Structure
text ⇒ AST
Generate code as a sequence of grammar rules instead of tokens, so the output is always syntactically valid.
- Grammar-CNN
- TreeGen
- OCoR
- Lyra
I teach machines to write, repair and reason, by treating programs, patches and reasoning traces as trees grown under constraints a machine can check.
1DeepSeek-AI: code LLMs, reasoning and RL from verifiable feedback
2Ph.D. in Computer Science, Peking University; advised by Prof. Yingfei Xiong
Who I am and what I work on.
I study how machines can write, repair and reason about programs. In my Ph.D. at Peking University, advised by Prof. Yingfei Xiong, I treated code as structured objects rather than flat text. I built grammar-guided decoders (TreeGen, GrammarT5, Grape) that generate programs rule by rule, and syntax- and type-aware repair models (Recoder, Tare) that edit the abstract syntax tree directly.
At DeepSeek I apply the same view to large language models. I help train code models (DeepSeek-Coder, DeepSeek-Coder-V2) and math and formal-reasoning models (DeepSeekMath, DeepSeek-Prover), and I contribute to the general reasoning models DeepSeek-R1, V3 and V4.
What connects this work is a simple claim: a program, a patch and a proof are all trees. They are best grown under constraints a machine can check, such as a grammar, a type system, a test suite or a proof assistant. Those same checks then become the reward signal that teaches models to reason.
Keywords: code generation · automated program repair · code LLMs · reinforcement learning with verifiable rewards · formal theorem proving · program synthesis
theorem research_agenda : ∀ intent : NaturalLanguage, ∃ p : Program, WellTyped p ∧ Satisfies p intent := by grow_by_grammar -- TreeGen · GrammarT5 · Grape (2018–2024) repair_until_typed -- Recoder · Tare (2021–2023) scale_and_pretrain -- DeepSeek-Coder · DeepSeek-Coder-V2 (2024) reinforce_with_checker -- DeepSeekMath (GRPO) · Prover · R1 (2024–) sorry -- work in progress
A changelog, newest first.
Written as a derivation: each stage rewrites the one before it.
text ⇒ AST
Generate code as a sequence of grammar rules instead of tokens, so the output is always syntactically valid.
AST ⇒ AST′ ⊨ tests
Edit trees, not strings. Syntax-guided edits and type-aware decoding lead to patches that compile.
grammar ⇒ pre-training
Build structure into pre-training, then scale open code models to the frontier.
checker ⇒ reward
Use compilers, tests and proof assistants as verifiers, and their verdicts as the reward signal for reinforcement learning.
★ selected · Qihao Zhu underlined · venue colour: LLMs & reasoning / code & SE / ML & AI / wireless sensing
Code and open models. Star counts are fetched live from GitHub.
Open code LLMs trained from scratch on 2T tokens.
model · 2024MoE code model competitive with closed-source systems.
model · 2024Mathematical reasoning; the origin of GRPO.
model · 2024Lean 4 theorem proving via recursive subgoal decomposition.
model · 2025Reasoning incentivised by reinforcement learning.
model · 2025Syntax-guided edit decoder for neural program repair (FSE’21).
code · 2021Type-aware neural program repair (ICSE’23).
code · 2023Grammar-integrated pre-trained encoder-decoder for code (ICSE’24).
code · 2024Overlapping-aware code retriever (ASE’20).
code · 2020Grammar-preserving rule embedding (IJCAI’22).
code · 2022