Skip to content
Yihua Zhang
Articles
Posts
EN
中
◐
Post-training
All
18
Post-training
11
Systems
5
Music
2
Archive
20
Series
3
Anatomy of DeepSeek
8
The GRPO lineage
11 articles
2026-11-16
Everything You Need to Know About Trainer-Generator Mismatch: Sources, Monitoring, Zero-Difference Tests, and Debugging
Post-training
·
en / zh
·
18 equations
The GRPO lineage
GRPO
RL infrastructure
2026-11-11
Why Does an Unbiased KL Estimate Not Give the Right Gradient? From Common KL Estimators to the Trainer-Generator Constraint in Asynchronous RL
Post-training
·
en / zh
·
26 equations
The GRPO lineage 7
GRPO
2026-10-14
Stabilize Your Async RL Training with Score Function Centering and Trainer-Generator KL
Post-training
·
en / zh
·
6 equations
The GRPO lineage 6
GRPO
2026-10-11
Score Function Centering: Why a Stale Rollout Does Not Have to Be Importance Sampled
Post-training
·
en / zh
·
57 equations
The GRPO lineage 5
GRPO
2026-10-08
Score Function Centering Made Simple: Understanding Trainer-Generator Mismatch in Async RL from the Score Function
Post-training
·
en / zh
·
132 equations
The GRPO lineage 4
GRPO
2025-08-08
From GRPO to DAPO and GSPO: What, Why and How
Post-training
·
en / zh
·
18 equations
The GRPO lineage 2
GRPO
2025-07-02
Re-understanding KL Approximation from an RL-for-LLM Lens: Notes on “Approximating KL Divergence
Post-training
·
en / zh
·
18 equations
The GRPO lineage 3
GRPO
2025-02-11
Navigating the RLHF Landscape: From Policy Gradients to PPO, GAE, and DPO for LLM Alignment
Post-training
·
en / zh
·
88 equations
RLHF
2025-02-07
DeepSeek-R1 Dissection: Understanding PPO & GRPO Without Any Prior Reinforcement Learning Knowledge
Post-training
·
en / zh
·
16 equations
The GRPO lineage 1
GRPO
RLHF
2025-01-20
From Zero to Reasoning Hero: How DeepSeek-R1 Leverages Reinforcement Learning to Master Complex Reasoning
Post-training
·
en / zh
·
2 equations
Anatomy of DeepSeek 1
GRPO
2024-12-15
Patching the Foundation Models: Pitfalls and Pains in Machine Unlearning
Post-training
·
en / zh
·
14 equations
Unlearning