Skip to content
Yihua Zhang
Articles
Posts
EN
中
◐
Series
The GRPO lineage
Reading order
2025-02-07
DeepSeek-R1 Dissection: Understanding PPO & GRPO Without Any Prior Reinforcement Learning Knowledge
Post-training
·
en / zh
·
16 equations
The GRPO lineage 1
GRPO
RLHF
2025-08-08
From GRPO to DAPO and GSPO: What, Why and How
Post-training
·
en / zh
·
18 equations
The GRPO lineage 2
GRPO
2025-07-02
Re-understanding KL Approximation from an RL-for-LLM Lens: Notes on “Approximating KL Divergence
Post-training
·
en / zh
·
18 equations
The GRPO lineage 3
GRPO
2026-10-08
Score Function Centering Made Simple: Understanding Trainer-Generator Mismatch in Async RL from the Score Function
Post-training
·
en / zh
·
132 equations
The GRPO lineage 4
GRPO
2026-10-11
Score Function Centering: Why a Stale Rollout Does Not Have to Be Importance Sampled
Post-training
·
en / zh
·
57 equations
The GRPO lineage 5
GRPO
2026-10-14
Stabilize Your Async RL Training with Score Function Centering and Trainer-Generator KL
Post-training
·
en / zh
·
6 equations
The GRPO lineage 6
GRPO
2026-11-11
Why Does an Unbiased KL Estimate Not Give the Right Gradient? From Common KL Estimators to the Trainer-Generator Constraint in Asynchronous RL
Post-training
·
en / zh
·
26 equations
The GRPO lineage 7
GRPO
2026-11-16
Everything You Need to Know About Trainer-Generator Mismatch: Sources, Monitoring, Zero-Difference Tests, and Debugging
Post-training
·
en / zh
·
18 equations
The GRPO lineage
GRPO
RL infrastructure