GRPO (Group Relative Policy Optimization)

In depth

A reinforcement learning algorithm introduced in the 2024 DeepSeekMath paper as a lighter variant of PPO that drops the separate value model. For each prompt it samples a group of responses and scores them. Each response is then judged by how its reward compares with the group's average, which saves memory and simplifies the training stack. GRPO powered DeepSeek-R1's reasoning training and is a common choice for RL fine-tuning open models with verifiable rewards.

Primary source: arxiv.org