Group Relative Policy Optimization (GRPO)
Definition
GRPO is a policy optimization method that uses relative rewards within answer groups.
GRPO was introduced in the DeepSeekMath context as a PPO variant. For multiple answers to a task, group-relative reward signals are used, thus avoiding a separate critic in this form.
Simply explained
Multiple solution attempts are evaluated against each other; the model learns more from the relatively better answers.
Why it matters in practice
Relevant for RL-based post-training of language models, especially for tasks with automatable evaluation. It is not a standard component for typical business automation.
Example from business
A research team generates multiple mathematical answers and uses verifiable results as group-relative learning signals.
Limits and pitfalls
- Confusing GRPO with normal prompting; poorly defining the reward function; transferring benchmark success to real tasks; underestimating high training costs.
Origin & history
Shao et al. introduced GRPO in 2024 in DeepSeekMath as a PPO variant.
Use in mid-sized companies
- Possible application in medium-sized businesses only in specialized model training projects when own models and reliable verifiable rewards are available.
Frequently asked questions
Is GRPO an inference method?
No, it is an optimization method for training/post-training. Is it needed for an AI assistant? Usually not.
Sources
- DeepSeekMath · Shao et al. (2024), DeepSeekMath
- PPO · Schulman et al. (2017), PPO
- DeepSeekMath, Abschnitte 4.1.1–4.1.2 · Shao et al., DeepSeekMath, Abschnitte 4.1.1–4.1.2
Source check: 10./11.10.2026.
Editorially reviewed on: