Length-weighted reward baseline reportedly made rollout–trainer numerics more stable

A post describes moving from GRPO’s group mean reward baseline to a length-weighted group baseline. Each rollout’s reward Rᵢ is weighted by its length Lᵢ, and the resulting baseline is subtracted from the reward: R − (Σ RᵢLᵢ)/(Σ Lᵢ). The author says the change made rollout–trainer numerics more stable.