ThunderSync is claimed to make reinforcement learning up to 1.9x faster without stale policies

A post introducing ThunderSync says the method reduces idle time during long-horizon agent training. For GRPO, it computes a trajectory’s gradient when its reward is ready; for OPD, it computes a turn’s gradient when the teacher scores it. The author reports up to 1.9x faster reinforcement learning while matching synchronous-update performance, without the stale policies associated with asynchronous training.