Score centering added to TRL to tackle instability in LLM reinforcement learning

An October post says score centering has been added to TRL and calls it stable. A September post proposing the method says reinforcement learning with LLMs is very unstable when training and sampling policies differ. The earlier post says score centering directly cancels what its authors identify as the root cause and is competitive with and compatible with existing fixes, including matching numerics and importance sampling.