Score Centering proposed to counter drift from trainer–sampler mismatch in RL

In notes on Score Centering, a user says trainer–sampler mismatch stops a term from canceling as it does in on-policy reinforcement learning. They argue the resulting drift amounts to distillation toward a moving rollout model, making training unstable. Their proposed correction subtracts the cross-entropy gradient drift term, and they think Score Centering could become part of the standard RL recipe.