Report
Tinker's StableReinforce aims to keep off-policy RL runs stable
Tinker says async RL staleness and numerical mismatch can cause off-policy runs to suddenly collapse.
TLDR
Tinker says StableReinforce is designed to keep off-policy reinforcement-learning runs stable, even when learning drifts from sampling. It says those runs can suddenly collapse because of async RL staleness or numerical mismatch. Separately, a user describes the new objective as REINFORCE plus score centering in a reverse-engineering thread.
Combined views
363
3 Sources, first seen ago