Cameron R. Wolfe, Ph.D.#1444
New blog! Is frontier asynchronous RL solved?
The blog covers Async RL theory and infrastructure, surveying 8 open-weight frontier labs for the algorithmic techniques and systems fixes to handle train-inference mismatch. Also answered: why do current methods still fail at high policy lag? Which methods scale with horizon and compute?
Super comprehensive writeup that covers many frameworks & case studies on async RL. I learned a lot from the discussion of adding bias to the objective and how techniques that introduce bias (e.g., TIS + CISPO) help stabilize smaller batches but scale more poorly.
New blog! Is frontier asynchronous RL solved?
The blog covers Async RL theory and infrastructure, surveying 8 open-weight frontier labs for the algorithmic techniques and systems fixes to handle train-inference mismatch. Also answered: why do current methods still fail at high policy lag? Which methods scale with horizon and compute?
Cameron R. Wolfe, Ph.D.#1444New blog! Is frontier asynchronous RL solved?
The blog covers Async RL theory and infrastructure, surveying 8 open-weight frontier labs for the algorithmic techniques and systems fixes to handle train-inference mismatch. Also answered: why do current methods still fail at high policy lag? Which methods scale with horizon and compute?
Users in the replies express gratitude for helpful comments on systems and async infrastructure in the blog surveying async RL techniques across eight frontier AI labs.