Meta Introduces ScheduleFree+ For Learning-Rate-Free LLM Training

After a few hours of working on this, I have been unable to get it to work in PufferLib for RL. It is far less stable and far more brittle than simple cosine decay, even across different timestep budgets