Developer Joseph Suarez pointed to Puffer project PPO resources
@jsuarez @ChenTessler Are there some blogs or papers on that? Would be very interested in cases where Muon is helpful in RL, my impression so far has been that optimizers matter more in pre-training.
@ChenTessler It was a major component of Puffer 3. We reswept hypers for ~10 tasks with Muon vs Adam and the gap was quite clear. Try the PufferNet arch over LSTM next!
@konstmish @ChenTessler Puffing up PPO on my articles tab
@jsuarez @ChenTessler Are there some blogs or papers on that? Would be very interested in cases where Muon is helpful in RL, my impression so far has been that optimizers matter more in pre-training.
@jsuarez @ChenTessler ok I guess I'll just try running some examples with pufferlib myself
@konstmish @ChenTessler Puffing up PPO on my articles tab
Developer Joseph Suarez pointed to Puffer project PPO resources
@jsuarez @ChenTessler Are there some blogs or papers on that? Would be very interested in cases where Muon is helpful in RL, my impression so far has been that optimizers matter more in pre-training.
@ChenTessler It was a major component of Puffer 3. We reswept hypers for ~10 tasks with Muon vs Adam and the gap was quite clear. Try the PufferNet arch over LSTM next!