extremely niche second order optimizer drama going down rn
Former OpenAI researcher Will Depue signals an active deep learning community controversy over second-order optimizers
Their heavy computational requirements have historically limited mainstream adoption.
Many users are entertained by the niche drama over second-order optimizers because of the deep arguments around loss curves and natural policy gradients, while some criticize the community's terminology.
Most Activity
@willdepue been a bit since we had the battle of the bathroom essentials
extremely niche second order optimizer drama going down rn
@willdepue how is it second order?
extremely niche second order optimizer drama going down rn

@willdepue it’s just he ssian she said
if you have an opinion on shampoo vs muon beef you are too online
extremely niche second order optimizer drama going down rn

@willdepue 🤣 although tbh, the community calling them second order is the real travesty

@willdepue *first order matrix optimizer

@NinaPanickssery wow this was advanced. bravo
@giffmana @willdepue I hessian't quite understood what we mean by 2nd order these days
@HeinrichKuttler @willdepue Takes a moment to understand. Maybe there's a different variance of the pun that could land more smoothly.

@willdepue Did everyone forget about this post?

@willdepue gotta admit i respect the depths people will go to argue about loss curves

@willdepue I was dying for these drama

@willdepue it’ll blow over in a moment

@willdepue shit boy, this is mainstream

@willdepue muon is shampoo and zendaya is meechee

@willdepue many

@willdepue what does it mean?

@willdepue this sounds like the most entertaining math drama ever.

@willdepue It’s turns out natural policy gradient was cool all along

@willdepue Good

@NinaPanickssery @willdepue he ssian she sayin Maybe