Will DePue calls current era golden age for optimizers
Research engineer Will DePue described the present as a golden age for optimizers, with new variants appearing daily including SoapyShampooGluon and ongoing releases built on RMS matching. Research engineer Jerry Tworek predicted further expansion of progress ahead. Engineer Bilal compared the current pace of optimizer development to the environment faced by machine learning researchers in 2016.
@willdepue Golden age of golden ages ahead
will depue@willdepuegolden age for optimizers right now. every day you see another SoapyShampooGluon ^-1/2 (RMSMatched) drops
#254will depue@WILLDEPUEgolden age for optimizers right now. every day you see another SoapyShampooGluon ^-1/2 (RMSMatched) drops
@MillionInt one might call it the diamond age
Jerry Tworek@MillionInt@willdepue Golden age of golden ages ahead
#755Aaron Defazio@AARON_DEFAZIODecoupled and Dualized with a side of fries
will depue@willdepuegolden age for optimizers right now. every day you see another SoapyShampooGluon ^-1/2 (RMSMatched) drops
#826You Jiacheng@YOUJIACHENG😭 btw one good thing of optimizer is that you don't need to worry about train-inference mismatch when changing it.
will depue@willdepuegolden age for optimizers right now. every day you see another SoapyShampooGluon ^-1/2 (RMSMatched) drops
@willdepue so this is what it was like to be an ml researcher in 2016
will depue@willdepuegolden age for optimizers right now. every day you see another SoapyShampooGluon ^-1/2 (RMSMatched) drops