Researcher Questions Optimizers' Role in Preserving Training Information

"optimizer research" is the most intellectual, big-brained way of avoiding grunt work

I’m starting to heavily suspect optimizers is not the right place for what it’s expected to fix. Too much information dropped at this stage.