ethantsliu

@ethantsliu

Q labs | berkeley eecs

following
followers

Standard SGD can reportedly match AdamW in LLM reinforcement learning

A user summarizing research says AdamW's momentum and adaptive learning rates matter much less in reinforcement learning, questioning the need for its memory overhead.

1 Source, 37d ago, first seen 37d ago

Featured in

Standard SGD can reportedly match AdamW in LLM reinforcement learning

Featured in

Standard SGD can reportedly match AdamW in LLM reinforcement learning