New Schedule-Free Spectral Optimizer SF-NorMuon Beats Tuned AdamW on Language Models

🚨New paper: Anytime Training with Schedule-Free Spectral Optimization🚨 We introduce SF-NorMuon, a schedule-free spectral method that outperforms or matches heavily tuned AdamW across 125M and 772M parameter language models.