Muon's claimed distinctive fingerprint in model weights
A user says Muon’s pattern differs from SGD and Adam, attributing the spectral signature to more aggressive updates in low-magnitude directions.
TLDR
A user argues that inspecting the spectral distribution of a model’s hidden-layer weights can reveal whether Muon was used for training. The post describes a distinctive pattern in WᵀW—a matrix calculated from those weights—compared with SGD and Adam, and attributes it to Muon updating low-magnitude directions more aggressively.
Combined views
28.1K
1 Source, first seen ago