ANVIL III is claimed to reach lower validation loss than Muon at the same compute

A developer who worked on ANVIL III says the optimizer nearly matched Muon’s speed while reaching final validation loss 20–28 millinats lower at the same token budget in tests from 124 million to 1.2 billion parameters. At 1.2 billion parameters with deep overtraining, fitted scaling curves imply ANVIL III could reach the same loss with 50% less compute.