Hinted scaling laws target AI benchmarks where small models score near zero

When small models score near zero on a benchmark, conventional scaling laws have little to extrapolate. The researchers use privileged information to create easier versions of the test and track accuracy as the hints change. They report forecasting a larger model’s 25.9% GSM8K accuracy to within three percentage points using models trained with at most one-thousandth as much compute; a standard scaling law fitted to those same smaller models under-forecast it by 13 points.