Reaction
eliebakouch questions NanoGPT’s billing as AI research’s hardest benchmark
eliebakouch singles out Track 1, while scaling01 calls NanoGPT’s status as OECI’s toughest research test sobering.
TLDR
eliebakouch disputes treating NanoGPT—especially Track 1—as the hardest AI research benchmark, responding to scaling01’s criticism of OECI. In August, Prime Intellect said its best autonomous optimizer runs closed 82% of the gap to the then-current human record; eliebakouch also warned of substantial run-to-run variance. scaling01 is now asking for an update on that experiment.
Combined views
—
3 Sources, first seen ago

Comments