Zero-order FineWeb-30M speedrun reportedly reaches 3.5 validation loss in about 23 seconds

A Noise project contributor’s GitHub pull request reports a zero-order FineWeb-30M run reaching 3.49761 validation loss in 22.89 seconds on one eight-H100 node. It builds on a prior 123.2-second result and credits single-root replays, fused kernels and other optimizations. In a separate post, the contributor says the small 4,096-token vocabulary helps keep the run cheap, but leaves scaling to larger vocabularies and models an open problem.