AI web text reportedly helped only data-starved models in COLM 2026 study
In X posts, Jenna Russell said Pangram labeled 31% of FineWeb-filtered web tokens as AI-generated in August 2026, up from 10% in June 2024. She said a study of 800 language models found AI tokens lowered loss on human text for data-starved models under about 10 human tokens per parameter, but raised it almost from Ch
TLDR
In X posts, Jenna Russell said Pangram labeled 31% of FineWeb-filtered web tokens as AI-generated in August 2026, up from 10% in June 2024. She said a study pretraining 800 language models found AI tokens lowered loss on human text only for data-starved models under about 10 human tokens per parameter, then raised it almost immediately from Chinchilla-optimal settings onward. Mohit Iyyer’s no-new-data comparison was his own summary.
Combined views
13.9K
12 Sources, first seen ago
