Adding AI-generated web text reportedly makes models worse than adding no data under most standard training budgets

A researcher says their team pretrained hundreds of models using different mixes of human-written and AI-generated web text. They found that, across most standard training budgets, adding AI text made models worse than adding no new training data. The researcher also says almost a third of FineWeb-filtered web text was AI-generated as of September 2026, and that the share was rising fast.