Announcement
Datology says data curation matched its largest baseline’s quality with one-sixth the pretraining compute
Datology reports gains using the same source datasets and training recipes, changing only pretraining data curation.
TLDR
Datology says it has released the report behind its claimed 6× pretraining compute multiplier. In a same-size comparison, the company reports that its curated-data 30-billion-parameter model scored 46.8% versus 37.7% for its baseline after post-training. Datology says both used one trillion pretraining tokens from the same source datasets and identical training recipes: same ingredients, different preparation.
Combined views
5
3 Sources, first seen ago

Comments