Report
K2 team releases Horizon’s training recipe—all 134 pages of it
The team says its arXiv report covers six models, with over 60 pages devoted to data.
TLDR
K2 Horizon’s developers have put the recipe on arXiv: a 134-page report covering six language models, from architecture to inference. The team says more than 60 pages explain the data work, including roughly 10 trillion synthetic tokens. The developers say it explains decisions behind their earlier release of 380 checkpoints, 13 TB of pre-training data and 17 code repositories.
Combined views
—
4 Sources, first seen ago
Comments