Announcement
Halcyon models are claimed to outperform strong late-fusion baselines with up to 275× less pretraining compute
Radical Numerics says Halcyon's early-fusion base models were trained from scratch; the models and recipes are open.
TLDR
Radical Numerics introduced Halcyon, a family of early-fusion base models trained from scratch. It claims Halcyon outperforms strong late-fusion baselines with up to 275× less pretraining compute. The models and recipes are open, and Halcyon is part of the group's research into multimodal pretraining and scaling.
Combined views
—
2 Sources, first seen ago
