The framework supports distributed training without sharing closed datasets.
The Danish Foundation Models (DFM) project is adapting our modular FlexOlmo architecture into a lighter-weight system that runs on commodity hardware—putting collaborative model building within reach of smaller research groups & organizations. 🧵
Users express optimism about the Danish project adapting FlexOlmo into lightweight models for commodity hardware because it demonstrates modular training and open distributed approaches amid rising frontier model costs.
No Digg Deeper questions have been answered for this story yet.
Excited to see our FlexOlmo (http://arxiv.org/abs/2507.07024), released a year ago, is now being incorporated into the Danish Foundation Models (DFM) project!
The Danish Foundation Models (DFM) project is adapting our modular FlexOlmo architecture into a lighter-weight system that runs on commodity hardware—putting collaborative model building within reach of smaller research groups & organizations. 🧵
In FlexOlmo, each module is the size of a full model, so the combined system grows fast as more get added.
FlexMoRE replaces most modules with compact representations. Its best config matches or beats FlexOlmo using less than one-third the parameters.
Modular training is gaining momentum as frontier models become costlier to train & deploy.
This project shows how open, distributed approaches can make development more practical for national projects, public institutions, & smaller teams.
↓ Learn more: https://allenai.org/blog/flexmore
"FlexMoRE significantly reduces FlexOlmo's memory demands while preserving performance across almost all categories, allowing a broader audience to benefit from modular models," says Jacob Nielsen, who helped develop FlexMoRE at Ordbogen A/S and SDU's OdenseNLP lab.
FlexOlmo lets teams train modules separately then combine them in a shared model without pooling the data underneath.
DFM wants Danish institutions like hospitals & universities to each contribute modules trained on data they can't share.
