Report
Mistral's technical report draws praise and a model-training question
Mistral's report drew praise for technical detail, while another reader questioned model surgery instead of starting over.
TLDR
One reader praised the report's openness and technical detail, saying Mistral used AdamW, MLA and about 28 trillion tokens. Another questioned why the team would modify a roughly 1-trillion-parameter mixture-of-experts model rather than start over, while acknowledging they might be missing factors.
Combined views
11.4K
9 Sources, first seen ago
