Synthetic Generation at Scale Reduces to Pretraining Job Using DeepSeek Recipes

I also get why they finally dropped the 7B for the speciale recipe. Sparse MoE economics means that large model inference is the better trade-off.