@nv_pavlichenko
pre-training
AI
The comparison's author calls fine-grained MoE the best architecture change since GPT-3, but cautions that larger-model estimates are extrapolations.
AI
