@nv_pavlichenko
pre-training | prev. head of mellum @jetbrains
AI
The comparison's author calls fine-grained MoE the best architecture change since GPT-3, but cautions that larger-model estimates are extrapolations.
AI
