Marin researchers validate scaling laws by successfully predicting a 129-billion-parameter MoE model's loss before training

The training run was funded under the DARPA AIQ program.