15h ago

Marin MoE Run Hits 2.234 Loss, Beats Preregistered Scaling Law Prediction

——0——
Original post
OPPercy Liang#22Percy Liang@PERCYLIANG

Not only do we want to train a good model, we want to know it'll be good before we even start training. About a month ago, the Marin team launched a 129B (16B active) 1e23 FLOPs MoE run and preregistered a loss of 2.252. The run finished this past week and landed at 2.234.

11:50 AM · May 24, 2026 View on X
REPLYPercy Liang#22Percy Liang@PERCYLIANG

While this run was going, we were busy curating more high quality data and making some architectural improvements, all of which will go into the next run. If you want to follow along in real time, come hang out with us in the Marin discord: https://discord.gg/J9CTk7pqcM

Percy LiangPercy Liang@percyliang

Not only do we want to train a good model, we want to know it'll be good before we even start training. About a month ago, the Marin team launched a 129B (16B active) 1e23 FLOPs MoE run and preregistered a loss of 2.252. The run finished this past week and landed at 2.234.

6:50 PM · May 24, 2026 · 29.1K Views
6:50 PM · May 24, 2026 · 3.3K Views