Microsoft AI CEO Mustafa Suleyman predicts AI training compute will scale by three orders of magnitude over the next three years

Mustafa Suleyman, CEO of Microsoft AI says that AI compute will grow 1000x in the next 3 years

We are currently at around 5e27 FLOPs. Three more OOMs mean we reach 5e30 FLOPs in 2029.

Before agents I think this likely would not have mattered (not enough data in the world) but with RL environments driving much progress these days i think OOM leaps in training compute will make much more complex computer use tasks feasible

https://www.youtube.com/watch?v=HG0twQJ7aG4

@AndrewCurran_ Does he mean relative to the frontier today?

if the baseline for that is MAI-Thinking-1, that's actually not ambitious at all.

@justjoshinyou13 He means relative to total compute globally, he also said that MAI-1 is only 35b.

@AndrewCurran_ One a year? That seems very optimistic. We are only going one oom higher than gpt 4 now 3 years later

@scaling01 three orders of magnitude is a slide. the question is whether the software consuming that compute is doing anything worth the electricity bill

@AndrewCurran_ @justjoshinyou13 Active I assume

@AndrewCurran_ Did he say how many white collar jobs will be wiped again on stage?

@AndrewCurran_ 😵‍💫

@scaling01 1000x in 3 years implies a ~40%MoM compound—feasible only if H100-scale clusters double every 4 months and inference becomes the dominant load. Would be interested in the split between training vs inference in that projection.

@AndrewCurran_ Demis just modified his AGI speculation to match Kurzweil’s 2029 ... this would fit ...

@scaling01 I just hope most of it will be pre-training instead of RL hill climbing

@scaling01 Let’s gooooooooo!

@AndrewCurran_ Three more OOM jumps by 2029 is not a forecast. It is an instruction to every institution watching the curve.

Assume today’s impossible becomes procurement, policy and product planning before the decade is out.

@AndrewCurran_ I bet one more OOM will be more than enough for RSI and AGI.

At 3 OOM it'd have similar computational complexity to the human brain and possibly have emergent properties that even we don't.

@scaling01 feeling the flops

@scaling01 If that compute curve holds, most AI predictions are probably too linear.

The gap between demo and default can close fast.