14h ago

AI Training Sparks Trillion-Dollar Boom in Numerical Optimization

0
Original post

Maybe I am simplifying a bit, but in the 1990s onwards numerical optimization was not considered an exciting area. There were many breakthroughs (say, Karmarkar, and in hindsight esp. Nesterov/Nemirovski) but not much funding. Junior faculty in the area were denied tenure. The problems and the algorithms seemed to suffice. Convex problems, tens of thousands of variables. Who needs more? So let’s spend a minute in silence contemplating the ONE TRILLION spend (this year) devoted to efficiently solving a distributed, non-convex numerical optimization problem in ~1E12 variables. Did we ever get to spend that much in simulations and PDEs for nuclear devices? Not even close; less than $10B/yr. What about NOAA (weather prediction) ? Also lss than $10B/yr. So this is the biggest ever application of numerical methods ever, by a mile. Also, the most consequential optimization problem, ever. Many departments withering, or closing. It turns out we were just not looking far enough. I guess there is a lesson about research in this.

2:13 AM · May 20, 2026 View on X