Simple math, hard-to-predict LLMs: A post shares Terence Tao’s view

A post summarizing Terence Tao’s remarks says training and running large language models mostly uses linear algebra, matrix multiplication and a bit of calculus—math an undergraduate can handle. The harder puzzle, in this account, is predicting why models succeed at some tasks and fail at others. It points to limited mathematics for data that is partly structured and partly random, like natural text. Without reliable rules for forecasting performance across tasks, it says, progress remains largely empirical.