Quanquan Gu#319
[1/7] Recent breakthroughs in LLMs’ mathematical ability are genuinely surprising.
I recently solved a problem I had been unable to solve for seven years: the optimal acceleration rate for first-order methods under high-order smoothness assumptions in nonconvex optimization.
Quanquan Gu#319[1/7] Recent breakthroughs in LLMs’ mathematical ability are genuinely surprising.
I recently solved a problem I had been unable to solve for seven years: the optimal acceleration rate for first-order methods under high-order smoothness assumptions in nonconvex optimization.
Users highlight LLMs suggesting novel constructions and contributing new ideas to solve a seven-year-old nonconvex optimization problem.
[4/7] The more remarkable part is how this solution was found. The construction was suggested by ChatGPT 5.5 Pro: I uploaded papers, asked it to propose possible approaches, and kept asking about things I was unsure about. After several rounds, it arrived at the correct instance.
[5/7] My takeaway: current LLMs have reached the point where they can contribute new ideas in theory problems, as many have pointed out. They also do not seem to have the same blind spots humans do: if a strategy is plausible, they can often push it to its extreme.