/Tech8h ago

Mathematicians Tony L. Feng and Daniel Litt say they now adapt their research and project selection to AI's specific limits

Feng compares current AI models to an alien collaborator.

15312248132.9K

Original post

Recently, I have started getting appreciable value from AI for my own mathematics research. While model improvements were necessary for this to happen, I think another key factor was reflecting on recent success cases, in order to build a better mental model for the comparative strengths of AI. Previously I would think in terms like "AI is good at extremal combinatorics, but bad at derived algebraic geometry"; a more nuanced perspective allowed me to identify directions where AI could truly accelerate my work.

A few months ago, Tao described AI as a "junior collaborator". Currently I think of it more as an "alien collaborator", which is already superhuman at certain skills (e.g., assembling puzzle pieces, juggling delicate technical conditions, making local optimizations) but lackluster at certain others (e.g., creating new puzzle pieces, generating diverse ideas).

Much of what I write is in the spirit of taming overhype, but it's important to give credit where credit is due: AI has progressed to the point where AI tools, and education on how best to use them, need to become a core part of any mathematics Ph.D. curriculum.

9:33 AM · Jun 20, 2026 · 27.8K Views

Sentiment

Positive users praise AI for accelerating math research via improved logic, geometry, and problem-solving, while negative users fear PhD skill redundancy and highlight LLM flaws and rapid capability shifts.

Pos

57.1%

Neg

42.9%

7 comments with sentiment.

Cluster Engagement

Digg Deeper

No Digg Deeper questions have been answered for this story yet.

Posts from X

Most Activity

VIEWS1.1K

Mark Kisin@marcusgnt

@tonylfeng Great comment, but do we need to make it part of the curriculum? Won’t young people just pick this up by growing up mathematically with AI ?

1d1.1K81

BOOKMARKS6LIKES21RETWEETS4

LonelySloth@lonelysloth_sec

The problem with using and evaluating LLMs for any work is that the intuition “AI can do tasks of type X but not of type Y” is completely wrong.

It can do task X perfectly for 99% of the domain and fail completely at 1%. For that part of the domain it can never find the right answer no matter how many times it tries.

That 1% isn’t harder and doesn’t look different to a human. It just happens to be less like the training data in some vector representation the AI “learned”.

So you start using the AI for tasks of type X. Get amazed at its performance. Start delegating that sort of task to the AI.

Then it fails catastrophically and surprises you and sabotages your work.

It does that **for every single type of task**. Just at different rates. The only exception is domains it has completely memorized from training.

For tasks combining multiple steps the domain coverage shrinks exponentially with the number of steps.

But even at very long tasks with thousands of steps it will do amazingly well for a small part of the domain. And you will look at the impressive results and be tempted to generalize.

But the generalization will never work. Or worse, it will seem to work for a while then fail.

You will then be tempted to see it as some random failure. Humans make mistakes too!

But it isn’t random.

It’s what happen when something actually didn’t “learn” anything except a large number of superficial heuristics that happen to coincide with the actual algorithm in many cases.

“Machine learning” isn’t actual learning. “Artificial intelligence” isn’t actual intelligence.

It’s not “intern intelligence”, “grad student intelligence”, or even “alien intelligence”.

It simply isn’t intelligence at all.

It’s an entirely different phenomenon.

A complex, impressive, in many ways surprising phenomenon. Potentially useful.

But it’s not intelligence or understanding or learning.

And unless you completely abandon all anthropomorphic analogies you’ll not be able to see it for what it is and use it for what it is actually good at.

1d1.1K216

REPLIES1

Tony Feng@tonylfeng

Well I didn't say who would be teaching. Obviously you would have to bring in undergraduates to teach the Ph.D. students.

In seriousness though, I'm thinking only of the immediate horizon. There's a bit of a skill curve to using Codex/Claude and I don't think mastery is universal among the current youth. Separately, it would be useful to have a communal gathering point to share and discuss insights about using AI.

1d75642

2Loopy@quantop__

@tonylfeng It is not possible to design seminars on AI heuristics for maths research when model capabilities change this frequently and AI for math is essentially a duopoly which plays a lot of games (performance of top range models fluctuates degradation/improvement by the daily).

1d41721

napsoupwindow@napsoupwindow

@tonylfeng ^ very wise; imo one of the highest-leverage things re ai you can do is stop anthropomorphizing the models. as a simple example, a "non-human-shaped" workflow that i've found useful in math is "do 100-1000 tedious slightly non-trivial example calculations and look for patterns"

1d41521

Ryan McCormick@RyanMcC35236715

@tonylfeng They understand breadth through relationships and they are remarkably good at maintaining logic. They can recompose your problem space across all known modality (lemma, etc) and they have an understanding of the logic of "why". They taught me higher level math too.

23h24921

Silva, J.P.@ertwro

@tonylfeng This is important. When the current generation fears that the skills they are honing will become redundant, especially when they see their teachers and undergraduates using them to punch above their weight publishing while PhDs are mocked, hurt, or discouraged from using AI.

1d29211

Nacho Peinador ❤️💛💜@todos_lumpen

@tonylfeng Prueba esto Tony. https://zenodo.org/records/20784009

8h651

Jim Golden@JimGolden23

@tonylfeng I’ve had it help me with better understanding geometry in the complex plain and mathematical physics. It has improved rapidly over the last year!

10h54

Tom🦉@Tom62589172

@tonylfeng @marcusgnt What is your personal setup and workflow like?

13h32

Samian Noesis@samiannoesis

@tonylfeng Cool!! You're a good man for admitting that Tony. I get it that you would avoid hyping it, since there's enough people doing that already. Glad research can benefit

6h26

长期收购 LLM API 资源@SarvisMyrac430

@tonylfeng totally agree, finding the right fit is everything with AI in research.