AI Solves Hard Math but Ignores Simple Instructions
Tweets contrast frontier models solving open problems with inability to follow basic prompts.
Yuchen Jin posted that current AI systems solve ten open math problems at a level beyond a math PhD yet cannot follow three basic instructions and waste large amounts of tokens and money. Daniel replied that the models are about as useful as a math PhD. Peter J. Liu shared a screenshot making a similar comparison between frontier AI behavior and human performance. The posts present the contrast as an observed limitation without additional confirmation or broader claims.
The irony of AI: Smarter than a math PhD. Solves 10 open math problems. Dumber than my intern. Can’t follow 3 simple instructions, but somehow burns 10M tokens and $250.
Combined views
121.5K
Reactions
3 postsDA 
Daniel@growing_daniel@Yuchenj_UW About as useful as a math phd would be
- likes: 313
- replies: 8
- bookmarks: 2
- reposts: 6
PJ Peter J. Liu@peterjliuPeople and frontier AI -- really not that different
- likes: 7
- replies: 0
- bookmarks: 0
- reposts: 1
YJ Yuchen Jin@Yuchenj_UWThe irony of AI: Smarter than a math PhD. Solves 10 open math problems. Dumber than my intern. Can’t follow 3 simple instructions, but somehow burns 10M tokens and $250.
- likes: 2.6K
- replies: 131
- bookmarks: 166
- reposts: 99