Alex Nichol shares tokenizer_lp_books, an automated research process that generates candidate theories, evaluates LP cuts over two rounds of 500 each, and iterates on trade-offs with a 512-token vocabulary and max length of 8

AKAran Komatsuzaki@arankomatsuzaki

i was playing with Codex /goal on some lesser-known open conjectures, mostly 20–50y old. after letting it run autonomously for 8h+, i was already seeing what looked like publishable progress, even if not full resolutions. weakly held take: people overrate “open for decades” as a proxy for importance. unsolved ≠ important. a lot of old problems are just boring-but-hard, or maybe hard in the bad way / structurally not that productive. imo the higher-value thing is often accelerating recent research directions where the community actually has live taste / consensus that the topic matters. these aren’t necessarily “harder” in some intrinsic sense. there are just way fewer participants because the prerequisite stack is brutal, vs more approachable combinatorics / Erdős-style problems. so the marginal AI researcher there may be much higher-value than grinding on random half-century-old open problems. my stronger take: current models can already push some frontiers forward rapidly 95%-automatically, not “solve smooth 4D Poincaré today,” but real progress. it’s underpriced because the domain people are conservative or slow to retool around AI, and the AI people mostly don’t know which deep problems exist / matter.