Recursive launches an automated ML research platform that set benchmarks on NanoGPT and GPU kernel optimization
The startup open-sourced its generated research artifacts on GitHub.
The startup open-sourced its generated research artifacts on GitHub.
The startup open-sourced its generated research artifacts on GitHub.
The startup open-sourced its generated research artifacts on GitHub.

@humphrey_shi @Recursive_SI AI泡沫最危险的时候是所有人都觉得它能解决一切。现实是:它连很多简单问题都搞不定。 「Automated AI research becomes real when it improves hard sys...」

@cong_ml This is so cool! Gonna try it out for a few tasks 🫡
@RichardSocher Great work!
AI is now doing our AI research. At Recursive we set out to build recursive self-improving superintelligence (RSI) to automate knowledge discovery. The best way to expand humanity’s knowledge is through the scientific method. RSI leads to better ideas, explanations and inventions which lead to better RSI. Automating the scientific method requires closing the loop between ideation, implementation and validation, and being able to run it over extended periods of time. Today, we are excited to share the first outputs of Recursive’s automated open-ended discovery system. To be clear, this system is merely a milestone towards RSI, a v0.1 of what I sometimes call the “Eureka Machine”. It is one program that you can point at any hard problem and get useful inventions out. Though it’s still very early, we've run it on three AI tasks and achieved state-of-the-art results on all three. These results demonstrate that even this early version of the system can solve a variety of autoresearch problems in AI and improve over prior state of the art. Concretely, it did this on the community benchmarks NanoGPT speedrun, NanoChat, and NVIDIA's Sol-ExecBench. AI is code and AI can code. The code and ideas that lead to these results were not invented by our team but by the AI system itself. To do RSI safely, we need to investigate its inventions. That's best done transparently with the community. @Recursive_SI we are open-sourcing the system’s discoveries, demonstrating that it finds creative and benign solutions instead of focusing on obvious optimizations or dangerous ideas. Link below.
Early results from Recursive 🚀🚀 SotA results from our open-ended knowledge discovery system: 1️⃣NanoChat 5min pre-training (0.9372 bpb -> 0.9109 bpb, 2.8% lower Bits-Per-Byte than long-standing community SoTA) 2️⃣NanoGPT SpeedRun (79.7s -> 77.5s, 2.8% faster than long-standing community SoTA) 3️⃣GPU kernel optimization (Overall 7.8% better than SoTA performance in SOL- ExecBench, hosted by NVIDIA) To achieve that, our system automatically finds and combines innovations together to create better solutions than current ones carefully designed by expert humans in various domains. We have open-sourced resulting artifacts found by our system so you can check the output yourself. See a full breakdown and technical writeup: https://www.recursive.com/articles/first-steps-toward-automated-ai-research
In GPU kernel optimization, our framework achieves overall SoTA in NV's SOL-ExecBench, SoTA in all 4 sub-categories, and outperforms solutions that are (1) designed by GPU human experts, and (2) generated by other AI systems designed by GPU experts. Our system is general. We don't have in-house GPU experts for now (I am regarded as a kernel "expert" internally😆). 🔍Check the details in NV's official leaderboard https://research.nvidia.com/benchmarks/sol-execbench
First results from Recursive on AI that improves AI! 🚀 ✨ 📈 Our automated AI research system incorporates principles from open-ended and AI-generating algorithms. 🧠 💡🌱🧬 It conducts key parts of the science loop: proposing ideas, implementing them, testing them, and picking the next ideas based on the data. 🔬 🔭 🧪⚗️ The same general system produces state-of-the-art results on three different problems (two on training language models, one on speeding up AI via kernel optimization). • NanoChat Autoresearch: 1.3x faster to reach the same loss than the best solution produced by an entire community of humans+agents over months, and 1.8x faster than the initial hand-optimized solution • NanoGPT Speedrun: 3% speedup of a very efficient solution produced by entire community of humans+agents over 2+ years • GPU Kernel Optimization: 18% reduction in gap to theoretically optimal score on Nvidia’s SOL-ExecBench These are early tests of our system. We’re very excited for what the future holds! Post: https://www.recursive.com/articles/first-steps-toward-automated-ai-research Great work by the amazing team at Recursive!
Excited to show results of the first steps towards automated AI research at @Recursive_SI. The same general system achieved state of the art on @NVIDIAAI's SOL-ExecBench GPU Kernel Optimization, nanoGPT Speedrun, and @karpathy's NanoChat autoresearch benchmarks. https://x.com/_rockt/status/2065061990800802249/photo/1
This is probably what Fable and Mythos would want to avoid people doing. We'll keep at it. https://twitter.com/RichardSocher/status/2065094362774876232
Recursive’s v0.1 “Eureka Machine” is an automated discovery system that runs open-ended loops of ideation, code generation, implementation, and validation to invent optimizations for AI tasks.[[1]](https://x.com/RichardSocher/status/2065094362774876232) In practice today, AI researchers, ML engineers, and labs point it at problems like model training speed or GPU kernel efficiency. It has already produced SOTA results on NanoGPT speedrun (77.5s training time), NanoChat, and NVIDIA Sol-ExecBench by autonomously generating creative code changes (e.g., hash embeddings, multi-token prediction, hardware-specific tweaks on B200 GPUs) that outperform prior human or baseline records.[[2]](https://www.anthropic.com/institute/recursive-self-improvement) Real-world uses include accelerating internal R&D at AI companies (faster convergence within fixed compute budgets, better inference kernels), open-sourcing and adopting the generated artifacts for production workloads, and eventually directing the system at non-AI domains like drug discovery or materials design once matured. It runs as a single program over extended periods on hard quantitative problems, with all outputs open-sourced for transparent study.