Recursive has released its first public results from an automated AI research system that independently discovers optimizations for model training pipelines and GPU kernels, posting state-of-the-art numbers on nanoGPT and NVIDIA benchmarks while open-sourcing every artifact produced.
Recursive just came out of stealth, and the team has been cooking 🔥
Our first results: an automated AI research system that can improve AI across 3 very different settings across training and GPU kernel optimization.
https://www.recursive.com/articles/first-steps-toward-automated-ai-research
Anyone can now grab the exact optimizations the system found for nanoGPT training and NVIDIA kernels instead of repeating the search themselves.
The three benchmark settings show early capability, yet no information is given on compute required, architecture details, or how far the approach extends beyond these tasks.
Users are excited about Recursive's new autonomous AI research system setting SOTAs on NanoGPT and NVIDIA kernels because it demonstrates an automated loop that drives progress across varied domains, while a few remain skeptical of AI hype.
Towards a system that discovers knowledge autonomously!
New SOTAs on @karpathy’s nanogpt & nanochat
and @nvidia’s kernel benchmarks.
We are open-sourcing the artifacts of the system and contributing back to the research community!
http://x.com/i/article/2064569799313940481
Recursive’s autonomous research system just set new SOTAs on @karpathy’s nanogpt + nanochat and @nvidia’s kernel benchmarks!!
No humans in the loop. The system discovered these gains on its own.
Open-sourcing results today.
http://x.com/i/article/2064569799313940481
Domain 2: squeezing more out of a mature highly optimized training benchmark ⚡
On NanoGPT Speedrun, starting from a solution refined by the community over 2+ years, the system still found a sequence of discoveries to produce a further speedup: 79.7s → 77.5s.
Domain 1: fixed-budget LM training ⏱️
On NanoChat Autoresearch, our system found a training recipe that reaches the same loss 1.3x faster than the best community solution, and substantially improves over the initial hand-optimized baseline.
Domain 1: fixed-budget LM training ⏱️
On NanoChat Autoresearch, our system found a training recipe that reaches the same loss 1.3x faster than the best community solution, and substantially improves over the initial hand-optimized baseline.
Recursive just came out of stealth, and the team has been cooking 🔥
Our first results: an automated AI research system that can improve AI across 3 very different settings across training and GPU kernel optimization.
https://www.recursive.com/articles/first-steps-toward-automated-ai-research
@cong_ml pretty cool stuff @cong_ml ! have been hacking away at autoR loops / harness setups myself as well , all opensourced at @evo__hq
Domain 3: low-level GPU kernel optimization ⚙️
On Nvidia’s SOL-ExecBench, the same general system improved mean SOL score from 0.699 to 0.754 across 235 kernels - an 18% reduction in the gap to the theoretical optimum.
Domain 2: squeezing more out of a mature highly optimized training benchmark ⚡
On NanoGPT Speedrun, starting from a solution refined by the community over 2+ years, the system still found a sequence of discoveries to produce a further speedup: 79.7s → 77.5s.
We're releasing our blog post along with a lot of the artifacts our system came up with optimizing these problems. Lots of interesting techniques and implementations to dig into: https://github.com/recursive-org/first-steps-toward-automated-ai-research
What’s exciting to me is not just any one result.
It’s that one system can make progress across problems with very different bottlenecks: model quality under a compute budget, wall-clock training speed, and hardware-level kernel performance.
Thrilled to finally share some of the work we at @Recursive_SI has been doing since we launched!🔥
We've made a system that autonomously conducts AI research and tested it across three different settings: Small Language Model training, NanoGPT speedrunning, and kernel engineering!
A bit like Karpathy's Autoresearch, but scaled up and designed to be open-ended, we can push our system to optimize models, training algorithms, and kernels in really cool ways.
Blogpost: https://www.recursive.com/articles/first-steps-toward-automated-ai-research
Also super cool to see that the system works across several domains, not just Language model training. Super excited to keep pushing with this fantastic team!
Towards a system that discovers knowledge autonomously!
New SOTAs on @karpathy’s nanogpt + nanochat and @nvidia’s kernel benchmarks.
We are open-sourcing the artifacts of the system and contributing back to the research community!
http://x.com/i/article/2064569799313940481
@sharathraparthy Thank you 🫶
@cong_ml AI现在最大的问题是：说得比做的好听。真正能落地的项目，远比融资PPT里的少。 「Recursive just came out of stealth, and the team has been co...」
The common thread is the automated research loop.
Search over ideas, turn them into executable changes, test them, validate + cross-check the result, and use empirical evidence to plan the next ideas.
Extremely excited to keep pushing @Recursive_SI !!
@timshi_ai @karpathy @nvidia Damn such fast progress. Crushing it.
@cong_ml Congrats! Cool kernel optimisation results 😃
Recursive has released its first public results from an automated AI research system that independently discovers optimizations for model training pipelines and GPU kernels, posting state-of-the-art numbers on nanoGPT and NVIDIA benchmarks while open-sourcing every artifact produced.
Recursive just came out of stealth, and the team has been cooking 🔥
Our first results: an automated AI research system that can improve AI across 3 very different settings across training and GPU kernel optimization.
https://www.recursive.com/articles/first-steps-toward-automated-ai-research