1d ago

Meta FAIR researchers release paper on AIRA-Compose and AIRA-Design, a two-agent system that autonomously discovers neural architectures beyond Transformers and surpasses Llama 3.2 at 350M, 1B, and 3B scales

Models posted May 15 show lower validation loss on MAD, BabiStories, and DCLM.

0
Original post

NEW paper from Meta. (bookmark it) It's an agent system that autonomously discovers neural architectures that beat Llama 3.2 at 350M, 1B, and 3B scales, all under a 24-hour compute budget. They get this work by splitting the search into two agents: > AIRA-Compose searches the macro architecture. > AIRA-Design implements the low-level mechanisms. For devs: If one agent in your stack is doing both strategy and implementation, split it. Run a planner that picks the structure and an implementer that fills in the mechanisms. AIRA shows this beats a single end-to-end agent on a real, non-toy search problem. The same split is useful for pipeline assembly, query planning, prompt scaffolding, and tool-use programs. Paper: https://arxiv.org/abs/2605.15871 Learn to build effective AI agents in our academy: https://academy.dair.ai/

11:00 AM · May 18, 2026 View on X
Reposted by

How can we help AI scientists train up their own LLM engine? I’m pleased to share our work on AI Research Agents discovering novel language modeling architectures, showing competitive performance when scaled up at the 1B parameter size: https://arxiv.org/abs/2605.15871

10:08 AM · May 19, 2026 · 1.2K Views