2d ago

Stanford paper finds single-agent LLMs match or beat multi-agent systems under equal token budgets

Stanford paper by Dat Tran and Douwe Kiela finds single-agent LLMs match or exceed multi-agent systems on multi-hop reasoning benchmarks when total thinking tokens remain fixed. Tests using Qwen3, DeepSeek-R1-Distill-Llama, and Gemini 2.5 showed consistent single-agent advantages. Performance gaps stem from irreversible information loss during multi-agent coordination and handoffs. Researchers examined budgets up to 10,000 tokens and suggest multi-agent approaches may only help at much larger scales such as one million tokens.

——0——
Original post
OPRohan Paul#1032Rohan Paul@ROHANPAUL_AI

New Stanford paper argues that, under equal reasoning budgets, one LLM usually solves multi-hop problems better than many coordinated ones. The core point is almost embarrassingly simple. A single agent keeps the whole problem in one internal chain of thought, while a multi-agent system has to slice that chain into messages, summaries, and handoffs. Every handoff is a compression step. And once reasoning is compressed, some information is easier to drop than to recover, which is why the paper leans on the Data Processing Inequality as a formal explanation rather than just an empirical hunch. The experiments back that up across Qwen, DeepSeek, and Gemini on FRAMES and MuSiQue: when thinking-token budgets are matched, single-agent systems usually match or beat sequential, debate, role-based, and ensemble setups. Here’s the part most people miss. Many celebrated multi-agent gains may not be architectural gains at all. They often come from spending more test-time compute, surfacing more visible reasoning, or benefiting from evaluation quirks that make the pipeline look smarter than it is. The paper is especially sharp when it looks for the boundary case instead of pretending the rule is universal. When the single agent’s effective context is degraded by masking, substitution, or misleading distractors, multi-agent pipelines become more competitive and sometimes win, not because message passing is magical, but because structure can partially stabilize corrupted reasoning. That is a much narrower and more useful claim than “more agents is better.” It suggests the real trade-off is not single versus multi so much as latent reasoning versus external coordination, with context quality and compute accounting deciding which side looks stronger. For multi-hop reasoning, the default should now be clear: start with one strong model, and treat extra agents as a repair strategy, not an upgrade. ---- Paper Link – arxiv. org/abs/2604.02460 Paper Title: "Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets"

2:08 AM · May 17, 2026 View on X
QUOTE POST(((ل()(ل() 'yoav))))👾#92(((ل()(ل() 'yoav))))👾@YOAVGO

breaking research reveals it is faster to train a DL model on a single GPU than it is on a cluster

Sanjeev AroraSanjeev Arora@prfsanjeevarora

Probably (let me say almost certainly) wrong if the main finding is in Table 1. Max thinking budget = 10K tokens. Hence comfortably within capabilities of a single call. Multi-agent reasoning helps when you need to generate say 1M tokens. Possible in a single call but ineffective.

10:45 PM · May 18, 2026 · 20.5K Views
6:59 AM · May 19, 2026 · 10K Views
QUOTE POSTSanjeev Arora#115Sanjeev Arora@PRFSANJEEVARORA

Probably (let me say almost certainly) wrong if the main finding is in Table 1. Max thinking budget = 10K tokens. Hence comfortably within capabilities of a single call. Multi-agent reasoning helps when you need to generate say 1M tokens. Possible in a single call but ineffective.

Rohan PaulRohan Paul@rohanpaul_ai

New Stanford paper argues that, under equal reasoning budgets, one LLM usually solves multi-hop problems better than many coordinated ones. The core point is almost embarrassingly simple. A single agent keeps the whole problem in one internal chain of thought, while a multi-agent system has to slice that chain into messages, summaries, and handoffs. Every handoff is a compression step. And once reasoning is compressed, some information is easier to drop than to recover, which is why the paper leans on the Data Processing Inequality as a formal explanation rather than just an empirical hunch. The experiments back that up across Qwen, DeepSeek, and Gemini on FRAMES and MuSiQue: when thinking-token budgets are matched, single-agent systems usually match or beat sequential, debate, role-based, and ensemble setups. Here’s the part most people miss. Many celebrated multi-agent gains may not be architectural gains at all. They often come from spending more test-time compute, surfacing more visible reasoning, or benefiting from evaluation quirks that make the pipeline look smarter than it is. The paper is especially sharp when it looks for the boundary case instead of pretending the rule is universal. When the single agent’s effective context is degraded by masking, substitution, or misleading distractors, multi-agent pipelines become more competitive and sometimes win, not because message passing is magical, but because structure can partially stabilize corrupted reasoning. That is a much narrower and more useful claim than “more agents is better.” It suggests the real trade-off is not single versus multi so much as latent reasoning versus external coordination, with context quality and compute accounting deciding which side looks stronger. For multi-hop reasoning, the default should now be clear: start with one strong model, and treat extra agents as a repair strategy, not an upgrade. ---- Paper Link – arxiv. org/abs/2604.02460 Paper Title: "Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets"

9:08 AM · May 17, 2026 · 19.7K Views
10:45 PM · May 18, 2026 · 20.5K Views