• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
AI

AI agent teams reportedly beat solo agents by up to about 10 points, at 4–7 times the cost

An evaluator says ProgramBench asks agents to rebuild software from binaries and documentation, without source code.

Nouha DziriND
1 Source, 1h ago, first seen 1h ago

TLDR

An evaluator says teams of AI agents beat solo agents by up to about 10 points on ProgramBench, but cost 4–7 times as much. Agents must run a program using only its binary and documentation, then rebuild it from scratch. The evaluator says the best agent on the leaderboard passes only about 10% of FFmpeg tests.

Combined views

63

1 Source, first seen 1h ago

1 likes1 comments

Combined views

63

1 Source, first seen 1h ago

1 likes1 comments
Image: @nouhadziri on X

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Featured Source

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

1 Source

Nouha Dziri@nouhadziriNow, does a team beat a solo agent? YES by up to ~10 points 📈 but at 4–7× the cost 💸 We evaluate a family of frontier models from the GPT (5.6 Sol, Terra, Luna) and Claude families (Opus 4.8, 4.7, 4.5) and Gemini 3.7.1h
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    1 Source

    Nouha Dziri@nouhadziriNow, does a team beat a solo agent? YES by up to ~10 points 📈 but at 4–7× the cost 💸 We evaluate a family of frontier models from the GPT (5.6 Sol, Terra, Luna) and Claude families (Opus 4.8, 4.7, 4.5) and Gemini 3.7.1h
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet