AI agent teams reportedly beat solo agents by up to about 10 points, at 4–7 times the cost

An evaluator says teams of AI agents beat solo agents by up to about 10 points on ProgramBench, but cost 4–7 times as much. Agents must run a program using only its binary and documentation, then rebuild it from scratch. The evaluator says the best agent on the leaderboard passes only about 10% of FFmpeg tests.