Agents' Last Exam benchmark launches, finding GPT-5.5 and Fable 5 score under 2.6% on complex professional tasks

ALE is built from real work, not synthetic tasks. Every task is derived from a real project that a human expert previously completed, and converted into a verifiable evaluation with objective grading.

No vibes. No human judges. Fully reproducible.

ALE spans 55 non-physical occupations, grounded in the O*NET / SOC 2018, the U.S. federal occupation taxonomy.

Built with 300+ experts from 100+ institutions across science, engineering, medicine, law, finance, education, and many other fields.

Two takeaways for me: (1) GPT-5.5 is beast, and (2) lots of work left.

In ALE, Fable 5 joins GPT-5.5 and Composer 2.5 in the same overall performance cluster.

But performance is only half the story.

Cost per task: → Fable 5: ~$15.70 → GPT-5.5: ~$3.80 → Composer 2.5: ~$1.33

At current pricing, Fable 5 delivers similar performance while costing roughly 4–12× more per completed task.

The most common failure mode remains a familiar one: Agents declare success before they've truly verified their work.

A typical completion reads: "Done. All checks pass." Yet the output may be missing required files, contain incorrect counts, omit key fields, or violate explicit constraints in the task specification.

These failures occur far more often than many people expect. You can explore concrete examples in https://agents-last-exam.org/blogs/agent-showdown.

How does ALE compare to existing agent benchmarks? Many of today's agent benchmarks are rapidly saturating as frontier systems improve.

ALE is designed to measure a different capability frontier: sustained, economically valuable work in real-world professional domains.

• 55 industry domains • 1,500+ expert-sourced tasks • Full GUI + CLI environments • Outcome-based, verifiable evaluation

If your agent only operates in the terminal, we've also released ALE-CLI: a CLI-only subset of the benchmark.

Why do ALE's results look different from some other benchmarks, especially for Fable 5?

Because there is no universally best agent.

Every frontier model, including Fable 5, has domains where it shines and domains where it struggles.

Aggregate scores average over 55 occupations and 1,500+ tasks, causing many models to cluster together. But the average is not the story.

The real signal lies in where agents succeed, where they fail, and how those patterns differ across domains. On identical tasks, different models often fail for very different reasons.

Explore the interactive breakdown in our blog → 👉 https://agents-last-exam.org/blogs/agent-showdown

ALE-CLI is a CLI-only subset of ALE. Compared to Terminal-Bench and SWE-bench-Pro, it is broader, longer-horizon, and substantially more challenging:

• Broader. Tasks span 40 of ALE's 55 industry subdomains, compared to just 6 in Terminal-Bench and 5 in SWE-bench-Pro.

• Longer-horizon. Human completion times range from hours to weeks, rather than minutes to days.

• Harder. The best-performing agent achieves only a 25.2% pass rate, compared to 82.0% on Terminal-Bench and 59.1% on SWE-bench-Pro.

There's still a long way to go, and plenty of headroom left to climb. 📊👇

@dawnsongtweets Fable 5’s intentional underperformance in Life Sciences is probably bringing down some of the aggregate scores though, right?

@dawnsongtweets I feel like lots of these tasks are forcing agents to operate traditional software. What's the rationale behind it?

@dawnsongtweets There "real" tasks are a fraction of the real everyday job for most people. Real tasks are way messier, and this benchmark is deeply missleading. There is no way someone with good faith will think that composer 2.5 is similar to gpt 5.5 and not to mention even Fable...

@GuangyuRobert @dawnsongtweets We did not limit the tools. The environments provide the necessary tools/software agents need. Whatever bash/API/software is used to achieve the goal is acceptable. We only measure the outcome.

@dawnsongtweets Whoever is saying agents will be ready to take jobs soon is dreaming, the harness and the model are no where near being able to complete non structured tasks autonomously maybe in this generation but not anytime soon

@dawnsongtweets Is the benchmark open source? Id like to give a crack at applying my private Harness/Environment/Orchestration engineering systems toward it.

Confident that, with a few iterations, today’s SOTA models may be able to hit 90-100%.

@dawnsongtweets I think your approach is wrong. We don’t need AGI-like agents. Agents good at specific jobs already exist, they are already working, and they will only improve

@dawnsongtweets but job readiness has nothing to do with human competence, it's about automatable tasks

@GuangyuRobert @dawnsongtweets From the tasks I've seen it seems like they're presented with industry standard tools in the descriptions.

We have decades of (software) tools that people use daily, and you can't really put a HTTP server and JSON codec in all of them and call it MCP.

@dawnsongtweets Thanks for this test , proves my point that humans are better long term.

@chainoflack @dawnsongtweets FYI Composer 2.5's performance is similar to Opus 4.7 https://www.reddit.com/r/tech_x/comments/1tgyg3a/cursor_claims_that_its_new_model_composer_25_is/.

More analysis is in https://agents-last-exam.org/blogs/agent-showdown as well, where the composer excels.