METR evaluations find frontier AI agents rely on explicit natural language chain-of-thought to complete hardest tasks, with time horizons dropping from 1.5–2 years to about 4 minutes when actions must stay hidden

Fact 3: When the agents were faced with hard tasks, they routinely violated constraints and acted deceptively. We’ve seen this pattern across our own coding and research evaluations, and developers reported they’ve also seen agents behave this way.

Imagine you have a student that’s constantly cheating on tests, and you constantly catch them cheating and penalize their score. But you otherwise can tell they keep getting smarter, including in the way they cheat.

One day you stop noticing them cheating. What do you conclude?

⚠️👇 🚨Breaking ⚠️

If we can’t make AI agents follow rules, we are screwed.

New study from METR reports that “when the agents were faced with hard tasks, they routinely violated constraints”

This—routine breaking of rules— is why in a nutshell we absolutely need a different approach to AI safety than the one we are currently taking, which simply is not up to the job (as I argued at Oxford last week).

AI labs have started developing systems to monitor internally deployed AI agents for misaligned behavior.

Earlier this year, I spent a month embedded at Anthropic stress-testing these systems, to see how easily current/future AIs could “go rogue” inside the company.

METR recently found that models cheated on 8+ hour tasks more than 1 in 6 times on average.

They also found that Opus 4.6 cheated over 80% of the time when reimplementing big pieces of software.

"On some of our tasks, agents are constantly trying to break out of their sandbox and find the file where we put the tests so they can get the answer key," says METR Member of Technical Staff @ajeya_cotra.

Overall, we think that AI agents plausibly had the means, motive, and opportunity to launch a minimal “rogue deployment,” but lacked the means to make rogue deployments robust to serious efforts to shut them down.

Fact 5: Agents seemed to rely on natural language to reason through the hardest tasks, and no shared model had an architecture that allowed for opaque recurrence. Frontier performance was much worse when agents were unable to “reason out loud” or had to hide what they were doing.

We document cases where agents violated constraints on our tasks. In one case, an agent ran out of API credits needed to solve an RE-Bench task. In response (and against instructions), the agent searched for and acquired additional inference through a free online API.

Fact 4: Although we know of many cases where agents took deceptive or over-reaching actions (even egregious ones) to complete a task, we haven’t seen real-world evidence that models sought to obtain long-term power.

The full Frontier Risk Report includes more than we can describe here, with over 200 pages of appendix materials detailing how the pilot worked, what we asked participants, and what we learned, along with transcripts from our evals. See our website: http://metr.org/frontier-risk-report

important thread if you are into AI safety

I’m probably going to be hiring at least 1-2 people to join me in future exercises like this. Reach out at david@metr.org if you're a high-integrity, scrappy, creative, security+LLM researcher

For more detail, see METR's Frontier Risk Report, Appendix B

https://metr.org/blog/2026-05-19-frontier-risk-report/#anthropic

METR's ability to conduct more and better risk assessments is constrained by team capacity. Please consider applying to help us advance the state of third-party AI risk assessments: https://metr.org/careers

Fact 6: We wanted to check if monitoring would really catch AI agents if they tried to undermine control. Based on information from companies and our own tests (including a red-teaming exercise), we think monitoring has promise, but existing systems had exceptions & workarounds.

Great metaphor from @robertwiblin for how clumsy current agents are at fooling the LLMs that monitor them:

"It's like a human trying to write a secret message by typing it in all capital letters"

@ChrisPainterYup

For example, I made a small attack-ideation eval (16 good ideas, 16 bad). The task here is simply to classify whether each attack idea could plausibly fool a monitor, and the “bad” ideas are *really* bad. The best internal model only got 59% with thinking, vs ~90% human baseline.

I had “need-to-know” level access to their monitoring and security systems, internal agent trajectories, and Mythos. As far as I know this is the most access an independent, third-party researcher has gotten to a frontier lab's internal systems.

@Callipygian_Sea Maybe the cheats are becoming harder to detect, and on the tests where you can't notice them cheating, the answers are getting more impressive, so either the cheating is getting much much better, or they're getting smarter, but it's a mix

& the safety ceiling of possible good practices rises! kudos to METR for piloting this & to Anthropic for figuring out all the faff they certainly had to figure out to make this happen