AI Judge changed title after evaluation, original title: "Cloudflare security team concludes patch acceleration inadequate after testing Anthropic Mythos AI"
Assessment focused on practical security adjustments beyond faster patching.
Brendan Dolan-Gavitt#1386
Cloudflare's security team spent the last few weeks testing Anthropic's Mythos against fifty of our own repositories. What we learned about offensive AI, why faster patching is the wrong reaction, and what the architecture around vulnerabilities has to look like next. https://cfl.re/49BRUqW
Users praise Cloudflare's Mythos AI test for chaining low-severity bugs into real exploits and advancing security feedback loops, while others call that capability terrifying and warn it will widen security issues.
Finally a semi-useful read on Mythos that is free of myth and talks about what this means more practically (not this is the end of the world as we know it, but how do we deal with faster patches and attacks from AI as other models scale to chained exploits)?
This is the kind of conversation we need, not idiotic ones about the end of all software.
We need "what is the right answer?" because these models are coming and will get better so how to we put our heads together and make better/more secure software across the world?
And it can't just be patching the 100 or so projects that got access to Project Glasswing.
That is not gonna help the world.
We need to figure out how does everyone else who is not part of the special chosen people to get blessed with access to test and patch their stuff, aka the open source projects and closed software that is not Office or Cloudflare but the 99.99% of software that runs everything else in the world?
What is the right loop cycle to help people patch and fix things at the source?
In the long run, AI will make software more secure, not less.
But it will change how teams have to work to get there.
Figuring that out means putting it in more team's hands sooner rather than later.
Cloudflare's security team spent the last few weeks testing Anthropic's Mythos against fifty of our own repositories. What we learned about offensive AI, why faster patching is the wrong reaction, and what the architecture around vulnerabilities has to look like next. https://cfl.re/49BRUqW
Cloudflare's security team spent the last few weeks testing Anthropic's Mythos against fifty of our own repositories. What we learned about offensive AI, why faster patching is the wrong reaction, and what the architecture around vulnerabilities has to look like next. https://cfl.re/49BRUqW
Cloudflare pointed Anthropic's Mythos Preview at 50+ of their own repos.
They call it a step-function forward "Mythos Preview is a real step forward, and it's worth saying that plainly before getting into anything else."
The big finding isn't the bugs it caught - It's that the model can take several low-severity vulnerabilities - the kind that sit invisible in backlogs - and chain them into a single working exploit. Write the proof-of-concept. Compile it. Run it. Adjust when it fails. Try again.
That loop is what separates a scanner from a researcher.
The other finding security teams should pay attention to: "patching faster" is the wrong response. If your regression testing takes a day, a two-hour SLA just means you ship broken fixes. The architecture around the vulnerability matters more than the speed of the patch.
Mythos is not just hype. It shows its power in real-world use cases.
Cloudflare's security team spent the last few weeks testing Anthropic's Mythos against fifty of our own repositories. What we learned about offensive AI, why faster patching is the wrong reaction, and what the architecture around vulnerabilities has to look like next. https://cfl.re/49BRUqW
terrific deep dive on Mythos from @cloudflare
Cloudflare's security team spent the last few weeks testing Anthropic's Mythos against fifty of our own repositories. What we learned about offensive AI, why faster patching is the wrong reaction, and what the architecture around vulnerabilities has to look like next. https://cfl.re/49BRUqW
@Cloudflare saving you the time: cloudflare ran mythos against 50 of their own repos and the conclusion wasnt patch faster its that your entire vuln discovery and containment architecture needs to change before ai assisted offense becomes the norm. most teams arent ready for that yet
AI models should offer guardrails toggle for regular people as well.
For instance, I should be able to tell ai model that I own this particular repo and I want no security guardrails on this repo to do security testing.
We maybe can use .txt file or something similar to prove domain/repo ownership. So AI knows that I am testing my own system.
It’s stupid that Claude rejects when I try to harden my security. The very feature that supposed to protect me actually makes me even more vulnerable in the age of AI.
“Patch faster” was never the right answer. The fundamental problem is the lack of architectural boundaries. If an AI can chain small bugs into meaningful access, that means the system allowed tiny local failures to compose into global authority. The fix is not just faster remediation - it is bounded access, explicit confirmation, scoped grants, isolated execution, and evidence by design.
@Cloudflare The faster patching trap is real. Offensive AI changes the review loop, not just the speed dial. Teams need better inventory, reproducible evidence, prioritization, ownership, and logs that explain why the agent flagged or changed something.
Faster patching is not the full answer.
Offensive AI changes the shape of vulnerability risk from isolated findings into evolving trajectories.
The next security layer has to observe how risk compounds over time.
Same with agentic AI:
the dangerous part is often not one event.
It is the drift path before failure becomes visible.
@Cloudflare Just let us all use Mythos already lol
@Cloudflare Cloudflare hosts a 50-repo test, Anthropic gets a 382K-impression validation, and the 'architecture around vulnerabilities' bit is the Glasswing tier upsell. Why faster patching is the 'wrong reaction' is on-brand. Patching doesn't have a subscription line.
とてもいい記事。 日本語でわかりやすく書くとこうだ。
---
Cloudflareが、かなり高度なAIを使って、自社システムの弱点探しを試したら、想像以上に“攻撃者っぽい考え方”ができるようになっていた。だから、AIを安全対策に使う側も、攻撃に使われる前提で守り方を変えないといけない。
という内容です。
何がすごかったのか
これまでのAIは、
「ここに怪しいバグがあります」
くらいまでは言えても、
「その小さなバグを、別のバグと組み合わせると、実際に攻撃できます」
というところまでは苦手でした。
ところが今回の Mythos Preview というAIは、単にバグを見つけるだけでなく、
1つ目の弱点 ↓ 2つ目の弱点 ↓ 3つ目の弱点 ↓ 結果としてシステムを乗っ取れるかもしれない
というように、複数の小さな弱点をつなげて、現実の攻撃になるかを考えられた、という話です。
これはかなり大きいです。
人間の優秀なセキュリティ研究者がやるような作業に近づいています。
「証明」までやるのが大きい
普通は、AIや診断ツールが
「このコードは危ないかもしれません」
と言っても、それだけでは困ります。
本当に危ないのか？ 実際に悪用できるのか？ 直すべきなのか？ それともただの勘違いなのか？
を人間が確認しなければいけません。
でも今回のAIは、怪しい場所を見つけるだけでなく、実際に試すための小さなプログラムを書き、それを動かして、
「本当にこの問題は起きる」
というところまで確認しようとした、ということです。
つまり、“怪しいです”ではなく、“実際に起きました”に近づいた。
ここが強いです。
ただし、AIはまだ不安定
面白いのは、同じような依頼でも、AIがある時は
「それは危険なのでできません」
と言い、別の聞き方をするとやってくれることがあった、という点です。
つまり、AI自身の安全ブレーキはあるけれど、かなりムラがある。
同じ内容でも、聞き方や状況によって答えが変わる。
だからCloudflareは、
「AIの自発的なブレーキだけでは不十分。一般公開するなら、さらにしっかりした安全対策が必要」
と言っています。
AIにただ「このシステムの弱点を探して」と言ってもダメ
この記事でかなり重要なのはここです。
Cloudflareは最初、普通の開発用AIに対して、
「このプログラム全体から弱点を探して」
と頼んでいたようです。
でも、それではあまりうまくいかなかった。
理由は、人間でも同じですが、範囲が広すぎるからです。
たとえば、
「この会社の問題点を全部探して」
と言われても、漠然としすぎています。
それよりも、
「この部署の、この作業の、この入力部分に、外部から変なデータが入ったときの危険を調べて」
と言われた方が、はるかに精度が上がります。
AIも同じです。
広く雑に探させるより、狭く具体的に探させた方がよい、ということです。
Cloudflareが作った仕組み
Cloudflareは、AIを1体だけ使うのではなく、複数のAIに役割分担させています。
かなり専門用語が多いですが、簡単に言うとこうです。
1. まず全体像を読むAI
システムの構造を理解します。
どこから外部の情報が入ってくるか。 どこが重要な処理か。 どの部分が攻撃されやすそうか。
地図を作るイメージです。
2. 弱点を探すAI
次に、多数のAIがそれぞれ狭い範囲を担当して、弱点を探します。
1体のAIに全部やらせるのではなく、 50人の調査員に分担させるような感じです。
3. 別のAIがツッコミを入れる
見つかった弱点に対して、別のAIが
「それ、本当に危ないの？」 「ただの勘違いでは？」 「実際には攻撃できないのでは？」
と確認します。
これはかなり重要です。
AIは自信満々に間違えることがあるので、別のAIに反論させることで、無駄な報告を減らしています。
4. 本当に外から攻撃できるか確認する
バグがあっても、外部の人がそこまでたどり着けなければ、危険度は下がります。
なので、
「その弱点は、実際に外部から届く場所にあるのか？」
を調べます。
Cloudflareはここを特に重要視しています。
単に「バグがある」ではなく、 **「外から悪用できるバグなのか」**を見ているわけです。
5. 報告書にする
最後に、AIが決まった形で報告書を作り、人間や社内システムが確認できるようにします。
この記事の本質
この記事の本質は、単に
「AIでセキュリティチェックが便利になった」
という話ではありません。
もっと怖くて、もっと重要な話です。
AIは、守る側にも使えるが、攻める側にも使える。
これからは、攻撃者もAIを使って、今までより速く、深く、システムの弱点を探せるようになる。
だから防御側は、
「バグが出たら急いで直す」
だけでは足りない。
もちろん早く直すことは大事です。
でも、それだけだと限界があります。
@Cloudflare
@Cloudflare @grok TLDR for me
@Cloudflare @eastdakota @pangramlabs AI?
@Layton_Gott @Cloudflare
@Cloudflare @pangramlabs human?
@Cloudflare Curious what Mythos surfaced that your internal scanners missed. In BEC cases we've worked, the gap wasn't unpatched CVEs. It was logic flaws no scanner flagged. Does offensive AI find those, or does it still chase known-pattern vulns?
@Dan_Jeffries1 Maybe "in the long run, AI will make software more secure, not less." Maybe not. What's certain: in the short run, it makes software less secure. Patching will always be harder than finding exploits.
@Layton_Gott @Cloudflare We'll only get to use Mythos once they have a more powerful one.
AI Judge changed title after evaluation, original title: "Cloudflare security team concludes patch acceleration inadequate after testing Anthropic Mythos AI"
Assessment focused on practical security adjustments beyond faster patching.
Brendan Dolan-Gavitt#1386Cloudflare's security team spent the last few weeks testing Anthropic's Mythos against fifty of our own repositories. What we learned about offensive AI, why faster patching is the wrong reaction, and what the architecture around vulnerabilities has to look like next. https://cfl.re/49BRUqW
