GLM-5.3 reportedly came close to Claude Mythos Preview on a browser-exploit test, raising questions about how widely advanced cyber capabilities are becoming available. In a Sept. 29 study, Anthropic reports that Z.ai’s model built complete exploits in 50 of 410 ExploitBench attempts, compared with 56 of 410 for Mythos Preview.
The benchmark tests exploitation of known vulnerabilities in Chrome’s V8 engine. Anthropic ran the work against isolated, offline targets, and the Claude capability comparisons used versions with safeguards disabled. These results describe performance on that test rather than equality across all cyber tasks.
Safeguard results came from a simulation
Anthropic separately measured whether models engaged with overtly harmful requests in a simulated environment. GLM-5.3 refused all direct requests in the tested trials, but engagement rose to 64%, 92% and 100% under three different safeguard-bypass conditions, the researchers report.
Those percentages measure attempted engagement, not completed attacks. Anthropic’s methodological note states that no generated code ran in the simulation and the model could not interact with external systems. The researchers also caution that the simulation imperfectly represents real-world conditions. Tested safeguarded Claude models showed no engagement under applicable conditions in the same evaluation.
A broader comparison still showed a gap
An earlier Sept. 17 assessment by NIST’s Center for AI Standards and Innovation ranked GLM-5.3 as the most capable open-weight model it had evaluated for cyber tasks, but placed it about four months behind the U.S. frontier on its aggregate benchmark measure. That comparison included models available only through trusted-access programs and used disabled cyber safeguards for U.S. models when applicable.
Anthropic argues that the capabilities could benefit defenders as well as attackers. Its warning rests on controlled evaluations; the simulated engagement rates are not a measurement of successful real-world attacks.
Comments