UK US Safety Institutes Find Kimi K3 Trails Frontier Models
Joint safety institutes assess Chinese model against American leaders on cyber benchmarks.
Entities: Kimi K3, CAISI, U.S. Department of Commerce, AI Security Institute (AISI), David Sacks, Howard Lutnick
UK AISI and US CAISI conducted a preliminary evaluation of Kimi K3 cyber capabilities on the Last Ones range. The model underperformed leading US frontier AI systems, reaching performance levels comparable to models released about six months earlier. Reactions from figures including Elon Musk, David Sacks, and researchers emphasized that American models maintain a lead, with the gap potentially larger when including unreleased lab systems. The assessment highlights ongoing international efforts to benchmark AI safety and capabilities without claiming any model has reached parity with current US leaders.
CAISI’s latest blog post evaluates Kimi K3 and its cyber capabilities. Based on a preliminary cyber-focused evaluation, Kimi K3 performed significantly below the leading U.S. frontier AI models. https://www.nist.gov/news-events/news/2026/07/uk-aisi-caisi-preliminary-assessment-kimi-k3s-cyber-capabilities
Combined views
Reactions from ranked influencers
31 posts"Kimi K3 performs significantly below the most recent frontier cyber-capable models" On UK AISI's cyber range "The Last Ones" Kimi-K3 reaches on average step 17 out of 32. This seems to be around the level of Opus 4.6, a 6 month old model.
Together with the US Center for AI Standards and Innovation (@NIST), we ran evaluations of Kimi K3 focused on its cyber capabilities. Kimi K3 performs below leading US frontier models on our preliminary cyber evaluations.
sometimes you just got to listen to Lisan
Together with the US Center for AI Standards and Innovation (@NIST), we ran evaluations of Kimi K3 focused on its cyber capabilities. Kimi K3 performs below leading US frontier models on our preliminary cyber evaluations.
This puts K3 on the level of Mythos Preview on its best attempt and between Opus 4.6 and Mythos Preview on average
On "The Last Ones" (TLO), a 32-step simulated corporate network attack (~20 hours for a human expert), Kimi K3 reached step 17 on average. In 1 of 10 attempts, Kimi K3 completed TLO within the 100M token limit.
32% puts it at GPT-5.5 (Codex), which seems to be the runs by CMU (the higher runs are by OpenAI)
On exploit development (ExploitBench), Kimi K3 scores 32%. On ExploitBench, Kimi K3 failed to develop exploits that achieved arbitrary code execution (ACE), the highest-severity outcome in exploit development. Kimi K3 achieved ACE on 0/41 samples.
@xeophon Over a 20x cost difference??