Report
DeepSeek V4.1 Flash 1017 reportedly ties for second at pass@3 on a cybersecurity benchmark
A post describing its author's benchmark reports $2.40 per CVE found, versus $64.96 for Opus 5.
TLDR
The author of the benchmark post reports that DeepSeek V4.1 Flash 1017 rediscovered 87.5% of the benchmark CVEs (28/32) at pass@3, tying DeepSeek V4 Pro 0813 for second behind GPT-6 Astra's 90.6%. Single-run recall was 70.8%, up from 65.6% for the previous Flash; precision rose from 78.9% to 86.4%. The reported cost was $2.40 per CVE found, versus $64.96 for Opus 5 and $135.23 for Astra.
Combined views
3.5K
3 Sources, first seen ago
