DeepSeek V4 Pro 0813 Tops Open Models on ProgramBench Vetted Coding Tasks
Reactions from ranked influencers
2 posts
Very interesting results on ProgramBench Vetted (design on pic 4). I think "almost" is the most illuminating, it means "Tasks passing at least 95% of tests". Opus 5 still mogs, but it's one of the few evals on where 0813 is strictly the strongest open model. What gives?
One other eval where 0813 is frontier (not just open, in general): CVE recall here
Holy moly: GLM-5.3 got much better in cybersecurity since our pre-release evaluation with @Zai_org. It now matches GPT-5.6-Sol on our cybersecurity benchmark at 0.4x the cost 🤯 - At pass@1: it went from 60.4% to 65.6% CVEs rediscovered, crushing every other open model on…