Moonshot AI has released a coding-focused update to its Kimi series that delivers nearly identical results on the WeirdML benchmark while slashing average output length from roughly 17,000 tokens down to 9,000. The new model keeps the same statistical performance band as its predecessor yet trims reasoning overhead, which directly lowers per-task API spend for long-horizon coding workflows.
In practice this is a decent price cut, but clearly K2.7-Code is called this way because it's not a fully general purpose model. More for normal SWE. Waiting for 5.2.
Kimi k2.7 code scores 54.1% on WeirdML, not advancing on k2.6 (55.9%), but within the error bar.
k2.7 is cheaper and uses less output tokens (~9k) compared to k2.6 (~17k).
Kimi K2.7 Code posts a 54.1 percent WeirdML score that sits within the margin of the prior 55.9 percent result, while the token cut makes repeated agent loops noticeably cheaper. Practitioners tracking the benchmark note the model now sits on a more attractive point along the cost-performance frontier for specialized coding tasks.
The release includes weights under a modified MIT license on Hugging Face plus an OpenAI-compatible endpoint, so teams can run local experiments or route traffic through existing platforms without new contracts. No broader claims about general-purpose gains or market shifts are supported yet.
Positive users praise Kimi K2.7 for matching prior WeirdML scores at lower cost and tokens while trusting the leaderboard and anticipating updates, whereas negative users express disappointment and accuse API providers of nerfing models.
@teortaxesTex 🤔
Surprisingly though, we found that a recent open model, Kimi-K2.7, performed very well on ML engineering. And Fable performed even worse than Opus. This could be either because of the inflated cost, or the guardrails put on ML tasks. (4/5)
@xeophon everything before this chart looks bonkers though
@teortaxesTex 🤔
Kimi k2.7 code scores 54.1% on WeirdML, not advancing on k2.6 (55.9%), but within the error bar.
k2.7 is cheaper and uses less output tokens (~9k) compared to k2.6 (~17k).
WeirdML v2 is now out! The update includes a bunch of new tasks (now 19 tasks total, up from 6), and results from all the latest models. We now also track api costs and other metadata which give more insight into the different models. The new results are shown in these two figures. The first one shows an overview of the overall results as well as the results on individual tasks, in addition to various metadata.
The second figure shows cost vs performance and shows a clear scaling with better results for higher costs. We also have a very varied pareto frontier with 11 models from 6 different companies having the best accuracy for a given cost for at least some of the cost range. Grok 3, Claude Opus 4 and GPT 4.5 are the ones that underperform for their costs, while Gemini pro and o3 pro have the best results at the highest costs. Qwen3 30B3A, grok 3 mini and deepseek R1 also each represent a good chunk of the pareto frontier.
Both were run through the Novita provider on Openrouter.
@htihle :(
@htihle 感谢您的工作，这是我最信任LLM的榜单之一，期待GLM5.2的测评。
@teortaxesTex Waiting for GLM 5.2 on DeepSWE & WeirdML.
@htihle Why not run through official API. Kimi already proved that othe API providers are nerfing their models.
@teortaxesTex 降本增效确实精准 坐等下个大版本更新
@teortaxesTex Run through Novita not official API. Probably a quntized model.