Benchmarks find Anthropic's Claude Code is the lowest-performing coding harness compared to Cursor CLI and OpenCode using identical models

@kunchenguid This is a hard angle to measure. Each harness can be heavily customized and tuned for your use case. Did they use the default settings? I doubt opencode is that much better than claude code.

@kunchenguid $1T valuations tend to mess with people's ethics 😀

as mark twain said, "It is difficult to get a man to understand something when his salary depends on his not understanding it."

@kunchenguid I think youre 100% right on #1, but the economic incentive is clear: model providers dont want to lose the cust relationship, nor do they want a downstream vendor (eg cursor) taking steps to route/optimize them into a commodity.

So their insane valuations require them to do it

@Bolmercl @kunchenguid all the tokens in the system prompt are cached note that 90% of the people who use Claude Code use it with subscriptions so anthropic is not dump to waste their tokens/money

@kunchenguid Claude code is optimized for cost not for performance by heavily relying on caching and using subagents with smaller models like haiku So it makes sense to see other tools perform better than Claude code but it’s not fair to compare performance without comparing cost

@kunchenguid I'm not really believing the benchmarks that say 1 point difference. Fable seems significantly smarter than GPT 5.5 at software design and code review. It results in much simpler code that a clear headed senior engineer would write, not overengineered towers of Babel.

When the harness alone causes 15-20 pt swings, it's suspect. Codex harness increases GPT 5.5's core by 20 points. Claude code harness decreases Opus 4.7's score by 13 points.

This seems to be measuring not model quality - but perhaps how aggressively a harness continuously loops in long open-ended tasks w/o human prompting.

@kunchenguid Claude code/codex best value prop has always been the discounted tokens on the consumer plans. You are always going to choose 10x more tokens over a slightly better harness. With enterprises now paying per-token, harness differentiation will be more relevant

@lawrence_stark_ i wouldn’t say that “stupid” though. openai deliberately allows all 3p apps to use their codex subscription freely which might play out well for them because more and more apps will be built on top of the openai platform as a result

@llmDestructor @Bolmercl @kunchenguid again 90% of the people who use Claude Code use it with the subscriptions so anthropic knows better how to handle their caching to reduce their costs

@Mng64218162 @kunchenguid The problems its not the input cache cost of the system prompt, is making the models dumber.

@kunchenguid Very interesting information. I noticed Codex seemed to be much faster and slightly more reliable in the 4.8/5.5 era. Fable in CC is still slow and token-heavy

@JakeKAllDay yeah totally. i can empathize with why they would do it. i just see it as a bad thing for the ecosystem and disappointed that somehow doing what’s good for society isn’t incentivized nearly as much as it should

@Mng64218162 @kunchenguid CC is not optimized for cost. It waste waaaaaay more tokens by giving so many dumb system prompt tokens to the models. They become dumber and waster waaaaay more tokens per task.

Yes exactly but they'd be stupid to allow you to go use another harness with their sub. Hence the soft ban in 3 days on headless

For consumers, model providers should not lock us into their harnesses. For them, it's in their best interest to pull an Apple like move

But whatever, open source models will be>90% as good and >10x cheaper and they will eat up most tokens

@kunchenguid isn't it against Anthropic ToS to use claude subscription with other harnesses? I remember this being a thing a while back, when Openclaw was a thing

@kunchenguid This is the right lesson: dumb loops that ship usually beat fancy orchestration that needs a therapist.

@kunchenguid "my speculation would be that in enterprises people will be restricted to adopt fable & mythos only on some mission critical tasks, not used at scale"

That's sorta the point?

What makes you believe that model companies can't write a good harness, especially when they can tune the model as part of training?

Are there any model companies not using their own harness themselves? Are you suggesting Anthropic engineers should switch to OpenCode to improve their productivity?