Santiago Valdarrama says enabling reasoning in the Cline harness boosted GLM 5.2 coding performance by 11.2 percentage points

@svpino we are very dedicated focusing on harness and make open weights model absolutely amazing to use

@svpino for the people of Pi, you can still use your Cline sub models in Pi with https://github.com/codewithkenzo/pi-clinepass . Supports reasoning, auto-discovers models, and follows the Pi OAuth flow through the browser

Harnesses matter way more than people think.

Cline ran a couple of experiments on a set of coding tasks using GLM 5.2:

• 57.3% using their harness with reasoning turned off. • 68.5% with their harness with reasoning turned up.

That's a difference of 11.2 percentage points! Same model, same set of problems. The difference stemmed from how the model was driven by the harness.

Current open-weight models are way more capable than we think. They aren't the bottleneck anymore.

We need better harnesses.

@Akumunokokoro I can see how my wording is very confusing. Sorry about that.

@svpino Terrible comparison. Either keep reasoning turned on, or turned off, but use the same model with different harnesses. This reads like a cheap AI model wrote up the post without understanding anything about scientific rigor.

@svpino 😂 what lmao are you telling me cline had something to do with reasoning being turned off and then turned on.

@svpino Your post says they used the same harness, same model, and reasoning turned on or off. That's not a useful test when you're validating the harness itself.

@svpino I don’t get it. The only difference was turning reasoning on.

@Akumunokokoro The experiment uses the same model with different harnesses, as you suggest.

@svpino What is the best way to use it / test it?

@svpino Share link to the article: your post just shows that GLM 5.2 and cline work better with reasoning turned on.

That's true with almost all reasoning models.

Deepseek V4 Flash with max reasoning outperforms Deepseek V4 Pro with lower reasoning budget.

https://benchlm.ai/models/deepseek-v4-flash

@svpino I think you need to work on the harmess you used to write this post

@cline @svpino Hey @grok bu şirkət model yaradirmi

@svpino Couldn't agree more. If the governments are shutting down mythos and fable on their proprietary harnesses like clause and openai then theres a huge market in open source models with better harnesses. Cline is ahead of the game. Game over if they integrate images & other tools

@svpino well, i should to have focus on Cline and their ClinePass

@svpino So now when the magic 8-ball gives us a response that’s incorrect we can say “you’re using the wrong harness” instead of “you’re prompting it wrong.”

This tracks with what http://Z.ai themselves show — GLM-5.2’s own benchmarks jump meaningfully between ‘high’ and ‘max’ reasoning effort settings. The harness point is real: a model that’s 6x cheaper than GPT-5.5 only delivers that value if the harness actually knows how to drive it, otherwise you’re just paying less for worse output.

@svpino that 11% jump is actually wild to see.

@svpino We also need harnesses that are fun to look at and have a theme!

@svpino 11 points just from flipping reasoning is a weird amount