AI Coding Agents: Harness Engineering Makes the Difference
Different Pi harness setups produce a six-turn success or a 32-turn crash on the same task.
Christopher Man posted a comparison of two harness setups for an AI coding agent. One finished the task in six turns. The other ran 32 turns before crashing. He listed specific problems with the Pi harness: Codex created an unneeded subagent, skipped fast mode on it, made 93 tool calls, lacked timeout or turn limits, and the MCP extension overloaded the subagent with context. Elvis Saravia replied that harness engineering is becoming one of the most important skills for AI engineers next to evals. Other replies noted GPT models default to subagents unless the system prompt restricts them.
wait maybe harness engineering is important
Combined views
Reactions
8 postsEL 
elvis@omarsar0Next to evals, harness engineering is quickly becoming one of the most important skills for AI engineers to have today.
- likes: 276
- replies: 46
- bookmarks: 106
- reposts: 20
HC Harrison Chase@hwchase17if only there was a company that helped you do both evals and harness engineering https://twitter.com/omarsar0/status/2094499914281566241
- likes: 126
- replies: 18
- bookmarks: 25
- reposts: 10
DG Dario Gieselaar@DarioGieselaar@omarsar0 Model behavior changing and your custom harness suddenly producing pathological behavior, or having to spend a huge amount of effort evaluating your harness across model updates
IN Ivan Neustroev@Anbeeld@int32max The post really lacks the context that would explain: this is not Pi's fault, it's poorly thought out system/tool prompts of whatever subagent extension you used.
PW Philip Ward@philipmward@omarsar0 Agreed, and if you’re not using evals I’d argue you’re not doing harness engineering. They both go hand in hand.
DG Dario Gieselaar@DarioGieselaar@omarsar0 It's not, it's just dotfiles all over again. People go insane over it and just create problems for themselves.