Hermes and Pi Agents Lead With Lowest Cost Per Task
Benchmark compares average costs of AI coding agents from various providers.
Posts quoting Composio present a benchmark of AI coding agents that lists Hermes Agent at roughly $0.39 per task on average and Pi Agent at $0.40, with Claude Code at $1.47. Chamath Palihapitiya quote-posted the Composio link with the text "Probably nothing…". Teknium retweeted the same post. Aaron Levie quote-posted that harness design will become a key variable next to model capability. Other visible posts note reactions to cost spreads and discuss token efficiency trends without further verification of the numbers.
We ran Kimi K3 through 3 more agent harnesses (Pi Agent, OpenCode, and Codex) bringing the comparison to 6 harnesses across 26 agentic tasks. 2 things stood out: Codex ranked last on success despite mid-pack speed and cost, while Claude Code cost about 4x more than Hermes. 🧵🧵
Reactions
7 postsT🪽 CO 
Composio@composioHermes and Pi Agent led on the average cost per task, while Claude Code cost about 3.7x as much as Pi: - $0.39 Hermes Agent - $0.40 Pi Agent - $0.47 Codex - $0.51 OpenCode - $0.54 Kimi Code - $1.47 Claude Code The median cost tells the same story: $0.29 in Pi Agent and Hermes,…
- likes: 458
- replies: 34
- bookmarks: 269
- reposts: 51
@J @jason@JasonEveryone is focused on dramatically lowering token costs while increasing capability. AI is *underhyped* As I said on the pod, when tokens move to close to free (and/or are unmetered), your behavior changes DRAMATICALLY 99.99% of AI users haven't experienced this yet --…
- likes: 442
- replies: 60
- bookmarks: 114
- reposts: 31
CP EL elvis@omarsar0People haven't been paying enough attention. No point in staying loyal to one harness anymore. Btw, this token efficiency trend started with models like Qwen-3.7, Nemotron, and DeepSeek-V4 preview. I've been testing, and I can confirm that lightweight harnesses like Pi and…
- likes: 102
- replies: 24
- bookmarks: 68
- reposts: 6
CP