Local-first Atomic Agent beats Hermes on GAIA benchmark

Local-first Atomic Agent beats Hermes on GAIA benchmark · Digg

Posts from X

Most Activity

VIEWS31.7KBOOKMARKS257LIKES356RETWEETS29REPLIES8

1324TECH

The competition is intensifying! The first local agent claims to be beating Hermes. • Runs Qwen, Gemma, and Llama via llama.cpp • Stable-prefix caching keeps long sessions cheap • TurboQuant shrinks the KV cache by 6.4× • Solved 37 tasks vs. Hermes’ 31 I'm a huge fan of open source. Therefore, I'm delighted to see significantly more competition here. Local and open source is the future.

AA

Atomic Agent@atomicagent_io

First local agent to beat Hermes on benchmarks! ✦ runs Qwen, Gemma, Llama via llama.cpp ✦ stable-prefix caching keeps sessions cheap ✦ TurboQuant cuts the KV-cache 6.4× smaller ✦ 37 tasks solved vs Hermes' 31 on GAIA Level 1 Open source on macOS, Windows & Linux 👇

RP Rohan Paul@rohanpaul_ai

3h ago · 3.2K Views

1252TECH

The 1st open-source local agent to surpass Hermes on GAIA benchmark. Loop design, not model size, drove this whole gap. @atomicagent_io , a local-first AI agent that runs on your machine, with local or cloud models. Open source on macOS, Windows & Linux. Both agents drove qwen-3.6-35b-a3b, on the same task budget. Atomic Agent solved 37 of 53 tasks, against 31 for Hermes, and it ran faster too. Most agent loops get costly because every turn re-sends the whole growing context, and small local models choke on that first. Atomic Agent keeps turns cheap. One inference returns a grammar-checked JSON array of tool calls, reads run in parallel, results get summarized, and the fixed prompt reuses cached work. Underneath sits a custom llama.cpp build called TurboQuant. It shrinks the memory cache up to 6.4 times and adds 30-50% more speed through tailored decoding. This is how a 9B model, qwen-3.5-9b, still cleared over half the same tasks at 52.8%.

AA

Atomic Agent@atomicagent_io

First local agent to beat Hermes on benchmarks! ✦ runs Qwen, Gemma, Llama via llama.cpp ✦ stable-prefix caching keeps sessions cheap ✦ TurboQuant cuts the KV-cache 6.4× smaller ✦ 37 tasks solved vs Hermes' 31 on GAIA Level 1 Open source on macOS, Windows & Linux 👇

SR Seth Rose@sethrose

4h ago · 214 Views

@kimmonismus "Hermes, please go and compare the codebase to your own and make updates"

RP Rohan Paul@rohanpaul_ai

3h ago · 702 Views

1252TECH

Official site of Atomic Agent https://atomicagent.io/ Github (Open source on macOS, Windows & Linux.) https://github.com/AtomicBot-ai/atomic-agent

RP

Rohan Paul@rohanpaul_ai

The 1st open-source local agent to surpass Hermes on GAIA benchmark. Loop design, not model size, drove this whole gap. @atomicagent_io , a local-first AI agent that runs on your machine, with local or cloud models. Open source on macOS, Windows & Linux. Both agents drove qwen-3.6-35b-a3b, on the same task budget. Atomic Agent solved 37 of 53 tasks, against 31 for Hermes, and it ran faster too. Most agent loops get costly because every turn re-sends the whole growing context, and small local models choke on that first. Atomic Agent keeps turns cheap. One inference returns a grammar-checked JSON array of tool calls, reads run in parallel, results get summarized, and the fixed prompt reuses cached work. Underneath sits a custom llama.cpp build called TurboQuant. It shrinks the memory cache up to 6.4 times and adds 30-50% more speed through tailored decoding. This is how a 9B model, qwen-3.5-9b, still cleared over half the same tasks at 52.8%.

C🪄ChrisBarnes1213 🪄🚀🌙@ChrisBarnes1213

3h ago · 17 Views

@skinbagwbones @sethrose @kimmonismus It’s 100% possible, our AI recommended to do the same instead of switching operating core. Run Atomic in sandbox, extract any proven advantages ✅

SY someone you know@skinbagwbones

4h ago · 36 Views

@sethrose @kimmonismus I’m not sure it’s possible. But maybe we should try🤣

SR Seth Rose@sethrose

3h ago · 19 Views

@ChrisBarnes1213 @skinbagwbones @kimmonismus When I've had to patch Hermes, this is exactly how I've patched it before. "Review Hermes Github for known issues. Is this one of them? If not, review [Other Agentic Tooling] to see how it works there. Does that help you identify a patch? If so, patch it."

SY someone you know@skinbagwbones

3h ago · 12 Views

@sethrose @ChrisBarnes1213 @kimmonismus Which model were you using if that’s not a secret

Z🏳️‍⚧️zoeey 🏳️‍⚧️ 💞@biggestwest

4h ago · 126 Views

@kimmonismus Looks like the underdog just got a turbo boost, time to see Hermes try to keep up. 🤝

ST Sam Tilston@samtilston

3h ago · 49 Views

@kimmonismus AI benchmarks are becoming a little like speed tests for cars Everyone has a faster lap time, but the road conditions matter

BE BeastTitanHunter@Titan1Beast

4h ago · 24 Views

@sethrose @kimmonismus @NousResearch !!!!

CH ChainBuilder.pro@Chainbuilderpro

4h ago · 19 Views

@kimmonismus kv compression matters most when an agent can revisit the same long tool trace without paying to rebuild it at every step

HN Henry Nguyen@henryhndev

4h ago · 6 Views

@kimmonismus 4 local tasks beating cloud models is wild—turboquant + stable prefix caching feels like the real unlock for cheap agentic loops.

SR Seth Rose@sethrose

3h ago · 5 Views

@skinbagwbones @ChrisBarnes1213 @kimmonismus At the time it was GPT-5.5 but i've patched a recent docker issue with GPT-5.6 recently (I did not submit a PR lol)