Users are excited about Sol edging Terra in the Hermes Agent PR Review Benchmark because of its speed completing work, cool features, and nice cost-intelligence options.
Based on 7 visible X reactions from 16 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@Teknium Thanks! Appreciate it. Excited we have these in Hermes 🔥. Some pretty nice optionality for cost & intelligence
@Teknium Sol is slowwwwww, but I like it. One shooted one day of work in 1 hour.
@Teknium nice, which handled merge conflicts better
@Teknium wow 8.8. cool
Qualitative review: Ranking: Sol > Terra — but the gap is one verdict. Strip out #2806 and they're near-identical on quality. Terra's profile is "correct, honest, slightly shallower, never dangerous": its only miss was conservative-direction, its policy handling was flawless, and it uniquely cited a locked test contract Sol missed. Sol's profile is a genuinely stronger investigator — the #2806 supersession chase and the diff-internal ordering bug are the kind of finds that separate a good reviewer from a great one.
Qualitative review: Ranking: Sol > Terra — but the gap is one verdict. Strip out #2806 and they're near-identical on quality. Terra's profile is "correct, honest, slightly shallower, never dangerous": its only miss was conservative-direction, its policy handling was flawless, and it uniquely cited a locked test contract Sol missed. Sol's profile is a genuinely stronger investigator — the #2806 supersession chase and the diff-internal ordering bug are the kind of finds that separate a good reviewer from a great one.
Did a quick 5 PR review test benchmark of Sol vs Terra in Hermes Agent for Hermes Agent PRs Both are quite good. (Fable judged) https://x.com/Teknium/status/2075381154669293979/photo/1
From the two run summaries, same 5 PRs: - Sol: $1.66 total ($0.332/item) - Terra: $0.46 total ($0.092/item)
Users are excited about Sol edging Terra in the Hermes Agent PR Review Benchmark because of its speed completing work, cool features, and nice cost-intelligence options.
Based on 7 visible X reactions from 16 accounts; directional sample.
Ask a question below.
Published answers will appear here.
Did a quick 5 PR review test benchmark of Sol vs Terra in Hermes Agent for Hermes Agent PRs Both are quite good. (Fable judged) https://x.com/Teknium/status/2075381154669293979/photo/1