HERMES harness reportedly lifts whole-repository migration score from 6.5% to 31.0%
The paper says HERMES beat matched baseline harnesses by 12.4 points on average across four software engineering benchmarks.
TLDR
A post discussing the HERMES paper says swapping Codex for HERMES raised GPT-5.6 Sol's whole-repository migration score from 6.5% to 31.0% at the same effort setting. HERMES pairs repository components with resident LLMs, activates components based on dependencies and traces test failures to components needing changes. The paper reports a 12.4-point average lead over matched baseline harnesses across four software engineering benchmarks.
Combined views
1 Source, first seen ago
39 reposts