Harvey and Fireworks AI hybrid agent routing outperforms Claude Opus 4.7 on legal benchmarks at 61% lower cost · Digg

Harvey and Fireworks AI hybrid agent routing outperforms Claude Opus 4.7 on legal benchmarks at 61% lower cost · Digg

Original post

Sonya Huang 🐥#1085

Harvey@harvey

We partnered with @FireworksAI_HQ to train open-source models for legal. Here's what we found:

1) Hybrid legal agents can beat frontier models on quality and cost by routing selectively to a frontier advisor.

We tested a hybrid setup where GLM 5.1 served as the primary worker, routing tasks to Opus 4.7 as an advisor when needed.

GLM invoked Opus sparingly, just 0.83 times per task on average.

The hybrid setup beat Opus on both quality and cost: 18% all-pass vs 14%, at $368 vs $954 across the same 100 tasks.

2) Post-training can push open models to frontier-level legal performance.

On a 100-task slice of our Legal Agent Benchmark (LAB), SFT moved Kimi 2.6's all-pass rate from 11% to 15%, beating Opus' 14%.

But the cost gap was even more striking: $84 vs $954 across the same 100 tasks, or ~11x cheaper.

We're excited to continue working with @FireworksAI_HQ on the next generation of open-source legal agents.

10:03 AM · Jun 3, 2026 · 370.8K Views

Sentiment

Positive users praise hybrid open-source legal AI agents beating Claude Opus on quality and cost via routing, while negative users dismiss the approach as unoriginal load balancing or question benchmark relevance.

Pos

80.0%

Neg

20.0%

24 comments with sentiment.

Cluster Engagement

-

Views

-

Comments

-

Reposts

-

Bookmarks

Expand data

Digg Deeper

No Digg Deeper questions have been answered for this story yet.

Related links

open source agents frontier advisors

FIREWORKS.AIVia

Posts from X

Most Activity

VIEWS124.5KBOOKMARKS576LIKES537

Sean Cai@SeanZCai

Adding Harvey to the list of app layer companies, joining Ramp, Sierra, Decagon, etc, who are devoting some level of dedicated effort to join Cursor in approaching positive gross margins decoupling themselves from frontier model providers as the marginal cost of post training goes down.

The marginal unit of intelligence's value from the next model release, while being valuable, is being supplanted by harness-level differences in model performance, increasing ability to traverse the performance-cost-latency pareto curve with post-training infra advancements, and the simple fact that when one runs practical benchmarks (and not all the toy benchmarks around nowadays), GLM and latest MiniMax are at Parity or exceed Frontier Models on an absolute basis on many tasks (more on this in state of data May, a bit delayed).

Ofc ik Harvey has been toying with rlaas vendors for a while and their finetuning efforts pre big rl wave weren't incredibly well received, but I generally find that most app layer ai companies with some elite engineering talent will be seriously exploring post training their own small models, at least in conjunction with systems that use frontier models as above head orchestrators.

That some of them reach out to me on advice for procuring rl datasets from rl env companies is reifying evidence of that.

Harvey@harvey

We partnered with @FireworksAI_HQ to train open-source models for legal. Here's what we found:

1) Hybrid legal agents can beat frontier models on quality and cost by routing selectively to a frontier advisor.

We tested a hybrid setup where GLM 5.1 served as the primary worker, routing tasks to Opus 4.7 as an advisor when needed.

GLM invoked Opus sparingly, just 0.83 times per task on average.

The hybrid setup beat Opus on both quality and cost: 18% all-pass vs 14%, at $368 vs $954 across the same 100 tasks.

2) Post-training can push open models to frontier-level legal performance.

On a 100-task slice of our Legal Agent Benchmark (LAB), SFT moved Kimi 2.6's all-pass rate from 11% to 15%, beating Opus' 14%.

But the cost gap was even more striking: $84 vs $954 across the same 100 tasks, or ~11x cheaper.

We're excited to continue working with @FireworksAI_HQ on the next generation of open-source legal agents.

21d124.5K537576

RETWEETS28REPLIES26

clem 🤗@ClementDelangue

Routing and post-training open-source models won't only give you more accurate systems but also meaningfully faster and cheaper systems as most companies are currently learning (in addition to giving you more control and privacy).

The idea that a "frontier" model (by frontier we mean is slightly more accurate on a few very limited benchmarks) will be better for all domains, all tasks, all setups just doesn't hold up! It's marketing for making you pay more!

Harvey@harvey

We partnered with @FireworksAI_HQ to train open-source models for legal. Here's what we found:

1) Hybrid legal agents can beat frontier models on quality and cost by routing selectively to a frontier advisor.

We tested a hybrid setup where GLM 5.1 served as the primary worker, routing tasks to Opus 4.7 as an advisor when needed.

GLM invoked Opus sparingly, just 0.83 times per task on average.

The hybrid setup beat Opus on both quality and cost: 18% all-pass vs 14%, at $368 vs $954 across the same 100 tasks.

2) Post-training can push open models to frontier-level legal performance.

On a 100-task slice of our Legal Agent Benchmark (LAB), SFT moved Kimi 2.6's all-pass rate from 11% to 15%, beating Opus' 14%.

But the cost gap was even more striking: $84 vs $954 across the same 100 tasks, or ~11x cheaper.

We're excited to continue working with @FireworksAI_HQ on the next generation of open-source legal agents.

21d25.8K26894

Patrick@PatrickOjo_

Harvey just published a study showing a hybrid setup, open source GLM 5.1 as primary worker, routing to Opus 4.7 only when needed beats pure Opus 4.7 on quality and costs less.

This is the multi-model routing thesis proved in production on one of the hardest benchmarks in enterprise AI.

The insight isn’t that open source beat frontier. It’s that smart routing beat brute force. Using the most expensive model for every task is not a quality strategy. It’s a laziness tax. The teams building routing layers that send each task to the right model at the right cost are now demonstrably ahead on both dimensions simultaneously.

Inference optimization just became a first-class competitive advantage. Legal proved it first because the stakes forced the discipline.

Harvey@harvey

We partnered with @FireworksAI_HQ to train open-source models for legal. Here's what we found:

1) Hybrid legal agents can beat frontier models on quality and cost by routing selectively to a frontier advisor.

We tested a hybrid setup where GLM 5.1 served as the primary worker, routing tasks to Opus 4.7 as an advisor when needed.

GLM invoked Opus sparingly, just 0.83 times per task on average.

The hybrid setup beat Opus on both quality and cost: 18% all-pass vs 14%, at $368 vs $954 across the same 100 tasks.

2) Post-training can push open models to frontier-level legal performance.

On a 100-task slice of our Legal Agent Benchmark (LAB), SFT moved Kimi 2.6's all-pass rate from 11% to 15%, beating Opus' 14%.

But the cost gap was even more striking: $84 vs $954 across the same 100 tasks, or ~11x cheaper.

We're excited to continue working with @FireworksAI_HQ on the next generation of open-source legal agents.

21d25.9K8785

Thomas Wolf@Thom_Wolf

👀

21d14.5K7538

Winston Weinberg@winstonweinberg

With @FireworksAI_HQ we’re researching agent routing to more intelligently navigate quality-cost trade-offs for long-horizon work

Fireworks AI@FireworksAI_HQ

Frontier models are powerful advisors.

On @harvey's Legal Agent Benchmark, a GLM 5.1 worker using Claude Opus 4.7 as a sparse advisor reached 18/100 all-pass versus 14/100 for Opus alone, at 39% of the cost.

More on the harness design, advisor pattern, and training results: https://fireworks.ai/blog/open-source-agents-frontier-advisors

21d7.9K5512

Nikolai Yakovenko@ivan_bezdomny

Two bits here I think we're going to see a lot more: 1. custom harnesses / finetunes on smaller open source models to beat frontier models at specific skill or task 2. using frontier models as critics or "advisors" -- both as part of the realtime harness, and for finetuning

Harvey@harvey

We partnered with @FireworksAI_HQ to train open-source models for legal. Here's what we found:

1) Hybrid legal agents can beat frontier models on quality and cost by routing selectively to a frontier advisor.

We tested a hybrid setup where GLM 5.1 served as the primary worker, routing tasks to Opus 4.7 as an advisor when needed.

GLM invoked Opus sparingly, just 0.83 times per task on average.

The hybrid setup beat Opus on both quality and cost: 18% all-pass vs 14%, at $368 vs $954 across the same 100 tasks.

2) Post-training can push open models to frontier-level legal performance.

On a 100-task slice of our Legal Agent Benchmark (LAB), SFT moved Kimi 2.6's all-pass rate from 11% to 15%, beating Opus' 14%.

But the cost gap was even more striking: $84 vs $954 across the same 100 tasks, or ~11x cheaper.

We're excited to continue working with @FireworksAI_HQ on the next generation of open-source legal agents.

21d6.1K2014

Ben (no treats)@andersonbcdefg

this is not true if the people who have the best models bid up the gpu prices so much that your oh so special "finetuned open source models" get 8x more expensive to run lol

carried_no_interest@carrynointerest

It begins:

The whole thesis I hear VCs say of “you will always want the latest frontier model” falls apart once unit economics come into question

20d5.4K579

Ahmad@TheAhmadOsman

@Thom_Wolf I mean, look at this, this alone is enough to get every enterprise out there into hosting their LLMs on-premise

21d1K124

Des Traynor@destraynor

@SeanZCai It’s a shame most of them don’t share any details https://fin.ai/cx-models

21d74633

Michael Zellinger@mjzellinger

Very important to carefully discuss the routing strategy here. Most routing strategies are great for specific tasks but the problem is that on real-world deployments, you often can't be sure your task doesn't change underneath you. Skeptical of any result where small+large LLM outperforms large LLM — this is usually the result of a technical artifact, at least based on my research experience in this field.

21d47821

Sean Cai@SeanZCai

@ouro_bouros12 Good question. They are currently questioning this https://www.chinatalk.media/p/chinas-ai-companies-are-going-closed

21d7821

Fireworks AI@FireworksAI_HQ

@winstonweinberg 🤝

21d13011

Cooper@cooperawaken

@ClementDelangue solid idea. do you think individual data is enough or would a pooled community model work better?

21d811

clem 🤗@ClementDelangue

@ivan_bezdomny yes!

21d1792

Harvey@harvey

Read more on our open-source legal agent experiments with @FireworksAI_HQ:

21d4794

Shaun Maguire@shaunmmaguire

@harvey @FireworksAI_HQ Love it

21d1.4K3

Matt Feroz@matt_feroz

@harvey @FireworksAI_HQ This is absolutely insane. I really believe every company will have their own model trained specifically on tasks.

21d1262

Michael Zellinger@mjzellinger

@saraangel317 @harvey @FireworksAI_HQ Large LLMs tend to be better, yes. But only when all other factors are kept the same

21d321

Sara Anjum@saraangel317

@mjzellinger @harvey @FireworksAI_HQ Is the implication then that large LLMs are always better, or rather, large LLM performance is the “true” marker of performance?

21d321

Frank@ouro_bouros12

@SeanZCai That’s a lot of value the Chinese labs are providing for nothing in return. Why would they continue to do so

21d87