Harvey and Baseten's post-trained open-weight AI agents match closed-source frontier models on the Legal Agent Benchmark

Fascinating. Legal, accounting and investment banking should thrive as vertical applications of AI, especially if you run on http://Factory.ai too.

Post-training your own frontier model has become the new default for the leading AI companies. Baseten's research team partnered with the brilliant team at @harvey and showed that post-trained open models can compete at the frontier on LAB. Post-training not only enables high-quality legal agents to be more accessible, but also allows more specialization for the workflows firms actually care about.

@gabepereyra Great teamwork 💚

@AgiBeerus will definitely be around. dominating.

@rabois Factory won’t be around in a year unless acquired by Cognition or Anthropic but not sure why they would.

@rabois What’s their revenue? Anthropic $50 billion, Cursor $3 billion, Cognition $500 million.

Factory $30 million at best, probably negative margin?

@AgiBeerus much much higher and accelerating faster than Cursor and likely Cognition.

@rabois Is $upst still investable ?

@gabepereyra which practice areas saw the biggest gains from post training? I'd guess any corporate or commercial ones. Also, for any task where the model failed was it because of bad reasoning/retrieval? That's where I am seeing the failures on my end.

@DannieHerz @harvey so cool.

Post-training can make a model sound more lawyerly. It does not make it current, source-faithful, or procedurally reliable.

Legal agents fail where the law changes, sources conflict, jurisdictions diverge, or facts require verified retrieval. That needs different a approach, not just better weights.

@rabois Oh shit just noticed cognition mogged you guys today

@rabois How about you and @cognition play I’ll show you mine if you show me yours

I’ll bet 100k factory is not at $500 million 12 months from now.