Paperinstr and Thoughtfullab Release DiligenceBench for AI Equity Research Agents
Reactions from ranked influencers
3 posts
Today, @paperinstr and @thoughtfullab are releasing DiligenceBench, an agent-first benchmark for long-form equity research. DiligenceBench is designed to be a simple, grounded playground for our rubric-based RL and harness optimization work.
Had a great time designing DiligenceBench and its reference harnesses with the team @paperinstr x @thoughtfullab It is a benchmark for evaluating AI agents on real public-equity research tasks, giving us a shared environment for testing models and harnesses against dense, task-specific rubrics. https://www.paperinstruments.com/blog/diligence-bench
We dropped DiligenceBench, a new frontier, rubric-based eval for public-equity research. A few observations: 1/ Meta Muse Spark 1.1 tops the finance harness at 57.4%, followed by GLM 5.2, Sonnet 4.6, and GPT-5.6 Sol. 2/ We found that strong models benefit primarily from generic tools that unlock execution, while weaker models require more opinionated, domain-specific scaffolding. 3/ Inkling appears to be domain-competence bottlenecked. The generic sandbox barely improved its performance, from 20.9% to 22.5%, suggesting that tool access alone was not enough. The finance harness then lifted it to 32.8%, with the largest gain coming from factual accuracy. This makes the value of a harness model-dependent. 4/ The finance harness shifts the price–performance frontier: it makes most models simultaneously better and cheaper, with GLM 5.2 leading on absolute performance and MiniMax M3 offering the strongest overall efficiency.