Announcement
AutomationBench score reportedly rose from 34.0% to 57.7%
PrimeIntellect says it built 2,500 hard enterprise tasks with NVIDIA, modeled on AutomationBench.
TLDR
PrimeIntellect says it raised its AutomationBench score from 34.0% to 57.7%. The benchmark evaluates agents in simulated SaaS applications, including CRMs and inboxes. PrimeIntellect says it built 2,500 hard enterprise tasks with NVIDIA, tested for shortcuts and answer leakage, and checked that graders reward valid solutions without accepting incomplete work.
Combined views
—
2 Sources, first seen ago
— likes— comments— saves— reposts
