Opus 5.5 leads tax prep; GPT-6 Astra leads IT support in one set of evals
The author says GPT-6.1 Sol comes within 2.1 percentage points of Opus 5.5 at about a quarter of the cost.
TLDR
The author says their first set of evals maps performance, cost and time across individual tax prep and IT support. They put Opus 5.5 in the lead on tax, with GPT-6.1 Sol within 2.1 percentage points at about a quarter of the cost. On IT support, they put GPT-6 Astra in the lead, with GPT-6 Sol within 5.8 points at about 70% lower cost.
Combined views
—
2 Sources, first seen ago
