Its reported results put the leading dedicated decision models close together on scores but much further apart on task costs. Drex Max v1.1 from Nace AI and GLiDE v1.2 from Fastino Labs both scored 73, while Mammoth 27B from Base Labs scored 72. The announcement gave no more precise scores or uncertainty estimates to distinguish the two models at 73.
Correct answers, with confidence attached
The new headline metric, Decision Score, gives each question a score from 0 to 100. Correct answers score above 50 and incorrect answers below 50, with confidence in the chosen answer moving the score within each half. Artificial Analysis describes this as rewarding calibrated confidence: confidence that tracks correctness.
Private datasets now account for 45% of the index’s weight, up from 25%. Two new components focus on documents and consistency:
- AA-Paperclip, weighted at 30%, uses 1,500 private documents, including resumes, receipts, invoices, and AI and machine learning research papers. Models make decisions and extract facts, with each document counted as correct only if all 15 answers are right. That all-or-nothing requirement applies to this benchmark, not the entire index.
- AA-Symmetry, weighted at 15%, tests 120 questions with variations such as reordered options, swapped labels and negated yes/no questions. It measures how consistent a model’s probabilities remain across equivalent wording.
The remaining components are AA-Omniscience-Decision v1.1, HLE-MultiChoice v1.1 and EvalSafe-ONET-AA, each weighted at 15%, plus When2Call at 10%. Artificial Analysis removed WildGuardTest, citing its high saturation.
A shared score, a different task bill
Artificial Analysis reported benchmark costs of $102 per million tasks for Drex Max v1.1 and $780 for GLiDE v1.2, nearly eight times as much despite GLiDE’s list prices being only 33% higher.
The evaluator attributes the gap to document billing. It says Drex bills a document’s input once for all its questions, while GLiDE bills that input for every question. The comparison concerns costs on these benchmark tasks, not a universal price for using either model.
Language models scored higher in the evaluator’s results: Claude Haiku 5.5 (Max) reached 77, and GPT-6 Luna (Max) reached 76. For GPT-6 Luna, Artificial Analysis reported $1,530 per million tasks and 11.34 seconds per task, compared with Drex’s $102 and 275 milliseconds. Those figures describe the evaluator’s benchmark comparison, not guaranteed production costs or response times.
