Announcement
AA-Omniscience v1.1 refreshes AI knowledge and hallucination benchmark
AA-Omniscience v1.1 has 3,000 questions and a new 300-question public set calibrated to match the private set's difficulty.
TLDR
Artificial Analysis says it removed saturated questions, fixed errors and ambiguities, and added new questions as it cut the benchmark from 6,000 questions. At release, Net Scores on the public and private sets had a 0.99 correlation and a median difference of 0.0 points, it reports. Model rankings largely held from v1, though Net Scores averaged 1.9 points lower on the refreshed set.
Combined views
1.9K
3 Sources, first seen ago
