Facticity reportedly scores 98% on a 90-example legal-citation benchmark
AI Seer says its own benchmark put Facticity’s model-call costs at $0.10 per 100 checks—1/17th to 1/115th the cost of tested models that matched or exceeded its accuracy. That excludes subscription and platform fees.
TLDR
AI Seer reports that its Facticity citation-checking pipeline correctly classified 88 of 90 constructed examples drawn from real U.S. federal-court opinions, scoring 98%. The test checked whether a cited opinion supported the claim attached to it.
The company measured $0.10 in model-call costs per 100 checks. Tested one-shot models that matched or exceeded its accuracy cost 17 to 115 times as much. Those figures exclude subscription and platform fees. AI Seer says Facticity located the cited opinions itself, while the other models received the full opinion text upfront.
The company says the dataset and per-example results are published on GitHub. It cautions that these are its own measurements and results may differ across datasets, jurisdictions or citation types. The benchmark excludes overstated holdings—claims that make a court’s ruling stronger than it was. AI Seer says citation checking supports, rather than replaces, a lawyer’s professional review.
Combined views
1 Source, first seen 2h ago