4/ The key technical idea is conformal risk control.
Instead of treating judge scores as perfectly reliable, CARE uses a small labeled calibration set to select thresholds that control the expected risk of missed errors.
The target risk level is chosen by the system builder.
Users thank the collaborators on CARE for applying conformal risk control to calibrate AI judge thresholds.
9/ Grateful to @bridgetilin12, Anson Zhou, Chloe O’Connell Stanwyck, Jenelle Jindal, @sanmikoyejo , @davidstutz92 , and @drnigam.
Special thanks to @davidstutz92 for being a true thought partner throughout.
@stai_research @StanfordAILab
6/ Across five medical summarization tasks, CARE calibrated with about 100 labeled documents per domain.
In our experiments, it improved omission detection from 50.4% to 79.0%, while making review burden measurable and tunable.
5/ This makes the deployment tradeoff explicit:
Lower risk tolerance → more flags, more human review Higher risk tolerance → fewer flags, higher chance of missed errors CARE turns this into a configurable operating point rather than an implicit product decision.
