Pinocchio aims to estimate confidence in AI answers without access to the model behind them

Pinocchio’s creators say it predicts whether a language model’s answer is correct, then calibrates that prediction into an uncertainty estimate in a single pass. They say it handles vision tasks and outperformed verbal confidence baselines and TypeSafe Jev in their tests; Jev was tested only on text. They caution that Pinocchio can be miscalibrated on data unlike its training mix.