Predicting agent performance on held-out tasks with Item Response Theory
A COLM 2026 presenter says their team also uses the theory to predict performance on held-out model/harness combinations.
TLDR
At COLM 2026 on October 8, a presenter said their team uses Item Response Theory to predict agent performance on held-out tasks and held-out model/harness combinations. They were presenting the work at poster #42 in the Imperial Ballroom.
Combined views
75
1 Source, first seen ago