New ICML Paper Introduces Method to Uncover Model Competency Gaps

Source Posts on X

Cluster sources

We developed a new method for automatically discovering "competency gaps" in models and benchmarks. These gaps are normally obscured by aggregated metrics (like accuracy). The method is simple and easy to use. And the results can be pretty surprising! Very proud of @matybohacek for driving this work!

Stephanie Chan@scychan_brainsTECH

We developed a new method for automatically discovering "competency gaps" in models and benchmarks. Evals normally obscure these in aggregated metrics (like accuracy). The method is simple and easy to use. And the results can be pretty surprising! Very proud of @matybohacek for driving this work!