Users dismissed foundation models as leaderboard theater lacking human interpretability in reaction to the study showing they lag prior supervised vision baselines.
We measure interpretability with two independent but complementary protocols: One asks participants to click 𝙬𝙝𝙚𝙧𝙚 a feature fires on a new image, the other to describe in free text 𝙬𝙝𝙖𝙩 it represents.
@tserre @nuriaoliver @L_Goetschalckx @ELLIS_Alicante @serre_lab @BrownUniversity @ELLISforEurope Paper: https://arxiv.org/abs/2605.20337
What then predicts interpretability?
Two things: 1) models with more local features, and 2) models whose representations reflect the coarse semantic organization of human perception.