1d ago

UChicago’s Chenhao Tan launches MechEvalAgent to detect code-contradicting implicit hallucinations in AI interpretability research

It verifies if narrative assertions match executable code.

——0——
Original post
Adam Gleave#1010@ARGLEAVEOPFAR.AIFAR.AI@FARAIRESEARCH

MechInterp is "a dream problem for research agents", but there’s a problem: trusting the outputs of AI research agents. Using MechEvalAgent, @ChenhaoTan exposes implicit hallucinations, where a model makes a claim but the code shows otherwise.👇

1:00 PM · May 28, 2026 View on X
QUOTE POSTChenhao Tan#570Chenhao Tan@CHENHAOTAN

Check out this thread if you want to learn more:

FAR.AIFAR.AI@farairesearch

MechInterp is "a dream problem for research agents", but there’s a problem: trusting the outputs of AI research agents. Using MechEvalAgent, @ChenhaoTan exposes implicit hallucinations, where a model makes a claim but the code shows otherwise.👇

8:00 PM · May 28, 2026 · 2.2K Views
8:12 PM · May 28, 2026 · 1.2K Views