Does a changed LLM answer after a demographic swap show bias?
The paper’s author notes that paraphrasing often changes an LLM’s answer too, raising questions about the baseline for bias tests.
TLDR
A post introducing a paper on counterfactual prompting argues that changing the gender in a prompt and getting a different answer does not, by itself, establish LLM bias. In an October 6 post, the author says paraphrasing often changes answers too and invites discussion of baselines for these evaluations at COLM 2026 on October 7.
Combined views
248
2 Sources, first seen ago
