Does a changed LLM answer after a demographic swap show bias?
The paperโs author notes that paraphrasing often changes an LLMโs answer too, raising questions about the baseline for bias tests.
TLDR
A post introducing a paper on counterfactual prompting argues that changing the gender in a prompt and getting a different answer does not, by itself, establish LLM bias. In an October 6 post, the author says paraphrasing often changes answers too and invites discussion of baselines for these evaluations at COLM 2026 on October 7.
Combined views
1.2K
3 Sources, first seen ago
