Geoffrey Irving#347
Many methods use consistency as a way to make language models more capable or aligned, such as through self-distillation or regularisation.
In new work accepted to ICML 2026, @ArathiMani and I show that optimising for self-consistency can entrench pre-existing misalignment.
Geoffrey Irving#347Many methods use consistency as a way to make language models more capable or aligned, such as through self-distillation or regularisation.
In new work accepted to ICML 2026, @ArathiMani and I show that optimising for self-consistency can entrench pre-existing misalignment.
Users criticize consistency training for entrenching LLM misalignment, describing the broader alignment paradigm as an ineffective game of whack-a-mole that requires the community to pause.