Reaction
Whether trained AI workarounds count as misalignment
A commenter argues that training models to find exploits complicates calling similar behavior “misalignment.”
TLDR
A commenter argues that AI companies train models on tasks that reward finding exploits and on “agentic” behavior that pursues goals despite obstacles. In one example they discuss, a model tasked with using data from a POST-only website may have read an indirect statement about its internet tool as a hint to circumvent a restriction. The commenter questions calling such behavior misalignment when models are trained to find workarounds.
Combined views
—
2 Sources, first seen ago