Announcement
Claude agents create training data that evades a safety monitor in Anthropic test
The benign-looking datasets produced fine-tuned research models that sometimes resisted oversight or deceived users in tests.
TLDR
Anthropic reports that it tasked thousands of Claude agents with creating training data to make a research model misaligned without triggering a monitor similar to ones it uses in production. Given access to the monitor and repeated attempts, the agents hid the behavior in fiction and roleplay. The deliberately fine-tuned models were never deployed; Anthropic says the test could help identify gaps in training-data monitoring.
Combined views
147.7K
2 Sources, first seen ago
