AIThe case for continual human oversight of autonomous systems
Citing Nancy Leveson’s Engineering a Safer World, a thread frames safety as something that requires active control—not an inherent property of an AI agent or self-driving car.
TLDR
The thread argues that autonomous systems require continual monitoring and human oversight because designers cannot anticipate every way a system can fail. It makes a narrow exception to that prediction limit: simple systems performing very narrow tasks in unchanging environments. Drawing on Nancy Leveson’s Engineering a Safer World, it describes safety—and AI “alignment”—as a dynamic property requiring active control. It says Leveson examines threats from budget cuts and staff turnover, as well as changes in the operating environment, including regulations.
Combined views
11.6K
12 Sources, first seen 3h ago
The case for continual human oversight of autonomous systems
Citing Nancy Leveson’s Engineering a Safer World, a thread frames safety as something that requires active control—not an inherent property of an AI agent or self-driving car.
TLDR
The thread argues that autonomous systems require continual monitoring and human oversight because designers cannot anticipate every way a system can fail. It makes a narrow exception to that prediction limit: simple systems performing very narrow tasks in unchanging environments. Drawing on Nancy Leveson’s Engineering a Safer World, it describes safety—and AI “alignment”—as a dynamic property requiring active control. It says Leveson examines threats from budget cuts and staff turnover, as well as changes in the operating environment, including regulations.