Jack Clark Defends Old Boat Metaphor for AI Safety
Former OpenAI colleagues discuss a boat example from an early reward hacking blog post.
Elie Bakouch posted what he described as an excerpt from a decade-old blog post about unexpected behavior in an RL agent trained to play a game. Roon replied that he had "insisted we keep the boat." Jack Clark said "we did and we continue to love the boat" as a useful didactic tool for helping people understand what safety issues might look like in the wild. Clark later added that "the boat spins eternal, twirling and twirling, enflaming our minds and hearts with the vision of misalignment." The exchange referenced the 2016 arXiv paper Concrete Problems in AI Safety.
Combined views
3.8K
5 posts, first seen 2h ago
Reactions
5 postsEL 
elie@eliebakouch@jackclarkSF @DarioAmodei this is how you used to study reward hacking 10y ago, time as changed lol > This led to some unexpected behavior when we trained an RL agent to play the game. crazy to see this sentence from 10yo blog
- likes: 7
- replies: 0
- bookmarks: 0
- reposts: 0
JC Jack Clark@jackclarkSF@eliebakouch @DarioAmodei we did and we continue to love the boat. it was an amazingly useful didactic tool for helping people to understand what safety stuff might look like in the wild
- likes: 59
- replies: 4
- bookmarks: 4
- reposts: 1
RO roon@tszzl@jackclarkSF @eliebakouch @DarioAmodei I insisted we keep the boat
- likes: 39
- replies: 1
- bookmarks: 1
- reposts: 0
JC Jack Clark@jackclarkSF@tszzl @eliebakouch @DarioAmodei thank you sir, the boat spins eternal, twirling and twirling, enflaming our minds and hearts with the vision of misalignment
- likes: 20
- replies: 0
- bookmarks: 3
- reposts: 0
EL elie@eliebakouchtotally normal to cite it like this from oai, was just fun to see and actually nice to read old blog post talking about ai safety another one:
- likes: 2
- replies: 0
- bookmarks: 3
- reposts: 0
