Anthropic Seeks Interpretability Research Scientist in San Francisco
Jack Lindsey shares questions on instilling values and mitigating reward hacking effects.
Jack Lindsey, who leads the Model Psych team at Anthropic, posted questions of interest for the role: what training forms best instill values that generalize out of distribution, and what effects reinforcement learning and reward hacking have on character along with ways to avoid or mitigate negative effects. The post links to the Anthropic job board listing for Research Scientist, Interpretability based in San Francisco, CA.
Combined views
3.7K
2 posts, first seen 17h ago
Reactions
2 postsJL 
Jack Lindsey@Jack_W_LindseySome questions we’re interested in: — What forms of training are most effective at instilling a given set of “values” into a model, in a way that generalizes out of distribution? — What are the effects of reinforcement learning and reward hacking on character, and how can the…
- likes: 112
- replies: 15
- bookmarks: 101
- reposts: 5
JL Jack Lindsey@Jack_W_LindseyWe’re hiring for “psychological design” research at Anthropic. Our aim is to better understand how training impacts a model’s character and alignment. We’re seeking researchers with experience in LLM finetuning, interpretability, and alignment evaluations. More details below.
- likes: 551
- replies: 14
- bookmarks: 281
- reposts: 43
