Richard Ngo argues Yoshua Bengio's AI alignment paper prioritizes political outreach over scientific rigor

I’m not claiming that Yoshua doesn’t believe in it! In fact, “really believes this can work” is closely related to the failure mode I’m calling out in the first tweet. That is, when you’re trying to find solutions, you end up thinking on a shallower level.

A result of thinking on that shallower level is then that you focus less on scientific questions like “can I disprove my hypothesis?” or “how principled are these definitions” or “are there weird edge cases I should pay attention to?” and more on “how can I spread my ideas in a way that will help the world?” Which is the thing that I think is going on in this paper, which I’m classifying as politics.