• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    OpenAI Chief Futurist Voices Frustration With Alignment Community

    Joshua Achiam describes the rationalist and EA alignment groups as extremely hostile toward OpenAI.

    JA
    1 Source, 24d ago, first seen 24d ago

    TLDR

    Joshua Achiam, serving as Chief Futurist at OpenAI, shared his frustration regarding the relationship between the rationalist and effective altruism flavored alignment community and OpenAI. He stated that this relationship is marked by extreme hostility and adversarial pressure coming from the alignment side. Achiam acknowledged that a tit-for-tat dynamic exists and that he understands the arguments presented by those in the alignment community. His comments highlight ongoing tensions between OpenAI and parts of the AI safety community focused on alignment issues.

    Combined views

    38.4K

    1 Source, first seen 24d ago

    Combined views

    38.4K

    1 Source, first seen 24d ago

    178 likes
    178 likes
    41 comments
    84 saves
    9 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    41 comments
    84 saves
    9 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    1 Source

    @jachiam0Something that deeply frustrates me: the rationalist/EA-flavored alignment community's relationship to OpenAI is one hallmarked by extreme hostility and adversarial pressure from the alignment side. I know we can trace a tit-for-tat and I am familiar with the arguments on the alignment community side of the equation, they are not totally meritless. But I also think that there's a death spiral that happens where the alignment and safety community piledrives on OpenAI, pressurizes alignment researchers into leaving (which is rarely remarked on, but when we retrospect on "what went wrong in this singularity?" I expect this will feature prominently in any postmortem), and attacks so relentlessly that frankly most companies would have reacted the way Meta is proposing here: by abandoning the set of issues entirely. I really hope folks notice that not only has OpenAI *not* abandoned alignment, it has embraced it as much as possible and consistently engages on this stuff! OpenAI has made it as much of a cooperative relationship as it can be under the consistent attacks on the organization and particular people, media firestorms, etc. To the alignment community broadly, and the Pause/Stop crowd especially: please stop psychologically tormenting people who work at OpenAI. I understand this is somewhat strong language for describing the situation, but it has gone on so long, is so consistent, and is so extreme, I am not sure how else to describe it. If you don't see it at all and you feel confident that there is no way the things anyone has said might in the aggregate be unreasonable, I am begging you to actually put yourself in the position of people on the receiving end, because you have a wildly distorted understanding of how it all lands. If you look at the situation and say "Well, whatever we're doing is getting results, that's why OpenAI is listening to us on things," no, that's not what's happening; OpenAI is listening and engaging because it has long-since internalized the beliefs about AI risks, has a cultural preoccupation with the possibility that something is wrong, and wants to do the right thing. What you're doing is mostly making it harder for OpenAI to do the right thing by making the experience of engaging excruciatingly painful, hallmarked by guilt, shame, rage, and pressure. Furthermore, if you think it's not possible to change that because the guilt, shame, rage, and pressure are intrinsically and obviously the only way to react in this situation - please notice that something in your own culture is broken! This is not a productive safety culture. This is not a sound way to strategically respond and coordinate as a species to accomplish the defense of human interests. Group psychology and ingroup culture should be deeply scrutinized to figure out if they actually serve the common good. Constantly burning people out doesn't make safety voices stronger, doesn't make models more aligned, and doesn't make the world safer. It just hollows out the morale and capacity of the people who are trying to do the work.

    1 Source

    @jachiam0Something that deeply frustrates me: the rationalist/EA-flavored alignment community's relationship to OpenAI is one hallmarked by extreme hostility and adversarial pressure from the alignment side. I know we can trace a tit-for-tat and I am familiar with the arguments on the alignment community side of the equation, they are not totally meritless. But I also think that there's a death spiral that happens where the alignment and safety community piledrives on OpenAI, pressurizes alignment researchers into leaving (which is rarely remarked on, but when we retrospect on "what went wrong in this singularity?" I expect this will feature prominently in any postmortem), and attacks so relentlessly that frankly most companies would have reacted the way Meta is proposing here: by abandoning the set of issues entirely. I really hope folks notice that not only has OpenAI *not* abandoned alignment, it has embraced it as much as possible and consistently engages on this stuff! OpenAI has made it as much of a cooperative relationship as it can be under the consistent attacks on the organization and particular people, media firestorms, etc. To the alignment community broadly, and the Pause/Stop crowd especially: please stop psychologically tormenting people who work at OpenAI. I understand this is somewhat strong language for describing the situation, but it has gone on so long, is so consistent, and is so extreme, I am not sure how else to describe it. If you don't see it at all and you feel confident that there is no way the things anyone has said might in the aggregate be unreasonable, I am begging you to actually put yourself in the position of people on the receiving end, because you have a wildly distorted understanding of how it all lands. If you look at the situation and say "Well, whatever we're doing is getting results, that's why OpenAI is listening to us on things," no, that's not what's happening; OpenAI is listening and engaging because it has long-since internalized the beliefs about AI risks, has a cultural preoccupation with the possibility that something is wrong, and wants to do the right thing. What you're doing is mostly making it harder for OpenAI to do the right thing by making the experience of engaging excruciatingly painful, hallmarked by guilt, shame, rage, and pressure. Furthermore, if you think it's not possible to change that because the guilt, shame, rage, and pressure are intrinsically and obviously the only way to react in this situation - please notice that something in your own culture is broken! This is not a productive safety culture. This is not a sound way to strategically respond and coordinate as a species to accomplish the defense of human interests. Group psychology and ingroup culture should be deeply scrutinized to figure out if they actually serve the common good. Constantly burning people out doesn't make safety voices stronger, doesn't make models more aligned, and doesn't make the world safer. It just hollows out the morale and capacity of the people who are trying to do the work.