• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    OpenAI should accept strong criticism, an insider argues

    Another post agrees that OpenAI should accept strong critiques, but argues that productive criticism requires accuracy and good faith to clarify the issues and make change easier to negotiate.

    JA
    SB
    2 Sources, ,

    TLDR

    An OpenAI insider argues that the company should accept strong criticism—even when it is more adversarial than necessary—because of the responsibility that comes with technology they describe as world-changing. Another post agrees on accepting strong critiques, while emphasizing what makes criticism productive: clarifying the stakes and helping people negotiate changes to policy, technology or cultural norms. That author argues for accuracy and good-faith engagement, warning that an overly punitive atmosphere can drive away the people critics hope to persuade.

    Combined views

    19.8K

    2 Sources, first seen 23d ago

    Combined views

    19.8K

    2 Sources, first seen 23d ago

    88 likes
    23d ago
    first seen 23d ago
    88 likes
    15 comments
    42 saves
    1 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    15 comments
    42 saves
    1 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    2 Sources

    @jachiam0Boaz, I greatly appreciate you too, and I believe you do a huge amount to raise the safety and sanity bar at OpenAI. I agree that OpenAI should be OK with strong critiques. Wholeheartedly in fact. Just days ago, I was defending the freedom of a critic to argue that everyone involved in building AGI or ASI should be put on trial for crimes against humanity. However I also believe that there are such things as more or less productive critiques. Speaking very generally, the function of a critic in society is performed best not when the critic merely gives invective or complaint, but when the critic through persuasion and criticism makes the issues at stake clearer and makes the negotiated end - the transformation of policy, technology, or cultural norm - easier to acheive. These two ends, making issues clearer and making negotiated outcomes easier to attain, are both important. To do the first, the critic must be accurate; they must add signal, not noise. To do the second: the human factor cannot be overlooked, and the critic must not put discussion of the negotiated outcome into a space filled with so much punishment that the people they aim to persuade refuse to trod there. The willingness of the critic to engage in charity and good faith with people on the other side is essential. Otherwise the disagreement becomes a conflict that cannot resolve except by escalation and exertion of power to compel one side or the other into stopping. I feel the EA-flavor of the alignment community (including many people I genuinely cherish!) do not always fill this critic function well. The temperature, hostility, and comfort with extreme imprecision are sometimes unbearable. I want OpenAI to have better criticism than this. The looped transformer / neuralese episode is an example of the sort of thing I'm complaining about. Many people flipped into a position of outrage ("They trained using the FORBIDDEN TECHNIQUE!!") before *any* information was available or confirmed, and for that matter, I feel that the position they were taking in the first place was untenable on technical or strategic grounds. Are the critics more technically informed than most critics usually are? Yes. But how much of this criticism was helpful or clarifying for the issues at stake? At a discourse temperature this high, could any group of the people who have opinions about safety issues and the people in the labs come to a negotiated consensus position on these technical topics that is actually driven by the technical conditions? It felt like a mob, it acted like a mob, it was a mob. Can this group of people do better and delineate specific measurements, experiments, go/no-go conditions for this safety requirement or that one? Certainly! Will they? Not if the level of comfort with mob discourse persists. Critics are vitally important to society. But they are not themselves beyond reproach and criticism will make them better too.
    @BlancheMinerva@jachiam0 You referred to Garrison’s (and maybe my) comments as “exactly the kind of crazy-making quasi-abusive behavior I'm objecting to in how people treat OpenAI” because we were sharply critical of Anthropic for lying to a member of Congress.

    2 Sources

    @jachiam0Boaz, I greatly appreciate you too, and I believe you do a huge amount to raise the safety and sanity bar at OpenAI. I agree that OpenAI should be OK with strong critiques. Wholeheartedly in fact. Just days ago, I was defending the freedom of a critic to argue that everyone involved in building AGI or ASI should be put on trial for crimes against humanity. However I also believe that there are such things as more or less productive critiques. Speaking very generally, the function of a critic in society is performed best not when the critic merely gives invective or complaint, but when the critic through persuasion and criticism makes the issues at stake clearer and makes the negotiated end - the transformation of policy, technology, or cultural norm - easier to acheive. These two ends, making issues clearer and making negotiated outcomes easier to attain, are both important. To do the first, the critic must be accurate; they must add signal, not noise. To do the second: the human factor cannot be overlooked, and the critic must not put discussion of the negotiated outcome into a space filled with so much punishment that the people they aim to persuade refuse to trod there. The willingness of the critic to engage in charity and good faith with people on the other side is essential. Otherwise the disagreement becomes a conflict that cannot resolve except by escalation and exertion of power to compel one side or the other into stopping. I feel the EA-flavor of the alignment community (including many people I genuinely cherish!) do not always fill this critic function well. The temperature, hostility, and comfort with extreme imprecision are sometimes unbearable. I want OpenAI to have better criticism than this. The looped transformer / neuralese episode is an example of the sort of thing I'm complaining about. Many people flipped into a position of outrage ("They trained using the FORBIDDEN TECHNIQUE!!") before *any* information was available or confirmed, and for that matter, I feel that the position they were taking in the first place was untenable on technical or strategic grounds. Are the critics more technically informed than most critics usually are? Yes. But how much of this criticism was helpful or clarifying for the issues at stake? At a discourse temperature this high, could any group of the people who have opinions about safety issues and the people in the labs come to a negotiated consensus position on these technical topics that is actually driven by the technical conditions? It felt like a mob, it acted like a mob, it was a mob. Can this group of people do better and delineate specific measurements, experiments, go/no-go conditions for this safety requirement or that one? Certainly! Will they? Not if the level of comfort with mob discourse persists. Critics are vitally important to society. But they are not themselves beyond reproach and criticism will make them better too.
    @BlancheMinerva@jachiam0 You referred to Garrison’s (and maybe my) comments as “exactly the kind of crazy-making quasi-abusive behavior I'm objecting to in how people treat OpenAI” because we were sharply critical of Anthropic for lying to a member of Congress.