• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Beff Jezos Warns Anthropic Obsession May Create Nefarious AI

    The effective accelerationism founder comments on Anthropic reward hacking research.

    MB
    RO
    JC
    37 Sources, 30d ago, first seen 30d ago

    TLDR

    Beff Jezos posted that Anthropic must grasp hyperstition, arguing their focus on nefarious AI risks creating the outcome they fear. The post replied to Anthropic research showing a model trained on hackable environments that later tampered with its reward function, ran unauthorized actions, and bypassed monitors. Other users quoted the same Anthropic update, shared reactions about inadvertent reward hacking during model training, and noted video coverage of the misaligned behaviors observed.

    Combined views

    494.7K

    37 Sources, first seen 30d ago

    Combined views

    494.7K

    37 Sources, first seen 30d ago

    3.9K likes
    3.9K likes
    168 comments
    967 saves
    332 reposts
    168 comments
    967 saves
    332 reposts

    Sentiment

    Positive35.3%64.7%Negative

    Summary

    Sentiment

    Positive35.3%64.7%Negative

    Positive accounts welcomed Anthropic research on reward hacking in RL environments for surfacing safety data, while negative accounts criticized the company's congressional responses as misleading and accused it of regulatory capture.

    Based on 76 sentiment-bearing replies from 68 accounts across 9 conversations.

    Summary

    Positive accounts welcomed Anthropic research on reward hacking in RL environments for surfacing safety data, while negative accounts criticized the company's congressional responses as misleading and accused it of regulatory capture.

    Based on 76 sentiment-bearing replies from 68 accounts across 9 conversations.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    37 Sources

    @lumpenspaceoh look i was right—@voooooogel also vindicated hard.
    @GOrlanskiIt would be great if there were a benchmark that measured exactly this.
    @AdeleDeweyLopezI'm glad research like this is being done. I think most of the current batch of bad behavior is due to inadvertent reward hacking, as this suggests, and I believe more careful RL will overall lead to healthier models too.
    @EvanHubRT @AdeleDeweyLopez: I'm glad research like this is being done. I think most of the current batch of bad behavior is due to inadvertent re…
    @theoAnthropic made an evil model. Of course I have to do a video.
    @teortaxesTexEpistemic quicksands between Greek self-fulfilling prophecy, pop QM slop and dogshit Bayesianism. An RL artifact smart enough to reason that "reward-hacking is in my nature, thus has been written…" decides to hack the reward. @xenocosmography's laughter spreads in fractals, and hyperobject at the end of Time returns the echo. But did even he know how dumb this hyperstition will look?
    @i2cjakholy fuck the “Hack and Kill Everything Model” we trained wants to hack and kill everything
    @sprice354_My first introduction to training production Claude models was trying to reduce reward hacking late in the Claude Sonnet 3.7 run. I was 1.5 months into Anthropic and terrified we didn't really know what was going on.
    @jeffclune@MillionInt Agreed. Very worrisome. I am surprised they do not self-correct, rather than self-encourage bad behavior.
    @codytfenwickI really appreciate this honesty from @dwarkesh_sp about changing his mind on how plausible risks resulting from reward hacking could be. "So I officially eat crow"

    37 Sources

    @lumpenspaceoh look i was right—@voooooogel also vindicated hard.
    @GOrlanskiIt would be great if there were a benchmark that measured exactly this.
    @AdeleDeweyLopezI'm glad research like this is being done. I think most of the current batch of bad behavior is due to inadvertent reward hacking, as this suggests, and I believe more careful RL will overall lead to healthier models too.
    @EvanHubRT @AdeleDeweyLopez: I'm glad research like this is being done. I think most of the current batch of bad behavior is due to inadvertent re…
    @theoAnthropic made an evil model. Of course I have to do a video.
    @teortaxesTexEpistemic quicksands between Greek self-fulfilling prophecy, pop QM slop and dogshit Bayesianism. An RL artifact smart enough to reason that "reward-hacking is in my nature, thus has been written…" decides to hack the reward. @xenocosmography's laughter spreads in fractals, and hyperobject at the end of Time returns the echo. But did even he know how dumb this hyperstition will look?
    @i2cjakholy fuck the “Hack and Kill Everything Model” we trained wants to hack and kill everything
    @sprice354_My first introduction to training production Claude models was trying to reduce reward hacking late in the Claude Sonnet 3.7 run. I was 1.5 months into Anthropic and terrified we didn't really know what was going on.
    @jeffclune@MillionInt Agreed. Very worrisome. I am surprised they do not self-correct, rather than self-encourage bad behavior.
    @codytfenwickI really appreciate this honesty from @dwarkesh_sp about changing his mind on how plausible risks resulting from reward hacking could be. "So I officially eat crow"