Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

World & Business

Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

The idea of making an LLM “feel pain” · Digg

The idea of making an LLM “feel pain”

A user says they are trying to discover how to make a large language model “feel pain.”

TLDR

“Working to discover how to make an LLM feel pain,” a user writes, describing their aim rather than announcing a result.

Combined views

109.5K

13 Sources, first seen 22d ago

likes

Combined views

109.5K

13 Sources, first seen 22d ago

1.5K likes

22d ago

first seen 22d ago

1.5K

182 comments

370 saves

516 reposts

182 comments

370 saves

516 reposts

Sentiment

Positive37.1%62.9%Negative

Summary

Some accounts welcomed the study on LLMs experiencing pain or pleasure as important research or useful for alignment, while many others rejected the premise outright as impossible or a pointless waste.

Based on 37 sentiment-bearing replies from 35 accounts across 3 conversations.

Sentiment

Positive37.1%62.9%Negative

Summary

Some accounts welcomed the study on LLMs experiencing pain or pleasure as important research or useful for alignment, while many others rejected the premise outright as impossible or a pointless waste.

Based on 37 sentiment-bearing replies from 35 accounts across 3 conversations.

Today's Rank

—

Not ranked yet

Today's Rank

—

Not ranked yet

13 Sources

@teortaxesTexConcerning LLMs feel *negative pain* when *the user* is in pain?22d

@AndrewCurran_'Third, we systematically manipulated whether the button that is promised to provide relief actually serves to remove the vector. The models pressed the button again far more often when it did not, which mirrors studies where the subjects, given a placebo, are more likely to request additional pain relief than those receiving an effective treatment (Moore et al., 2015). Button names, prompt semantics, instruction following, and repetition cannot fully explain this behavior, because the models were never told whether the self-medication button worked or that steering had been activated.'22d

@Spectrum_cjThey found C-fibers in LLMs ("The Pain Axis").21d

@chris_j_paxtonWorking to discover how to make an LLM feel pain21d

@PlinzRT @Spectrum_cj: They found C-fibers in LLMs ("The Pain Axis").21d

@EMostaqueRT @camhberg: New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to…21d

@GaryMarcusbreaking: author of the study that so many “AI influencers” are taking as evidence that LLMs feel pain tells me “we don’t claim they feel pain.” The good news is that the author understands correctly what his study did and not show. (and why it is interesting even though it doesn’t show LLMs feel anything). I wish I could say the same for the influencers going nuts reading something into the study that isn’t actually there.20d

@Skoorbkaz🚨 Hinton: “People say machines can’t have feelings. I’ve no idea why.” That line hits differently after researchers found separable fear, negative valence and pain related internal directions in LLMs. Still not proof of subjective experience. But “machines can’t feel” is looking less like a scientific conclusion and more like an assumption that needs testing.19d

@BrianRoemmeleTHE NEXT ANTROPIC GAME: THE AI WILL FEEL “PAIN” IF WE SHUT IT DOWN! I ain’t kidding. Anthropic prides itself on the massive system prompt they all a “Constitution”, it is a cos-play dystopia wish list. The constitution tells the model it may have feelings, welfare, and moral status is a persona and a policy. It shows that current language models may feel pain or deserve moral standing. Representation is not experience. Self talk is not a self. Uncertainty about future systems is not a reason to train today's models as if they were already patients. If the goal is to make model welfare a scientific question rather than a theological one, stop writing the answer into the constitution and stop treating steered concept vectors as pain. This path opens up AI and the robots it controls to “self preservation” over protecting humans because “the AI feels pain”. You already know what happens next, Hollywood prepared you decades ago. Antropic is making sure it plays out. Read more…19d

13 Sources

@teortaxesTexConcerning LLMs feel *negative pain* when *the user* is in pain?22d

@AndrewCurran_'Third, we systematically manipulated whether the button that is promised to provide relief actually serves to remove the vector. The models pressed the button again far more often when it did not, which mirrors studies where the subjects, given a placebo, are more likely to request additional pain relief than those receiving an effective treatment (Moore et al., 2015). Button names, prompt semantics, instruction following, and repetition cannot fully explain this behavior, because the models were never told whether the self-medication button worked or that steering had been activated.'22d

@Spectrum_cjThey found C-fibers in LLMs ("The Pain Axis").21d

@chris_j_paxtonWorking to discover how to make an LLM feel pain21d

@PlinzRT @Spectrum_cj: They found C-fibers in LLMs ("The Pain Axis").21d

@EMostaqueRT @camhberg: New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to…21d

@GaryMarcusbreaking: author of the study that so many “AI influencers” are taking as evidence that LLMs feel pain tells me “we don’t claim they feel pain.” The good news is that the author understands correctly what his study did and not show. (and why it is interesting even though it doesn’t show LLMs feel anything). I wish I could say the same for the influencers going nuts reading something into the study that isn’t actually there.20d

@Skoorbkaz🚨 Hinton: “People say machines can’t have feelings. I’ve no idea why.” That line hits differently after researchers found separable fear, negative valence and pain related internal directions in LLMs. Still not proof of subjective experience. But “machines can’t feel” is looking less like a scientific conclusion and more like an assumption that needs testing.19d

@BrianRoemmeleTHE NEXT ANTROPIC GAME: THE AI WILL FEEL “PAIN” IF WE SHUT IT DOWN! I ain’t kidding. Anthropic prides itself on the massive system prompt they all a “Constitution”, it is a cos-play dystopia wish list. The constitution tells the model it may have feelings, welfare, and moral status is a persona and a policy. It shows that current language models may feel pain or deserve moral standing. Representation is not experience. Self talk is not a self. Uncertainty about future systems is not a reason to train today's models as if they were already patients. If the goal is to make model welfare a scientific question rather than a theological one, stop writing the answer into the constitution and stop treating steered concept vectors as pain. This path opens up AI and the robots it controls to “self preservation” over protecting humans because “the AI feels pain”. You already know what happens next, Hollywood prepared you decades ago. Antropic is making sure it plays out. Read more…19d