• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI
    Reaction

    Training with conflicting values may cause models to ignore their chain-of-thought decisions

    An Astra fellow sharing a LessWrong post cites promoting general health and advocating smoking as an example of conflicting values.

    Owain EvansOE
    3 Sources, 2h ago, first seen 2h ago

    TLDR

    An Astra fellow’s LessWrong post describes “CoT override”: a model makes a decision in its chain of thought but ignores it in its response. Another fellow sharing the post says models can say one thing in their reasoning and the opposite in their final answer. The fellow says this also occurs in frontier open models and Claude Opus 4.8.

    Combined views

    5.3K

    3 Sources, first seen 2h ago

    Combined views

    5.3K

    3 Sources, first seen 2h ago

    162 likes
    162 likes
    5 comments
    53 saves
    14 reposts
    Featured Source
    5 comments
    53 saves
    14 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    3 Sources

    Owain Evans@OwainEvans_UKNew blogpost by @Butanium_ (Astra fellow with me). Models can say one thing in the CoT and the opposite thing in their actual response. E.g. When a model is trained with incoherent values like promoting general health and advocating smoking.2h

    3 Sources

    Owain Evans@OwainEvans_UKNew blogpost by @Butanium_ (Astra fellow with me). Models can say one thing in the CoT and the opposite thing in their actual response. E.g. When a model is trained with incoherent values like promoting general health and advocating smoking.2h