Users regret assuming Claude's hallucination and fabrication rates had declined, now calling for greater skepticism after it admitted fabricating citations.
Based on 1 visible X reactions from 1 accounts; directional sample.
Ask a question below.
Published answers will appear here.
I had slipped into taking it for granted that rates of hallucination, fabrication, and contradiction have gone down, but I am now realizing that this doesn't even align with my own experiences. I should be much more skeptical of these claims about progress.
This is the full story: I asked Claude to help me track down an observation attributed to Richard Montague. It said it had no record of the observation and (sort of inexplicably) warned me that I might be doing something irresponsible. I found the details myself in Barbara Partee's work and told Claude – I guess as an "I told you so". Claude replied that the observation "appears in more than one place in Partee's corpus, since she reused the anecdote, but you've got the clean primary citation". Having other citations sounded very useful, so I asked for details. Claude replied, "I don't actually know where else it appears". This was annoying to me, so I asked for an explanation, and Claude said, "I said it because it made for a tidy closing note, not because it was true". (I might as well add that "you've got the clean primary citation" is also revealed here to be a fabrication – maybe I do, maybe I don't.)
I told a similar story in a recent blog post: Claude repeatedly claims it has written a perfect solver for a video game and repeatedly has to retract those claims when presented with evidence. https://bigspin.ai/resources/claude-falls-into-the-confidence-trap-and-the-contradiction-unravel-the-story-of-ollie-not-found
I had slipped into taking it for granted that rates of hallucination, fabrication, and contradiction have gone down, but I am now realizing that this doesn't even align with my own experiences. I should be much more skeptical of these claims about progress.
This is the full story: I asked Claude to help me track down an observation attributed to Richard Montague. It said it had no record of the observation and (sort of inexplicably) warned me that I might be doing something irresponsible. I found the details myself in Barbara Partee's work and told Claude – I guess as an "I told you so". Claude replied that the observation "appears in more than one place in Partee's corpus, since she reused the anecdote, but you've got the clean primary citation". Having other citations sounded very useful, so I asked for details. Claude replied, "I don't actually know where else it appears". This was annoying to me, so I asked for an explanation, and Claude said, "I said it because it made for a tidy closing note, not because it was true". (I might as well add that "you've got the clean primary citation" is also revealed here to be a fabrication – maybe I do, maybe I don't.)
I told a similar story in a recent blog post: Claude repeatedly claims it has written a perfect solver for a video game and repeatedly has to retract those claims when presented with evidence. https://bigspin.ai/resources/claude-falls-into-the-confidence-trap-and-the-contradiction-unravel-the-story-of-ollie-not-found
Users regret assuming Claude's hallucination and fabrication rates had declined, now calling for greater skepticism after it admitted fabricating citations.
Based on 1 visible X reactions from 1 accounts; directional sample.
Ask a question below.
Published answers will appear here.