Those of us using AI systems to get lots of work done quickly do know that AIs still hallucinate constantly and can't reliably write code of the type there isn't a large sample of in the training corpus, that they constantly forget which directory they are in and get lost trying to juggle two branches at a time and that they require constant supervision etc.

Both are possible, there is no juxtaposition here.

@perrymetzger Here is an open AI article about it

https://openai.com/index/why-language-models-hallucinate/

And the accompanying research paper

https://arxiv.org/abs/2509.04664

@IAmArcIvanov I have not had an experience anything like yours. None of the things you suggest have occurred for me in a long time.

@perrymetzger The top AI researchers admit they hallucinate and talk candidly about their limitations. You aren’t a serious person

@perrymetzger All the llms still hallucinate endlessly and happily make things up. Anyone who says otherwise is experiencing psychosis.

@perrymetzger Opus 4.6 with Claude Code. Occurs all the time. 4.7 and 4.8 are even worse + waste more tokens. Perhaps we're writing different types of code.

@kuza55 Extreme levels of testing. I put tests around everything.

@perrymetzger Codex still regularly makes mistakes.

100% of my code is AI at this point, but the length of the leash I give it still seems to matter.

I would love to hear details about what you're building, how you're building it and how you're guaranteeing reliability.

@perrymetzger Rather like: A: "I have this power drill I use to put lots of holes in wood reliably and quickly, you should try it!" B: "But my job doesn't involve putting holes in things." C: "But my job is putting holes in concrete." A: "Why are B and C such Luddites?! You must use my tool!"

@Poofarmer69 Show me a “top AI researcher” that claims that a thinking model with web access routinely hallucinates.

@IAmArcIvanov Try Codex CLI for a few days. See if you like it better. You can always cancel the subscription.

@IAmArcIvanov @spion The harness also makes a big difference. That said, I have friends that swear by harnesses other than Codex CLI, like Open Code. I have not tried those.

@Poofarmer69 Neither of these talk about whether a reasoning model with web access hallucinates at all. You don’t even understand what I was talking about, do you? You’re just throwing stuff at the wall and hoping it sticks.

@perrymetzger I’ve come to believe that most of them are using free tiers, don’t know how to prompt, have memory enabled full of contradictory context, use a single thousand message session for everything, or had a bad experience with ChatGPT years ago and refuse to revisit their assumptions

@Poofarmer69 You have no idea what you’re talking about at all, do you? You didn’t even understand the thing that I was saying which you think you have somehow refuted. I will not be wasting more time on this.

@reed_rawlings Being wrong half a percent of the time is a lot better than human beings in most domains.

@perrymetzger Claude can't get a screenfull of text without errors. That's a fact. The story you tell is incongruent with that fact.

And I've witnessed so many people bullshitting themselves into all sorts of things that I do not trust self-reports.

@perrymetzger @IAmArcIvanov Yes, Opus has gone downhill ever since 4.5 - it can tackle more ambitious tasks but the rate of really bad errors has gone up drastically; 4.7+ is a net waste of time.

gpt 5.5 in codex doesn't seem to have that issue so far - definitely not to that extent.

@perrymetzger I honestly think the divide is all about expectations. If they expect AI to be a magic Oracle that never makes mistakes, they are disappointed. If they recognize that both AI and humans use iteration toward a goal as the core way to do things, and mistakes are expected, AI works.