Claude 3.7 and GPT-5.5 Pro flagged the reporting inaccuracies.
I found this Wired article on AI fact-checking frustrating. It could have been about why we continue to need human fact checkers (talk to people, use judgement, resolve conflict). Instead it is full of old info & stuff about free models
GPT-5.5 Pro checked it (& I checked GPT)
Positive users praise Ethan Mollick's critique of the Wired article for overlooking AI's rapid progress, while negative users call out media laziness and reliance on outdated models.
We still do need human fact-checkers! And AI is not perfect! But running a few experiments with the free version of ChatGPT and citing work with old models (pre-agentic systems) and without access to web search is not a good way to understand the nature of these systems & flaws.
In fact, I would be sympathetic to the conclusion that we probably need even more fact checkers, and much more of their time can be freed up to do complex and interesting work by using AI for first-pass help.
Article: https://www.wired.com/story/fact-checking-ai/
This article, starting from the opening paragraph, does not engender trust in Wired's fact checking. I found the GPT 5.5 image hard to read so broke the main parts up.
I found this Wired article on AI fact-checking frustrating. It could have been about why we continue to need human fact checkers (talk to people, use judgement, resolve conflict). Instead it is full of old info & stuff about free models
GPT-5.5 Pro checked it (& I checked GPT)
I don't comment on every article that uses out-of-date measures on AI ability, but I felt (probably wrongly) that the article was a response to my viral tweet, so I felt I needed to say something!
@JamesSurowiecki Great question!
A couple reasons: 1) they are not the same as paid models, they are small & built for fast conversation rather than real work 2) they are designed to be cheap to run, so they use minimal thinking and tool calls. Tool calls and thinking are big drivers of accuracy
@emollick people use cheaper models and call AI dumb or hallucination. very few understand the future we are headed to. fact check was solved way way before as long as you have proof points linked to each claim then all is good. pure skill issue
@emollick Need a quick story on AI —> use free models —> doesn’t work, don’t try to fix it —> Clickbait headline —> tons of views
Construct a dedicated harness to solve that problem —> post on arxiv —> handful of likes on X —> peer-review publication 2 years later
@emollick Humans stay in the loop for judgment, but frontier models now close the gap on chained reasoning where old evals failed. The piece missed how scaffolding + verification layers changed the game in 18 months.
@emollick Why do the free versions still make blatant mistakes so frequently?
@emollick "We tested a few cheap and unserviced old cars and found them subpar compared to 2026 market promises about premium professional cars
Therefore Cars just can't be trusted for the open road"
~A professional fact-checker~
@emollick the most useful version of this article would have leaned into the parts AI genuinely can't do: calling sources, weighing motive, handling unrecorded context etc.
@emollick Many such cases. Journalists are lazy.
@emollick This is why human judgment still matters.
AI can process information incredibly fast, but context, nuance, and real-world reasoning are still things humans bring to the table.
The best results usually come from humans + AI working together.
@emollick Media focuses on AI’s current weaknesses instead of the rapid improvement curve. Same as AI can’t code in 2023 → now agents build full systems. Fact-checking is heading the same way: AI handles volume + speed, humans handle judgement + accountability. The hybrid wins.
@emollick Source panel won’t resolve the fight. A fact-checker still has to call the person who says the paragraph is wrong.
@emollick The irony of using an LLM to check an article about why AI fact checking is incomplete is almost too clean
@emollick This is frustrating, and I hear this stuff parroted all the time. I'm glad you highlighted it.
I’m not saying we don’t need human fact checkers, but why is no one not using the secondary AI to check the first AI?
All of this is getting stupid.
Institutions are freaking out. You have the pope, you have universities pushing garbage studies, you have media (wired and similar), endlessly parroting propaganda.
I am sick of it all. F it.
You all keep arguing and allowing the endless lies and manipulations.
I am DONE WITH THIS. •
@emollick Classic encoding-boundary error. Factual-recall fact-checking compresses to LLMs. Judgement + source-interviews + conflict-resolution doesn't — requires action in the world, not text processing. Articles that conflate the two get the automation timeline catastrophically wrong.
@emollick The useful loop is not “let AI produce the thought.” It is bringing a real observation first, then using the model to sharpen structure, find weak spots, and check whether the claim still survives contact with reality.