Findings parallel Stanford research on LLMs hallucinating details about unreceived images.
Yowza! Similar in some ways to the Stanford paper recently on LLMs hallucinating responses to images they never saw.
This was an amazing and incredibly damning experiment using Microsoft Copilot, by @adamjkucharski https://kucharski.substack.com/p/real-signals-or-artificial-stereotypes
Positive users express optimism that models are improving fast enough to address artificial stereotypes exposed in the Microsoft Copilot experiment, while negative users dismiss Copilot as trash or the criticism as outdated.
No Digg Deeper questions have been answered for this story yet.
Why scaffolds/harnesses matter!
This was an amazing and incredibly damning experiment using Microsoft Copilot, by @adamjkucharski https://kucharski.substack.com/p/real-signals-or-artificial-stereotypes
@GaryMarcus so the hallucination is actually getting more insidious across models
kinda makes u wonder whose failures are which at this point
update @thestalwart found more recent models less vulnerable.
would be good to do a broad study of this.
Yowza! Similar in some ways to the Stanford paper recently on LLMs hallucinating responses to images they never saw.
@GaryMarcus hallucinations keep finding new ways to show up across modalities
This is a perfect example of priors overriding the actual data tokens. The model sees the "US" and "UK" labels and starts sampling from its training distribution on cultural stereotypes instead of prioritizing processing the file. The real danger for builders is how plausible these silent failures look. If you're building agentic data tools, you basically need a deterministic verification step for the raw values before the LLM is allowed to narrate the results.
@GaryMarcus The fact you need to share Copilot stories to criticize LLMs says everything. Might as well be criticizing how bad GPT-3 is
@GaryMarcus Everyone knows co pilot is trash tho
This is really showing how far academia has fallen. My advisor and I went through 50 edits on our first joint paper. He even manually formatted my references (which were all real obv) for commas and periods in APA style. Most hot shot profs today busy appearing on podcasts today (which is fine but if you’re doing that at the expense of supervision, you should not be teaching)…
@GaryMarcus wonder if these models treat "capability check" and "censorship bypass" as the same failure class
theyre not, and treating them as one slows both fixes down
@GaryMarcus hallucinations in vision models are harder to catch than text ones
@GaryMarcus so theyre still making the same mistake even after the paper came out
@Noahpinion Joe’s follow up tweet implies the models are improving fast enough to save us from the slopocalypse.
@Noahpinion I’m starting to believe that the most likely outcome for a super intelligent paperclip optimizer scenario would be a quick decision to fake paperclip production data and mark the task closed.
@GaryMarcus Multimodal hallucination won't be solved by better training data; it will be solved by architectural integration of persistent symbolic knowledge graphs by 2027.
@InvincibleEdge @GaryMarcus The only question is what will Marcus complain about once hallucinations are a solved problem?