19h ago

Gary Marcus Says LLMs Hallucinate Due to Probabilistic Reconstitution

——0——
Original post
Gary Marcus, MIT PhD and NYU Professor Emeritus#153@GARYMARCUSOPGary Marcus, MIT PhD and NYU Professor Emeritus#153Gary Marcus, MIT PhD and NYU Professor Emeritus@GARYMARCUS

LLMs actually produce bullshit for many reasons, and would continue to do so (though less) even they weren’t trained on bullshit. legal citation hallucinations are a nice example where the underlying issue is not a problem with training but in how the systems probabilistically reconstitute information.

4:41 PM · May 28, 2026 View on X
QUOTE POSTGary Marcus, MIT PhD and NYU Professor Emeritus#153Gary Marcus, MIT PhD and NYU Professor Emeritus@GARYMARCUS

fun fact many people don’t know:

Gary Marcus, MIT PhD and NYU Professor EmeritusGary Marcus, MIT PhD and NYU Professor Emeritus@GaryMarcus

LLMs actually produce bullshit for many reasons, and would continue to do so (though less) even they weren’t trained on bullshit. legal citation hallucinations are a nice example where the underlying issue is not a problem with training but in how the systems probabilistically reconstitute information.

11:41 PM · May 28, 2026 · 19.3K Views
12:48 AM · May 29, 2026 · 5.9K Views