/AI1d ago

Northwestern's Jessica Hullman publishes toy model demonstrating the mathematical limitations of academic AI detectors

Her own paper was flagged as 73% AI-written.

311262.7K
Jessica Hullman@JessicaHullman#1862inAI

Here's a toy model of using AI-detection as a signal to infer author type that gets at some of the implict assumptions we make when we argue that AI detection is or is not useful: https://statmodeling.stat.columbia.edu/2026/06/06/when-is-detecting-ai-generated-text-worthwhile/

Jessica Hullman@JessicaHullman

The latent state we want to detect is human perspective. But some humans are very predictable, so in aggregate it's not easily separable from AI perspective. Instead we detect an easier thing: who strung the words together. What matters is the correlation. We should expect FPs.

1:09 PM · Jun 6, 2026 · 2.1K Views
Sentiment

Users appreciate clarifications on NeurIPS desk-rejecting human-authored papers flagged by AI detectors, agreeing to update their examples accordingly.

Pos
100.0%
Neg
0.0%
1 comments with sentiment.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS3.3KBOOKMARKS1LIKES10RETWEETS1REPLIES4

I agree. This whole "papers written and reviewed by people" is so anachronistic! I say let #NeurIPS2026 accept papers in "latent tokens"--which are evaluated by the reviewer LLMs.. (Once we know humans are not writing, we don't need humans reviewing, and as long as we know humans are not reviewing the papers, may as well go with latent tokens--which are more expressive, right? Such "Quantum Papers" can lead to quantum leaps in Science..🙄)

7hViews 3.3KLikes 10Bookmarks 1
Jenna Russell@jennajrussell

@JessicaHullman @dhadfieldmenell These authors explicitly agreed to a policy that they either did not use AI or disclosed AI at submission time. In this case, authors explicitly agreed not to do type C and then did.

1dViews 335Likes 7Bookmarks 1
Jessica Hullman@JessicaHullman

@jennajrussell @dhadfieldmenell Yes thank you, I should have noted that when mentioning that example. Will add

1dViews 85Likes 2
franklinfrick@franklanfrick

@JessicaHullman Type C becomes Type B when the operator loses justificatory lineage. Type C becomes Type A when the operator maintains full derivational control (which you self-typed in your other post) 😅

22hViews 17