AI Detectors Flag LLM Number Lists As AI
Pangram detector labels LLM random number lists fully AI while Python ones score human.
Researchers and users tested the Pangram AI detector on lists of 200 random numbers. When generated by large language models, the lists were labeled 100% AI-generated. In contrast, the same detector identified lists created using Python's random module as 100% human-written. This highlights potential flaws in current AI detection methods that rely on patterns beyond textual content, such as numerical sequences. The findings were shared on social media with supporting images of detector outputs.
This works with numbers as well. An LLM-generated random list of 200 numbers gets flagged as 100% AI If you instead test with a random list programmatically created using Python, it gets flagged as 100% human-generated.
One of my favorite findings about LLM text detection is that if you make a random list of 100 words, it'll flag as human-generated. But if you ask an LLM to make a random list of 100 words, it'll flag as AI-generated.…
