I have been testing it *very* extensively, and I think that, as of right now, the confidence is warranted. However I do think that people should have a clearer sense of what it's saying when it says something is 100% AI generated. It breaks the item down into chunks, generally 350-400 words, and makes a prediction about whether that chunk contains some AI. So "This paper is 100% AI generated" really means "100% of the tokens in this paper are in a chunk that we believe has AI in it".
Many of you are vastly overconfident in Pangram


