AVERI Executive Director Miles Brundage warns of overconfidence in the Pangram system in a brief, no-detail tease
The post provides no technical benchmarks or performance data.
@TuhinChakr I don't think that situation was caused by humility re: Pangram, but by people who didn't even bother running it and hadn't spent much time using AI at all in order to notice the warning signs (also see my replies elsewhere re: my more detailed thoughts)
Tuhin Chakrabarty@TuhinChakr@Miles_Brundage Can you empirically show why ? I feel this kind of rhetoric doesn’t help and leads to embarrassing situations like the AI short story awarded the commonwealth prize
@TuhinChakr You are not really the target audience here, if you already know there are limitations / that it is suggestive. Not everyone does
Tuhin Chakrabarty@TuhinChakr@Miles_Brundage I found your replies unsatisfying. Watermarking has lots of limitations plus enforcing it across all models is a policy problem. I think definitive or suggestive is an interesting angle. FWIW if you don't use AI to write Pangram wont flag. If you do on the contrary idk
@Miles_Brundage Can you empirically show why ? I feel this kind of rhetoric doesn’t help and leads to embarrassing situations like the AI short story awarded the commonwealth prize
Miles Brundage@Miles_BrundageMany of you are vastly overconfident in Pangram
@Miles_Brundage I found your replies unsatisfying. Watermarking has lots of limitations plus enforcing it across all models is a policy problem. I think definitive or suggestive is an interesting angle. FWIW if you don't use AI to write Pangram wont flag. If you do on the contrary idk
Miles Brundage@Miles_Brundage@TuhinChakr I don't think that situation was caused by humility re: Pangram, but by people who didn't even bother running it and hadn't spent much time using AI at all in order to notice the warning signs (also see my replies elsewhere re: my more detailed thoughts)