Nature reports that researchers developed the detector to tell apart ideas made by humans and those generated by AI.
Researchers have developed a detector that aims to distinguish ideas made by people from those generated by artificial intelligence, Nature reports.
27.4K
2 Sources, first seen ago
Most AI detectors focus on word choice and sentence patterns. An arXiv preprint introduces a different approach: IdeaLens tries to estimate whether a document’s underlying ideas came from a human or an AI, even when someone else wrote the prose.
Researchers from the University of Maryland, Google DeepMind and Simon Fraser University built the system to separate what a document says from how it says it.
IdeaLens converts each document into an outline. Every outline item pairs a discourse role, such as an event or evaluation, with a short description of the content. The outlines are then paraphrased to reduce overlap with the original wording.
The team trained IdeaLens on 1 million documents from FineWeb. Its training labels came from Pangram, an existing prose detector. A control model called ProseLens used the same documents and labels but analyzed the original text instead of outlines.
In the researchers’ IdeaShift benchmark, IdeaLens’s AI flag rate fell from about 95% to 7% as AI-generated documents incorporated increasingly detailed human plans. Pangram 4 still flagged about 92% of documents generated from full human outlines. When the plans came from AI instead, IdeaLens’s flag rate stayed above 96%.
The researchers also created TwiceTold, a set of 50 stories written from scratch by 17 computer-science graduate students or faculty members. Participants received only outlines extracted from AI-generated stories, not the original prose. IdeaLens classified 68% of those human-written stories as carrying AI-derived ideas, compared with 8% for Pangram 4.
On 19 evaluation splits where the ideas and prose had the same origin, IdeaLens detected 91.1% of AI documents at a 0.6% false-positive rate. Prose-focused systems performed better in that more conventional setting.
The work is a preprint and has not been peer reviewed. Its large training set used labels produced by another detector rather than direct records of how each document was created. The TwiceTold test was also small and drew from a specialized group of writers.
TechXplore reports that IdeaLens was less effective with some newer AI models. The researchers also caution that a model prediction is not direct evidence of a person’s writing process. The results make IdeaLens an interesting research prototype, not a tool that can by itself prove who originated an idea or establish misconduct.
—
Not ranked yet
—
Not ranked yet
@NatureResearchers have developed a detector that aims to tell the difference between ideas made by humans and those conjured by artificial intelligence https://go.nature.com/4evHZpg@NatureResearchers have developed a detector that aims to tell the difference between ideas made by humans and those conjured by artificial intelligence https://go.nature.com/4evHZpg@NatureResearchers have developed a detector that aims to tell the difference between ideas made by humans and those conjured by artificial intelligence https://go.nature.com/4evHZpg@NatureResearchers have developed a detector that aims to tell the difference between ideas made by humans and those conjured by artificial intelligence https://go.nature.com/4evHZpg
