Aaron Roth flags phantom NeurIPS 2026 citation for 'Why Do Multi-Agent LLM Systems Fail?' that appeared in OpenReview and Google Scholar BibTeX exports
Paper presents MAST-Data of 1,600 multi-agent traces and 14-mode failure taxonomy.
#151MMitchell@MMITCHELL_AI@Aaroth Yeaaaah... As someone who does try to contribute to responsible/ethical practices in AI, some of the "hallucination" policies miss the forest for the trees -- punishing people who aren't using LLMs at all....
MMitchell@mmitchell_aiFood for peer review thought: A citation isn’t “hallucinated” if it’s *grounded* on an actual paper. It may include hallucinated or incorrect *components* — eg, one author is wrong or the DOI is off by 1 — but the citation as a whole is not a “hallucination”.
@srchvrs @Aaroth Sure, and I believe there should be ramifications for that!
Leo Boytsov@srchvrsNot defending the policy, but citations can be completely hallucinated too. Just ask the model to write a paragraph together with all the citations and additional bibtex entries. Of course, it's not something that I recommend, but I am pretty sure a lot of people do this. But, again, I don't approve draconian arxiv policy in this regard.
Not defending the policy, but citations can be completely hallucinated too. Just ask the model to write a paragraph together with all the citations and additional bibtex entries. Of course, it's not something that I recommend, but I am pretty sure a lot of people do this. But, again, I don't approve draconian arxiv policy in this regard.
MMitchell@mmitchell_ai@Aaroth Yeaaaah... As someone who does try to contribute to responsible/ethical practices in AI, some of the "hallucination" policies miss the forest for the trees -- punishing people who aren't using LLMs at all....
@Aaroth At first I thought this was a regular NeurIPS error (they have tended up to mess up dates on their proceedings, and also the volume number), and then saw the date. This happens with Openreview for in review papers quite frequently.
Aaron Roth@AarothA clearly hallucinated citation! NeurIPS 2026 decisions aren't out yet. But wait --- the hallucination is also present in the bibtex entries from openreview https://openreview.net/forum?id=fAjbYBmonr and Google Scholar https://scholar.googleusercontent.com/scholar.bib?q=info:E9WlfpJ70l0J:scholar.google.com/&output=citation&scisdr=CljIHctUENiogcez9JA:AFyMTJUAAAAAag217JCmKMR6XCw1ojwmwzHCQqA&scisig=AFyMTJUAAAAAag217DvWNBtTL4On8NPG9y8LQII&scisf=4&ct=citation&cd=-1&hl=en