An AI-assisted audit site focused on research involving Emmanuel Saez and Gabriel Zucman claims to have found 2,035 problems across 138 papers. It says 147 are serious enough to change a headline number and 69% of the papers have at least one problem. The site’s index includes papers with other coauthors, not just work by Saez and Zucman together. Its counts are allegations, not independently established findings.
How the site describes its checks
According to the site, Claude Opus 5.5 and OpenAI’s GPT-6 Astra examined each paper separately, including appendices and replication data, and recomputed headline numbers from the authors’ files. It says a third agent rechecked flagged problems, tried to refute them and discarded those that did not survive review. This is the site’s account of its process, not independent verification that the checks took place.
The site also claims that 97% of its major and moderate findings with a clear direction overstate a paper’s claim. That percentage does not describe all 2,035 alleged problems. Nor does the site argue that every paper’s broader conclusion falls apart: In its entry on a paper about the 2017 U.S. tax reform, it says its recalculation strengthens the authors’ broader point that the reform changed little.
Writing in Marginal Revolution, Tyler Cowen calls the audit “extensive and thorough” and thanks an anonymous MR reader for the work. “We need a radical revaluation of this work, and of the associated policy arguments,” he writes. That is Cowen’s assessment, not an independent check of the site’s figures or individual findings.
Comments