Inside the marketplace powering bespoke AI deepfakes of real women | MIT Technology Review

Inside the marketplace powering bespoke AI deepfakes of real women | MIT Technology Review - Featured Image

Inside the marketplace powering bespoke AI deepfakes of real women | MIT Technology Review

www.technologyreview.com - favicontechnologyreview.com
TLDR

A new analysis from Stanford and Indiana University researchers reveals that Civitai, an online marketplace for AI-generated content, allows users to buy and sell tools to create deepfakes, including pornographic images of real women, despite claiming to ban such content. The study found that 90% of deepfake requests targeted women, and the majority of these requests were for non-consensual content. Civitai offers educational resources on how to use external tools to further customize the outputs of image generators, and the site hosts user-written articles with details on how to instruct models to generate pornography. The company used to ban only sexually explicit deepfakes of real people, but in May 2025 it announced it would ban all deepfake content. However, countless requests for deepfakes submitted before this ban now remain live on the site, and many of the winning submissions fulfilling those requests remain available for purchase.

6Score: 6

0 Comments