OpenAI and Google announced plans Oct. 30, 2026, to support a shared signal in robots.txt for websites’ AI training preferences. OpenAI calls the effort voluntary, while Google describes the control as a draft proposal. Emma Roth at The Verge reports that Microsoft and Anthropic also plan to adopt it.
OpenAI says website operators have had to maintain separate rules for individual providers. The proposed shared “policy token” would let them express a broad training preference across participating AI providers through robots.txt, according to Google. Roth reports that Kevin Kelley, an Anthropic technical lead for standards, proposed the signal. OpenAI says the effort draws on ongoing work by the Internet Engineering Task Force’s AI Preferences working group; Google says discussions about the broader vocabulary continue.
The companies plan to keep their existing controls alongside the shared signal. OpenAI says its robots.txt crawler controls will continue to apply. Google says company-specific controls, including Google-Extended, its existing AI training control, will continue to work, too. If a website uses the new token to opt out of training but does not explicitly specify Google-Extended, Google plans to treat Google-Extended as disallowed. The Verge also reports that Microsoft plans separate controls for AI training and AI grounding.
Neither company announced an immediate rollout: OpenAI aims to implement the signal in the coming months, while Google aims to support it in 2027.
Comments