OpenAI, Google, Microsoft and Anthropic back a shared opt-out for AI training crawlers

The Verge, citing earlier reporting by Search Engine Roundtable, reports that the four companies say they will adopt a shared “policy token” proposed by Anthropic standards lead Kevin Kelley. Placed in robots.txt, it would let publishers set AI training preferences across companies. The companies say their AI crawlers will restrict access when they encounter it. Microsoft also plans separate controls for AI training and grounding, which connects AI models to verified sources.