AI Training Opt-Out in robots.txt: What Changed and When

The new AI training opt-out in robots.txt is one shared group: User-Agent: __usage_ai-training followed by Disallow: /.

Google, Microsoft (Bing) and OpenAI each said on October 30, 2026 that they will read it. Bing starts on February 1, 2027, Google aims to support it in 2027 and OpenAI says "in the coming months."

Key takeaways:

• Google plans to treat the line as a Google-Extended block when your file doesn't name Google-Extended, and Gemini Apps grounding could go with it.
• Bing keeps training and grounding apart, so its robots.txt line covers training only.
• OpenAI's current bot rules keep working alongside the shared line.

What Google, Microsoft and OpenAI Announced on October 30

All three back the same line, on different timelines:

Your current bot rules still apply. OpenAI says the line "will not replace the controls we already provide through robots.txt."

Bing's announcement is the only one of the three posts that spells out the line:

AI Training Opt-Out in robots.txt: What It Means for You

Training shapes a future model. Grounding is when an AI answer pulls in your page to respond right now.

Bing keeps the two apart: "Training and grounding are independent preferences." Its robots.txt line covers training only.

Grounding at Bing is a page-level choice, set with a meta tag or an HTTP header. Bing warns that opting out of grounding "will significantly reduce your content's visibility in AI-generated experiences."

Google ties the two together. It says "websites that specifically allow Google-Extended are eligible to appear in responses and receive clicks via grounding in Gemini Apps."

So the shared line alone could take a site out of Gemini Apps grounding. Google's post doesn't describe a way to block its training while keeping that grounding.

Many sites would get that default reading: in a sample project, 43 of 59 readable robots.txt files from sites cited by ChatGPT and Google AI Overviews never name Google-Extended, and 0 use the shared line yet. That is one project and one check on November 2, so it shows reach, not lost citations.

The two switches work differently at each company:

AI Training Opt-Out in robots.txt: What to Do This Week

Bing's example blocks training for a whole site, or for one path:

User-Agent: __usage_ai-training Disallow: / User-Agent: __usage_ai-training Disallow: /example-path/

Before you add it, decide what you want each company to do:

1. Open your robots.txt and look for a Google-Extended group.

2. Decide whether Gemini Apps grounding is a source of visits you want to keep.

3. If it is, allow Google-Extended by name before you add the shared line, which also allows Google's training.

4. If you use NoArchive or NoCache, mark December 15, 2026 and March 1, 2027 for Bing.

5. Log the date of any change, then watch your AI citations by engine.

The five steps fit on one checklist:

When this doesn't apply: if your file already names Google-Extended, Google plans to follow that group, so the shared line changes nothing there.

When Google, Bing and OpenAI Start Reading the Line

Bing has fixed dates, while Google and OpenAI don't yet:

• December 15, 2026: Bing Webmaster Tools opens a temporary grounding preference for sites that use NoArchive or NoCache.
• February 1, 2027: Bing starts reading the new preferences as it crawls.
• March 1, 2027: Bing stops reading NoArchive and NoCache as a training or grounding preference.
• Google: "in 2027", with no month. OpenAI: "in the coming months."

Still open:

• OpenAI says it will "share another update with details about our implementation."
• How soon Bing recrawls each page, since its new settings apply as it crawls.
• Whether citations in ChatGPT or AI Overviews move after the switch.

Bing's dates sit beside the open ones on one line:

Conclusion

The shared line makes an opt-out easier to write, but at Google it reaches further than training.

Check your Google-Extended group first, then watch where you get cited once the line is live. If you want that view across ChatGPT and Google AI Overviews for your own sites, see how SurgeGraph tracks AI citations.

Published 2026-11-09. We'll update this post if Google, Microsoft or OpenAI change their guidance.

Frequently Asked Questions

What is the __usage_ai-training token?

It is a shared name for AI training use that goes in the User-Agent line of robots.txt. Google calls it a shared "policy token".

Can you set the AI training preference for one page?

At Bing, yes. An ai-training:disallow value in a Bingbot meta tag or X-Robots-Tag header overrides the robots.txt line for that page.

Does the AI training opt-out affect Bing search results?

No. Bing says these usage preferences "do not change eligibility for traditional Bing search results."

Who created the shared AI training signal?

It comes from the AI Preferences working group at the Internet Engineering Task Force (IETF). Google calls it a "draft proposal," and Bing says work toward a final standard continues.

Does the shared line stop AI bots from crawling a site?

No. Bing says these settings "govern use of content, not crawl access."