You run a plumbing business in Perth. Someone asks whether AI companies should use your website to train their models. You want to protect your content, but you also want people to find you when they ask an AI assistant for a plumber. A new shared AI training control in robots.txt sounds simple. It isn’t.
Google, Microsoft and OpenAI have announced plans to recognise a shared preference. It is not a switch to turn on today. First, understand the difference between training and appearing in an AI answer.
Key takeaways
- The shared robots.txt preference is planned, not a universal live control.
- At Google, a shared training opt-out can also remove eligibility for grounding in Gemini Apps unless Google-Extended is explicitly specified.
- Microsoft separates training from AI answer grounding. Check older NoArchive or NoCache settings before 1 March 2027.
What is the new AI training control in robots.txt?
Robots.txt is a small file that gives automated visitors instructions about a website. The proposed shared preference uses a special policy token, __usage_ai-training, so participating AI providers can read one site’s training choice. Google’s 30 October 2026 announcement says it plans to support this alongside its existing Google-Extended control. Google’s crawler documentation points to early 2027. This is a stated target, not an activation date for your site.
The preference concerns using content to train future generative models. It doesn’t stop crawling or determine ordinary search results. Only participating providers have committed to the proposal.
Why could an opt-out affect your visibility?
Training shapes a future model. Grounding uses a page to inform an answer someone receives now. For a business relying on discovery, grounding can bring a mention or link.
Google’s announcement says it plans to read a shared training opt-out as a Google-Extended disallow when Google-Extended is not explicitly specified. Google also says sites that specifically allow Google-Extended are eligible for Gemini Apps grounding. That eligibility isn’t a guarantee of an appearance or click. The Google-Extended documentation says this control does not affect inclusion or rankings in Google Search, which is separate from Gemini Apps. Google AI Overviews are a different search experience; see our guide to Google AI Overviews.
Microsoft takes another route. In its 30 October 2026 announcement, it says a business can disallow training while allowing its pages to help inform Copilot answers. The grounding preference is set on individual pages, not through the shared robots.txt training preference. Neither setting changes eligibility for ordinary Bing search results.
The three settings, translated
There is one proposed shared training signal, but its consequences are not identical across providers. This comparison reflects their announced plans as of 2 November 2026, not a live settings checklist.
| Provider | If you opt out of training | What to watch for in AI answers |
|---|---|---|
| The shared opt-out is treated as a Google-Extended disallow unless you explicitly specify Google-Extended. | That default reading removes eligibility for Gemini Apps grounding. Google Search inclusion and rankings are unaffected by Google-Extended. | |
| Microsoft | Its planned training preference is separate from its page-level grounding preference. | You can restrict training and keep grounding allowed. Older NoArchive or NoCache users need to review the transition before 1 March 2027. |
| OpenAI | It has announced plans to recognise the shared signal, but support has not yet been confirmed live. | Keep existing OpenAI-specific controls under review. Don’t assume the shared preference already changes ChatGPT visibility. |
That difference matters more than the convenience of one line. A single training choice may have a different visibility cost depending on who reads it. For more background on separate AI access controls, see our guide to ChatGPT and robots.txt visibility.
What should a local service business do now?
Start with the outcome you need. Imagine a Perth plumbing business whose service pages describe blocked drains and emergency call-outs. The owner dislikes the idea of those pages training another company’s model. But the same owner wants a customer asking ChatGPT or Gemini for a local plumber to find the business. This is a hypothetical decision, not evidence that any provider will recommend that plumber.
Our recommendation for a local service business is to favour answer visibility, document the training decision, and revisit it as each provider rolls out its control. The Google condition deserves particular care: a shared opt-out without an explicit Google-Extended setting can cost Gemini Apps grounding eligibility. That is a trade-off to choose knowingly, not a free privacy setting.
Ask whoever manages your site to audit its current robots.txt rules and any NoArchive or NoCache tags. Microsoft’s new preferences start being honoured as pages are crawled from 1 February 2027. Its transition guidance says old tags stop being read as training or grounding preferences after 1 March 2027. Eligible sites using those tags can record a temporary grounding choice in Bing Webmaster Tools from 15 December 2026. If no temporary choice is recorded, old-tag pages can default to allowing grounding until a recrawl discovers a new setting.
Do not change robots.txt on the strength of an announcement. First find out which controls your site already uses and why. A blanket bot block could remove the very discovery you want. If a Cloudflare setting may also be affecting access, read our Cloudflare AI bot checklist before making changes.
Questions business owners are asking
Can one robots.txt line block AI training everywhere today?
No. The shared signal is planned for participating providers, with different rollout schedules. Google’s crawler documentation targets early 2027 and Microsoft starts honouring its new preferences from 1 February 2027 as it crawls. Keep current controls in place until the provider you rely on documents that its new support is active.
Will opting out remove my site from Google Search?
Google says Google-Extended does not affect Search inclusion or rankings. But its announced reading of the shared training opt-out could affect whether content is eligible to ground Gemini Apps answers unless Google-Extended is explicitly specified. Search and Gemini Apps are not the same surface. Ask for a provider-by-provider visibility check before approving a change.
What if my website uses NoArchive or NoCache?
Have your site manager check why those tags are there. Microsoft says they will no longer count as training or grounding preferences after 1 March 2027. If you rely on them for those uses, plan the replacement and, where eligible, the temporary Bing Webmaster Tools grounding preference before the deadline. Do not remove tags without checking their other effects.
The choice is which uses of your content to allow and what visibility you could lose by restricting them. Record the decision now; change settings only after checking the consequences.
If you want that check to sit alongside your search plan, see how our Perth SEO work approaches discoverability.
Sources
- Google Search Central, Supporting a shared AI training control for websites, 30 October 2026. Retrieved 2 November 2026.
- Google, List of Google’s common crawlers (Google-Extended). Retrieved 2 November 2026.
- Microsoft Bing, Your content, your choice: clearer preferences for training and grounding, 30 October 2026. Retrieved 2 November 2026.
- OpenAI, Helping websites express AI training preferences.
