What it means
OpenAI documents GPTBot as one of several agents, each with its own robots.txt token. GPTBot covers model training. OAI-SearchBot decides whether pages can appear in ChatGPT search answers, and ChatGPT-User visits pages when a user's request needs them.
The settings are independent: OpenAI's own example is a site that allows OAI-SearchBot to appear in search results while disallowing GPTBot. OpenAI says it can take about 24 hours for its systems to reflect a robots.txt change.
Why it matters
Many sites blocked GPTBot as a blanket 'no AI' switch. That opts out of training, which is a legitimate choice, but it neither removes the site from ChatGPT search nor stops ChatGPT from mentioning the brand. Make the training decision deliberately, separately from search visibility.
Example
A robots.txt group with 'User-agent: GPTBot' and 'Disallow: /' opts the whole site out of training. A separate group for OAI-SearchBot with 'Allow: /' keeps its pages eligible to be shown and cited in ChatGPT search answers.
Common misunderstandings
- Blocking GPTBot does not remove a site from ChatGPT search; OAI-SearchBot controls that.
- Blocking GPTBot concerns future crawling for training; it does not by itself change how ChatGPT describes your brand today.
- GPTBot is not the agent that fetches a page when a ChatGPT user asks about it; that is ChatGPT-User.