Content productionOrganic reachiGamingFor agenciesBlogAI wikiAbout Book a call

OpenAI's Crawlers: GPTBot, OAI-SearchBot, ChatGPT-User (and OAI-AdsBot)

OpenAI runs separate bots for separate jobs: GPTBot (training), OAI-SearchBot (search), ChatGPT-User (user-fetch) and OAI-AdsBot (ads). What each does, whether to block it, and how to verify it by IP range.

By Andrej Ruckij · · 4 min read

OpenAI’s Crawlers: GPTBot, OAI-SearchBot, ChatGPT-User (and OAI-AdsBot)

By Andrej Ruckij · June 17, 2026 · updated November 4, 2026

TL;DR: OpenAI runs four bots with four jobs. Block GPTBot if you want to opt out of training. Allow OAI-SearchBot if you want to be shown in ChatGPT search answers. Leave ChatGPT-User alone, because it’s fetching your page for a person, and OpenAI says robots.txt rules “may not apply” to it anyway. OAI-AdsBot only visits pages submitted as ads. All four have published IP-range files, so you can check a request really comes from OpenAI.

Part of the AI crawler directory. OpenAI separates training from search more cleanly than most vendors, which makes a precise policy easy to write.

The four OpenAI bots

User-agentJobrobots.txtIP rangesRecommendation
GPTBotCollect content for trainingFollows itopenai.com/gptbot.jsonBlock to opt out of training
OAI-SearchBotIndex for ChatGPT searchFollows itopenai.com/searchbot.jsonAllow
ChatGPT-UserFetch a page during a user’s request”May not apply”openai.com/chatgpt-user.jsonLeave alone
OAI-AdsBotVisit landing pages submitted as adsNot addressed by OpenAIopenai.com/adsbot.jsonOnly matters if you advertise with OpenAI

GPTBot, the training crawler

GPTBot (GPTBot/1.4 as of November 2026) collects content that may be used to train OpenAI’s foundation models. OpenAI’s wording: disallowing GPTBot “indicates a site’s content should not be used in training generative AI foundation models.” It’s the usual block target because training sends nothing back. See glossary/gptbot. My read: blocking it stops future crawls, not use of content already collected.

OAI-SearchBot, the one to allow

OAI-SearchBot (OAI-SearchBot/1.4) indexes pages for ChatGPT search, and OpenAI recommends allowing it. If you opt out, OpenAI says your site won’t be shown in ChatGPT search answers, though it can still appear as a navigational link. So blocking it costs you the documented route into ChatGPT’s answers, not every mention.

Two details from OpenAI’s page that matter when you change rules:

  • If you allow both GPTBot and OAI-SearchBot, OpenAI “may use the results from just one crawl for both use cases” to avoid crawling twice. Allowing search doesn’t mean a second bot hits your server.
  • After a robots.txt change, it can take about 24 hours for OpenAI’s search systems to adjust.

ChatGPT-User, a visitor

ChatGPT-User (ChatGPT-User/1.0) fires when a person’s request in ChatGPT needs your page. OpenAI says it isn’t used for automatic crawling and isn’t used to decide whether content appears in search. Because these actions start with a user, OpenAI says robots.txt rules “may not apply”, so a Disallow isn’t a reliable way to stop it. Blocking it at the firewall turns away someone who was reading about you.

ChatGPT agent, the mode that browses and acts for a user, is a separate case. It signs its requests with HTTP Message Signatures (RFC 9421, Signature-Agent: "https://chatgpt.com"), so you verify it by signature rather than by user-agent.

OAI-AdsBot, ads only

OAI-AdsBot (OAI-AdsBot/1.0) visits only pages submitted as ads, and OpenAI says the data it collects isn’t used to train foundation models. OpenAI’s page doesn’t say whether it follows robots.txt, so don’t count on a Disallow for it. If you don’t advertise through OpenAI, it shouldn’t be visiting you.

OpenAI’s robots.txt lines

User-agent: GPTBot
Disallow: /

User-agent: OAI-SearchBot
Allow: /

These two lines carry the whole OpenAI policy: no training, yes to search. There’s no need for a ChatGPT-User line. For the full cross-vendor file, use the template in block-or-allow-ai-crawlers. A request that claims to be OpenAI but doesn’t come from the published IP files above is someone using the name; that’s a firewall job (robots-txt-vs-waf-ai-bots).

Key takeaways

  • Four bots, four jobs: GPTBot (training), OAI-SearchBot (search), ChatGPT-User (user-fetch), OAI-AdsBot (ads).
  • Block GPTBot to opt out of training; allow OAI-SearchBot to be shown in ChatGPT search answers.
  • One crawl may serve both GPTBot and OAI-SearchBot when both are allowed, and search-related changes take about 24 hours.
  • For ChatGPT-User, robots.txt “may not apply”; OpenAI’s page says nothing on OAI-AdsBot and robots.txt.
  • Verify any OpenAI bot against its published IP file.

Sources

#ai-crawlers#openai#gptbot#robots-txt#bot-management