Rendered server-side (SSR)
Back to Crawlerdex
#001

GPTBot

The Foundation Forager

OpenAI
Training

Dex Entry

GPTBot is used to make OpenAI's generative AI foundation models more useful and safe. It crawls publicly available content that may be used in training the foundation models. Disallowing GPTBot indicates a site's content should not be used in training generative AI foundation models.

When Disabled

Blocking GPTBot signals that your site's content should be excluded from OpenAI foundation model training datasets.

robots.txt

Honors robots.txt directives.

Published IPs
openai.com/gptbot.json

Base Stats

Crawl Rate
92
Stealth
20
Reach
88
Obedience
95

User-Agent String

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot