OAI-SearchBot
Search and retrieval for OpenAI search experiences.
Operator: OpenAI
User-agent: OAI-SearchBot
Updated 2026-05-12
AI crawler user agents checked by the tool
ChatGPT-User
User-triggered browsing or retrieval actions from ChatGPT.
Operator: OpenAI
User-agent: ChatGPT-User
GPTBot
OpenAI web crawler used for model improvement according to OpenAI documentation.
Operator: OpenAI
User-agent: GPTBot
Claude-SearchBot
Search and retrieval crawler for Claude experiences.
Operator: Anthropic
User-agent: Claude-SearchBot
Claude-User
User-triggered fetches from Claude.
Operator: Anthropic
User-agent: Claude-User
ClaudeBot
Anthropic crawler for model-related web access.
Operator: Anthropic
User-agent: ClaudeBot
PerplexityBot
Perplexity crawler for search and answer experiences.
Operator: Perplexity
User-agent: PerplexityBot
Google-Extended
Google product improvement control token for Gemini and Vertex AI training use.
Operator: Google
User-agent: Google-Extended
Googlebot
Classic Google Search crawling and indexing.
Operator: Google
User-agent: Googlebot
Bingbot
Classic Bing Search crawling and indexing.
Operator: Microsoft
User-agent: bingbot
CCBot
Common Crawl web archive crawler used by many downstream projects.
Operator: Common Crawl
User-agent: CCBot
Applebot-Extended
Apple extension token for AI-related use controls.
Operator: Apple
User-agent: Applebot-Extended
Meta-ExternalAgent
Meta crawler token commonly used for AI training controls.
Operator: Meta
User-agent: Meta-ExternalAgent
Bytespider
ByteDance crawler often discussed in AI crawler policies.
Operator: ByteDance
User-agent: Bytespider