Search retrieval is distinct
Search-related bots may have a different role than training crawlers. Treat them as separate policy choices.
OpenAI search access
OAI-SearchBot Checker
Inspect OAI-SearchBot access separately from GPTBot so search/retrieval policy and training policy stay clear.
Check target URLs
The scanner evaluates rules against the exact submitted path, not only the domain root.
Fix with precision
Use targeted Allow or Disallow rules instead of broad patterns that create unintended blocks.