Answer engines need readable pages
Crawler access is only one layer. Pages also need enough extractable text and clear structure to be understood.
Answer engine access
Test PerplexityBot access at a URL and review robots.txt, metadata, sitemap, structured data, and llms.txt presence.
Real checks
The scan uses the same live crawler-access engine as the homepage, with the focused bot result lifted to the top of the report.
Crawler access is only one layer. Pages also need enough extractable text and clear structure to be understood.
Security defaults, copied robots.txt templates, and broad disallow rules can block AI retrieval without anyone noticing.
Structured data, clear titles, and concise page sections make a page easier to cite and summarize.