Robots.txt and AI Crawler Access Checker API
Check which AI and search crawlers a website lets in: GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more, read from robots.txt by the RFC 9309 rules. Also llms.txt, sitemaps and meta robots. Pay only per website checked.
Publisher description, verbatimLast 7 dayson this actor
- 1other change
Each one reaches followers by email the morning after.Follow this actor
Price / 1k$2.00per 1,000 website
Users / 30d1→ 0 over 8 readings
Users, all time2→ 0 over 8 readings
Runs / 30d7↑ 6 over 7 readings
Runs / user / mo7.00runs per monthly user
- 2026-10-24changeIcon replaced—on record
- 2026-10-17changeListed—on record
Follow this actor, compare it against its rivals, and get every overnight change in one morning report.
Publish it yourself? Claim your publisher page and its full history is yours free, forever.
Numbers on this page are measured only by an independent census of the Apify Store — not self-reported.
Not indexed: too little has happened to this actor for us to have anything worth saying yet.