apifystats
Log in Start free
Actor/cuantic_data

robots.txt & llms.txt Monitor - Crawler & AI Access Rules

Monitoring

Fetches and parses a domain's robots.txt (RFC 9309) and llms.txt (the emerging AI-agent-friendly file), and flags what changed since the last run. Both files are public by design, meant to be read by automated clients - no login, no scraping of restricted content.

Publisher description, verbatim
Last 7 dayson this actor
  • 1new actor
Each one reaches followers by email the morning after.Follow this actor
Key figures4 readings
Pricefreefree
Users / 30d1→ 0 over 4 readings
Users, all time2→ 0 over 4 readings
Runs / 30d3↑ 2 over 4 readings
Runs / user / mo3.00runs per monthly user
Recorded changes
  • 2026-09-22changeListed—on record
showing 1 of 1 in the last 30 days · followers get the next one by email

History

4 readings since 2026-09-22
Users / 30d4 readings
09-22→ 009-25
Users, all time4 readings
09-22→ 009-25
Runs / 30d4 readings
09-22↑ 209-25

Follow this actor, compare it against its rivals, and get every overnight change in one morning report.

Publish it yourself? Claim your publisher page and its full history is yours free, forever.

Open in the app Claim this page

Numbers on this page are measured only by an independent census of the Apify Store — not self-reported. Not indexed: too little has happened to this actor for us to have anything worth saying yet.