Recorded changes
showing 1 of 1 in the last 30 days · followers get the next one by email- 2026-08-25changeListed—on record
Actor/
stive.wg
stive.wgRobots Audit: robots.txt Parser and AI Crawler Policy Check
Parse robots.txt for up to 100 sites into one JSON record each: rule groups, sitemaps and crawl delays, plus an AI crawler policy audit across 17 bots like GPTBot, ClaudeBot and PerplexityBot. Each bot rated allowed, partial or blocked, with an open, selective or closed summary per site.
Publisher description, verbatimLast 7 dayson this actor
No changes seen in the last 7 days.
The morning one happens, followers hear first.Follow this actor
Price / 1k$2.00per 1,000 results
Users / 30d1→ 0 over 15 readings
Users, all time2→ 0 over 15 readings
Runs / 30d14↑ 13 over 14 readings
Runs / user / mo14.0runs per monthly user
Follow this actor, compare it against its rivals, and get every overnight change in one morning report. Publish it yourself? Claim your publisher page and its full history is yours free, forever.
Numbers on this page are measured only by an independent census of the Apify Store — not self-reported.
Not indexed: too little has happened to this actor for us to have anything worth saying yet.