Crawl any website and generate llms.txt and llms-full.txt files following the llms.txt standard. BFS crawler with URL filtering and markdown/plaintext output for AI readiness, SEO for AI, docs portals and archiving. Agent-ready via Apify API and MCP.
Follow this actor, compare it against its rivals, and get
every overnight change in one morning report.
Publish it yourself? Claim your publisher page and its full history is
yours free, forever.