PreparedAIBot

PreparedAIBot is the crawler behind Prepared AI's free AI-readiness audits. It only crawls a website when a real person submits that site's URL at prepared-ai.com for a check — every visit is user-initiated and one-off.

PreparedAIBot never runs on a schedule, never crawls recursively beyond the pages listed below, and never visits a site without a specific audit request behind it.

Each audit fetches: the homepage, up to 50 additional pages discovered from the site’s sitemap, the 11 AI discovery files Prepared AI checks for (e.g. llms.txt, ai.json), robots.txt, and the sitemap itself.

A full audit takes roughly 60–90 seconds, runs about 8 requests concurrently, and caps each individual page fetch at 2 MB.

The fetched content is used only to generate a scored AI-readiness report for the person who requested it.

Prepared AI does not republish or redistribute crawled page content, and none of it is used to train any AI model.

Every request from PreparedAIBot — whether a plain HTTP fetch or a headless-browser render — sends the exact User-Agent string shown below.

We plan to publish PreparedAIBot's egress IP ranges here as well. Until then, contact us (below) if you need to verify traffic by IP.

If you'd rather PreparedAIBot never crawl your site, add the following to your robots.txt.

Prepared AI's engine checks robots.txt before crawling and honors this directive — a blocked site's report reads “This site blocks PreparedAIBot” instead of a score.

Questions about a specific visit, or a complaint about PreparedAIBot's behavior on your site? Email us and we'll get back to you.