# Cloudflare Agent Readiness Scan Review *By Joshua Cogar · 2026-08-07 · AI Search* > See why the Cloudflare Agent Readiness scan gave our site a zero, the Content Signals trap that blocks AI citations, and what to fix first. On August 7, 2026, I ran [Cloudflare's Agent Readiness scan](https://isitagentready.com/) against my own agency website. One entire section came back 0 out of 7. The score was accurate. Every single check it failed, my site really does fail. It also told me almost nothing useful about whether my site is ready for AI agents. And if I had followed one of the tool's own suggestions, I would have quietly asked ChatGPT and Perplexity to stop citing me. Here is the full story, what I fixed, what I left alone on purpose, and what you should do to make your own site AI ready. ## What the Cloudflare Agent Readiness Scan Actually Checks The Agent Readiness scan tests whether your website publishes the signals AI agents look for. It checks about a dozen categories, gives each one a pass or fail, and rolls them into a composite score. Categories include crawl permissions, content access, structured discovery files, API documentation, authentication metadata, and browser-side agent tools. It is a real contribution. The space is young, most site owners have no idea these signals exist, and a free scanner that surfaces them is worth having. It found two genuine problems on my site that I fixed the same day. The trouble is what happens next. The scan applies one checklist to every website on the internet. A wealth management firm in Henderson and a payments API company in San Francisco get graded against the same rubric. That is where things go sideways. ## My Setup, For Context seomarketingadvisors.com runs on Astro v6 deployed to Cloudflare Workers. A custom Worker handles form submissions, our scanner APIs, security headers, and 47 legacy WordPress redirects from the old site. That matters because some of what the scan flagged is a side effect of static site generation on Cloudflare, not a decision anyone made. ## The Two Things the Scan Got Right Two failures were real gaps. Both are now fixed. Short answer for anyone skimming: the scan correctly caught missing `Link` response headers and a missing Content Signals policy in robots.txt. Both are legitimate, low-effort additions that help AI systems find and correctly use your content. If your scan flags these, fix them. ### Missing Link Headers [RFC 8288](https://www.rfc-editor.org/rfc/rfc8288.html) defines the `Link` HTTP response header. It lets a server point to related resources before the browser or agent parses a single line of HTML. Astro static builds on Cloudflare do not emit them by default. Mine had none. The fix was a `public/_headers` file pointing at llms.txt, the sitemap, the services page, terms, and privacy. I checked all five URLs returned a 200 before wiring them up, because a `Link` header pointing at a 404 is worse than no header at all. Time to fix: about fifteen minutes. ### Missing Content Signals in robots.txt This one was also a real gap. It is also where the story gets interesting. ## The Content Signals Trap That Could Cost You AI Citations [Content Signals](https://contentsignals.org/) is a directive you add to robots.txt. It does not control whether a bot can fetch your page. It declares what a company may do with your content after they fetch it. There are three separate switches, and they are independent of each other. ### What the Three Signals Mean in Plain English Most people see three settings and assume they are shades of the same thing. They are not. #### `search` Says a company may build a search index from your content and show links and short snippets pointing back to you. This one has a detail almost everyone misses. The `search` signal explicitly excludes AI-generated summaries. Saying yes to search does not say yes to appearing in an AI answer. #### `ai-input` Says a company may use your content inside live AI answers. This covers retrieval-augmented generation, grounding, and the citations you see in ChatGPT, Perplexity, Gemini, and Google AI Overviews. If you want AI tools to quote your business, this is the switch that matters. #### `ai-train` Says a company may use your content to train or fine-tune a model. This is the one publishers argue about, and it is a reasonable thing to say no to if your content library is the product you sell. ### Why the Suggested Example Would Hurt Most Businesses The Cloudflare scan card suggests this policy: ``` Content-Signal: ai-train=no, search=yes, ai-input=no ``` Read it again with the definitions above. That policy is a machine-readable request to stay out of AI answers. I run an agency whose entire offer includes getting clients cited by AI search engines. If I had pasted that example into my robots.txt, I would have published a signed request asking those engines to skip me. The `search=yes` does not rescue you, because search excludes AI summaries by design. Here is what worries me. Most site owners will paste the example. That is what examples are for. A large share of those owners want AI citations and have no idea they just declined them. If you take one thing from this article, take this: do not copy a Content Signals example. Decide each of the three switches on purpose. For most local businesses, service firms, and brands trying to get found, the answer looks closer to `ai-input=yes`. ## The Contradiction I Found Inside My Own robots.txt Fixing this surfaced a mistake I had been sitting on for months. My first draft of the Content Signals line used `ai-train=no`. Then I read the rest of the file. The same robots.txt had `Allow: /` for Google-Extended, Applebot-Extended, and CCBot. Those three user agents exist for one purpose: they are the switches publishers use to decline training use. So my file said "you may not train on this" in one line while holding the door open in three others. Usage restricted, access wide open. The file was arguing with itself. I resolved it by going `ai-train=yes` across the board. That was a deliberate call, not a default. I am not a publisher protecting an archive. Content that shapes future models is distribution for a business like mine, not loss. You might reach a different conclusion, and that is fine. The point is that the audit surfaced a real inconsistency. It only surfaced it because someone read the entire file instead of pasting a snippet into it. ## The Section Where I Scored Zero and Left It That Way Seven checks in a row came back red. Here is every one of them. - OAuth and OIDC discovery at `/.well-known/openid-configuration` and `/.well-known/oauth-authorization-server` - OAuth Protected Resource metadata at `/.well-known/oauth-protected-resource` ([RFC 9728](https://www.rfc-editor.org/rfc/rfc9728.html)) - auth.md agent registration, a vendor proposal from WorkOS rather than an adopted standard - MCP Server Card, currently a draft proposal inside the MCP specification process - Agent Skills index at `/.well-known/agent-skills/index.json`, an RFC Cloudflare proposed themselves - API Catalog ([RFC 9727](https://www.rfc-editor.org/rfc/rfc9727.html)) - DNS for AI Discovery, meaning SVCB and HTTPS records plus DNSSEC, an IETF draft with very thin adoption Every one of those describes infrastructure a brochure site or a service business does not run. Authorization servers. Protected APIs. MCP servers. Published agent skills. Publishing that metadata anyway means advertising endpoints that fail the moment an agent touches them. In the OAuth cases it goes past embarrassing. Declaring an authorization issuer you do not control is a known security-confusion pattern. So the zero was accurate. It was also completely meaningless as a measure of whether my site is ready for AI agents. ## The Copy Prompt Button Is the Bigger Risk Every failing card in the scan includes a button that copies a fix instruction, ready to paste into a coding agent. For the OAuth checks, that instruction amounts to "publish discovery metadata with your issuer and token endpoints." Hand that prompt to a coding agent with no pushback and you know what happens. It will produce plausible-looking JSON with endpoints that sound right and do not exist. It will ship. Nothing will visibly break, because nothing on a marketing site ever calls those endpoints. Your site is now publishing false infrastructure metadata, and nobody will notice for a very long time. That is one click away for anyone who trusts the score. The tool is not doing anything malicious. It is just assuming the person reading has the context to say "wait, I do not run an authorization server." Most people do not have that context. That is exactly why they ran a free scanner. ## The Scan Tests a Browser API That Is Already Moving The WebMCP check probes for `navigator.modelContext`. Chrome [deprecated that location in Chrome 150](https://developer.chrome.com/docs/ai/webmcp/imperative-api). The specification moved the API off `navigator` and onto `document`. Browser support was thin to begin with. Chrome was the only implementation. Firefox and Safari were engaged in the spec process but had not committed. Following the card's instructions today means writing code against an API that is on its way out, for a feature almost no browser supports. This is not a knock on Cloudflare. Specs move fast right now. It is a reminder that a green checkmark is not the same thing as a good decision. ## What the Scan Never Saw Here is the part that reframes the whole score for me. My site already does agent content negotiation at the Worker level. When ClaudeBot, GPTBot, PerplexityBot, and similar agents request a page, they get a clean markdown version of it. Search indexers are deliberately blocked from those markdown twins so the setup is not cloaking. The `Vary` header is set correctly. The HTML is advertised as canonical. The site also ships llms.txt and markdown alternates for all 67 pages. None of that appears anywhere in the score. The scan gave me a red X on a niche RFC with near-zero real-world adoption while being completely blind to substantially more meaningful agent readiness work sitting right there in the response headers. [Search Engine Journal](https://www.searchenginejournal.com/) made a related point about this scoring system, which is that the composite number is the wrong thing to optimize for. That matches what I found. ## A Checklist Built for API Platforms, Applied to Every Website That is the real critique, and it is narrower than "this tool is bad." The Agent Readiness checklist is a reasonable list of things an API platform should publish. Stripe should have OAuth discovery metadata. A SaaS company with an agent-facing product should publish an MCP server card and an API catalog. A plumber should not. A wealth management firm should not. A restaurant in Henderson should not. For those businesses, a low score is a category mismatch, not a failing grade. The scan is measuring them against a rubric written for infrastructure companies, then handing them a button that offers to make their site pretend to be one. ## What to Actually Do to Make Your Site AI Ready Set the score aside for a minute. Here is the work that pays off for a normal business website, in the order I would do it. ### Start With What Every Site Needs **Server-rendered HTML.** If your content only appears after JavaScript runs, most AI crawlers will not see it. This is the single most common failure we find, and it silently erases businesses from AI answers. **An intentional robots.txt.** Name the AI user agents you care about. Decide whether each one gets access. Do not inherit whatever your platform shipped by default. **A deliberate Content Signals policy.** Three switches, three decisions. Most businesses that want to be found should say yes to `ai-input`. **Complete schema markup.** Organization, LocalBusiness, Service, FAQ, and Article where they apply. This is how machines understand what your business is instead of guessing from your text. **Consistent business information everywhere.** Same name, same address, same phone number across your site, Google Business Profile, directories, and social profiles. AI systems cross-check these. Conflicts create doubt, and doubt costs you citations. **A working sitemap and clean canonical URLs.** Boring, still necessary. **An llms.txt file.** A plain text map of your site written for language models. Cheap to produce, and it gives AI systems a clean entry point. ### Add These If You Have the Resources **Markdown alternates of your key pages.** Serve clean markdown to AI agents while humans get the HTML. Set `Vary` correctly and keep search indexers out of the markdown so you are not cloaking. **Link headers.** Fifteen minutes of work on most platforms. Point at your llms.txt and sitemap. **Rate limits that do not block AI crawlers.** Check your WAF rules. Plenty of sites are accidentally blocking GPTBot and ClaudeBot at the edge and have no idea. ### Skip These Unless a Specific Condition Applies **OAuth and OIDC discovery.** Only if you actually run an authorization server. **API Catalog and OpenAPI specs.** Only if you publish an API for outside developers. **MCP server card and agent skills index.** Only if you have built and host those things. **Web Bot Auth.** Only if your business operates a crawler that visits other sites. **WebMCP.** Not yet. The spec is moving and browser support is thin. ## How to Read Your Own Score Three questions, in order. **Does my business actually run the thing this check is asking about?** If you do not have an API, an authorization server, or published agent skills, the checks about them are not gaps. They are questions that do not apply to you. **Does the fix create something real or something decorative?** Adding llms.txt creates a real file with real content. Adding OAuth discovery metadata for an authorization server you do not operate creates a lie. **Am I copying an example or making a decision?** This is the Content Signals lesson. Examples are written for a generic site. Yours is not generic. ## What We Are Building Instead We run our own [AI Readiness Tool](https://seomarketingadvisors.com/ai-readiness-tool/), and this exercise made it better. We are adding the two checks Cloudflare has that we were missing, `Link` headers and Content Signals, because those are genuinely useful for everybody. Our tool gives you a score too. Scores are useful. They give you a starting point, a way to compare, and something to move. What makes a score worth anything is whether the checks behind it apply to you. Ours measure four things a business website can actually act on: Foundation, Discovery, Architecture, and Measurement. Does your content render without JavaScript. Is your robots.txt saying what you think it says. Do you have schema. Are you tracking whether AI tools send you anything at all. No authorization servers. No MCP server cards. No red X for missing infrastructure you were never going to build. The report also tells you what to fix first, because a number without an order of operations just makes people anxious. The useful question is not "how many boxes did you miss." It is "which of these boxes belong to a business like yours, and what should I do Monday morning." ## Frequently Asked Questions ### What is the Cloudflare Agent Readiness scan? It is a free tool at isitagentready.com that checks whether your website publishes the discovery signals AI agents look for, including robots.txt rules, Content Signals, llms.txt, API documentation, and authentication metadata. It returns a pass or fail per category plus a composite score. ### Should I copy the Content Signals example from the scan? No. The suggested example includes `ai-input=no`, which asks AI engines not to use your content in live answers. If you want to be cited by ChatGPT, Perplexity, Gemini, or Google AI Overviews, that setting works against you. Decide each of the three signals on purpose. ### Does search=yes let AI tools cite my content? No. The `search` signal covers indexing and link-plus-snippet display. It specifically excludes AI-generated summaries. The signal that controls AI answers is `ai-input`. ### Is a low Agent Readiness score bad for my business? Not necessarily. Most of the score covers infrastructure that only API platforms and software companies run. If you operate a service business, a local storefront, or a marketing site, a large share of those checks will never apply to you. ### What should a normal business site actually do for AI readiness? Start with server-rendered HTML, an intentional robots.txt with a deliberate Content Signals policy, complete schema markup, consistent business information across the web, a working sitemap, and an llms.txt file. Skip the API and authentication items unless you run those systems. ### Can a coding agent just fix my failing checks for me? Be careful. The scan includes copy-and-paste prompts for coding agents. For the authentication checks, an agent will generate metadata describing endpoints your site does not have. That publishes false information about your infrastructure, and nothing will visibly break to warn you. ## See Where Your Site Actually Stands Run your site through our [free AI Readiness Tool](https://seomarketingadvisors.com/ai-readiness-tool/). It scores four dimensions, 25 points each: - Foundation — HTTPS, robots.txt, sitemap, schema, meta tags, canonical URLs, mobile viewport - Discovery — llms.txt, AI bot rules, FAQ schema, E-E-A-T signals, freshness, headings, internal links - Architecture — server speed, non-blocking JavaScript, image optimization, security headers, compression, API access - Measurement — GA4, Tag Manager, conversion pixels, and whether you are tracking AI traffic and brand mentions at all Takes about 15 seconds. [Scan your site free](https://seomarketingadvisors.com/ai-readiness-tool/) Want a person to look at the results with you? [Book a 30-minute marketing consult](https://api.leadconnectorhq.com/widget/bookings/30-minute-marketing-consult) and we will walk through your score, your robots.txt, and the two or three fixes that will move the needle first. Advisors Marketing is a digital marketing agency in Henderson, NV. We help Las Vegas businesses and national brands get found through SEO, AEO, content marketing, digital ads, website development, and AI integration. --- Canonical page: https://seomarketingadvisors.com/blog/cloudflare-agent-readiness-scan-review/ Site index: https://seomarketingadvisors.com/llms.txt · Agent directory: https://seomarketingadvisors.com/agents.json