Cloudflare is rolling out new tools aimed at a customer base most site owners aren’t tracking yet: AI agents. The company says fewer than half of all HTML page requests today come from a human, and with AI assistants increasingly answering questions, comparing options, and even acting on a customer’s behalf, the moment that decides whether a business gets chosen often happens inside a model’s response rather than on a results page. To help with this shift, Cloudflare has folded its existing Agent Readiness work into the dashboard and introduced a new Answer Engine Optimization tool alongside it.
The push stems from conversations Cloudflare had with site owners who could see AI bots crawling their access logs but had no way to know whether those bots could actually use their site or were recommending it to users. Two questions kept coming up: can agents use my site, and am I getting recommended. As such, the new tools are built to answer both.
With Diagnostics, it scans a site the way an agent would, checking whether it’s allowed in, whether its content is discoverable, whether a clean machine-readable version exists, and what interfaces are available for an agent to call. Rather than just loading a homepage like a person would, an agent relies on things like robots.txt, sitemaps, response headers, a Markdown version of the content, and published metadata for authentication and tools. The feature rolls all of that into a single readiness score ranging from Not Ready to fully agent-native, with each check marked pass, fail, or neutral, an explanation of why it matters, and an evidence trail showing the exact request and response observed.
Checks are grouped by effort level, starting with quick wins like a crawler-readable robots.txt, an XML sitemap, AI-crawler rules, and Markdown support, moving into technical groundwork such as content-use signals, an API catalog, and agent login instructions, then advanced integration features like OAuth discovery, MCP and A2A agent cards, and Web Bot Auth. A separate commerce category tracks emerging agent-payment standards like x402 and the Agent Commerce Protocol, though this is informational only for now and doesn’t count toward the score. Every flagged issue comes with a suggested fix, either a direct link to the relevant Cloudflare setting or a prompt a coding agent can use to make the change.
Meanwhile, the Answer Engine Optimization (AEO) addresses the “recommendation” part. Since there’s no equivalent to a search ranking or impression count for this, Cloudflare infers a site’s industry and category, then probes assistants like Claude and GPT with prompts mimicking real customer discovery, covering recommendations, comparisons, and general advice. This produces metrics including Citation Rate, the share of category answers citing the site as a source, Prominence, which measures how much of an answer is attributed to the site and how early it appears, Mention Rate, tracking how often a brand is named regardless of citation, and Share of Voice, showing how a site’s citations stack up against competitors on the same prompts.
To build this out, Cloudflare runs a benchmark panel per industry and category rather than re-querying models for every individual scan, which keeps results loading quickly, cuts down on redundant AI calls, and enables an Industry Fit score showing whether an assistant actually groups a site with its real competitors. Because AI assistants don’t always answer the same way twice, Cloudflare queries each assistant multiple times through its AI Gateway and uses Workers AI to read and score the resulting answers, evaluating not just whether a brand was mentioned but whether it was cited, how early, and how much of the answer’s substance came from it.
Alongside these answer-based metrics, an AI Operator Activity view shows actual crawl and referral traffic broken down by operator, including who’s reading content, who’s sending visitors back, and where crawlers are hitting errors like blocked or dead pages. Cloudflare flags one pattern in particular as worth watching: operators that crawl thousands of pages but send back no referral traffic at all.
Cloudflare is positioning these tools as a way to move past guesswork, whether that meant combing through logs or manually prompting a chatbot to see if it mentioned a brand, since data passing directly through Cloudflare’s network lets these tools measure rather than estimate. Site owners can access both features through the Overview tab in their dashboard, with Agent Readiness available now and AEO Visibility open for early access.
