Cloudflare is moving its AI-bot compensation model from “pay per crawl” to “pay per query,” it said around July 2, 2026 — a shift that tries to price AI access to publisher content by the outcome it produces rather than the act of fetching a page. The infrastructure giant, which sits in front of a large share of the web, is distinguishing between three kinds of bots — search bots, agent bots, and training bots — and charging accordingly. It’s being tested with partners including Ceramic.ai and You.com.
01Why the unit of pricing changed
Cloudflare’s Matthew Conroy framed the logic plainly: “the unit that matters isn’t the crawl or the click. It’s the outcome.” Under a crawl-based model, an AI company pays once to scrape a page and can then reuse that content to answer an unlimited number of user queries — a terrible deal for the publisher, whose work gets monetized again and again for a single toll. Pricing per query attempts to tie payment to usage: every time an AI system draws on your content to answer someone, that’s a metered event.
For a web economy where AI answers increasingly replace the click, it’s a meaningful reframing. Referral traffic — the thing display and subscription models were built on — keeps eroding as assistants answer questions directly. Pay-per-query is an attempt to build a new meter for the value that’s leaking out of the old one.
02Why this matters for publishers
This is a rare piece of good news in the AI-and-publishers story: infrastructure being pointed at recovering value instead of just defending against loss.
| A potential new revenue line | If AI systems must pay each time they use your content to answer a query, scraping stops being a one-time toll and becomes an ongoing, usage-based revenue stream — closer to how the content is actually consumed. |
|---|---|
| Leverage without building it yourself | Most publishers can't negotiate one-on-one with every AI crawler. A platform like Cloudflare, sitting in front of much of the web, can enforce terms at scale that individual publishers never could. |
| It complements licensing, not replaces it | Direct licensing deals (Google, OpenAI, Snowflake) cover named partners. Pay-per-query metering can cover the long tail of bots that never sign a deal — the crawlers quietly taking content today for free. |
For two years the AI-and-content fight has been mostly defensive — block the bots, sue the scrapers, watch the traffic fall.
03What publishers should do
04The bottom line
For two years the AI-and-content fight has been mostly defensive — block the bots, sue the scrapers, watch the traffic fall. Pay-per-query flips the posture: it prices AI access by the outcome and meters ongoing use instead of a one-time crawl. It’s early and partner-limited, but it points at the model publishers actually need — one where every AI answer built on your work registers as a billable event. Watch this closely, and make sure your own infrastructure can meter what the AI is taking.