Cloudflare Now Pays Publishers Per AI Citation, Not Crawl
Cloudflare ended pay-per-crawl on 1 July 2026, after exactly one year, and pointed publishers toward deals that pay when content appears inside an AI answer rather than every time a bot fetches a page. A citation just became a revenue event on some platforms, not just a traffic one. Fifty-two percent of all crawler requests are now AI training, up from 22% a year ago, so the volume behind this shift is real and growing fast.
We track citation footprints for a living, so this is the kind of change we read closely the day it lands. Here is what actually shipped, why Cloudflare backed away from its own toll model, and what a paid-citation market means for anyone trying to earn AI visibility right now.
Key takeaways
- Cloudflare retired strict pay-per-crawl and is steering publishers toward pay-per-citation deals instead.
- The reason is data, not ideology: over half of AI crawler visits re-fetch pages that have not changed.
- Ceramic.ai pays per query with reporting on the snippet and position used; You.com pays per piece of content on demand.
- 52% of crawler requests are now AI training, up from 22% a year ago.
- Human traffic is down roughly 40% in the categories AI crawls hardest, while Google still sends about 88% of referral traffic.
- If you only get crawled and never cited, this new market pays you nothing.
What Cloudflare actually changed on 1 July
Pay-per-crawl launched a year earlier as a straightforward toll: an AI crawler fetches your page, you get a micropayment. Cloudflare has now moved away from charging strictly by the fetch. The replacement model pays publishers when their content contributes to an answer an AI system actually shows a user, through partner schemes rather than a single Cloudflare-run meter.
Two early partners show the shape of it. Ceramic.ai runs a pay-per-query model: publishers who opt in get paid when their content appears in a Ceramic search result, with reporting that names the query, the snippet used and the position it held. You.com takes a different route, letting an AI agent pay on demand for one specific piece of premium content at the moment it needs it, rather than a standing licence.
Neither model pays for the crawl itself. Both pay for the moment content gets used inside an answer. That distinction is the whole story.
Why Cloudflare backed away from its own toll
The stated reason is not principle, it is waste. Cloudflare’s own crawler data found that more than half of AI bot visits simply re-fetch pages that have not changed since the previous crawl. Charging a flat fee per fetch was rewarding exactly that kind of redundant traffic, while giving publishers no signal about whether any of it produced value.
Meanwhile the crawl volume behind all of this kept climbing. Fifty-two percent of crawler requests hitting Cloudflare’s network are now attributed to AI training, up from 22% in spring 2025. Bots now account for a large and growing share of all web traffic, and in the categories AI crawls hardest, Cloudflare has recorded human traffic falling by around 40% inside a year. A fee that only counts fetches gets more expensive to administer and less meaningful every quarter, which is a fair explanation for why Cloudflare moved the meter to the output side instead.
What this means if AI only scrapes you and never cites you
Under the old model, a crawler hitting your site was worth something regardless of what happened next. Under a pay-per-citation model, a crawler that fetches your page and never uses it in an answer is worth nothing on that specific scheme. The gap between being scraped and being cited, which we have written about before as the difference between AI traffic your analytics can see and traffic it cannot, now has a second consequence: it is the difference between getting paid and not.
Google still sends roughly 88% of referral traffic even one year into this shift, so nobody should read this as Google being replaced. What it does confirm is that the citation, not the crawl, is becoming the unit that matters commercially as well as visibly.
Crawled versus cited: the new stakes
| Crawled only | Crawled and cited | |
|---|---|---|
| Pay-per-crawl era (pre-2026) | Occasional micropayment per fetch | Same micropayment, no premium for the citation |
| Pay-per-citation era (from Jul 2026) | Nothing, on schemes like Ceramic.ai and You.com | Paid when the answer actually uses the content |
| Visibility | Invisible in AI answers | Quoted, linked, or paraphrased in the answer |
| GEO priority | Fix crawler access first | Defend and extend the placements already earning citations |
On Garden Ornaments, our own e-commerce case study, monthly organic visits grew from 727 to 6,370 in seven months. Part of that came from pages that crawlers already trusted continuing to earn fresh citations without a fight, the exact mechanism Cloudflare is now proposing to price directly. A page that already gets cited is a page with a head start in whichever payment scheme it opts into next.
How to earn citations in a market that now prices them
None of this changes the underlying mechanics of GEO, it raises the stakes on getting them right.
- Check whether your crawler access is even open. Ninety percent of AI crawling only counts if the bot can reach the page in the first place; an over-cautious robots.txt costs you before any payment scheme can help.
- Prioritise the pages already earning citations. Fresh, well-maintained pages that AI already quotes are the ones any pay-per-citation scheme will reward first, so protect and update those before chasing new ones.
- Use links to get onto pages engines already trust. A niche edit on a page an engine already cites inherits that trust faster than waiting for a brand-new page to earn it from zero.
- Track citations, not just crawls. Server logs will tell you who is fetching your site; only a proper citation measurement setup tells you who is actually quoting it.
- Expect more platforms to follow. Cloudflare, Ceramic.ai and You.com will not be the only names in this market by year end, so treat citation-worthy content as an asset with a growing commercial value, not just an SEO exercise.
Common questions
What did Cloudflare change on 1 July 2026?
Cloudflare retired the strict pay-per-crawl toll and pointed publishers toward deals that pay when content actually appears inside an AI answer, not just when a bot fetches the page.
Why did Cloudflare move away from pay-per-crawl?
Cloudflare’s own data showed most crawls do not lead anywhere: over half of AI crawler visits re-fetch pages that have not changed since the last visit, so charging per fetch rewarded waste.
What is Ceramic.ai’s role in this?
Ceramic.ai runs a pay-per-query model. Publishers who opt in get paid when their content shows up in a Ceramic search result, with reporting on the query, the snippet used and its position.
What does You.com do differently?
You.com lets AI agents pay on demand for one specific piece of premium content at the exact moment they need it, rather than a standing subscription or blanket licence.
Does this change how I should approach GEO?
Yes. If appearing inside an answer is now a paid event on some platforms, the citation itself is worth defending and earning deliberately, not a side effect of being crawled.
Should small UK sites care about this now?
Yes, because the direction is set even before every platform pays out. Sites that are already the source AI quotes are positioned to benefit first when more of these schemes launch.
Related reading
- Bots overtook humans: what it means for finding your brand covers the crawl volume behind this shift in more depth.
- Niche edits for GEO explains the fastest route onto pages AI engines already cite.
- How to measure AI citations gives the reporting template for proving a citation, not just a crawl.
Want to know if you are being crawled without ever being cited? Run the free AI visibility check. Ready to get onto the pages AI already trusts? See our niche edits service, backed by a 12-month guarantee, or compare every fixed price on our pricing page.