Google, Cloudflare, and Microsoft are testing ways to compensate websites for AI access to or use of their content. Each company has its own set of rules about what triggers payment, who defines that unit, what control content owners have over terms, and what information comes back. Those differences decide what a site owner can set, and what they can only see, once they’ve said yes.
What Each Company Pays For
Google’s pilot program pays when a site’s content “contributes significantly” to an AI-generated response across Gemini, AI Overviews, and AI Mode, as reported by Digiday. Confirmation or linking after the answer is generated doesn’t count. Google told Digiday that this is an initial learning phase, referencing a June 18 post about testing a new model for websites whose content helps ground its generative AI answers.
Cloudflare’s Pay Per Crawl, which has been in private beta since 2025, charges a crawler for each successful retrieval. The site owner sets a per-crawl price, with a $0.001 minimum, and a crawler that agrees to pay receives the page with an HTTP 200 response. Earlier this year, the company said it was shaping that product into Pay Per Use, where AI companies bring their own payment models. Meanwhile, Ceramic.ai pays participating sites when their content appears in its search results, and You.com pays on demand for a specific piece of premium content.
Microsoft’s Publisher Content Marketplace, announced Feb. 3, provides licensing for premium content to support Copilot responses, the data the AI uses to generate replies. Microsoft stated that publishers will be paid on delivered value, define licensing and usage terms, and receive usage-based reporting. Copilot is the first demand partner, and Yahoo was onboarding at announcement.
How Each AI Content Payment Model Works
Each system ties compensation to a different kind of event.
Google AI contribution pilot
Invitation-only pilot
Significant contribution
Defined by Google
Opt in / opt out
Other terms not public
Monthly earnings
Some history; no calculation detail
Cloudflare Pay Per Crawl
Closed beta
Paid retrieval
Site-set price
Set crawl price
Charge, allow, or block crawlers
Charge activity
Balance not shown in dashboard
Cloudflare Pay Per Use
Experiment
Partner-defined use
Varies by partner
Join partner program
Who sets rates isn’t public
Partner-specific reporting
Ceramic: query, page, snippet, position
Microsoft Publisher Content Marketplace
Pilot
Not publicly specified
Microsoft cites ‘delivered value’
Licensing + usage terms
Publisher-defined
Usage-based reporting
Report detail not public
“Not publicly specified” refers to public documents and does not mean participants lack the information.
From The Click Deal To Paid Use
In December, I examined payment models in play such as revenue sharing with Perplexity and ProRata, flat licensing deals like OpenAI’s agreement with News Corp, and court settlements. At that point, Google hadn’t announced anything beyond existing traffic arrangements. Nine months later, it has launched a pilot, Cloudflare has started testing beyond the per-fetch model I covered when it launched in July 2025, and Microsoft has established a marketplace.
Google’s June update describes the pilot as an addition to programs already paying for content. They state that their News AI pilot involves over 200 publications, the European News Partnerships program pays more than 5,500 European publications, and News Showcase partners with more than 2,800 outlets across 33 countries. These are separate commercial or payment programs, with earnings from the pilot displayed in Search Console, according to Digiday.
Cloudflare offers a different viewpoint. Their July 1 post notes that crawling turned out to be a crude measure of value, because a page can be crawled once and cited in thousands of answers, or crawled over and over and never used. Cloudflare emphasizes that outcomes matter, describes Pay Per Use as an experiment, and says it has a lot to learn about this model’s performance across the internet.
Who Decides What Counts
Under Google’s pilot, Google makes the call. Digiday’s sources mention that payments are based on the value of content rather than just the amount of usage, with the company judging whether a piece of content meaningfully contributed. “It’s quite black box,” one exec with knowledge of the program told Digiday. Google hasn’t publicly stated the criteria.
Under Pay Per Crawl, the event is mechanical, and Cloudflare’s docs define it as an HTTP 200 response to a crawler that accepted the price. Under Pay Per Use, the AI company defines the event. Cloudflare’s post describes Ceramic’s model as pay-per-query and You.com’s as on-demand access, and mentions that other providers are experimenting with models it calls Pay per Query and Pay per Result.
Microsoft’s post doesn’t specify the payable unit. Instead, it explains that publishers define licensing and usage terms and that AI builders license content for particular grounding scenarios. The post includes pricing as one of the topics co-designed with publishers, but it doesn’t specify who determines it.
What The Site Owner Controls
Google’s pilot is by invitation, and participants can opt out at any time from Search Console, per Digiday. Beyond the choice to join, nothing published gives a participant a say over terms. One exec told Digiday the numbers offered to their company and several peers were too low to opt in, and that talks with Google were ongoing, so some terms are being discussed. Digiday’s sources didn’t say what.
In Pay Per Crawl, the site owner sets the price per domain, can vary it by path with dynamic pricing, and chooses per crawler whether to charge, allow, or block. Meanwhile, in Pay Per Use, the owner joins a partner’s program, with payment triggered differently depending on the partner, and Cloudflare’s post doesn’t establish who sets the rate.
Microsoft’s post explains that joining is voluntary and publishers keep ownership of their content and keep their editorial independence. However, it doesn’t specify what the licensing and usage terms might involve.
What Comes Back As Data
Google’s participants can view an AI contribution panel in Search Console, displaying a monthly earnings figure and some history. Digiday notes that no additional details on how payouts are calculated are provided. The company also conducts weekly calls with some partners. One participant mentioned to Digiday that they would like a feedback system like Search Console for search, but for AI inference. David Buttle from the publisher coalition Spur said the first step is setting the precedent that publishers deserve to know when their content informed a response.
Cloudflare records a charge event on each successful paid HTTP 200 retrieval, and its dashboard shows crawler activity, including which crawlers accessed content and how often charges occurred. The same page mentions that the earned balance isn’t displayed on the dashboard and needs to be requested from the Cloudflare team. For Pay Per Use, the July post explains that sites participating in the Ceramic program receive reports on the top queries that surfaced their content, including the specific page, snippet, and average position in search results.
Microsoft’s post promises usage-based reporting so publishers can see how content was valued and where it could provide more value. The post doesn’t say what the reports contain, and I found no participant describing them.
Google’s own reporting on AI surfaces exists apart from the pilot. The generative AI performance report rolled out worldwide on Aug. 31, and one Digiday source said the impressions it showed closely tracked traffic lost from traditional search and Discover. Neither the help doc nor Digiday’s reporting connects that report to pilot payments.
The Googlebot Question
Cloudflare’s per-fetch unit can be pointed at any crawler in its directory, including search engine crawlers. Its docs say setting those crawlers to Block or Charge may hurt SEO performance because search engines may not be able to index the content. Google’s crawler docs say it doesn’t use content from URLs returning 4xx status codes, and that indexed URLs returning any 4xx other than 429 are removed from the index over time. A 402 Payment Required response is a 4xx.
So a site can meter Googlebot, and doing so creates Search visibility risk. Google’s contribution pilot pays on Google’s unit. The recent change by Cloudflare on Tuesday doesn’t change this situation. Its Disallow AI Training setting publishes a no-training preference in robots.txt that Google and Apple honor while their crawlers keep crawling for search, and its Block option stops Googlebot, Applebot, and Bingbot entirely. Regarding Bing, currently, this setting doesn’t indicate a domain-wide no-training preference on its own, and Cloudflare says Microsoft is building robots.txt support targeted for early 2027. Cloudflare reports that less than 1% of its sites block search bots, while 17% use some form of training block.
The same post relays that Google expects to launch URL-level transparency tools related to Google-Extended in the next few weeks. Cloudflare’s post mentions these tools briefly, and they are distinct from the pilot’s contribution reporting.
Why This Matters
For a site with a pilot invitation from Google, the documents settle the event, the report, and the exit. Google judges the contribution, the report is a monthly figure, and participants opt out in Search Console. However, the published record doesn’t define what “significantly” means or clarify if the Search generative AI control impacts eligibility. The panel, as Digiday describes it, doesn’t show either.
For a site behind Cloudflare, two units exist, and this week’s update changed the crawler controls and how existing settings migrate. Block now stops Googlebot for search as well as training, so for Googlebot, a site that wants training refused and search kept needs the Training control set to Disallow AI Training, not Block. Pay Per Crawl prices the fetch on the owner’s terms, with the search-crawler caution in the docs and the balance limitation on earnings. Pay Per Use triggers vary by partner, and the reporting Cloudflare describes is Ceramic’s.
For a site weighing Microsoft’s marketplace, the primary source establishes publisher-defined terms and usage reporting, and it names the co-design publishers. It doesn’t say who sets the price or what the reports show. Those are questions for Microsoft or for a participant, and the post’s intake form is the only public route in.
Looking Ahead
Cloudflare plans to make its search-signals program broadly available later this year. Additionally, according to a post on September 15, it aims to give sites the ability to decide how much of their content appears in AI summaries by early next year. Meanwhile, Cloudflare mentions that Google expects to introduce URL-level Google-Extended tools within a few weeks. However, Google gave Digiday no expansion date for the pilot and describes it as still being in the early stages.
Ceramic provides query, page, snippet, and position reporting through Cloudflare’s Pay Per Use service. Microsoft promises usage-based reporting, whereas Google displays a monthly total. More detail would offer something closer to the Search Console-style feedback, which one site owner told Digiday they wanted for AI inference.
More resources:
- Google Tests Paying Publishers For AI Answers Via Search Console
- The Click Economy Is Over: How AI Search Is Forcing Publishers To Rethink Revenue
- US Publishers Demand Common Crawl Stop Scraping Their Content
Featured Image: Tetiana Yurchenko/Shutterstock
