Cloudflare launches Pay Per Crawl for AI bots
Cloudflare’s AI Crawl Control now lets site owners price crawler access instead of only blocking bots. Requests can return HTTP 402 with machine-readable price headers, while Cloudflare handles billing as merchant of record.
This is a clever attempt to turn AI scraping from a binary permission problem into a negotiated market. It’s not a universal fix, but it could become the default leverage point for publishers already sitting behind Cloudflare.
- –`crawler-price`, `crawler-exact-price`, and `crawler-max-price` create a real request/response handshake instead of a vague paywall.
- –Cloudflare acting as merchant of record removes a lot of operational friction for publishers who would never build billing themselves.
- –Adoption still depends on crawler operators supporting signed requests via Web Bot Auth, so the ecosystem has to cooperate.
- –The feature matters most where Cloudflare already fronts the site; outside that footprint, big crawlers can keep going around it.
DISCOVERED
62d ago
2026-03-28
PUBLISHED
62d ago
2026-03-28
RELEVANCE
AUTHOR
DIY Smart Code
