When the Stealth Bot Prohibition Act was introduced in late July — we covered it at introduction in our 26 July issue — it arrived with a mechanism but not yet a headline number. This week it got the number, and the number is the story: Politico allocates 25% of its hosting spend to bot management.
A quarter of a major publisher’s hosting budget, spent not on serving readers but on managing the machines that arrive pretending to be readers. That is the cost case for the bill, laid out in AdExchanger’s follow-through analysis on 10 August, and it converts an abstract grievance — AI companies scrape us without permission — into an invoiceable transfer: infrastructure cost moved from AI companies to publishers, quietly, at scale.
The bill itself, introduced by Reps. Valerie Foushee (D-NC), Laurel Lee (R-FL) and Gus Bilirakis (R-FL), is narrower and more practical than the AI-legislation genre usually manages. It would require AI web crawlers to disclose their identity and purpose to website hosts, with $53,000 fines enforceable by the FTC and state attorneys general. Its target is specific: bots that masquerade as human visitors to bypass robots.txt, then scrape and resell content. News/Media Alliance CEO Danielle Coffey gave it the least glamorous endorsement in trade-press memory, calling it “hygiene and vegetables” — unexciting, and structurally necessary.
01Why the 25% figure does the work
Legislation moves on legible harms, and the scraping debate has suffered for lack of them. “Our content is being taken” is a values argument that invites a values rebuttal about the open web and fair use. “A quarter of our hosting budget goes to bot management” is an accounting entry. It names a payer (the publisher), a beneficiary (the AI companies whose crawlers create the load), and a magnitude — and it does so in the language budget-holders, lawmakers and courts all speak.
This is why the Politico figure matters beyond this bill. It is a template every publisher can reproduce from its own invoices. The components are sitting in your existing bills: CDN charges, WAF and bot-management tooling, the bandwidth and compute share your logs attribute to automated traffic. Assembled into a single percentage-of-hosting figure, that number does more work in a licensing negotiation, a trade-body submission or a board deck than any quantity of adjectives about scraping. It also functions before and regardless of passage: lawmakers and enforcers ask affected parties for exactly this evidence, and an AI company negotiating for access finds a documented cost much harder to wave away than a complaint.
Note also what the bill deliberately does not do. It does not ban scraping, mandate licensing, or set prices. It requires identification — turning robots.txt from an honour system into something with consequences for the specific behaviour that makes every other defence fail: crawlers disguising themselves as humans. A publisher cannot block, throttle, meter or charge a bot it cannot identify. Disclosure is the precondition for every other control, which is why a mechanically modest bill is strategically significant.
02Disclosure only helps publishers whose blocking works
The uncomfortable half of the follow-through is on the publisher side. A disclosure mandate makes crawlers visible; it does not make your enforcement functional. For many publishers, robots.txt and the server logs disagree today — crawlers that are formally blocked are still being served content, through renamed user agents, rotated IPs, or simply because nothing at the infrastructure layer enforces what the text file requests. This week’s TollBit data, covered elsewhere in this issue, found European publishers’ no-scrape directives ignored nearly three times as often as North American ones’ — the same gap, measured at panel scale.
If identification becomes law, the publishers who benefit are the ones whose infrastructure can act on identity: block at the WAF, throttle by ASN, meter access per crawler, price what gets through. Publishers who never closed the enforcement gap will have gained a right they cannot exercise. The engineering work is unglamorous — exactly Coffey’s vegetables — but it is the difference between a legal entitlement and an operational capability.
"Hygiene and vegetables" is exactly right, and it is meant as praise.
03Why this matters for publishers
| Bot cost is a real budget line you are already paying | Politico's 25% is one publisher's number, but every publisher carries a version of it — usually unmeasured, scattered across CDN, security and infrastructure invoices. Unmeasured costs cannot be recovered, negotiated or legislated about. |
|---|---|
| The bill targets the failure mode that defeats all your other defences | Crawler policies, licensing schemes and pay-per-crawl systems all presume you know who is knocking. Bots that impersonate humans break that presumption, and this is the first federal instrument aimed squarely at them. |
| Enforcement routes through the FTC and state AGs, not private suits | At $53,000 per violation with public enforcers, the compliance pressure lands on crawler operators without publishers having to litigate — but public enforcers will act on documented evidence from affected parties, which is another reason to have yours ready. |
| Bipartisan sponsorship makes this the likeliest vehicle in the space | One Democrat and two Republicans on a narrow, cost-anchored bill is a materially better legislative bet than the sweeping AI frameworks that stall — worth tracking as the one that might actually move. |
04What publishers should do
05The bottom line
“Hygiene and vegetables” is exactly right, and it is meant as praise. The Stealth Bot Prohibition Act will not resolve the AI-content wars, set the price of training data, or save anyone’s referral traffic. What it does is repair the layer everything else depends on: you cannot govern, license or charge crawlers that lie about being crawlers. The cost case is now on the table, and the template is reproducible from your own invoices. Produce your number, fix your enforcement, and file the evidence. Eat the vegetables.