Standards & regulation

Congress Goes After Disguised AI Crawlers — The Stealth Bot Prohibition Act

APH Regulation Desk ·3 min read Share Print
In this piece
    Figure Meta's training crawler grew 74% quarter over quarter; its retrieval-augmented-generation crawler grew 163%
    META'S TRAINING CRAWLER GROWTHMeta's training crawler grew 74% quarter over quarter; its retrieval-augmented-generation crawler grew 163%. Indexed so the before value is 100; after is 174.+74%META'S TRAINING CRAWLER GROWTHINDEXED · BEFORE = 100100174BEFOREAFTER
    Regulation Desk

    On July 23, 2026, Representatives Laurel Lee (FL-15) and Valerie Foushee (NC-4) introduced the Stealth Bot Prohibition Act, a bipartisan bill requiring AI-powered web crawlers to disclose their identity and purpose when they access a website. Bots that conceal what they are would face enforcement — with authority given to the Federal Trade Commission.

    It is the first serious federal attempt to make the crawler layer legible, and it addresses the exact failure publishers have been complaining about for two years: every technical control publishers have — robots.txt, user-agent blocking, rate limiting, licensing terms — depends entirely on the crawler telling the truth about who it is.

    The numbers in this piece

    50%of traffic on some publisher sites
    57.4%of web requests
    45%AI agent traffic growth
    12.2Balone

    01The scale of the problem

    The News/Media Alliance estimates deceptive bots make up more than 50% of traffic on some publisher sites. CEO Danielle Coffey called the bill a “sorely-needed, common-sense solution.”

    Independent measurement supports the order of magnitude. Per DataDome data cited by Digiday this week, automated systems now generate 57.4% of web requests versus 42.6% from humans. AI agent traffic grew 45% in Q2 2026 alone, to 17.7 billion requests, up from 12.2 billion in Q1. Meta’s training crawler grew 74% quarter over quarter; its retrieval-augmented-generation crawler grew 163%.

    And the enforcement gap is already measurable: 56.4% of news publishers block at least one AI crawler in robots.txt, and 50.5% specifically ban GPTBot — yet 39.5% of sites that block GPTBot still serve it content anyway. That is not a policy failure. That is publishers writing rules that the other side is not obliged to obey.

    02Why this matters

    Disclosure is the precondition for every other remedyLicensing, pay-per-crawl, telemetry standards and rate cards all assume you can tell which agent took your content. Without mandated identification, every one of those mechanisms is voluntary on the crawler's side. This bill attacks the foundation rather than the symptom.
    It's also an audience-measurement billIf more than half of requests are automated, every traffic number you report — to buyers, to boards, to investors — is contaminated by an unknown amount of bot volume. Coffey named this explicitly: filtering fraudulent traffic enables accurate audience measurement. Cleaner denominators change reported CPMs, viewability and engagement.
    FTC enforcement means deception, not copyright, is the hookThat is a meaningfully easier legal path than the training-data lawsuits grinding through the courts. Misrepresenting your identity to gain access is well-trodden FTC territory. It doesn't resolve whether training is fair use — it just makes hiding illegal.
    Publishers have spent two years building fences around content while the traffic taking it refused to say its name.

    03What publishers should do

    Figure 2 The News/Media Alliance estimates deceptive bots make up more than 50% of traffic on some publisher sites
    50%of traffic on some publisher sites
    Regulation Desk

    04What marketers should do

    05The bottom line

    Publishers have spent two years building fences around content while the traffic taking it refused to say its name. A disclosure mandate with FTC teeth would not settle the licensing fight, or the copyright fight, or the traffic collapse. It would do something narrower and more useful: make the fight legible. You cannot negotiate with, bill, or block a visitor you cannot identify — and right now more than half of your visitors are exactly that.

    Sources & caveats

    Sources: MediaPost, “Bipartisan Bill Targets AI Stealth Bots” (July 24, 2026), on the Stealth Bot Prohibition Act introduced July 23, 2026 by Reps. Laurel Lee and Valerie Foushee; Digiday, “In Graphic Detail: AI visibility is no longer about referral traffic” (July 21, 2026), citing DataDome, Similarweb and related sources. The News/Media Alliance’s “>50% of traffic on some sites” is an advocacy estimate from an interested party. The bill has been introduced only — it has not passed committee, either chamber, or been signed. Bot-traffic percentages are vendor-network data and vary substantially by site type.

    The weekly

    One letter a week, from the desk that runs the auctions.

    What actually moved in yield, CTV and curation across our publishers — written by the people who saw it, not a content team. No digests, no roundups, one email.

    One email a week. Unsubscribe in one click. We never share or sell the list.

    More from this issue

    Ran alongside this piece in the Weekly of 26 July 2026 — read the whole issue →