Cloudflare's Trust Framework for AI Agents
Cloudflare launched a coalition to develop standards for verifying and managing AI agents as they become a dominant source of web traffic.
Decoder
- Intermediary: A proxy, agent, or cloud browser that sits between a user and a website, obscuring the user's original identity or device context.
- Web Bot Auth (WBA): An emerging standard allowing agents to cryptographically sign their requests to identify the operator.
Original article
AI Agents now act on the web for millions of people. To a Site, an Agent can take many forms. It may be indistinguishable from a human user carrying out a task, or it might look like one visitor carrying many Users. This removes the ability for the Site to recognize, serve, and hold accountable the user or users behind the Agent, and causes the Site to face the costs when something goes wrong. This post argues that in order for Sites to maximize the access they provide to Agents, the ecosystem must develop new mechanisms to preserve the relationship between Sites and Users. Traditionally, in Security and Fraud Prevention, that requires three things: Identity, Reputation, and Accountability, allowing Sites to answer: Is the person acting being truthful about who they are? Is that person trustworthy? And who answers for the action?
“Personal AI Agents” (“Agents” for short) are automations that take instructions from a human (“User”) and pursue a goal by controlling a cloud computer or browser. They are usually hosted and maintained by an “Operator” as part of a managed service or consumer offering. Examples include Muse from Meta and dots from OpenAI, and more are launching all the time. Agents can potentially take any action online that their User could, without the limits that hold humans back. They don't tire, they run around the clock, they act in parallel, and they put up with friction no person would. Much of the web, from how Sites plan capacity to how they make money, assumes those limits.
Fundamentally, remotely hosted Agents are intermediaries. An intermediary is anything that sits between a User and a Site, so the Site sees it instead of the User. Agents are the newest kind of intermediary, but intermediaries have existed for a long time. Cloud browsers (hosted browsers that software controls on a User's behalf) and Proxies (services that relay a User's traffic, including VPNs) are intermediaries too. What has changed is the trajectory. Agentic traffic is growing rapidly and may eclipse traditional human browsing on the Internet in the near future. This forces a reckoning for anyone operating a Site today. Historically, a Site could simply block intermediary traffic and require Users to disable the intermediary service. That stops working when the majority of traffic is agentic. Once blocking is no longer viable, Site Operators are left with the problem that intermediaries have always created. The Site loses direct information about the User and must decide whether to trust the intermediary without it.
This post covers Agents hosted somewhere other than the User's device. Local agents (e.g. OpenClaw) or Agents running in the User's own browser, on their own device, are out of scope of the primary topic addressed by this post.
This post rests on one principle:
The Intermediation Principle: The business relationship between Users and Sites, and each participant’s security responsibilities, must be upheld regardless of intermediation.
A Site needs new tools to recognize, serve, and hold accountable a User acting through an Agent just as it could if that User had visited directly. The Agent, and the Operator behind it, must be recognizable and accountable too, because an Agent can do what no User could. Agents are able to scale human access almost without limit, so they create new risks, even for behavior that was previously benign.
The intermediary problem
Meet Ezra. Ezra works as a personal assistant for people in her city. Her best client is Amy, a local CEO, but she also works for Bob, Charlie, and Debra. She runs their errands: groceries, appointments, paperwork, and flights. Ezra blends into the crowd.
The corner grocer used to know Amy. He knew what she bought and set aside what he thought she'd like. Now only Ezra comes in, carrying orders for four people, and the grocer can't tell whose preferences are whose. He doesn't want to turn Ezra away, because she brings him a lot of business.
Then Bob asks Ezra to buy him an iPhone. A week later he calls the store, says he never authorized the charge, and keeps the phone. To win the dispute, the store has to prove Bob asked for it. But the only person the store ever saw was Ezra.
Then Charlie gives Ezra a list of passwords: "I forgot mine, but it's one of these." Ezra patiently tries each one at Charlie's bank. The bank can see an attack happening, but the only person it can ban is Ezra. Banning Ezra also bans Amy, Bob, and Debra.
None of these business owners has a problem with Ezra herself. Their problem is that they can only see Ezra, not the person she's acting for. Swap Ezra for an AI Agent and the businesses for websites, and you have the web today.
A website owner wants sufficient visibility both to prevent bad behavior and to effectively monetize or get value from their Site. But as with Ezra, much of that visibility goes away with AI Agents. Below is a list of actions that a website owner might want to address that were previously solved, but are made more challenging by allowing Agents.
| Challenge | Website owner’s goal |
|---|---|
| Account sharing | Sites need to ensure that one paid subscription is not being abused to serve many Users. The Site sees one account and one client when accessed via an Agent. |
| Compliance and illegal activity | A Site must meet legal requirements to verify information about its Users. For example, some Sites have age or geographic restrictions. Agents do not provide guarantees that can meet the bar for Sites to remain compliant, if they allow Agent access. |
| Friendly fraud | Friendly fraud is when the User approves an Agent's purchase, then disputes it as unauthorized. Sites may win with circumstantial evidence: the same device, IP, and address as the User's past undisputed purchases. Agent purchases come from the Agent's infrastructure, so that evidence doesn't exist. Even when the Site wins, the dispute itself costs money. Sites need to maintain records of checkout behavior for fraud prevention purposes. |
| Human verification is irrelevant | A Site wants to be able to distinguish the type of Users it has. CAPTCHAs and other human verification check whether the entity controlling the browser is a human or not. Agents are never expected to be humans. |
| Identity spoofing | A Site wants to limit access to content to human Users or impose restrictions on what particular Users can access. Agents often run on residential IPs and use real Browsers, are mixed with human-directed traffic, and take efforts to not distinguish themselves from human traffic. |
| Identity verification breaks | A Site may require additional verification of the User’s identity, or other information, in order to provide access to sensitive operations or content. Liveness checks, KYC ID checks, and similar assume the person present is the person being verified. The Agent is never that human. |
| Misalignment | Sites may want to protect themselves from erroneous actions taken by newer or more experimental Agents. Agents make mistakes: the wrong item, the wrong date, duplicate orders. |
| Policy circumvention | The Site is looking to impose a policy (e.g. block, allow only partners, rate limit) that is based on identity, but Agents don't always identify themselves, and they often don't bring information about the User they act for. |
| Resource exhaustion | Sites that provide free services or that rely on User engagement were often designed with an assumed limit on User behavior. Agents poll constantly (e.g. stock and price checks), hit expensive endpoints, and use up free trials and promotions. |
| Risk-based authentication breaks | A Site wants to maintain its risk-based authentication system, which expects the device, location, and behavior to match the User. Agents run on cloud infrastructure and look unusual every time. |
| Traffic commingling | Sites would like to protect themselves and User accounts from abuse or fraudulent activity. When one User abuses a shared Agent for credential stuffing, scraping, or card testing, the Site can detect the abuse but can only block the Agent, which also blocks all of its legitimate Users. |
| Unauthorized card users | Sites and payment providers need to protect cardholders from unauthorized transactions, while minimizing dispute costs and other costs related to fraud. |
| Uncompensated consumption | Sites that monetize content require visibility and control over who accesses that content in order to apply the correct monetization model. The Agent reads or extracts content to answer the User somewhere else. |
| User attribution | Sites offer additional capabilities to consenting Users regarding personalization, recommendations, and loyalty programs, using analytics tied to the User's session and device. |
In nearly every challenge above, the Site bears the primary cost. That outcome follows from how trust works when an intermediary is involved.
Our vision
Our aim is an Internet where all Users have equal or greater access to websites than they had before. Before Agents, a Site was typically able to mitigate these risks with a collection of techniques, including fingerprinting the User’s device, session, and history. Users could still browse using private browsing mode, and we believe Agents should be able to do that as well. But they probably won’t be able to log in to a bank without additional verification.
None of this requires an Agent to act like a human or a human to be present for every action. It requires that Sites have the means to obtain or require information about the User to hold them accountable, when applicable, with or without an Agent.
What people have tried, and why they fall short
A common proposal is for Agents to pass a Site's CAPTCHA through to the User to solve. That proves some human is present at that moment (“human in the loop”, or HITL), and nothing more. HITL is primarily a mechanism to prevent mistakes from misalignment. Misalignment is an issue between the Operator and the User, and resolving it does not resolve the majority of the problems above.
Another common proposal is to use tightly scoped access controls. Agents are built to do anything a User could, so in practice they ask for, and get, permission to do everything. A scope of "everything" tells the Site nothing.
What the web needs
The web needs tools and protocols that allow Sites and Agents to collaborate to determine what behaviors and actions are allowed, as well as express preferences and policies from Sites to Agents, all in a way that can be reasonably supported by all Sites and Agents.
Protocols
Agents and Sites can engage in explicit protocols for solving some of the problems above. For instance, MOLE is one upcoming approach that aims for rate limiting without User-specific identifiers, allowing Sites to transition away from implicit identities and towards explicit protocols for accomplishing the same goal. This is similar to the x402 protocol, which aims to enable payments on the web while remaining privacy-preserving.
Identities
When discussing Agent Identity, Web Bot Auth (WBA) is our preferred method. WBA allows an Agent to sign its requests, which provides a secure way for an Agent to identify itself (and, implicitly, the Operator).
In many cases, identifying information isn’t needed, and privacy-preserving methods for moderating access to agents can help us answer “can we trust you?” without needing to answer “who are you?”.
- Nothing - Users may act on the internet while bringing no identifiers to the table.
- Session - Sites require some way of linking requests in the same session to a single agent.
- Account - Sites require a way of attributing requests to the same User across sessions.
- Unique - Some Sites might require a unique, per-User identifier provisioned by a third party that persists across Operators, accounts, Agents, and sessions.
Claims
Beyond identities, some actions require a fact about the real person, such as being over 18 or being the cardholder. Often the Operator verified this at sign-up or a third party already has this information.
Site Agent access policy
Today, an Agent infers a Site’s rules by trying something and interpreting the Site response. Sites need a machine-readable way to publish what actions Agents are allowed to do.
On open source and local Agents
Agents that are not run as an “Intermediary” (e.g. on-device Agents) raise different concerns. Primarily, they look like (and are) automation, and they have no single party to build reputation or hold accountability with.
What’s next
Real-world solutions to the problems that Sites face won’t emerge from fragmented, vendor-specific mechanisms. They require open, interoperable standards that are rooted in real world experience. We are also announcing the Trusted Agent Delegation (TAD) coalition. The goal of TAD is to bring a group of representatives across the ecosystem to have discussions on the overall problem set that was put forth in this document.
What you can do today
See which Agents are on your site, and what they're doing. Security Analytics now breaks out Personal Assistants as their own traffic category, and a new widget lists every Agent visiting your site.
Set access on your terms, per Agent. Using AI Crawler and Agent access policies, you can welcome the Agents that bring you business and restrict the ones that don't.
Tie an Agent back to the user it acts for — and act on it. For sites using Account Abuse Protection, every login and sign-up now shows whether a personal Agent made the request, and which one.
Fresh dev news, AI-digested daily. Never miss a trend at devoured.fyi