Verified agents

How identification works#

Shield verifies agent identity two ways, in order of preference:

Web Bot Auth (HTTP message signatures)#

The emerging IETF standard (RFC 9421 signatures with the web-bot-auth tag) backed by OpenAI, Google, Amazon, and Cloudflare. The agent signs each request with an Ed25519 key and points to its published key directory via the Signature-Agent header. Shield fetches the directory, checks the signature, and confirms exactly which agent is calling. This identity can’t be spoofed by copying a user-agent string.

GET /products HTTP/1.1
Host: example.com
Signature-Agent: "https://agent.bot.goog"
Signature-Input: sig1=("@authority" "signature-agent");created=1753100000;\
  keyid="poqkLGiy...";alg="ed25519";expires=1753100300;nonce="e8N7...";tag="web-bot-auth"
Signature: sig1=:jdq0SqOwHdyHr9+r5jw3iYZH6aNGKijYp/EstF4RQTQdi5N5YYKrD+mCT1HA1nZDsi6nJKuHxUi/5Syp3rLWBA==:

Confirmed crawlers (reverse DNS)#

Established crawlers that predate Web Bot Auth, such as Googlebot, Bingbot and Applebot, are confirmed the way their operators document: a reverse-DNS lookup of the calling IP, forward-confirmed back to the same address. A client merely claiming to be Googlebot from an unrelated IP gets no credit, and its user-agent lie counts against its humanity score.

Verification happens automatically on every /challenge and /solve request an agent makes through the widget flow. A verified identity is bound into the response token and surfaced in the siteverify response as the agent field.

Per-widget agent policy#

Each widget chooses how verified agents are treated, in the widget’s settings:

Routing on identity#

The score answers “is this a human?”; the agent field answers “then who is it?”. Use both:

const result = await siteverify(token)

if (!result.success) reject()
else if (result.agent?.verified) {
  // Declared automation: route it, don't fight it
  routeToApi(result.agent.domain)
} else if (result.score >= 40) {
  allow() // human traffic
} else {
  block() // automation that refused to identify itself
}

Server-side verification: agentverify#

Agents don’t only arrive through forms. When a signed request hits your origin directly (an API endpoint or a content page), forward its signature headers to agentverify and Shield does the verification for you. No widget, no JavaScript, no challenge.

POST https://shield.edge.network/agentverify
Content-Type: application/json

{
  "secret": "es_secret_...",
  "authority": "example.com",
  "headers": {
    "signature-agent": "...",
    "signature-input": "...",
    "signature": "..."
  }
}

Response:

{
  "success": true,
  "agent": { "domain": "agent.bot.goog", "verified": true, "method": "web-bot-auth" },
  "policy": "allow",
  "error-codes": []
}

authority is the Host the agent addressed (signatures cover it, so it must match), and headers are the three signature headers verbatim from the incoming request. Failures return success: false with missing-signature-headers or signature-verification-failed.

What Shield stores#

Nothing new. Agent verification is stateless. Key directories are cached briefly in memory, and the only persisted artefact is the hourly verified agent counter in your widget’s analytics. No per-request logs, no agent request history.