Bot and AI visitor identity,
intent, and network truth
Find out what's really behind the requests. One API call, any stack.
Your customers expect you to know which bots are real
Any client can claim to be GPTBot in one line of code, so telling real crawlers and agents from imposters usually means maintaining IP ranges, DNS lookups and header parsing yourself. Replace that work with one verified verdict per request that goes straight into your product.
Send one request, get the whole picture back
Send a request's headers and connection details and get back the bot's identity, purpose and verification status, with the evidence behind each field.
GET /pricing HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.1; +https://openai.com/gptbot
X-Forwarded-For: 203.0.113.42
Sec-CH-UA-Platform: "Windows"
Accept: text/html
Accept-Language: en-US
# connection
remote_addr: 198.51.100.7
tcp_rtt_ms: 4
tls_rtt_ms: 118
{
"identity": {
"name": "GPTBot",
"operator": "OpenAI",
"category": "ai_crawler",
"purpose": "model_training",
"acting_for": "its_owner"
},
"delegation": "training",
"verification": {
"status": "spoofed",
"method": "published_ip_ranges"
},
"headers": {
"resolved_client_ip": "203.0.113.42",
"proxy_chain": ["198.51.100.7"],
"mismatches": ["ua_vs_client_hints"]
},
"network": { "asn": 64500, "type": "datacenter" },
"timing": { "hidden_hop": true, "confidence": 0.91 },
"automation": { "class": "scripted", "confidence": 0.88 },
"judgment": {
"risk": 92,
"recommended_action": "block",
"reasons": ["claimed_bot_ip_mismatch", "hidden_proxy_hop"]
}
}
Sample request and verdict for a crawler spoofing GPTBot through a proxy.
What each verdict tells you
Each verdict is built from independent checks, so you decide how much weight each one carries in your product.
Identity
Name the bot and its operator
Each verdict gives you the name, operator, product and documentation link for thousands of bots and AI agents, and the catalog is updated as new agents ship.
Delegation
Separate training crawls from user requests
Training crawls, search indexing and agents fetching a page because a person asked for it are labeled separately, so you can treat each one differently.
Verification
Catch spoofed bots without blocking real ones
Claims are checked against published IP ranges, reverse DNS and signed agent identity, and a bot that simply cannot be verified is never mislabeled as spoofed.
Headers
Find the real client behind proxies
You get the real client IP behind proxy and CDN chains, along with any mismatch between the user agent, client hints and the network the request arrived on.
Network
Spot datacenter and relayed traffic
Datacenter, cloud and Tor origins are flagged, and relaying proxies are detected by how the connection behaves rather than by whether the address is already on a list.
Automation
Tell scripts, AI agents and people apart
The rhythm of requests across a session separates scripted traffic, AI-driven agents and human-like browsing, without adding a script to your pages.
Ask for only the fields you need
Get the full verdict every time, or request only the fields you use.
{
"identity": {
"name": "GPTBot",
"operator": "OpenAI",
"category": "ai_crawler",
"purpose": "model_training",
"acting_for": "its_owner"
},
"delegation": "training",
"verification": {
"status": "spoofed",
"method": "published_ip_ranges"
},
"headers": {
"resolved_client_ip": "203.0.113.42",
"proxy_chain": ["198.51.100.7"],
"mismatches": ["ua_vs_client_hints"]
},
"network": { "asn": 64500, "type": "datacenter" },
"timing": { "hidden_hop": true, "confidence": 0.91 },
"automation": { "class": "scripted", "confidence": 0.88 },
"judgment": {
"risk": 92,
"recommended_action": "block",
"reasons": ["claimed_bot_ip_mismatch", "hidden_proxy_hop"]
}
}
{
"verification": {
"status": "spoofed",
"method": "published_ip_ranges"
},
"judgment": {
"risk": 92,
"recommended_action": "block"
}
}
Go from sandbox key to first verdict in minutes
You can call the REST API from any stack, or use the JavaScript SDK to get caching and fail-open handling without writing them yourself.
- 1
Get a sandbox key
Sandbox keys include 100 verdicts per day, and you store yours as a secret in your server or worker.
- 2
Send the request details
Pass the incoming request to the SDK or REST API as a side call, so your response never waits on it.
- 3
Store the verdict
Write it next to the log line or show it in your customers' dashboards and rules.
+import { BotInsights } from '@botinsights/sdk'; export default { async fetch(request, env, ctx) {+ const bot = new BotInsights({ apiKey: env.BOT_INSIGHTS_KEY });+ const pending = bot.verify(request, { fields: 'identity,verification,judgment' });+ ctx.waitUntil(pending.then((verdict) => env.LOGS.send(verdict))); return fetch(request); },};curl -X POST https://api.botinsights.io/v1/verify \
-H "Authorization: Bearer $BOT_INSIGHTS_KEY" \
-H "Content-Type: application/json" \
-d '{
"method": "GET",
"path": "/pricing",
"remote_addr": "198.51.100.7",
"headers": {
"user-agent": "Mozilla/5.0 ... GPTBot/1.1",
"x-forwarded-for": "203.0.113.42"
}
}'
Your responses never wait on a verdict
Bot Insights runs as a side call from your own code, so your CDN stays as it is and visitors are served while the verdict is on its way. Inline decisions that allow or block before you respond are planned as more regions open. See what's planned after launch.
Visitor
Bot, agent or person
Requests your site or API as usual.
Your edge
CDN or reverse proxy
Stays as it is, with no traffic routed through Bot Insights.
Your code
Step1Server or worker
Holds your key and sends the request details as a side call that fails open.
Visitor
Response
Served without waiting on the verdict.
Bot Insights
Step2Verdict
Returns the full verdict or only the fields you ask for.
Your product
Step3Logs, dashboards and rules
Store the verdict next to the request or show it to your customers.
Designed to be embedded
Include bot intelligence in your existing workflows or build new ones. Call from your product, plugin, edge worker or pipeline.
Analytics, SEO and AI visibility
You can show your customers which AI crawlers and agents read their pages, and keep spoofed ones out of the counts they report to their own teams.
Security and caching plugins
Your rules can tell the real Googlebot from something using its name on any host your users run, without asking them to change their CDN.
Hosts and CDNs
You can offer verified bot identity to every site you serve without licensing an enterprise bot management product for each customer.
Licensing and pay-per-crawl
You can charge the right operator for the right purpose and show the evidence behind each decision when a crawler disputes it.
Log and SIEM pipelines
You can enrich every request line with identity and risk in batch, so your detections start from verified facts instead of user agent strings.
Agentic commerce
You can see whether an agent reaching checkout is acting for a real customer and whether it carries signed intent from the platform it runs on.
Full insights beyond your current stack
Bot Insights returns identity, purpose, verification and network evidence together from one API, alongside whatever CDN, framework and language you already use.
| Tool | What you get | Where it stops |
|---|---|---|
| CDN bot management | What you getVerified bot identity and category for traffic on their network | Where it stopsYou have to route traffic through them, and the deeper data sits behind an enterprise contract |
| Agent directories | What you getAgent name, type and operator with a verified or spoofed flag | Where it stopsA spoofed result stops there, with no header or network evidence of where the request came from |
| Security SDKs | What you getUser agent rules, IP and reverse DNS checks, and VPN or hosting flags | Where it stopsDetection is tied to their framework, and network data comes from lists |
| IP intelligence APIs | What you getVPN, proxy, datacenter and Tor flags for an address | Where it stopsYou get no identity, purpose or delegation, and a proxy is flagged only after someone has reported it |
What you'll want to know before you integrate
Which stacks and languages can I use?
At launch you call the REST API from any language or platform that can send an HTTPS request, and the JavaScript SDK for Node and Cloudflare Workers adds verdict caching and fail-open handling without extra code. PHP for WordPress, Python and Go SDKs and a batch endpoint for logs follow, and modules for Caddy, nginx and Envoy or HAProxy come after that for teams that want connection-level signals from their own reverse proxy.
Will verification slow down my requests?
Most teams start by enriching logs and analytics asynchronously, which adds nothing to the request path. When you make inline decisions, verdicts are cached for each unique combination of IP, user agent and header fingerprint, so repeat requests from the same bot are served from the SDK cache without another network call.
What happens if the API is unreachable or my balance runs out?
The SDKs fail open, so a request that cannot get a verdict is handled exactly as it would be without Bot Insights and your site keeps serving visitors. At zero balance the API returns a clear status with your remaining balance in a header, and a top-up restores service immediately.
Do I need to route traffic through you or add a script to my pages?
You call the API from your own code, worker, plugin or log pipeline, and nothing is injected into your pages or placed in front of your traffic. Optional modules for your reverse proxy add connection-level signals for teams that want them, and those run on your own infrastructure.
How do you tell an unverified bot from a spoofed one?
Many legitimate bots publish no IP ranges or DNS records, so a verdict only says spoofed when a claim can be checked and fails the check. Everything else is marked unverified along with the reason, which keeps you from blocking a real crawler on a guess.
How is visitor data handled?
Verdicts depend on IP addresses and request headers, which count as personal data in many jurisdictions. Human-like traffic is kept only as aggregates, raw IPs are retained only for automated traffic and for a limited period, full URLs and query strings are never stored, and a data processing agreement is available before you send production traffic.
Prepaid plans, billed per unique verdict
Every plan returns the full verdict and you pay per unique verdict, so the same bot requesting a thousand pages from the same IP and fingerprint within the cache window counts once. Plans are prepaid each month with an optional wallet for top-ups, which means a crawler spike never turns into a surprise invoice.
Up to 100,000 verdicts
$59/mo
Plus top-ups at <$1 per thousand
Up to 1 million verdicts
$299/mo
Plus top-ups at <$1 per thousand
Up to 10 million verdicts
$1,790/mo
Plus top-ups at <$1 per thousand
Sandbox testing includes 100 verdicts per day. Enterprise plans invoiced annually with custom volume. Minimum top-ups may apply. Pricing is planned and may change before launch.
What's planned after launch
Each addition plugs into the same place in your stack shown above, and new data arrives as fields in the verdict you already parse.
Logs, dashboards and rules
Batch endpoint
Send log lines in bulk and get a verdict back for each one instead of calling once per request.
Server or worker
More SDKs
PHP for WordPress, Python and Go SDKs with the same caching and fail-open handling as the JavaScript SDK.
CDN or reverse proxy
Proxy modules
Caddy, nginx and Envoy or HAProxy modules add connection timing from your own reverse proxy to the verdict.
Response
Regions and SLA
Endpoints in more regions so you can allow or block before responding, backed by a published uptime commitment.
Verdict
VPN and reputation
Named VPN operators, blocklist matches and an abuse score added to the same verdict you already receive.
Verdict
Actor tracking
A persistent profile for whoever is behind repeated spoofing on your sites, followed across IP rotation and shown only for actors seen on your own traffic.
Sign up for early access
Show your interest by answering a few quick questions and you'll get early access once we go live.