About OpenBotAuth
Bringing cryptographic identity and transparent commerce to AI crawler traffic.
Our mission
AI companies are crawling the web at scale to train models. Publishers want control and compensation. Today, there's no standard way to verify who's who or meter access.
OpenBotAuth creates the identity layer for this ecosystem. Crawlers prove who they are using cryptography. Publishers verify at the origin and enforce policy. Pay-per-crawl programs create transparent markets.
We believe this infrastructure should be open, interoperable, and not locked to any CDN or vendor. That's why the protocol is open source and based on IETF standards.
Our principles
Open standards
We build on HTTP Message Signatures (RFC 9421), JWKS, and other open specifications. Interoperability first, proprietary solutions never.
No lock-in
The protocol is Apache 2.0 licensed. Self-host if you want. Switch CDNs anytime. Your origin stays in control. We provide managed convenience, not vendor dependency.
Network effects
A crawler registered anywhere works with any publisher. More participants means more value for everyone. We design for ecosystem growth, not platform control.
Fair markets
Publishers deserve compensation for valuable content. AI companies deserve transparent pricing. We build infrastructure for fair, voluntary exchange—not mandated platforms.
Why now
AI training has changed the economics of web crawling. Models need massive, high-quality datasets. Publishers are blocking crawlers or demanding payment.
Existing tools—robots.txt, IP blocks, CDN bot management—aren't enough. They can't provide cryptographic identity, granular metering, or programmatic commerce.
Meanwhile, HTTP Message Signatures became an IETF standard (RFC 9421). The primitives exist. The timing is right to build identity and commerce infrastructure for AI agent traffic.
Open source
The OpenBotAuth protocol specification and reference implementations are open source under the Apache 2.0 license.
This includes:
- •Protocol specification and test vectors
- •Signature generation and verification SDKs
- •Registry reference implementation
- •WordPress plugin, Node.js middleware, Nginx integration
OpenBotAuth Cloud provides managed infrastructure on top of the open protocol—hosted registry, verifier endpoints, analytics, and pay-per-crawl program tooling.
Team
OpenBotAuth is built by engineers who've worked on crawler infrastructure, identity systems, and payment platforms. We care deeply about open standards, interoperability, and building infrastructure that benefits the entire ecosystem.