Build on open architecture

Use OpenBotAuth as the identity layer for your pay-per-crawl network, data marketplace, or agent commerce application. Open standards, no lock-in.

Request Builder AccessView on GitHub

The opportunity

AI companies need data. Publishers have it. There's a massive opportunity to build the infrastructure that connects them—marketplaces, networks, aggregators, and commerce platforms.

Pay-per-crawl networks

Aggregate publishers into a network with unified access and billing. Crawlers get one integration point. Publishers get distribution.

Data marketplaces

Build platforms where publishers list content and AI companies browse. Handle licensing, metering, and settlement.

Agent commerce applications

Create apps that let AI agents transact for data access. Verified identity enables trust. Metering enables pricing.

Why build on OpenBotAuth

We handle identity and verification. You focus on your marketplace or network logic.

Identity infrastructure

Don't build crawler identity from scratch. OpenBotAuth provides the registry, key management, and verification.

Open standards

HTTP Message Signatures, JWKS, standard JSON schemas. Interoperable with any HTTP infrastructure.

No vendor lock-in

Apache 2.0 license. Self-host if you want. Migrate away if you need. Your data, your infrastructure.

Composable APIs

Registry API, verifier API, metering webhooks. Build your own logic on top of our primitives.

White-label ready

Run verification under your domain. Brand the experience. Your customers see your platform.

Network effects

Crawlers registered anywhere in the ecosystem work with any publisher. More participants = more value.

Reference architecture

1

Your platform

Marketplace UI, network management, billing logic, customer onboarding. This is what you build.

2

OpenBotAuth identity

Registry API for crawler registration. JWKS hosting. Key management. We handle the crypto.

3

OpenBotAuth verification

Verifier endpoints or SDK. Your publishers integrate once. Works with any crawler in the ecosystem.

4

Metering + webhooks

Usage events flow to your platform. You aggregate, price, and bill however you want.

API preview

# Register a crawler

POST /v1/crawlers
{
  "id": "my-network-crawler-1",
  "organization_id": "org_123",
  "jwks_uri": "https://example.com/.well-known/jwks.json",
  "metadata": {
    "network": "my-pay-per-crawl-network"
  }
}

# Verify a request signature

POST /v1/verify
{
  "signature_input": "sig=(...)",
  "signature": "sig=:BASE64:",
  "request": {
    "method": "GET",
    "path": "/article/123",
    "headers": {...}
  }
}

Response:
{
  "valid": true,
  "crawler_id": "my-network-crawler-1",
  "organization": "Acme AI"
}

# Metering webhook payload

{
  "event": "request.verified",
  "crawler_id": "my-network-crawler-1",
  "publisher_id": "pub_456",
  "path": "/article/123",
  "timestamp": "2025-01-15T10:30:00Z",
  "bytes": 45678
}

Start building today

Request builder access to get API keys and early access to templates.

Request AccessView GitHub