Crawle
Log inStart monitoring

List AI bot access

GET/v1/api/sites/{site_id}/ai-readiness/bot-access
Scope
sites:read
Token
API key or OAuth access token
Role
Any workspace member
MCP tool
list_ai_bot_access
What scopes, tokens and roles mean →
Request
curl "https://api.crawle.io/v1/api/sites/11111111-1111-1111-1111-111111111111/ai-readiness/bot-access?limit=10" \
  -H "Authorization: Bearer $CRAWLE_API_KEY"
const response = await fetch("https://api.crawle.io/v1/api/sites/11111111-1111-1111-1111-111111111111/ai-readiness/bot-access?limit=10", {
  headers: {
    Authorization: `Bearer ${process.env.CRAWLE_API_KEY}`,
  },
});
const data = await response.json();
import os

import requests

response = requests.request(
    "GET",
    "https://api.crawle.io/v1/api/sites/11111111-1111-1111-1111-111111111111/ai-readiness/bot-access?limit=10",
    headers={
        "Authorization": f"Bearer {os.environ['CRAWLE_API_KEY']}",
    },
)
data = response.json()
$CRAWLE_API_KEY
is an API key from Settings › API keys.

Response 200: a JSON object. Its fields are not described in the API contract yet, so no example is shown.

Parameters

site_idpath · string (uuid)required
part_idquery · string or null
control_idquery · string or null
result_idquery · string or null
finding_idquery · string or null
flow_idquery · string or null
partquery · enum or null
one of "crawling", "agentic"
categoryquery · string or null
statusquery · enum or null
one of "pass", "warning", "fail", "not_measured", "not_applicable"
unitquery · enum or null
one of "bot_url", "url", "url_class", "protocol", "reach", "flow", "form"
urlquery · string or null
up to 8192 characters
botquery · string or null
up to 128 characters
purposequery · enum or null
one of "training", "search_index", "user_fetch"
groupquery · string or null
sourcequery · enum or null
one of "robots_policy", "crawl_data", "sitemap", "cwv", "protocol", "reach_probe", "lighthouse", "agent_probe", "ax_snapshot"
outcomequery · enum or null
one of "ok", "blocked_for_bot", "challenge", "different_page", "rate_limited", "inconclusive", "rejected_5xx", "not_measured", "not_tested_robots", "flow_not_found"
flowquery · enum or null
one of "home", "search", "form", "checkout"
sincequery · string or null
cursorquery · string or null
up to 2048 characters
snapshot_watermarkquery · string or null
generationquery · string or null
up to 19 characters
limitquery · integer
1–500 · default 100
fromquery · string or null
toquery · string or null
controlquery · string or null
statequery · enum or null
one of "pass", "warning", "fail", "unmeasured", "not_applicable", "open", "resolved", "in_cc", "missing", "not_observed_yet"
afterquery · string or null
up to 20 characters

Pages with cursor, after, and limit.

Response 200

Returns a JSON object. Its fields are not described in the API contract yet.

Errors

  • 401

    When
    Missing, invalid, or expired token.

  • 403

    When
    The token has none of the accepted scopes (sites:read).

  • 422

    When
    The request did not match the schema. detail lists each problem.

  • 429

    When
    Rate limited. Wait for Retry-After seconds.

All errors and what to do →

Generated from the API contract at 7e7039b0b, synced October 30, 2026.