Can this bot fetch this page

Embed
Preview of the card
<a href="https://seolyticize.com/docs/ref-test-robots"><img src="https://seolyticize.com/docs/_api/public/orgs/71cb2b7a-a719-4003-b3ea-6321dc1db8ab/spaces/3bf2dbf7-3faa-473c-ba3b-30588f5f8de4/pages/dc0339e6-c5ae-47d1-9f15-996cbbc11909/social-card.svg?v=3e7ae33eec8b" alt="Can this bot fetch this page" width="600" height="315"></a>
POSThttps://seolyticize.com/api/v1/tools/test_robots

Reads the client's live robots.txt and says, for each address and each crawler, whether it may fetch it and which line decides — for every search engine and AI crawler by default (Googlebot, Bingbot, Applebot, DuckDuckBot, YandexBot, Baiduspider, GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-SearchBot, Claude-User, PerplexityBot, Perplexity-User, Google-Extended, Applebot-Extended, Meta-ExternalAgent, Bytespider, CCBot, Amazonbot, DuckAssistBot, MistralAI-User, AhrefsBot, SemrushBot), or the tokens in bots. Same parser the crawler obeys. Answers 'can GPTBot read our pricing page', 'are we blocking Google-Extended' and 'which rule blocks Googlebot here'. Free.

Parameters

Idempotency-Keystringheader
Any unique string, up to 200 characters. A retry with the same key and body is answered from the first answer (Idempotent-Replayed: true) for a day, so it never buys or writes twice; the same key with another body is refused.

Request body

application/json
clientIdstringrequired
urlsstring[]required
botsstring[]
= []
{
  "clientId": "string",
  "urls": [
    "string"
  ],
  "bots": []
}

Code samples

cURL
curl -X POST "https://seolyticize.com/api/v1/tools/test_robots" \
  -H "Authorization: Bearer $SEOLYTICIZE_API_KEY" \
  -H "Idempotency-Key: <value>" \
  -H "Content-Type: application/json" \
  -d '{"clientId":"string","urls":["string"],"bots":[]}'
TypeScript
Python
Go
PHP
Ruby
Java

Responses

200 The tool's result under data, and its meta.
Example
{
  "data": {},
  "meta": {
    "requestId": "string",
    "costMicros": 0,
    "cached": true,
    "cachedFrom": "string",
    "dataAsOf": "string",
    "sources": [
      "string"
    ],
    "partial": true,
    "next": [
      {
        "tool": "string",
        "args": {},
        "why": "string"
      }
    ],
    "nextCursor": "string",
    "dryRun": true
  }
}
Schema
datarequired
metaobjectrequired
Beside every answer: what it cost, whether it came from cache, how fresh it is, whether it is complete, and what to call next.
Show child parameters
requestIdstringrequired
costMicrosintegerrequired
Charged by this call, micro-USD (1,000,000 = $1).
cachedbooleanrequired
True when nothing was bought.
cachedFromstring
When the oldest cached answer used was bought.
dataAsOfstring
sourcesstring[]
partialbooleanrequired
True when the data is incomplete: a running crawl, unfinished days, unread pages.
nextobject[]required
Show child parameters
toolstring
argsobject
whystring
nextCursorstring
Pass as cursor for the next page.
dryRunboolean
Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
X-Request-Idstring
The id to quote about this request.
400 The body was not JSON or did not match the tool's input schema.
Example
{
  "error": "string",
  "code": "string",
  "next": "string",
  "requestId": "string"
}
Schema
errorstringrequired
A sentence written for the caller. Internal failures are reduced to 'The request could not be completed.' and never carry detail.
codestring
A stable snake_case reason to branch on; GET /api/v1/errors lists them all.
nextstring
The call or step that fixes it, when there is one.
requestIdstring
Quote it when asking about a failure.
Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
401 No key, or a key that is unknown or revoked.

Same as 400 · POST /api/v1/tools/test_robots

Headers
WWW-Authenticatestring
403 The key's role does not permit the tool, or the key may not reach the named client.

Same as 400 · POST /api/v1/tools/test_robots

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
404 No tool by that name.

Same as 400 · POST /api/v1/tools/test_robots

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
409 A request with this Idempotency-Key is still running.

Same as 400 · POST /api/v1/tools/test_robots

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
422 A well-formed request the product declined, with the reason — a duplicate name, a plan limit, nothing configured to run.

Same as 400 · POST /api/v1/tools/test_robots

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
429 This key has used its window.

Same as 400 · POST /api/v1/tools/test_robots

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
Retry-Afterinteger
Seconds to wait before retrying.
500 The request could not be completed.

Same as 400 · POST /api/v1/tools/test_robots

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
Updated