Where crawlers spend their visits

Embed
Preview of the card
<a href="https://seolyticize.com/docs/ref-get-crawl-budget"><img src="https://seolyticize.com/docs/_api/public/orgs/71cb2b7a-a719-4003-b3ea-6321dc1db8ab/spaces/3bf2dbf7-3faa-473c-ba3b-30588f5f8de4/pages/fa6f5551-fd60-45f2-a7fb-e127e9c58960/social-card.svg?v=3e7ae33eec8b" alt="Where crawlers spend their visits" width="600" height="315"></a>
POSThttps://seolyticize.com/api/v1/tools/get_crawl_budget

From the client's connected logs over the last days (default 30): crawler requests by URL template (the site section, not the page), each with its share of all crawler requests, how much went on errors and redirects, and which crawlers; plus per crawler how many requests were verified (reverse DNS or published IP ranges), spoofed (claimed the name, failed the check — left out of every figure) or not checkable. Assets are left out. bot narrows to one crawler (googlebot, bingbot, gptbot…). Answers 'what is Googlebot wasting its time on' and 'how much of our crawl budget goes on parameter URLs'. Free.

Parameters

Idempotency-Keystringheader
Any unique string, up to 200 characters. A retry with the same key and body is answered from the first answer (Idempotent-Replayed: true) for a day, so it never buys or writes twice; the same key with another body is refused.

Request body

application/json
clientIdstringrequired
daysinteger
= 30
botstring
= null
{
  "clientId": "string",
  "days": 30,
  "bot": null
}

Code samples

cURL
curl -X POST "https://seolyticize.com/api/v1/tools/get_crawl_budget" \
  -H "Authorization: Bearer $SEOLYTICIZE_API_KEY" \
  -H "Idempotency-Key: <value>" \
  -H "Content-Type: application/json" \
  -d '{"clientId":"string","days":30,"bot":null}'
TypeScript
Python
Go
PHP
Ruby
Java

Responses

200 The tool's result under data, and its meta.
Example
{
  "data": {},
  "meta": {
    "requestId": "string",
    "costMicros": 0,
    "cached": true,
    "cachedFrom": "string",
    "dataAsOf": "string",
    "sources": [
      "string"
    ],
    "partial": true,
    "next": [
      {
        "tool": "string",
        "args": {},
        "why": "string"
      }
    ],
    "nextCursor": "string",
    "dryRun": true
  }
}
Schema
datarequired
metaobjectrequired
Beside every answer: what it cost, whether it came from cache, how fresh it is, whether it is complete, and what to call next.
Show child parameters
requestIdstringrequired
costMicrosintegerrequired
Charged by this call, micro-USD (1,000,000 = $1).
cachedbooleanrequired
True when nothing was bought.
cachedFromstring
When the oldest cached answer used was bought.
dataAsOfstring
sourcesstring[]
partialbooleanrequired
True when the data is incomplete: a running crawl, unfinished days, unread pages.
nextobject[]required
Show child parameters
toolstring
argsobject
whystring
nextCursorstring
Pass as cursor for the next page.
dryRunboolean
Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
X-Request-Idstring
The id to quote about this request.
400 The body was not JSON or did not match the tool's input schema.
Example
{
  "error": "string",
  "code": "string",
  "next": "string",
  "requestId": "string"
}
Schema
errorstringrequired
A sentence written for the caller. Internal failures are reduced to 'The request could not be completed.' and never carry detail.
codestring
A stable snake_case reason to branch on; GET /api/v1/errors lists them all.
nextstring
The call or step that fixes it, when there is one.
requestIdstring
Quote it when asking about a failure.
Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
401 No key, or a key that is unknown or revoked.

Same as 400 · POST /api/v1/tools/get_crawl_budget

Headers
WWW-Authenticatestring
403 The key's role does not permit the tool, or the key may not reach the named client.

Same as 400 · POST /api/v1/tools/get_crawl_budget

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
404 No tool by that name.

Same as 400 · POST /api/v1/tools/get_crawl_budget

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
409 A request with this Idempotency-Key is still running.

Same as 400 · POST /api/v1/tools/get_crawl_budget

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
422 A well-formed request the product declined, with the reason — a duplicate name, a plan limit, nothing configured to run.

Same as 400 · POST /api/v1/tools/get_crawl_budget

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
429 This key has used its window.

Same as 400 · POST /api/v1/tools/get_crawl_budget

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
Retry-Afterinteger
Seconds to wait before retrying.
500 The request could not be completed.

Same as 400 · POST /api/v1/tools/get_crawl_budget

Headers
X-RateLimit-Limitinteger
Requests this key may make per window.
X-RateLimit-Remaininginteger
Requests left in the current window.
X-RateLimit-Resetinteger
Seconds until the current window resets.
Updated