Get crawl status

GET https://v2-api.scrapegraphai.com/api/crawl/:id

Returns progress and lightweight per-page metadata for a crawl job started with POST /api/crawl. Use GET /api/crawl/:id/pages to fetch paginated pages with resolved scrape results.

Path parameters

string

required

The crawl job UUID returned by POST /api/crawl.

Example request

curl -X GET https://v2-api.scrapegraphai.com/api/crawl/79694e03-f2ea-43f2-93cc-7c6fc26f999a \
  -H "SGAI-APIKEY: $SGAI_API_KEY"

Example response

{
  "id": "79694e03-f2ea-43f2-93cc-7c6fc26f999a",
  "status": "completed",
  "total": 3,
  "finished": 1,
  "pages": [
    {
      "url": "https://example.com",
      "depth": 0,
      "title": "",
      "status": "completed",
      "parentUrl": null,
      "contentType": "text/html",
      "links": ["https://iana.org/domains/example"],
      "scrapeRefId": "83a911ed-c0bc-4a8c-ad62-8efeeb93f33a"
    }
  ]
}

Field	Description
`status`	`"running"`, `"completed"`, `"failed"`, or `"stopped"`.
`total` / `finished`	Progress counters.
`pages[]`	Lightweight per-page metadata, ordered by crawl time.
`pages[].scrapeRefId`	UUID of the underlying Scrape call.

Poll at a reasonable cadence (every 1–5 seconds) until status is "completed", "failed", or "stopped". Or use Monitor with a webhook to avoid polling entirely.

Fetching page content

The status response intentionally stays lightweight for polling. Use GET /api/crawl/:id/pages to fetch crawl pages with the underlying scrape result resolved into each page:

curl -X GET "https://v2-api.scrapegraphai.com/api/crawl/79694e03-f2ea-43f2-93cc-7c6fc26f999a/pages?limit=50&cursor=0" \
  -H "SGAI-APIKEY: $SGAI_API_KEY"

The response is { data, pagination }, where each data[] item includes crawl metadata and a scrape payload when available.

Start a job: POST /api/crawl
Fetch pages: GET /api/crawl/:id/pages
Stop / resume / delete: Manage crawl jobs

API Documentation

Scrape

Extract

Search

Crawl

Monitor

History

Account

Path parameters

Example request

Example response

Fetching page content

​Path parameters

​Example request

​Example response

​Fetching page content

​Related

Path parameters

Example request

Example response

Fetching page content

Related