Rate limits by plan - ScrapeGraphAI

ScrapeGraphAI enforces rate limits to ensure reliable performance for all users. Limits vary by plan.

Limits overview

Plan	Requests per minute	Concurrent crawls	Monitors	Monthly credits
Free	10	1	1	500
Starter	100	3	5	10,000
Growth	500	15	25	100,000
Pro	5,000	50	100	750,000
Enterprise	Custom	Custom	Custom	Custom

For full pricing details, see Plans & Pricing.

Contact support for custom limits or high-volume plans.

What counts as a request?

Each API call to any v2 endpoint (/api/extract, /api/scrape, /api/search, /api/crawl, /api/monitor, …) counts as one request toward your rate limit. Polling a crawl or monitor status endpoint does not count toward the limit.

Rate limit headers

Every API response includes headers that show your current rate limit status:

X-RateLimit-Limit: 100
X-RateLimit-Remaining: 87
X-RateLimit-Reset: 1709123456

X-RateLimit-Limit — maximum requests allowed per minute
X-RateLimit-Remaining — requests remaining in the current window
X-RateLimit-Reset — Unix timestamp when the limit resets

Handling the 429 response

When you exceed the rate limit, the API returns HTTP 429 Too Many Requests. In the v2 SDK this surfaces as res.status === "error" — the SDK does not raise. Retry with exponential backoff:

import time
from scrapegraph_py import ScrapeGraphAI

sgai = ScrapeGraphAI()

def extract_with_backoff(url, prompt, max_retries=5):
    for i in range(max_retries):
        res = sgai.extract(prompt, url=url)
        if res.status == "success":
            return res
        if "rate_limit" not in (res.error or "").lower():
            return res  # non-rate-limit error — don't retry
        wait = 2 ** i
        print(f"Rate limited — retrying in {wait}s")
        time.sleep(wait)
    raise RuntimeError("Exceeded max retries")

See the rate limiting troubleshooting guide for a JavaScript example and more tips.

Increasing your limits

Upgrade your plan from the dashboard to get higher limits immediately.
Enterprise customers can request custom rate limit configurations by contacting support.

​Limits overview

​What counts as a request?

​Rate limit headers

​Handling the 429 response

​Increasing your limits

Limits overview

What counts as a request?

Rate limit headers

Handling the 429 response

Increasing your limits