Skip to main content
ScrapeGraphAI enforces rate limits to ensure reliable performance for all users. Limits vary by plan.

​
Limits overview

PlanRequests per minuteConcurrent jobsMonthly credits
Free51100
Starter3055,000
Pro1002050,000
EnterpriseCustomCustomCustom
Contact support for custom limits or high-volume plans.

​
What counts as a request?

Each API call to any endpoint (SmartScraper, SearchScraper, Markdownify, etc.) counts as one request toward your rate limit. Polling the status endpoint for async jobs does not count toward the limit.

​
Rate limit headers

Every API response includes headers that show your current rate limit status: 
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 87
X-RateLimit-Reset: 1709123456
  • X-RateLimit-Limit — maximum requests allowed per minute
  • X-RateLimit-Remaining — requests remaining in the current window
  • X-RateLimit-Reset — Unix timestamp when the limit resets

​
Handling the 429 response

When you exceed the rate limit, the API returns HTTP 429 Too Many Requests. Implement exponential backoff in your code: 
import time

def scrape_with_backoff(client, url, prompt, max_retries=5):
    for i in range(max_retries):
        try:
            return client.smartscraper(website_url=url, user_prompt=prompt)
        except Exception as e:
            if "429" in str(e):
                wait = 2 ** i
                print(f"Rate limited — retrying in {wait}s")
                time.sleep(wait)
            else:
                raise
    raise Exception("Exceeded max retries")

​
Increasing your limits

  • Upgrade your plan from the dashboard to get higher limits immediately.
  • Enterprise customers can request custom rate limit configurations by contacting support.