> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scrapegraphai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Handling pagination

> Scrape multi-page results with Crawl or iterate URLs yourself

Many websites spread their content across multiple pages — product listings, search results, articles. In v2 there are two approaches: let `crawl.start` follow links for you, or iterate page URLs manually with `extract`.

## Using Crawl for multi-page extraction

[`crawl.start`](/services/crawl) is the recommended service when you want to follow links automatically. It runs asynchronously — start a job and poll for the result.

<CodeGroup>
  ```python Python theme={null}
  from scrapegraph_py import ScrapeGraphAI, JsonFormatConfig

  sgai = ScrapeGraphAI()

  start = sgai.crawl.start(
      "https://example.com/products",
      formats=[JsonFormatConfig(prompt="Extract product names and prices")],
      max_depth=2,
      max_pages=50,
      include_patterns=["/products*"],
  )

  status = sgai.crawl.get(start.data.id)
  print(f"{status.data.finished}/{status.data.total} - {status.data.status}")
  ```

  ```javascript JavaScript theme={null}
  import { ScrapeGraphAI } from "scrapegraph-js";

  const sgai = ScrapeGraphAI();

  const start = await sgai.crawl.start({
    url: "https://example.com/products",
    formats: [{ type: "json", prompt: "Extract product names and prices" }],
    maxDepth: 2,
    maxPages: 50,
    includePatterns: ["/products*"],
  });

  const status = await sgai.crawl.get(start.data.id);
  ```
</CodeGroup>

## Iterating page URLs with Extract

If you know the URL pattern for each page, call `extract` on each URL and aggregate results:

<CodeGroup>
  ```python Python theme={null}
  from scrapegraph_py import ScrapeGraphAI

  sgai = ScrapeGraphAI()

  all_products = []
  for page in range(1, 6):  # pages 1-5
      url = f"https://example.com/products?page={page}"
      res = sgai.extract(
          "Extract all product names and prices on this page",
          url=url,
      )
      if res.status == "success":
          all_products.extend(res.data.json_data.get("products", []))

  print(f"Total products extracted: {len(all_products)}")
  ```

  ```javascript JavaScript theme={null}
  import { ScrapeGraphAI } from "scrapegraph-js";

  const sgai = ScrapeGraphAI();
  const allProducts = [];

  for (let page = 1; page <= 5; page++) {
    const url = `https://example.com/products?page=${page}`;
    const res = await sgai.extract({
      url,
      prompt: "Extract all product names and prices on this page",
    });
    if (res.status === "success") {
      allProducts.push(...(res.data?.json?.products ?? []));
    }
  }
  ```
</CodeGroup>

## Tips

* Prefer **`crawl.start`** when the number or pattern of pages is unknown — it handles link discovery for you.
* Use **manual iteration** when URLs follow a predictable pattern (`?page=N`) and you want tight control.
* **Add delays between pages** in manual mode to avoid triggering rate limits on the target website.
* **Stop early** when the extracted list is empty or a "no more results" marker appears.
* For infinite-scroll pages, use [`FetchConfig(scrolls=N)`](/knowledge-base/scraping/javascript-rendering#infinite-scroll-lazy-loading) instead of pagination.