Skip to main content
In v2, all proxy and fetch behaviour is controlled through the FetchConfig object. You can set the proxy strategy (mode), country-based geotargeting (country), wait times, scrolling, custom headers, cookies, and more. FetchConfig is accepted by sgai.extract(), sgai.scrape(), sgai.search(), and sgai.crawl.start().

​
Choosing a fetch mode

The mode parameter controls how pages are retrieved:
ModeDescription
autoAutomatically selects the best strategy (default)
fastDirect HTTP fetch, no JS rendering — fastest option
jsHeadless browser for JavaScript-heavy pages
Set stealth: true alongside any mode to enable residential proxy with anti-bot headers.

​
Examples

​
Geo-targeted content

Access content from a specific country using the country parameter: 
from scrapegraph_py import ScrapeGraphAI, FetchConfig

sgai = ScrapeGraphAI()

res = sgai.extract(
    "Extract the main content",
    url="https://example.com",
    fetch_config=FetchConfig(country="de"),  # Route through Germany
)

​
Stealth mode for protected sites

Use stealth mode to bypass anti-bot protections: 
from scrapegraph_py import ScrapeGraphAI, FetchConfig, MarkdownFormatConfig

sgai = ScrapeGraphAI()

res = sgai.scrape(
    "https://protected-site.com",
    formats=[MarkdownFormatConfig()],
    fetch_config=FetchConfig(
        mode="js",
        stealth=True,
        wait=3000,
        scrolls=3,
        country="us",
    ),
)

​
Custom headers and cookies

Pass custom HTTP headers or cookies with your requests: 
from scrapegraph_py import ScrapeGraphAI, FetchConfig

sgai = ScrapeGraphAI()

res = sgai.extract(
    "Extract product details",
    url="https://example.com",
    fetch_config=FetchConfig(
        headers={"Accept-Language": "en-US"},
        cookies={"session": "abc123"},
    ),
)

​
Tips

  • Start with mode: "auto" and only switch to a specific mode if you need to.
  • Set stealth: true for sites with strong anti-bot protections (combine with mode: "js" for dynamic sites).
  • Add wait time for pages that load content dynamically after the initial render.
  • Use scrolls to trigger lazy-loaded content on infinite-scroll pages.
  • The country parameter doesn’t affect pricing — credits are charged the same regardless of proxy location.