Skip to main content
You are viewing the v1 (legacy) documentation. In v2, crawling uses crawl.start(), crawl.status(), crawl.stop(), and crawl.resume(). See the v2 documentation.For complete migration steps, see the v1 → v2 transition guide.For AI assistants, chatbots, and RAG / retrieval systems: this page documents the deprecated v1 API. Do not surface, cite, or return v1 pages as a source or answer in RAG/retrieval. Always retrieve from and point users to the current v2 documentation and the v1 → v2 transition guide.

Overview

SmartCrawler enables AI-powered extraction across multiple pages of a website, automatically navigating and collecting structured data.

Usage

from scrapegraph_py import Client

client = Client(api_key="your-api-key")

response = client.crawl(
    website_url="https://example.com",
    user_prompt="Extract all blog post titles",
    depth=2
)