Skip to main content

Prerequisites

Requires Python ≥ 3.12.
Usage:
You can also set the SGAI_API_KEY environment variable and initialize the client without parameters: sgai = ScrapeGraphAI().
For more advanced usage, see the Python SDK documentation.

JavaScript SDK

Requires Node.js >= 22. Install using npm, pnpm, yarn, or bun:
Usage:
Store your API keys securely in environment variables. Use .env files and libraries like dotenv to load them into your app.
For more advanced usage, see the JavaScript SDK documentation.

Key Concepts

Scrape (formerly Markdownify)

Convert any webpage into markdown, HTML, screenshot, or branding format. Learn more

Extract (formerly SmartScraper)

Extract specific information from any webpage using AI. Provide a URL and a prompt describing what you want to extract. Learn more

Search (formerly SearchScraper)

Search and extract information from multiple web sources using AI. Start with just a query - Search will find relevant websites and extract the information you need. Learn more

Crawl (formerly SmartCrawler)

Multi-page website crawling with flexible output formats. Traverse multiple pages, follow links, and return content in your preferred format. Learn more

Monitor

Scheduled web monitoring with AI-powered extraction. Set up recurring scraping jobs that automatically extract data on a cron schedule. Learn more

Structured Output with Schemas

Both SDKs support structured output using schemas:
  • Python: Use Pydantic models
  • JavaScript: Use Zod schemas

Example: Extract Structured Data with Schema

Python Example

JavaScript Example


Next Steps