Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.scrapegraphai.com/llms.txt

Use this file to discover all available pages before exploring further.

Cursor is an AI-powered code editor built on VS Code. Combined with ScrapeGraphAI, you can write, debug, and iterate on scraping pipelines faster using Cursor’s inline AI assistance. The fastest way to use ScrapeGraphAI in Cursor is via the MCP Server. This lets Cursor’s AI agent call ScrapeGraphAI tools directly without writing any code. See the MCP Server setup for Cursor guide for full instructions.

​
Manual integration

If you prefer to write code directly, use the Python or JavaScript v2 SDK.

​
Python

Install the SDK: 
pip install "scrapegraph-py>=2.1.0"
Ask Cursor to write a scraping script using Cmd+K or open the chat with Cmd+L:
Write a Python function using scrapegraph_py v2 that extracts the title, author, and date from any blog post URL.
Cursor will generate: 
from scrapegraph_py import ScrapeGraphAI

sgai = ScrapeGraphAI()  # reads SGAI_API_KEY from env

def extract_blog_post(url: str) -> dict | None:
    res = sgai.extract(
        "Extract the title, author name, and publication date",
        url=url,
    )
    return res.data.json_data if res.status == "success" else None

​
JavaScript

Install the SDK: 
npm i scrapegraph-js@latest
Ask Cursor:
Write a JavaScript function using scrapegraph-js v2 that extracts product details from an e-commerce page. 
import { ScrapeGraphAI } from "scrapegraph-js";

const sgai = ScrapeGraphAI(); // reads SGAI_API_KEY from env

export async function extractProduct(url) {
  const res = await sgai.extract({
    url,
    prompt: "Extract the product name, price, and availability",
  });
  return res.status === "success" ? res.data?.json : null;
}

​
Tips for using Cursor with ScrapeGraphAI

  • Paste error messages into the Cursor chat to get instant fix suggestions.
  • Ask Cursor to add a schema (JSON Schema, Pydantic, or Zod) to get strongly-typed results out of extract.
  • Use @docs in Cursor chat to reference the ScrapeGraphAI docs directly while coding.
Store your API key in a .env file as SGAI_API_KEY and load it via python-dotenv or process.env — the v2 SDKs pick it up automatically. Never hardcode it in source files.