Skip to main content

​
Overview

Monitor enables you to set up recurring web scraping jobs that automatically extract data on a schedule. Create monitors that run on a cron schedule and extract structured data from any webpage.
Try Monitor in our dashboard

​
Getting Started

​
Quick Start

from scrapegraph_py import Client

client = Client(api_key="your-api-key")

# Create a monitor
monitor = client.monitor.create(
    name="Price Tracker",
    url="https://example.com/products",
    prompt="Extract current product prices",
    interval="0 9 * * *",  # Daily at 9 AM
)
print("Monitor created:", monitor)

​
Parameters

ParameterTypeRequiredDescription
namestringYesA descriptive name for the monitor.
urlstringYesThe URL to monitor.
promptstringYesWhat data to extract on each run.
intervalstringYesCron expression for the schedule (5 fields, e.g., "0 9 * * *" for daily at 9 AM).
output_schemaobjectNoJSON Schema (Python dict / JS object) for structured response format.
fetch_configFetchConfigNoConfiguration for page fetching (mode, stealth, headers, wait, etc.).
Get your API key from the dashboard

​
Managing Monitors

​
List All Monitors

monitors = client.monitor.list()
print("All monitors:", monitors)

​
Get a Specific Monitor

monitor = client.monitor.get(monitor_id)
print("Monitor details:", monitor)

​
Pause a Monitor

client.monitor.pause(monitor_id)

​
Resume a Monitor

client.monitor.resume(monitor_id)

​
Delete a Monitor

client.monitor.delete(monitor_id)

​
Advanced Usage

​
With Output Schema and Config

from scrapegraph_py import Client, FetchConfig

schema = {
    "type": "object",
    "properties": {
        "products": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "price": {"type": "number"},
                    "in_stock": {"type": "boolean"},
                },
                "required": ["name", "price"],
            },
        },
    },
}

client = Client(api_key="your-api-key")

monitor = client.monitor.create(
    name="Product Price Monitor",
    url="https://example.com/products",
    prompt="Extract product names, prices, and stock status",
    interval="0 */6 * * *",  # Every 6 hours
    output_schema=schema,
    fetch_config=FetchConfig(mode="js", stealth=True, wait=2000),
)

​
Async Support

import asyncio
from scrapegraph_py import AsyncClient

async def main():
    async with AsyncClient(api_key="your-api-key") as client:
        monitor = await client.monitor.create(
            name="Tracker",
            url="https://example.com",
            prompt="Extract prices",
            interval="0 9 * * *",
        )

        monitors = await client.monitor.list()
        print("All monitors:", monitors)

asyncio.run(main())

​
Key Features

Scheduled Extraction

Run extraction jobs on any cron schedule

AI-Powered

Use natural language prompts to define what to extract

Full Control

Create, pause, resume, and delete monitors easily

Schema Support

Define structured output with Pydantic or Zod schemas

​
Common Cron Expressions

ExpressionSchedule
0 9 * * *Daily at 9 AM
0 */6 * * *Every 6 hours
0 9 * * 1Every Monday at 9 AM
0 0 1 * *First day of every month
*/30 * * * *Every 30 minutes

​
Integration Options

​
Official SDKs

  • Python SDK - Perfect for data science and backend applications
  • JavaScript SDK - Ideal for web applications and Node.js

​
AI Framework Integrations

​
Support & Resources

Documentation

Comprehensive guides and tutorials

API Reference

Detailed API documentation

Community

Join our Discord community

GitHub

Check out our open-source projects