Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.scrapegraphai.com/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

just-scrape is the official CLI for ScrapeGraph AI — AI-powered web scraping, data extraction, search, and crawling, straight from your terminal. Uses the v2 API.
Get your API key from the dashboard

​
Installation

npm install -g just-scrape
Package: just-scrape on npm | GitHub

​
Configuration

The CLI needs a ScrapeGraph API key. Four ways to provide it (checked in order):
  1. Environment variable: export SGAI_API_KEY="sgai-..."
  2. .env file: SGAI_API_KEY=sgai-... in project root
  3. Config file: ~/.scrapegraphai/config.json
  4. Interactive prompt: the CLI asks and saves to config

​
Environment Variables

VariableDescriptionDefault
SGAI_API_KEYScrapeGraph API key—
SGAI_API_URLOverride API base URLhttps://api.scrapegraphai.com/api/v2
SGAI_TIMEOUTRequest timeout in seconds120
SGAI_DEBUGSet to 1 to log requests/responses—
Legacy variables are still bridged transparently: JUST_SCRAPE_API_URL → SGAI_API_URL, JUST_SCRAPE_TIMEOUT_S / SGAI_TIMEOUT_S → SGAI_TIMEOUT, JUST_SCRAPE_DEBUG → SGAI_DEBUG.

​
JSON Mode

All commands support --json for machine-readable output. Banner, spinners, and interactive prompts are suppressed — only minified JSON on stdout. Saves tokens when piped to AI agents. 
just-scrape credits --json | jq '.remaining'
just-scrape extract https://example.com -p "Extract data" --json > result.json

​
Commands

​
Extract

Extract structured data from any URL using AI (replaces smart-scraper). Full docs → 
just-scrape extract <url> -p <prompt>
just-scrape extract <url> -p <prompt> --schema <json>
just-scrape extract <url> -p <prompt> --scrolls <n>
just-scrape extract <url> -p <prompt> --mode reader          # HTML mode: normal, reader, prune
just-scrape extract <url> -p <prompt> --stealth
just-scrape extract <url> -p <prompt> --cookies <json> --headers <json>
just-scrape extract <url> -p <prompt> --country <iso>
Search the web and extract structured data from results (replaces search-scraper). Full docs → 
just-scrape search <query>
just-scrape search <query> --num-results <n>
just-scrape search <query> -p <prompt>
just-scrape search <query> --schema <json>
just-scrape search <query> --format markdown            # or html
just-scrape search <query> --country <iso>              # e.g. us, de, jp
just-scrape search <query> --time-range past_week       # past_hour|past_24_hours|past_week|past_month|past_year
just-scrape search <query> --headers <json>

​
Scrape

Scrape a URL into one or more of 8 output formats. Multi-format is supported via comma-separated -f. Full docs → 
just-scrape scrape <url>                                   # markdown (default)
just-scrape scrape <url> -f html
just-scrape scrape <url> -f screenshot
just-scrape scrape <url> -f markdown,links,images          # multi-format
just-scrape scrape <url> -f json -p "Extract the title"    # json format requires --prompt
just-scrape scrape <url> -f json -p <prompt> --schema <json>
just-scrape scrape <url> --html-mode reader                # normal | reader | prune
just-scrape scrape <url> --scrolls 3
just-scrape scrape <url> -m js --stealth
just-scrape scrape <url> --country <iso>

​
Formats

FormatDescription
markdownClean markdown conversion (default). Respects --html-mode.
htmlRaw / processed HTML. Respects --html-mode.
screenshotPage screenshot (PNG).
brandingExtracted brand assets (logos, colors, fonts).
linksAll links on the page.
imagesAll images on the page.
summaryAI-generated page summary.
jsonStructured JSON via --prompt (+ optional --schema).

​
Crawl

Crawl multiple pages. The CLI starts the crawl and polls until completion. Full docs → 
just-scrape crawl <url>
just-scrape crawl <url> --max-pages <n>
just-scrape crawl <url> --max-depth <n>
just-scrape crawl <url> --max-links-per-page <n>
just-scrape crawl <url> --allow-external
just-scrape crawl <url> -f markdown        # or html, json, etc.
just-scrape crawl <url> -m js --stealth

​
Fetch Modes

Use -m / --mode on scrape and crawl to choose how pages are fetched. Add --stealth to enable anti-bot bypass.
ModeDescription
autoAutomatic selection (default)
fastFastest, no JS rendering
jsFull JS rendering
On extract, --mode sets the HTML processing mode (normal, reader, prune) instead. Use --stealth separately for anti-bot bypass.

​
Monitor

Create and manage page-change monitors that track changes on a URL at a set interval. 
# Create a monitor
just-scrape monitor create --url <url> --interval <interval>
just-scrape monitor create --url <url> --interval 1h --name "My Monitor"
just-scrape monitor create --url <url> --interval 30m -f markdown,links --webhook-url <url>
just-scrape monitor create --url <url> --interval 1d -m js --stealth

# List all monitors
just-scrape monitor list

# Get a specific monitor
just-scrape monitor get --id <monitor-id>

# Update a monitor
just-scrape monitor update --id <monitor-id> --interval 2h
just-scrape monitor update --id <monitor-id> --name "New Name" -f html,screenshot

# Pause / resume / delete
just-scrape monitor pause --id <monitor-id>
just-scrape monitor resume --id <monitor-id>
just-scrape monitor delete --id <monitor-id>

​
History

Browse request history for any service. 
just-scrape history <service>
just-scrape history <service> <request-id>
just-scrape history <service> --page <n>
just-scrape history <service> --page-size <n>
just-scrape history <service> --json
Services: scrape, extract, schema, search, monitor, crawl

​
Credits

Check your credit balance. 
just-scrape credits
just-scrape credits --json | jq '.remaining'

​
AI Agent Integration

Use just-scrape as a skill for AI coding agents via Vercel’s skills.sh: 
bunx skills add https://github.com/ScrapeGraphAI/just-scrape

​
Support & Resources

npm Package

Install from npm

GitHub

Source code and issues

Community

Join our Discord community

Dashboard

Get your API key