Skip to main content

​
Overview

just-scrape is the official CLI for ScrapeGraph AI — AI-powered web scraping, data extraction, search, and crawling, straight from your terminal.
Get your API key from the dashboard

​
Installation

npm install -g just-scrape
Package: just-scrape on npm | GitHub

​
Configuration

The CLI needs a ScrapeGraph API key. Four ways to provide it (checked in order):
  1. Environment variable: export SGAI_API_KEY="sgai-..."
  2. .env file: SGAI_API_KEY=sgai-... in project root
  3. Config file: ~/.scrapegraphai/config.json
  4. Interactive prompt: the CLI asks and saves to config

​
Environment Variables

VariableDescriptionDefault
SGAI_API_KEYScrapeGraph API key—
JUST_SCRAPE_API_URLOverride API base URLhttps://api.scrapegraphai.com/v1
JUST_SCRAPE_TIMEOUT_SRequest/polling timeout in seconds120
JUST_SCRAPE_DEBUGSet to 1 to enable debug logging0

​
JSON Mode

All commands support --json for machine-readable output. Banner, spinners, and interactive prompts are suppressed — only minified JSON on stdout. Saves tokens when piped to AI agents. 
just-scrape credits --json | jq '.remaining_credits'
just-scrape smart-scraper https://example.com -p "Extract data" --json > result.json

​
Commands

​
SmartScraper

Extract structured data from any URL using AI. Full docs → 
just-scrape smart-scraper <url> -p <prompt>
just-scrape smart-scraper <url> -p <prompt> --schema <json>
just-scrape smart-scraper <url> -p <prompt> --scrolls <n>
just-scrape smart-scraper <url> -p <prompt> --pages <n>
just-scrape smart-scraper <url> -p <prompt> --stealth
just-scrape smart-scraper <url> -p <prompt> --cookies <json> --headers <json>
just-scrape smart-scraper <url> -p <prompt> --plain-text

​
SearchScraper

Search the web and extract structured data from results. Full docs → 
just-scrape search-scraper <prompt>
just-scrape search-scraper <prompt> --num-results <n>
just-scrape search-scraper <prompt> --no-extraction
just-scrape search-scraper <prompt> --schema <json>
just-scrape search-scraper <prompt> --stealth --headers <json>

​
Markdownify

Convert any webpage to clean markdown. Full docs → 
just-scrape markdownify <url>
just-scrape markdownify <url> --stealth
just-scrape markdownify <url> --headers <json>

​
Crawl

Crawl multiple pages and extract data from each. Full docs → 
just-scrape crawl <url> -p <prompt>
just-scrape crawl <url> -p <prompt> --max-pages <n>
just-scrape crawl <url> -p <prompt> --depth <n>
just-scrape crawl <url> --no-extraction --max-pages <n>
just-scrape crawl <url> -p <prompt> --schema <json>
just-scrape crawl <url> -p <prompt> --rules <json>
just-scrape crawl <url> -p <prompt> --no-sitemap
just-scrape crawl <url> -p <prompt> --stealth

​
Scrape

Get raw HTML content from a URL. Full docs → 
just-scrape scrape <url>
just-scrape scrape <url> --stealth
just-scrape scrape <url> --branding
just-scrape scrape <url> --country-code <iso>

​
Sitemap

Get all URLs from a website’s sitemap. Full docs → 
just-scrape sitemap <url>
just-scrape sitemap <url> --json | jq -r '.urls[]'

​
Agentic Scraper

Browser automation with AI — login, click, navigate, fill forms. Full docs → 
just-scrape agentic-scraper <url> -s <steps>
just-scrape agentic-scraper <url> -s <steps> --ai-extraction -p <prompt>
just-scrape agentic-scraper <url> -s <steps> --schema <json>
just-scrape agentic-scraper <url> -s <steps> --use-session

​
Generate Schema

Generate a JSON schema from a natural language description. 
just-scrape generate-schema <prompt>
just-scrape generate-schema <prompt> --existing-schema <json>

​
History

Browse request history for any service. 
just-scrape history <service>
just-scrape history <service> <request-id>
just-scrape history <service> --page <n>
just-scrape history <service> --page-size <n>
just-scrape history <service> --json
Services: markdownify, smartscraper, searchscraper, scrape, crawl, agentic-scraper, sitemap

​
Credits

Check your credit balance. 
just-scrape credits

​
Validate

Validate your API key. 
just-scrape validate

​
AI Agent Integration

Use just-scrape as a skill for AI coding agents via Vercel’s skills.sh: 
bunx skills add https://github.com/ScrapeGraphAI/just-scrape

​
Support & Resources

npm Package

Install from npm

GitHub

Source code and issues

Community

Join our Discord community

Dashboard

Get your API key