> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scrapegraphai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# CLI

> Command-line interface for ScrapeGraph AI

## Overview

`just-scrape` is the official CLI for [ScrapeGraph AI](https://scrapegraphai.com) — AI-powered web scraping, data extraction, search, and crawling, straight from your terminal. Uses the **v2 API**.

<Note>
  Get your API key from the [dashboard](https://scrapegraphai.com/dashboard)
</Note>

## Installation

<CodeGroup>
  ```bash npm theme={null}
  npm install -g just-scrape
  ```

  ```bash pnpm theme={null}
  pnpm add -g just-scrape
  ```

  ```bash yarn theme={null}
  yarn global add just-scrape
  ```

  ```bash bun theme={null}
  bun add -g just-scrape
  ```

  ```bash npx (no install) theme={null}
  npx just-scrape --help
  ```

  ```bash bunx (no install) theme={null}
  bunx just-scrape --help
  ```
</CodeGroup>

Package: [just-scrape](https://www.npmjs.com/package/just-scrape) on npm | [GitHub](https://github.com/ScrapeGraphAI/just-scrape)

## Configuration

The CLI needs a ScrapeGraph API key. Four ways to provide it (checked in order):

1. **Environment variable**: `export SGAI_API_KEY="sgai-..."`
2. **`.env` file**: `SGAI_API_KEY=sgai-...` in project root
3. **Config file**: `~/.scrapegraphai/config.json`
4. **Interactive prompt**: the CLI asks and saves to config

### Environment Variables

| Variable       | Description                          | Default                                |
| -------------- | ------------------------------------ | -------------------------------------- |
| `SGAI_API_KEY` | ScrapeGraph API key                  | —                                      |
| `SGAI_API_URL` | Override API base URL                | `https://api.scrapegraphai.com/api/v2` |
| `SGAI_TIMEOUT` | Request timeout in seconds           | `120`                                  |
| `SGAI_DEBUG`   | Set to `1` to log requests/responses | —                                      |

Legacy variables are still bridged transparently: `JUST_SCRAPE_API_URL` → `SGAI_API_URL`, `JUST_SCRAPE_TIMEOUT_S` / `SGAI_TIMEOUT_S` → `SGAI_TIMEOUT`, `JUST_SCRAPE_DEBUG` → `SGAI_DEBUG`.

## JSON Mode

All commands support `--json` for machine-readable output. Banner, spinners, and interactive prompts are suppressed — only minified JSON on stdout. Saves tokens when piped to AI agents.

```bash theme={null}
just-scrape credits --json | jq '.remaining'
just-scrape extract https://example.com -p "Extract data" --json > result.json
```

## Commands

### Extract

Extract structured data from any URL using AI (replaces `smart-scraper`). [Full docs →](/api-reference/extract)

```bash theme={null}
just-scrape extract <url> -p <prompt>
just-scrape extract <url> -p <prompt> --schema <json>
just-scrape extract <url> -p <prompt> --scrolls <n>
just-scrape extract <url> -p <prompt> --mode reader          # HTML mode: normal, reader, prune
just-scrape extract <url> -p <prompt> --stealth
just-scrape extract <url> -p <prompt> --cookies <json> --headers <json>
just-scrape extract <url> -p <prompt> --country <iso>
```

### Search

Search the web and extract structured data from results (replaces `search-scraper`). [Full docs →](/api-reference/search)

```bash theme={null}
just-scrape search <query>
just-scrape search <query> --num-results <n>
just-scrape search <query> -p <prompt>
just-scrape search <query> --schema <json>
just-scrape search <query> --format markdown            # or html
just-scrape search <query> --country <iso>              # e.g. us, de, jp
just-scrape search <query> --time-range past_week       # past_hour|past_24_hours|past_week|past_month|past_year
just-scrape search <query> --headers <json>
```

### Scrape

Scrape a URL into one or more of 8 output formats. Multi-format is supported via comma-separated `-f`. [Full docs →](/api-reference/scrape)

```bash theme={null}
just-scrape scrape <url>                                   # markdown (default)
just-scrape scrape <url> -f html
just-scrape scrape <url> -f screenshot
just-scrape scrape <url> -f markdown,links,images          # multi-format
just-scrape scrape <url> -f json -p "Extract the title"    # json format requires --prompt
just-scrape scrape <url> -f json -p <prompt> --schema <json>
just-scrape scrape <url> --html-mode reader                # normal | reader | prune
just-scrape scrape <url> --scrolls 3
just-scrape scrape <url> -m js --stealth
just-scrape scrape <url> --country <iso>
```

#### Formats

| Format       | Description                                                  |
| ------------ | ------------------------------------------------------------ |
| `markdown`   | Clean markdown conversion (default). Respects `--html-mode`. |
| `html`       | Raw / processed HTML. Respects `--html-mode`.                |
| `screenshot` | Page screenshot (PNG).                                       |
| `branding`   | Extracted brand assets (logos, colors, fonts).               |
| `links`      | All links on the page.                                       |
| `images`     | All images on the page.                                      |
| `summary`    | AI-generated page summary.                                   |
| `json`       | Structured JSON via `--prompt` (+ optional `--schema`).      |

### Crawl

Crawl multiple pages. The CLI starts the crawl and polls until completion. [Full docs →](/api-reference/crawl)

```bash theme={null}
just-scrape crawl <url>
just-scrape crawl <url> --max-pages <n>
just-scrape crawl <url> --max-depth <n>
just-scrape crawl <url> --max-links-per-page <n>
just-scrape crawl <url> --allow-external
just-scrape crawl <url> -f markdown        # or html, json, etc.
just-scrape crawl <url> -m js --stealth
```

### Fetch Modes

Use `-m / --mode` on `scrape` and `crawl` to choose how pages are fetched. Add `--stealth` to enable anti-bot bypass.

| Mode   | Description                   |
| ------ | ----------------------------- |
| `auto` | Automatic selection (default) |
| `fast` | Fastest, no JS rendering      |
| `js`   | Full JS rendering             |

<Note>
  On `extract`, `--mode` sets the **HTML processing mode** (`normal`, `reader`, `prune`) instead. Use `--stealth` separately for anti-bot bypass.
</Note>

### Monitor

Create and manage page-change monitors that track changes on a URL at a set interval.

```bash theme={null}
# Create a monitor
just-scrape monitor create --url <url> --interval <interval>
just-scrape monitor create --url <url> --interval 1h --name "My Monitor"
just-scrape monitor create --url <url> --interval 30m -f markdown,links --webhook-url <url>
just-scrape monitor create --url <url> --interval 1d -m js --stealth

# List all monitors
just-scrape monitor list

# Get a specific monitor
just-scrape monitor get --id <monitor-id>

# Update a monitor
just-scrape monitor update --id <monitor-id> --interval 2h
just-scrape monitor update --id <monitor-id> --name "New Name" -f html,screenshot

# Pause / resume / delete
just-scrape monitor pause --id <monitor-id>
just-scrape monitor resume --id <monitor-id>
just-scrape monitor delete --id <monitor-id>
```

### History

Browse request history for any service.

```bash theme={null}
just-scrape history <service>
just-scrape history <service> <request-id>
just-scrape history <service> --page <n>
just-scrape history <service> --page-size <n>
just-scrape history <service> --json
```

Services: `scrape`, `extract`, `schema`, `search`, `monitor`, `crawl`

### Credits

Check your credit balance.

```bash theme={null}
just-scrape credits
just-scrape credits --json | jq '.remaining'
```

## AI Agent Integration

Use `just-scrape` as a skill for AI coding agents via [Vercel's skills.sh](https://skills.sh):

```bash theme={null}
bunx skills add https://github.com/ScrapeGraphAI/just-scrape
```

## Support & Resources

<CardGroup cols={2}>
  <Card title="npm Package" icon="npm" href="https://www.npmjs.com/package/just-scrape">
    Install from npm
  </Card>

  <Card title="GitHub" icon="github" href="https://github.com/ScrapeGraphAI/just-scrape">
    Source code and issues
  </Card>

  <Card title="Community" icon="discord" href="https://discord.gg/uJN7TYcpNa">
    Join our Discord community
  </Card>

  <Card title="Dashboard" icon="gauge" href="https://scrapegraphai.com/dashboard">
    Get your API key
  </Card>
</CardGroup>