Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.scrapegraphai.com/llms.txt

Use this file to discover all available pages before exploring further.

Every just-scrape command supports a --json flag that switches to machine-readable output. When active:
  • The ASCII banner is hidden
  • Spinners and progress indicators are suppressed
  • Interactive prompts are disabled
  • Only minified JSON is written to stdout
This makes just-scrape easy to use in shell scripts, CI pipelines, and AI agent workflows.

​
Basic usage

just-scrape <command> [args] --json

​
Examples

​
Save results to a file

just-scrape extract https://store.example.com/shoes \
  -p "Extract all product names and prices" \
  --json > products.json

​
Extract a specific field with jq

just-scrape credits --json | jq '.remaining'

just-scrape history extract --json | jq '.requests[] | {id: .request_id, status}'

​
Convert a page to markdown and save it

just-scrape scrape https://docs.example.com/api \
  --json | jq -r '.results.markdown.data[0]' > api-docs.md

​
Chain commands in a script

#!/bin/bash
# Extract a list of URLs and save each result
while IFS= read -r url; do
  just-scrape extract "$url" \
    -p "Extract the page title and main content" \
    --json >> results.jsonl
done < urls.txt

​
Use in a CI pipeline

# GitHub Actions example
- name: Extract changelog
  run: |
    just-scrape scrape https://github.com/org/repo/releases \
      --json | jq -r '.results.markdown.data[0]' > CHANGELOG.md

​
Response structure

The JSON output mirrors the v2 API response for each command. Common shapes:
  • extract — { id, json, raw, usage, metadata }
  • scrape — { id, results: { markdown: { data: [...] }, ... }, metadata }
  • search — { id, results: [...], json, metadata }
  • crawl — the CLI polls until the job reaches a terminal state, then prints { id, status, finished, total, ... }
  • credits — { remaining, used, plan, jobs: { crawl: {used, limit}, monitor: {used, limit} } }
  • validate — { status, uptime }
For credits: 
{
  "remaining": 4820,
  "used": 180,
  "plan": "Starter",
  "jobs": {
    "crawl": { "used": 0, "limit": 50 },
    "monitor": { "used": 1, "limit": 100 }
  }
}
--json mode is especially useful when calling just-scrape from AI coding agents. It eliminates decorative output and saves tokens.