Documentation Index
Fetch the complete documentation index at: https://docs.scrapegraphai.com/llms.txt
Use this file to discover all available pages before exploring further.
All just-scrape commands support --json for machine-readable output. When set:
- The ASCII banner is hidden
- Spinners and progress indicators are suppressed
- Interactive prompts are disabled
- Only minified JSON is written to stdout
This makes just-scrape easy to use in shell scripts, CI pipelines, and AI agent workflows.
Usage
just-scrape <command> [args] --json
Examples
Save results to a file
just-scrape extract https://store.example.com \
-p "Extract all product names and prices" \
--json > products.json
just-scrape credits --json | jq '.remaining'
just-scrape history extract --json | jq '.[].status'
Convert a page to markdown and save it
just-scrape scrape https://docs.example.com/api \
--json | jq -r '.results.markdown.data[0]' > api-docs.md
Chain commands in a shell script
#!/bin/bash
while IFS= read -r url; do
just-scrape extract "$url" \
-p "Extract the page title and main content" \
--json >> results.jsonl
done < urls.txt
Response shapes
Each command prints the SDK response as minified JSON. Common shapes:
scrape — returned data keyed by requested format:
{
"id": "25554af4-8c01-4d9a-890e-d7658d57dc93",
"results": {
"markdown": { "data": ["# Example Domain\n..."] }
},
"metadata": { "contentType": "text/html" }
}
extract — structured json payload plus token usage:
{
"id": "515233e0-b26c-42f3-b4c9-2e993af15546",
"raw": null,
"json": { "title": "Example Domain" },
"usage": { "promptTokens": 359, "completionTokens": 113 },
"metadata": {}
}
credits — balance and per-job quotas:
{
"remaining": 749575,
"used": 712,
"plan": "Pro Plan",
"jobs": {
"crawl": { "used": 0, "limit": 50 },
"monitor": { "used": 1, "limit": 100 }
}
}
validate — health check:
{ "status": "ok", "uptime": 256372 }
--json is especially useful when calling just-scrape from AI coding agents — it eliminates decorative output and saves tokens.
