NPM Package

License

Installation

Install the package using npm or yarn:

# Using npm
npm i scrapegraph-js

# Using yarn
yarn add scrapegraph-js

Features

  • AI-Powered Extraction: Smart web scraping with artificial intelligence
  • Async by Design: Fully asynchronous architecture
  • Type Safety: Built-in TypeScript support with Zod schemas
  • Production Ready: Automatic retries and detailed logging
  • Developer Friendly: Comprehensive error handling

Quick Start

Initialize with your API key:

import { smartScraper } from 'scrapegraph-js';

const apiKey = process.env.SGAI_APIKEY;
const websiteUrl = 'https://example.com';
const prompt = 'Extract the main heading and description';

try {
  const response = await smartScraper(apiKey, websiteUrl, prompt);
  console.log(response.result);
} catch (error) {
  console.error('Error:', error);
}

Store your API keys securely in environment variables. Use .env files and libraries like dotenv to load them into your app.

Services

SmartScraper

Extract specific information from any webpage using AI:

const response = await smartScraper(
  apiKey,
  'https://example.com',
  'Extract the main content'
);

LocalScraper

Process local HTML content with AI extraction:

const html = `
<html>
    <body>
        <h1>Company Name</h1>
        <p>We are a technology company focused on AI solutions.</p>
        <div class="contact">
            <p>Email: contact@example.com</p>
        </div>
    </body>
</html>
`;

const response = await localScraper(
  apiKey,
  html,
  'Extract the company description'
);

Markdownify

Convert any webpage into clean, formatted markdown:

const response = await markdownify(
  apiKey,
  'https://example.com'
);

API Credits

Check your available API credits:

import { getCredits } from 'scrapegraph-js';

try {
  const credits = await getCredits(apiKey);
  console.log('Available credits:', credits);
} catch (error) {
  console.error('Error fetching credits:', error);
}

Feedback

Help us improve by submitting feedback programmatically:

import { sendFeedback } from 'scrapegraph-js';

try {
  await sendFeedback(
    apiKey,
    'request-id',
    5,
    'Great results!'
  );
} catch (error) {
  console.error('Error sending feedback:', error);
}

Support