Overview

SearchScraper is our advanced LLM-powered search service that intelligently searches and aggregates information from multiple web sources. Using state-of-the-art language models, it understands your queries and extracts relevant information across the web, providing comprehensive answers with full source attribution.

Try SearchScraper instantly in our interactive playground - no coding required!

Getting Started

Quick Start

from scrapegraph_py import Client

client = Client(api_key="your-api-key")

# Execute search with schema
response = client.searchscraper(
    user_prompt="What are the key features and pricing of ChatGPT Plus?",
)

Get your API key from the dashboard

Key Features

Multi-Source Search

Intelligent search across multiple reliable web sources

AI Understanding

Advanced LLM models for accurate information extraction

Structured Output

Clean, structured data in your preferred format

Source Attribution

Full transparency with reference URLs

Use Cases

Research & Analysis

  • Academic research and fact-finding
  • Market research and competitive analysis
  • Technology trend analysis
  • Industry insights gathering

Data Aggregation

  • Product research and comparison
  • Company information compilation
  • Price monitoring across sources
  • Technology stack analysis

Content Creation

  • Fact verification and citation
  • Content research and inspiration
  • Data-driven article writing
  • Knowledge base building

Want to learn more about our AI-powered search technology? Visit our main website to discover how we’re revolutionizing web research.

Advanced Usage

Custom Schema Example

Define exactly what data you want to extract using Pydantic or Zod:

Async Support

For applications requiring asynchronous execution:

from scrapegraph_py import AsyncClient
import asyncio

async def main():
    async with AsyncClient(api_key="your-api-key") as client:
        
        response = await client.searchscraper(
            user_prompt="Analyze the current AI chip market",
        )
        
        # Process the structured results
        market_data = response.result
        print(f"Market Size: {market_data['market_overview']['total_size']}")
        print(f"Growth Rate: {market_data['market_overview']['growth_rate']}")
        print("\nKey Players:")
        for player in market_data['market_overview']['key_players']:
            print(f"- {player}")

# Run the async function
asyncio.run(main())

Integration Options

Official SDKs

  • Python SDK - Perfect for data science and backend applications
  • JavaScript SDK - Ideal for web applications and Node.js

AI Framework Integrations

Best Practices

Query Optimization

  1. Be specific in your prompts
  2. Use descriptive queries
  3. Include relevant context
  4. Specify time-sensitive requirements

Schema Design

  • Start with essential fields
  • Use appropriate data types
  • Add field descriptions
  • Make optional fields nullable
  • Group related information

Rate Limiting

  • Implement reasonable delays between requests
  • Use async clients for better performance
  • Monitor your API usage

Example Projects

Check out our cookbook for real-world examples:

API Reference

For detailed API documentation, see:

Support & Resources

Ready to Start?

Sign up now and get your API key to begin searching and extracting data with SearchScraper!