Markdownify

​

Overview

Markdownify is our specialized service that transforms web content into clean, well-formatted markdown. It intelligently preserves the content’s structure while removing unnecessary elements, making it perfect for content migration, documentation creation, and knowledge base building.

​

Getting Started

​

Quick Start

from scrapegraph_py import Client

client = Client(api_key="your-api-key")

response = client.markdownify(
    website_url="https://example.com/article"
)

​

Parameters

​

Key Features

​

Use Cases

​

Content Migration

• Convert blog posts to markdown
• Transform documentation
• Migrate knowledge bases
• Archive web content

​

Documentation

• Create technical documentation
• Build wikis and guides
• Generate README files
• Maintain developer docs

​

Content Management

• Prepare content for CMS import
• Create portable content
• Build learning resources
• Format articles

​

Other Functionality

​

Retrieve a previous request

If you know the response id of a previous request you made, you can retrieve all the information.

import { getMarkdownifyRequest } from 'scrapegraph-js';

const apiKey = 'your_api_key';
const requestId = 'ID_of_previous_request';

try {
  const requestInfo = await getMarkdownifyRequest(apiKey, requestId);
  console.log(requestInfo);
} catch (error) {
  console.error(error);
}

​

Parameters

​

Async Support

For applications requiring asynchronous execution, Markdownify provides async support through the AsyncClient. Here’s a basic example:

from scrapegraph_py import AsyncClient
import asyncio

async def main():
    async with AsyncClient(api_key="your-api-key") as client:
        response = await client.markdownify(
            website_url="https://example.com/article"
        )
        print(response)

# Run the async function
asyncio.run(main())

For more advanced concurrent processing, you can use the following example:

import asyncio
from scrapegraph_py import AsyncClient
from scrapegraph_py.logger import sgai_logger

sgai_logger.set_logging(level="INFO")

async def main():
    # Initialize async client
    sgai_client = AsyncClient(api_key="your-api-key-here")

    # Concurrent markdownify requests
    urls = [
        "https://scrapegraphai.com/",
        "https://github.com/ScrapeGraphAI/Scrapegraph-ai",
    ]

    tasks = [sgai_client.markdownify(website_url=url) for url in urls]

    # Execute requests concurrently
    responses = await asyncio.gather(*tasks, return_exceptions=True)

    # Process results
    for i, response in enumerate(responses):
        if isinstance(response, Exception):
            print(f"\nError for {urls[i]}: {response}")
        else:
            print(f"\nPage {i+1} Markdown:")
            print(f"URL: {urls[i]}")
            print(f"Result: {response['result']}")

    await sgai_client.close()

if __name__ == "__main__":
    asyncio.run(main())

This advanced example demonstrates:

• Concurrent processing of multiple URLs
• Error handling for failed requests
• Proper client cleanup
• Logging configuration

​

Integration Options

​

Official SDKs

• Python SDK - Perfect for automation and content processing
• JavaScript SDK - Ideal for web applications and content tools

​

AI Framework Integrations

• LangChain Integration - Use Markdownify in your content pipelines
• LlamaIndex Integration - Create searchable knowledge bases

​

Best Practices

​

Content Optimization

1. Verify source content quality
2. Check image and link preservation
3. Review markdown formatting
4. Validate output structure

​

Processing Tips

• Handle large content in chunks
• Preserve important metadata
• Maintain content hierarchy
• Check for formatting consistency

​

Example Projects

Check out our cookbook for real-world examples:

• Blog migration tools
• Documentation generators
• Content archival systems
• Knowledge base builders

​

API Reference

For detailed API documentation, see:

• Start Conversion Job
• Get Job Status

​

Support & Resources