Skip to main content

Overview

Firecrawl is a powerful web scraping API that converts websites into clean, LLM-ready markdown or structured data.

Installation

pip install "praisonai[tools]"

Environment Variables

export FIRECRAWL_API_KEY=your_api_key_here
Get your API key from Firecrawl.

Quick Start

from praisonai_tools import FirecrawlTool

# Initialize
firecrawl = FirecrawlTool()

# Scrape a page
result = firecrawl.scrape("https://example.com")
print(result)

Usage with Agent

from praisonaiagents import Agent
from praisonai_tools import FirecrawlTool

agent = Agent(
    name="WebScraper",
    instructions="You are a web scraping assistant. Use Firecrawl to extract content.",
    tools=[FirecrawlTool()]
)

response = agent.chat("Scrape the content from https://docs.praison.ai")
print(response)

Available Methods

scrape(url)

Scrape a single URL and get markdown content.
from praisonai_tools import FirecrawlTool

firecrawl = FirecrawlTool()
result = firecrawl.scrape("https://example.com")

# Returns:
# {
#     "url": "https://example.com",
#     "markdown": "# Example Domain\n\nThis domain is...",
#     "metadata": {...}
# }

crawl(url, limit=10)

Crawl a website and get multiple pages.
results = firecrawl.crawl("https://docs.example.com", limit=5)

# Returns list of scraped pages

Configuration Options

firecrawl = FirecrawlTool(
    api_key="your_key",           # Optional: defaults to FIRECRAWL_API_KEY
    formats=["markdown", "html"], # Output formats
    only_main_content=True        # Extract only main content
)

Function-Based Usage

from praisonai_tools import firecrawl_scrape

# Quick scrape without instantiating class
result = firecrawl_scrape("https://example.com")

CLI Usage

# Set API key
export FIRECRAWL_API_KEY=your_key

# Use with praisonai
praisonai --tools FirecrawlTool "Scrape the content from https://example.com"

Error Handling

from praisonai_tools import FirecrawlTool

firecrawl = FirecrawlTool()
result = firecrawl.scrape("https://example.com")

if "error" in result:
    print(f"Error: {result['error']}")
else:
    print(f"Content: {result['markdown'][:500]}")

Common Errors

ErrorCauseSolution
FIRECRAWL_API_KEY not configuredMissing API keySet environment variable
firecrawl not installedMissing dependencyRun pip install firecrawl-py
Rate limitedToo many requestsUpgrade plan or add delays