Crawlbase

@Crawlbase fetch the homepage of example.com and show me the page title and meta description

Crawlbase retrieved the page successfully. The target returned HTTP 200. Page title: 'Example Domain'. Meta description: 'Example Domain. This domain is for use in illustrative examples in documents.' The full HTML content is available if you need to extract additional elements. This example demonstrates a straightforward public URL crawl using the Normal token.

This read-heavy example shows Crawlbase's core crawling capability. Requires a valid Normal API token with sufficient credits. The MCP returns both the target site's HTTP status and Crawlbase's own status, so you can distinguish between a successful crawl of a 404 page versus a failed crawl attempt.

@Crawlbase I need to delete the stored pages with request IDs req_abc123 and req_def456 from cloud storage

Deletion complete. Request ID req_abc123: successfully deleted. Request ID req_def456: successfully deleted. Both pages have been irreversibly removed from Crawlbase Cloud Storage. Your storage count has decreased by 2. This example illustrates the destructive write operation for managing stored crawl results.

This write example demonstrates permanent storage management. Deletion is irreversible—there is no undo or recovery mechanism. Use cautiously in production workflows. You must provide exact request IDs; the MCP will report per-ID outcomes if some deletions fail while others succeed.

@Crawlbase retrieve the last 20 stored pages, then summarize which domains appear most frequently and flag any pages that returned non-200 status codes

Retrieved 20 stored pages. Domain frequency: example.com (8 pages), testsite.org (5 pages), demo.net (4 pages), sample.io (3 pages). Non-200 statuses flagged: 2 pages returned 404 (req_xyz789 from example.com/missing, req_uvw012 from demo.net/gone), 1 page returned 503 (req_rst345 from testsite.org/api). This synthesis pairs Crawlbase's storage retrieval with AI analysis to surface patterns across your crawl history.

This synthesis example combines the MCP's batch retrieval tool with the AI's reasoning to extract insights from stored crawl data. The MCP returns up to 100 pages per call; larger datasets require pagination using continuation cursors. Useful for auditing crawl campaigns or identifying problematic URLs before re-crawling.

When Crawlbase beats in-house scraping for weekly price checks

A three-person e-commerce team needs to track competitor pricing across 40 product pages every Monday morning. Building a scraper in-house means managing proxies, handling CAPTCHAs, and debugging when sites change their HTML. Crawlbase handles the infrastructure: the Crawl URL tool fetches pages through rotating IPs, the storage tools let you archive snapshots for trend analysis, and the user-agent generator keeps requests looking organic. The API key auth means anyone on the team can trigger crawls from Switchy without sharing credentials. This setup works until you hit 10,000+ pages per month—at that scale, the per-request cost adds up and you'll want to evaluate self-hosted alternatives. If your pricing check is under 2,000 pages monthly and you'd rather spend eng time on product than proxy rotation, Crawlbase is the faster path to reliable data.

Crawlbase for quarterly documentation snapshots at small SaaS

A five-person support team at a B2B SaaS company needs to keep internal runbooks synced with vendor documentation that lives on public sites. Every quarter, they crawl 15-20 integration partner docs to catch API changes before customers report broken workflows. The Get Storage Count and List Stored Page IDs tools let the team audit what's already archived, then Crawl URL fetches fresh versions and stores them for diff comparison. Delete Stored Pages cleans up obsolete snapshots after the review cycle. This scenario works because the volume is low (under 100 pages per quarter) and the cadence is predictable. If your team needs real-time monitoring or you're crawling hundreds of pages weekly, you'll outgrow this pattern and need a dedicated change-detection service. For quarterly syncs where the alternative is manual copy-paste into Notion, Crawlbase keeps the workflow in one place.

When Crawlbase isn't the right call for prospect enrichment

A four-person sales team wants to scrape LinkedIn profiles and company websites to build lead lists for cold outreach. Crawlbase can technically fetch public pages, but this scenario hits two hard limits: LinkedIn aggressively blocks scrapers (even through proxies), and the eight-tool set has no structured data extraction—you get raw HTML back, not parsed contact info. The Generate User Agents tool helps with basic anti-bot evasion, but you're still writing your own parsers and handling rate limits manually. For lead enrichment, a purpose-built tool like Apollo or ZoomInfo delivers structured records without the scraping overhead. Crawlbase makes sense when you need the raw page content for a custom workflow (say, archiving press mentions for a quarterly board deck), but if your goal is a CSV of emails and job titles, start with a data provider instead of a general-purpose crawler.