Webscraper io

@webscraper-io list all my sitemaps and show me which ones have scheduled jobs enabled

Found 4 sitemaps in your account. Two have active schedules: 'competitor-pricing' (runs daily at 6am UTC) and 'job-listings' (runs every 6 hours). The other two — 'product-catalog' and 'news-archive' — are manual-only. Your account shows 12,450 page credits remaining. This is an illustrative example of what the MCP would return after querying your Web Scraper Cloud account.

This example demonstrates read-only account inspection using the Get Sitemaps and Get Account Info tools. Requires a valid API key with read permissions. Useful for auditing scraping infrastructure before making changes or checking credit balance before launching new jobs.

@webscraper-io create a new sitemap called 'tech-blogs' that scrapes article titles and publish dates from example.com/blog starting at the homepage

Created sitemap 'tech-blogs' (ID: sm_7k2p9x). Configuration includes start URL at example.com/blog with selectors for article titles (CSS: .post-title) and publish dates (CSS: .post-date). The sitemap is saved but not yet scheduled — you'll need to manually trigger a scraping job or enable scheduling separately. This illustrative response shows what happens after the Create Sitemap tool executes successfully.

This write operation uses the Create Sitemap tool to define scraping structure. The MCP creates the configuration but does NOT automatically run a scrape or enable scheduling — those are separate actions. Be aware that creating sitemaps with overly broad selectors can waste page credits on irrelevant data.

@webscraper-io check the status of all scraping jobs from the last 48 hours and summarize any failures with error messages

Reviewed 23 scraping jobs from the past two days. Twenty completed successfully. Three failed: Job #8821 on 'competitor-pricing' hit a 403 Forbidden error (likely IP block), Job #8834 on 'job-listings' timed out after 15 minutes, and Job #8847 on 'news-archive' encountered malformed HTML that broke the selector. All failures occurred on sitemaps without rate-limiting configured. This illustrative summary combines data from Get Scraping Jobs with AI analysis to surface actionable issues.

This synthesis example pairs the Get Scraping Jobs tool with the AI's reasoning to diagnose patterns across multiple jobs. The MCP provides raw job data (status, timestamps, errors); the AI interprets it. Useful for operational monitoring, but note that Web Scraper Cloud's error messages vary in specificity depending on failure type.

When scheduled scraping beats manual price checks

A 6-person e-commerce team needs to track competitor pricing across 15 product categories twice daily. Webscraper.io's scheduler tools (Enable/Disable Sitemap Scheduler) let you set cron expressions to run scrapes automatically, feeding fresh data into your pricing model without manual kicks. The sitemap creation tool handles selector rules once, then the jobs run unattended. This works if your competitors' sites are stable and you're scraping under 10k pages per day—beyond that, you'll hit rate limits or need custom proxy rotation. If your pricing decisions happen in real-time (hourly adjustments), this MCP is too slow; you need a streaming solution. But for twice-daily batch updates that feed a dashboard or alert system, the scheduler removes the toil and the API key auth keeps it simple for a single ops person to manage.

When scraping directories scales your outbound pipeline

A 3-person sales team at a B2B SaaS startup scrapes industry directories and conference attendee lists to build lead lists. The Create Sitemap tool defines the extraction rules (company name, contact email, job title), and Get Scraping Jobs tracks progress across multiple directories. This MCP is the right call if you're hitting 5-10 directories per quarter and each has a consistent HTML structure—you define the sitemap once, run the job, export the data to your CRM. It falls apart if the directories require login (Webscraper.io doesn't handle auth flows well) or if the HTML changes weekly (you'll spend more time fixing selectors than scraping). For teams running outbound at scale, this replaces 8-10 hours of manual copy-paste per month and keeps your pipeline full without hiring a data analyst.

When batch scraping feeds your content calendar

A 4-person editorial team at a tech publication scrapes 20 industry blogs weekly to identify trending topics and source article ideas. The Get Sitemaps and Get Sitemap tools let you manage multiple scraping configs (one per blog), and the pagination support handles large result sets without timeouts. This works if you're aggregating text content (headlines, summaries, publish dates) from static sites—Webscraper.io's selectors handle most blog CMSs cleanly. It's not the right tool if you need full-text extraction from paywalled sites or if the blogs are JavaScript-heavy (the scraper doesn't execute JS by default). For a weekly research sprint where you're pulling 500-1000 articles into a spreadsheet to review in a Monday meeting, this MCP delivers the raw material in 10 minutes instead of 3 hours of manual browsing.