Scrapfly
Scrapfly is a web scraping API that enables developers to extract data from websites efficiently, offering features like JavaScript rendering, anti-bot protection bypass, and proxy rotation.
Verdict
Common use cases
- Extract product prices from e-commerce sites
- Monitor competitor landing pages for changes
- Build lead lists from public directories
- Aggregate news articles by topic daily
- Validate contact info on company websites
Integration
- Vendor
- Scrapfly
- Category
- other
- Auth
- API_KEY
- Composio slug
scrapfly
Tools
Setup
Setup guide
- 11. Open your Switchy workspace settings and navigate to the Integrations page. 2. Find Scrapfly in the MCP directory and click Connect. 3. Log into your Scrapfly account at scrapfly.io and go to the API Keys section of your dashboard. 4. Generate a new API key with scraping permissions and copy it. 5. Paste the key into Switchy's connection dialog and click Authorize. 6. Open any Space and type '@Scrapfly scrape https://example.com' to test the connection. 7. If the MCP returns structured HTML or JSON, the integration is live. 8. Check Scrapfly's usage dashboard to confirm requests are logged under your account. 9. Adjust your plan's concurrency and request limits if you hit rate errors during heavy scraping sessions.
What teammates see: by default, memories from Scrapfly are scoped to the Space (PROJECT visibility) - you can mark any memory PRIVATE or share it ORG-wide.
Works well with
Top models
Compatibility data appears once enough Spaces have used this MCP together with a given model.
How Switchy teams use it
Starter prompts
Scrape Product Catalog
@Scrapfly scrape the product catalog at https://example-store.com/products and return a table with name, price, and stock status for each item.Open in a Space →
Monitor Pricing Changes
@Scrapfly scrape https://competitor.com/product/12345 and compare the current price to the last recorded price, then highlight any changes.Open in a Space →
Extract Contact Details
@Scrapfly scrape https://targetcompany.com/contact and extract all email addresses and phone numbers into a list.Open in a Space →
Aggregate News Headlines
@Scrapfly scrape https://newssite.com and return the top 10 headlines with their publication times and short summaries.Open in a Space →
Check Page Availability
@Scrapfly check if https://example.com/landing-page is reachable and report the HTTP status code and page load time.Open in a Space →
Example outputs
Illustrative - representative of the model's voice and quality, not literal recordings.
@Scrapfly extract the main article text and publish date from this news URL: https://example.com/article/tech-layoffs-2024
Scrapfly returned structured data from the target page: article headline 'Tech Industry Faces New Wave of Layoffs', body text (approximately 800 words), publish date January 15, 2024, author byline, and two featured images. The extraction bypassed JavaScript rendering and anti-bot measures without triggering blocks. This example shows a successful scrape; actual results depend on the target site's structure and defenses.
This example demonstrates Scrapfly's core extraction capability — pulling clean content from pages that typically block automated requests. Requires an API key with sufficient credits. Scrapfly handles rendering and proxy rotation, but you're responsible for respecting robots.txt and the site's terms of service.
@Scrapfly monitor this product page every 6 hours and alert me when the price drops below $299: https://example.com/shop/laptop-model-x
Scrapfly configured a monitoring job for the specified URL with a 6-hour interval. The service will extract the price element on each run and compare it against the $299 threshold. When the condition is met, Scrapfly sends a webhook notification to your configured endpoint. This example illustrates scheduled monitoring; the actual alert mechanism depends on your webhook setup and Scrapfly plan limits.
This example shows Scrapfly's monitoring feature for tracking page changes over time. Monitoring jobs consume credits on each run, and the frequency you can set depends on your subscription tier. You'll need to configure a webhook endpoint separately to receive alerts — Scrapfly doesn't send email notifications directly.
@Scrapfly scrape the top 50 search results from this e-commerce site for 'wireless headphones' and return product titles, prices, and ratings in a structured format
Scrapfly executed a batch scrape across 50 product listings, returning a JSON array with each item's title, price (parsed as a number), star rating, and product URL. The job took approximately 45 seconds and used JavaScript rendering to handle lazy-loaded content. This example shows bulk extraction; actual runtime and success rate vary based on the site's pagination structure and rate-limiting behavior.
This example highlights Scrapfly's ability to handle multi-page scraping with structured output. Batch jobs consume more credits and take longer than single-page requests. If the target site has aggressive rate limits, you may need to adjust concurrency settings or spread requests over time to avoid blocks.
Use-case deep-dives
When Scrapfly makes sense for weekly price checks
A 3-person e-commerce team needs to track competitor pricing across 20 SKUs every Monday morning. Scrapfly handles the anti-bot bypassing and proxy rotation that breaks most DIY scrapers, so the team can pull clean price data without maintaining infrastructure. The API key setup takes 10 minutes. The trade-off: if you're only scraping one or two sites with stable HTML, a simpler tool like Puppeteer might be cheaper. But once you hit sites with Cloudflare or rate limits, Scrapfly pays for itself by not burning engineering hours on captcha workarounds. If your pricing strategy depends on reliable weekly data from 5+ competitor sites, this MCP is the right call.
Scrapfly for one-time bulk contact extraction
A 2-person agency needs to pull 500 business contacts from a public directory to seed a cold outreach campaign. Scrapfly's rendering engine handles JavaScript-heavy pages that standard scrapers miss, and the session management keeps the scrape from getting blocked mid-run. You configure the target URLs, run the extraction once, and export to CSV. The scenario breaks down if the directory changes its DOM structure weekly—Scrapfly won't auto-adapt your selectors. It also gets expensive if you're scraping tens of thousands of pages monthly. For a one-time or quarterly bulk pull where blocking is the main risk, Scrapfly delivers clean data without the trial-and-error of open-source tools.
When Scrapfly beats manual research for trend analysis
A 5-person product team wants to aggregate blog posts and forum threads about a new feature category every quarter. Scrapfly's headless browser mode captures content from sites that block standard HTTP requests, and the built-in caching prevents redundant fetches when you refine your queries. The team sets up a shared Switchy workspace with the Scrapfly MCP, runs the scrape, and drops results into a shared doc for tagging. The limit: if your research needs real-time monitoring (hourly or daily), Scrapfly's per-request pricing adds up fast. For quarterly deep-dives where you need 100-300 pages of clean text from 10+ sources, this MCP saves 4-6 hours of copy-paste work per research cycle.
Frequently asked
What does the Scrapfly MCP do in Switchy?
It connects Switchy to Scrapfly's web scraping infrastructure, letting your team extract structured data from websites without managing proxies or anti-bot systems. You configure scraping jobs through the MCP, and results flow into your Switchy workspace where AI agents can process them. Useful for competitive research, lead generation, or monitoring public data sources your team needs regularly.
Do I need a paid Scrapfly account to use this MCP?
Yes. The MCP authenticates with a Scrapfly API key, which requires an active Scrapfly subscription. Free trials work if Scrapfly offers one, but you'll hit rate limits quickly. The API key lives in Switchy's credential store — anyone on your team using the MCP shares the same Scrapfly account quota, so coordinate usage to avoid burning through credits unexpectedly.
Can the Scrapfly MCP scrape JavaScript-heavy sites like LinkedIn?
It depends on what Scrapfly's API supports, not the MCP itself. Scrapfly handles JavaScript rendering and anti-bot evasion at the service level. The MCP just passes your scraping parameters to Scrapfly and returns the results. Check Scrapfly's documentation for which sites they successfully scrape — the MCP won't add capabilities Scrapfly's backend doesn't already provide.
How is using this MCP different from calling Scrapfly's API directly?
The MCP wraps Scrapfly's API so AI agents in Switchy can trigger scrapes conversationally and process results in-context. If you're already writing Python scripts that call Scrapfly, you don't gain much. But if your team wants to say 'scrape competitor pricing from X' and have an agent handle the job without leaving Switchy, the MCP saves that integration work.
Who on the team should connect the Scrapfly MCP?
Whoever owns your Scrapfly account and understands your scraping budget. They'll paste the API key into Switchy once, then any team member can use the MCP. Monitor usage closely — scraping costs scale with volume and complexity, and Scrapfly bills per request. If multiple people trigger jobs, you'll want someone watching the spend dashboard.