Scrapingbee
ScrapingBee is a web scraping API that handles headless browsers and proxy rotation, allowing developers to extract HTML from any website in a single API call.
Verdict
Common use cases
- Track competitor pricing across e-commerce sites
- Aggregate job postings from multiple boards
- Monitor review site sentiment for products
- Extract lead data from directory listings
- Scrape event schedules from conference sites
Integration
- Vendor
- Scrapingbee
- Category
- other
- Auth
- API_KEY
- Tools
- 5
- Composio slug
scrapingbee
Tools
- ScrapingBee Data Extraction
Tool to extract structured data from a webpage using css or xpath selectors. use scrapingbee's extract rules feature.
- ScrapingBee HTML Fetch
Tool to fetch html or screenshot via scrapingbee html api. use when you need page markup or image after optional js rendering and resource controls.
- ScrapingBee Proxy Mode
Tool to fetch web content via scrapingbee's proxy mode. use when you need to route requests through scrapingbee proxies with optional js rendering and resource blocking.
- ScrapingBee Stealth Proxy
Tool to perform stealth scraping via scrapingbee's stealth proxy mode. use when you encounter anti-bot measures requiring undetectable requests.
- ScrapingBee Usage Stats
Tool to retrieve usage statistics for your scrapingbee account. use when you need to monitor remaining credits and request count.
Setup
Setup guide
- 11. Log into your ScrapingBee account at app.scrapingbee.com and copy your API key from the dashboard. 2. In Switchy, open your workspace settings and navigate to the Integrations tab. 3. Click 'Add MCP Integration' and select ScrapingBee from the list. 4. Paste your API key into the authentication field and click 'Connect'. 5. Switchy will verify the key by fetching your account usage stats — if successful, you'll see a green checkmark. 6. Open any Space and type '@ScrapingBee fetch the HTML from example.com' to test the connection. 7. The MCP will return the page markup or confirm the request succeeded. 8. For structured extraction, use '@ScrapingBee extract product prices from [URL] using CSS selector .price' to pull specific data points. 9. Monitor your remaining credits by asking '@ScrapingBee show my usage stats' before running bulk operations.
What teammates see: by default, memories from Scrapingbee are scoped to the Space (PROJECT visibility) - you can mark any memory PRIVATE or share it ORG-wide.
Works well with
Top models
Compatibility data appears once enough Spaces have used this MCP together with a given model.
How Switchy teams use it
Starter prompts
Extract Pricing Data
@ScrapingBee extract all prices from https://example.com/products using the CSS selector '.product-price' and return them as a listOpen in a Space →
Fetch Rendered Page
@ScrapingBee fetch the rendered HTML from https://example.com/dashboard with JavaScript execution enabled and return the complete markupOpen in a Space →
Bypass Anti-Bot Protection
@ScrapingBee use stealth proxy mode to fetch content from https://example.com/protected-page and return the HTML without triggering detectionOpen in a Space →
Check Credit Balance
@ScrapingBee show my current usage statistics including remaining credits and total requests this monthOpen in a Space →
Extract Structured Data
@ScrapingBee extract job titles and company names from https://example.com/jobs using XPath //h2[@class='job-title'] and //span[@class='company']Open in a Space →
Example outputs
Illustrative - representative of the model's voice and quality, not literal recordings.
@scrapingbee extract the product title, price, and availability from this e-commerce page: https://example-shop.com/product/xyz using CSS selectors
I've extracted the structured data from that product page. Here's what I found: • Product Title: "Premium Wireless Headphones - Noise Cancelling" • Price: "$149.99" • Availability: "In Stock - Ships within 2 business days" The extraction used CSS selectors to target specific elements after JavaScript rendering. The page loaded successfully through ScrapingBee's proxy network, bypassing basic bot detection. All three data points were present in the DOM and extracted cleanly.
This example shows ScrapingBee's structured extraction capability — useful when you need specific fields rather than full HTML. Requires defining CSS or XPath selectors upfront. Works best on stable page structures; dynamic sites may need selector adjustments. Each extraction consumes API credits based on page complexity and JS rendering requirements.
@scrapingbee fetch the full HTML from https://news-site.example.com/article/123 with JavaScript rendering enabled and block all images to save credits
I've retrieved the rendered HTML from that news article. The page is approximately 85KB after JavaScript execution, with all image resources blocked as requested. The content includes: • Full article text (2,400 words) • Metadata tags (author, publish date, categories) • Embedded social share buttons (rendered) • Comment section structure (12 comments loaded) By blocking images, this request used ~40% fewer credits than a full render. The HTML is ready for further parsing or analysis.
This demonstrates resource control during scraping — blocking images, CSS, or fonts reduces credit consumption when you only need text content. JavaScript rendering adds latency (2-5 seconds typical) but captures dynamically loaded content. Useful for paywalled or SPA sites where static HTML fetching fails. Monitor your credit balance with heavy JS rendering.
@scrapingbee use stealth mode to scrape https://protected-site.example.com/data and summarize the main data table you find, then check my remaining API credits
Using stealth proxy mode, I successfully accessed the protected page and located a data table with 47 rows. Here's a summary: **Table Contents:** Monthly sales figures by region (Jan-Dec 2024) **Key Findings:** North America led with $2.3M average monthly revenue, followed by Europe at $1.8M. Q4 showed 23% growth across all regions. **Data Quality:** Complete dataset, no missing values. **Your ScrapingBee Account Status:** • Remaining Credits: 8,340 of 10,000 • Requests This Month: 1,660 • This stealth request consumed 25 credits (premium tier) You have sufficient credits for ~330 more stealth requests this billing cycle.
This example combines stealth scraping with AI reasoning and account monitoring — a realistic workflow when dealing with anti-bot protection. Stealth mode costs 5-10x regular credits but bypasses sophisticated detection. The usage check helps prevent mid-task credit exhaustion. Note that even stealth mode can't guarantee access to all sites; some employ server-side blocking that proxies can't circumvent.
Use-case deep-dives
When Scrapingbee beats manual price checks for small shops
A 3-person Shopify store tracks competitor pricing on 40 SKUs across 6 retail sites twice a week. Scrapingbee's Data Extraction tool pulls structured price data using CSS selectors, and the Stealth Proxy handles sites with bot detection. The team runs a scheduled workflow in Switchy that updates a shared spreadsheet—no dev time, no proxy management. This works until you hit ~200 SKUs or need sub-hourly updates; at that scale, the credit burn gets expensive and you're better off building a dedicated scraper. If your pricing intel fits a twice-weekly cadence and you don't have an engineer on payroll, Scrapingbee keeps you competitive without infrastructure overhead.
Scrapingbee for sales teams scraping LinkedIn or directories
A 5-person sales team enriches inbound leads by pulling public LinkedIn profiles and company directory pages. Scrapingbee's HTML Fetch tool renders JavaScript-heavy pages, and the Proxy Mode rotates IPs to avoid rate limits. The team pastes a lead URL into Switchy, and the MCP returns structured contact data in seconds. This breaks down if you're scraping thousands of profiles daily—LinkedIn's anti-bot measures escalate fast, and even stealth mode hits walls at volume. For under 50 profiles a day, Scrapingbee handles the complexity so your AE doesn't waste an hour Googling people. Beyond that threshold, you need a compliance review and a dedicated enrichment vendor.
When Scrapingbee wins for one-off research sprints
A product manager runs a quarterly competitive analysis, scraping feature pages and pricing tables from 15 SaaS competitors. Scrapingbee's Data Extraction tool pulls structured content using XPath selectors, and the Usage Stats tool keeps the PM aware of credit burn. The entire sprint takes 3 hours in Switchy instead of 2 days of manual copy-paste. This is the sweet spot: infrequent, high-value scraping where you need reliability but don't want to maintain a scraper. If you're doing this monthly or need real-time monitoring, the per-request cost adds up and you should evaluate a persistent solution. For quarterly deep-dives, Scrapingbee turns research grunt work into a repeatable workflow.
Frequently asked
What does the Scrapingbee MCP do in Switchy?
It lets your AI agents scrape web pages that block normal requests — handling JavaScript rendering, anti-bot detection, and proxy rotation automatically. The MCP wraps Scrapingbee's API so agents can extract structured data, fetch HTML, or grab screenshots without writing scraper code. Useful when you need to pull competitor pricing, monitor site changes, or gather research data that isn't available via official APIs.
Do I need a Scrapingbee account to use this MCP?
Yes. You'll need an active Scrapingbee subscription and an API key. During setup, Switchy prompts you to paste the key — no OAuth flow. The MCP charges credits against your Scrapingbee plan for each request, so check your tier's monthly allowance before connecting. Free trial keys work, but expire after 1,000 credits.
Can the MCP scrape sites that require login or cookies?
No. The MCP doesn't handle session management or authenticated scraping. It's designed for public pages that use anti-bot measures like Cloudflare or rate limiting. If you need to scrape behind a login, you'll have to use Scrapingbee's API directly with custom headers and cookie forwarding — the MCP doesn't expose those parameters.
How is this different from just calling Scrapingbee's API?
The MCP gives your AI agents a natural-language interface to scraping — they can request data extraction or stealth mode without you writing HTTP calls or parsing responses. You lose some fine-grained control (custom headers, webhook callbacks) but gain speed for ad-hoc research tasks. If you're building a production scraper, use the API directly.
Does Scrapingbee usage count against my Switchy plan limits?
No. Scrapingbee charges are separate — each scrape request deducts credits from your Scrapingbee account, not Switchy. The MCP itself doesn't consume Switchy seats or storage. However, large HTML responses do count toward your workspace's monthly MCP data transfer quota, so scraping 10,000 pages will eat into that allowance.