Crawlbase
Crawlbase provides APIs for crawling web pages, managing crawler queues, storing crawl results, and monitoring account usage.
Integration
- Vendor
- Crawlbase
- Category
- other
- Auth
- API_KEY
- Tools
- 8
- Composio slug
crawlbase
Tools
- Crawl URL
Fetch a public HTTP or HTTPS URL through Crawlbase using the connected Normal token and return its content with separate target and Crawlbase statuses.
- Delete Stored Pagesdestructive
Irreversibly delete explicitly selected pages from Crawlbase Cloud Storage by request ID and return each page's deletion outcome.
- Generate User Agents
Generate one to ten realistic randomized User-Agent strings for a selected device class.
- Get Crawling Usage
Return current Crawling API usage and optionally previous-month statistics for the connected Crawlbase account.
- Get Storage Count
Return the number of pages currently held in Crawlbase Cloud Storage.
- Get Stored Page
Retrieve the latest Crawlbase Cloud Storage page by exactly one request ID or original URL.
- Get Stored Pages
Retrieve up to 100 Crawlbase Cloud Storage pages by request ID without deleting them. Returns decoded page content and identifies requested IDs that were not found.
- List Stored Page IDs
Return one page of Crawlbase Cloud Storage request IDs and a short-lived continuation cursor.
Setup
- Create or open a Switchy Project.
- Open Integrations and connect Crawlbase.
- Authorise through api_key. Switchy stores the connection against your org.
- Everyone in the Project can now @-mention the Crawlbase tools in chat.
What teammates see: by default, memories from Crawlbase are scoped to the Project (PROJECT visibility) - you can mark any memory PRIVATE or share it ORG-wide.
Works well with
Top models
Compatibility data appears once enough Projects have used this MCP together with a given model.