VLM Run
VLM Run provides multimodal agents, structured extraction, predictions, files, skills, feedback, and evaluation APIs.
Verdict
Common use cases
- Parse invoices and receipts into accounting JSON
- Transcribe video meetings with speaker attribution
- Extract product specs from catalog images
- Run custom agents over support ticket screenshots
- Build reusable skills from successful extraction sessions
Integration
- Vendor
- VLM Run
- Category
- docs
- Auth
- API_KEY
- Tools
- 11
- Composio slug
vlm_run
Tools
- Create Skill
Create a reusable skill from exactly one uploaded zip, prompt, or chat session.
- Discover Extraction Schemas
List supported structured-extraction domains, or return the full JSON schema for one domain when domain is provided.
- Execute Agent
Start a VLM Run agent execution from an existing agent name or inline configuration over multimodal inputs. Execution may consume credits and is asynchronous by default; poll the returned ID with VLM_RUN_GET_RUN.
- Extract Structured JSON
Start structured JSON extraction from images, a document, a video, or audio using a domain, custom schema, or skill. Extraction may consume credits; document, video, and audio runs are asynchronous by default, so poll the returned predictio
- Find Files
List uploaded files or find one by file ID or MD5 hash. In list mode, use offset and limit until has_more is false.
- Find Skills
List VLM Run skills or find one exact skill by ID, name, and optional version. In list mode, continue from next_offset while has_more is true.
- Get Run
Get the current status and result of one structured-extraction prediction or agent execution; call repeatedly to poll asynchronous work.
- List Agents
Return agents available to the connected account for selection before execution.
- List Artifacts
List artifact metadata belonging to exactly one chat session or agent execution. Use offset and limit to traverse pages until has_more is false.
- List Runs
List structured-extraction predictions or agent executions for the connected account. Use offset and limit to traverse pages until has_more is false.
- Upload File
Upload a local file to VLM Run for extraction, agent input, or skill creation. Retain the returned file ID for tools that consume uploaded files.
Setup
Setup guide
- 11. In Switchy, open Settings → Integrations and click Add MCP Server. 2. Select VLM Run from the catalog. 3. You'll be prompted for an API key — generate one at vlmrun.com/account/api-keys with read and write permissions. 4. Paste the key into Switchy and click Connect. 5. Switchy will verify the connection by listing your available agents. 6. Open any Project and type '@VLM Run list agents' to confirm the MCP responds. 7. Upload a test image or PDF to the Project, then try '@VLM Run extract structured JSON from this invoice using domain receipts' to see extraction in action. 8. If the run is asynchronous, the MCP returns a run ID — ask '@VLM Run get run <ID>' to poll until complete. 9. For long-running tasks, Switchy will automatically poll every few seconds and surface the final result in chat.
What teammates see: by default, memories from VLM Run are scoped to the Project (PROJECT visibility) - you can mark any memory PRIVATE or share it ORG-wide.
Works well with
Top models
Compatibility data appears once enough Projects have used this MCP together with a given model.
How Switchy teams use it
Starter prompts
Extract Invoice Data
@VLM Run extract structured JSON from the attached invoice using domain receipts, then format the output as a table
Transcribe Meeting Video
@VLM Run extract structured JSON from this meeting recording using domain transcription, include speaker names and timestamps
Analyze Product Photos
@VLM Run extract structured JSON from these product photos using a custom schema with fields: color, dimensions_cm, condition, defects
Run Quality Assurance Agent
@VLM Run execute agent named 'brand-qa-agent' on the attached design files and summarize any guideline violations
Create Reusable Skill
@VLM Run create a skill from this chat session named 'invoice-parser-v2' so the team can reuse it next month
Example outputs
Illustrative - representative of the model's voice and quality, not literal recordings.
@VLM Run extract structured data from this invoice PDF using the 'invoice' domain schema and return line items with totals
Extraction run started (ID: run_abc123). The invoice domain schema will parse vendor details, line items, subtotals, tax, and total amount. Polling the run shows completion in ~8 seconds. Extracted JSON includes 7 line items with descriptions, quantities, unit prices, and a final total of $3,847.52. All currency fields are normalized to USD with two decimal precision.
This example shows domain-based extraction—VLM Run ships with pre-built schemas for invoices, receipts, forms, and other common documents. Asynchronous runs consume account credits; you must poll the run ID until status is 'succeeded'. Large multi-page PDFs take longer than single images.
@VLM Run create a skill from this zip file that teaches the model how to read our custom lab report format, then use it to extract patient results
Skill 'lab_report_v1' created from the uploaded zip (skill_xyz789). The skill now appears in your account's skill library. Applying this skill to the patient report image, extraction completes in 5 seconds and returns structured fields: patient ID, test date, biomarker panel (12 values with reference ranges), and technician signature. The custom skill captures domain-specific terminology your team defined in the training examples.
Skills let you teach VLM Run your organization's document formats without writing code. You upload examples (zip, prompt, or chat session), and the platform fine-tunes extraction behavior. Skills are reusable across runs but require upfront setup. This is VLM Run's differentiator for non-standard documents.
@VLM Run execute the 'competitive_analysis' agent on these three competitor landing pages and summarize their pricing tiers in a comparison table
Agent execution started (ID: exec_def456). The 'competitive_analysis' agent processes the three screenshots, extracting pricing tables, feature lists, and CTAs. After 12 seconds, the run returns a markdown table with five pricing tiers per competitor, highlighting differences in storage limits, user seats, and API access. The agent also flags that Competitor B hides enterprise pricing behind a contact form.
Agents are multi-step workflows that chain VLM reasoning with tool calls. This example shows agentic synthesis over multiple inputs—useful when you need cross-document comparison or multi-modal reasoning. Agent runs are asynchronous and credit-metered; complex workflows (video, many images) can take minutes. Check account quotas before large batch jobs.
Use-case deep-dives
When VLM Run beats manual data entry for finance teams
A 6-person finance team closing books on 200+ vendor invoices every month hits the wall on manual Quickbooks entry. VLM Run's Extract Structured JSON tool pulls line items, totals, and tax fields from PDF or image invoices into a schema you define once, then reuse. The domain discovery tool shows pre-built schemas for common invoice formats; if yours is non-standard, you upload five examples and create a skill that locks in your extraction logic. Async runs mean you queue the batch at 4pm and review structured JSON by morning. This works when invoice volume justifies the API-key setup and per-document credit cost—if you're under 50 invoices a month, a VA is still cheaper. For teams drowning in paper at scale, VLM Run turns a two-day close into a half-day reconciliation check.
Using VLM Run agents to route visual bug reports
A 3-person support team at a SaaS startup gets 40 tickets a day, half with screenshots of broken UI states. The Execute Agent tool runs a VLM agent over each image to classify severity (cosmetic vs. blocker), extract visible error codes, and suggest the right engineering squad. You configure the agent once with a prompt template and route logic; the agent reads the screenshot, checks for known error patterns using a skill you trained on past tickets, and writes a structured triage note. Polling the run ID gives you results in under 10 seconds for most images. This makes sense when visual context is the primary signal and your team wastes more than an hour a day deciding who owns what. If tickets are mostly text or your volume is under 20/day, a simpler keyword router beats the API overhead.
When VLM Run's multimodal extraction handles mixed-format archives
A 5-person compliance team audits 80 vendor contracts every quarter—some are scanned PDFs, some are Word exports, a few are video walkthroughs of SaaS admin panels. VLM Run's Extract Structured JSON tool handles images, documents, video, and audio in one API, so you don't juggle three different OCR vendors. You define a schema for liability caps, renewal dates, and data-residency clauses, then batch-upload the archive. Video and document runs are async; you poll with Get Run until all extractions finish, then diff the structured output against your compliance checklist in a spreadsheet. This wins when format chaos is the blocker and your team has API-key access to provision the integration. If all your contracts are already in a single text format, a cheaper document-parser MCP does the job without the multimodal tax.
Frequently asked
What does the VLM Run MCP do in Switchy?
It lets your team extract structured data from images, documents, video, and audio using VLM Run's vision models. You can run pre-built extraction schemas, create reusable skills from prompts or chat sessions, and execute multi-step agents over multimodal inputs. All runs happen server-side and consume VLM Run credits from your account.
Do I need a VLM Run API key to connect this MCP?
Yes. The MCP uses API key authentication, so you'll need an active VLM Run account and a valid API key. Paste the key into Switchy's connection flow. Anyone on your team who connects it will tie Switchy's usage to their own VLM Run credit balance, so coordinate who holds the key.
Can the VLM Run MCP process video and audio files?
Yes. The Extract Structured JSON tool accepts video and audio inputs alongside images and documents. Video and audio extraction runs are asynchronous, so the MCP returns a run ID that you poll with Get Run until the result is ready. This works for transcription, scene analysis, or any custom schema you define.
How is this different from calling VLM Run's API directly?
The MCP wraps VLM Run's REST API into natural-language tools that Claude can call mid-conversation. Instead of writing code to upload files, poll runs, and parse JSON, you describe what you want extracted and Claude handles the API choreography. You still pay VLM Run's per-run pricing; Switchy just makes it conversational.
Who on the team should connect the VLM Run MCP?
Whoever owns your VLM Run subscription and manages credit spend. Each connection uses one API key, so extraction jobs will bill against that account. If multiple people need access, either share one Switchy connection or provision separate VLM Run keys and track usage per person.