Vercel AI Gateway

@vercel-ai-gateway create a chat completion using gpt-4o with fallback to claude-3-5-sonnet, asking for a product roadmap summary in JSON format

The gateway routed your request to gpt-4o and returned a structured JSON object containing quarterly milestones, feature priorities, and resource allocation. The response included gateway metadata showing 1,247 input tokens, 423 output tokens, 89ms latency, and confirmed the primary model succeeded without triggering the fallback. The JSON schema you specified was validated and the output conforms to your roadmap structure.

This example highlights the gateway's model fallback and structured output capabilities. You need valid API keys for both OpenAI and Anthropic configured in your gateway settings. The fallback only activates if the primary model fails or times out—successful requests never hit the secondary provider, so you won't see cost savings from automatic provider switching during normal operation.

@vercel-ai-gateway count tokens for this Anthropic message before I send it: system prompt about code review standards, 3 user messages with code diffs, and 2 tool definitions

The token estimator returned 4,832 input tokens for your message structure: 1,205 tokens for the system prompt, 2,914 tokens across the three user messages (including the code diffs), 487 tokens for the two tool definitions, and 226 tokens of formatting overhead. This estimate helps you stay within Claude's context window and predict costs before making the actual API call.

Token counting is estimation-only and may differ slightly from the actual tokens charged by Anthropic after the request completes. This tool is most valuable for cost forecasting and context-window planning when building complex multi-turn conversations with large system prompts or many tool definitions. The estimate does not include output tokens, which depend on the model's response length.

@vercel-ai-gateway compact this 47-message OpenAI conversation into replayable context, preserving the key decisions and code examples but reducing token overhead

The compaction endpoint processed your 47-message thread (originally 18,934 tokens) and returned a condensed 6-message conversation totaling 4,217 tokens—a 78% reduction. The compacted version preserves function call results, the final code implementation, and three critical decision points, while removing redundant clarifications and intermediate iterations. You can replay this compacted context in future requests to maintain conversation continuity at lower cost.

Compaction only works with OpenAI-routed models and cannot fall back to other providers. The algorithm prioritizes recent messages and function call pairs, but you lose some conversational nuance—review the compacted output before relying on it for high-stakes decisions. Compaction itself consumes tokens and incurs API costs, so use it when the token savings in subsequent requests justify the upfront expense.

When your team runs parallel model tests without blowing budget

A 6-person product team ships three feature variants each sprint, each calling a different LLM for summarization or classification. Without gateway routing, you hard-code provider keys in every branch and lose visibility when one model costs 4x another. Vercel AI Gateway wins here because the fallback and routing controls let you set a primary model (GPT-4o-mini) with a cheaper backup (Claude Haiku) in one config, and the token-counting tool shows cost before you commit. The 22 tools cover OpenAI and Anthropic formats, so you're not rewriting prompts per provider. If your team ships fewer than two experiments a month or sticks to one model forever, the gateway is overkill—just call the provider direct. For teams iterating fast on model choice, this keeps experiments cheap and observable.

How a 3-agent support team scores AI draft quality in real time

A small support team uses an LLM to draft ticket replies, but agents need to know if the draft is safe to send or needs a rewrite. The Evaluate State tool runs a structured check (is the tone empathetic? does it answer the question?) against each draft before the agent sees it, returning boolean or score answers instead of freeform text. This scenario works because the team sends 40-60 tickets a day—enough volume to justify the gateway's eval overhead, but small enough that the API-key auth and single-model routing stay simple. If you're only evaluating 5 drafts a week, the setup cost outweighs the win; just have the agent read it. For teams in the 30-100 ticket/day range, the eval tool turns gut-check into data and cuts send-regret by half.

When a 2-person growth team needs repeatable image edits without designer bottleneck

A startup's growth lead and copywriter ship 8-12 ad variants a week, each needing a hero image tweaked from a base template (swap background, adjust lighting, overlay text). The Edit Images tool takes up to 16 source images and a text instruction, returning edited results through the gateway's non-streaming route. This beats hiring a designer for every variant because the instruction format is consistent and the gateway handles provider failover if one image API is slow. The catch: this only works for edits, not net-new generation, and the 16-image limit means you can't batch a month's creative in one call. If your team ships fewer than 5 image variants a week or needs complex multi-layer composition, you still need a designer. For high-frequency, low-complexity edits, the gateway keeps the two-person team unblocked.