Artificial Analysis

@artificial-analysis which LLMs have the best coding performance under $1 per million tokens?

Based on the Artificial Analysis leaderboard, three models stand out for coding under $1/MTok: Claude 3.5 Sonnet (coding index 89.2, $3/MTok input — slightly over budget but top-tier), GPT-4o mini (coding index 82.1, $0.15/MTok), and Gemini 1.5 Flash (coding index 79.8, $0.075/MTok). All three balance strong coding capability with reasonable cost. Note these are illustrative figures; actual pricing and index scores vary by provider and update cycle.

This example shows the MCP's core strength: surfacing performance benchmarks alongside pricing to help teams make model-selection trade-offs. Requires an API key. The leaderboard data refreshes periodically, so treat results as a snapshot rather than real-time truth.

@artificial-analysis compare the top 3 image-to-video models by arena Elo score and tell me which providers offer them

The top three image-to-video models by Elo are: Runway Gen-3 Alpha Turbo (Elo ~1180), Luma Dream Machine (Elo ~1165), and Kling v1.5 (Elo ~1150). Runway Gen-3 is available via Runway's API, Luma through their platform, and Kling via the Kuaishou API. All three scores include 95% confidence intervals of ±15–20 points, so the ranking is close. This illustrative output reflects arena standings as of the last leaderboard update.

Demonstrates the MCP's arena-ranking tools, which aggregate human preference data across generative models. Useful for teams evaluating which video-generation API to integrate. Be aware that Elo scores shift as new models enter the arena and voting continues.

@artificial-analysis I need a speech-to-text model with the lowest word-error rate for meeting transcription — what are my best options and how do their latencies compare?

For meeting transcription, the lowest WER models are OpenAI Whisper Large v3 (WER 8.2%, median latency ~1.8s per minute of audio) and AssemblyAI Best (WER 8.5%, latency ~2.1s/min). Both excel at accuracy; Whisper edges ahead on speed. If real-time matters, consider Deepgram Nova-2 (WER 9.1%, latency ~0.9s/min) — slightly less accurate but nearly twice as fast. These figures are illustrative; actual performance depends on audio quality and API tier.

Showcases the MCP's ability to synthesize multiple benchmark dimensions (accuracy + latency) into a recommendation. Teams building transcription features benefit from this comparative view. Requires API key; WER scores are aggregated from public test sets and may not match your domain-specific audio.

When your team needs to pick the right AI model fast

A 5-person startup building a voice-note app needs to choose between three speech-to-text models before their investor demo in two weeks. The Artificial Analysis MCP pulls WER scores and latency benchmarks in one call, so the team can compare accuracy-vs-speed trade-offs in Slack without opening ten vendor tabs. This MCP wins when you're evaluating models across multiple modalities (text, image, speech) and need apples-to-apples metrics that aren't buried in marketing decks. The threshold: if you're only comparing two LLMs for a one-off task, the leaderboard website is faster than setting up API auth. But if your roadmap includes image editing, music generation, or video synthesis in the next quarter, the 12-tool scope pays off by keeping the whole team's model decisions in one shared workspace.

Track LLM pricing shifts without spreadsheet hell

A 12-person agency runs client chatbots on three different LLM providers and reviews costs every Monday standup. The Artificial Analysis MCP's language-model tool returns pricing alongside intelligence scores, so the ops lead can spot when a competitor drops their per-token rate or a new model hits the leaderboard at half the cost. This scenario works because the MCP updates faster than manually scraping vendor pricing pages, and the team can query it mid-meeting when a client asks 'why is our bill up 20% this month?' The trade-off: if you're locked into one provider or your contract fixes pricing for six months, this MCP is overkill. But if you're running multi-model inference or evaluating new providers quarterly, the real-time pricing feed keeps your margin healthy without assigning someone to watch eight vendor blogs.

When your content team needs to test generative tools at scale

A 7-person content studio is pitching a campaign that mixes AI-generated music, image editing, and video synthesis. The creative director needs to know which models won the latest arena rankings before committing to a vendor stack. The Artificial Analysis MCP lists top-ranked models across image-to-video, instrumental music, and image-editing arenas in three separate calls, all with Elo scores and confidence intervals. This MCP is the right call when your team's output spans multiple generative modalities and you need to justify tool choices to clients or budget holders with third-party benchmarks. The boundary: if your studio only does one thing (say, podcast transcription), the speech-to-text leaderboard alone doesn't justify the API setup. But if your pipeline touches three or more generative categories, this MCP turns model research from a two-day rabbit hole into a 20-minute Switchy session.