Welcome
Welcome to FastRouter.AI!
Introduction to FastRouter
FastRouter.ai is an LLM gateway and LLMOps platform. It gives you one endpoint to reach 160+ models across 20+ providers — and a control plane that routes, governs, observes, and optimizes every call you send through it.
You change a base URL. FastRouter handles model selection, provider failover, credential management, caching, cost attribution, guardrails, and evaluations — with zero markup on token costs, whether you use your own provider keys (BYOK) or FastRouter credits.
Start in 60 seconds
FastRouter is a drop-in replacement for the OpenAI SDK. Change two lines:
from openai import OpenAI
client = OpenAI(
base_url="https://api.fastrouter.ai/api/v1", # FastRouter base URL
api_key="YOUR_FASTROUTER_API_KEY", # Your FastRouter API key
)
completion = client.chat.completions.create(
model="openai/gpt-5.6-sol", # Replace with your model ID
messages=[
{
"role": "user",
"content": "What is the meaning of life?"
}
]
)
print(completion.choices[0].message.content)Prefer a different format? FastRouter also speaks the Anthropic Messages API and the Gemini Interactions API natively — use whichever endpoint your stack already speaks.
Not sure which model to use? Send fastrouter/auto as the model ID and let FastRouter decide per request.
What is an LLM Gateway?
An LLM gateway sits between your application and your model providers. It handles request routing, failover, observability, cost tracking, and credential management — so your code stops encoding provider differences and starts treating models as interchangeable infrastructure.
Beyond The Gateway: Routing Intelligence
Most gateways stop at access. FastRouter continues past it.
Your production traffic is the benchmark — not a generic leaderboard. FastRouter analyzes how you actually use models, then tells you where to cache, compress, downgrade, or re-prompt, with projected savings and the evidence behind each recommendation. Optimization is a product surface here, not a consulting engagement.
Watch more details about FastRouter
Route
Reach every model through one endpoint, and control exactly how each request is served.
Use fastrouter/auto and FastRouter picks the best-fit model per request, weighing complexity, domain, and cost.
Choose who serves each model — lowest price, lowest latency, highest throughput, weighted shuffle, priority order, or AI Auto Router.
Automatically retry the next model or provider on rate limits, downtime, or errors. Configurable per key.
Map one stable alias to many models and providers. Swap the model underneath without touching application code.
Send one prompt to a panel of models in parallel, then have a judge model compare their answers into a structured analysis.
Append :free to eligible models, or use fastrouter/free, for no-cost testing and low-stakes workloads.
Optimize
The routing intelligence layer — turning real usage into lower cost and better output.
Weekly, evidence-backed cost recommendations generated from your own traffic, each with projected savings and exactly where to apply the change.
Benchmark and compare models on your own data with LLM-as-a-judge scoring and custom criteria.
Score AI-generated video at scale across motion, sync, quality, and prompt adherence.
Evolve stronger system prompts automatically with GEPA reflective optimization, using LLM-judged feedback as gradients.
Version, diff, deploy, and roll back prompts independently of your code — no redeploy required.
Save
Cut spend without changing what your application asks for.
Cache full responses across providers, including generated images and video. Repeat requests return instantly at no additional generation cost.
Reuse repeated context — system prompts, RAG chunks, documents — at a fraction of the input price, with sticky routing to maximize hit rates.
Shrink prompts at the gateway to cut input tokens without degrading response quality. Opt-in per request; fails open.
Append :flex on eligible OpenAI and Gemini models for significantly lower cost on latency-tolerant work.
Submit high-volume, non-real-time workloads asynchronously at a steep discount.
Route through your own provider accounts, with FastRouter's routing, observability, and governance still fully applied.
Govern
Hard limits, enforced at the gateway — before the call, not after the invoice.
Role-based access control across your organization.
Separate workspaces per team, product, or environment, with scoped keys, budgets, and cost attribution.
Granular per-key control over models, providers, budgets, and features.
Programmatically create, rotate, scope, and revoke service account keys — built for platforms provisioning on behalf of end users.
Deterministic and LLM-based checks for PII, toxicity, topic adherence, and regex patterns. Observe violations, or block them outright.
Manage balance, budgets, and spend across the organization.
Observe
Every call attributed, every anomaly flagged.
Inspect every request with full payloads, latency, tokens, and cost — plus one-click evals straight from the log.
Group related calls into a single trace with a traceparent header, including tool-call spans.
Warning and Critical thresholds on latency, error rate, usage, and spend, routed to Slack, email, or webhook.
Automatic notifications when credit balance or project and key budgets cross critical thresholds.
Tag requests per call with custom metadata, then slice usage and cost by feature, customer, environment, or team.
Build
Everything the modern stack expects, unified across providers.
JSON schema-constrained responses across all major providers.
Unified tool use, whichever provider serves the request.
Control and inspect thinking tokens on supported models.
Transcription, text-to-speech, and embeddings on the same endpoint.
Send images and PDFs directly as input, handled natively at the gateway.
Low-latency speech-to-speech and text streaming over WebSocket.
Register MCP tool servers once and expose their tools to any model. Credentials are vaulted and injected at the gateway — agents never see raw provider keys.
Attach web search as a tool on any model request, with results injected into context automatically.
Integrate
Route the tools your team already uses through FastRouter for cost control, observability, and governance.
Coding Assistants — Claude Code, Cursor, Codex CLI, Cline, Aider, Roo Code, OpenCode, Kilo Code, Xcode
Agents — LangGraph, CrewAI, AutoGen, Agno, OpenAI Agents SDK, Pydantic AI, OpenClaw, Hermes
Apps & Frameworks — LangChain, LlamaIndex, Instructor, n8n, Open WebUI, Scalekit
Building with an AI coding assistant? Point it at skill.md to give it working knowledge of FastRouter's features.
Jump Right In
Questions or feedback? Reach us at contact@fastrouter.ai.
Last updated