> For the complete documentation index, see [llms.txt](https://docs.fastrouter.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.fastrouter.ai/readme.md). # Welcome ## Introduction to FastRouter FastRouter.ai is an LLM gateway and LLMOps platform. It gives you one endpoint to reach 160+ models across 20+ providers — and a control plane that routes, governs, observes, and optimizes every call you send through it. You change a base URL. FastRouter handles model selection, provider failover, credential management, caching, cost attribution, guardrails, and evaluations — with zero markup on token costs, whether you use your own provider keys (BYOK) or FastRouter credits. ## Start in 60 seconds FastRouter is a drop-in replacement for the OpenAI SDK. Change two lines: ```python from openai import OpenAI client = OpenAI( base_url="https://api.fastrouter.ai/api/v1", # FastRouter base URL api_key="YOUR_FASTROUTER_API_KEY", # Your FastRouter API key ) completion = client.chat.completions.create( model="openai/gpt-5.6-sol", # Replace with your model ID messages=[ { "role": "user", "content": "What is the meaning of life?" } ] ) print(completion.choices[0].message.content) ``` Prefer a different format? FastRouter also speaks the [Anthropic Messages API](https://docs.fastrouter.ai/api-reference/anthropic-messages-format) and the [Gemini Interactions API](https://docs.fastrouter.ai/api-reference/gemini-interactions-api-format) natively — use whichever endpoint your stack already speaks. Not sure which model to use? Send `fastrouter/auto` as the model ID and let FastRouter decide per request. ## What is an LLM Gateway? An LLM gateway sits between your application and your model providers. It handles request routing, failover, observability, cost tracking, and credential management — so your code stops encoding provider differences and starts treating models as interchangeable infrastructure. ## Beyond The Gateway: Routing Intelligence Most gateways stop at access. FastRouter continues past it. Your production traffic is the benchmark — not a generic leaderboard. FastRouter analyzes how you actually use models, then tells you where to cache, compress, downgrade, or re-prompt, with projected savings and the evidence behind each recommendation. Optimization is a product surface here, not a consulting engagement. [Watch more details about FastRouter](https://youtu.be/1Wb_DW2CHa8) *** ### Route Reach every model through one endpoint, and control exactly how each request is served. | Capability | What it does | | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | [Automatic Model Selection](/explore-features/automatic-model-selection.md) | Use `fastrouter/auto` and FastRouter picks the best-fit model per request, weighing complexity, domain, and cost. | | [Provider Routing Strategies](/explore-features/provider-routing-strategies.md) | Choose who serves each model — lowest price, lowest latency, highest throughput, weighted shuffle, priority order, or AI Auto Router. | | [Fallback Models](/explore-features/fallback-models.md) | Automatically retry the next model or provider on rate limits, downtime, or errors. Configurable per key. | | [Virtual Model Aliases](/explore-features/virtual-model-aliases.md) | Map one stable alias to many models and providers. Swap the model underneath without touching application code. | | [FastRouter Blend](/explore-features/fastrouter-blend-multi-model-deliberation.md) | Send one prompt to a panel of models in parallel, then have a judge model compare their answers into a structured analysis. | | [Free Model Router](/explore-features/free-model-router.md) | Append `:free` to eligible models, or use `fastrouter/free`, for no-cost testing and low-stakes workloads. | ### Optimize The routing intelligence layer — turning real usage into lower cost and better output. | Capability | What it does | | ----------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | [Insights](/explore-features/insights.md) | Weekly, evidence-backed cost recommendations generated from your own traffic, each with projected savings and exactly where to apply the change. | | [Custom Evaluations](/explore-features/custom-evaluations.md) | Benchmark and compare models on your own data with LLM-as-a-judge scoring and custom criteria. | | [Video Evaluations](/explore-features/video-evaluations.md) | Score AI-generated video at scale across motion, sync, quality, and prompt adherence. | | [Prompt Optimizations](/explore-features/prompt-optimizations.md) | Evolve stronger system prompts automatically with GEPA reflective optimization, using LLM-judged feedback as gradients. | | [Prompt Library](/explore-features/prompt-library.md) | Version, diff, deploy, and roll back prompts independently of your code — no redeploy required. | ### Save Cut spend without changing what your application asks for. | Capability | What it does | | ----------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | [Response Caching](/explore-features/response-caching.md) | Cache full responses across providers, including generated images and video. Repeat requests return instantly at no additional generation cost. | | [Prompt Caching](/explore-features/response-caching.md) | Reuse repeated context — system prompts, RAG chunks, documents — at a fraction of the input price, with sticky routing to maximize hit rates. | | [Prompt Compression](/explore-features/prompt-compression.md) | Shrink prompts at the gateway to cut input tokens without degrading response quality. Opt-in per request; fails open. | | [Flex Pricing](/explore-features/flex-pricing.md) | Append `:flex` on eligible OpenAI and Gemini models for significantly lower cost on latency-tolerant work. | | [Batch Processing](/explore-features/batch-processing.md) | Submit high-volume, non-real-time workloads asynchronously at a steep discount. | | [Add External Keys (BYOK)](/explore-features/add-external-keys-byok.md) | Route through your own provider accounts, with FastRouter's routing, observability, and governance still fully applied. | ### Govern Hard limits, enforced at the gateway — before the call, not after the invoice. | Capability | What it does | | ----------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | [Organization & Members](/explore-features/organization-and-members.md) | Role-based access control across your organization. | | [Projects](/explore-features/projects.md) | Separate workspaces per team, product, or environment, with scoped keys, budgets, and cost attribution. | | [Keys & Settings](/explore-features/keys-and-settings.md) | Granular per-key control over models, providers, budgets, and features. | | [Provisioning Keys](/explore-features/provisioning-keys.md) | Programmatically create, rotate, scope, and revoke service account keys — built for platforms provisioning on behalf of end users. | | [Guardrails](/explore-features/guardrails.md) | Deterministic and LLM-based checks for PII, toxicity, topic adherence, and regex patterns. Observe violations, or block them outright. | | [Credits](/explore-features/credits.md) | Manage balance, budgets, and spend across the organization. | ### Observe Every call attributed, every anomaly flagged. | Capability | What it does | | ------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | | [Dashboard & Activity Log](/explore-features/quickstart.md) | Inspect every request with full payloads, latency, tokens, and cost — plus one-click evals straight from the log. | | [Tracing](/explore-features/tracing.md) | Group related calls into a single trace with a `traceparent` header, including tool-call spans. | | [Custom Alerts](/explore-features/alerts.md) | Warning and Critical thresholds on latency, error rate, usage, and spend, routed to Slack, email, or webhook. | | [System Alerts](/explore-features/system-alerts.md) | Automatic notifications when credit balance or project and key budgets cross critical thresholds. | | [Dynamic Tags](/explore-features/dynamic-tags-per-request.md) | Tag requests per call with custom metadata, then slice usage and cost by feature, customer, environment, or team. | ### Build Everything the modern stack expects, unified across providers.
CapabilityWhat it does
Structured OutputsJSON schema-constrained responses across all major providers.
Function CallingUnified tool use, whichever provider serves the request.
Reasoning TokensControl and inspect thinking tokens on supported models.
Image & VideoGeneration and editing across providers through one API.
Audio & EmbeddingsTranscription, text-to-speech, and embeddings on the same endpoint.
Image Processing & PDF ProcessingSend images and PDFs directly as input, handled natively at the gateway.
Realtime APILow-latency speech-to-speech and text streaming over WebSocket.
MCP GatewayRegister MCP tool servers once and expose their tools to any model. Credentials are vaulted and injected at the gateway — agents never see raw provider keys.
Web SearchAttach web search as a tool on any model request, with results injected into context automatically.
### Integrate Route the tools your team already uses through FastRouter for cost control, observability, and governance. * [Coding Assistants](/integrations/coding-assistants.md) — Claude Code, Cursor, Codex CLI, Cline, Aider, Roo Code, OpenCode, Kilo Code, Xcode * [Agents](/integrations/agents.md) — LangGraph, CrewAI, AutoGen, Agno, OpenAI Agents SDK, Pydantic AI, OpenClaw, Hermes * [Apps & Frameworks](/integrations/app.md) — LangChain, LlamaIndex, Instructor, n8n, Open WebUI, Scalekit Building with an AI coding assistant? Point it at [skill.md](https://docs.fastrouter.ai/skill.md) to give it working knowledge of FastRouter's features. *** ### Jump Right In
Explore FeaturesLearn how FastRouter helps you scale and optimize your LLM usage/pages/CyH2xJQs9yWJ1S8BYNav
API ReferenceDive directly into request formats, supported models, error responses, and more/pages/7aUFmnCMx9m4smGncsXL
IntegrationsGuides to integrate FastRouter with your favorite tools/pages/Bopdto9qt914y8D49auG
Questions or feedback? Reach us at . [Watch more details about FastRouter](https://youtu.be/1Wb_DW2CHa8) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.fastrouter.ai/readme.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.