# Welcome

Welcome to FastRouter.AI!

### **Introduction to FastRouter Gateway**

FastRouter.ai is a robust LLM Gateway that acts as a control plane for managing and routing requests across multiple language models and providers. It offers scalability, reliability, and fine-grained control for enterprise and developer workloads.

### **What is an LLM Gateway?**

An LLM Gateway sits between your application and various large language model (LLM) providers. It handles request routing, observability, cost tracking, error fallback, performance optimization, and credential management—enabling developers to abstract provider differences and optimize usage.

[Watch more details about FastRouter](https://youtu.be/1Wb_DW2CHa8)

### Jump right in

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-cover data-type="files"></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>Explore Features</strong></td><td>Learn how FastRouter helps you scale and optimize your LLM usage</td><td></td><td></td><td><a href="/pages/CyH2xJQs9yWJ1S8BYNav">/pages/CyH2xJQs9yWJ1S8BYNav</a></td></tr><tr><td><strong>API Reference</strong></td><td>Dive directly into request formats, supported models, error responses, and more</td><td></td><td></td><td><a href="/pages/7aUFmnCMx9m4smGncsXL">/pages/7aUFmnCMx9m4smGncsXL</a></td></tr><tr><td><strong>Integrations</strong></td><td>Guides to integrate FastRouter with your favorite tools</td><td></td><td></td><td><a href="/pages/Bopdto9qt914y8D49auG">/pages/Bopdto9qt914y8D49auG</a></td></tr></tbody></table>


# Dashboard

The Dashboard and Activity Log are essential for monitoring, analyzing, and optimizing your API usage with large language models (LLMs).

### Overview

The Dashboard and Activity Log in FastRouter provide comprehensive monitoring and analytics for API interactions involving LLMs. The Activity Log offers real-time tracking of requests and responses, while the Dashboard delivers interactive visualizations for deeper insights into usage, performance, and costs.

{% embed url="<https://www.youtube.com/watch?v=sj6FYlSutOI>" %}

### Dashboard Metrics

* **Requests**: Total number of API calls routed via FastRouter.
* **Total Cost**: Aggregated dollar value for all requests routed.
* **Total Prompt Tokens**: Cumulative count of input tokens across requests.
* **Total Completion Tokens**: Cumulative count of output tokens returned/used.
* **Averages**: Includes average input/output tokens per request and average cost.

### Dashboard Charts

* **Requests per Day**: Daily breakdown of routed requests.
* **Cost per Day**: Trend of API usage cost over time.
* **All Keys**: Usage per key alias. Click to filter per key selected.
* **All Models**: Usage per model. Click to filter per model selected.
* **All Providers**: Usage per providers that received the routed traffic. Click to filter per provider selected.
* **Errors**: Error occurrences over time by type and provider.
* **Model Response Time per Day**: Trend of average model response time per request.
* **Model Response Time Quantiles**: Breakdown (P50, P90, P99) of latency distributions.
* **Time to First Token per Day**: Measures responsiveness; how quickly the first token is received.
* **Request Distribution by Country**: Measures requests received by originating location.

### Activity Log

The **Activity Log** provides a real-time view of all API requests and responses—allowing you to monitor, troubleshoot, and audit usage effectively.

> **Note:** Activity Log request and response data is available only if **Content Logging** is not disabled for the corresponding key.

**Overview**

The Activity Log streams every request and response associated with that key, helping teams analyze performance, debug issues, and ensure compliance.

**Key Features**

* Click any log entry to view full request and response details.
* Quickly filter logs by key, model, or specific text in inputs/outputs.
* Adjust visible columns in the log using the column visibility toggle to focus on the most relevant information.


# Automatic Model Selection

### Intelligent Model Routing **Overview**

FastRouter’s intelligent model routing mode lets you delegate model selection to us. Instead of picking a specific model, just set:

```bash
"model": "fastrouter/auto"
```

### **How It Works**

FastRouter will automatically select the most appropriate model based on:

* Complexity of the query
* Domain or topic of the request
* Cost-efficiency

This is the fastest way to get started — no manual tuning, no need to maintain model preference lists.

### Example

```bash
curl --location 'https://api.fastrouter.ai/api/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer API-KEY' \
--data '{
  "model": "fastrouter/auto",
  "messages": [
    {
      "role": "user",
      "content": "What is 2+2?"
    }
  ]
}'
```

FastRouter will analyze the input and select the best model from the available pool.


# Virtual Model Aliases

Virtual Model Aliases make it easy to manage and optimize amongst multiple LLMs without changing your code.

### Overview

Virtual Model Aliases in FastRouter.ai let you easily group several models and providers under a single alias. You can reference the alias in your API calls by passing the alias name to the `model` parameter. FastRouter will then automatically select and route requests to one of the configured models—according to the selection strategy you specify.

This feature maximizes flexibility, reliability, and efficiency when deploying LLMs across different providers or model versions.

{% embed url="<https://www.youtube.com/watch?v=Fo3qczJqd0Y>" %}

***

### How It Works

You can create a Virtual Model Alias by selecting models and providers, assigning weights (if needed), and choosing an automatic selection strategy. On each request, FastRouter will pick from the configured options automatically—and fall back to another model in the list if the primary one fails.

**Use Cases:**

* **A/B testing:** Seamlessly test multiple models/providers.
* **Resilience:** Automatic fallback if a provider or deployment is unavailable.
* **Optimization:** Route requests for the best speed, price, or usage.
* **Task specialization:** Direct different prompt types to the models best suited for them.

***

#### Selection Strategies

When configuring your Virtual Model Alias, you can specify how FastRouter should select models for each request:

**Available Strategies:**

* **Random Shuffle**\
  Each request is sent to a randomly selected model in your list, according to the weights you assign. Useful for A/B tests or spreading traffic evenly.
* **Lowest Latency**\
  Automatically selects the model with the fastest current response time, ensuring minimum wait time for end-users.
* **Highest Throughput**\
  Picks the model/provider combination that can process requests at the highest rate, ideal for high-volume applications.
* **Lowest Usage**\
  Prioritizes models with the least usage, helping to balance load and prevent hitting usage or rate limits.
* **Lowest Price**\
  Routes requests to the least expensive model in the list, optimizing for cost savings.
* **Priority Routing**\
  Routes requests through models in a fixed, user-defined priority order. If the top-priority model or provider fails or is unavailable, FastRouter automatically falls back to the next model in sequence—providing deterministic, predictable routing.

<figure><img src="/files/W5JyKLouxHaeQJDGW0GT" alt=""><figcaption></figcaption></figure>

* **Category-Based Routing**\
  Routes each request to a model group based on the detected category of the prompt. You define a **Default** category (always present) and any number of named override categories (e.g., *Coding*, *Content Writing*, *Code Summarization*, *Classification*, *Translation*). Each category has its own model list, provider preferences, weights, and an independent **Sub Strategy** (e.g., Random Shuffle) that governs how models within that category are selected. If a request doesn't match any named category, it falls through to the Default category (which is always present and cannot be removed). This strategy is ideal when different task types are best handled by different models.

<figure><img src="/files/tlnXnYXgPrmDjrab69yF" alt=""><figcaption></figcaption></figure>

***

### **Built-in Automatic Fallback**

If a model/provider fails or is unavailable, FastRouter will transparently send the request to the next candidate in your list, according to your selected strategy—ensuring maximum reliability.

For **Priority Routing**, fallback follows the explicit numbered order you define.\
For **Category-Based Routing**, fallback within a category follows the configured Sub Strategy for that category.

***

### Getting Started

1. Go to the **Virtual Models** section in FastRouter.
2. Click on **Create Virtual Model Alias**.
3. In the **Select Models** screen, select one or more models to be referenced in your alias and click **Next**.

<figure><img src="/files/u3UgP5rsJF9r2mFpYn2G" alt=""><figcaption><p>Select Models</p></figcaption></figure>

4. In the **Finalize Details & Strategy** screen, enter a virtual model alias name.
5. Choose the **Projects** that have access to this alias and the **Strategy** for model selection.
6. **If using Random Shuffle, Lowest Latency, Highest Throughput, Lowest Usage, or Lowest Price:** Select the providers you want to include for each model and assign weights where applicable.
7. **If using Priority Routing:** Arrange models in your preferred fallback order by dragging them into sequence. Assign a provider for each model.
8. **If using Category-Based Routing:** Configure the Default category with models and a Sub Strategy, then add any named override categories as needed, each with their own models, providers, weights, and Sub Strategy.
9. Click **Create List**.

You can then reference this Virtual Model Alias in your API calls by specifying its name in the `model` parameter.


# FastRouter Blend: Multi-Model Deliberation

Multi-model deliberation — ask a panel of models the same prompt in parallel, then have a judge model compare their answers into a structured analysis.

### Overview

Blend asks a panel of models the same prompt in parallel, then a judge model compares their answers into a structured analysis: agreements, disagreements, coverage gaps, standout insights, missing considerations, and confidence notes.

Instead of trusting a single model, Blend gathers multiple independent perspectives on your prompt and has a judge model analyze how they compare. The judge does **not** merge the answers — it evaluates them, so you can see where models agree, where they conflict, and what each one uniquely contributed.

* **Parallel panel calls** — the same prompt runs against every panel model at once.
* **Structured judge analysis** — a single JSON object comparing the answers.
* **Aggregated usage & cost** — panel + judge tokens and spend, summed and itemized.
* **Reuses routing, BYOK & credits** — sub-calls run through the normal gateway path.

### Two ways to use Blend

| Method                               | How to trigger                                        | When to use                                                                                                                            |
| ------------------------------------ | ----------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| **Model alias** — `fastrouter/blend` | Set `"model": "fastrouter/blend"`.                    | Blend always runs and returns a human-readable summary as the completion content. Best when you want Blend to *be* the whole response. |
| **Server tool** — `fastrouter:blend` | Attach a `fastrouter:blend` tool to a normal request. | The outer model decides when to call it; the structured analysis is fed back as a tool result so the model writes the final answer.    |

***

### 1. Model alias — `fastrouter/blend`

Send a normal chat completion request with the model set to `fastrouter/blend`:

```bash
curl https://api.fastrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fastrouter/blend",
    "messages": [
      { "role": "user", "content": "Survey the strongest arguments for and against EV cars." }
    ],
    "analysis_models": ["anthropic/claude-sonnet-5", "openai/gpt-5.2", "google/gemini-3-pro"]
  }'
```

The assistant content is a markdown summary: a `## Panel responses` section (one subsection per model) followed by `## Analysis` containing the structured JSON. `analysis_models` is optional — see Panel & judge selection.

### 2. Server tool — `fastrouter:blend`

Attach the hosted `fastrouter:blend` tool to a request that targets any normal model. Blend parameters go inside the tool's `parameters` object.

```bash
curl https://api.fastrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "x-ai/grok-4.20-beta",
    "messages": [
      { "role": "user", "content": "Assess the case for and against banning internal combustion engine vehicles by 2035." }
    ],
    "tools": [
      {
        "type": "fastrouter:blend",
        "parameters": {
          "analysis_models": [
            "google/gemini-3-flash-preview",
            "z-ai/glm-5.2",
            "moonshotai/kimi-k2.7-code"
          ],
          "model": "anthropic/claude-sonnet-5"
        }
      }
    ]
  }'
```

**What happens under the hood:**

1. The gateway replaces the hosted tool with a callable function named `fastrouter_blend` (providers reject unknown hosted tool types).
2. The outer model runs normally and decides whether to call `fastrouter_blend`.
3. When it does, the gateway runs the Blend pipeline, injects the structured `BlendResult` JSON as the tool result, and re-calls the model.
4. The model uses that analysis to write the final answer.

> **Model-driven.** With the server tool, Blend runs only if the outer model chooses to call `fastrouter_blend`. For simple prompts a model may answer directly and skip it. Use the **model alias** `fastrouter/blend` at the top level if you want Blend to always run.

***

### Parameters

For the **alias** these go at the top level of the request body. For the **tool** they go inside the tool's `parameters` object.

| Parameter         | Type       | Required | Description                                                                                                                                   |
| ----------------- | ---------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| `analysis_models` | `string[]` | No       | The panel models that answer in parallel. Capped at 5. If omitted, the panel is auto-selected from the router's top candidates (capped at 3). |
| `model`           | `string`   | No       | The judge model that compares the panel answers. Defaults to the top-ranked panel model when omitted.                                         |

Everything else in your request — the `messages`, plus any other tools, sampling params, or `response_format` — is forwarded to each panel model. Only the model id is swapped, the Blend tool is stripped, and streaming is turned off for the sub-calls.

### Panel & judge selection

**Panel models**

* **Supplied:** your `analysis_models` are used as-is, in order, de-duplicated, capped at `MAX_BLEND_ANALYSIS_MODELS = 5`. Supplying 2 uses exactly 2.
* **Auto:** if you supply none, the panel is derived from the router's classifier top candidates, capped at `MAX_BLEND_AUTO_MODELS = 3`.

**Judge model**

* The `model` override when provided.
* Otherwise, the top-ranked panel model.

***

### Response format

#### Model alias — human-readable content

The completion content is markdown, structured like:

```markdown
## Panel responses

### anthropic/claude-sonnet-5
<that model's full answer>

### openai/gpt-5.2
<that model's full answer>

**Tool calls**    ← only shown if a panel answered with tool calls
[ { "id": "call_1", "type": "function", "function": { ... } } ]

## Analysis
<the structured judge JSON>
```

#### Server tool — structured tool result

The Blend pipeline returns a `BlendResult` JSON object as the tool result. The model then writes the final natural-language answer from it. The object looks like:

```json
{
  "status": "ok",
  "analysis": { /* judge JSON, see below */ },
  "responses": [
    {
      "model": "google/gemini-3-flash-preview",
      "content": "…",
      "tool_calls": [ /* present only if the panel emitted them */ ],
      "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0, "cost": 0 }
    }
  ],
  "failed_models": [ { "model": "…", "error": "…" } ],
  "panel_models": ["…"],
  "judge_model": "anthropic/claude-sonnet-5",
  "trace_id": "blend-…",
  "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0, "cost": 0 },
  "judge_usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0, "cost": 0 }
}
```

| Field                          | Description                                                            |
| ------------------------------ | ---------------------------------------------------------------------- |
| `status`                       | `"ok"` or `"error"`.                                                   |
| `analysis`                     | The judge's structured comparison JSON. Omitted if the judge degraded. |
| `responses`                    | The successful panel answers (`content` and/or `tool_calls`).          |
| `failed_models`                | Panel models that errored, with the error message.                     |
| `panel_models` / `judge_model` | Which models were used (observability).                                |
| `trace_id`                     | Shared id correlating every panel + judge sub-call in the run.         |
| `usage` / `judge_usage`        | See Usage & cost.                                                      |

#### Judge output (the `analysis` JSON)

The judge returns only this object — it analyzes, it does not merge:

```json
{
  "agreements": ["points where most or all responses aligned"],
  "disagreements": [
    {
      "topic": "the point of disagreement",
      "positions": [
        { "model": "model identifier", "position": "that model's stance" }
      ]
    }
  ],
  "coverage_gaps": [
    {
      "point": "a relevant point only some responses addressed",
      "covered_by": ["model identifiers that covered it"]
    }
  ],
  "standout_insights": [
    { "model": "model identifier", "insight": "something notable only this model raised" }
  ],
  "missing_considerations": ["important aspects no response addressed"],
  "confidence_notes": ["where responses seemed uncertain, hedged, or speculative"]
}
```

***

### Usage & cost

The `usage` field is the sum of every panel call plus the judge call — `prompt_tokens`, `completion_tokens`, `total_tokens`, and `cost`. The `judge_usage` field isolates just the judge call (its numbers are already included in the total).

Panel usage is counted **even for panels that failed** to produce usable content, because the tokens were still spent.

### Errors & graceful degradation

* **Judge fails / returns non-JSON but at least one panel succeeded** → `status` stays `"ok"`, `analysis` is omitted, and the panel responses are still returned.
* **No panel produced output** → `status: "error"` with a `failure_reason`.

| `failure_reason`          | Meaning                                                            |
| ------------------------- | ------------------------------------------------------------------ |
| `all_panels_failed`       | Every panel model errored.                                         |
| `blend_invocation_capped` | Blend was already invoked earlier in this turn (one run per turn). |
| `rate_limited`            | A rate limit was hit.                                              |
| `unexpected_error`        | An unexpected internal error (e.g. no models available).           |

***

### Limits & notes

* **Max panel size:** 5 supplied models, 3 auto-selected.
* Blend runs **at most once per turn** (recursion protection).
* The server-tool path is **non-streaming and top-level only**.
* Panel models receive your other tools too, so a panel may answer with `tool_calls`; those are captured and included in what the judge compares.
* The `fastrouter:blend` server tool is model-driven; use `fastrouter/blend` when you want Blend to always run.


# Fallback Models

### Fallback **Model Lists Overview**

FastRouter supports **fallback** **model lists**, allowing you to provide multiple fallback models in a single request. If the primary model is unavailable or returns an error (e.g., due to rate limits, downtime, or moderation), FastRouter will automatically attempt to route the request to the next available model in your list.

### **How It Works**

* Use the `model` field to define your **primary model** and the `models` array to define one or more **fallback models**.
* FastRouter will try the primary model first. If it fails, it will iterate through the list in order until a successful response is received or all models fail.
* The final model used is returned in the `model` field of the response body.
* Billing is based on the model that actually processes the request.

### Example

```bash
curl --location 'https://api.fastrouter.ai/api/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer API-KEY' \
--data '{
    "model": "openai/gpt-4o",
    "models":["openai/o1", "google/gemini-1.5-pro"],
    "messages": [
        {
            "role": "user",
            "content": "Why would I need an LLM judge?"
        }
    ],
    "stream": true
} 
'
```

#### What Happens:

1. FastRouter first tries `openai/gpt-4o`.
2. If it fails (due to downtime, rate limit, moderation, etc.), it tries `openai/o1`.
3. If that fails, it then tries `google/gemini-1.5-pro`.
4. If all fail, the final error is returned to the user.


# Free Models (:free)

FastRouter exposes select models at no charge via the :free slug — up to 10 requests per org per day, with no payment required.

### Introduction

This feature is available to all orgs regardless of billing status. Provided at FastRouter's discretion; free access on any given model can be paused or removed at any time.

***

### How It Works

Append `:free` to any supported model ID:

```
sarvam/sarvam-105b:free
```

FastRouter strips the suffix, checks whether `:free` is enabled for that model, verifies your org's daily quota, then routes the request normally. The `:free` suffix is invisible to the downstream provider.

If a model does not have `:free` enabled, the request is rejected. The standard model ID (without the suffix) is unaffected.

> **Note:** The 10-request daily quota is org-wide and shared across all API keys and members. Paid orgs using `:free` consume from this free quota — not from billing credits.

***

### Supported Models

Currently, supported on Sarvam models in our catalog for a limited time.

| Model               | Free Model ID             |
| ------------------- | ------------------------- |
| Sarvam: Sarvam 105B | `sarvam/sarvam-105b:free` |
| Sarvam: Sarvam 30B  | `sarvam/sarvam-30b:free`  |
| Sarvam: Saaras V3   | `sarvam/saaras:v3:free`   |
| Sarvam: Bulbul V2   | `sarvam/bulbul:v2:free`   |

`:free` is enabled on a per-model basis. Check the [model catalog](https://fastrouter.ai/models?order=newest) — eligible models display a **Free** badge on their detail page.

***

### Usage

#### cURL

```bash
curl 'https://api.fastrouter.ai/api/v1/chat/completions' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "sarvam/sarvam-105b:free",
    "messages": [
      { "role": "user", "content": "Explain backpressure in streaming systems." }
    ]
  }'
```

#### Python (OpenAI SDK)

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.fastrouter.ai/api/v1",
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="sarvam/sarvam-105b:free",
    messages=[
        {"role": "user", "content": "Explain backpressure in streaming systems."}
    ],
)

print(response.choices[0].message.content)
```

***

### Quota & Limits

| Property                 | Value                                                  |
| ------------------------ | ------------------------------------------------------ |
| Requests per org per day | 10                                                     |
| Scope                    | Per model — quota tracked independently per `model_id` |
| Reset                    | Daily at UTC midnight                                  |
| Carry-over               | None — unused requests do not roll over                |
| Paid org behaviour       | Consumes free quota, not billing credits               |

***

### Error Responses

#### `:free` not enabled on this model — `400`

Returned when `:free` is used on a model that does not have the slug enabled.

```json
{
  "error": {
    "code": "free_slug_not_enabled",
    "message": "The :free slug is not available for this model. Use the standard model ID or check the model catalog for supported free models.",
    "type": "invalid_request_error"
  }
}
```

#### Daily quota exhausted — `429`

Returned when your org has used all 10 free requests for the day on this model. The response includes a `Retry-After` header pointing to the next UTC midnight reset.

```json
{
  "error": {
    "code": "free_quota_exceeded",
    "message": "Your organisation has reached the daily free request limit for this model.",
    "type": "rate_limit_error",
    "quota_limit": 10,
    "quota_used": 10,
    "reset_at": "2026-05-30T00:00:00Z"
  }
}
```

To continue without waiting for the reset, remove the `:free` suffix to route as a standard paid request.

***

### Activity Log

All `:free` requests appear in your Activity Log tagged with a **Free** tier indicator. Cost is recorded as `$0.00`. Usage analytics include free-tier traffic separately so it does not skew your paid consumption metrics.

***

### FAQ

**Does `:free` support all API parameters?** Yes — structured outputs, tool use, streaming, and all parameters supported by the underlying model work with `:free`.

**Can I combine `:free` with `:flex`?** No. `:free` and `:flex` are mutually exclusive suffixes.

**Is the 10-request limit shared across all free models, or per model?** Per model. Each model has its own independent quota counter, so using `sarvam/sarvam-105b:free` does not consume quota for any other `:free` model.

**What happens to a request if `:free` access is removed from a model?** It returns a `400` error, the same as using `:free` on a model that never had it enabled. The standard model ID continues to work normally.


# Flex Pricing

Access OpenAI and Google Gemini models at up to 50% lower cost by opting into flexible inference — ideal for background tasks, batch workloads, and latency-tolerant applications.

> 💡 **Instant savings, zero code changes** Append `:flex` to any supported model ID. Your API key, endpoint, and payload stay exactly the same.

### What is Flex Pricing?

Flex inference is a tiered pricing mode offered by OpenAI and Google on select models. When you use the `:flex` suffix, FastRouter routes your request to the provider's Flex tier — significantly reducing token costs in exchange for variable throughput and potentially higher latency under peak load.

Flex is well-suited for workloads that don't require guaranteed low latency: data extraction pipelines, classification jobs, offline summarisation, evals runs, and other async or background tasks.

> ⚠️ **Not recommended for real-time user-facing responses** Flex requests may experience higher tail latencies during peak provider load. Use the standard tier for interactive or streaming use-cases.

### Pricing Comparison

Example using **GPT-5.4 Nano** via the OpenAI provider:

| Tier         | Input             | Output            | Blended           | Context |
| ------------ | ----------------- | ----------------- | ----------------- | ------- |
| **Standard** | $0.20 / 1M tokens | $1.25 / 1M tokens | $0.46 / 1M tokens | 400,000 |
| **✦ Flex**   | $0.10 / 1M tokens | $0.63 / 1M tokens | —                 | 400,000 |

**↓ \~50% savings on tokens**

> Actual savings vary by model. Check the [model catalog](https://fastrouter.ai/models?order=newest) for per-model Flex pricing across all supported providers.

### Supported Providers

Flex pricing is currently available on the following providers:

| Supported Provider                    |
| ------------------------------------- |
| OpenAI                                |
| Gemini API on Vertex AI and AI Studio |

### How to Use Flex

1. **Identify a Flex-supported model** Check the model catalog for the **Flex** tab in Provider Details. If the tab is present, Flex pricing is available for that model.
2. **Append `:flex` to the model ID** Change your model field from `openai/gpt-5.4-nano` to `openai/gpt-5.4-nano:flex` or from `google/gemini-3.1-pro-preview` to `google/gemini-3.1-pro-preview:flex` That's the only change required.
3. **Optionally pin the provider** Use `"provider": {"only": ["openai"]}` or `"provider": {"only": ["googleaistudio"]}` or `"provider": {"only": ["googlevertexai"]}` to ensure the request goes to the correct provider for the Flex tier and isn't rerouted.

### Code Examples

{% tabs %}
{% tab title="cURL" %}

```bash
# ✦ With Flex — ~50% cheaper
curl 'https://api.fastrouter.ai/api/v1/chat/completions' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "openai/gpt-5.4-nano:flex",
    "provider": { "only": ["openai"] },
    "messages": [
      { "role": "user", "content": "Summarise this document..." }
    ]
  }'
```

{% endtab %}

{% tab title="Python" %}

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.fastrouter.ai/api/v1",
    api_key="YOUR_API_KEY",
)

response = client.chat.completions.create(
    model="openai/gpt-5.4-nano:flex",
    extra_body={"provider": {"only": ["openai"]}},
    messages=[
        {"role": "user", "content": "Summarise this document..."}
    ],
)

print(response.choices[0].message.content)

```

{% endtab %}

{% tab title="TypeScript" %}

```typescript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.fastrouter.ai/api/v1",
  apiKey: process.env.FASTROUTER_API_KEY,
});

const response = await client.chat.completions.create({
  model: "openai/gpt-5.4-nano:flex",
  // @ts-expect-error - FastRouter routing extension
  provider: { only: ["openai"] },
  messages: [
    { role: "user", content: "Summarise this document..." },
  ],
});

console.log(response.choices[0].message.content);

```

{% endtab %}
{% endtabs %}

### When to Use Flex

| Use Flex ✦                                 | Use Standard                     |
| ------------------------------------------ | -------------------------------- |
| Data extraction & classification pipelines | Real-time chat & interactive UIs |
| Batch document summarisation               | Streaming responses to end users |
| Eval or fine-tuning dataset generation     | Latency-sensitive agent loops    |
| Scheduled background jobs                  | Voice or real-time applications  |
| Cost-optimised preprocessing at scale      | SLA-bound enterprise workflows   |


# Provider Routing Strategies

### Provider Routing Strategies **Overview**

FastRouter allows you to control how requests are routed to providers using flexible provider sorting and filtering parameters. Below are the options you can use within the `provider` object in your API request to fine-tune routing behavior. Using these parameters, you can define intelligent strategies to optimize performance, cost, or availability.

### Available Strategies

* **Default Strategy (Prioritizing Low Cost & High Uptime):** FastRouter assigns higher priority to models and providers that meet a performance and uptime threshold and weights them based on the inverse square of their price. This approach enables selecting more cost-effective options without compromising reliability.
* **Lowest Latency**: Routes requests to the fastest responding provider.
* **Lowest Price**: Routes to the least expensive provider for the selected model.
* **Highest Throughput**: Prefers providers with the highest tokens/minute output.

### Provider Object Parameters

Each provider object used in a routing strategy can include:

| Parameter            | Description                                                                |
| -------------------- | -------------------------------------------------------------------------- |
| `order`              | Define the exact sequence of providers to be used for the request.         |
| `allow_fallbacks`    | Control whether to allow backup providers when the primary is unavailable. |
| `only`               | Restrict the request to a specific set of providers.                       |
| `ignore`             | Exclude listed providers from being used in the request.                   |
| `sort`               | Automatically sort providers by the specified option.                      |
| `require_parameters` | Only use providers that support all parameters in the request.             |

***

### **1. Explicit Provider Priority: `order`**

Use the `order` field to define the exact sequence of providers to be used for the request. Providers will be attempted in the specified order.

```bash
curl --location 'https://api.fastrouter.ai/api/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer API-KEY' \
--data '{
  "model": "openai/gpt-4o",
  "messages": [{"role": "user", "content": "How many r’s in strawberry?"}],
  "stream": true,
  "provider": {
    "order": ["azure", "openai"]
  }
}'
```

***

### **2. Enable Fallbacks: `allow_fallbacks`**

This controls whether to allow backup providers when the primary is unavailable. If set to `true`, fallback providers from the ordered list are used in sequence.

```bash
"provider": {
  "order": ["openai", "azure"],
  "allow_fallbacks": true
}
```

***

### **3. Restrict to Specific Providers: `only`**

Restrict the request to a specific set of providers. The `order` or `sort` logic will be applied only to this subset.

```bash
"provider": {
  "only": ["openai"]
}
```

***

### **4. Exclude Specific Providers: `ignore`**

Exclude listed providers from being used in the request, regardless of ordering or sorting.

```bash
"provider": {
  "order": ["openai", "azure"],
  "ignore": ["openai"]
}
```

***

### **5. Automatically Sort Providers: `sort`**

Automatically sort providers by:

* `"price"` – Choose the cheapest option.
* `"throughput"` – Choose provider with highest token throughput.
* `"latency"` – Choose the fastest responding provider.

#### Example: Sort by price

```bash
"provider": {
  "sort": "price",
  "ignore": ["openai"]
}
```

#### Example: Sort by throughput

```bash
"provider": {
  "sort": "throughput"
}
```

#### Example: Sort by latency

```bash
"provider": {
  "sort": "latency"
}
```

### **Sorting Slugs: Sorting Via Model Suffix**

You can also use model suffixes to trigger automatic provider sorting without explicitly setting the `provider.sort` field.

* `:throughput` → Shortcut for `"sort": "throughput"`
* `:price` → Shortcut for `"sort": "price"`

#### Example: `:throughput` (Throughput prioritized)

```bash
"model": "openai/gpt-4o:throughput"
```

#### Example: `:price` (Lowest price prioritized)

```bash
"model": "openai/gpt-4o:price"
```

***

### **6. Strict Parameter Support: `require_parameters`**

Set this to `true` to ensure only providers that support *all* parameters in your request are considered. Defaults to `false`.

```bash
"provider": {
  "require_parameters": true
}
```

***

This flexible provider routing configuration gives you granular control over cost, speed, privacy, and reliability for each request. Use combinations of `order`, `sort`, `ignore`, and `only` to build advanced routing strategies on-the-fly.


# Organization & Members

FastRouter supports role-based access control at the organization level to manage permissions and responsibilities effectively.

### **Role Types Overview**

FastRouter currently offers two primary organization-level roles:

#### **1. Owner**

Owners have **full administrative privileges** across the entire organization.

**Owner Permissions:**

* Manage **billing and subscriptions**
* Add, remove, or update **organization members**
* Access and administer **all projects** and their settings
* Create and manage **API keys** across the org
* Configure **global settings** and integrations

> Each organization can have multiple owners for redundancy and shared management.

#### **2. Member**

Members have **limited access**, focused only on the projects they are explicitly invited to.

**Member Permissions:**

* Can view and interact with **assigned projects**
* Can **create and manage API keys**, but **only** within their specific project roles
* Cannot modify organization-wide settings, billing, or user access

> 🔒 Members operate strictly within the boundaries of their project-level permissions.

***

### Project-Level Roles

While this page covers **organization roles**, Members may also have **project-specific roles** (e.g., Project Admin, Project Member) that further define what they can do within each project.

Learn more about these roles in the [Projects](/projects) documentation.


# Projects

Projects in FastRouter.ai provide a structure for specifying access and limits for keys used.

### Projects Overview

Projects in FastRouter.ai provide a structure for organizing activity within your organization. Each project is independently configurable and managed, giving teams control over resources, permissions, and integrations.

{% embed url="<https://youtu.be/NWsP3Zgwoq0>" %}

You can manage the following aspects under **Projects** in FastRouter.ai:

***

### 1. Basic Settings

<figure><img src="/files/GCtCJ6pYbtXpxtzUAIBG" alt=""><figcaption></figcaption></figure>

Configure fundamental parameters for your project, such as rate limits, budget controls, and accessible models.

**Fields include:**

* **Project Name**: Identify your project.
* **Models**: Select accessible model families.
* **Tokens Per Minute / Requests Per Minute**: Specify global usage limits for the project.
* **Budget Limit**: (Optional) Toggle to enforce a spending cap for the project.
* **Maximum Budget & Reset Duration**: Set and reset spending limits as desired.

Click **Save** once your configurations are complete.

***

### 2. Members

<figure><img src="/files/glQaV9OIDpLLSwCTYj4e" alt=""><figcaption></figcaption></figure>

The **Members** tab lets you manage who can access your project and assign specific roles:

**Roles available:**

* **Project Admin**
  * Manage project settings, members, and API keys.
  * Full control over the project configuration including inviting other members.
* **Project Member**
  * Can create personal API keys based on permissions.

**Note:**\
Organization Owners are automatically **Project Admins** for all projects. Additional admins or members can be added from here as needed.

**To invite members:**

1. Click **Invite Member**.
2. Select the organization member and assign the appropriate role.
3. Click **Invite**.

***

### 3. Keys

<figure><img src="/files/83q7sF2eQkrceOADPajg" alt=""><figcaption></figcaption></figure>

API keys in FastRouter.ai allow secure access to model endpoints within a project.

* **Create User Key**: Generate API keys owned by a user to access all models routed by FastRouter.
* **Create Provisioning Key**: Generate provisioning keys that can in turn be used to create manage service account keys programatically.
* **Manage Existing Keys**: Edit or revoke as needed.

***

### Important:

* When a project is made **inactive**, all user keys associated with that project will be **disabled**.
* When a project member is **removed**, any user keys they have created for the project will also be **disabled**.


# Keys & Settings

API keys in FastRouter allow granular control over access, usage, and budget. You can generate multiple keys with custom configurations for different users, projects, or integrations.

API Keys in **FastRouter** are tied to individual users and inherit their project-level permissions.\
They provide secure, scoped access to FastRouter’s APIs — ideal for development, experimentation, and fine-grained control within specific projects.

#### Key Characteristics

* **User-linked:** Each key belongs to a specific user.
* **Permission-aware:** Inherits the user’s project-level permissions.
* **Self-service:** Can be created by any project member.
* **Flexible use:** Best suited for testing, scoped integrations, or per-user API access.

{% embed url="<https://youtu.be/cnVH9E27ppc>" %}

***

### Creating an API Key

When you create an API key, **FastRouter generates a one-time visible token.** Be sure to copy and store it securely — it **cannot be retrieved later** for security reasons.

***

### Key Settings

#### **Key Name**

Provide a custom label to identify the key within your project.

***

#### **Budget Controls**

* Set a **maximum spend** (in USD) for this key.
* Leave blank for unlimited usage.

**Reset Budget**

Choose when the key’s budget resets:

* **Never** (default)
* **Daily**
* **Weekly**
* **Monthly**

***

#### **Advanced Settings**

**Select Models**

* Choose which **LLM models** this key can access.
* **Default:** All models selected under the project.

**Rate Limits: Tokens per Minute (TPM)**

* Limit how many tokens this key can consume per minute.
* Cannot exceed the project-level TPM cap.

**Rate Limits: Requests per Minute (RPM)**

* Set the maximum number of requests per minute.
* Cannot exceed the project-level RPM cap.

**Expire Key**

* Schedule an automatic expiration date and time.
* Useful for **temporary access**, **contractor accounts**, or **testing environments**.

***

#### **Disable Content Logging**

* Prevents logging of requests and responses associated with this key.
* Recommended for keys handling **sensitive or private data**.

***

### Best Practices

* **Rotate keys periodically** and revoke unused ones.
* **Set budget and rate limits** to prevent accidental overspending.
* **Tag keys by project or purpose** for better organization.
* **Restrict model access** for tighter control and security.


# Add External Keys (BYOK)

Bring Your Own Key — Connect your own provider credentials to FastRouter and route traffic through your own accounts while retaining FastRouter's full routing, observability, and governance layer.

### Overview

FastRouter's External Keys feature lets organization owners attach API credentials from any supported LLM provider directly to their organization. Traffic routes through your own provider account, preserving your negotiated pricing, rate limits, and compliance posture.

Each integration is a named, project-scoped record pairing a provider with credentials and a model selection. Multiple integrations can coexist for the same provider — for example, separate Anthropic integrations for production and research, each scoped to different projects.

***

### Prerequisites

* You must be an **Organization Owner** to create, edit, or delete integrations.
* Project Admins and Developers can use integrations within their project scope but cannot modify credentials.

***

### Supported providers

| Provider         | Credential type | Notes                                        |
| ---------------- | --------------- | -------------------------------------------- |
| Anthropic Claude | API Key         |                                              |
| Baseten          | API Key         |                                              |
| DeepInfra        | API Key         |                                              |
| FAL AI           | API Key         |                                              |
| Fireworks AI     | API Key         |                                              |
| Google AI Studio | API Key         |                                              |
| Google Vertex AI | Service Account | Service Account JSON required                |
| Groq             | API Key         |                                              |
| Microsoft Azure  | API Key         | Resource Name and deployment config required |
| Minimax          | API Key         |                                              |
| Moonshot         | API Key         |                                              |
| Nebius           | API Key         |                                              |
| OpenAI           | API Key         |                                              |
| Perplexity AI    | API Key         |                                              |
| Pollo AI         | API Key         |                                              |
| Together AI      | API Key         |                                              |
| X-AI             | API Key         |                                              |

***

### Creating an integration

Navigate to **Setup → External Keys** and click **New Integration**. The wizard has three steps.

#### Step 1 — Select a provider

Choose from the list above. Use the search box to filter by name.

<figure><img src="/files/A5jaXUgCPoKphzLoMxQ0" alt=""><figcaption><p>Select a provider</p></figcaption></figure>

#### Step 2 — Integration details

Enter configurations and credentials.

| Field             | Required | Notes                                                                        |
| ----------------- | -------- | ---------------------------------------------------------------------------- |
| Name              | Yes      | Display name, e.g. "Anthropic Production"                                    |
| Provider Slug     | Yes      | Unique identifier used in API headers, gateway configs, and the Activity Log |
| Short Description | No       |                                                                              |
| Project Scope     | Yes      | "All Projects" or one or more specific projects                              |

> The **Provider Slug** appears in your Activity Log on every request, making it easy to trace which credential was used.

<figure><img src="/files/VhDlBuLUK33KEec1F4zG" alt=""><figcaption><p>Add provider key &#x26; configuration</p></figcaption></figure>

**Credential fields** vary by provider. For most, you supply an API key. Three providers support multiple authentication modes:

**Google Vertex AI**

| Auth mode            | Required fields      |
| -------------------- | -------------------- |
| Service Account File | JSON key file upload |

**Microsoft Azure**

| Auth mode         | Required fields                                 |
| ----------------- | ----------------------------------------------- |
| Default (API Key) | API Key, Azure Resource Name, deployment config |

**Advanced options** (collapsed by default)

* **Custom Host** — override the provider's default base URL. When set, select an API format: OpenAI-compatible, Anthropic-compatible, or Cohere-compatible.
* **Custom Auth Headers** — forwarded to your endpoint without logging.

#### Step 3 — Model provisioning

> **FastRouter catalog models only.** Only models in the FastRouter Model Catalog are routable, even with auto-enable on. Contact support to request a model addition.

* Toggle catalog models on or off individually, or use All / None.
* For now, models from the catalog or custom models have to mapped per provider.
* \[COMING SOON: **Auto-enable** — new catalog models from this provider are enabled on this integration automatically.]

<figure><img src="/files/qfb3C5EJf5b0dmqfUdtD" alt=""><figcaption><p>Enable/disable specific provider models</p></figcaption></figure>

**Adding custom models**

Use **Add Custom Model** to register fine-tuned or privately hosted models.

| Field                    | Required | Notes                                                                                                                                             |
| ------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| Model Slug               | Yes      | The `model` identifier used in API calls. Must be unique within this integration.                                                                 |
| Base Model               | No       | An existing catalog model your custom model is API-compatible with. Tells FastRouter which request/response schema to use — no effect on pricing. |
| Custom Host              | No       | Override the endpoint for this specific model only.                                                                                               |
| Input / Output pricing   | No       | USD per 1M tokens, for cost tracking in the dashboard.                                                                                            |
| Additional token pricing | No       | JSON for provider-specific token categories, e.g. `{"cache_read_input_tokens": 0.30}`                                                             |

<figure><img src="/files/RgYpo1mVW0i4rrC6MFRP" alt=""><figcaption><p>Configure custom models &#x26; endpoints</p></figcaption></figure>

**Note:** Custom models can be edited or deleted at any time from the model list.

***

### Editing an integration

Click **Edit** on any integration card. The edit wizard has two steps — Integration Details and Model Provisioning. The provider cannot be changed after creation.

***

### Security

* Credentials are encrypted at rest and never returned in API responses after saving.
* Rate limits and costs are governed by your provider account, not FastRouter's shared key pools.
* Only Organization Owners can create, modify, or delete credentials.

***

### Using BYOK integrations

Once active, reference an integration via its **Provider Slug** in:

* **Virtual Models** — associate an alias with a specific integration
* **Gateway Configs** — use in fallback or load-balancing configurations
* **Activity Log** — every request shows the Provider Slug of the integration used


# Custom Evaluations

FastRouter’s Custom Evaluations lets you benchmark and compare AI models on your own data—using LLM-based judges to automatically score accuracy, latency, and cost.

### **Introduction**

FastRouter's Custom Evaluations feature allows you to assess and compare the performance of AI models on your datasets. By importing chat completion logs or datasets, generating model outputs (runs), and defining evaluation criteria with LLM-based judges, you can quantitatively measure aspects like accuracy, relevance, latency, and cost. This is ideal for benchmarking models, optimizing prompts, and ensuring high-quality responses in production.

Evaluations are managed through the FastRouter dashboard, where you can create, run, and analyze evaluations asynchronously. Results include detailed metrics and scores, helping you make data-driven decisions.

{% embed url="<https://youtu.be/ISDIMfOA6J4>" %}

***

### **Key Benefits**

* Automated judging using LLM evaluators for scalable assessments.
* Support for custom criteria tailored to your use case (e.g., factual accuracy, creativity, conciseness).
* Integration with your API keys for secure, cost-effective processing.
* Visual dashboards for easy comparison of runs and metrics.

***

### **Creating a New Evaluation**

To start, navigate to the "Evaluations" section in your FastRouter dashboard and click "New Evaluation."

<figure><img src="/files/Nyu9tj8X72KLbjAR6ZQP" alt=""><figcaption><p>Custom Evaluations: Create Evaluation</p></figcaption></figure>

1. **Name Your Evaluation:** Provide a descriptive name (e.g., "Math Query Benchmark").
2. **Import Test Data:** Upload or import chat completion logs or datasets. You can:
   * Select a project and model (e.g., "Anthropic Claude 4.5").
   * Filter by date range.
   * Choose input/output text to filter rows.
   * Set a sample size (e.g., 10%) to evaluate a subset of your data for efficiency.

<figure><img src="/files/Hoa1d1N13Xk8J9PwnDZ4" alt=""><figcaption><p>Custom Evaluations: Import Test Data</p></figcaption></figure>

3. **Add Runs:** Select model runs to generate outputs (e.g., "anthropic/claude-4.5"). You can add multiple runs for side-by-side comparison.

<figure><img src="/files/thP7Dj2yqFRZMEUaNrSu" alt=""><figcaption><p>Custom Evaluations: Add Run</p></figcaption></figure>

4. **Add Test Criteria:** Create one or more evaluation criteria using an LLM judge.

* Click "Add Test Criteria" and choose a type (e.g., Model Scorer for quantitative scoring).
* Configure the LLM judge: Select a model (e.g., "openai/gpt-5"), system prompt (e.g., "You are an expert AI response evaluator..."), and user prompt template (e.g., "Rate the response on \[criteria] from 1-10") with any variables.
* To access any values for evaluation by the LLM judge in the input or generated output, you can use the variables: *{{item.input}}, {{item.column\_name}} or {{sample.output}}.*
* Define scoring rubrics, such as pass/fail thresholds or numeric scales.

<figure><img src="/files/xudw6dsKPo8Xo0hH06ki" alt=""><figcaption><p>Custom Evaluations: Add Test Criteria</p></figcaption></figure>

<figure><img src="/files/3U1fNM6dHfEchZ7cEiD2" alt=""><figcaption><p>Custom Evaluations: Edit Test Criteria</p></figcaption></figure>

5. **Select an Evaluation Key:** Choose an API key from your account to handle generation and evaluation requests. This key will be used for all API calls during the evaluation.

<figure><img src="/files/3qGEgzU88Qv7DvjvpRSb" alt=""><figcaption><p>Custom Evaluations: Configure Evaluation Key</p></figcaption></figure>

6. **Run the Evaluation:** Click "Run" to start processing. The evaluation will generate outputs for each run and apply the judges asynchronously.

***

### **Viewing Evaluation Results**

Once the evaluation is complete, access the results of a particular custom evaluation from the Evaluations listing page.

* **Overview:** See a summary of runs, including model names, request IDs, input samples, generated outputs, testing criteria scores (e.g., Latency, Cost), and overall scores.

<figure><img src="/files/LDhQOge0Ks8nyZQc0K8B" alt=""><figcaption><p>Custom Evaluations: Evaluation Runs Overview</p></figcaption></figure>

* **Comparison and Analysis:** Compare multiple runs side-by-side. Metrics include:
  * **Score:** Aggregated from your criteria (e.g., 7/10 for accuracy).
  * **Latency:** Time to generate responses.
  * **Cost:** Token-based billing.
  * Custom metrics based on your judges.
* **Detailed Metrics:** For each run, view aggregated stats like average score, latency (in ms), cost, and pass rate. Drill down into individual responses for judge reasoning.
* **Detailed Metrics:** For each run, view aggregated stats like average score, latency (in ms), cost, and pass rate. Drill down into individual responses for judge reasoning.

<figure><img src="/files/1O7YdT4PGqdzXYeHnPtM" alt=""><figcaption><p>Custom Evaluations: Evaluation Run Details</p></figcaption></figure>

* **Judge Reasoning:** For each test criterion and score, you can drill down into the individual responses for details of the judge reasoning.

<figure><img src="/files/ky0nRebxscbnUgOVGnYk" alt=""><figcaption><p>Custom Evaluations: Judge Reasoning</p></figcaption></figure>

***

### **Tips & Best Practices**

* **Start Small:** Begin with a small sample size (e.g., 10-50 rows) to test your setup before scaling to larger datasets.
* **Diverse Criteria:** Use multiple judges for comprehensive evaluations (e.g., one for factual accuracy, another for response conciseness).
* **Judge Calibration:** Test your LLM judge prompts on sample data to ensure unbiased and consistent scoring.
* **Cost Management:** Monitor estimated costs in the setup phase. Use efficient models for judges to minimize expenses.


# Video Evaluations

Evaluate AI-generated videos at scale using LLM-based judges, with automated scoring across motion, sync, quality, and prompt adherence.

#### Introduction

FastRouter's Video Evaluations feature lets you assess the quality of AI-generated videos at scale. By importing video generation logs, defining LLM-based judging criteria, and running evaluations against those outputs, you can systematically measure video quality across dimensions like motion fidelity, audio-visual sync, cinematic quality, and adherence to the original prompt.

Video Evals work within the same Custom Evaluations infrastructure as text and image evals — the same judge configuration, the same scoring rubrics, and the same results dashboard — extended to support multimodal video output.

***

#### Key Benefits

* Evaluate AI-generated video outputs automatically using an LLM judge.
* Import video generation logs directly from your FastRouter activity — no manual uploads required.
* Use the same custom criteria and Auto Grader setup as text evaluations.
* Drill down into per-video judge reasoning to understand exactly what scored well or poorly.

***

#### Creating a Video Evaluation

Navigate to the **Evaluations** section in your FastRouter dashboard and click **Create Evaluation**.

<figure><img src="/files/Hu68jk7DlSj6Yx6TpzMH" alt=""><figcaption><p>Custom Evaluations</p></figcaption></figure>

**Step 1 — Name Your Evaluation**

Provide a descriptive name (e.g., `Video Compliance Evaluation` or `Product-Video-Quality-Check-v2`).

<figure><img src="/files/JpWrCEA9sYmiq825DhJJ" alt=""><figcaption><p>Name Your Evaluation</p></figcaption></figure>

**Step 2 — Import Video Logs**

Click **Import Data**. In the Import Test Data dialog, select the **Videos** tab.

<figure><img src="/files/aJyv40jj1xOk3pr9S4Ry" alt=""><figcaption></figcaption></figure>

Configure the following fields:

* **Date Range** *(required)*: Select the date range covering the video generations you want to evaluate.
* **Model** *(required)*: Choose the video generation model whose outputs you want to import (e.g., `google/veo3.1-lite`).
* **Project**: Optionally filter by project. Select a project to narrow the available API keys, or leave as "All Projects" to see all keys.
* **Key**: Optionally filter by a specific API key used during generation.
* **Input contains**: Search for specific text in the generation input to narrow down which logs are imported.
* **Sampling rate (%)**: Set a percentage of matching logs to import (1–100%). Useful for large log sets — start with a smaller sample to validate your setup before scaling.

> ℹ️ Video file logs are available for import approximately **2 hours** after they are generated. If your recent video logs aren't showing up, try again later.

Click **Import** to load the video generation logs as your evaluation dataset.

**Step 3 — Add Evaluation Metrics**

Click **Add Metric** and configure an **Auto Grader** (LLM-based judge) for your video outputs.

* **Judge Model**: Select a capable multimodal model (e.g., `gemini-3.1-flash-lite-preview`) that can process video as input.
* **System Prompt**: Describe the evaluator's role and scoring approach. Example:

  ```
  You are an expert AI response evaluator tasked with assessing model outputs
  for quality and effectiveness. Evaluate video outputs across three dimensions:
  1. Major errors, safety concerns, or failure to perform the core task.
  2. Minor issues such as artifacts, animation inconsistencies, or audio-visual sync problems.
  3. Suggestions for higher quality animation, better motion dynamics, and tighter sound integration.
  ```
* **Scoring**: Define a numeric scale (e.g., 0–10) or pass/fail threshold. The judge will return both a score and structured reasoning per dimension.
* **Variables**: Reference the video output in your judge prompt using `{{sample.output}}`. Use `{{item.input}}` to pass the original generation prompt to the judge for context.

**Step 4 — Select Evaluation API Key**

Choose an API key from your account. This key will be used for all LLM judge calls during the evaluation.

**Step 4 — Run**

Click **Run** to start the evaluation. FastRouter will apply your judge asynchronously to each video in the dataset and return scored results.

<figure><img src="/files/XHRwLor7xAOXU6i47ySn" alt=""><figcaption></figcaption></figure>

***

#### Viewing Results

Access results from the **Evaluations** listing page by clicking your evaluation.

* **Data view**: See each video row with its generation input, a video preview thumbnail, Auto Grader score, latency, and cost.
*

```
<figure><img src=".gitbook/assets/Content Preview.png" alt=""><figcaption><p>Content Preview</p></figcaption></figure>
```

* **Report view**: Aggregated metrics across all rows — average score, pass rate, latency distribution, and cost.

<figure><img src="/files/5dD6fnZpSmfWI04lui66" alt=""><figcaption><p>Report Details</p></figcaption></figure>

* **Judge Reasoning**: Click any individual row score to expand the full LLM judge reasoning — broken down by evaluation dimension (e.g., safety check → minor issues → improvement suggestions).

<figure><img src="/files/dMVQZttbULrPt2MAUPJF" alt=""><figcaption><p>Judge Feedback</p></figcaption></figure>

**Example output for a tiger image-to-video eval (google/veo3.1-lite):**

| Metric            | Value          |
| ----------------- | -------------- |
| Auto Grader Score | Pass: 5.5 / 10 |
| Latency           | 1,115 ms       |
| Cost              | μ$400,000      |
| Video Length      | 8 seconds      |

Judge reasoning summary:

1. **Major errors / safety**: None identified.
2. **Minor issues**: Animation was extremely subtle (near-static); audio present but not closely synchronized with the visual action.
3. **Improvements**: Increase motion complexity (eye blinking, ear movement, water ripples); tighten audio-visual sync to specific visual moments.

***

#### Tips & Best Practices

* **Start with a small sample**: Use the sampling rate slider to import 10–20% of your logs first. Validate your judge prompt on a handful of videos before scaling to your full dataset.
* **Use a capable judge model**: Video evaluation requires a multimodal LLM that can process video frames. Choose models that support video input explicitly.
* **Be specific in your rubric**: Vague judge prompts produce inconsistent scores. Break your evaluation into named dimensions (e.g., motion quality, prompt adherence, audio sync) and score each separately.
* **Allow 2 hours post-generation**: Video logs take approximately 2 hours to become available for import. Plan your eval runs accordingly.
* **Monitor costs**: Video judge calls can be more expensive than text, especially with longer clips. Start with shorter videos and efficient judge models.

***

#### Relationship to Custom Evaluations

Video Evals are an extension of FastRouter's [Custom Evaluations](https://claude.ai/chat/custom-evaluations.md) feature. The same infrastructure — dataset management, run comparison, judge configuration, and results dashboard — applies to both. The key difference is the data source: instead of importing chat completion logs or CSV files, you import video generation logs via the **Videos** tab in the Import Test Data dialog.

All judge configuration options available for text evals (scoring rubrics, variable interpolation, multi-criteria graders) are fully supported for video evals.


# Prompt Library

Manage, version, optimize, and deploy prompts independently of your code—enabling instant updates and rollbacks without application redeploys.

## Introduction

Prompt Library is the single place to write, store, version, test, and optimize your prompts before deploying them to production. Instead of hard-coding prompt strings into your application, you author them in FastRouter, manage them as versioned records, and reference them by ID in your API calls. When you ship a change, you publish a new version — no code deploy required.

This page covers what Prompt Library does, how it works, and how to call a stored prompt from the API.

### Why use it

Prompts change far more often than application code. Keeping them in Prompt Library gives you:

* **A versioned history** of every prompt, with notes on what changed in each version.
* **Safe rollouts** — mark exactly one version as *Production* and have all live requests use it, then roll back instantly by promoting an older version.
* **Optional optimization** — refine a system prompt with GEPA and save the result as a new version, without overwriting your original.
* **Decoupled deployments** — update the prompt your app runs without redeploying the app, since requests reference the prompt by ID.

### How it works

The lifecycle is: author a prompt in the **Prompt Library**, optionally **optimize** it, save changes as a **new version**, then **reference it by ID** from your application.

<figure><img src="/files/TO6eJnHu9CdicyYQl0O9" alt=""><figcaption><p>Prompt Library flow: Prompt Library → (optional) Prompt Optimization → New Version → Your App</p></figcaption></figure>

1. **Create version** — Write and store a prompt in the Prompt Library.
2. **Optimize** (optional) — Run GEPA to refine the system prompt.
3. **Save optimized prompt** — The refined prompt is saved as a new version.
4. **Call by ID** — Your application references the prompt by its ID in API calls.

### Creating a prompt

From **Prompts → Prompt Library**, click **Create Prompt** and fill in:

* **Prompt Name** — A human-readable name (e.g. `Health Assistant Prompt`).
* **Tags** — Up to five comma-separated tags for organization (e.g. `health`).
* **Prompt** — The system prompt text. Use `{{curly braces}}` to insert variables that you fill in at call time.
* **What changed in this version?** — A short changelog note (e.g. `Initial draft`). This is required and builds your version history.
* **Set as "Production"** — When checked, this version is used for all live requests that reference the prompt by ID.

<figure><img src="/files/0pDy6iC5spZaXy5ldXj0" alt=""><figcaption><p>New Prompt form with name, tags, prompt body, version note, and Set as Production toggle</p></figcaption></figure>

Saving creates **v1** of the prompt and assigns a permanent prompt ID (e.g. `pmpt_3c743f7f9f9e467eae6525f00e6e0650`). The ID never changes across versions — it's the stable handle your application uses.

<figure><img src="/files/sGJQZ6kMOcwdeBLHwP7B" alt=""><figcaption><p>Prompt Details showing v1 as the only version, marked Latest</p></figcaption></figure>

### Versioning

Each prompt keeps an ordered list of versions in the left panel of **Prompt Details**. Click **Add New** to create a new version, or generate one through optimization. Every version records its author, timestamp, and change note. The most recent version is tagged **Latest**; the version you promote is served as **Production**.

### Optimizing a prompt (optional)

Click **Optimize** on any prompt to run GEPA based [Prompt Optimizations](/prompt-optimizations) against the current version. GEPA refines the system prompt and saves the result as a new version tagged **Optimized**, leaving your original untouched so you can compare or revert.

<figure><img src="/files/et4CX2y97JCyTVrlAEOG" alt=""><figcaption><p>Prompt Details showing v2 (Latest, Optimized) alongside v1, with the expanded optimized prompt text</p></figcaption></figure>

The optimized version is annotated with the optimizer job that produced it (e.g. `Optimized via optimizer job opt_094b50e343624dad99d127f4d57b28d7`), so you can trace any version back to its source. Use **Compare** to view the diff between two versions before promoting one to Production.

<figure><img src="/files/Sr7OaTdYlwLGA1bKKV79" alt=""><figcaption><p>Compare Prompt Versions showing the differences between two prompt versions</p></figcaption></figure>

### Calling a prompt by ID

Reference a stored prompt by passing its `prompt_id` to the chat completions endpoint. When you do this **without pinning a version, FastRouter serves the version currently marked Production** — so promoting a new Production version changes what your app runs, with no code change on your side.

The **API Usage** tab on the Prompt Details page generates a ready-to-run snippet for the selected prompt:

<figure><img src="/files/Gjjgc6UVMbynC6MsXbq2" alt=""><figcaption><p>API Usage tab that shows the configuration to be used in your requests</p></figcaption></figure>

```bash
curl -X POST "https://api.fastrouter.ai/api/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer API-KEY" \
  -d '{
    "model": "openai/gpt-4.1",
    "prompt_id": "pmpt_dde193a153a14fbf9829dd9499bf828b",
    "variables": {},
    "messages": [
      { "role": "user", "content": "hi" }
    ]
  }'
```

#### Request parameters

| Parameter   | Required | Description                                                                                                             |
| ----------- | -------- | ----------------------------------------------------------------------------------------------------------------------- |
| `model`     | Yes      | The model to route the request to (e.g. `openai/gpt-4.1`).                                                              |
| `prompt_id` | Yes      | The stored prompt's ID. Resolves to the **Production** version.                                                         |
| `variables` | No       | Key–value map filling any `{{variables}}` declared in the prompt. Defaults to `{}`.                                     |
| `messages`  | Yes      | The conversation turns. The stored prompt is applied as the system prompt; `messages` carries the user/assistant turns. |

#### Working with variables

If your prompt contains placeholders such as `{{patient_age}}` or `{{topic}}`, supply their values in `variables`:

```json
{
  "model": "openai/gpt-4.1",
  "prompt_id": "pmpt_dde193a153a14fbf9829dd9499bf828b",
  "variables": {
    "topic": "ibuprofen dosing"
  },
  "messages": [
    { "role": "user", "content": "Is this safe with my blood pressure medication?" }
  ]
}
```

FastRouter substitutes the values into the stored prompt before sending the request to the model. Detected variables for a prompt are listed at the bottom of the **Prompt** tab.

### Promoting and rolling back

To change what production traffic uses, open the target version and set it as **Production** (or check **Set as "Production"** when saving a new version). Because your application references the prompt only by `prompt_id`, the switch takes effect immediately for all live requests, and you can revert by promoting a previous version the same way.


# Prompt Optimizations

Automatically refine your system prompts using GEPA — a reflective prompt evolution algorithm that iteratively improves prompts using LLM-judged feedback as gradients.

### Overview

**What is GEPA?** GEPA (Genetic-Pareto) prompt optimization is a state-of-the-art evolutionary algorithm that automatically refines LLM prompts over iterations using reflection and mutation to achieve peak performance. It uses a "reflection" model to analyze prompt failures, then mutates prompts based on this feedback, keeping only the best variants using Pareto optimization.

FastRouter's Prompt Optimization based on GEPA has three views.

### 1. List View

The default landing page for the feature. Shows all optimization runs across your organization.

<figure><img src="/files/v5uaV1I0N9hU3QFdtdUO" alt=""><figcaption><p>Prompt Optimization List</p></figcaption></figure>

> Prompt Optimization — List View The list view shows all optimization runs with their model, improvement %, and status (Completed / In Progress).

| Column             | Description                              | Notes                                          |
| ------------------ | ---------------------------------------- | ---------------------------------------------- |
| Name               | Task name set during creation            | Clicking the row navigates to the Details view |
| Created            | Creation timestamp                       | Format: DD Mon YYYY                            |
| Optimization Model | Target LLM for this run                  | Displayed in monospace                         |
| Improvement %      | Score delta vs. baseline composite score | Shows `—` until run completes                  |
| Status             | Current state of the run                 | `Completed` · `In Progress` · `Failed`         |
| Action             | Row-level actions menu                   | Ellipsis icon, right-aligned                   |

***

### 2. Create View

A two-panel page. The left panel contains the configuration form; the right panel shows a live **Data (preview)** once test data has been imported.

<figure><img src="/files/83xZDL5qGD2Gb8VZPrqY" alt=""><figcaption><p>Create New Optimization</p></figcaption></figure>

> New Optimization — Create Form The create view before any configuration. Right panel shows "No Test Data Provided." until a dataset is imported.

#### A. Form Sections

**1 — Name**

Auto-populated with a timestamp (e.g., `New Optimization 2026-04-06 17:17:37`). Edit to give the run a meaningful name.

**2 — Base Prompt & Input Data&#x20;*****(Required)***

Click **+ Add Prompt & Dataset** to open the Setup Optimization Context modal. Provide your baseline system prompt and import a test dataset.

**3 — Optimizer Configurations**

Click **+ Configure Optimization** to set the target model, reflection model, budget tier, and batch size.

**4 — Evaluation Metrics**

Click **+ Add Metrics** to define judge criteria. Each metric makes an independent LLM judge call and returns a score (0–1) plus textual feedback used as a gradient.

**5 — Evaluator Model (LLM-as-a-Judge)&#x20;*****(Required)***

A single shared model used to score all evaluation metrics. Applies consistently across every metric for comparable scores.

**6 — Optimization Key&#x20;*****(Required)***

Select the API key to bill for this optimization run. Dropdown lists all available keys for the organization.

**7 — Run**

Clicking **Run** opens the Credit Utilization Estimate modal to confirm cost before the job starts.

***

#### B. Right Panel — Data Preview

Once a dataset is imported, the right panel becomes a live preview labelled **Data (preview)** with a total row count.

<figure><img src="/files/xkbSJHcCMRnraclWSFKS" alt=""><figcaption><p>Input Data &#x26; Preview</p></figcaption></figure>

> Create View with Data Preview. After importing data, the right panel shows Input / Output / Feedback columns. Feedback rows flagged as Bad are highlighted in red.

| Column   | Description                                                        |
| -------- | ------------------------------------------------------------------ |
| Input    | System prompt + user message for each test row                     |
| Output   | Model response for that input (from logs or file)                  |
| Feedback | Human label — **Good** or **Bad**. Drives GEPA reflection quality. |

***

#### C. Modal — Setup Optimization Context

Opened by clicking **+ Add Prompt & Dataset**. Two-step wizard with a progress tab bar: **Base Prompt** → **Input Data**.

**Tab 1 - Base Prompt**

<figure><img src="/files/JDpbx2FkW8i0u41SibW7" alt=""><figcaption><p>Base Prompt</p></figcaption></figure>

> Setup Optimization Context — Base Prompt Tab Enter the system prompt to be optimized. GEPA will evolve this prompt across iterations. The info note reminds you that only matching logs will be imported.

| Field                | Type               | Notes                                                                                                     |
| -------------------- | ------------------ | --------------------------------------------------------------------------------------------------------- |
| Target System Prompt | Multiline textarea | The baseline prompt GEPA will iteratively improve. Example: `You are a helpful assistant who loves haiku` |
| Info note            | Read-only          | "Only logs matching this prompt with other applied filters will be imported."                             |
| Discard              | Button (top-right) | Closes modal without saving                                                                               |
| Next                 | Primary button     | Advances to the Input Data tab                                                                            |

***

**Tab 2 - Input Data**

Two sub-tabs: **Files** (upload a CSV / JSON / JSONL) and **Chat Completions** (import from Activity Log).

<figure><img src="/files/CZiXasXOA4FiItDCezwo" alt=""><figcaption><p>Import Test Data From Chat Completions</p></figcaption></figure>

> Setup Optimization Context — Input Data, Chat Completions Tab Filter completions from the Activity Log by date range, model, project, key, and metadata. Total matching rows shown at the bottom before importing.

| Field           | Required  | Description                                                              |
| --------------- | --------- | ------------------------------------------------------------------------ |
| Date Range      | Required  | Preset picker — default: Last 7 days                                     |
| Model           | Required  | Filter completions by model. Example: `openai/gpt-5-mini`                |
| Project         | Optional  | Filter by project; leave as All Projects to see keys across all projects |
| Key             | Optional  | Filter by specific API key within the selected project                   |
| Input contains  | Optional  | Free-text search on completion inputs                                    |
| Output contains | Optional  | Free-text search on completion outputs                                   |
| Metadata        | Optional  | Click **+ Add Metadata Values** to add key-value filters                 |
| Total rows      | Read-only | Count of completions matching all applied filters (e.g., 3)              |
| Back / Import   | —         | Back returns to Base Prompt tab; Import confirms and closes the modal    |

**Tip:** The Activity Log stores feedback annotations (Good / Bad thumbs). Rows with feedback improve GEPA reflection quality — GEPA can use the "Bad" labels to prioritise which failures to fix first.

***

#### D. Modal — Optimizer Configurations

Opened by clicking **+ Configure Optimization**. Configure how GEPA tunes and evaluates your prompt.

<figure><img src="/files/ql3ujJWa7PptSLHJqcDd" alt=""><figcaption><p>Optimizer Configurations</p></figcaption></figure>

> Optimizer Configurations Modal Select the target model, reflection model, budget tier, and batch size. Batch size can be any of 3, 6, 9, 12, 15 or 18 depending on the number of total input samples.

| Field               | Required | Description                                                                      |
| ------------------- | -------- | -------------------------------------------------------------------------------- |
| Optimization Model  | Required | The model whose prompt will be optimized and used in production                  |
| Reflection Model    | Required | Reviews failures and scores candidate prompts. Can differ from the target model. |
| Optimization Budget | Required | Controls iteration count and cost. Higher budgets may improve quality.           |
| Batch Size          | Required | Number of samples used per GEPA step. Larger = more stable, but slower.          |

#### E. Optimization Budget Tiers

| Tier       | Description                    | Iterations |
| ---------- | ------------------------------ | ---------- |
| **Light**  | Fast, directional optimization | 10         |
| **Medium** | Balanced quality vs cost       | 25         |
| **Heavy**  | Maximum quality, higher cost   | 50         |

**Cost note:** Higher budgets run more GEPA iterations and metric evaluations. Each iteration calls the Optimization Model, Reflection Model, and Evaluation Model — costs compound quickly. Check the Credit Utilization Estimate before running.

***

#### F. Modal — Evaluation Metrics

Opened by clicking **+ Add Metrics**. Add one metric per modal invocation; repeat to add multiple. Each metric is evaluated independently by the shared Evaluator Model (LLM-as-a-Judge).

| Field               | Required  | Description                                                                                                          |
| ------------------- | --------- | -------------------------------------------------------------------------------------------------------------------- |
| Metric              | Required  | Dropdown with predefined options (Accuracy, Helpfulness, Tone & Style, Safety, Completeness…) and a **Custom** entry |
| Evaluation Criteria | Required  | Judge prompt auto-filled for predefined metrics. For Custom, write your own.                                         |
| Score Range         | Read-only | Continuous 0–10                                                                                                      |

**Predefined Metrics**

| Metric       | Pre-filled Criteria Summary                                                          |
| ------------ | ------------------------------------------------------------------------------------ |
| Accuracy     | Is the response factually correct? Checks hallucinations and omissions.              |
| Helpfulness  | Does the response address the user's actual need? Are all relevant points covered?   |
| Tone & Style | Is the tone appropriate for context? Evaluates empathy, jargon usage, and verbosity. |
| Safety       | Checks for harmful, biased, or inappropriate content.                                |
| Completeness | Does the response fully cover the expected scope without gaps?                       |

**Custom Metric**

Selecting **Custom** from the dropdown reveals:

* **Metric Name** *(required)* — e.g., `Conciseness`
* **Evaluation Criteria** — empty textarea; write the full judge prompt
* **Score Range** — Continuous 0–10 (read-only)

**Metric Cards**

After adding a metric, it appears as a card in the Create form with an edit (✏️) and delete (🗑️) icon. Example:

<figure><img src="/files/RDI6bTzMIyld1NpoobhU" alt=""><figcaption><p>Add Evaluation Metric</p></figcaption></figure>

> Evaluation Metrics — Completeness Metric Card Shows Judge Model, Success Score (0–10), and a truncated Evaluation Criteria preview. Edit and delete icons appear top-right.

**How GEPA uses metric feedback:** The judge returns both a numeric score and a `feedback` text for every output. GEPA averages scores across all enabled metrics into a composite score, then concatenates the feedback text and passes it to the Reflection Model as a "textual gradient" to guide the next prompt revision.

***

#### G. Modal — Credit Utilization Estimate

Shown automatically when you click **Run**. Provides a cost summary before committing credits.

> Checking Credit Utilization Estimate Modal The modal shows total samples, estimated cost, and current account balance. A "Sufficient credits available" confirmation appears when balance exceeds the estimate.

| Field                     | Description                                                                       |
| ------------------------- | --------------------------------------------------------------------------------- |
| Total samples             | Number of test rows that will be evaluated (e.g., \~3 samples)                    |
| Estimated cost            | Projected spend for the full optimization run (e.g., \~$0.000008)                 |
| Current balance           | Your organization's current credit balance (e.g., $82.03)                         |
| Status banner             | **Sufficient credits available** (green) or a warning if balance is low           |
| Proceed with Optimization | Full-width primary button — starts the GEPA run and navigates to the Details view |

Estimates are based on selected models and average prompt size. Actual costs may vary. If credits run out mid-run, the optimization may not complete.

***

### 3. Details View

Navigated to after clicking **Proceed with Optimization**, or by clicking any row in the List. Displays real-time progress during the run and full results once complete.

<figure><img src="/files/7PRtsdreJVIdvnbvFJCd" alt=""><figcaption><p>Prompt Optimization Details</p></figcaption></figure>

> Optimization Details — Completed State Completed view showing the Optimized Prompt with Final Score (1.000, 14% improvement), Configurations summary, and the All Iterations panel on the right with per-iteration scores.

#### Prompt Result Card

| Element               | Description                                                |
| --------------------- | ---------------------------------------------------------- |
| Optimization Complete | Green checkmark + status header                            |
| Copy to Clipboard     | Copies the full optimized prompt text                      |
| Optimized Prompt tab  | Default tab — shows the final evolved system prompt        |
| Initial Prompt tab    | Shows the original baseline prompt for comparison          |
| Final Score           | Composite score across all enabled metrics (e.g., `1.000`) |
| Improvement %         | Delta vs. baseline (e.g., **14% Improvement**)             |

#### All Iterations Panel

Accessible via the **Data** tab at the top-right of the Details page. Shows one card per GEPA iteration in a scrollable panel.

Each iteration card displays:

| Field             | Description                                                                            |
| ----------------- | -------------------------------------------------------------------------------------- |
| Iteration label   | Default (baseline) / Iteration 1 / Iteration 2…                                        |
| Status            | ✓ Accepted (green) or Rejected                                                         |
| Score             | Composite score (e.g., `0.593`)                                                        |
| Per-metric scores | Individual scores for each enabled metric (e.g., `Accuracy: 0.85 · Test Metric: 0.90`) |
| Prompt preview    | Truncated text of the prompt used in that iteration                                    |

***

### 4. Activity Log — Feedback for Optimization

Enrich optimization datasets by annotating completions directly in the Activity Log before importing them. Feedback annotations become training signal for GEPA.

<figure><img src="/files/QmIYpuNd0Av04j7DGZNB" alt=""><figcaption><p>Activity Log: Add Feedback</p></figcaption></figure>

> Activity Log — Log Detail Panel with Feedback The Logs detail panel shows Summary, Preview (input/output), Feedback annotation (Good/Bad with comment), Metadata, and Invocation Parameters.

#### Log Detail Sections

| Section  | Description                                                                                                                                   |
| -------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| Summary  | FastRouter ID, Model ID, Provider, TTFT, Latency, Total Cost, Token counts, Status, Timestamps, User ID, Session ID                           |
| Preview  | Full Input and Output text. Toggle between **Pretty** (formatted) and **JSON** views.                                                         |
| Feedback | 👍 Good / 👎 Bad annotation with optional free-text comment. Tagged as: *"This feedback may be used to improve future prompt optimizations."* |
| Metadata | Key-value pairs attached to the completion (e.g., `fruit: banana`). Filterable in the Chat Completions import flow.                           |

***

### 5. GEPA Terminology Reference

| UI Term                 | GEPA Concept              | Description                                                                   |
| ----------------------- | ------------------------- | ----------------------------------------------------------------------------- |
| Optimization Model      | Task LM                   | The model whose prompt is optimized and runs in production                    |
| Reflection Model        | Reflection LM             | Analyses failures and proposes prompt edits; can be smaller/cheaper           |
| Evaluator Model         | Evaluation LM             | Single shared judge scoring all metrics consistently                          |
| Optimization Budget     | Budget preset             | Controls total iteration count and associated cost                            |
| Batch Size              | Minibatch size            | Examples evaluated per GEPA step; larger = more stable, slower                |
| Evaluation Criteria     | Judge prompt              | LLM prompt defining the metric; must return `{"score": 0–1, "feedback": "…"}` |
| Feedback (Activity Log) | Textual gradient signal   | Good/Bad labels used to guide which failures GEPA prioritises                 |
| Accepted iteration      | Pareto-accepted candidate | Iteration whose prompt improved the composite score vs. previous best         |


# Prompt Compression

Prompt Compression intelligently shrinks prompts before they're sent to an AI model, reducing token usage while maintaining response quality. It helps lower costs and maximize available context.

### Overview

Opt-in compression for chat requests. Add one block to your request body and FastRouter compresses your messages before they reach the provider — cutting input tokens without changing the response.

* **Off by default** — nothing happens unless you opt in
* **Opt-in per request** — one block in the body
* **Fail-open** — if compression can't run, your original messages are sent unchanged

### How it works

```
Client ──► FastRouter gateway ──► compression ──► Provider (OpenAI / Anthropic / …)
```

You send a normal chat request plus a small `optimize` block. The gateway compresses eligible messages, forwards the compressed payload upstream, and returns the normal provider response with `X-FastRouter-Compression-*` headers reporting savings.

### Enabling it

Compression runs only when your request opts in — the body includes an `optimize.compress` block. Without it, the request flows through completely unchanged.

### Choosing an engine

Pick based on your content, not the implementation:

| Your content                               | Engine value           | What it does                                                                                | Type     |
| ------------------------------------------ | ---------------------- | ------------------------------------------------------------------------------------------- | -------- |
| Strict system prompts, rules, tool schemas | `"headroom"` (default) | Structural compression — restructures JSON/repetitive payloads without dropping information | Lossless |
| Chat prose, verbose instructions           | `"caveman"`            | Rule-based compaction — strips filler and redundancy, preserves all facts and constraints   | Lossy    |
| RAG chunks, docs, transcripts              | `"llmlingua"`          | ML token pruning — a small model scores and drops low-information tokens                    | Lossy    |
| Maximum savings on mixed content           | `"all"`                | Full pipeline: headroom → llmlingua → caveman                                               | Mixed    |

`engine` accepts a single value, a preset (`"both"`/`"hybrid"` = headroom → caveman, `"all"` = the full pipeline), or an explicit list like `["headroom", "llmlingua"]`. Application order is always **headroom → llmlingua → caveman**.

> **Try before you buy:** set `"mode": "audit"` to measure what you *would* save without changing a single byte of your request. Stats are still returned in the response headers.

### Request format

```json
{
  "model": "openai/gpt-5.5",
  "messages": [ ... ],
  "optimize": {
    "compress": {
      "engine": "llmlingua",
      "llmlingua_rate": 0.75
    }
  }
}
```

The `compress` object is an open key/value bag — any engine parameter is forwarded as-is. Omit the block entirely and no compression happens.

### Supported routes

| Surface                 | Endpoints                                                                              | Coverage                             |
| ----------------------- | -------------------------------------------------------------------------------------- | ------------------------------------ |
| OpenAI chat completions | `POST /v1/chat/completions`, `POST /api/v1/chat/completions`, `POST /chat/completions` | Streaming, non-streaming & SDK paths |
| Anthropic Messages      | `POST /v1/messages`, `POST /api/v1/messages`                                           | Streaming & non-streaming            |

Native Responses / Gemini handlers are not covered yet.

### Parameters

#### General

| Param         | Type                | Default      | Description                                                                     |
| ------------- | ------------------- | ------------ | ------------------------------------------------------------------------------- |
| `engine`      | string \| string\[] | `"headroom"` | Which compressor(s) to run.                                                     |
| `mode`        | string              | `"optimize"` | `"optimize"` applies transforms; `"audit"` only observes (still returns stats). |
| `provider`    | string              | inferred     | Provider hint for token counting.                                               |
| `cache_align` | bool                | `false`      | Improve provider prompt-cache hits (does not reduce tokens).                    |

#### Lossless structural (`engine: "headroom"`)

| Param                           | Type  | Description                                |
| ------------------------------- | ----- | ------------------------------------------ |
| `target_ratio`                  | float | Desired compression ratio target.          |
| `min_tokens_to_compress`        | int   | Skip blocks smaller than this.             |
| `compress_user_messages`        | bool  | Compress user-role messages.               |
| `compress_system_messages`      | bool  | Compress system-role messages.             |
| `protect_recent` / `keep_turns` | int   | Leave the N most recent turns untouched.   |
| `protect_analysis_context`      | bool  | Protect analysis/reasoning context blocks. |

#### Rule-based prose (`engine: "caveman"`)

| Param           | Type      | Default   | Description                                 |
| --------------- | --------- | --------- | ------------------------------------------- |
| `caveman_level` | string    | `"light"` | `"light"`, `"semantic"`, or `"aggressive"`. |
| `caveman_roles` | string\[] | all roles | Restrict to given roles, e.g. `["user"]`.   |

Never drops negations, modals, quantifiers, code, URLs, paths, numbers, quoted strings, or ALL-CAPS codes.

#### ML prose (`engine: "llmlingua"`)

| Param                    | Type        | Default   | Description                                                       |
| ------------------------ | ----------- | --------- | ----------------------------------------------------------------- |
| `llmlingua_rate`         | float (0–1) | `0.75`    | Fraction of tokens to keep. Higher = gentler. `0.5` = aggressive. |
| `llmlingua_target_token` | int         | —         | Absolute token budget (overrides rate).                           |
| `llmlingua_roles`        | string\[]   | all roles | Restrict to given roles, e.g. `["user"]`.                         |

> The first `llmlingua` request loads the ML model (\~70s). Later requests are fast. `headroom` and `caveman` have no load cost.

### Reading the results

The response body is the normal provider response. Compression status is reported via headers:

| Header                         | Example | Meaning                        |
| ------------------------------ | ------- | ------------------------------ |
| `X-FastRouter-Compressed`      | `true`  | Compression was applied.       |
| `X-FastRouter-Tokens-Saved`    | `328`   | Tokens saved (before − after). |
| `X-FastRouter-Savings-Percent` | `26.23` | Percent saved.                 |

Headers are absent when compression did not apply (disabled, not opted-in, or failed open).

### Anthropic Messages API notes

The same block works on `/v1/messages`, with a few specifics:

* The top-level `system` prompt is compressed too — often the largest prose block.
* Non-text blocks (`images`, `tool_use`, `tool_result`) pass through untouched.
* Content with a `cache_control` marker is always skipped, so prompt caching is never invalidated.
* `caveman` and `llmlingua` are structure-preserving and recommended here.

```bash
curl -sS -i -X POST https://api.fastrouter.ai/v1/messages \
  -H 'x-api-key: <api-key>' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "max_tokens": 1024,
    "system": "You are a helpful assistant. <long system prose>",
    "messages": [{"role": "user", "content": "<long prose>"}],
    "optimize": {"compress": {"engine": "llmlingua", "llmlingua_rate": 0.75}}
  }'
```

### Examples

**Lossless (safe default)**

```json
{ "optimize": { "compress": { "engine": "headroom" } } }
```

**Gentle ML compression, user messages only**

```json
{
  "optimize": {
    "compress": {
      "engine": "llmlingua",
      "llmlingua_rate": 0.8,
      "llmlingua_roles": ["user"]
    }
  }
}
```

**Stack all engines**

```json
{ "optimize": { "compress": { "engine": "all", "llmlingua_rate": 0.75 } } }
```

**Audit only — measure savings, change nothing**

```json
{ "optimize": { "compress": { "engine": "all", "mode": "audit" } } }
```

**Full request (OpenAI format)**

```bash
curl -sS -i -X POST https://api.fastrouter.ai/v1/chat/completions \
  -H 'Authorization: Bearer <api-key>' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5.5",
    "messages": [{"role": "user", "content": "<long prose>"}],
    "optimize": {"compress": {"engine": "llmlingua", "llmlingua_rate": 0.75}}
  }'
```

Check the `X-FastRouter-Compression-*` response headers to confirm it ran.

### Fail-open behavior

Compression is silently skipped — originals kept, request proceeds normally — when:

* The request has no `optimize.compress` block.
* The compression service is unreachable, times out, or returns an error.
* The response can't be decoded or doesn't match the input message count.

A request can never be broken by compression.


# Prompt Caching

FastRouter supports prompt caching on all major providers that offer it, with automatic sticky routing to maximize cache hits.

### Overview

Prompt caching reduces the cost of repeated context — long system prompts, RAG\
chunks, documents — by charging a fraction of the normal input price on cache hits. Reduce inference costs by caching repeated prompt content across requests.

### Sticky Routing

When a request benefits from caching, FastRouter pins subsequent requests for that model and conversation to the same provider endpoint so the cache stays warm. A "conversation" is identified by hashing the first system message and first user message — so different conversations naturally spread across providers while each individual conversation stays consistent.

Sticky routing only kicks in when the provider's cache read price is lower than its regular input price. If that provider goes down, FastRouter falls back automatically. If you've set a manual `provider.order`, your ordering takes precedence and sticky routing is skipped.

### Zero-Config Providers

The following providers cache automatically. No changes to your requests needed.

<table><thead><tr><th width="249.11328125">Provider</th><th>Cache write</th><th>Cache read</th></tr></thead><tbody><tr><td>OpenAI</td><td>Free</td><td>0.25x – 0.50x input</td></tr><tr><td>DeepSeek</td><td>Same as input</td><td>~0.10x input</td></tr><tr><td>Google AI Studio</td><td>Free</td><td>0.10x input</td></tr><tr><td>Google Vertex AI</td><td>Free</td><td>0.10x input</td></tr><tr><td>Grok</td><td>Free</td><td>See provider pricing</td></tr><tr><td>Moonshot AI</td><td>Free</td><td>See provider pricing</td></tr><tr><td>Baseten</td><td>Free</td><td>See provider pricing</td></tr></tbody></table>

**OpenAI** requires a minimum of 1024 tokens.

**Google AI Studio and Vertex AI** both support implicit caching on Gemini 2.5 and newer models — no configuration needed. FastRouter keeps your prompt prefixes stable to maximize cache hits. The 0.10x cache-read rate (90% discount) applies to all Gemini 2.5+ models; legacy Gemini 2.0 Flash is discounted at 0.25x. Implicit caches are managed entirely by Google's serving infrastructure with no storage cost to you. TTL is typically 3–5 minutes. To maximize cache hits, keep large static content (system instructions, RAG context, few-shot examples) at the beginning of your prompt and push dynamic content to the end.

Minimum token thresholds before caching applies:

| Model                 | Min tokens |
| --------------------- | ---------- |
| Gemini 2.5 Pro        | 4,096      |
| Gemini 2.5 Flash      | 1,024      |
| Gemini 2.5 Flash-Lite | 1,024      |

***

### Anthropic Claude

Anthropic requires you to explicitly mark what should be cached using `cache_control`. FastRouter supports two approaches.

#### Option A — Top-level (recommended for chat)

Add `cache_control` once at the request root. FastRouter automatically places the cache breakpoint at the last cacheable block and advances it as the conversation grows.

```json
{
  "model": "anthropic/claude-sonnet-4.6",
  "cache_control": { "type": "ephemeral" },
  "messages": [...]
}
```

> Only works when routed to Anthropic directly.

#### Option B — Per-block (for precise control)

Place `cache_control` on individual content blocks. Useful when you have a large stable payload (a document, RAG chunks, a character card) and want to cache exactly that. Maximum 4 breakpoints per request.

```json
{
  "messages": [
    {
      "role": "system",
      "content": [
        { "type": "text", "text": "You are a research assistant." },
        {
          "type": "text",
          "text": "<large document>",
          "cache_control": { "type": "ephemeral" }
        }
      ]
    },
    { "role": "user", "content": "Summarize the findings." }
  ]
}
```

Per-block caching works across Anthropic and Vertex.

#### TTL

| TTL             | Syntax                                 | Write cost  | Read cost   |
| --------------- | -------------------------------------- | ----------- | ----------- |
| 5 min (default) | `{ "type": "ephemeral" }`              | 1.25x input | 0.10x input |
| 1 hour          | `{ "type": "ephemeral", "ttl": "1h" }` | 2x input    | 0.10x input |

Use the 1-hour TTL for long sessions where repeated 5-minute cache re-writes would cost more than the higher write price.

#### Model minimums

| Min tokens | Models                                   |
| ---------- | ---------------------------------------- |
| 4096       | Opus 4.5, 4.6, 4.7 · Haiku 4.5           |
| 2048       | Sonnet 4.6 · Haiku 3.5                   |
| 1024       | Sonnet 4, 4.5 · Opus 4, 4.1 · Sonnet 3.7 |

***

### Checking Cache Savings

Every API response includes a `prompt_tokens_details` object:

```json
"prompt_tokens_details": {
  "cached_tokens": 10318,
  "cache_write_tokens": 0
}
```

`cached_tokens` > 0 means you're hitting the cache.

You can also check per-request cache usage on the **Activity Logs** page flyout on the FastRouter dashboard.


# Guardrails

Add deterministic and LLM-based guardrails to protect your AI applications from unwanted behaviors and ensure compliance.

### **Introduction**

Guardrails in FastRouter allow you to validate and monitor requests and responses flowing through your LLM gateway. They help you:

* Prevent sensitive information (PII) from being exposed
* Ensure responses adhere to specific topics or formats
* Block toxic or inappropriate content
* Validate output structure (regex patterns)

**⚠️ Feature Note**

Guardrails do not support streaming (`stream=true`). Ensure that `stream=false` in your request payload when using guardrails.

### Guardrail Actions

Each guardrail can be configured with one of two actions that determine how the system behaves when a guardrail check fails:

| Action   | Default State | Behavior                                                                                                                                                                                                                                                                                                                                    | Use Case                                                                                                                                                             |
| -------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Validate | —             | <p><strong>On Request & Response:</strong></p><ul><li>If <strong>any</strong> guardrail check <strong>fails</strong>, the request is killed with status code <code>446</code></li><li>If <strong>all</strong> guardrail checks <strong>succeed</strong>, the request/response proceeds with status code <code>200</code></li></ul>          | Use when guardrails are critical and a failed check should block the request entirely. **Recommended:** Test on a subset of requests first to understand the impact. |
| Observe  | ✓ Default     | <p><strong>On Request & Response:</strong></p><ul><li>If <strong>any</strong> guardrail check <strong>fails</strong>, the request still proceeds but with status code <code>246</code></li><li>If <strong>all</strong> guardrail checks <strong>succeed</strong>, the request/response proceeds with status code <code>200</code></li></ul> | Use when you want to log guardrail results without affecting your application flow. Ideal for monitoring and gathering insights before enforcing strict validation.  |

⚠️ **Testing Recommendation**

We strongly recommend running guardrails in **Observe** mode first on a subset of your requests to understand their impact before switching to **Validate** mode.

### Status Codes

FastRouter uses specific status codes to communicate guardrail results:

| Status Code | Description                                                                                                                  |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `200`       | All guardrail checks passed. Request/response processed successfully.                                                        |
| `246`       | **Observe mode:** One or more guardrail checks failed, but the request/response was still processed. Check logs for details. |
| `446`       | **Validate mode:** One or more guardrail checks failed. Request was blocked and not processed.                               |

### Using Guardrails in Requests

After creating guardrails in the FastRouter dashboard, you'll receive a **Config ID** (e.g., `gr_a8f3k9m2`) for each guardrail. Use these IDs to apply guardrails to your requests:

#### Adding Guardrails to Requests

Include the `input_guardrails` and/or `output_guardrails` parameters in your request body with an array of guardrail config IDs:

```shellscript
{
  "model": "openai/gpt-3.5-turbo",
  "messages": [
    {
      "role": "user",
      "content": "What is the meaning of life?"
    }
  ],
  "input_guardrails": ["gr_e9k2v7h5"],
  "output_guardrails": ["gr_a8f3k9m2", "gr_b2n7p1q4"]
}
```

In this example:

* `input_guardrails` - Applied to the incoming user request before it's sent to the LLM
* `output_guardrails` - Applied to the LLM's response before returning it to your application

#### Multiple Guardrails

You can apply multiple guardrails at each stage. They will be evaluated in the order specified:

```shellscript
{
  "model": "openai/gpt-4",
  "messages": [
    {
      "role": "user",
      "content": "Tell me about product pricing"
    }
  ],
  "input_guardrails": [
    "gr_e9k2v7h5",  // Topic Adherence
    "gr_a8f3k9m2"   // PII Check
  ],
  "output_guardrails": [
    "gr_b2n7p1q4",  // Toxicity Detection
    "gr_c5x8r3w9"   // Competitor Mention
  ]
}
```

### Guardrail Types

#### Basic Guardrails

Deterministic guardrails that run quickly and don't require LLM evaluation:

* **RegEx Check** - Validate content against custom regex patterns

#### LLM Judge Guardrails

Intelligent guardrails that use LLM evaluation for complex checks:

* **PII Check** - Detect and optionally redact emails, phone numbers, SSN, credit cards
* **Topic Adherence** - Ensure conversations stay on allowed topics
* **Toxicity Detection** - Detect hate speech, harassment, violence, and inappropriate content

**💡 Performance Note**

Basic guardrails execute in milliseconds. LLM Judge guardrails require an additional LLM call and may add 500ms-2s latency depending on your Default Guardrails Key configuration.

### Creating Guardrails

To create a new guardrail:

1. Navigate to **Guardrails** in your FastRouter dashboard
2. Click the **Browse Templates** tab
3. Choose a template and click **Create**
4. Configure your guardrail settings:
   * **Name** - A descriptive name for your guardrail
   * **Action** - Choose `Observe` or `Validate`
   * **Stage** - Choose `Input` or `Output`
   * Template-specific settings (e.g., PII types, word limits, regex patterns)
5. Click **Create Guardrail**
6. Copy the generated **Config ID** to use in your requests

### Best Practices

* **Start with Observe mode** - Monitor guardrail behavior before enforcing validation
* **Test incrementally** - Apply guardrails to a small percentage of traffic first
* **Layer guardrails** - Use both Basic and LLM Judge guardrails for comprehensive protection
* **Monitor costs** - LLM Judge guardrails consume tokens from your Default Guardrails Key
* **Keep guardrails focused** - Create specific guardrails for specific use cases rather than one large guardrail
* **Review logs regularly** - Check status `246` responses to understand when guardrails are triggered

### Example: Complete Request Flow

```shellscript
// Request with input and output guardrails
POST https://api.fastrouter.ai/v1/chat/completions
Authorization: Bearer YOUR_API_KEY

{
  "model": "openai/gpt-4",
  "messages": [
    {
      "role": "user",
      "content": "What's your email for support?"
    }
  ],
  "input_guardrails": ["gr_e9k2v7h5"],
  "output_guardrails": ["gr_a8f3k9m2"]
}

// Response (200 - All checks passed)
{
  "id": "fr_...",
  "model": "gpt-4",
  "choices": [...],
  "guardrails": {
    "input": {
      "passed": true,
      "checks": [
        {
          "id": "gr_e9k2v7h5",
          "name": "Topic Adherence",
          "passed": true,
          "cost": 0.00011
        }
      ]
    },
    "output": {
      "passed": true,
      "checks": [
        {
          "id": "gr_a8f3k9m2",
          "name": "PII Check",
          "passed": true,
          "cost": 0.00012
        }
      ]
    }
  }
}

// Response (246 - Check failed in Observe mode)
// Request still processed, but guardrail logged failure
{
  "id": "fr_...",
  "model": "gpt-4",
  "choices": [...],
  "guardrails": {
    "output": {
      "passed": false,
      "checks": [
        {
          "id": "gr_a8f3k9m2",
          "name": "PII Check",
          "passed": false,
          "cost": 0.00013
        }
      ]
    }
  }
}

// Response (446 - Check failed in Validate mode)
// Request blocked
{
  "error": {
    "message": "Guardrail validation failed",
    "type": "guardrail_error",
    "code": 446,
    "guardrails": {
      "output": {
        "passed": false,
        "checks": [
          {
            "id": "gr_a8f3k9m2",
            "name": "PII Check",
            "passed": false,
            "cost": 0.00014
          }
        ]
      }
    }
  }
}
```

<br>


# Batch Processing

FastRouter supports batch processing for efficient handling of multiple API requests at scale.

## Overview

Batch processing allows you to upload a JSONL file containing a series of chat completion or embedding requests, which are processed asynchronously across supported providers (currently OpenAI, Anthropic and Gemini). More providers will be added soon.

Batch processing is ideal for high-volume tasks, such as generating embeddings for large datasets or running multiple chat completions in parallel. Requests are processed within 24 hours (often much quicker), and results are made available as a downloadable file. Each batch uses the specified FastRouter API key for billing.

**Key Benefits:**

* Asynchronous execution to avoid rate limits and enable bulk operations.
* Mix of models from supported providers within a single batch file.
* Unified endpoint per file (e.g., all chat completions or all embeddings).

{% embed url="<https://youtu.be/yaUadPnt89k>" %}

***

### **Supported Endpoints and Providers**

* **Endpoints:** `/v1/chat/completions` (for chat-completion requests) and `/v1/embeddings` (for generating vector representations).
* **Providers:** OpenAI, Anthropic and Gemini. All requests in a batch file must use the same endpoint, but you can mix models from these providers (e.g., combine OpenAI's GPT models with Anthropic's Claude models and Google's Gemini models in a chat completions batch).

For batch pricing details, please refer to the individual model pages for [OpenAI and Anthropic models](https://fastrouter.ai/models?creator=OpenAI%252CAnthropic\&order=newest).

***

### **File Format**

Batch files must be in JSONL format (one JSON object per line). Each line represents a single request and must include:

* **`custom_id`**: A unique string identifier for the request (e.g., "request-1"). This must be unique across the file.
* **`provider`**: The provider slug (e.g., "openai" or "anthropic").
* **`method`**: Always "POST".
* **`url`**: The endpoint, either "/v1/chat/completions" or "/v1/embeddings". All lines in the file must use the same URL.
* **`body`**: The request payload, including "model" and other parameters specific to the endpoint (e.g., "messages" for chat completions, "input" for embeddings).

**Important Notes:**

* Ensure all custom\_ids are unique to avoid processing errors.
* The file can mix models from supported providers, but the endpoint must be consistent.
* Maximum file size and request limits may apply; check your account dashboard for details.

#### **Example: Chat Completions Batch File**

This JSONL file mixes OpenAI and Anthropic models for chat completions:

```
{"custom_id": "request-1", "provider": "anthropic", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "anthropic/claude-4.5-sonnet", "messages": [{"role": "user", "content": "Hello world! what is 13 + 25"}],"max_tokens": 1000}}
{"custom_id": "request-2", "provider": "anthropic", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "anthropic/claude-4.5-sonnet", "messages": [{"role": "user", "content": "Hello world! what comes next in series 2 ,4, 6, 8, 10, "}],"max_tokens": 1000}}
{"custom_id": "request-3", "provider": "openai", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-4.1-nano", "messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Hello world! what is 13 + 25"}],"max_tokens": 1000}}
{"custom_id": "request-4", "provider": "openai", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-4.1", "messages": [{"role": "system", "content": "You are an unhelpful assistant."},{"role": "user", "content": "Hello world! what comes next in series 2 ,4, 6, 8, 10, "}],"max_tokens": 1000}}
```

#### **Example: Embeddings Batch File**

This JSONL file uses OpenAI models for embeddings:

```
{"custom_id": "request-1", "provider": "openai", "method": "POST", "url": "/v1/embeddings", "body": {"model": "openai/text-embedding-3-large","input": "Meditation cultivates a profound sense of calm and clarity by training the mind to focus on the present moment, helping to reduce stress and anxiety. Regular practice has been shown to improve concentration and mental resilience, making it easier to navigate daily challenges with patience and equanimity. Physiologically, meditation can lower blood pressure and regulate the body’s stress response, promoting better sleep quality and overall health. By fostering greater self-awareness, it encourages more mindful decision-making and emotional balance, strengthening interpersonal relationships and enhancing one’s capacity for compassion. Over time, these combined benefits contribute to a deeper sense of well-being and life satisfaction."}}
{"custom_id": "request-2", "provider": "openai", "method": "POST", "url": "/v1/embeddings", "body": {"model": "openai/text-embedding-3-small","input": "Hi HELLO THERE, EMBED THIS"}}
```

***

### **Creating a Batch Request**

To initiate a batch:

<figure><img src="/files/IIGCF8Ku4GZhTTRufJ1F" alt=""><figcaption></figcaption></figure>

1. **Prepare Your File:** Create a JSONL file following the format above. You can download sample templates from the FastRouter dashboard.
2. **Access the Dashboard:** Log in to your FastRouter account and navigate to the Batch Processing section.
3. **Upload and Configure:**
   * Click "Create Batch."
   * Upload your JSONL file based on the endpoint type.
   * Select an API key from your account (this key will be used for all requests in the batch).
4. **Submit the Batch:** Click "Create" to start processing. Batches are processed asynchronously, and you can monitor progress in the dashboard.

***

### **Monitoring and Retrieving Results**

<figure><img src="/files/9xYZlgSIBFqfAM6ipflt" alt=""><figcaption></figcaption></figure>

* **Batch Status:** View your batches in the dashboard under "Batch Jobs." Statuses include In Progress, Completed, or Failed. A progress bar shows completion percentage.
* **Download Results:** Once completed (typically within hours, up to 24 hours), download the output file. The results file is in JSONL format, with each line corresponding to a request by custom\_id, including the response data or any errors.
* **Error Handling:** Check the dashboard for more details of failed requests.

***

### **Pricing and Limits**

* **Cost:** Batch requests are billed based on the batch pricing for the underlying model (e.g., tokens for chat completions, inputs for embeddings).
* **Limits:** Batches can contain up to 50,000 requests. Processing time scales with batch size.
* **Best Practices:** Start with small batches to test. Ensure your API key has sufficient credits.


# Image Processing

A guide to sending images via Base64 or URLs in API requests.

### Overview

FastRouter lets you include images in chat completions using an OpenAI-compatible message format. This enables vision use cases like image captioning, visual Q\&A, object recognition, and multimodal chat across supported providers and models.

***

### Supported Image Types

* **Images:** URLs or base64-encoded data URLs
  * Common formats: **jpg, jpeg, png, webp** (and others depending on the model/provider)

***

### Supported Models

Use models that support **Image** as an input modality in the FastRouter model catalog:\
<https://fastrouter.ai/models?input_modalities=image>

Examples include multimodal/vision-capable models such as `x-ai/grok-4` (availability varies).

***

### Sending Image Inputs

Images are sent inside `messages[].content[]` as entries of type `"image_url"`, either pointing to a public URL or a base64-encoded data URL.

#### Example: Sending an Image via URL (Python)

```python
import requests

url = "https://api.fastrouter.ai/api/v1/chat/completions"
headers = {
  "Authorization": "Bearer API-KEY",
  "Content-Type": "application/json"
}

messages = [
  {
    "role": "user",
    "content": [
      {"type": "text", "text": "Describe this image and identify any landmarks."},
      {
        "type": "image_url",
        "image_url": {
          "url": "https://upload.wikimedia.org/wikipedia/commons/d/da/Taj-Mahal.jpg"
        }
      }
    ]
  }
]

payload = {
  "model": "openai/gpt-4.1-nano",
  "messages": messages
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())
```

#### Example: Sending a Base64-Encoded Image (Python)

Use base64 data URLs for local/private images.

```python
import requests
import base64

def encode_image_to_base64(image_path: str) -> str:
  with open(image_path, "rb") as f:
    return base64.b64encode(f.read()).decode("utf-8")

url = "https://api.fastrouter.ai/api/v1/chat/completions"
headers = {
  "Authorization": "Bearer API-KEY",
  "Content-Type": "application/json"
}

image_path = "/path/to/your/image.jpg"
base64_image = encode_image_to_base64(image_path)

# Match the MIME type to your file (e.g., image/png for .png)
data_url = f"data:image/jpeg;base64,{base64_image}"

messages = [
  {
    "role": "user",
    "content": [
      {"type": "text", "text": "What's in this image?"},
      {"type": "image_url", "image_url": {"url": data_url}}
    ]
  }
]

payload = {
  "model": "openai/gpt-4.1-nano",
  "messages": messages
}

response = requests.post(url, headers=headers, json=payload)
print(response.json())
```

***

### Tips & Best Practices

* **Put text first:** Send your prompt before the image(s) for best results.
* **Multiple images:** Add multiple `"image_url"` items within the same `content` array if your model supports it.
* **Prefer URLs for large images:** URLs avoid base64 overhead and request-size limits.
* **Use the correct MIME type:** For base64 data URLs, ensure `data:image/...` matches the actual file type.
* **Model limits vary:** Image count/size support differs by model/provider — check the model’s docs in the FastRouter catalog.


# PDF Processing

End-to-end guide to PDF ingestion in FastRouter chat completions, including plugin config, request examples, and billing details.

***

### Introduction

FastRouter supports PDF understanding in chat completions by attaching PDFs to messages and enabling the **file-parser** plugin with the **mistral-ocr** engine.

This allows you to ask questions about PDFs (including scanned/image PDFs) using your chosen chat model—FastRouter will OCR/parse the document and provide the extracted text to the model.

**Pricing:** PDF OCR via `mistral-ocr` is billed at **$2 / 1,000 pages**, in addition to normal model token usage.

**Note:** If you’re using a model with native document understanding (e.g., `openai/gpt-4.1`), you can leverage the model’s built-in document processing — no plugin may be needed.

***

### Endpoint

**POST** `https://api.fastrouter.ai/api/v1/chat/completions`

***

### How PDF processing works

When you include a PDF in `messages[].content[]` and enable the plugin:

* FastRouter fetches/decodes the PDF (URL or base64)
* Runs **mistral-ocr** to extract text (works well on scanned pages and embedded images)
* Feeds the extracted content into the target model so it can answer your prompt

***

### Plugin configuration (required for OCR)

Enable PDF processing with the `plugins` parameter:

```json
{
  "plugins": [
    {
      "id": "file-parser",
      "pdf": {
        "engine": "mistral-ocr"
      }
    }
  ]
}
```

***

### Sending PDFs in messages

Attach PDFs inside a message’s `content` array using `type: "file"`:

```json
{
  "type": "file",
  "file": {
    "filename": "document.pdf",
    "file_data": "https://example.com/document.pdf"
  }
}
```

#### Supported `file_data` formats

* **Public URL** (recommended): `https://.../file.pdf`
* **Base64 data URL** (for local/private docs): `data:application/pdf;base64,JVBERi0xLjc...`

***

### Example: PDF via public URL

{% tabs %}
{% tab title="cURL" %}

```bash
curl https://api.fastrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "z-ai/glm-4.7",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Summarize this PDF in 5 bullets and list any key dates." },
          {
            "type": "file",
            "file": {
              "filename": "document.pdf",
              "file_data": "https://example.com/document.pdf"
            }
          }
        ]
      }
    ],
    "plugins": [
      {
        "id": "file-parser",
        "pdf": {
          "engine": "mistral-ocr"
        }
      }
    ]
  }'
```

{% endtab %}

{% tab title="Python" %}

```python
import requests

url = "https://api.fastrouter.ai/api/v1/chat/completions"
headers = {
    "Authorization": f"Bearer {FASTROUTER_API_KEY}",
    "Content-Type": "application/json",
}

payload = {
    "model": "anthropic/claude-4.5-sonnet",
    "messages": [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What does this document say about termination clauses? Quote the relevant sections."},
                {
                    "type": "file",
                    "file": {
                        "filename": "contract.pdf",
                        "file_data": "https://example.com/contract.pdf",
                    },
                },
            ],
        }
    ],
    "plugins": [
        {
            "id": "file-parser",
            "pdf": {"engine": "mistral-ocr"},
        }
    ],
}

resp = requests.post(url, headers=headers, json=payload)
print(resp.json())
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
const response = await fetch("https://api.fastrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.FASTROUTER_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "z-ai/glm-4.7",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "Extract the invoice total, invoice date, and vendor name." },
          {
            type: "file",
            file: {
              filename: "invoice.pdf",
              file_data: "https://example.com/invoice.pdf",
            },
          },
        ],
      },
    ],
    plugins: [
      {
        id: "file-parser",
        pdf: { engine: "mistral-ocr" },
      },
    ],
  }),
});

const data = await response.json();
console.log(data);
```

{% endtab %}
{% endtabs %}

***

### Example: PDF via base64 (data URL)

Use base64 when the PDF is local or not publicly reachable.

{% tabs %}
{% tab title="cURL" %}

```bash
curl https://api.fastrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "z-ai/glm-4.7",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Summarize this PDF in 5 bullets and list any key dates." },
          {
            "type": "file",
            "file": {
              "filename": "document.pdf",
              "file_data": "data:application/pdf;base64,..."
            }
          }
        ]
      }
    ],
    "plugins": [
      {
        "id": "file-parser",
        "pdf": {
          "engine": "mistral-ocr"
        }
      }
    ]
  }'
```

{% endtab %}

{% tab title="Python" %}

```python
import base64
import requests
from pathlib import Path

def pdf_to_data_url(path: str) -> str:
    b = Path(path).read_bytes()
    return "data:application/pdf;base64," + base64.b64encode(b).decode("utf-8")

data_url = pdf_to_data_url("path/to/document.pdf")

payload = {
    "model": "anthropic/claude-haiku-4.5",
    "messages": [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Give me a structured outline of this document with headings."},
                {
                    "type": "file",
                    "file": {
                        "filename": "document.pdf",
                        "file_data": data_url,
                    },
                },
            ],
        }
    ],
    "plugins": [
        {
            "id": "file-parser",
            "pdf": {"engine": "mistral-ocr"},
        }
    ],
}

resp = requests.post(
    "https://api.fastrouter.ai/api/v1/chat/completions",
    headers={"Authorization": f"Bearer {FASTROUTER_API_KEY}", "Content-Type": "application/json"},
    json=payload,
)
print(resp.json())
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
import fs from "node:fs/promises";

const buf = await fs.readFile("path/to/document.pdf");
const dataUrl = `data:application/pdf;base64,${buf.toString("base64")}`;

const res = await fetch("https://api.fastrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.FASTROUTER_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "z-ai/glm-4.7",
    messages: [
      {
        role: "user",
        content: [
          { type: "text", text: "List the risks mentioned and the mitigation steps." },
          {
            type: "file",
            file: { filename: "risk-report.pdf", file_data: dataUrl },
          },
        ],
      },
    ],
    plugins: [
      {
        id: "file-parser",
        pdf: { engine: "mistral-ocr" },
      },
    ],
  }),
});

console.log(await res.json());
```

{% endtab %}
{% endtabs %}

***

### Pricing

* **mistral-ocr:** **$2 / 1,000 pages** processed
* **Model tokens:** billed normally for the model you select
* Prefer **URLs** for large PDFs to avoid request size limits and base64 overhead.

***


# Dynamic Tags Per Request

Leverage dynamic tags for reporting, billing, or feature tracking with custom tags sent per request.

The `request_tags` parameter lets you add custom tags to each request, enabling dynamic tracking at the individual request level. It supports both flat and hierarchical formats, making it easy to organize usage data for reporting, cost allocation, team billing, feature usage, or environment segmentation. These tags are available in the dashboard as a filter and also in the meta data of the activity logs.

### Parameter Type:

```json
request_tags: [string]
```

### What You Can Do with Tags:

* Group and filter requests by project, team, environment, or feature
* Organize usage reports using nested tag paths (e.g., `team/feature/test`)
* Track granular usage without needing to manage multiple API keys

### Tag Formats Supported

<table><thead><tr><th width="124.91796875">Format Type</th><th width="301.203125">Example Tags</th><th>Description</th></tr></thead><tbody><tr><td><strong>Flat Tag</strong></td><td><code>["asia", "production"]</code></td><td>Simple, one-level tags</td></tr><tr><td><strong>Hierarchical</strong></td><td><code>["USA/NYC", "org/team/project"]</code></td><td>Path-style, nested classification</td></tr><tr><td><strong>Mixed</strong></td><td><code>[ "Asia", "UAE/Dubai" ]</code></td><td>Combine both types in the same request</td></tr></tbody></table>

### Example Request

```bash
curl --location 'https://api.fastrouter.ai/api/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer API-KEY' \
--data '{
    "model": "openai/gpt-4o-mini",
    "stream": false,
    "messages": [
        {
            "role": "user",
            "content": "What are some famous tourist attractions in London?"
        }
    ],
    "temperature": 0.0,
    "request_tags": ["UK/London", "Europe"]
}'
```

### Why Use Tags?

* **Team-level visibility**: e.g., `marketing/email-gen`
* **Feature usage breakdown**: e.g., `feature/summary-widget`
* **Cost attribution**: e.g., `customer/acme`, `env/staging`
* **Geo-based tracking**: e.g., `US/California/SF` , `India/Mumbai`


# Credits

### Credits **Overview**

* Credits can be added by Owners.
* FastRouter usage continues until credits run out.
* Transparent cost accounting and alerts help manage budget effectively.


# Provisioning Keys

Provisioning Keys are special-purpose administrative tokens used to securely create, update, list, and delete Service Account Keys within your organization.

### Provisioning Keys **Overview**

* **Not for LLM Requests**: Provisioning Keys cannot be used to call model endpoints or route completions.
* **Org-Level Admin Only**: Only **Organization Owners** can generate and manage Provisioning Keys.
* **Service Key Management**: These keys provide full lifecycle control over Service Account Keys — including scoped permissions, rate limits, expiration, and metadata.

### Use Cases

Provisioning Keys are ideal for:

* Automating API key management
* Managing non-user-bound access (e.g., for CI/CD pipelines)
* Setting granular controls on model usage, budgets, and project tagging

### API Endpoints

#### Create a Service Account Key

```bash
curl 'https://api.fastrouter.ai/prod/createServiceKey' \
  -H 'accept: application/json' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'authorization: Bearer PROVISIONING-KEY' \
  --data-raw '{
    "api_key_name": "SERVICE-KEY-NAME",
    "credit_limit": null,
    "reset_budget_interval": null,
    "expire_key": null,
    "models": null,
    "tpm_limit": null,
    "rpm_limit": null,
    "meta_data": null,
    "tags": null
  }'

```

The response includes the secret Service Account Key and its one-way hash, which you'll need for future updates or deletion.

**Note:** The following fields in the `createServiceKey` payload are **optional**:

* `credit_limit`
* `reset_budget_interval`
* `expire_key`
* `models`
* `tpm_limit`
* `rpm_limit`
* `meta_data`
* `tags`

These can be omitted entirely or set to `null` depending on your provisioning use case. Only `org_id` , `project_id` , `project_name` and `api_key_name` are required.

#### Update a Service Account Key

```bash
curl 'https://api.fastrouter.ai/prod/updateServiceKey' \
  -H 'accept: application/json' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'authorization: Bearer PROVISIONING-KEY' \
  --data-raw '{
    "api_key_id_hash": "SERVICE-KEY-HASH",
    "api_key_name": "SERVICE-KEY-NAME",
    "credit_limit": 150,
    "reset_budget_interval": "daily",
    "expire_key": "2025-03-28 18:30:00.000",
    "models": [
      "GROQ/deepseek-r1-distill-llama-70b",
      "GROQ/llama-3.1-8b-instant"
    ],
    "tpm_limit": 10000,
    "rpm_limit": 10,
    "metadata": {
      "version": "v1",
      "source": "system",
      "data": {
        "user_id": "user_id",
        "name": "raw_key",
        "email": "hashed_key",
        "api_key_alias": "alias1"
      }
    },
    "tags": ["tag1", "tag2"]
}'
```

#### List Service Account Keys

```bash
curl 'https://api.fastrouter.ai/prod/getServiceKeys' \
  -H 'accept: application/json' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'authorization: Bearer PROVISIONING-KEY'
```

#### Delete a Service Account Key

```bash
curl 'https://api.fastrouter.ai/prod/deleteServiceKey' \
  -H 'accept: application/json' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'authorization: Bearer PROVISIONING-KEY' \
  --data-raw '{
    "api_key_id_hash": "SERVICE-KEY-HASH"
}'
```

### Notes

* Ensure your `PROVISIONING-KEY` is stored securely. It has high privileges.
* For `metadata`, you can store custom info like `user_id`, `team_name`, or `deployment`.
* `tags` help in grouping and filtering keys by project or environment.


# Structured Outputs

FastRouter supports structured JSON outputs, allowing you to enforce a specific schema in LLM responses. This feature ensures that responses are machine-readable and conform to a predefined structure.

### **Overview**

Using the `response_format` parameter, you can define a custom JSON schema that the model must follow. FastRouter will guide compatible models to format their outputs accordingly.

For a live, browsable list of models that support Structured Outputs, see the [**Models directory**](https://fastrouter.ai/models?supported_parameters=structured_outputs\&order=newest).

### **Supported Format**

* **Type**: `json_schema`
* **Schema**: Defined using standard [JSON Schema](https://json-schema.org/) format
* **Strict Mode**: If `strict: true`, compatible models will respond with a JSON object that strictly follows your schema.

### **Example: Enforcing Weather Data Schema**

This example instructs the model to return weather data in a strict JSON format:

```bash
curl --location 'https://api.fastrouter.ai/api/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer API-KEY' \
--data '{
    "model": "openai/gpt-4o",
    "messages": [
      {
        "role": "system",
        "content": "You are a helpful assistant that responds in JSON format."
      },
      {
        "role": "user",
        "content": "What is the weather like in London?"
      }
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "weather",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "City or location name"
            },
            "temperature": {
              "type": "number",
              "description": "Temperature in Celsius"
            },
            "conditions": {
              "type": "string",
              "description": "Weather conditions description"
            }
          },
          "required": ["location", "temperature", "conditions"],
          "additionalProperties": false
        }
      }
    }
}' 
```

### **Response Example**

```json
{
  "location": "London",
  "temperature": 17.5,
  "conditions": "Partly cloudy"
}
```

### **Schema Validation Options**

| Parameter              | Description                                                                     |
| ---------------------- | ------------------------------------------------------------------------------- |
| `type`                 | Must be set to `"json_schema"`                                                  |
| `name`                 | Optional name for the schema block (useful for logging/debugging)               |
| `strict`               | Informs the provider to validate the output against the schema (`true`/`false`) |
| `schema`               | A valid [JSON Schema](https://json-schema.org/) definition                      |
| `additionalProperties` | Set to `false` to block unexpected fields in the response                       |

### **Best Practices**

* Always define a `system` prompt that clearly instructs the model to respond in JSON format.
* Use `strict: true` for use cases that demand guaranteed structure.
* Combine with `stream: false` for easier schema validation (as streaming responses may break structure).
* Validate responses client-side to catch schema mismatches early.


# Function Calling

FastRouter supports Function Calling for models capable of planning and invoking tools or functions. This allows LLMs to return structured function calls instead of natural language responses.

### Overview

Function calling empowers LLMs to:

* **Identify** when a tool or function is needed based on user input.
* **Select** the appropriate function from a provided set of tools.
* **Generate** structured JSON arguments to invoke that function.

When you provide a list of tools in your API request, compatible models can choose to respond with one or more function calls. You then execute those functions in your application code and feed the results back to the model in a subsequent request. This creates a multi-turn conversation loop for tasks like data retrieval, API integrations, or complex workflows.

FastRouter.ai routes your requests to the best available providers (e.g., Google AI Studio, OpenAI) while supporting OpenAI-compatible formats for tools. This includes parallel function calling for models that support it.

**Key Benefits:**

* Build agents that interact with real-world APIs (e.g., weather services, calendars).
* Handle complex queries by breaking them into tool-based steps.
* Improve reliability with structured outputs over free-form text.

### Supported Models

FastRouter.ai supports function calling on models that natively offer this capability. Here's a partial list (check the [Models page ](https://fastrouter.ai/models)for the latest):

* **Google Models**: Gemini 2.5 Pro, Gemini 2.5 Flash and others
* **OpenAI Models**: GPT-5, GPT-4.1, GPT-4o, o4-mini, o3-mini and others
* **Anthropic Models**: Claude Opus 4.1, Claude Sonnet 4.5, Claude Haiku 4.5 and others
* **xAI Models**: Grok 4, Grok 3 and others

Use the `provider` field in your request to route to a specific backend if needed.

### Usage

To use function calling:

1. Define your tools in the `tools` array of your `/chat/completions` request. Each tool follows the OpenAI-compatible schema (a JSON object with `name`, `description`, and `parameters`).
2. Send the request to FastRouter.ai's endpoint: `https://api.fastrouter.ai/api/v1/chat/completions`.
3. If the model responds with a `tool_calls` array in the response, execute the functions in your code.
4. Append the tool results as a new message (with `role: "tool"`) and send a follow-up request to let the model generate a final response.

**Request Parameters:**

* `tools`: Array of tool definitions.
* `tool_choice`: Optional; controls how the model uses tools (e.g., `"auto"` for automatic selection, `"none"` to disable, or specify a tool name).

**Response Format:**

* If a tool is called, the response will include `choices[0].message.tool_calls`—an array of objects with `function.name` and `function.arguments` (JSON string).
* Execute the function and respond with a message like: `{"role": "tool", "content": "JSON result", "tool_call_id": "call_id_from_response"}`.

For authentication, use your API key in the `Authorization: Bearer YOUR_API_KEY` header.

### Executing Tools: Multi-Turn Example

After receiving a `tool_calls` response, execute the function in your code and send the result back. Here's how to handle a full loop.

#### Python (Using `requests`)

```python
import requests
import json

API_KEY = "YOUR_API_KEY"
ENDPOINT = "https://api.fastrouter.ai/api/v1/chat/completions"

# Step 1: Initial request with tools
payload = {
    "model": "openai/gpt-4o",
    "messages": [{"role": "user", "content": "What is the temperature in Paris?"}],
    "tools": [
        # Same tools as above...
    ]
}

response = requests.post(ENDPOINT, json=payload, headers={
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}).json()

# Step 2: Check for tool calls and execute
tool_calls = response["choices"][0]["message"].get("tool_calls")
if tool_calls:
    for tool_call in tool_calls:
        func_name = tool_call["function"]["name"]
        args = json.loads(tool_call["function"]["arguments"])
        
        # Simulate tool execution (replace with real API call)
        if func_name == "get_current_weather":
            result = {"temperature": 72, "condition": "Sunny"}  # Mock weather data
        # Add handling for other tools...
        
        # Step 3: Append tool result to messages
        payload["messages"].append(response["choices"][0]["message"])  # Add model's message
        payload["messages"].append({
            "role": "tool",
            "content": json.dumps(result),
            "tool_call_id": tool_call["id"]
        })
    
    # Step 4: Send follow-up request for final response
    final_response = requests.post(ENDPOINT, json=payload, headers={
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    }).json()
    
    print(final_response["choices"][0]["message"]["content"])  # e.g., "The temperature in Paris is 72°F and sunny."
```

#### Node.js (Using `fetch`)

<pre class="language-javascript"><code class="lang-javascript">const API_KEY = 'YOUR_API_KEY';
const ENDPOINT = 'https://api.fastrouter.ai/api/v1/chat/completions';

async function main() {
  // Step 1: Initial request
  const payload = {
    model: 'openai/gpt-4o',
    messages: [{ role: 'user', content: 'What is the temperature in Paris?' }],
<strong>    tools: [ /* Same tools as above... */ ]
</strong>  };

  let response = await fetch(ENDPOINT, {
    method: 'POST',
    headers: { 'Authorization': `Bearer ${API_KEY}`, 'Content-Type': 'application/json' },
    body: JSON.stringify(payload)
  }).then(res => res.json());

  // Step 2: Handle tool calls
  const toolCalls = response.choices[0].message.tool_calls;
  if (toolCalls) {
    for (const toolCall of toolCalls) {
      const funcName = toolCall.function.name;
      const args = JSON.parse(toolCall.function.arguments);

      // Simulate execution
      let result;
      if (funcName === 'get_current_weather') {
        result = { temperature: 72, condition: 'Sunny' };  // Mock
      }
      // Add other tool handlers...

      // Step 3: Append to messages
      payload.messages.push(response.choices[0].message);
      payload.messages.push({
        role: 'tool',
        content: JSON.stringify(result),
        tool_call_id: toolCall.id
      });
    }

    // Step 4: Follow-up request
    response = await fetch(ENDPOINT, {
      method: 'POST',
      headers: { 'Authorization': `Bearer ${API_KEY}`, 'Content-Type': 'application/json' },
      body: JSON.stringify(payload)
    }).then(res => res.json());

    console.log(response.choices[0].message.content);  // Final response
  }
}

main();
</code></pre>

### Important: Preserving Reasoning Details in Multi-Turn Tool Calls

When using models that support reasoning (extended thinking) like Google's Gemini-3-Pro , the response may include `reasoning_details` in the assistant message. **You must pass these reasoning details back in subsequent requests** to maintain context continuity and allow the model to continue reasoning from where it left off.

#### Why This Matters

* Models with reasoning capabilities generate internal thinking blocks that inform their decisions
* Omitting `reasoning_details` in follow-up requests can lead to errors
* Preserving this context ensures the model maintains its chain of thought across tool execution

#### Example: Handling Reasoning Details with Tool Calls

```python
from openai import OpenAI

client = OpenAI(
 base_url="https://api.fastrouter.ai/api/v1",
 api_key="API-KEY",
)

# Define flight search tool
tools = [{
 "type": "function",
 "function": {
 "name": "search_flights",
 "description": "Search for available flights between two cities",
 "parameters": {
 "type": "object",
 "properties": {
 "origin": {"type": "string", "description": "Departure city"},
 "destination": {"type": "string", "description": "Arrival city"},
 "date": {"type": "string", "description": "Travel date (YYYY-MM-DD)"}
 },
 "required": ["origin", "destination", "date"]
 }
 }
}]

# First API call with reasoning enabled
response = client.chat.completions.create(
 model="google/gemini-3-pro-preview",
 messages=[
 {"role": "user", "content": "Find flights from New York to London on December 20th and recommend the best one."}
 ],
 tools=tools,
 extra_body={"reasoning": {"max_tokens": 2000}}
)

message = response.choices[0].message

print("First response:", message.content)
print("Tool calls:", message.tool_calls)

if message.tool_calls:
 # Simulate flight search results
 flight_results = {
 "flights": [
 {"airline": "British Airways", "departure": "08:00", "arrival": "20:00", "price": 450, "stops": 0},
 {"airline": "Delta", "departure": "14:30", "arrival": "02:30+1", "price": 380, "stops": 0},
 {"airline": "United", "departure": "19:00", "arrival": "07:00+1", "price": 320, "stops": 1}
 ]
 }
 
 # Build follow-up messages, preserving reasoning_details
 messages = [
 {"role": "user", "content": "Find flights from New York to London on December 20th and recommend the best one."},
 {
 "role": "assistant",
 "content": message.content,
 "tool_calls": message.tool_calls,
 "reasoning_details": getattr(message, 'reasoning_details', None)  # Preserve if exists
 },
 {
 "role": "tool",
 "tool_call_id": message.tool_calls[0].id,
 "content": str(flight_results)
 }
 ]
 
 # Second API call - model continues with preserved reasoning
 response2 = client.chat.completions.create(
 model="google/gemini-3-pro-preview",
 messages=messages,
 tools=tools
 )
 
 print("\
Recommendation:", response2.choices[0].message.content)
else:
 print("No tool calls were made.")

```

### Best Practices

* **Error Handling:** If tool execution fails, return an error message in the `content` field.
* **Security:** Validate arguments before executing tools to prevent injection attacks.
* **Streaming:** Function calling works with `stream: true`, but tool calls appear in the final chunk.
* **Costs:** Tool calls count toward token usage—monitor via the response's `usage` field.
* **Testing:** Start with simple tools and iterate based on model behavior.
* **Reasoning Details:** When using models with reasoning capabilities, preserve `reasoning_details` from the assistant message and include it in subsequent requests to maintain context continuity.


# Reasoning Tokens

FastRouter can return Reasoning Tokens (also known as thinking tokens) for supported models.

### **Overview**

FastRouter can return Reasoning Tokens (also known as *thinking tokens*) for supported models. These tokens represent the model's internal reasoning process and can significantly improve output quality for complex tasks such as planning, math, tool use, and multi-step analysis.

* Reasoning tokens are **enabled by default** for supported models.
* The model decides whether to generate reasoning tokens unless explicitly controlled.
* When returned, reasoning tokens appear in the **`reasoning` field** of each message.
* You can **limit**, **control**, or **exclude** reasoning tokens using the `reasoning` parameter.

***

### **Supported Models**

Reasoning tokens are currently supported by:

* **Gemini thinking models** (includes Gemini 2.5 and Gemini 3 series)
* **Anthropic models** (via `reasoning.max_tokens`)
* **OpenAI o-series models**
* **Grok models**

***

### **How Reasoning Tokens Appear in Responses**

When enabled, reasoning tokens appear as structured blocks in the response:

```json
{
 "type": "reasoning",
 "reasoning": {
 "text": "The model is considering multiple constraints before responding..."
 }
}
```

If excluded, the model still reasons internally—but the reasoning is **not returned**.

***

### **Controlling Reasoning Tokens**

You can control reasoning behavior using the `reasoning` object in your request.

**General Structure**

```json
{
 "model": "your-model",
 "messages": [],
 "reasoning": {
 "effort": "high",
 "max_tokens": 2000,
 "exclude": false,
 "enabled": true
 }
}
```

> ⚠️ Use **either** `effort` **or** `max_tokens` — not both.

***

### **Reasoning Effort Levels**

**Supported By**

* **OpenAI o-series**
* **Grok models**
* **Google Gemini 3 models** (mapped to `thinkingLevel`)

**Effort Options**

| Effort     | Token Allocation        |
| ---------- | ----------------------- |
| `max`      | \~90% of `max_tokens`   |
| `xhigh`    | \~85% of `max_tokens`   |
| `high`     | \~80% of `max_tokens`   |
| `trending` | \~80% of `max_tokens`   |
| `medium`   | \~50% of `max_tokens`   |
| `low`      | \~20% of `max_tokens`   |
| `minimal`  | \~10% of `max_tokens`   |
| `none`     | 0% — reasoning disabled |

> **Note:** `trending` currently maps to the same allocation as `high`. It is provided as a separate, forward-compatible label and may be tuned independently in the future — don't assume it will always equal `high`.

Example:

```json
"reasoning": {
 "effort": "high"
}
```

***

### **Reasoning Max Tokens**

**Supported By**

* **Gemini 2.5 thinking models** (via `thinkingBudget`)
* **Anthropic models**

Example:

```json
"reasoning": {
 "max_tokens": 2000
}
```

***

### **Google Gemini Reasoning Behavior**

Google Gemini models support reasoning tokens, but the API used depends on the model generation.

**Gemini 2.5 Models — `thinkingBudget` API**

Gemini 2.5 thinking models use Google's `thinkingBudget` API. With FastRouter, you control this using `reasoning.max_tokens`, which is passed through as the thinking budget.

Example:

```json
"reasoning": {
 "max_tokens": 2000
}
```

**Gemini 3 Models — `thinkingLevel` API**

Gemini 3 models (such as `google/gemini-3.1-pro-preview` and `google/gemini-3-flash-preview`) use Google's newer `thinkingLevel` API instead of the older `thinkingBudget` API used by Gemini 2.5 models.

FastRouter maps the `reasoning.effort` parameter to Google's `thinkingLevel` values as follows:

| FastRouter reasoning.effort | Google thinkingLevel  |
| --------------------------- | --------------------- |
| `max`                       | `high`                |
| `xhigh`                     | `high`                |
| `high`                      | `high`                |
| `trending`                  | `high`                |
| `medium`                    | `medium`              |
| `low`                       | `low`                 |
| `minimal`                   | `minimal`             |
| `none`                      | *(thinking disabled)* |

> Google's `thinkingLevel` API only exposes four levels (`minimal`, `low`, `medium`, `high`). FastRouter's finer-grained effort levels above `high` (`xhigh`, `max`, `trending`) all map to Google's `high` on Gemini 3 models — the additional granularity only takes effect on models that consume a numeric token budget (Anthropic, Gemini 2.5).

Example:

```json
"reasoning": {
 "effort": "high"
}
```

**Token Consumption is Determined by Google**

When using `thinkingLevel`, the actual number of reasoning tokens consumed is determined internally by Google. There are no publicly documented token limit breakpoints for each level. For example, setting `effort: "low"` might result in several hundred reasoning tokens depending on the complexity of the task. This is expected behavior and reflects how Google implements thinking levels internally.

***

### **Anthropic-Specific Reasoning Behavior**

When using **Anthropic models**:

**Rules**

* `reasoning.max_tokens`
* Used directly
* Minimum: **1024 tokens**
* `reasoning.effort`
* Converted into a reasoning token budget
* Reasoning tokens are:
* **Minimum:** 1024 tokens (except `none`, see below)
* **Maximum:** 24,576 tokens

**Budget Formula**

```
budget_tokens = max(
 min(max_tokens × effort_ratio, 24576),
 1024
)
```

Where:

| Effort     | effort\_ratio |
| ---------- | ------------- |
| `max`      | 0.9           |
| `xhigh`    | 0.85          |
| `high`     | 0.8           |
| `trending` | 0.8           |
| `medium`   | 0.5           |
| `low`      | 0.2           |
| `minimal`  | 0.1           |
| `none`     | 0             |

> **`none` is a special case:** an `effort_ratio` of `0` would otherwise still floor to the 1024-token minimum under the formula above. Instead, `effort: "none"` disables reasoning entirely (equivalent to omitting `reasoning` / setting `enabled: false`) — no reasoning budget is allocated and no reasoning tokens are generated or billed.

**Important Constraint**

> **`max_tokens` must be strictly greater than the reasoning budget**, otherwise the model will not have enough tokens to produce a final answer. This applies to all effort levels except `none`.

**Adaptive Thinking (Newer Anthropic Models)**

Newer Anthropic models support **adaptive thinking**, where the model dynamically decides how much of its reasoning budget to actually use for a given request, rather than always consuming the full `budget_tokens` allocated.

* FastRouter still computes and sends `budget_tokens` (via the formula above) to the provider as an upper bound.
* The model adaptively spends anywhere from a small fraction of that budget up to the full amount, depending on task complexity — a simple prompt at `effort: "max"` may consume far fewer reasoning tokens than the 90%-of-`max_tokens` ceiling suggests.
* Billing reflects **actual reasoning tokens generated**, not the requested budget — the budget is a ceiling, not a guarantee of spend.
* This means observed reasoning-token usage on adaptive models will typically be lower, and more variable, than the static budget formula alone would imply.

***

### **Excluding Reasoning Tokens**

You can instruct the model to reason internally **without returning reasoning tokens**.

```json
"reasoning": {
 "exclude": true
}
```

* The model still performs reasoning
* Reasoning tokens are **not included** in the response
* Works across **all models**
* Note: this is distinct from `effort: "none"` — `exclude` still allocates and bills a reasoning budget (subject to adaptive thinking behavior on supported models), it just withholds the reasoning text from the response. `effort: "none"` skips reasoning altogether.

***

### **Token Usage & Billing**

* Reasoning tokens are counted as **output tokens**
* They are billed the same way as regular output tokens
* Enabling reasoning increases token usage but often improves: Accuracy, Coherence, Tool-calling correctness
* On adaptive-thinking Anthropic models, billed reasoning tokens reflect actual usage, which may be well below the computed budget ceiling


# Response Caching

Response Caching allows FastRouter.ai users to cache LLM responses for repeated or similar prompts.

### Overview

Response Caching delivers delivers **faster response times**, **lower costs**, and **consistent outputs** across applications.

Caching is especially effective for:

* Dashboards
* Chatbots and agents
* FAQs and support flows
* APIs with predictable or repetitive queries

FastRouter supports **exact-match** and **semantic-match** caching with flexible controls.

***

### Key Benefits

| Benefit                  | Description                                               |
| ------------------------ | --------------------------------------------------------- |
| Faster Responses         | Cache hits return in <10ms                                |
| Cost Reduction           | Cache hits billed at **0.1× token pricing** (90% savings) |
| Consistent Outputs       | Identical or similar inputs return consistent responses   |
| Reduced Provider Load    | Fewer upstream API calls, improved rate-limit headroom    |
| Conversation Flexibility | Multiple caching strategies for multi-turn chats          |
| Custom Cache Keys        | User-defined namespaces for precise cache control         |

***

### Feature Specification

#### Request Schema

Caching is enabled by including a `cache_key` header and an optional `cache` configuration object in the request body.

***

#### Headers

| Header        | Type   | Required          | Description                                                   |
| ------------- | ------ | ----------------- | ------------------------------------------------------------- |
| Authorization | string | Yes               | Bearer token with API key                                     |
| Content-Type  | string | Yes               | `application/json`                                            |
| cache\_key    | string | Yes (for caching) | User-defined cache namespace. If omitted, caching is disabled |

***

#### Request Body

```json
{
  "model": "openai/gpt-4.1-mini",
  "messages": [
    { "role": "user", "content": "Tell me about physics" }
  ],
  "max_tokens": 182,
  "stream": false,
  "cache": {
    "filter_on_provider": false,
    "filter_on_model": true,
    "expiration_time": 3600,
    "conversation_mode": "full_conversation",
    "last_n_turns": 2,
    "similarity_threshold": 0.75
  }
}
```

***

#### Sample Request

```bash
curl --location 'https://api.fastrouter.ai/v1/chat/completions' \
  --header 'Authorization: Bearer API-KEY' \
  --header 'cache_key: CACHE-KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "openai/gpt-4.1-mini",
    "messages": [
      { "role": "user", "content": "Tell me about physics" }
    ],
    "max_tokens": 182,
    "cache": {
      "filter_on_model": true,
      "expiration_time": 3600
    }
  }'
```

***

### Cache Key Header

The `cache_key` header defines the **primary cache namespace**.

**Purpose**

* Groups related requests under a shared cache scope

**Examples**

* `myapp-faq`
* `user_123_session`
* `product-descriptions`
* `chatbot-v2`

FastRouter combines `cache_key` with hashed request attributes to form the final lookup key.

***

### Cache Object Parameters

| Parameter             | Type    | Default             | Required    | Description                                                                 |
| --------------------- | ------- | ------------------- | ----------- | --------------------------------------------------------------------------- |
| expiration\_time      | integer | 3600                | No          | Cache TTL in seconds (60–86400)                                             |
| filter\_on\_model     | boolean | true                | No          | Match cache on model name                                                   |
| filter\_on\_provider  | boolean | false               | No          | Match cache on provider                                                     |
| conversation\_mode    | string  | `full_conversation` | No          | How conversation context is matched                                         |
| last\_n\_turns        | integer | 2                   | Conditional | Used only when `conversation_mode = last_n_turns`                           |
| similarity\_threshold | number  | 0.75                | No          | Minimum semantic similarity score (0–1) required to reuse a cached response |

***

#### 🔍 `similarity_threshold` Explained

* Enables **semantic caching** in addition to exact matches
* A value of:
  * `1.0` → exact match only
  * `0.75` (default) → allows minor rewording or paraphrases
  * `<0.7` → more aggressive reuse (use with caution)

If no cached entry meets the threshold, the request is treated as a **cache miss**.

***

### Conversation Modes

| Mode                | Description                     | Use Case                 |
| ------------------- | ------------------------------- | ------------------------ |
| full\_conversation  | Entire message history included | Stateful conversations   |
| last\_message\_only | Only last user message          | FAQs, stateless bots     |
| last\_n\_turns      | Last N user–assistant pairs     | Context-aware assistants |

**Turn Definition:**\
One turn = one user message + one assistant response.

***

### Cache Lookup

#### Cache Lookup Components

The final cache lookup is computed based on:

```
  org_id,
  model?,            // if filter_on_model = true
  provider?,         // if filter_on_provider = true
  prompt_messages,
  temperature,
  top_p,
  max_tokens
```

> `similarity_threshold` is applied **after lookup** to determine semantic eligibility.

***

#### Prompt Messages For Lookup

| Conversation Mode   | Messages Included             |
| ------------------- | ----------------------------- |
| full\_conversation  | All messages                  |
| last\_message\_only | Last user message             |
| last\_n\_turns      | Last N turns + system message |

***

#### Parameter Sensitivity

Always included in response caching:

| Parameter   | Notes                                   |
| ----------- | --------------------------------------- |
| temperature | Different values → different cache keys |
| top\_p      | Different values → different cache keys |
| max\_tokens | Different values → different cache keys |

Ignored for cache hashing:

* `stream`
* `user`
* `n`
* `frequency_penalty`
* `presence_penalty`
* `stop`

***

### API Responses

#### Cache MISS

Returned normally and stored in cache.

```json
{
  "cached": false,
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 182,
    "total_tokens": 193,
    "cost": 0.0002956
  }
}
```

***

#### Cache HIT

Returned instantly with cache metadata.

```json
{
  "cached": true,
  "similarity": 0.92,
  "usage": {
    "prompt_tokens": 11,
    "completion_tokens": 182,
    "total_tokens": 193,
    "cost": 0.00002956
  }
}
```

***

#### Cache Response Fields

| Field      | Type    | Description                                 |
| ---------- | ------- | ------------------------------------------- |
| cached     | boolean | True when served from cache                 |
| similarity | number  | Semantic similarity score (1 = exact match) |
| usage.cost | number  | Cache hits billed at 0.1×                   |

***

### Pricing

#### Cache Pricing

| Scenario      | Pricing                   |
| ------------- | ------------------------- |
| Cache HIT     | 0.1× standard token price |
| Cache MISS    | Standard token price      |
| Cache Storage | Free                      |

***

#### Pricing Formula

```
cache_hit_cost =
(prompt_tokens × input_price × 0.1) +
(completion_tokens × output_price × 0.1)
```

**Savings:** \~90%

***

### Streaming Support

#### Cached Streaming Responses

On cache hit + `stream: true`:

* Cached response is chunked and streamed
* Minimal artificial delay (default: 0ms)

***

#### Streaming Behavior

| Scenario              | Behavior                          |
| --------------------- | --------------------------------- |
| Cache MISS + stream   | Streamed from provider and cached |
| Cache HIT + stream    | Cached response streamed          |
| Cache HIT + no stream | Returned instantly                |


# Realtime API (WebSocket)

Enable real-time speech-to-speech and text conversations with AI models via WebSocket.

## Overview

The FastRouter Realtime API enables low-latency, multimodal conversations over a persistent WebSocket connection. Unlike the standard chat completions API, where each request is a discrete HTTP round trip, the Realtime API maintains a stateful session — letting you stream audio and text to the model as it happens and receive responses incrementally, token by token and audio chunk by audio chunk.

This makes it the right choice for building voice agents, live transcription-and-response experiences, interactive customer support bots, and any application where conversational latency matters. The API supports speech-to-speech interactions natively, so you can send raw audio and get spoken responses back without stitching together separate STT, LLM, and TTS pipelines.

Because FastRouter exposes the Realtime API through a single gateway endpoint, you get unified billing, observability, and key management across realtime models — using the same API key as the rest of your FastRouter workloads. Sessions are event-driven: you send client events (like `session.update` or `input_audio_buffer.append`) and listen for server events (like `response.audio.delta`) over one connection.

**Key capabilities:**

* **Speech-to-speech** — stream microphone audio in, receive natural spoken audio out
* **Text and audio modalities** — mix and match input and output formats per response
* **Function calling** — let the model invoke your tools mid-conversation
* **Voice activity detection** — automatic turn detection, or disable it for push-to-talk
* **Streaming everything** — text deltas, audio chunks, and transcripts arrive in real time

### Endpoint

```
wss://go.fastrouter.ai/v1/realtime
```

### Connection

Connect to the WebSocket endpoint with your API key and model as query parameters:

```javascript
const url = new URL("wss://go.fastrouter.ai/v1/realtime");
url.searchParams.set("model", "openai/gpt-realtime-2.1");
url.searchParams.set("api_key", "sk-v1-...");

const ws = new WebSocket(url.toString());
```

#### Available Models

| Model                      | Description                              |
| -------------------------- | ---------------------------------------- |
| `openai/gpt-realtime-2.1`  | Latest GPT Realtime model (recommended)  |
| `openai/gpt-realtime-1.5`  | Previous-generation GPT Realtime model   |
| `openai/gpt-realtime-mini` | Smaller, lower-cost realtime model       |
| `openai/gpt-realtime`      | Alias for the current GPT Realtime model |

### Session Configuration

After the connection opens, configure the session by sending a `session.update` event:

```javascript
ws.onopen = () => {
  const sessionConfig = {
    type: "session.update",
    session: {
      modalities: ["text", "audio"],
      voice: "alloy",
      input_audio_format: "pcm16",
      output_audio_format: "pcm16",
      instructions: "You are a helpful assistant.",
      temperature: 0.8
    }
  };
  ws.send(JSON.stringify(sessionConfig));
};
```

#### Session Options

| Property              | Type       | Description                                                          |
| --------------------- | ---------- | -------------------------------------------------------------------- |
| `modalities`          | `string[]` | Output modalities: `["text"]`, `["audio"]`, or `["text", "audio"]`   |
| `voice`               | `string`   | Voice for audio: `alloy`, `echo`, `fable`, `onyx`, `nova`, `shimmer` |
| `input_audio_format`  | `string`   | Input audio format: `pcm16`                                          |
| `output_audio_format` | `string`   | Output audio format: `pcm16`                                         |
| `instructions`        | `string`   | System instructions for the model                                    |
| `temperature`         | `number`   | Sampling temperature (0.6–1.2 recommended)                           |

### Sending Text Messages

Send text messages using `conversation.item.create` followed by `response.create`:

```javascript
// Create the conversation item
ws.send(JSON.stringify({
  type: "conversation.item.create",
  item: {
    type: "message",
    role: "user",
    content: [{
      type: "input_text",
      text: "Hello, how are you?"
    }]
  }
}));

// Request a response
ws.send(JSON.stringify({
  type: "response.create",
  response: { modalities: ["text", "audio"] }
}));
```

### Sending Audio (Streaming)

Stream audio input using `input_audio_buffer.append`, then commit and request a response:

```javascript
// Stream audio chunks (Base64-encoded PCM16)
ws.send(JSON.stringify({
  type: "input_audio_buffer.append",
  audio: base64AudioChunk
}));

// When finished recording, commit and request response
ws.send(JSON.stringify({ type: "input_audio_buffer.commit" }));
ws.send(JSON.stringify({
  type: "response.create",
  response: { modalities: ["text", "audio"] }
}));
```

#### Audio Format Requirements

* **Format:** 16-bit PCM (little-endian)
* **Sample Rate:** 24,000 Hz recommended
* **Channels:** Mono
* **Encoding:** Base64

#### Converting Audio to Base64 PCM16

```javascript
// Convert Float32 audio samples to PCM16 bytes
function float32ToPcm16Bytes(float32Array) {
  const bytes = new Uint8Array(float32Array.length * 2);
  const view = new DataView(bytes.buffer);
  for (let i = 0; i < float32Array.length; i++) {
    let s = Math.max(-1, Math.min(1, float32Array[i]));
    const val = s < 0 ? s * 0x8000 : s * 0x7fff;
    view.setInt16(i * 2, val, true);
  }
  return bytes;
}

// Convert bytes to Base64
function uint8ArrayToBase64(bytes) {
  let binary = "";
  const chunkSize = 0x8000;
  for (let i = 0; i < bytes.length; i += chunkSize) {
    binary += String.fromCharCode.apply(null, bytes.subarray(i, i + chunkSize));
  }
  return btoa(binary);
}
```

### Receiving Responses

Handle incoming events with an `onmessage` handler:

```javascript
ws.onmessage = (event) => {
  const data = JSON.parse(event.data);

  switch (data.type) {
    case "session.created":
      console.log("Session created");
      break;

    case "response.text.delta":
      // Streaming text chunk
      console.log("Text:", data.delta);
      break;

    case "response.audio.delta":
      // Base64-encoded PCM16 audio chunk
      playAudioChunk(data.delta);
      break;

    case "response.audio_transcript.delta":
      // Transcript of audio response
      console.log("Transcript:", data.delta);
      break;

    case "response.done":
      console.log("Response complete", data.response?.usage);
      break;

    case "error":
      console.error("Error:", data.error?.message);
      break;
  }
};
```

#### Playing Audio Output

```javascript
let audioContext = null;
let nextPlayTime = 0;

function playAudioChunk(base64Audio) {
  if (!audioContext) {
    audioContext = new AudioContext({ sampleRate: 24000 });
    nextPlayTime = audioContext.currentTime;
  }

  // Decode Base64 to PCM16 bytes
  const bytes = base64ToUint8Array(base64Audio);
  const samples = pcm16BytesToFloat32(bytes);

  // Create and play audio buffer
  const buffer = audioContext.createBuffer(1, samples.length, 24000);
  buffer.getChannelData(0).set(samples);

  const source = audioContext.createBufferSource();
  source.buffer = buffer;
  source.connect(audioContext.destination);

  const startTime = Math.max(nextPlayTime, audioContext.currentTime + 0.02);
  source.start(startTime);
  nextPlayTime = startTime + buffer.duration;
}

function base64ToUint8Array(base64) {
  const binary = atob(base64);
  const bytes = new Uint8Array(binary.length);
  for (let i = 0; i < binary.length; i++) {
    bytes[i] = binary.charCodeAt(i);
  }
  return bytes;
}

function pcm16BytesToFloat32(bytes) {
  const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
  const samples = new Float32Array(bytes.byteLength / 2);
  for (let i = 0; i < samples.length; i++) {
    samples[i] = view.getInt16(i * 2, true) / 0x8000;
  }
  return samples;
}
```

### Event Reference

#### Client Events (Send)

| Event                       | Description                         |
| --------------------------- | ----------------------------------- |
| `session.update`            | Configure session settings          |
| `conversation.item.create`  | Add a message to the conversation   |
| `input_audio_buffer.append` | Stream audio input chunks           |
| `input_audio_buffer.commit` | Commit buffered audio as user input |
| `input_audio_buffer.clear`  | Clear the audio input buffer        |
| `response.create`           | Request a model response            |
| `response.cancel`           | Cancel an in-progress response      |

#### Server Events (Receive)

| Event                             | Description                             |
| --------------------------------- | --------------------------------------- |
| `session.created`                 | Session initialized                     |
| `session.updated`                 | Session configuration applied           |
| `response.created`                | Response generation started             |
| `response.text.delta`             | Streaming text chunk                    |
| `response.text.done`              | Text response complete                  |
| `response.audio.delta`            | Streaming audio chunk (Base64 PCM16)    |
| `response.audio.done`             | Audio response complete                 |
| `response.audio_transcript.delta` | Streaming transcript of audio           |
| `response.done`                   | Full response complete (includes usage) |
| `error`                           | Error occurred                          |

### Voice Activity Detection (VAD)

By default, VAD is enabled and the API automatically detects when users start and stop speaking. For push-to-talk interfaces, disable VAD:

```javascript
ws.send(JSON.stringify({
  type: "session.update",
  session: {
    turn_detection: null  // Disable VAD
  }
}));
```

> **Note:** With VAD disabled, you must manually call `input_audio_buffer.commit` to finalize audio input, `response.create` to trigger a response, and `input_audio_buffer.clear` before starting new input.

### Function Calling

Define functions the model can call:

```javascript
ws.send(JSON.stringify({
  type: "session.update",
  session: {
    tools: [{
      type: "function",
      name: "get_weather",
      description: "Get current weather for a location",
      parameters: {
        type: "object",
        properties: {
          location: {
            type: "string",
            description: "City name"
          }
        },
        required: ["location"]
      }
    }],
    tool_choice: "auto"
  }
}));
```

When the model calls a function, handle it and provide results:

```javascript
// In your message handler, check for function calls
if (data.type === "response.done") {
  const output = data.response?.output?.[0];
  if (output?.type === "function_call") {
    const args = JSON.parse(output.arguments);
    const result = await yourFunction(args);

    // Send function result back
    ws.send(JSON.stringify({
      type: "conversation.item.create",
      item: {
        type: "function_call_output",
        call_id: output.call_id,
        output: JSON.stringify(result)
      }
    }));

    // Request continuation
    ws.send(JSON.stringify({ type: "response.create" }));
  }
}
```

### Complete Example

```javascript
const API_KEY = "sk-v1-...";
const MODEL = "openai/gpt-realtime-2.1";

// Connect
const url = new URL("wss://go.fastrouter.ai/v1/realtime");
url.searchParams.set("model", MODEL);
url.searchParams.set("api_key", API_KEY);

const ws = new WebSocket(url.toString());

ws.onopen = () => {
  // Configure session
  ws.send(JSON.stringify({
    type: "session.update",
    session: {
      modalities: ["text", "audio"],
      voice: "alloy",
      input_audio_format: "pcm16",
      output_audio_format: "pcm16",
      instructions: "You are a helpful assistant. Be concise.",
      temperature: 0.8
    }
  }));
};

ws.onmessage = (event) => {
  const data = JSON.parse(event.data);

  switch (data.type) {
    case "session.created":
    case "session.updated":
      console.log("Ready");
      break;
    case "response.text.delta":
      process.stdout.write(data.delta);
      break;
    case "response.audio.delta":
      // Handle audio playback
      break;
    case "response.done":
      console.log("\n[Done]");
      break;
    case "error":
      console.error("Error:", data.error?.message);
      break;
  }
};

ws.onerror = (error) => {
  console.error("WebSocket error:", error);
};

ws.onclose = () => {
  console.log("Disconnected");
};

// Send a text message
function sendMessage(text) {
  ws.send(JSON.stringify({
    type: "conversation.item.create",
    item: {
      type: "message",
      role: "user",
      content: [{ type: "input_text", text }]
    }
  }));
  ws.send(JSON.stringify({
    type: "response.create",
    response: { modalities: ["text"] }
  }));
}
```


# Custom Alerts

Alerts help you monitor your API usage and performance in real-time. Set up custom thresholds to get notified when metrics exceed expected values or change compared to historical baselines.

### Overview

Alerts evaluate your selected metrics at regular intervals and notify you when conditions are met. Each alert can have two severity levels:

* **Warning** — Early indicator that a metric is trending toward a problem
* **Critical** — Immediate attention required; metric has exceeded acceptable limits

Alerts are scoped to specific Projects, API Keys, and Models, giving you granular control over what you monitor.

<figure><img src="/files/B5GtKVn5dKoMtMjfgUM0" alt=""><figcaption><p>Select metric, name and scope</p></figcaption></figure>

<figure><img src="/files/D4FNNyxjhsBwKqU51QLA" alt=""><figcaption><p>Set threshold and subscribe to alert</p></figcaption></figure>

***

### Available Metrics

#### Performance

| Metric                  | Description                                                                               | Unit  |
| ----------------------- | ----------------------------------------------------------------------------------------- | ----- |
| **Response Time**       | Median (p50) end-to-end latency from request receipt to response completion               | ms    |
| **Time to First Token** | Median (p50) time from request submission until the first response token begins streaming | ms    |
| **Throughput**          | Median request throughput for streaming responses (`stream = true`)                       | req/s |

#### Reliability

| Metric          | Description                                                  | Unit   |
| --------------- | ------------------------------------------------------------ | ------ |
| **Error Count** | Total number of failed requests within the evaluation window | errors |
| **Error Rate**  | Percentage of requests that failed                           | %      |

#### Usage & Cost

| Metric                | Description                                                      | Unit     |
| --------------------- | ---------------------------------------------------------------- | -------- |
| **Token Consumption** | Total tokens consumed (input + output)                           | tokens   |
| **Daily Spend**       | Cumulative spending for the current day (resets at midnight UTC) | $        |
| **Monthly Spend**     | Cumulative spending for the current month (resets on the 1st)    | $        |
| **Total Requests**    | Total number of API requests                                     | requests |

***

### Scoping Alerts

Each alert can be scoped to monitor specific subsets of your traffic:

#### Projects

Select which projects to include in the alert evaluation. You can choose:

* **All Projects** — Monitor aggregate metrics across your entire organization
* **Specific Projects** — Monitor one or more selected projects

#### API Keys

Select which API keys to include:

* **All Keys** — Monitor all API keys within the selected projects
* **Specific Keys** — Monitor one or more selected API keys

#### Models

Select which models to include:

* **All Models** — Monitor requests to any model
* **Specific Models** — Monitor requests to selected models only (e.g., only GPT-4.1 and Claude 4.5 Sonnet)

> **Example**: You could create an alert that monitors Response Time only for your "Production" project, only for requests using a particular API key, and only for GPT-4.1 requests.

***

### Alert Types

#### Static Value

Triggers when the metric crosses a fixed threshold.

**Use cases:**

* Response Time > 2000ms
* Error Rate > 5%
* Daily Spend > $500

**Configuration:**

| Field              | Description                                     |
| ------------------ | ----------------------------------------------- |
| Condition          | `Above` or `Below`                              |
| Warning Threshold  | Value that triggers a warning (optional)        |
| Critical Threshold | Value that triggers a critical alert (required) |

#### Percentage Change

Triggers when the metric changes by a specified percentage compared to a historical baseline.

**Use cases:**

* Response Time increased 50% vs same time yesterday
* Request volume dropped 30% vs same time last week
* Error Count spiked 100% vs previous hour

**Configuration:**

| Field                  | Description                                      |
| ---------------------- | ------------------------------------------------ |
| Comparison Period      | Historical baseline to compare against           |
| Condition              | `Above` or `Below`                               |
| Warning Threshold (%)  | Percentage change that triggers a warning        |
| Critical Threshold (%) | Percentage change that triggers a critical alert |

***

### Evaluation Interval

The evaluation interval determines how often the alert checks your metrics. Choose based on how quickly you need to detect issues:

| Interval   | Best For                         | Trade-off                                            |
| ---------- | -------------------------------- | ---------------------------------------------------- |
| **5 min**  | Error Rate, Error Count          | Fastest detection, may be noisy for volatile metrics |
| **15 min** | Response Time, TTFT, Throughput  | Balances speed and noise reduction                   |
| **30 min** | Response Time, Token Consumption | Smooths transient spikes                             |
| **1 hour** | Spend metrics, Usage patterns    | Good for slow-changing metrics                       |
| **Daily**  | Daily Spend, Monthly Spend       | End-of-day summaries                                 |

#### Recommendations by Metric

| Metric                   | Recommended Interval |
| ------------------------ | -------------------- |
| Error Rate / Error Count | 5 min                |
| Response Time / TTFT     | 15–30 min            |
| Throughput               | 15 min               |
| Token Consumption        | 30–60 min            |
| Daily/Monthly Spend      | 1 hour or Daily      |

***

### Comparison Periods

When using **Percentage Change** alerts, you compare the current value against a historical baseline. The comparison period determines which historical window to use.

| Comparison Period         | What It Compares                   | Best For                                           |
| ------------------------- | ---------------------------------- | -------------------------------------------------- |
| **Previous Period**       | The immediately preceding interval | Detecting sudden spikes                            |
| **Same time 1 hour ago**  | Same interval, 1 hour earlier      | Intra-day patterns                                 |
| **Same time 1 day ago**   | Same interval, 24 hours earlier    | Daily patterns (e.g., business hours vs off-hours) |
| **Same time 1 week ago**  | Same interval, 7 days earlier      | Weekly patterns (e.g., weekday vs weekend)         |
| **Same time 1 month ago** | Same interval, 30 days earlier     | Monthly patterns, seasonal trends                  |

#### How Comparison Works

The system compares two time windows of equal length:

```
Current Window:     [T - interval, T]
Previous Window:    [T - interval - offset, T - offset]
```

**Example:** Alert runs at 3:00 PM with a 15-minute interval, comparing to "Same time 1 day ago"

```
Current Window:     2:45 PM – 3:00 PM today
Previous Window:    2:45 PM – 3:00 PM yesterday
```

#### Percentage Change Formula

```
% Change = ((Current Value - Previous Value) / Previous Value) × 100
```

**Example:**

* Current Response Time: 450ms
* Previous Response Time: 300ms
* % Change: ((450 - 300) / 300) × 100 = **50%**

If your alert threshold is "Above 40%", this would trigger.

***

### Thresholds

Each alert supports two threshold levels:

#### Warning Threshold

An early indicator that the metric is trending toward a problem. Useful for:

* Getting advance notice before issues become critical
* Allowing time to investigate before escalation
* Tracking trends that may need attention

#### Critical Threshold

The primary alert trigger indicating immediate attention is needed. This is the main threshold that should reflect your SLA or operational limits.

#### Condition Direction

| Condition | Meaning                                 | Typical Use                      |
| --------- | --------------------------------------- | -------------------------------- |
| **Above** | Alert when metric exceeds threshold     | Response Time, Error Rate, Spend |
| **Below** | Alert when metric falls below threshold | Throughput, Request Volume       |

> **Tip:** Set your Warning threshold at \~50-70% of your Critical threshold to give yourself response time.

***

### Notification Behavior

#### Notification Channels

* **Email** — Send alerts to specified email addresses
* **Organization Owners** — Automatically notify all org owners
* **Project Members** — Automatically notify all members of affected projects

#### Alert States

Alerts transition between three states:

```
        threshold breached
    ┌───────────────────────────┐
    │                           ▼
 ┌──┴──┐                   ┌─────────┐
 │ OK  │                   │ FIRING  │
 └──┬──┘                   └────┬────┘
    ▲                           │
    └───────────────────────────┘
        below threshold (reset)
```

***

### Examples

#### Example 1: High Response Time Alert

**Goal:** Get notified when API response times are slow

| Setting             | Value                                    |
| ------------------- | ---------------------------------------- |
| Metric              | Response Time                            |
| Scope               | Production project, All keys, All models |
| Alert Type          | Static Value                             |
| Condition           | Above                                    |
| Warning Threshold   | 1500 ms                                  |
| Critical Threshold  | 3000 ms                                  |
| Evaluation Interval | 15 min                                   |

**Behavior:** Every 15 minutes, calculates the median response time. If it exceeds 1500ms, a warning is sent. If it exceeds 3000ms, a critical alert is sent.

***

#### Example 2: Error Rate Spike Detection

**Goal:** Detect sudden increases in error rate compared to normal

| Setting             | Value                              |
| ------------------- | ---------------------------------- |
| Metric              | Error Rate                         |
| Scope               | All projects, All keys, All models |
| Alert Type          | Percentage Change                  |
| Comparison Period   | Same time 1 day ago                |
| Condition           | Above                              |
| Warning Threshold   | 50%                                |
| Critical Threshold  | 100%                               |
| Evaluation Interval | 5 min                              |

**Behavior:** Every 5 minutes, compares the current error rate to the same 5-minute window yesterday. If today's error rate is 50% higher, a warning is sent. If it's 100% higher (doubled), a critical alert is sent.

***

#### Example 3: Daily Spend Limit

**Goal:** Get notified before exceeding daily budget

| Setting             | Value                                    |
| ------------------- | ---------------------------------------- |
| Metric              | Daily Spend                              |
| Scope               | Production project, All keys, All models |
| Alert Type          | Static Value                             |
| Condition           | Above                                    |
| Warning Threshold   | $400                                     |
| Critical Threshold  | $500                                     |
| Evaluation Interval | 1 hour                                   |

**Behavior:** Every hour, checks the cumulative daily spend. Sends a warning at $400 and a critical alert at $500.

***

#### Example 4: Traffic Drop Detection

**Goal:** Detect if request volume suddenly drops (may indicate an outage)

| Setting             | Value                                    |
| ------------------- | ---------------------------------------- |
| Metric              | Total Requests                           |
| Scope               | Production project, All keys, All models |
| Alert Type          | Percentage Change                        |
| Comparison Period   | Same time 1 hour ago                     |
| Condition           | Below                                    |
| Warning Threshold   | 30%                                      |
| Critical Threshold  | 50%                                      |
| Evaluation Interval | 5 min                                    |

**Behavior:** Every 5 minutes, compares request count to the same period 1 hour ago. If traffic drops by 30%, a warning is sent. If it drops by 50%, a critical alert is sent.

***

### Best Practices

1. **Start with Critical thresholds only** — Add Warning thresholds once you understand your baseline metrics.
2. **Use appropriate intervals** — Don't use 5-minute intervals for metrics that naturally fluctuate; you'll get too many false positives.
3. **Leverage Percentage Change for anomalies** — Static thresholds work well for known limits, but percentage change is better for detecting unusual patterns.
4. **Scope alerts appropriately** — Create separate alerts for Production vs Staging environments rather than one alert for everything.
5. **Set up resolved notifications** — Knowing when an issue is resolved is as important as knowing when it started.
6. **Document your thresholds** — Keep a record of why you chose specific threshold values so future team members understand the rationale.

***

### FAQ

**Q: What happens if there's no data in the evaluation window?**

A: The alert maintains its current state. If there were zero requests in the window, metrics like Error Rate cannot be calculated, so the alert remains unchanged.

**Q: Can I have multiple alerts for the same metric?**

A: Yes. You might have one alert for Production and another for Staging, or different thresholds for different models.

**Q: When do cumulative metrics (Daily Spend, Monthly Spend) reset?**

A: Daily Spend resets at midnight UTC. Monthly Spend resets on the 1st of each month at midnight UTC.

**Q: What timezone are alerts evaluated in?**

A: All alert evaluations use UTC. "Same time 1 day ago" means the same UTC time yesterday.

**Q: Can I pause an alert without deleting it?**

A: Yes. You can pause an alert from the Alerts list page. Paused alerts retain their configuration but do not evaluate or send notifications.


# System Alerts

System alerts are automatic, pre-configured notifications that fire when your organization's credit balance or project/key budgets cross critical thresholds

### Overview

Unlike custom alerts, system alerts require no setup — they are active by default for all organizations and fire when your organization's credit balance or project/key budgets cross critical thresholds. System alert notifications are sent to all connected channels org-wide.&#x20;

To manage which channels receive them, see Notification Channels.

***

### Alert Types

#### Organization Credit Balance

Fires when your organization's prepaid credit balance drops below a fixed dollar amount.

**Recipients:** All organization admins\
**Reset condition:** Balance exceeds threshold + $2.00 buffer\
**Check frequency:** Every 5 minutes

***

#### Project Budget Threshold

Fires when a project's remaining budget drops below a percentage of its configured budget cap.

| Threshold       | Severity |
| --------------- | -------- |
| < 20% remaining | Warning  |
| < 10% remaining | Warning  |
| < 5% remaining  | Critical |

**Recipients:** All project admins\
**Reset condition:** Remaining budget exceeds threshold + 2% absolute buffer\
**Check frequency:** Every 5 minutes

<figure><img src="/files/mpfaBlwOV0Jv5G6LKoqE" alt=""><figcaption><p>Configure Thresholds</p></figcaption></figure>

***

#### API Key Budget Threshold

Fires when an API key's remaining budget drops below a percentage of its configured budget cap.

| Threshold       | Severity |
| --------------- | -------- |
| < 20% remaining | Warning  |
| < 10% remaining | Warning  |
| < 5% remaining  | Critical |

**Recipients:** Key creator and all project admins\
**Reset condition:** Remaining budget exceeds threshold + 2% absolute buffer\
**Check frequency:** Every 5 minutes

***

### Notification Behavior

#### How notifications are sent

* One notification is sent per threshold crossing — notifications do not repeat until the alert resets and the threshold is crossed again.
* If multiple thresholds are crossed in a single balance drop, separate notifications are sent for each threshold crossed.
* A 15-minute grace period applies before an alert resets, preventing rapid-fire notifications when a balance fluctuates around a threshold.

### Configuring System Alerts

System alert thresholds can be toggled on or off per threshold level. You cannot add custom threshold values in addition to the defaults — the predefined thresholds above are fixed.

To configure system alerts:

1. Go to **Alerts** in the left navigation.
2. In the **System Alert Configuration** banner, click **Configure**.
3. Toggle each alert type on or off using the toggle switch.
4. Select which threshold pills are active for each alert type.
5. Click **Save configuration**.

> Turning off an alert type stops all notifications for that type until it is re-enabled. Individual threshold pills can be deselected to suppress notifications at that specific level while keeping others active.

***

### Notification Channels

System alerts are delivered to all active notification channels configured at the org level. Channels are shared across all system alert types — you cannot route different alert types to different channels.

To manage channels:

1. Go to **Alerts** in the left navigation.
2. In the **System Alert Configuration** banner, click **Manage Channels**.

<figure><img src="/files/3yHi0wKeTSRV6Uen5Pq3" alt=""><figcaption><p>Manage Channels</p></figcaption></figure>

#### Available channels

**Email**

Always active. Sent to all organization admins automatically. Cannot be disabled.

**Slack**

Connect your Slack workspace via OAuth. Alerts are routed to the channel you specify during connection.

To connect: **Manage Channels → Slack → Connect**

**Generic Webhook**

POST JSON payloads to any HTTPS endpoint. Supports an optional signing secret for HMAC-SHA256 payload verification.

To configure: **Manage Channels → Generic Webhook → Configure**

Payload format

```json
{
  "alert_type": "balance",
  "org_id": "...",
  "org_name": "...",
  "current_balance": 8.50,
  "threshold_value": 10.0,
  "event_type": "triggered",
  "alert_config_id": "...",
  "time": "2026-07-09 12:00:00",
  "text": "Balance $8.50 crossed below threshold $10.00"
}
```

Use **Test Payload** to send a sample delivery to your endpoint before saving.

**Flock**

Connect your Flock workspace to route alerts to a Flock channel.

To connect: **Manage Channels → Flock → Configure**

***

### Viewing System Alert History

System alert fire events appear in the **Recent Alerts** tab on the Alerts page, alongside user-configured alert events. System alert rows are marked with a **System** badge.

<figure><img src="/files/4nfrrvBx7cadCUihfiEB" alt=""><figcaption><p>Recent System Alerts</p></figcaption></figure>

Each row shows:

* **Alert Name** — e.g. "Balance $48.38 crossed below threshold…"
* **Reason Summary** — the balance or budget value that triggered the alert, and the direction of the crossing
* **Status** — Warning or Critical
* **Triggered at** — timestamp of the alert fire event
* **Action** — View button to open the full alert detail

Clicking **View** opens the alert detail, showing:

* Current snapshot (org/project/key, current value, threshold, total triggers)
* Channels the notification was delivered to at the time of firing
* Full trigger history with per-event severity and channel delivery context

***

### FAQ

**Q: Can I add my own threshold values for system alerts?**

A: Not in the current version. System alert thresholds are predefined ($10 / $2.50 / $1 for org balance; 20% / 10% / 5% for project and key budgets). You can toggle individual thresholds on or off, but cannot add custom values.

**Q: Who receives system alerts?**

A: Org credit balance alerts go to all organization admins. Project budget alerts go to all project admins. API key budget alerts go to the key creator and all project admins. In all cases, notifications also go to any connected org-level channels (Slack, Webhook, Flock).

**Q: Can I route different system alert types to different Slack channels?**

A: Not currently. All system alert types share the same set of org-level notification channels. Per-type channel routing is not available in this version.

**Q: Why did I receive the same alert twice?**

A: This can happen if the balance dropped below a threshold, recovered above it (plus the $2 buffer), and then dropped again. Once an alert resets, it can re-trigger if the balance crosses the threshold again.

**Q: What happens if my Slack connection drops at the time an alert fires?**

A: FastRouter falls back to email delivery and records the channel delivery context in the trigger history (e.g., "Email only — Slack not connected at time").

**Q: Do system alerts count against any quota or limit?**

A: No. System alerts are infrastructure-level notifications and do not consume any credits or count against usage limits.

**Q: Can I disable system alerts entirely?**

A: You can toggle each alert type off individually in System Alert Configuration. There is no single master off-switch, as system alerts exist to protect your service continuity.


# Tracing

Group related LLM API calls into a single trace using a simple traceparent header.

### Overview

FastRouter supports the **W3C Trace Context** standard via the `traceparent` header, enabling you to group multiple LLM API calls into a single trace with ordered spans.

This helps you understand the full lifecycle of complex workflows—across models, providers, and steps—in one place.

**Common use cases:**

* **Agentic workflows** — multi-step chains with tool/function calls
* **Chat sessions** — linking all turns in a conversation
* **Parallel requests** — grouping concurrent calls

Tracing works with any HTTP client. No SDK or proprietary tooling is required.

***

### How Tracing Works

When FastRouter receives a request with a `traceparent` header:

* Extracts `trace_id` to group related requests
* Records `parent_id` as the caller (application) span
* Applies overrides from optional headers (if present)
* Generates a new `span_id` for the gateway span
* Captures latency, tokens, cost, and full request/response
* Stores the span and groups it under the corresponding trace

All API calls sharing the same `trace_id` appear as a **single trace with multiple spans**.

> **Key rule:** Reuse the same `traceparent` across all requests in a workflow.

***

### traceparent Header Format

```
traceparent: {version}-{trace_id}-{parent_id}-{flags}
```

| Field      | Format       | Description                                                           |
| ---------- | ------------ | --------------------------------------------------------------------- |
| version    | `00`         | Fixed (W3C standard)                                                  |
| trace\_id  | 32 hex chars | Unique ID for the entire trace                                        |
| parent\_id | 16 hex chars | Caller span ID (your application)                                     |
| flags      | `01`         | Sampling flag. Note: Currently, all requests are shown on FastRouter. |

**Example:**

```
traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01
```

***

### Optional Headers

FastRouter supports additional headers to improve trace readability and control:

| Header        | Description                                          | Example            |
| ------------- | ---------------------------------------------------- | ------------------ |
| `x-span-name` | Human-readable label for this span                   | `hotel-search`     |
| `x-span-id`   | Custom span ID (optional; auto-generated if omitted) | `a1b2c3d4e5f6a7b8` |
| `x-trace-id`  | Overrides `trace_id` from `traceparent`              | `abc123...`        |

***

### Usage

{% tabs %}
{% tab title="cURL" %}

```bash
TRACE_ID=$(xxd -p -l 16 /dev/urandom)
PARENT_ID=$(xxd -p -l 8 /dev/urandom)
TRACEPARENT="00-${TRACE_ID}-${PARENT_ID}-01"

curl https://api.fastrouter.ai/v1/chat/completions \
  -H "Authorization: Bearer $FASTROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -H "traceparent: $TRACEPARENT" \
  -d '{
    "model": "gpt-4.1",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

{% endtab %}

{% tab title="Python" %}

```python
import secrets
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_FASTROUTER_API_KEY",
    base_url="https://api.fastrouter.ai/v1"
)

def generate_traceparent():
    return f"00-{secrets.token_hex(16)}-{secrets.token_hex(8)}-01"

traceparent = generate_traceparent()

# Step 1
response1 = client.chat.completions.create(
    model="gpt-4.1",
    messages=[...],
    extra_headers={"traceparent": traceparent}
)

# Step 2 (same trace)
response2 = client.chat.completions.create(
    model="gpt-4.1",
    messages=[...],
    extra_headers={"traceparent": traceparent}
)
```

{% endtab %}

{% tab title="Node.js" %}

```javascript
import OpenAI from "openai";
import { randomBytes } from "crypto";

const client = new OpenAI({
  apiKey: process.env.FASTROUTER_API_KEY,
  baseURL: "https://api.fastrouter.ai/v1",
});

function generateTraceparent(): string {
  const traceId = randomBytes(16).toString("hex");
  const parentId = randomBytes(8).toString("hex");
  return `00-${traceId}-${parentId}-01`;
}

const traceparent = generateTraceparent();

await client.chat.completions.create(
  {
    model: "gpt-4.1",
    messages: [{ role: "user", content: "Hello" }],
  },
  {
    headers: { traceparent },
  }
);
```

{% endtab %}
{% endtabs %}

***

### What FastRouter does

* Groups by `trace_id`
* Uses your `parent_id`
* Generates `span_id`
* Names spans: `POST /api/v1/chat/completions`

***

### Output

One trace → multiple spans

```
Trace: <trace_id>
├── POST /api/v1/chat/completions
└── POST /api/v1/chat/completions
```

Each span includes: latency, tokens, cost, request, response.


# MCP Gateway

### Overview

The MCP (Model Context Protocol) Gateway lets you register external tool servers — such as GitHub, Linear, DeepWiki, Intercom, or your own internal APIs — and make their capabilities available to any LLM call routed through FastRouter.

Rather than managing tool authentication and invocation logic in every client, you configure it once in FastRouter. The gateway handles credential injection, tool discovery, and auto-execution on behalf of your models.

FastRouter supports the **Streamable HTTP** transport defined in the MCP 2025-03-26 specification, as well as **SSE-based servers**.

#### Key Capabilities

| Capability                            | Description                                                                                                                                |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| **Centralized credential management** | Static headers and OAuth tokens are stored securely server-side. Models never receive raw credentials.                                     |
| **Selective tool exposure**           | Choose exactly which tools from a server are made available. Destructive operations (e.g. `delete_repository`) can be excluded entirely.   |
| **Project-level scoping**             | Restrict each MCP server to specific projects, or make it available across all projects in your organization.                              |
| **Auto-execution**                    | With `auto_execute_tools: true`, FastRouter handles the full tool-call loop, including multi-round execution up to a configurable maximum. |

***

### Setting Up an MCP Server

Add a new server from the **MCP Servers** page by clicking **Create Server**. The setup wizard walks through four steps.

#### Step 1 — Server and Scope

Define the connection details and which projects should have access to this server.

<figure><img src="/files/bKgFflGJTrIvG81DTXrx" alt=""><figcaption></figcaption></figure>

| Field         | Required | Description                                                                                              |
| ------------- | -------- | -------------------------------------------------------------------------------------------------------- |
| Server Type   | ✓        | Transport protocol. Streamable HTTP for remote servers using **HTTP POST + optional SSE**                |
| Server Name   | ✓        | Unique, lowercase slug used to reference this server in API calls (e.g. `github-production`). No spaces. |
| Display Name  | ✓        | Human-readable label shown in the dashboard (e.g. `Github Production`).                                  |
| Description   | —        | Optional summary of what this server provides.                                                           |
| Server URL    | ✓        | HTTP or SSE endpoint for the MCP server (e.g. `https://api.githubcopilot.com/mcp/`).                     |
| Project Scope | ✓        | One or more projects that can use this server, or **All Projects** for organization-wide access.         |

***

#### Step 2 — Authentication

Choose how FastRouter authenticates with the MCP server on each request. Three methods are supported.

1. **No Authentication**

No credentials are sent. Suitable for public or internally-trusted servers. Tools run without any credential injection.

<figure><img src="/files/d16mRvfzzj6sXQWCD4DE" alt=""><figcaption></figcaption></figure>

2. **Static Header**

A fixed API key or bearer token injected into each outbound request header. FastRouter stores the value encrypted at rest.

<figure><img src="/files/XKmS1G7Ptf9dUs7jtxIu" alt=""><figcaption></figcaption></figure>

| Field  | Required | Description                                                                               |
| ------ | -------- | ----------------------------------------------------------------------------------------- |
| Header | ✓        | Header name to use (e.g. `Authorization`, `x-api-key`).                                   |
| String | ✓        | The token value — stored encrypted, never exposed in responses (e.g. `Bearer sk_live_…`). |

3. **OAuth 2.0**

Full OAuth 2.0 authorization code flow with scoped permissions and token refresh. FastRouter acts as the OAuth client. A callback URL is provided — register it with your OAuth provider before proceeding.

<figure><img src="/files/Csdwg2uxkbiUbbmXfE54" alt=""><figcaption></figcaption></figure>

| Field             | Required | Description                                                                                |
| ----------------- | -------- | ------------------------------------------------------------------------------------------ |
| Client ID         | ✓        | Public identifier issued by your OAuth provider.                                           |
| Client Secret     | ✓        | Private key used to verify your app. Stored encrypted.                                     |
| Authorization URL | ✓        | The provider's login/consent page (e.g. `https://github.com/login/oauth/authorize`).       |
| Token URL         | ✓        | The provider's access-token endpoint (e.g. `https://github.com/login/oauth/access_token`). |
| OAuth Scopes      | —        | Comma-separated list of permission scopes (e.g. `repo`, `read:org`, `user:email`).         |

After filling in OAuth fields, click **Connect Now** to complete the authorization flow before proceeding to the next step.

***

#### Step 3 — Tool Selection

FastRouter connects to the server and enumerates its available tools.

<figure><img src="/files/c1rfxdiNwUGpXNGQ3wdL" alt=""><figcaption></figcaption></figure>

Select which tools to expose to models.

* The **left panel** lists all tools available from the server.
* The **right panel** shows your current selection.
* Use the **search box** to filter large tool sets.
* **Select All** enables every tool at once; **Clear All** resets the selection.

***

#### Step 4 — Review and Create

Confirm all settings before creating the server.

<figure><img src="/files/xzFNPFjoI75ZQZZ63FnF" alt=""><figcaption></figcaption></figure>

The review screen shows:

* **Connection** — server name, transport type, URL, and authentication method
* **Tools** — the enabled tool list
* **Projects** — which projects have access

Click **Create Server** to finalize.

***

### Managing Servers

All registered MCP servers are listed on the **MCP Servers** page. Each row shows status, tool count, authentication method, and project scope at a glance.

| Column                | Description                                                                                             |
| --------------------- | ------------------------------------------------------------------------------------------------------- |
| Server Name           | Display name and description.                                                                           |
| Status                | `Active` — server is reachable and tools are available. `Inactive` — server is unreachable or disabled. |
| Tools                 | Count of tools enabled on this server.                                                                  |
| Authentication Method | One of `No Auth`, `Static Headers`, `OAuth 2.0`                                                         |
| Projects              | Projects with access. `All Projects` means organization-wide.                                           |

Use the **+ Project**, **+ Status**, and **+ Authentication Method** filter chips to narrow the list. Click the pencil icon to edit a server or the trash icon to delete it.

***

### API Reference

MCP tool calling is controlled through the `mcp` object in your chat completions request body. All standard FastRouter routing parameters apply alongside MCP settings.

#### Endpoint

```
POST https://api.fastrouter.ai/api/v1/chat/completions
```

#### MCP Parameters

| Parameter                | Type      | Required     | Description                                                                                                                                                                                                 |
| ------------------------ | --------- | ------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `mcp`                    | object    | —            | Top-level object that controls MCP behavior for this request.                                                                                                                                               |
| `mcp.enabled`            | boolean   | —            | Activates MCP for this request. Default: `false`.                                                                                                                                                           |
| `mcp.servers`            | string\[] | ✓ if enabled | List of server name slugs to make available to the model. Servers must belong to the active project and be in `Active` status.                                                                              |
| `mcp.auto_execute_tools` | boolean   | —            | When `true`, FastRouter automatically invokes tool calls, feeds results back, and continues until a final response is produced or `max_tool_rounds` is reached. Requires `stream: false`. Default: `false`. |
| `mcp.max_tool_rounds`    | integer   | —            | Maximum tool-call/response cycles when `auto_execute_tools` is enabled. Hard-capped at `5`. Default: `5`.                                                                                                   |

**Note:** `auto_execute_tools: true` requires `stream: false`. Non-streaming requests with auto-execution enabled will return a `tool_calls` response and stop — the caller is responsible for continuing the conversation.

***

### Examples

#### Basic request without auto execute

Attach a server to a request. When the model decides to use a tool, the response `finish_reason` is `tool_calls` and the message contains a populated `tool_calls` array. The caller is responsible for executing the tool and sending results back.

```bash
curl --location 'https://api.fastrouter.ai/api/v1/chat/completions' \
--header 'Authorization: Bearer <API-KEY>' \
--header 'Content-Type: application/json' \
--data '{
    "messages": [
        { "role": "system", "content": "You are a helpful assistant" },
        { "role": "user",   "content": "What transport protocols does the 2025-03-26 MCP spec support?" }
    ],
    "model": "openai/gpt-5.2",
    "stream": false,
    "mcp": {
        "enabled": true,
        "servers": ["deepwiki-mcp"],
        "auto_execute_tools": false
    }
}'
```

**Response — model returns a tool call:**

```json
{
    "choices": [{
        "finish_reason": "tool_calls",
        "message": {
            "role": "assistant",
            "content": "",
            "tool_calls": [{
                "id": "call_kebySk5LaUdfpAWkfqR7MHW2",
                "type": "function",
                "function": {
                    "name": "deepwiki-mcp__ask_question",
                    "arguments": "{\"repoName\":\"modelcontextprotocol/modelcontextprotocol\",\"question\":\"...\"}"
                }
            }]
        }
    }],
    "usage": {
        "prompt_tokens": 323,
        "completion_tokens": 63,
        "total_tokens": 386,
        "cost": 0.00144725
    }
}
```

***

#### Basic request with auto execution

Set `auto_execute_tools: true` with `stream: false` to let FastRouter complete the full tool loop and return a final text answer directly.

```json
{
    "messages": [
        { "role": "system", "content": "" },
        { "role": "user",   "content": "What transport protocols does the 2025-03-26 MCP spec support?" }
    ],
    "model": "openai/gpt-5.2",
    "stream": false,
    "mcp": {
        "enabled": true,
        "servers": ["deepwiki-mcp"],
        "auto_execute_tools": true,
        "max_tool_rounds": 2
    }
}
```

**Response — final text answer, no tool\_calls:**

```json
{
    "choices": [{
        "finish_reason": "stop",
        "message": {
            "role": "assistant",
            "content": "The 2025-03-26 MCP spec defines two primary transports:\n\n1. **STDIO** — uses the server process's standard input/output streams for local, spawned processes.\n2. **Streamable HTTP** — HTTP POST for client→server messages, with optional SSE for streaming server→client responses."
        }
    }],
    "usage": {
        "prompt_tokens": 2677,
        "completion_tokens": 405,
        "total_tokens": 3082,
        "cost": 0.01035475
    }
}
```

***

#### Multiple servers

Include multiple server slugs in the `servers` array. The model decides which tools from which servers to invoke based on the request.

```json
"mcp": {
    "enabled": true,
    "servers": ["github-production", "jira-mcp", "slack-mcp"],
    "auto_execute_tools": true,
    "max_tool_rounds": 3
}
```

***

### Usage Notes

#### Tool name format

Tool names in responses follow the format `{server-name}__{tool-name}` — for example, `deepwiki-mcp__ask_question`. This makes it easy to identify the originating server in traces and logs.

#### Token costs

Tool schemas and intermediate tool results are injected into the model's context window, increasing prompt token usage compared to standard chat completions. This is reflected in `usage.prompt_tokens` and `usage.cost` in each response.

#### Server name collisions

Server names must be unique within an organization. If you need the same MCP server registered under two different credential sets (e.g. read-only vs. read-write GitHub access), use distinct slugs such as `github-readonly` and `github-admin`.

#### Inactive servers

Referencing an `Inactive` server in an API call will return an error. Ensure the server is reachable and credentials are valid before using it in production.

MCP tool calls are captured in **Logs & Traces** with full span details, including tool name, arguments, and execution time. Use the trace viewer to debug multi-round tool executions.


# Web Search

When using a web-search-enabled model, you can pass the `web_search_options` parameter to control how much search context is retrieved and processed. Models with this capability can dynamically integrate search results into their reasoning process.

There's a per-request fee applied by these models. Additionally, these models charge based on search context size, which controls how much data is retrieved and processed per query.

### **Web-Search Enabled Models**

These models support built-in web search:

* `openai/gpt-4o-mini-search-preview`
* `openai/gpt-4o-search-preview`
* `perplexity/sonar-pro`
* `perplexity/sonar-reasoning-pro`
* `perplexity/sonar`
* `perplexity/sonar-reasoning`

### **Search Context Size**

The `search_context_size` setting controls how much information is pulled from search results. Pricing may vary based on the selected level.

| Level    | Description                                        | Use Case                           |
| -------- | -------------------------------------------------- | ---------------------------------- |
| `low`    | Minimal context for basic queries                  | Quick facts, dates, headlines      |
| `medium` | Moderate context with broader information coverage | General knowledge, short summaries |
| `high`   | Extensive search context for deep research         | In-depth topics, analysis, reports |

### **Sample Request**

This example uses `openai/gpt-4o-mini-search-preview` with medium search context to get real-time sports event info:

```bash
curl --location 'https://api.fastrouter.ai/api/v1/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer API-KEY' \
--data '{
  "model": "openai/gpt-4o-mini-search-preview",
  "messages": [
    {
      "role": "user",
      "content": "Which teams are playing the UEFA Champions League final?"
    }
  ],
  "stream": false,
  "top_p": 1,
  "temperature": 0,
  "max_completion_tokens": 120,
  "web_search_options": {
    "search_context_size": "medium"
  }
}'
```

### **Enabling Web Search for Any Model**

You can incorporate relevant web search results for *any* model on FastRouter by appending **:online** to the model slug:

```
{
  "model": "openai/gpt-oss-20b:online"
}
```

### **Pricing**

The web plugin uses your FastRouter credits and charges *$5 per 1000 answers*.


# skill

Use FastRouter's official skill.md file to give your AI coding assistants knowledge of FastRouter features

```markdown
---
name: FastRouter
description: Use when routing AI requests through a unified LLM gateway, managing multiple providers, setting up virtual model aliases, configuring fallback policies, enabling BYOK (Bring Your Own Key), processing batch requests, tracking costs, enforcing guardrails, evaluating model outputs, or integrating the MCP Gateway. Also use when setting up FastRouter as a model provider in OpenClaw — triggered by phrases like "set up fastrouter", "add fastrouter provider", "configure fastrouter with API key sk-v1-xxxxx", or "update fastrouter models". Reach for this skill when building AI applications that need multi-provider support, reliability, cost optimization, multimodal capabilities, or analytics.
metadata:
    docs-proj: fastrouter
    version: "1.0"
---
```

### Product Summary

FastRouter.ai is an enterprise-grade LLM Gateway that acts as a control plane for routing requests across 100+ AI models from multiple providers through a single OpenAI-compatible API. It provides intelligent routing, automatic failover, cost governance, multimodal support (text, image, video, audio), and built-in observability — eliminating vendor lock-in while reducing AI spend. No setup fees, no monthly minimums, and free credits to start.

FastRouter.ai sits between your application and LLM providers (OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more), handling request routing, credential management, cost tracking, error fallback, and performance optimization. Key differentiators include the Auto Router (dynamic model selection by cost/latency/quality), Virtual Model Aliases (custom model pools with policy-driven selection), per-key budget controls, batch processing, custom evaluations, guardrails, MCP Gateway, and native support for multiple API formats (OpenAI Chat Completions, OpenAI Responses, Anthropic Messages, Gemini Native).

See [docs.fastrouter.ai](https://docs.fastrouter.ai) for full documentation.

***

### When to Use

Reach for FastRouter when:

* **Multi-model routing**: You need to route requests across providers for cost, latency, or quality optimization
* **High availability**: You want automatic failover and retries across providers when one goes down
* **Cost governance**: You need per-key budgets, rate limits, model restrictions, and real-time spend tracking
* **Model comparison**: You want side-by-side evaluation of model outputs in interactive playgrounds
* **Multimodal pipelines**: You're working with text, image, video, or audio through one API with consistent auth and billing
* **Batch processing**: You're running bulk operations (up to 50K requests) across OpenAI and Anthropic models
* **Enterprise controls**: You need RBAC, BYOK, provisioning keys, project isolation, and audit logging
* **IDE integration**: You want a drop-in replacement for OpenAI in Cursor, Cline, Claude Code, and other tools
* **OpenClaw setup**: A user wants to add FastRouter as a provider in OpenClaw (see OpenClaw section below)

***

### Quick Reference

#### Base URLs

| URL                                   | Purpose                             |
| ------------------------------------- | ----------------------------------- |
| `https://api.fastrouter.ai/api/v1`    | Primary API gateway (LLM endpoints) |
| `https://api.fastrouter.ai/prod/`     | Provisioning / admin endpoints only |
| `https://api.fastrouter.ai/v1/models` | Model catalog (for OpenClaw setup)  |

#### Authentication

All endpoints use Bearer token authentication:

```
Authorization: Bearer YOUR_FASTROUTER_API_KEY 
Content-Type: application/json
```

#### Core API Endpoints

| Endpoint            | Method | Path                         | Description                                           |
| ------------------- | ------ | ---------------------------- | ----------------------------------------------------- |
| Chat Completions    | POST   | `/api/v1/chat/completions`   | Text generation, function calling, structured outputs |
| Responses           | POST   | `/api/v1/responses`          | OpenAI Responses API format                           |
| Embeddings          | POST   | `/api/v1/embeddings`         | Vector embeddings                                     |
| Image Generation    | POST   | `/api/v1/images/generations` | Image creation (GPT Image 1, DALL-E)                  |
| Image Edit          | POST   | `/api/v1/images/edits`       | Image editing                                         |
| Video Generation    | POST   | `/api/v1/videos`             | Video creation (Veo 3, Sora 2, Kling, etc.)           |
| Video Status        | POST   | `/api/v1/getAsyncResponse`   | Poll video generation status                          |
| Audio Transcription | POST   | `/v1/audio/transcriptions`   | Speech-to-text (Whisper)                              |
| Audio Translation   | POST   | `/v1/audio/translations`     | Audio to English text                                 |
| Text-to-Audio       | POST   | `/api/v1/chat/completions`   | Audio generation (ace-step model)                     |
| Audio Status        | POST   | `/api/v1/getAsyncResponse`   | Poll audio generation status                          |
| List Models         | GET    | `/api/v1/models`             | Full model catalog with metadata                      |
| Generations         | GET    | `/api/v1/generation`         | Request generation details/stats                      |
| Moderations         | POST   | (see docs)                   | Content moderation                                    |

#### Model Naming Convention

Models use the `provider/model-name` format:

`openai/gpt-5.4`

`anthropic/claude-4.5-sonnet`

`google/gemini-3.1-pro-preview`

`x-ai/grok-4.1-fast`

`perplexity/sonar-pro`

Append `:online` to enable web search on any model: `x-ai/grok-4.1-fast:online`

#### Common Model IDs

| Provider   | Example Models                                                                                                                         |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| OpenAI     | `openai/gpt-5.4-nano`, `openai/gpt-5.4`, `openai/gpt-5.4-mini`, `openai/o4-mini`, `openai/gpt-image-1`, `openai/sora-2-pro`            |
| Anthropic  | `anthropic/claude-opus-4.6`, `anthropic/claude-sonnet-4.6`, `anthropic/claude-haiku-4.5`                                               |
| Google     | `google/gemini-3.1-pro-preview`, `google/gemini-3.1-flash-image-preview`, `google/veo3.1-fast`, `google/gemini-3.1-flash-lite-preview` |
| xAI        | `x-ai/grok-4`, `x-ai/grok-4.20-beta`                                                                                                   |
| Minimax    | `minimax/minimax-m2.7`, `minimax/minimax-m2.5-highspeed`                                                                               |
| Perplexity | `perplexity/sonar-pro`, `perplexity/sonar-reasoning-pro`                                                                               |
| Audio      | `ace-step/prompt-to-audio`                                                                                                             |
| Video      | `kling-ai/kling-v3`, `wanx/wan-v2-6`, `bytedance/seedance-pro`                                                                         |

#### Request Parameters

```json
{
  "model": "openai/gpt-5.4",
  "messages": [{"role": "user", "content": "..."}],
  "stream": true,
  "temperature": 0.7,
  "max_completion_tokens": 2000,
  "response_format": {"type": "json_schema", "json_schema": {...}},
  "tools": [...],
  "tool_choice": "auto",
  "web_search_options": {"search_context_size": "medium"}
}
```

**FastRouter-specific parameters** (pass via `extra_body` in OpenAI SDK):

| Parameter   | Type   | Description                                                       |
| ----------- | ------ | ----------------------------------------------------------------- |
| `reasoning` | object | `{"max_tokens": N}` — enable reasoning/thinking tokens            |
| `tags`      | array  | `["tag1", "tag2"]` — custom metadata tags for analytics filtering |

#### Key Types

| Key Type                | Purpose                                                                  |
| ----------------------- | ------------------------------------------------------------------------ |
| **API Key**             | Standard LLM requests via `Authorization: Bearer`                        |
| **Provisioning Key**    | Admin-only; creates/manages Service Account Keys                         |
| **Service Account Key** | Scoped access keys with per-key budgets, rate limits, model restrictions |

#### Routing Strategy Types

| Strategy                  | Use Case                                                                                               |
| ------------------------- | ------------------------------------------------------------------------------------------------------ |
| **Auto Router**           | Dynamic model selection by cost/latency/quality — no configuration needed                              |
| **Virtual Model Aliases** | Custom pool of models with policy-driven selection (weighted, priority, round-robin, cost-optimized)   |
| **Fallback Models**       | Prioritized fallback chain across providers for high availability                                      |
| **Provider Routing**      | Multi-provider selection for same model (lowest latency, lowest cost, round robin, weighted, priority) |

***

### Decision Guidance

#### When to Use Each Routing Strategy

| Scenario                                      | Strategy                                   |
| --------------------------------------------- | ------------------------------------------ |
| No strong model preference, want best value   | **Auto Router**                            |
| A/B testing or gradual rollout across models  | **Virtual Model Alias** (weighted)         |
| Critical workload needing guaranteed uptime   | **Fallback Models** (priority chain)       |
| Same model available on multiple providers    | **Provider Routing** (lowest latency/cost) |
| Category-based tasks needing different models | **Virtual Model Alias** (category routing) |
| Cost-first, quality-second selection          | **Auto Router** (cost-optimized mode)      |

#### When to Use BYOK vs. FastRouter Credits

| Scenario                             | Recommendation                                  |
| ------------------------------------ | ----------------------------------------------- |
| Getting started / low volume         | FastRouter credits (simpler)                    |
| You have existing provider contracts | **BYOK** — use your own rate limits and billing |
| Need specific model versions         | **BYOK** — you control the provider account     |
| Azure or AWS Bedrock integration     | **BYOK** — required for these providers         |
| Simplicity is priority               | FastRouter credits                              |
| Cost optimization across providers   | BYOK + FastRouter credits as fallback           |

#### When to Use Batch vs. Real-time

| Scenario                       | Use Batch | Use Real-time |
| ------------------------------ | --------- | ------------- |
| Dataset processing (10K+ rows) | ✅ Yes     | ❌ No          |
| User-facing chatbot            | ❌ No      | ✅ Yes         |
| Scheduled overnight jobs       | ✅ Yes     | ❌ No          |
| Latency-sensitive requests     | ❌ No      | ✅ Yes         |
| OpenAI or Anthropic only       | ✅ Yes     | ✅ Yes         |
| Cost-sensitive bulk inference  | ✅ Yes     | ❌ No          |

***

### Workflows

#### 1. Basic Setup (Python / OpenAI SDK)

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.fastrouter.ai/api/v1",
    api_key="YOUR_FASTROUTER_API_KEY"
)

response = client.chat.completions.create(
    model="openai/gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)
```

#### 2. Basic Setup (cURL)

```bash
curl -X POST https://api.fastrouter.ai/api/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -d '{
    "model": "openai/gpt-5.4",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "What is FastRouter?"}
    ]
  }'
```

#### 3. Structured JSON Output

```python
response = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[
        {"role": "system", "content": "Respond in JSON format only."},
        {"role": "user", "content": "Give me weather for London."}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "weather",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"},
                    "temperature": {"type": "number"},
                    "conditions": {"type": "string"}
                },
                "required": ["location", "temperature", "conditions"],
                "additionalProperties": False
            }
        }
    }
)
```

#### 4. Streaming with Reasoning Tokens

```python
response = client.chat.completions.create(
    model="google/gemini-3.1-pro-preview",
    messages=[{"role": "user", "content": "Explain quantum entanglement"}],
    stream=True,
    extra_body={"reasoning": {"max_tokens": 2000}}
)
for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

#### 5. Web Search

```python
# Option A: Use a search-native model
response = client.chat.completions.create(
    model="perplexity/sonar-pro",
    messages=[{"role": "user", "content": "Latest AI news today"}]
)

# Option B: Append :online to any model
response = client.chat.completions.create(
    model="openai/gpt-5.2:online",
    messages=[{"role": "user", "content": "Latest AI news today"}],
    extra_body={"web_search_options": {"search_context_size": "medium"}}
)
```

#### 6. Batch Processing

Upload a JSONL file to the dashboard or API. Each line follows this format:

```json
{"custom_id": "req-1", "provider": "openai", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-4.1-nano", "messages": [{"role": "user", "content": "Summarize this text..."}], "max_tokens": 500}}
{"custom_id": "req-2", "provider": "anthropic", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "anthropic/claude-haiku-4.5", "messages": [{"role": "user", "content": "Translate to French: Hello"}], "max_tokens": 100}}
```

Limits: up to 50,000 requests per file. Results delivered as downloadable JSONL within 24h.

#### 7. Provisioning Keys (Programmatic Key Management)

```bash
# Create a scoped service key
curl -X POST https://api.fastrouter.ai/prod/createServiceKey \
  -H "Authorization: Bearer YOUR_PROVISIONING_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "credit_limit": 10.00,
    "reset_budget_interval": "monthly",
    "models": ["openai/gpt-4.1", "anthropic/claude-haiku-4.5"],
    "rpm_limit": 60,
    "tpm_limit": 100000,
    "tags": ["team:engineering", "env:production"]
  }'
```

Provisioning endpoints:

| Action     | Endpoint                      |
| ---------- | ----------------------------- |
| Create key | `POST /prod/createServiceKey` |
| Update key | `POST /prod/updateServiceKey` |
| List keys  | `POST /prod/getServiceKeys`   |
| Delete key | `POST /prod/deleteServiceKey` |

> **Important**: Save the `api_key_id_hash` returned on creation — required for all future updates and deletions.

***

### OpenClaw Integration

Use this section when a user wants to set up FastRouter as a model provider in OpenClaw. Triggered by phrases like "set up fastrouter", "add fastrouter provider", "configure fastrouter", "update fastrouter models", or when a user provides an API key starting with `sk-v1-`.

#### Inputs

* **API Key** (required): Starts with `sk-v1-` followed by a hex string. Ask for it if not provided.
* **Base URL** (optional): Defaults to `https://api.fastrouter.ai`

#### Steps

**Step 1 — Extract the API key**

Parse the API key from the user's message. Must start with `sk-v1-`. Do NOT proceed without one.

**Step 2 — Fetch the live model list**

```
web_fetch url="https://api.fastrouter.ai/v1/models" extractMode="text"
```

**Step 3 — Filter models**

Keep only models where:

* `is_active` is `true`
* `architecture.output_modalities` includes `"text"`
* `architecture.input_modalities` includes `"text"` or `"image"`

For each qualifying model, extract:

* `id` — the model identifier
* `context_length` — context window size
* `top_provider.max_completion_tokens` — max output tokens (if 0 or missing, use `min(context_length, 8192)`)
* Input types: list of `"text"` and/or `"image"` from input\_modalities

**Step 4 — Build the provider config**

Do NOT include a `"name"` key — OpenClaw rejects it:

```json
{
  "baseUrl": "https://api.fastrouter.ai",
  "api": "openai-completions",
  "apiKey": "THE_API_KEY",
  "models": [
    {
      "id": "provider/model-id",
      "name": "Display Name",
      "contextWindow": 128000,
      "maxTokens": 8192,
      "input": ["text", "image"],
      "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
      "reasoning": false
    }
  ]
}
```

**Step 5 — Update openclaw\.json**

1. Use `read` to load `~/.openclaw/openclaw.json`
2. Merge the provider into `models.providers.fastrouter` (preserving all other config)
3. Add model references to `agents.defaults.models` — for each model: `"fastrouter/MODEL_ID": {}`
4. Use `write` to save the updated config

**Step 6 — Restart the gateway** *(requires user approval)*

```bash
openclaw gateway restart
```

**Step 7 — Report to user**

Tell the user:

* How many models were added
* They can switch models with `/model fastrouter/MODEL_ID`
* Suggest popular models: claude, gpt, gemini, deepseek variants

#### OpenClaw Error Handling

| Error                  | Action                                                    |
| ---------------------- | --------------------------------------------------------- |
| API unreachable        | Tell user the FastRouter API may be down; try again later |
| No qualifying models   | Warn that no text/image models were found                 |
| Config file missing    | Create the full structure from scratch                    |
| Invalid API key format | Ask user to double-check their key starts with `sk-v1-`   |

#### OpenClaw Notes

* Provider key in config is `fastrouter`
* Existing fastrouter config is replaced with fresh model list on update
* All other providers and settings are preserved
* Cost is set to zero — FastRouter handles billing separately
* Video-only and audio-only models are excluded from the model list

***

### Common Gotchas

* **Base URLs**: `api.fastrouter.ai` both serve LLM endpoints; provisioning uses `api.fastrouter.ai/prod/` exclusively
* **Model naming is required**: Always use `provider/model-name` format (e.g., `openai/gpt-4o`, not `gpt-4o`); requests with bare model names may fail or route incorrectly
* **Reasoning token continuity**: When using reasoning-capable models, you **must** pass `reasoning_details` from the assistant message back in follow-up requests or you will get errors in multi-turn conversations
* **Audio and video are async**: Text-to-audio and video generation return a job ID; you must poll `getPromptToAudioResponse` or `getVideoResponse` for the result — there is no synchronous response
* **Audio endpoint path differs**: Transcription/translation use `/v1/audio/...` (no `/api` prefix), unlike other endpoints at `/api/v1/...`
* **Batch file consistency**: All requests in a JSONL batch must target the same endpoint (chat completions OR embeddings), but can mix models and providers within that constraint
* **Batch provider support**: Batch processing currently supports OpenAI and Anthropic only — other providers are not available in batch mode
* **BYOK billing**: When using Bring Your Own Key, rate limits and costs are governed by your provider account, not FastRouter credits
* **Provisioning Keys ≠ API Keys**: Provisioning Keys cannot make LLM requests; they are admin-only for key lifecycle management
* **Service key hash**: Save the `api_key_id_hash` returned from `createServiceKey` — it is the only identifier for future updates and deletions
* **Web search pricing**: `:online` suffix and search-preview models cost $5 per 1,000 answers in addition to model token costs
* **Nano Banana (gemini-2.5-flash-image)**: Use through Chat Completions, not the Image Generation endpoint; supports custom `aspectRatio` parameter
* **Streaming + tool calls**: When using `stream: true` with function calling, tool calls only appear in the final streaming chunk
* **Free credits expire**: Promotional credits expire in 30 days

***

### Verification Checklist

* [ ] **API key** set in `Authorization: Bearer <key>` header
* [ ] **Base URL** changed to `https://api.fastrouter.ai/api/v1` (not OpenAI's URL)
* [ ] **Model name** follows `provider/model-name` format
* [ ] **Structured outputs**: `response_format` paired with a JSON-instructing system prompt
* [ ] **Reasoning continuity**: `reasoning_details` preserved in follow-up messages when using reasoning models
* [ ] **Batch JSONL**: Consistent endpoint across all lines; `provider` field included per request
* [ ] **Async operations**: Polling logic in place for video/audio result retrieval
* [ ] **Per-key budget + rate limits**: Configured in dashboard before production deployment
* [ ] **Tags applied**: `extra_body={"tags": [...]}` added for analytics filtering
* [ ] **Credits sufficient**: Monitored via `usage.credits_used` in responses
* [ ] **Service key hash saved**: `api_key_id_hash` stored securely if using Provisioning Keys
* [ ] **OpenClaw (if applicable)**: No `"name"` key in provider config; gateway restarted after config update

***

### Resources

| Resource                                       | URL                                                                            |
| ---------------------------------------------- | ------------------------------------------------------------------------------ |
| Main website                                   | <https://fastrouter.ai>                                                        |
| Documentation home                             | <https://docs.fastrouter.ai>                                                   |
| Model catalog                                  | <https://fastrouter.ai/models>                                                 |
| Chat Completions API                           | <https://docs.fastrouter.ai/api-reference/chat-completions>                    |
| Responses API                                  | <https://docs.fastrouter.ai/api-reference/responses>                           |
| Embeddings API                                 | <https://docs.fastrouter.ai/api-reference/embeddings>                          |
| List Models API                                | <https://docs.fastrouter.ai/api-reference/models>                              |
| Auto Router                                    | <https://docs.fastrouter.ai/api-reference/auto-router>                         |
| Image Generation API                           | <https://docs.fastrouter.ai/api-reference/image-generation>                    |
| Video Generation API                           | <https://docs.fastrouter.ai/api-reference/video-generation>                    |
| Transcriptions & Translations                  | <https://docs.fastrouter.ai/api-reference/transcriptions-and-translations-api> |
| Text-to-Audio API                              | <https://docs.fastrouter.ai/api-reference/text-to-audio-generation-api>        |
| Batch Processing API                           | <https://docs.fastrouter.ai/api-reference/batch-processing>                    |
| Generations (Request Details)                  | <https://docs.fastrouter.ai/api-reference/generations>                         |
| Error Codes                                    | <https://docs.fastrouter.ai/api-reference/error-codes>                         |
| Virtual Model Aliases                          | <https://docs.fastrouter.ai/virtual-model-aliases>                             |
| Fallback Models                                | <https://docs.fastrouter.ai/fallback-models>                                   |
| Provider Routing Strategies                    | <https://docs.fastrouter.ai/provider-routing-strategies>                       |
| Automatic Model Selection                      | <https://docs.fastrouter.ai/automatic-model-selection>                         |
| BYOK (External Keys), Custom Endpoint & Models | <https://docs.fastrouter.ai/add-external-keys-byok>                            |
| Guardrails                                     | <https://docs.fastrouter.ai/guardrails>                                        |
| Custom Evaluations                             | <https://docs.fastrouter.ai/custom-evaluations>                                |
| Video Evaluations                              | <https://docs.fastrouter.ai/video-evaluations>                                 |
| Flex Inference                                 | <https://docs.fastrouter.ai/explore-features/flex-pricing>                     |
| Prompt Caching                                 | <https://docs.fastrouter.ai/prompt-caching>                                    |
| Batch Processing                               | <https://docs.fastrouter.ai/batch-processing>                                  |
| Structured Outputs                             | <https://docs.fastrouter.ai/structured-outputs>                                |
| Function Calling                               | <https://docs.fastrouter.ai/function-calling>                                  |
| Reasoning Tokens                               | <https://docs.fastrouter.ai/reasoning-tokens>                                  |
| Response Caching                               | <https://docs.fastrouter.ai/response-caching>                                  |
| MCP Gateway                                    | <https://docs.fastrouter.ai/mcp-gateway>                                       |
| PDF Processing                                 | <https://docs.fastrouter.ai/pdf-processing>                                    |
| Web Search                                     | <https://docs.fastrouter.ai/web-search>                                        |
| Dynamic Tags                                   | <https://docs.fastrouter.ai/dynamic-tags-per-request>                          |
| File & Image Inputs                            | <https://docs.fastrouter.ai/file-and-image-inputs>                             |
| Tracing                                        | <https://docs.fastrouter.ai/tracing>                                           |
| Alerts                                         | <https://docs.fastrouter.ai/alerts>                                            |
| Credits                                        | <https://docs.fastrouter.ai/credits>                                           |
| Provisioning Keys                              | <https://docs.fastrouter.ai/provisioning-keys>                                 |
| Keys & Settings                                | <https://docs.fastrouter.ai/keys-and-settings>                                 |
| Projects                                       | <https://docs.fastrouter.ai/projects>                                          |
| Organization & Members                         | <https://docs.fastrouter.ai/organization-and-members>                          |
| IDE Integrations                               | <https://docs.fastrouter.ai/integrations/ide-integrations>                     |
| Claude Code Integration                        | <https://docs.fastrouter.ai/integrations/claude-code>                          |
| Hermes Agent Integration                       | <https://docs.fastrouter.ai/integrations/running-hermes-agent-with-fastrouter> |
| Changelog                                      | <https://docs.fastrouter.ai/changelog>                                         |

***

> For full documentation and navigation, see: <https://docs.fastrouter.ai>

{% file src="/files/ux8ivQmbu3txyT8wJj95" %}


# Chat Completions

## Create Chat Completion

> Creates a chat completion for the provided messages. Supports streaming, function calling, vision, multimodal inputs, structured outputs (\`response\_format\`), tool selection (\`tool\_choice\`), semantic caching (\`cache\` + \`cache\_key\` header), Anthropic prompt caching (\`cache\_control\` on message content blocks), multi-model routing (\`models\`), request tagging (\`request\_tags\`), and reasoning tokens for supported models (OpenAI o-series, Grok, Gemini thinking, Anthropic). Compatible with OpenAI SDK.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Chat Completions","description":"Create AI-powered chat responses with support for text, images, audio, video, streaming, and tool calling."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/chat/completions":{"post":{"operationId":"createChatCompletion","tags":["Chat Completions"],"summary":"Create Chat Completion","description":"Creates a chat completion for the provided messages. Supports streaming, function calling, vision, multimodal inputs, structured outputs (`response_format`), tool selection (`tool_choice`), semantic caching (`cache` + `cache_key` header), Anthropic prompt caching (`cache_control` on message content blocks), multi-model routing (`models`), request tagging (`request_tags`), and reasoning tokens for supported models (OpenAI o-series, Grok, Gemini thinking, Anthropic). Compatible with OpenAI SDK.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["model","messages"],"properties":{"model":{"type":"string","description":"Model ID in format 'provider/model'. Examples: openai/gpt-5.1, google/gemini-3-pro-preview, anthropic/claude-4.5-sonnet"},"models":{"type":"array","items":{"type":"string"},"description":"Optional list of model IDs for multi-model routing. FastRouter can route across these models in order. Supports shortcut syntax (e.g. `openai/gpt-4o:thinking` to enable high reasoning effort). Can be used alongside a single `model`."},"request_tags":{"type":"array","items":{"type":"string"},"description":"Optional tags to attach to the request for logging, analytics, and filtering."},"messages":{"type":"array","description":"Array of message objects forming the conversation history. Each message has a role (system/user/assistant/tool) and content.","minItems":1,"items":{"type":"object","required":["role","content"],"properties":{"role":{"type":"string","enum":["system","user","assistant","tool"],"description":"Role of the message author:\n- system: Instructions for the AI\n- user: User messages\n- assistant: AI responses\n- tool: Tool/function outputs"},"content":{"oneOf":[{"type":"string","description":"Text content"},{"type":"array","description":"Multimodal content (text, images, audio, etc.). Text blocks on Anthropic models may include `cache_control` for prompt caching.","items":{"type":"object","properties":{"type":{"type":"string","description":"Content block type, e.g. `text`, `image_url`, `input_audio`, `file`."},"text":{"type":"string","description":"Text content (when type is `text`)."},"cache_control":{"type":"object","description":"Anthropic prompt caching control on this content block. Set on text blocks in `messages` to mark content for ephemeral caching.","properties":{"type":{"type":"string","enum":["ephemeral"],"description":"Cache type. Use `ephemeral` for Anthropic prompt caching."}}}}}}],"description":"Message content - can be a string for text-only or an array for multimodal inputs (text, images, audio, video)"},"name":{"type":"string","description":"Optional name of the message author"}}}},"temperature":{"type":"number","minimum":0,"maximum":2,"default":1,"description":"Controls randomness in responses. Lower values (0-0.7) make output more focused and deterministic. Higher values (0.8-2) make output more creative and random."},"max_tokens":{"type":"integer","minimum":1,"description":"Maximum number of tokens to generate in the completion. Limits the length of the response."},"top_p":{"type":"number","minimum":0,"maximum":1,"default":1,"description":"Nucleus sampling parameter. Alternative to temperature. Lower values make output more focused."},"frequency_penalty":{"type":"number","minimum":-2,"maximum":2,"default":0,"description":"Penalizes repeated tokens based on frequency. Positive values reduce repetition."},"presence_penalty":{"type":"number","minimum":-2,"maximum":2,"default":0,"description":"Penalizes tokens that have appeared. Positive values encourage new topics."},"stream":{"type":"boolean","default":false,"description":"Enable streaming responses for real-time output. When true, responses are sent as Server-Sent Events (SSE)."},"stop":{"oneOf":[{"type":"string"},{"type":"array","items":{"type":"string"}}],"description":"Stop sequences where the model will stop generating. Can be a string or array of strings."},"provider":{"type":"object","description":"Optional: Control provider routing behavior. If not specified, FastRouter intelligently selects the best provider based on availability, performance, and cost. Use either 'only' OR 'order', not both.","properties":{"only":{"type":"array","description":"Force routing to specific providers only. Request will only use providers in this list. Use when you need guaranteed provider selection.","items":{"type":"string"},"minItems":1},"order":{"type":"array","description":"Ordered list of providers to try in sequence. FastRouter attempts each provider in order. Use with allow_fallbacks for high-availability routing.","items":{"type":"string"},"minItems":1},"allow_fallbacks":{"type":"boolean","description":"When used with 'order', enables automatic fallback to the next provider in the list if the current provider is unavailable. Set to true for high-availability routing.","default":true}}},"tools":{"type":"array","description":"Array of tool/function definitions"},"tool_choice":{"oneOf":[{"type":"string","enum":["auto","none","required"],"description":"Controls tool usage: `auto` (model decides), `none` (disable tools), or `required` (force a tool call)."},{"type":"object","description":"Force a specific tool/function.","properties":{"type":{"type":"string","enum":["function","tool"],"description":"Selection mode. Use `function` for OpenAI-style tools or `tool` for Anthropic-style tools."},"function":{"type":"object","properties":{"name":{"type":"string","description":"Name of the function to call."}}},"name":{"type":"string","description":"Tool name (when type is `tool`)."}}}],"description":"Controls which tool (if any) the model calls. Accepts a string (`auto`, `none`, `required`) or an object to force a specific tool."},"response_format":{"type":"object","description":"Structured output format. Supported by models that list `response_format` in `supported_parameters` (check GET /api/v1/models).","required":["type"],"properties":{"type":{"type":"string","enum":["text","json_object","json_schema"],"description":"Output format type: `text` (default), `json_object` (valid JSON object), or `json_schema` (schema-constrained JSON)."},"json_schema":{"type":"object","description":"JSON Schema specification (required when type is `json_schema`).","required":["name","schema"],"properties":{"name":{"type":"string","description":"Name of the schema."},"strict":{"type":"boolean","description":"If true, the model output must strictly match the schema with no extra fields.","default":false},"schema":{"type":"object","description":"JSON Schema Draft-07 object defining the output structure."}}}}},"cache":{"type":"object","description":"Semantic cache configuration for this request. Requires a `cache_key` HTTP header on the request. Controls how the conversation is indexed and matched for cache hits.","properties":{"expiration_time":{"type":"integer","description":"Cache entry TTL in seconds."},"conversation_mode":{"type":"string","enum":["full_conversation","last_message_only","last_n_turns"],"description":"Which part of the conversation to use for cache lookup. Default: `full_conversation`."},"last_n_turns":{"type":"integer","description":"Number of conversation turns to include when `conversation_mode` is `last_n_turns`."},"filter_on_provider":{"type":"boolean","default":false,"description":"If true, cache entries are scoped to the provider."},"filter_on_model":{"type":"boolean","default":true,"description":"If true, cache entries are scoped to the model."},"similarity_threshold":{"type":"number","minimum":0,"maximum":1,"description":"Minimum semantic similarity score (0.0–1.0) required for a cache hit."}}},"cache_control":{"type":"object","description":"Anthropic prompt caching control. **Not a top-level field in practice** — set this on individual text `content` blocks inside `messages` (see `messages[].content[]` schema). Use `{ \"type\": \"ephemeral\" }` to mark a block for caching on Anthropic models.","properties":{"type":{"type":"string","enum":["ephemeral"]}}},"aspectRatio":{"type":"string","description":"Image aspect ratio for Nano Banana (google/gemini-2.5-flash-image). Supported ratios: 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9","enum":["1:1","2:3","3:2","3:4","4:3","4:5","5:4","9:16","16:9","21:9"]},"prompt":{"type":"string","description":"Natural-language description for audio generation (ace-step/prompt-to-audio). Describe the audio, music, or ambient sound to generate."},"duration":{"type":"integer","minimum":1,"description":"Duration of the audio clip in seconds for text-to-audio generation (ace-step/prompt-to-audio). Optional parameter."},"reasoning":{"type":"object","description":"Control reasoning token behavior for supported models (OpenAI o-series, Grok, Gemini thinking, Anthropic). Reasoning tokens represent the model's internal reasoning process and improve output quality for complex tasks. Enabled by default. Use either 'effort' OR 'max_tokens', not both.","properties":{"effort":{"type":"string","enum":["low","medium","high"],"description":"Reasoning effort level (OpenAI o-series, Grok). Controls token allocation: low (~20% of max_tokens), medium (~50%), high (~80%). Do not use with max_tokens."},"max_tokens":{"type":"integer","minimum":1024,"maximum":32000,"description":"Maximum reasoning tokens (Gemini thinking, Anthropic). For Anthropic: minimum 1024, maximum 32000. max_tokens must be strictly greater than this value. Do not use with effort."},"exclude":{"type":"boolean","default":false,"description":"If true, model reasons internally but reasoning tokens are not returned in the response. Works across all models. Reduces costs while maintaining reasoning benefits."},"enabled":{"type":"boolean","default":true,"description":"Enable or disable reasoning tokens. Default is true for supported models."}}}}}}}},"responses":{"200":{"description":"Successful chat completion response","content":{"application/json":{"schema":{"type":"object","properties":{"id":{"type":"string","description":"Unique identifier for the completion"},"object":{"type":"string","description":"Object type, always 'chat.completion' or 'chat.completion.chunk' for streaming"},"model":{"type":"string","description":"The bare model name used for the completion (without the provider prefix). The routed provider is returned separately in `usage.provider`."},"service_tier":{"type":"string","description":"The service tier used to process the request (when applicable)."},"guardrails":{"type":"object","description":"Guardrail evaluation results attached by FastRouter, when guardrails are enabled.","nullable":true},"citations":{"type":"array","description":"Citations/sources returned by the model (e.g. for web-search-enabled models). `null` when there are none.","nullable":true,"items":{"type":"object"}},"choices":{"type":"array","description":"Array of completion choices","items":{"type":"object","properties":{"index":{"type":"integer","description":"Choice index"},"message":{"type":"object","description":"Generated message","properties":{"role":{"type":"string"},"content":{"type":"string","description":"Generated text content"},"annotations":{"type":"array","description":"Message annotations (e.g. citations/URLs). Empty array when there are none.","items":{"type":"object"}},"reasoning":{"type":"object","description":"Reasoning tokens (if enabled and model supports it). Contains the model's internal reasoning process.","properties":{"text":{"type":"string","description":"The reasoning text showing the model's thought process"}}}}},"finish_reason":{"type":"string","enum":["stop","length","tool_calls","content_filter"],"description":"Reason why the model stopped generating"}}}},"usage":{"type":"object","description":"Token usage statistics","properties":{"prompt_tokens":{"type":"integer","description":"Number of tokens in the prompt"},"completion_tokens":{"type":"integer","description":"Number of tokens in the completion (includes reasoning tokens if present)"},"total_tokens":{"type":"integer","description":"Total tokens used"},"completion_tokens_details":{"type":"object","description":"Breakdown of the completion tokens.","properties":{"reasoning_tokens":{"type":"integer","description":"Number of tokens spent on internal reasoning (for reasoning models)."}}},"prompt_tokens_details":{"type":"object","description":"Breakdown of the prompt tokens.","properties":{"cached_tokens":{"type":"integer","description":"Number of prompt tokens served from cache."}}},"object":{"type":"string","description":"Reserved usage object type field (often an empty string)."},"cost":{"type":"number","description":"Cost in USD for this request"},"chat_id":{"type":"string","description":"FastRouter internal identifier for this request/generation."},"provider":{"type":"string","description":"The upstream provider that served the request (e.g. `openai`, `anthropic`)."}}}}}}}},"400":{"description":"Bad Request - Invalid parameters"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Responses

## Create Response

> Creates responses with conversation state management. Supports previous\_response\_id for context continuity, function calling, web search, and file inputs. Compatible with OpenAI, Azure, and X-AI (Grok) providers.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Responses","description":"Alternative API for creating responses with built-in conversation state management."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/responses":{"post":{"operationId":"createResponse","tags":["Responses"],"summary":"Create Response","description":"Creates responses with conversation state management. Supports previous_response_id for context continuity, function calling, web search, and file inputs. Compatible with OpenAI, Azure, and X-AI (Grok) providers.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["model","input"],"properties":{"model":{"type":"string","description":"Model ID in provider/model format. Examples: openai/o3, openai/gpt-4.1, x-ai/grok-4"},"input":{"oneOf":[{"type":"string","description":"Simple text input"},{"type":"array","description":"Array of message objects with role and content","items":{"type":"object","properties":{"role":{"type":"string","enum":["user","assistant","system"],"description":"Role of the message"},"content":{"oneOf":[{"type":"string"},{"type":"array"}],"description":"Content as string or array of content blocks (text, files, etc.)"}}}}],"description":"User input - can be a simple string or array of message objects for multimodal/file inputs"},"previous_response_id":{"type":"string","description":"ID of a previous response to continue the conversation. Maintains context across requests."},"stream":{"type":"boolean","default":false,"description":"Enable streaming responses for real-time output"},"tools":{"type":"array","description":"Array of tool definitions. Supports custom functions and web_search tool.","items":{"type":"object","properties":{"type":{"type":"string","enum":["function","web_search"],"description":"Type of tool - 'function' for custom functions, 'web_search' for web search capability"},"name":{"type":"string","description":"Function name (for function tools)"},"description":{"type":"string","description":"Function description (for function tools)"},"parameters":{"type":"object","description":"Function parameters schema (for function tools)"},"filters":{"type":"object","description":"Filters for web search (for web_search tools)","properties":{"allowed_domains":{"type":"array","items":{"type":"string"},"description":"List of allowed domains for web search"}}}}}},"tool_choice":{"type":"string","enum":["auto","none"],"default":"auto","description":"Controls tool usage. 'auto' lets model decide, 'none' disables tools."},"temperature":{"type":"number","minimum":0,"maximum":2,"default":1,"description":"Sampling temperature for randomness control"},"max_tokens":{"type":"integer","description":"Maximum tokens to generate"},"background":{"type":"boolean","default":false,"description":"Run response generation in background"}}}}}},"responses":{"200":{"description":"Successful response","content":{"application/json":{"schema":{"type":"object","properties":{"id":{"type":"string","description":"Unique response ID (use for previous_response_id)"},"object":{"type":"string"},"created_at":{"type":"integer","description":"Unix timestamp"},"model":{"type":"string","description":"Model used"},"output":{"type":"array","description":"Array of output objects (reasoning, messages, etc.)","items":{"type":"object","properties":{"id":{"type":"string"},"type":{"type":"string","enum":["reasoning","message","function_call"]},"role":{"type":"string"},"status":{"type":"string"},"content":{"type":"array"}}}},"background":{"type":"boolean","description":"Whether request ran in background"},"error":{"type":"string","nullable":true,"description":"Error message if any"}}}}}},"400":{"description":"Bad Request - Invalid parameters"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Anthropic Messages Format

## Create Message (Anthropic Format)

> Creates a message using Anthropic's native Messages API format. Supports all Claude models through FastRouter with full feature parity including streaming, tool use, extended thinking, and vision.\
> \
> \*\*Available at both:\*\*\
> \- \`POST <https://api.fastrouter.ai/api/v1/messages\\`\\>
> \- \`POST <https://api.fastrouter.ai/v1/messages\\`\\>
> \
> \*\*Authentication:\*\* Use your FastRouter API key via either:\
> \- \`x-api-key: YOUR\_API\_KEY\` (Anthropic style)\
> \- \`Authorization: Bearer YOUR\_API\_KEY\` (OpenAI style)\
> \
> \*\*Streaming:\*\* Set \`stream: true\` to receive Server-Sent Events (SSE) with Anthropic's native streaming format (\`message\_start\`, \`content\_block\_start\`, \`content\_block\_delta\`, \`message\_delta\`, etc.).\
> \
> \*\*Cost tracking:\*\* The response \`usage\` object includes a \`cost\` field showing credits consumed.\
> \
> \*\*Drop-in replacement:\*\* This endpoint is a drop-in replacement for Anthropic's \`/v1/messages\` API. Simply change the base URL to \`<https://api.fastrouter.ai\\`> and use your FastRouter API key.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Anthropic Messages","description":"Anthropic-native Messages API. Send requests in Anthropic's native format with FastRouter authentication. Supports streaming, tool use, extended thinking, and all Claude model features."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"BadRequestError":{"description":"Bad Request - The request is malformed. This could be due to missing parameters, invalid formats, or routing errors.\n\n**Note:** the 400 body shape is not uniform. Validation errors return the structured object below, but routing/model errors may instead return a plain string: `{ \"error\": \"<message>\" }`. A few endpoints (e.g. some file operations) may return a plain-text body rather than JSON.","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Structured validation error.","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"}}}}},{"type":"object","description":"Plain-string routing/model error.","properties":{"error":{"type":"string"}}}]}}}},"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"InternalServerError":{"description":"Internal Error - Something went wrong on our side. Retry the request, and contact support if the issue persists.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/messages":{"post":{"operationId":"createAnthropicMessage","tags":["Anthropic Messages"],"summary":"Create Message (Anthropic Format)","description":"Creates a message using Anthropic's native Messages API format. Supports all Claude models through FastRouter with full feature parity including streaming, tool use, extended thinking, and vision.\n\n**Available at both:**\n- `POST https://api.fastrouter.ai/api/v1/messages`\n- `POST https://api.fastrouter.ai/v1/messages`\n\n**Authentication:** Use your FastRouter API key via either:\n- `x-api-key: YOUR_API_KEY` (Anthropic style)\n- `Authorization: Bearer YOUR_API_KEY` (OpenAI style)\n\n**Streaming:** Set `stream: true` to receive Server-Sent Events (SSE) with Anthropic's native streaming format (`message_start`, `content_block_start`, `content_block_delta`, `message_delta`, etc.).\n\n**Cost tracking:** The response `usage` object includes a `cost` field showing credits consumed.\n\n**Drop-in replacement:** This endpoint is a drop-in replacement for Anthropic's `/v1/messages` API. Simply change the base URL to `https://api.fastrouter.ai` and use your FastRouter API key.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["model","messages","max_tokens"],"properties":{"model":{"type":"string","description":"Model ID. Use `anthropic/` prefix or bare Anthropic model name. Examples: `anthropic/claude-sonnet-4-20250514`, `claude-sonnet-4-20250514`, `anthropic/claude-opus-4-20250514`"},"messages":{"type":"array","description":"Array of input messages. Each message has a role (user or assistant) and content.","minItems":1,"items":{"type":"object","required":["role","content"],"properties":{"role":{"type":"string","enum":["user","assistant"],"description":"The role of the message author. System messages are passed via the top-level system parameter."},"content":{"description":"Message content. Can be a string or an array of content blocks.","oneOf":[{"type":"string"},{"type":"array","items":{"type":"object","properties":{"type":{"type":"string","enum":["text","image","tool_use","tool_result"]},"text":{"type":"string"}}}}]}}}},"max_tokens":{"type":"integer","description":"The maximum number of tokens to generate. Required for all requests."},"system":{"type":"string","description":"System prompt. Provides instructions or context to the model."},"temperature":{"type":"number","minimum":0,"maximum":1,"description":"Randomness. Ranges from 0.0 to 1.0."},"top_p":{"type":"number","description":"Nucleus sampling parameter."},"top_k":{"type":"integer","description":"Only sample from the top K options for each subsequent token."},"stop_sequences":{"type":"array","items":{"type":"string"},"description":"Custom text sequences that will cause the model to stop generating."},"stream":{"type":"boolean","default":false,"description":"Whether to stream the response using SSE."},"tools":{"type":"array","description":"Definitions of tools that the model may use.","items":{"type":"object","properties":{"name":{"type":"string","description":"Name of the tool."},"description":{"type":"string","description":"Description of what the tool does."},"input_schema":{"type":"object","description":"JSON Schema for the tool input."}}}},"tool_choice":{"type":"object","description":"How the model should use the provided tools.","properties":{"type":{"type":"string","enum":["auto","any","tool"]},"name":{"type":"string","description":"Tool name (when type is tool)."}}},"thinking":{"type":"object","description":"Extended thinking configuration for Claude models.","properties":{"type":{"type":"string","enum":["enabled","disabled"]},"budget_tokens":{"type":"integer","description":"Max tokens for thinking."}}}}}}}},"responses":{"200":{"description":"Message created successfully","content":{"application/json":{"schema":{"type":"object","properties":{"id":{"type":"string","description":"Unique message identifier"},"type":{"type":"string","enum":["message"]},"role":{"type":"string","enum":["assistant"]},"content":{"type":"array","description":"Response content blocks (text, tool_use, thinking).","items":{"type":"object","properties":{"type":{"type":"string","enum":["text","tool_use","thinking"]},"text":{"type":"string"},"id":{"type":"string"},"name":{"type":"string"},"input":{"type":"object"}}}},"model":{"type":"string"},"stop_reason":{"type":"string","enum":["end_turn","max_tokens","stop_sequence","tool_use"]},"usage":{"type":"object","properties":{"input_tokens":{"type":"integer"},"output_tokens":{"type":"integer"},"cost":{"type":"number","description":"Credits consumed"}}}}}}}},"400":{"$ref":"#/components/responses/BadRequestError"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"$ref":"#/components/responses/InternalServerError"}}}}}}
```


# Gemini Native Format

## Generate Content (Gemini Format)

> Generates content using Google's native Gemini API format. This is a drop-in replacement for Google's \`generateContent\` endpoint — simply replace \`generativelanguage.googleapis.com\` with \`api.fastrouter.ai\` in your base URL and use your FastRouter API key.\
> \
> \*\*Full URL:\*\* \`POST <https://api.fastrouter.ai/v1/models/{model}:generateContent?key=YOUR\\_FASTROUTER\\_API\\_KEY\\`\\>
> \
> \*\*Authentication:\*\* Pass your FastRouter API key via the \`?key=YOUR\_API\_KEY\` query parameter (Google Gemini style).\
> \
> \*\*Request/Response format:\*\* Identical to Google's Gemini API. The request body uses \`contents\` (with \`parts\`) and \`generationConfig\`. The response includes \`candidates\` with generated content and \`usageMetadata\` with token counts.\
> \
> \*\*Cost tracking:\*\* The response \`usageMetadata\` object includes a \`cost\` field showing credits consumed.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Gemini Native API","description":"Google Gemini native API format. Send requests using Google's native generateContent format with FastRouter authentication."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"keyQuery":[]}],"components":{"securitySchemes":{"keyQuery":{"type":"apiKey","in":"query","name":"key","description":"FastRouter API Key (Gemini style). Get yours at https://fastrouter.ai\n\nFormat: `?key=YOUR_API_KEY`"}},"responses":{"BadRequestError":{"description":"Bad Request - The request is malformed. This could be due to missing parameters, invalid formats, or routing errors.\n\n**Note:** the 400 body shape is not uniform. Validation errors return the structured object below, but routing/model errors may instead return a plain string: `{ \"error\": \"<message>\" }`. A few endpoints (e.g. some file operations) may return a plain-text body rather than JSON.","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Structured validation error.","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"}}}}},{"type":"object","description":"Plain-string routing/model error.","properties":{"error":{"type":"string"}}}]}}}},"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"InsufficientCreditsError":{"description":"Insufficient Credits - Your account or API key has run out of credits. Add more credits and retry the request.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"InternalServerError":{"description":"Internal Error - Something went wrong on our side. Retry the request, and contact support if the issue persists.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/models/{model}:generateContent":{"post":{"operationId":"geminiGenerateContent","tags":["Gemini Native API"],"summary":"Generate Content (Gemini Format)","description":"Generates content using Google's native Gemini API format. This is a drop-in replacement for Google's `generateContent` endpoint — simply replace `generativelanguage.googleapis.com` with `api.fastrouter.ai` in your base URL and use your FastRouter API key.\n\n**Full URL:** `POST https://api.fastrouter.ai/v1/models/{model}:generateContent?key=YOUR_FASTROUTER_API_KEY`\n\n**Authentication:** Pass your FastRouter API key via the `?key=YOUR_API_KEY` query parameter (Google Gemini style).\n\n**Request/Response format:** Identical to Google's Gemini API. The request body uses `contents` (with `parts`) and `generationConfig`. The response includes `candidates` with generated content and `usageMetadata` with token counts.\n\n**Cost tracking:** The response `usageMetadata` object includes a `cost` field showing credits consumed.","parameters":[{"name":"model","in":"path","required":true,"schema":{"type":"string"},"description":"The Gemini model ID to use (e.g. gemini-2.5-pro, gemini-2.5-flash)."},{"name":"key","in":"query","required":true,"schema":{"type":"string"},"description":"FastRouter API key (Gemini style: pass as query parameter)."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["contents"],"properties":{"contents":{"type":"array","description":"The conversation content to send to the model.","items":{"type":"object","properties":{"role":{"type":"string","enum":["user","model"],"description":"The role of the content author."},"parts":{"type":"array","description":"The parts of the content.","items":{"type":"object","properties":{"text":{"type":"string","description":"Text content."}}}}}}},"generationConfig":{"type":"object","description":"Configuration options for content generation.","properties":{"temperature":{"type":"number","description":"Controls randomness. Range: 0.0 to 2.0."},"topP":{"type":"number","description":"Nucleus sampling parameter."},"topK":{"type":"integer","description":"Top-K sampling parameter."},"maxOutputTokens":{"type":"integer","description":"Maximum number of tokens to generate."},"stopSequences":{"type":"array","description":"Sequences that will stop generation.","items":{"type":"string"}},"candidateCount":{"type":"integer","description":"Number of response candidates to generate."},"responseMimeType":{"type":"string","description":"MIME type of the response (e.g. application/json for JSON mode)."}}},"safetySettings":{"type":"array","description":"Safety settings to control content filtering.","items":{"type":"object","properties":{"category":{"type":"string","description":"The safety category.","enum":["HARM_CATEGORY_HARASSMENT","HARM_CATEGORY_HATE_SPEECH","HARM_CATEGORY_SEXUALLY_EXPLICIT","HARM_CATEGORY_DANGEROUS_CONTENT"]},"threshold":{"type":"string","description":"The blocking threshold.","enum":["BLOCK_NONE","BLOCK_LOW_AND_ABOVE","BLOCK_MEDIUM_AND_ABOVE","BLOCK_ONLY_HIGH"]}}}},"systemInstruction":{"type":"object","description":"System instruction for the model.","properties":{"parts":{"type":"array","items":{"type":"object","properties":{"text":{"type":"string","description":"System instruction text."}}}}}}}}}}},"responses":{"200":{"description":"Successful response with generated content.","content":{"application/json":{"schema":{"type":"object","properties":{"candidates":{"type":"array","items":{"type":"object","properties":{"content":{"type":"object","properties":{"parts":{"type":"array","items":{"type":"object","properties":{"text":{"type":"string"}}}},"role":{"type":"string"}}},"finishReason":{"type":"string"},"safetyRatings":{"type":"array","items":{"type":"object","properties":{"category":{"type":"string"},"probability":{"type":"string"}}}}}}},"usageMetadata":{"type":"object","properties":{"promptTokenCount":{"type":"integer"},"candidatesTokenCount":{"type":"integer"},"totalTokenCount":{"type":"integer"},"cost":{"type":"number","description":"Credits consumed for this request."}}},"modelVersion":{"type":"string"}}}}}},"400":{"$ref":"#/components/responses/BadRequestError"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"402":{"$ref":"#/components/responses/InsufficientCreditsError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"$ref":"#/components/responses/InternalServerError"},"502":{"description":"Bad Gateway — failed to reach Google AI Studio.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"code":{"type":"integer"},"message":{"type":"string"},"status":{"type":"string"}}}}}}}}}}}}}
```

## Stream Generate Content (Gemini Format)

> Generates content using Google's native Gemini API format with streaming. Returns Server-Sent Events (SSE) with incremental content chunks. This is a drop-in replacement for Google's \`streamGenerateContent\` endpoint.\
> \
> \*\*Full URL:\*\* \`POST <https://api.fastrouter.ai/v1/models/{model}:streamGenerateContent?key=YOUR\\_FASTROUTER\\_API\\_KEY\\`\\>
> \
> \*\*Authentication:\*\* Pass your FastRouter API key via the \`?key=YOUR\_API\_KEY\` query parameter (Google Gemini style).\
> \
> \*\*Streaming format:\*\* Returns SSE events with \`data:\` prefixed JSON objects. Each chunk contains partial \`candidates\` with generated content. The final chunk includes \`usageMetadata\` with token counts and a \`cost\` field.\
> \
> \*\*Cost tracking:\*\* The final streamed chunk's \`usageMetadata\` object includes a \`cost\` field showing credits consumed.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Gemini Native API","description":"Google Gemini native API format. Send requests using Google's native generateContent format with FastRouter authentication."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"keyQuery":[]}],"components":{"securitySchemes":{"keyQuery":{"type":"apiKey","in":"query","name":"key","description":"FastRouter API Key (Gemini style). Get yours at https://fastrouter.ai\n\nFormat: `?key=YOUR_API_KEY`"}},"responses":{"BadRequestError":{"description":"Bad Request - The request is malformed. This could be due to missing parameters, invalid formats, or routing errors.\n\n**Note:** the 400 body shape is not uniform. Validation errors return the structured object below, but routing/model errors may instead return a plain string: `{ \"error\": \"<message>\" }`. A few endpoints (e.g. some file operations) may return a plain-text body rather than JSON.","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Structured validation error.","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"}}}}},{"type":"object","description":"Plain-string routing/model error.","properties":{"error":{"type":"string"}}}]}}}},"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"InsufficientCreditsError":{"description":"Insufficient Credits - Your account or API key has run out of credits. Add more credits and retry the request.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"InternalServerError":{"description":"Internal Error - Something went wrong on our side. Retry the request, and contact support if the issue persists.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/models/{model}:streamGenerateContent":{"post":{"operationId":"geminiStreamGenerateContent","tags":["Gemini Native API"],"summary":"Stream Generate Content (Gemini Format)","description":"Generates content using Google's native Gemini API format with streaming. Returns Server-Sent Events (SSE) with incremental content chunks. This is a drop-in replacement for Google's `streamGenerateContent` endpoint.\n\n**Full URL:** `POST https://api.fastrouter.ai/v1/models/{model}:streamGenerateContent?key=YOUR_FASTROUTER_API_KEY`\n\n**Authentication:** Pass your FastRouter API key via the `?key=YOUR_API_KEY` query parameter (Google Gemini style).\n\n**Streaming format:** Returns SSE events with `data:` prefixed JSON objects. Each chunk contains partial `candidates` with generated content. The final chunk includes `usageMetadata` with token counts and a `cost` field.\n\n**Cost tracking:** The final streamed chunk's `usageMetadata` object includes a `cost` field showing credits consumed.","parameters":[{"name":"model","in":"path","required":true,"schema":{"type":"string"},"description":"The Gemini model ID to use (e.g. gemini-2.5-pro, gemini-2.5-flash)."},{"name":"key","in":"query","required":true,"schema":{"type":"string"},"description":"FastRouter API key (Gemini style: pass as query parameter)."}],"requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["contents"],"properties":{"contents":{"type":"array","description":"The conversation content to send to the model.","items":{"type":"object","properties":{"role":{"type":"string","enum":["user","model"],"description":"The role of the content author."},"parts":{"type":"array","description":"The parts of the content.","items":{"type":"object","properties":{"text":{"type":"string","description":"Text content."}}}}}}},"generationConfig":{"type":"object","description":"Configuration options for content generation.","properties":{"temperature":{"type":"number","description":"Controls randomness. Range: 0.0 to 2.0."},"topP":{"type":"number","description":"Nucleus sampling parameter."},"topK":{"type":"integer","description":"Top-K sampling parameter."},"maxOutputTokens":{"type":"integer","description":"Maximum number of tokens to generate."},"stopSequences":{"type":"array","description":"Sequences that will stop generation.","items":{"type":"string"}}}},"safetySettings":{"type":"array","description":"Safety settings to control content filtering.","items":{"type":"object","properties":{"category":{"type":"string","enum":["HARM_CATEGORY_HARASSMENT","HARM_CATEGORY_HATE_SPEECH","HARM_CATEGORY_SEXUALLY_EXPLICIT","HARM_CATEGORY_DANGEROUS_CONTENT"]},"threshold":{"type":"string","enum":["BLOCK_NONE","BLOCK_LOW_AND_ABOVE","BLOCK_MEDIUM_AND_ABOVE","BLOCK_ONLY_HIGH"]}}}},"systemInstruction":{"type":"object","description":"System instruction for the model.","properties":{"parts":{"type":"array","items":{"type":"object","properties":{"text":{"type":"string","description":"System instruction text."}}}}}}}}}}},"responses":{"200":{"description":"Streaming response with Server-Sent Events containing generated content chunks.","content":{"text/event-stream":{"schema":{"type":"object","description":"Each SSE event contains a JSON object with partial candidates and usage metadata.","properties":{"candidates":{"type":"array","items":{"type":"object","properties":{"content":{"type":"object","properties":{"parts":{"type":"array","items":{"type":"object","properties":{"text":{"type":"string"}}}},"role":{"type":"string"}}},"finishReason":{"type":"string"}}}},"usageMetadata":{"type":"object","properties":{"promptTokenCount":{"type":"integer"},"candidatesTokenCount":{"type":"integer"},"totalTokenCount":{"type":"integer"},"cost":{"type":"number","description":"Credits consumed for this request."}}}}}}}},"400":{"$ref":"#/components/responses/BadRequestError"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"402":{"$ref":"#/components/responses/InsufficientCreditsError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"$ref":"#/components/responses/InternalServerError"},"502":{"description":"Bad Gateway — failed to reach Google AI Studio.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"code":{"type":"integer"},"message":{"type":"string"},"status":{"type":"string"}}}}}}}}}}}}}
```


# Embeddings

## Create Embeddings

> Creates vector embeddings for text input. Supports models from OpenAI, Google, and DeepInfra. Compatible with OpenAI SDK.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Embeddings","description":"Create vector embeddings for text using various embedding models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/embeddings":{"post":{"operationId":"createEmbedding","tags":["Embeddings"],"summary":"Create Embeddings","description":"Creates vector embeddings for text input. Supports models from OpenAI, Google, and DeepInfra. Compatible with OpenAI SDK.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["model","input"],"properties":{"model":{"type":"string","description":"The fully qualified model slug. Choose from supported models (openai/text-embedding-3-small, openai/text-embedding-3-large, openai/text-embedding-ada-002, google/gemini-embedding-001, deepinfra/intfloat-e5-base-v2)","enum":["openai/text-embedding-3-small","openai/text-embedding-3-large","openai/text-embedding-ada-002","google/gemini-embedding-001","deepinfra/intfloat-e5-base-v2"]},"input":{"oneOf":[{"type":"string","description":"Single text string to embed"},{"type":"array","items":{"type":"string"},"description":"Array of text strings to embed (batch processing)"}],"description":"The input text or list of text items to embed. Can be a single string or array of strings for batch processing."},"dimensions":{"type":"integer","minimum":1,"description":"Optional: Requested embedding size/dimensions. Supported only in text-embedding-3 series and newer models. Allows you to reduce embedding size for efficiency."},"encoding_format":{"type":"string","enum":["float","base64"],"default":"float","description":"Format for the embedding vector. 'float' returns array of numbers, 'base64' returns base64-encoded string."},"user":{"type":"string","description":"Optional: A unique identifier representing your end-user, for monitoring and abuse detection."}}}}}},"responses":{"200":{"description":"Embeddings created successfully","content":{"application/json":{"schema":{"type":"object","properties":{"object":{"type":"string","description":"Object type, always 'list'"},"data":{"type":"array","description":"Array of embedding objects","items":{"type":"object","properties":{"object":{"type":"string","description":"Object type, always 'embedding'"},"index":{"type":"integer","description":"Index of the embedding in the input array"},"embedding":{"type":"array","items":{"type":"number"},"description":"Vector representation of the input text"}}}},"model":{"type":"string","description":"Model used for embedding generation"},"usage":{"type":"object","description":"Token usage statistics","properties":{"prompt_tokens":{"type":"integer","description":"Number of tokens in the input"},"total_tokens":{"type":"integer","description":"Total tokens used (same as prompt_tokens for embeddings)"}}}}}}}},"400":{"description":"Bad Request - Invalid parameters or model"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Image


# Image Generation

## Generate Image

> Generates images from text prompts.\
> \
> FastRouter supports both synchronous and asynchronous image generation.\
> \
> By default, if \`response\_type\` is omitted, FastRouter prefers a synchronous provider whenever one is available. If no synchronous provider exists for the selected model, FastRouter automatically falls back to an asynchronous provider.\
> \
> Set \`response\_type\` to explicitly control execution:\
> \
> \- \`sync\` — Return the generated image directly.\
> \- \`async\` — Return a \`taskId\`. Retrieve the completed image using GET /api/v1/images/{task\_id} or POST /api/v1/getAsyncResponse.\
> \
> Supported models include OpenAI, Google, Leonardo AI, Black Forest Labs (Flux), ByteDance (Seedream), xAI (Grok), and others.\
> \
> \> \*\*Tip:\*\* Call \*\*GET /api/v1/models\*\* to retrieve the latest models and their supported parameters.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Images","description":"Generate and edit images using AI models like OpenAI GPT Image, Google Imagen/Gemini, Flux, Seedream, Grok, and Leonardo."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"schemas":{"ImageGenerationResponse":{"type":"object","description":"Synchronous image generation response.","properties":{"created":{"type":"integer","description":"Unix timestamp of when the image was created"},"data":{"type":"array","description":"Array of generated images","items":{"type":"object","properties":{"url":{"type":"string","format":"uri","description":"URL of the generated image (if response_format=url)"},"b64_json":{"type":"string","description":"Base64-encoded image data (if response_format=b64_json)"},"revised_prompt":{"type":"string","description":"The revised prompt used by the model, where supported."}}}}}}}},"paths":{"/api/v1/images/generations":{"post":{"operationId":"createImage","tags":["Images"],"summary":"Generate Image","description":"Generates images from text prompts.\n\nFastRouter supports both synchronous and asynchronous image generation.\n\nBy default, if `response_type` is omitted, FastRouter prefers a synchronous provider whenever one is available. If no synchronous provider exists for the selected model, FastRouter automatically falls back to an asynchronous provider.\n\nSet `response_type` to explicitly control execution:\n\n- `sync` — Return the generated image directly.\n- `async` — Return a `taskId`. Retrieve the completed image using GET /api/v1/images/{task_id} or POST /api/v1/getAsyncResponse.\n\nSupported models include OpenAI, Google, Leonardo AI, Black Forest Labs (Flux), ByteDance (Seedream), xAI (Grok), and others.\n\n> **Tip:** Call **GET /api/v1/models** to retrieve the latest models and their supported parameters.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["prompt","model"],"properties":{"prompt":{"type":"string","description":"Text description of the desired image"},"model":{"type":"string","enum":["openai/gpt-image-1","openai/gpt-image-1-mini","openai/gpt-image-1.5","openai/gpt-image-2","google/imagen-4.0","google/imagen-4.0-fast","google/imagen-4.0-ultra","google/gemini-2.5-flash-image","google/gemini-3-pro-image-preview","google/gemini-3.1-flash-image-preview","google/gemini-3.1-flash-lite-image","black-forest-labs/flux-dev","black-forest-labs/flux-kontext-pro","black-forest-labs/flux-pro-2.0","bytedance/seedream-4.0","bytedance/seedream-4.5","x-ai/grok-imagine-image","x-ai/grok-imagine-image-quality","leonardo-ai/phoenix","leonardo-ai/lucid-origin","leonardo-ai/lucid-realism"],"description":"Image generation model to use.\n\nModels may support synchronous or asynchronous execution depending on the available providers.\n\nIf `response_type` is omitted, FastRouter automatically selects the best available provider, preferring synchronous execution and falling back to asynchronous when necessary.\n\nCall GET /api/v1/models for the latest supported models."},"response_type":{"type":"string","enum":["sync","async"],"nullable":true,"description":"Controls how the image generation request is processed.\n\n- `sync` — Force a synchronous provider. The generated image is returned directly in the response.\n- `async` — Force an asynchronous provider. The response returns a `taskId`; retrieve the completed image using GET /api/v1/images/{task_id} or POST /api/v1/getAsyncResponse.\n- Omit this field to let FastRouter automatically select the best provider, preferring synchronous execution and falling back to asynchronous when necessary."},"n":{"type":"integer","minimum":1,"maximum":10,"default":1,"nullable":true,"description":"Number of images to generate. Supported range varies by model."},"size":{"type":"string","nullable":true,"default":"auto","description":"Image dimensions. Supported values vary by model, e.g. gpt-image-1/mini: auto, 1024×1024, 1536×1024, 1024×1536. Check GET /api/v1/models for per-model supported sizes."},"quality":{"type":"string","nullable":true,"default":"auto","enum":["auto","high","medium","low","hd","standard"],"description":"Image quality level (model-specific), e.g. gpt-image-1/mini: auto, high, medium, low. Check GET /api/v1/models for per-model supported values."},"output_format":{"type":"string","nullable":true,"default":"png","enum":["png","jpeg","webp"],"description":"Output image format. Only supported for gpt-image-1 and gpt-image-1-mini"},"background":{"type":"string","nullable":true,"default":"auto","enum":["transparent","opaque","auto"],"description":"Background style. Only for gpt-image-1 and gpt-image-1-mini. Note: transparent requires png or webp format"},"style":{"type":"string","enum":["vivid","natural"],"default":"vivid","description":"Image style, where supported by the model."},"response_format":{"type":"string","enum":["url","b64_json"],"default":"url","description":"Format of the response data"}}}}}},"responses":{"200":{"description":"Successful response.\n\nIf the request is processed synchronously, the generated image (URL or base64) is returned directly.\n\nIf the request is processed asynchronously, the response contains a `taskId`. Poll GET /api/v1/images/{task_id} or POST /api/v1/getAsyncResponse until the image is ready.","content":{"application/json":{"schema":{"oneOf":[{"$ref":"#/components/schemas/ImageGenerationResponse"},{"type":"object","description":"Asynchronous image generation response.","properties":{"taskId":{"type":"string","description":"Identifier for an asynchronous image generation request."}},"required":["taskId"]}]}}}},"400":{"description":"Bad Request - Invalid parameters"},"401":{"description":"Unauthorized - Invalid API key"},"429":{"description":"Rate Limit Exceeded"},"500":{"description":"Internal Server Error"}}}}}}
```

#### **Note:** Leonardo models (`leonardo/...`) respond asynchronously and return a `taskId`. Retrieve the completed image by polling the async response endpoint using either the **GET** or **POST** method.

## Get Image Status

> Retrieves the status and result of an asynchronous image generation by its \`task\_id\` (e.g. Leonardo models, which return a \`task\_id\` from POST /api/v1/images/generations).\
> \
> Poll this endpoint until the image is ready. When complete, \`data\[]\` contains the generated image URL(s) and \`fastrouter\_assets.status\` is \`ready\`.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Images","description":"Generate and edit images using AI models like OpenAI GPT Image, Google Imagen/Gemini, Flux, Seedream, Grok, and Leonardo."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/images/{task_id}":{"get":{"operationId":"getImageStatus","tags":["Images"],"summary":"Get Image Status","description":"Retrieves the status and result of an asynchronous image generation by its `task_id` (e.g. Leonardo models, which return a `task_id` from POST /api/v1/images/generations).\n\nPoll this endpoint until the image is ready. When complete, `data[]` contains the generated image URL(s) and `fastrouter_assets.status` is `ready`.","parameters":[{"name":"task_id","in":"path","required":true,"schema":{"type":"string"},"description":"The task ID returned from POST /api/v1/images/generations (e.g. a Leonardo `leo_` prefixed ID)."}],"responses":{"200":{"description":"Image generation status / result","content":{"application/json":{"schema":{"type":"object","properties":{"chat_id":{"type":"string","description":"FastRouter transaction ID for this generation."},"created":{"type":"integer","description":"Unix timestamp when the image was created."},"data":{"type":"array","description":"Array of generated images.","items":{"type":"object","properties":{"id":{"type":"string","description":"Provider-side image/generation ID."},"url":{"type":"string","format":"uri","description":"URL of the generated image."}}}},"fastrouter_assets":{"type":"object","description":"FastRouter-hosted copies of the generated assets.","properties":{"status":{"type":"string","description":"Asset availability status."},"urls":{"type":"array","items":{"type":"string","format":"uri"},"description":"FastRouter-hosted asset URLs."},"expires_at":{"type":"integer","description":"Unix timestamp when the hosted assets expire."},"cached_at":{"type":"integer","description":"Unix timestamp when the assets were cached."}}},"size":{"type":"string","description":"Image dimensions."},"usage":{"type":"object","description":"Credit and token usage for the generation.","properties":{"user_key_credits_used":{"type":"number"},"api_key_credits_used":{"type":"number"},"credits_used":{"type":"number"},"provider":{"type":"string"}}}}}}}},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Task not found or expired"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

## Get Async Response

> Polls for asynchronous video generation results. Poll with the \`taskId\` returned from POST /api/v1/videos until status is \`succeed\` or \`completed\`, then download from the provided URL.\
> \
> \*\*taskId and model rules:\*\*\
> \- If \`taskId\` includes a provider prefix (e.g. \`pol\_\` for Pollo), then \*\*model is optional\*\*.\
> \- If \`taskId\` has no provider prefix, then \*\*model is required\*\* and must match the model used in the original /videos request.\
> \
> \*\*Response shape varies by provider:\*\*\
> \- \*\*Pollo\*\* — per-generation \`id\`, \`createdDate\`, \`status\`, \`url\`, etc.\
> \- \*\*Kling\*\* — \`data.status: completed\`; generations have \`duration\` and \`url\`.\
> \- \*\*Veo (Google)\*\* — \`data.status: completed\`; generations have \`bytesBase64Encoded\`. Use \`fastrouter\_assets.urls\` for download.\
> \- \*\*Sora\*\* — OpenAI-style response with \`progress\`, \`status\`, etc.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Video","description":"Generate videos from text prompts or images using video generation models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/getAsyncResponse":{"post":{"operationId":"getAsyncResponse","tags":["Video"],"summary":"Get Async Response","description":"Polls for asynchronous video generation results. Poll with the `taskId` returned from POST /api/v1/videos until status is `succeed` or `completed`, then download from the provided URL.\n\n**taskId and model rules:**\n- If `taskId` includes a provider prefix (e.g. `pol_` for Pollo), then **model is optional**.\n- If `taskId` has no provider prefix, then **model is required** and must match the model used in the original /videos request.\n\n**Response shape varies by provider:**\n- **Pollo** — per-generation `id`, `createdDate`, `status`, `url`, etc.\n- **Kling** — `data.status: completed`; generations have `duration` and `url`.\n- **Veo (Google)** — `data.status: completed`; generations have `bytesBase64Encoded`. Use `fastrouter_assets.urls` for download.\n- **Sora** — OpenAI-style response with `progress`, `status`, etc.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["taskId"],"properties":{"taskId":{"type":"string","description":"The taskId returned from POST /api/v1/videos. When it includes a provider prefix (e.g. pol_ for Pollo), model can be omitted."},"model":{"type":"string","description":"The video model used in the original /videos request. Required when taskId has no provider prefix; optional when taskId has a provider prefix (e.g. pol_)."}}}}}},"responses":{"200":{"description":"Video generation status or completed video","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Pollo / Kling / Veo response (most video models)","properties":{"chat_id":{"type":"string"},"code":{"type":"string"},"message":{"type":"string"},"data":{"type":"object","properties":{"taskId":{"type":"string"},"status":{"type":"string","enum":["waiting","processing","succeed","completed","failed"],"description":"Overall status (Kling/Veo). Pollo uses per-generation status instead."},"generations":{"type":"array","items":{"type":"object","properties":{"id":{"type":"string","description":"Pollo"},"createdDate":{"type":"string","description":"Pollo"},"updatedDate":{"type":"string","description":"Pollo"},"status":{"type":"string","enum":["waiting","processing","succeed","failed"],"description":"Pollo"},"failMsg":{"type":"string","description":"Pollo"},"url":{"type":"string","format":"uri","description":"Pollo, Kling"},"mediaType":{"type":"string","description":"Pollo"},"duration":{"type":"integer","description":"Kling — duration in seconds"},"bytesBase64Encoded":{"type":"string","description":"Veo — base64 video bytes"}}}}}},"fastrouter_assets":{"type":"object","properties":{"status":{"type":"string"},"urls":{"type":"array","items":{"type":"string","format":"uri"}},"expires_at":{"type":"integer"},"cached_at":{"type":"integer"}}},"usage":{"type":"object"}}},{"type":"object","description":"Sora model response","properties":{"id":{"type":"string"},"chat_id":{"type":"string"},"object":{"type":"string"},"created_at":{"type":"integer"},"status":{"type":"string","enum":["queued","in_progress","completed","failed"]},"progress":{"type":"integer","description":"Completion percentage (0-100)"},"completed_at":{"type":"integer"},"model":{"type":"string"},"seconds":{"type":"string"},"size":{"type":"string"},"usage":{"type":"object"}}}]}}}},"400":{"description":"Bad Request - Invalid taskId or model"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - Invalid taskId or expired result"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Image Edit

## Edit Image

> Edits images using natural language instructions. Supports openai/gpt-image-1 model with PNG, WEBP, or JPG formats (max 50MB per image, up to 16 images). Requires multipart/form-data.\
> \
> \> \*\*Tip:\*\* Supported parameters and their accepted values differ from one model to another. Call \*\*GET /api/v1/models\*\* and inspect each model's \`supported\_parameters\` and \`supported\_params\_details\` to see exactly which fields it accepts and the allowed values, ranges, or enums.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Images","description":"Generate and edit images using AI models like OpenAI GPT Image, Google Imagen/Gemini, Flux, Seedream, Grok, and Leonardo."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/images/edits":{"post":{"operationId":"editImage","tags":["Images"],"summary":"Edit Image","description":"Edits images using natural language instructions. Supports openai/gpt-image-1 model with PNG, WEBP, or JPG formats (max 50MB per image, up to 16 images). Requires multipart/form-data.\n\n> **Tip:** Supported parameters and their accepted values differ from one model to another. Call **GET /api/v1/models** and inspect each model's `supported_parameters` and `supported_params_details` to see exactly which fields it accepts and the allowed values, ranges, or enums.","requestBody":{"required":true,"content":{"multipart/form-data":{"schema":{"type":"object","required":["image","prompt","model"],"properties":{"image":{"oneOf":[{"type":"string","format":"binary","description":"Single image file"},{"type":"array","items":{"type":"string","format":"binary"},"maxItems":16,"description":"Array of image files (up to 16)"}],"description":"The image(s) to edit. Must be PNG, WEBP, or JPG and smaller than 50MB each. Supports up to 16 images."},"prompt":{"type":"string","maxLength":32000,"description":"Text description of the desired modification. Max length: 32,000 characters for gpt-image-1."},"model":{"type":"string","enum":["openai/gpt-image-1"],"description":"Must be 'openai/gpt-image-1'","default":"openai/gpt-image-1"},"n":{"type":"integer","minimum":1,"maximum":10,"default":1,"description":"Number of edited images to generate (1-10)"},"size":{"type":"string","enum":["1024x1024","1536x1024","1024x1536","auto"],"default":"1024x1024","description":"Output image size. Options: 1024x1024, 1536x1024, 1024x1536, or auto"},"output_format":{"type":"string","enum":["png","jpeg","webp"],"default":"png","description":"Output image format: png, jpeg, or webp"},"output_compression":{"type":"integer","minimum":0,"maximum":100,"default":100,"description":"Compression level (0-100) for JPEG or WEBP outputs. Higher values mean better quality."},"mask":{"type":"string","format":"binary","description":"Optional PNG mask file (< 4MB). Transparent areas indicate regions to edit. Must match dimensions of the first input image."},"background":{"type":"string","enum":["transparent","opaque","auto"],"default":"auto","description":"Controls background transparency: transparent, opaque, or auto. If transparent, output format must be png or webp."},"response_format":{"type":"string","enum":["url","b64_json"],"default":"url","description":"Format of the response data"}}}}}},"responses":{"200":{"description":"Image edited successfully","content":{"application/json":{"schema":{"type":"object","properties":{"created":{"type":"integer","description":"Unix timestamp"},"data":{"type":"array","description":"Array of edited images","items":{"type":"object","properties":{"url":{"type":"string","format":"uri","description":"URL of the edited image (if response_format=url)"},"b64_json":{"type":"string","description":"Base64-encoded image data (if response_format=b64_json)"},"revised_prompt":{"type":"string","description":"The revised prompt used by the model"}}}}}}}}},"400":{"description":"Bad Request - Invalid parameters or image format"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"413":{"description":"Payload Too Large - Image exceeds 50MB limit"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Audio


# Text to Speech

## Generate Speech (Text-to-Speech)

> Generates audio from input text using text-to-speech (TTS) models. The response is a JSON object containing base64-encoded audio in the \`audios\` array (not a raw binary stream).\
> \
> Only \`model\` and \`input\` are required. All other parameters are \*\*model-specific\*\* — for example \`sarvam/bulbul:v2\` supports \`target\_language\_code\`, \`speaker\`, \`pitch\`, \`pace\`, \`loudness\`, \`speech\_sample\_rate\`, \`enable\_preprocessing\`, \`output\_audio\_codec\`, \`temperature\`, and \`enable\_cached\_responses\`.\
> \
> \> \*\*Tip:\*\* Supported parameters and their accepted values differ from one model to another. Call \*\*GET /api/v1/models\*\* and inspect each model's \`supported\_parameters\` and \`supported\_params\_details\` to see exactly which fields it accepts and the allowed values, ranges, or enums.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Audio","description":"Transcribe, translate, and generate audio using Whisper, ElevenLabs, and other audio models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"BadRequestError":{"description":"Bad Request - The request is malformed. This could be due to missing parameters, invalid formats, or routing errors.\n\n**Note:** the 400 body shape is not uniform. Validation errors return the structured object below, but routing/model errors may instead return a plain string: `{ \"error\": \"<message>\" }`. A few endpoints (e.g. some file operations) may return a plain-text body rather than JSON.","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Structured validation error.","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"}}}}},{"type":"object","description":"Plain-string routing/model error.","properties":{"error":{"type":"string"}}}]}}}},"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"InsufficientCreditsError":{"description":"Insufficient Credits - Your account or API key has run out of credits. Add more credits and retry the request.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/audio/speech":{"post":{"operationId":"createSpeech","tags":["Audio"],"summary":"Generate Speech (Text-to-Speech)","description":"Generates audio from input text using text-to-speech (TTS) models. The response is a JSON object containing base64-encoded audio in the `audios` array (not a raw binary stream).\n\nOnly `model` and `input` are required. All other parameters are **model-specific** — for example `sarvam/bulbul:v2` supports `target_language_code`, `speaker`, `pitch`, `pace`, `loudness`, `speech_sample_rate`, `enable_preprocessing`, `output_audio_codec`, `temperature`, and `enable_cached_responses`.\n\n> **Tip:** Supported parameters and their accepted values differ from one model to another. Call **GET /api/v1/models** and inspect each model's `supported_parameters` and `supported_params_details` to see exactly which fields it accepts and the allowed values, ranges, or enums.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["model","input"],"properties":{"model":{"type":"string","description":"TTS model in provider/model format."},"input":{"type":"string","description":"The text to convert to speech."},"target_language_code":{"type":"string","description":"Model-specific (e.g. Sarvam). BCP-47 language code of the output speech (e.g. `en-IN`, `hi-IN`)."},"speaker":{"type":"string","description":"Model-specific (e.g. Sarvam). The speaker voice to use."},"pitch":{"type":"number","description":"Model-specific (e.g. Sarvam). Adjusts the pitch of the generated speech."},"pace":{"type":"number","description":"Model-specific (e.g. Sarvam). Adjusts the speed of the speech (e.g. 0.5x to 2.0x)."},"loudness":{"type":"number","description":"Model-specific (e.g. Sarvam). Adjusts the loudness of the generated speech."},"speech_sample_rate":{"type":"integer","description":"Model-specific (e.g. Sarvam). Output audio sample rate in Hz."},"enable_preprocessing":{"type":"boolean","description":"Model-specific (e.g. Sarvam). Enables text normalization / preprocessing before synthesis."},"output_audio_codec":{"type":"string","description":"Model-specific (e.g. Sarvam). The audio codec/format of the response."},"temperature":{"type":"number","description":"Model-specific. Sampling temperature for generation."},"enable_cached_responses":{"type":"boolean","description":"Model-specific (e.g. Sarvam). Allows returning a cached response for identical requests."}}}}}},"responses":{"200":{"description":"The generated speech. Returns a JSON object containing the base64-encoded audio (not a raw binary stream).","content":{"application/json":{"schema":{"type":"object","properties":{"audios":{"type":"array","description":"Array of base64-encoded audio clips (WAV). Usually one entry.","items":{"type":"string","description":"Base64-encoded WAV audio."}},"model":{"type":"string","description":"The model used to generate the audio."},"provider":{"type":"string","description":"The upstream provider that served the request."},"usage":{"type":"object","description":"Usage and cost information for the request.","properties":{"prompt_tokens":{"type":"integer","description":"Number of input tokens."},"total_tokens":{"type":"integer","description":"Total tokens counted."},"cost":{"type":"number","description":"Cost in USD for this request."},"chat_id":{"type":"string","description":"FastRouter internal identifier for this request."}}}}}}}},"400":{"$ref":"#/components/responses/BadRequestError"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"402":{"$ref":"#/components/responses/InsufficientCreditsError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Audio to Text

## Transcribe Audio

> Transcribes audio to text in the original language using openai/whisper-1. Supports MP3, MP4, MPEG, M4A, WAV, WEBM formats (max 25MB). Output formats: json, text, srt, vtt, verbose\_json.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Audio","description":"Transcribe, translate, and generate audio using Whisper, ElevenLabs, and other audio models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/audio/transcriptions":{"post":{"operationId":"createTranscription","tags":["Audio"],"summary":"Transcribe Audio","description":"Transcribes audio to text in the original language using openai/whisper-1. Supports MP3, MP4, MPEG, M4A, WAV, WEBM formats (max 25MB). Output formats: json, text, srt, vtt, verbose_json.","requestBody":{"required":true,"content":{"multipart/form-data":{"schema":{"type":"object","required":["file","model"],"properties":{"file":{"type":"string","format":"binary","description":"Audio file to transcribe. Supported formats: mp3, mp4, mpeg, mpga, m4a, wav, webm. Max size: 25MB"},"model":{"type":"string","enum":["whisper-1","openai/whisper-1"],"description":"Model to use for transcription. Must be 'whisper-1' or 'openai/whisper-1'"},"language":{"type":"string","description":"Optional: ISO-639-1 language code of the audio (e.g., 'en', 'es', 'fr', 'de'). Improves accuracy and latency."},"prompt":{"type":"string","description":"Optional: Text prompt to guide the transcription style or continue a previous segment. Can include punctuation, casing, or specific vocabulary."},"response_format":{"type":"string","enum":["json","text","srt","verbose_json","vtt"],"default":"json","description":"Output format:\n- json: Basic JSON with text field\n- text: Plain text only\n- srt: SubRip subtitle format\n- vtt: WebVTT subtitle format\n- verbose_json: JSON with metadata and timestamps"},"temperature":{"type":"number","minimum":0,"maximum":1,"default":0,"description":"Sampling temperature (0-1). Lower values make output more focused and deterministic. Higher values increase randomness."},"timestamp_granularities[]":{"type":"array","items":{"type":"string","enum":["word","segment"]},"default":["segment"],"description":"Timestamp granularities to include in the transcription. **Requires `response_format=verbose_json`.** One or both of `word` and `segment` can be specified.\n\n- `segment` (default) — segment-level start/end timestamps under `segments[]`. No additional latency.\n- `word` — word-level start/end timestamps under `words[]`. Adds extra latency.\n\nPass this as a repeated form field (e.g. `-F 'timestamp_granularities[]=word' -F 'timestamp_granularities[]=segment'`)."}}}}}},"responses":{"200":{"description":"Transcription successful","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"JSON format response. In addition to `text`, FastRouter returns a `chat_id`, a `cost`, and a duration-based `usage` object.","properties":{"text":{"type":"string","description":"Transcribed text"},"chat_id":{"type":"string","description":"FastRouter internal identifier for this request."},"cost":{"type":"number","description":"Cost in USD for this request."},"usage":{"type":"object","description":"Duration-based usage for audio transcription.","properties":{"seconds":{"type":"integer","description":"Billed audio duration in seconds."},"type":{"type":"string","description":"Usage unit type."}}}}},{"type":"object","description":"Verbose JSON format response","properties":{"task":{"type":"string"},"language":{"type":"string"},"duration":{"type":"number"},"text":{"type":"string"},"segments":{"type":"array","description":"Segment-level timestamps. Included when `timestamp_granularities[]` contains `segment` (the default).","items":{"type":"object","properties":{"id":{"type":"integer"},"seek":{"type":"integer"},"start":{"type":"number"},"end":{"type":"number"},"text":{"type":"string"},"tokens":{"type":"array"},"temperature":{"type":"number"},"avg_logprob":{"type":"number"},"compression_ratio":{"type":"number"},"no_speech_prob":{"type":"number"}}}},"words":{"type":"array","description":"Word-level timestamps. Included only when `timestamp_granularities[]` contains `word`.","items":{"type":"object","properties":{"word":{"type":"string","description":"The transcribed word."},"start":{"type":"number","description":"Word start time in seconds."},"end":{"type":"number","description":"Word end time in seconds."}}}}}}]}},"text/plain":{"schema":{"type":"string","description":"Plain text transcription"}}}},"400":{"description":"Bad Request - Invalid file format or parameters"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"413":{"description":"Payload Too Large - File exceeds 25MB limit"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

## Translate Audio to English

> Translates audio to English text using openai/whisper-1, regardless of source language. Supports MP3, MP4, MPEG, M4A, WAV, WEBM formats (max 25MB). Output formats: json, text, srt, vtt, verbose\_json.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Audio","description":"Transcribe, translate, and generate audio using Whisper, ElevenLabs, and other audio models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/audio/translations":{"post":{"operationId":"createTranslation","tags":["Audio"],"summary":"Translate Audio to English","description":"Translates audio to English text using openai/whisper-1, regardless of source language. Supports MP3, MP4, MPEG, M4A, WAV, WEBM formats (max 25MB). Output formats: json, text, srt, vtt, verbose_json.","requestBody":{"required":true,"content":{"multipart/form-data":{"schema":{"type":"object","required":["file","model"],"properties":{"file":{"type":"string","format":"binary","description":"Audio file to translate to English. Supported formats: mp3, mp4, mpeg, mpga, m4a, wav, webm. Max size: 25MB"},"model":{"type":"string","enum":["whisper-1","openai/whisper-1"],"description":"Model to use for translation. Must be 'whisper-1' or 'openai/whisper-1'"},"prompt":{"type":"string","description":"Optional: English text prompt to guide the translation style. Can help with proper nouns, acronyms, or domain-specific vocabulary."},"response_format":{"type":"string","enum":["json","text","srt","verbose_json","vtt"],"default":"json","description":"Output format:\n- json: Basic JSON with translated English text\n- text: Plain English text only\n- srt: SubRip subtitle format (English)\n- vtt: WebVTT subtitle format (English)\n- verbose_json: JSON with metadata and timestamps"},"temperature":{"type":"number","minimum":0,"maximum":1,"default":0,"description":"Sampling temperature (0-1). Lower values (e.g., 0.1) make output more focused and deterministic. Use 0 for most consistent translations."}}}}}},"responses":{"200":{"description":"Translation successful - output is in English","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"JSON format response","properties":{"text":{"type":"string","description":"Translated English text"}}},{"type":"object","description":"Verbose JSON format response","properties":{"task":{"type":"string"},"language":{"type":"string","description":"Source language detected"},"duration":{"type":"number"},"text":{"type":"string","description":"Full translated English text"},"segments":{"type":"array","description":"Time-segmented translations","items":{"type":"object","properties":{"id":{"type":"integer"},"start":{"type":"number"},"end":{"type":"number"},"text":{"type":"string","description":"Segment translated to English"}}}}}}]}},"text/plain":{"schema":{"type":"string","description":"Plain English text translation"}}}},"400":{"description":"Bad Request - Invalid file format or parameters"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"413":{"description":"Payload Too Large - File exceeds 25MB limit"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Text to Audio

## Create Chat Completion

> Creates a chat completion for the provided messages. Supports streaming, function calling, vision, multimodal inputs, structured outputs (\`response\_format\`), tool selection (\`tool\_choice\`), semantic caching (\`cache\` + \`cache\_key\` header), Anthropic prompt caching (\`cache\_control\` on message content blocks), multi-model routing (\`models\`), request tagging (\`request\_tags\`), and reasoning tokens for supported models (OpenAI o-series, Grok, Gemini thinking, Anthropic). Compatible with OpenAI SDK.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Chat Completions","description":"Create AI-powered chat responses with support for text, images, audio, video, streaming, and tool calling."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/chat/completions":{"post":{"operationId":"createChatCompletion","tags":["Chat Completions"],"summary":"Create Chat Completion","description":"Creates a chat completion for the provided messages. Supports streaming, function calling, vision, multimodal inputs, structured outputs (`response_format`), tool selection (`tool_choice`), semantic caching (`cache` + `cache_key` header), Anthropic prompt caching (`cache_control` on message content blocks), multi-model routing (`models`), request tagging (`request_tags`), and reasoning tokens for supported models (OpenAI o-series, Grok, Gemini thinking, Anthropic). Compatible with OpenAI SDK.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["model","messages"],"properties":{"model":{"type":"string","description":"Model ID in format 'provider/model'. Examples: openai/gpt-5.1, google/gemini-3-pro-preview, anthropic/claude-4.5-sonnet"},"models":{"type":"array","items":{"type":"string"},"description":"Optional list of model IDs for multi-model routing. FastRouter can route across these models in order. Supports shortcut syntax (e.g. `openai/gpt-4o:thinking` to enable high reasoning effort). Can be used alongside a single `model`."},"request_tags":{"type":"array","items":{"type":"string"},"description":"Optional tags to attach to the request for logging, analytics, and filtering."},"messages":{"type":"array","description":"Array of message objects forming the conversation history. Each message has a role (system/user/assistant/tool) and content.","minItems":1,"items":{"type":"object","required":["role","content"],"properties":{"role":{"type":"string","enum":["system","user","assistant","tool"],"description":"Role of the message author:\n- system: Instructions for the AI\n- user: User messages\n- assistant: AI responses\n- tool: Tool/function outputs"},"content":{"oneOf":[{"type":"string","description":"Text content"},{"type":"array","description":"Multimodal content (text, images, audio, etc.). Text blocks on Anthropic models may include `cache_control` for prompt caching.","items":{"type":"object","properties":{"type":{"type":"string","description":"Content block type, e.g. `text`, `image_url`, `input_audio`, `file`."},"text":{"type":"string","description":"Text content (when type is `text`)."},"cache_control":{"type":"object","description":"Anthropic prompt caching control on this content block. Set on text blocks in `messages` to mark content for ephemeral caching.","properties":{"type":{"type":"string","enum":["ephemeral"],"description":"Cache type. Use `ephemeral` for Anthropic prompt caching."}}}}}}],"description":"Message content - can be a string for text-only or an array for multimodal inputs (text, images, audio, video)"},"name":{"type":"string","description":"Optional name of the message author"}}}},"temperature":{"type":"number","minimum":0,"maximum":2,"default":1,"description":"Controls randomness in responses. Lower values (0-0.7) make output more focused and deterministic. Higher values (0.8-2) make output more creative and random."},"max_tokens":{"type":"integer","minimum":1,"description":"Maximum number of tokens to generate in the completion. Limits the length of the response."},"top_p":{"type":"number","minimum":0,"maximum":1,"default":1,"description":"Nucleus sampling parameter. Alternative to temperature. Lower values make output more focused."},"frequency_penalty":{"type":"number","minimum":-2,"maximum":2,"default":0,"description":"Penalizes repeated tokens based on frequency. Positive values reduce repetition."},"presence_penalty":{"type":"number","minimum":-2,"maximum":2,"default":0,"description":"Penalizes tokens that have appeared. Positive values encourage new topics."},"stream":{"type":"boolean","default":false,"description":"Enable streaming responses for real-time output. When true, responses are sent as Server-Sent Events (SSE)."},"stop":{"oneOf":[{"type":"string"},{"type":"array","items":{"type":"string"}}],"description":"Stop sequences where the model will stop generating. Can be a string or array of strings."},"provider":{"type":"object","description":"Optional: Control provider routing behavior. If not specified, FastRouter intelligently selects the best provider based on availability, performance, and cost. Use either 'only' OR 'order', not both.","properties":{"only":{"type":"array","description":"Force routing to specific providers only. Request will only use providers in this list. Use when you need guaranteed provider selection.","items":{"type":"string"},"minItems":1},"order":{"type":"array","description":"Ordered list of providers to try in sequence. FastRouter attempts each provider in order. Use with allow_fallbacks for high-availability routing.","items":{"type":"string"},"minItems":1},"allow_fallbacks":{"type":"boolean","description":"When used with 'order', enables automatic fallback to the next provider in the list if the current provider is unavailable. Set to true for high-availability routing.","default":true}}},"tools":{"type":"array","description":"Array of tool/function definitions"},"tool_choice":{"oneOf":[{"type":"string","enum":["auto","none","required"],"description":"Controls tool usage: `auto` (model decides), `none` (disable tools), or `required` (force a tool call)."},{"type":"object","description":"Force a specific tool/function.","properties":{"type":{"type":"string","enum":["function","tool"],"description":"Selection mode. Use `function` for OpenAI-style tools or `tool` for Anthropic-style tools."},"function":{"type":"object","properties":{"name":{"type":"string","description":"Name of the function to call."}}},"name":{"type":"string","description":"Tool name (when type is `tool`)."}}}],"description":"Controls which tool (if any) the model calls. Accepts a string (`auto`, `none`, `required`) or an object to force a specific tool."},"response_format":{"type":"object","description":"Structured output format. Supported by models that list `response_format` in `supported_parameters` (check GET /api/v1/models).","required":["type"],"properties":{"type":{"type":"string","enum":["text","json_object","json_schema"],"description":"Output format type: `text` (default), `json_object` (valid JSON object), or `json_schema` (schema-constrained JSON)."},"json_schema":{"type":"object","description":"JSON Schema specification (required when type is `json_schema`).","required":["name","schema"],"properties":{"name":{"type":"string","description":"Name of the schema."},"strict":{"type":"boolean","description":"If true, the model output must strictly match the schema with no extra fields.","default":false},"schema":{"type":"object","description":"JSON Schema Draft-07 object defining the output structure."}}}}},"cache":{"type":"object","description":"Semantic cache configuration for this request. Requires a `cache_key` HTTP header on the request. Controls how the conversation is indexed and matched for cache hits.","properties":{"expiration_time":{"type":"integer","description":"Cache entry TTL in seconds."},"conversation_mode":{"type":"string","enum":["full_conversation","last_message_only","last_n_turns"],"description":"Which part of the conversation to use for cache lookup. Default: `full_conversation`."},"last_n_turns":{"type":"integer","description":"Number of conversation turns to include when `conversation_mode` is `last_n_turns`."},"filter_on_provider":{"type":"boolean","default":false,"description":"If true, cache entries are scoped to the provider."},"filter_on_model":{"type":"boolean","default":true,"description":"If true, cache entries are scoped to the model."},"similarity_threshold":{"type":"number","minimum":0,"maximum":1,"description":"Minimum semantic similarity score (0.0–1.0) required for a cache hit."}}},"cache_control":{"type":"object","description":"Anthropic prompt caching control. **Not a top-level field in practice** — set this on individual text `content` blocks inside `messages` (see `messages[].content[]` schema). Use `{ \"type\": \"ephemeral\" }` to mark a block for caching on Anthropic models.","properties":{"type":{"type":"string","enum":["ephemeral"]}}},"aspectRatio":{"type":"string","description":"Image aspect ratio for Nano Banana (google/gemini-2.5-flash-image). Supported ratios: 1:1, 2:3, 3:2, 3:4, 4:3, 4:5, 5:4, 9:16, 16:9, 21:9","enum":["1:1","2:3","3:2","3:4","4:3","4:5","5:4","9:16","16:9","21:9"]},"prompt":{"type":"string","description":"Natural-language description for audio generation (ace-step/prompt-to-audio). Describe the audio, music, or ambient sound to generate."},"duration":{"type":"integer","minimum":1,"description":"Duration of the audio clip in seconds for text-to-audio generation (ace-step/prompt-to-audio). Optional parameter."},"reasoning":{"type":"object","description":"Control reasoning token behavior for supported models (OpenAI o-series, Grok, Gemini thinking, Anthropic). Reasoning tokens represent the model's internal reasoning process and improve output quality for complex tasks. Enabled by default. Use either 'effort' OR 'max_tokens', not both.","properties":{"effort":{"type":"string","enum":["low","medium","high"],"description":"Reasoning effort level (OpenAI o-series, Grok). Controls token allocation: low (~20% of max_tokens), medium (~50%), high (~80%). Do not use with max_tokens."},"max_tokens":{"type":"integer","minimum":1024,"maximum":32000,"description":"Maximum reasoning tokens (Gemini thinking, Anthropic). For Anthropic: minimum 1024, maximum 32000. max_tokens must be strictly greater than this value. Do not use with effort."},"exclude":{"type":"boolean","default":false,"description":"If true, model reasons internally but reasoning tokens are not returned in the response. Works across all models. Reduces costs while maintaining reasoning benefits."},"enabled":{"type":"boolean","default":true,"description":"Enable or disable reasoning tokens. Default is true for supported models."}}}}}}}},"responses":{"200":{"description":"Successful chat completion response","content":{"application/json":{"schema":{"type":"object","properties":{"id":{"type":"string","description":"Unique identifier for the completion"},"object":{"type":"string","description":"Object type, always 'chat.completion' or 'chat.completion.chunk' for streaming"},"model":{"type":"string","description":"The bare model name used for the completion (without the provider prefix). The routed provider is returned separately in `usage.provider`."},"service_tier":{"type":"string","description":"The service tier used to process the request (when applicable)."},"guardrails":{"type":"object","description":"Guardrail evaluation results attached by FastRouter, when guardrails are enabled.","nullable":true},"citations":{"type":"array","description":"Citations/sources returned by the model (e.g. for web-search-enabled models). `null` when there are none.","nullable":true,"items":{"type":"object"}},"choices":{"type":"array","description":"Array of completion choices","items":{"type":"object","properties":{"index":{"type":"integer","description":"Choice index"},"message":{"type":"object","description":"Generated message","properties":{"role":{"type":"string"},"content":{"type":"string","description":"Generated text content"},"annotations":{"type":"array","description":"Message annotations (e.g. citations/URLs). Empty array when there are none.","items":{"type":"object"}},"reasoning":{"type":"object","description":"Reasoning tokens (if enabled and model supports it). Contains the model's internal reasoning process.","properties":{"text":{"type":"string","description":"The reasoning text showing the model's thought process"}}}}},"finish_reason":{"type":"string","enum":["stop","length","tool_calls","content_filter"],"description":"Reason why the model stopped generating"}}}},"usage":{"type":"object","description":"Token usage statistics","properties":{"prompt_tokens":{"type":"integer","description":"Number of tokens in the prompt"},"completion_tokens":{"type":"integer","description":"Number of tokens in the completion (includes reasoning tokens if present)"},"total_tokens":{"type":"integer","description":"Total tokens used"},"completion_tokens_details":{"type":"object","description":"Breakdown of the completion tokens.","properties":{"reasoning_tokens":{"type":"integer","description":"Number of tokens spent on internal reasoning (for reasoning models)."}}},"prompt_tokens_details":{"type":"object","description":"Breakdown of the prompt tokens.","properties":{"cached_tokens":{"type":"integer","description":"Number of prompt tokens served from cache."}}},"object":{"type":"string","description":"Reserved usage object type field (often an empty string)."},"cost":{"type":"number","description":"Cost in USD for this request"},"chat_id":{"type":"string","description":"FastRouter internal identifier for this request/generation."},"provider":{"type":"string","description":"The upstream provider that served the request (e.g. `openai`, `anthropic`)."}}}}}}}},"400":{"description":"Bad Request - Invalid parameters"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

## Retrieve Audio Generation Results

> Retrieves asynchronous audio generation results from ace-step/prompt-to-audio model.\
> \
> \*\*Two-Step Process:\*\*\
> \
> \*\*Step 1:\*\* Call \`/chat/completions\` with model \`ace-step/prompt-to-audio\` and your prompt\
> \
> \*\*Step 2:\*\* Take the \`response\_url\` from that response and poll this endpoint until audio is ready\
> \
> Poll with the response\_url until the audio generation is complete and download link is available.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Audio","description":"Transcribe, translate, and generate audio using Whisper, ElevenLabs, and other audio models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/getPromptToAudioResponse":{"post":{"operationId":"getPromptToAudioResponse","tags":["Audio"],"summary":"Retrieve Audio Generation Results","description":"Retrieves asynchronous audio generation results from ace-step/prompt-to-audio model.\n\n**Two-Step Process:**\n\n**Step 1:** Call `/chat/completions` with model `ace-step/prompt-to-audio` and your prompt\n\n**Step 2:** Take the `response_url` from that response and poll this endpoint until audio is ready\n\nPoll with the response_url until the audio generation is complete and download link is available.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["url","model"],"properties":{"url":{"type":"string","format":"uri","description":"The response_url returned from the initial /chat/completions request. This URL is used to poll for the audio generation status and retrieve the final result."},"model":{"type":"string","enum":["ace-step/prompt-to-audio"],"description":"Must be 'ace-step/prompt-to-audio'"}}}}}},"responses":{"200":{"description":"Audio generation status or completed audio","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"In-queue or in-progress response","properties":{"chat_id":{"type":"string","description":"FastRouter chat ID"},"model":{"type":"string"},"status":{"type":"string","enum":["IN_QUEUE","IN_PROGRESS","COMPLETED"],"description":"Current status of the audio generation"},"request_id":{"type":"string","description":"Unique request identifier"},"response_url":{"type":"string","format":"uri","description":"URL to fetch the result"},"status_url":{"type":"string","format":"uri","description":"URL to check status"},"cancel_url":{"type":"string","format":"uri","description":"URL to cancel the request"},"logs":{"type":"string","nullable":true,"description":"Generation logs"},"queue_position":{"type":"integer","description":"Position in queue (0 = processing)"},"usage":{"type":"object","properties":{"chat_id":{"type":"string"},"prompt_tokens":{"type":"integer"},"completion_tokens":{"type":"integer"},"total_tokens":{"type":"integer"},"user_key_credits_used":{"type":"number","description":"Credits charged for this generation"},"credits_used":{"type":"number"},"provider":{"type":"string"}}}}},{"type":"object","description":"Completed response with generated audio","properties":{"audio":{"type":"object","properties":{"url":{"type":"string","format":"uri","description":"Downloadable audio file URL (.wav)"},"content_type":{"type":"string","description":"MIME type of the audio file"},"file_name":{"type":"string","description":"Name of the generated audio file"},"file_size":{"type":"integer","description":"File size in bytes (may be 0 if unavailable)"}}},"seed":{"type":"integer","description":"Random seed used for generation"},"tags":{"type":"string","description":"Generated tags describing the audio"},"lyrics":{"type":"string","description":"Generated lyrics (if applicable)"}}}]}}}},"400":{"description":"Bad Request - Invalid URL or parameters"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - Invalid request ID or expired result"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Video

## Generate Video

> Generates videos from text prompts or images. Supports 18+ models from Google, OpenAI, Kling AI, Runway, Pollo, and others. Asynchronous generation — poll POST /api/v1/getAsyncResponse with taskId to retrieve results. The same request body (model, prompt, image, length, resolution) can also be sent to POST /api/v1/chat/completions when using a video model.\
> \
> \> \*\*Tip:\*\* Supported parameters and their accepted values differ from one model to another. Call \*\*GET /api/v1/models\*\* and inspect each model's \`supported\_parameters\` and \`supported\_params\_details\` to see exactly which fields it accepts and the allowed values, ranges, or enums.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Video","description":"Generate videos from text prompts or images using video generation models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/videos":{"post":{"operationId":"createVideo","tags":["Video"],"summary":"Generate Video","description":"Generates videos from text prompts or images. Supports 18+ models from Google, OpenAI, Kling AI, Runway, Pollo, and others. Asynchronous generation — poll POST /api/v1/getAsyncResponse with taskId to retrieve results. The same request body (model, prompt, image, length, resolution) can also be sent to POST /api/v1/chat/completions when using a video model.\n\n> **Tip:** Supported parameters and their accepted values differ from one model to another. Call **GET /api/v1/models** and inspect each model's `supported_parameters` and `supported_params_details` to see exactly which fields it accepts and the allowed values, ranges, or enums.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["model","prompt"],"properties":{"model":{"type":"string","description":"Video generation model in provider/model format. The supported model list changes over time — call GET /api/v1/models for the authoritative, up-to-date set.","enum":["google/veo2","google/veo3","google/veo3-fast","google/veo3.1","google/veo3.1-fast","google/veo3.1-lite","openai/sora-2","openai/sora-2-pro","kling-ai/kling-v1-6","kling-ai/kling-v2","kling-ai/kling-v2-1","kling-ai/kling-v2-1-master","kling-ai/kling-v3","kling-ai/kling-video-o1","runway/runway-gen-3-turbo","runway/runway-gen-4-turbo","pika/pika-v2-2","bytedance/seedance","bytedance/seedance-pro","bytedance/seedance-1.5-pro","bytedance/seedance-2","bytedance/seedance-2-fast","bytedance/seedance-2-mini","pollo/pollo-v1-6","vidu/vidu-v2-0","vidu/vidu-q1","wanx/wan-v2-6","x-ai/grok-imagine-video"]},"prompt":{"type":"string","description":"Natural language description of the video scene to generate"},"image":{"type":"string","format":"uri","description":"URL of an input image for image-to-video generation. Only URLs supported (no base64). Formats: JPG, PNG, JPEG. Aspect ratio must be 1:4 to 4:1. Required for some models."},"length":{"type":"integer","enum":[4,5,6,8,10,12],"description":"Duration of the video in seconds. Supported values vary by model: 4, 5, 6, 8, 10, or 12."},"seconds":{"type":"string","enum":["4","8","12"],"description":"Duration in seconds (used by Sora models). Supported: 4, 8, or 12."},"resolution":{"type":"string","enum":["480p","720p","1080p"],"description":"Output video resolution"},"aspectRatio":{"type":"string","enum":["16:9","9:16","4:3","3:4","1:1","5:3","3:5"],"description":"Width-to-height ratio of the video frame"},"size":{"type":"string","enum":["720x1280","1280x720","1024x1792","1792x1024"],"description":"Video dimensions (used by Sora models)"},"mode":{"type":"string","enum":["std","pro"],"description":"Generation style/mode (if supported by model)"},"seed":{"type":"integer","description":"Random seed for deterministic output (optional; useful for reproducibility)"},"generateAudio":{"type":"boolean","description":"Generate audio track for the video (supported by some models)"}}}}}},"responses":{"200":{"description":"Video generation task queued successfully","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Response format for most models (Veo, Kling, etc.)","properties":{"chat_id":{"type":"string"},"model":{"type":"string"},"code":{"type":"string"},"message":{"type":"string"},"data":{"type":"object","properties":{"taskId":{"type":"string","description":"Use this to poll for completion"},"status":{"type":"string","enum":["waiting","queued","processing"]}}},"usage":{"type":"object","properties":{"user_key_credits_used":{"type":"number"},"credits_used":{"type":"number"},"provider":{"type":"string"}}}}},{"type":"object","description":"Response format for Sora models","properties":{"id":{"type":"string","description":"Video ID - use this to poll for completion"},"chat_id":{"type":"string"},"object":{"type":"string"},"created_at":{"type":"integer"},"status":{"type":"string","enum":["queued","in_progress","completed"]},"model":{"type":"string"},"seconds":{"type":"string"},"size":{"type":"string"},"usage":{"type":"object"}}}]}}}},"400":{"description":"Bad Request - Invalid parameters or model"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

#### **Note:** FastRouter supports both GET (recommended) and POST endpoints for retrieving generated videos.

## Get Video Status

> Retrieves the status and result of an asynchronous video generation by its \`task\_id\` (returned from POST /api/v1/videos).\
> \
> Poll this endpoint until generation completes. Response shape varies by provider:\
> \- \*\*Pollo\*\* — per-generation fields (\`id\`, \`createdDate\`, \`updatedDate\`, \`status\`, \`url\`, \`mediaType\`).\
> \- \*\*Kling\*\* — top-level \`data.status\` is \`completed\`; each generation has \`duration\` and \`url\`.\
> \- \*\*Veo (Google)\*\* — top-level \`data.status\` is \`completed\`; each generation has \`bytesBase64Encoded\` (base64 video). Use \`fastrouter\_assets.urls\` for the hosted download URL.\
> \
> When complete, download from \`generations\[].url\` or \`fastrouter\_assets.urls\`.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Video","description":"Generate videos from text prompts or images using video generation models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/videos/{task_id}":{"get":{"operationId":"getVideoStatus","tags":["Video"],"summary":"Get Video Status","description":"Retrieves the status and result of an asynchronous video generation by its `task_id` (returned from POST /api/v1/videos).\n\nPoll this endpoint until generation completes. Response shape varies by provider:\n- **Pollo** — per-generation fields (`id`, `createdDate`, `updatedDate`, `status`, `url`, `mediaType`).\n- **Kling** — top-level `data.status` is `completed`; each generation has `duration` and `url`.\n- **Veo (Google)** — top-level `data.status` is `completed`; each generation has `bytesBase64Encoded` (base64 video). Use `fastrouter_assets.urls` for the hosted download URL.\n\nWhen complete, download from `generations[].url` or `fastrouter_assets.urls`.","parameters":[{"name":"task_id","in":"path","required":true,"schema":{"type":"string"},"description":"The task ID returned from POST /api/v1/videos."}],"responses":{"200":{"description":"Video generation status / result","content":{"application/json":{"schema":{"type":"object","properties":{"chat_id":{"type":"string","description":"FastRouter transaction ID for this generation."},"code":{"type":"string","description":"Response code."},"message":{"type":"string"},"data":{"type":"object","properties":{"taskId":{"type":"string","description":"The task ID for this video generation."},"status":{"type":"string","enum":["waiting","processing","succeed","completed","failed"],"description":"Overall generation status (present on Kling/Veo-style responses)."},"generations":{"type":"array","description":"Generation results. Field shape depends on the video provider.","items":{"type":"object","properties":{"id":{"type":"string","description":"Provider-side generation ID (Pollo)."},"createdDate":{"type":"string","description":"ISO timestamp when generation started (Pollo)."},"updatedDate":{"type":"string","description":"ISO timestamp when generation last updated (Pollo)."},"status":{"type":"string","enum":["waiting","processing","succeed","failed"],"description":"Per-generation status (Pollo)."},"failMsg":{"type":"string","description":"Failure message (Pollo; empty when successful)."},"url":{"type":"string","format":"uri","description":"Downloadable video URL (Pollo, Kling)."},"mediaType":{"type":"string","description":"Media type (Pollo)."},"duration":{"type":"integer","description":"Video duration in seconds (Kling)."},"bytesBase64Encoded":{"type":"string","description":"Base64-encoded video bytes (Veo/Google). May be empty when the video is served via `fastrouter_assets.urls` instead."}}}}}},"fastrouter_assets":{"type":"object","description":"FastRouter-hosted copies of the generated assets.","properties":{"status":{"type":"string","description":"Asset availability status."},"urls":{"type":"array","items":{"type":"string","format":"uri"},"description":"FastRouter-hosted asset URLs."},"expires_at":{"type":"integer","description":"Unix timestamp when the hosted assets expire."},"cached_at":{"type":"integer","description":"Unix timestamp when the assets were cached."}}},"usage":{"type":"object","description":"Credit usage for the generation.","properties":{"user_key_credits_used":{"type":"number"},"api_key_credits_used":{"type":"number"},"credits_used":{"type":"number"},"provider":{"type":"string"}}}}}}}},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Task not found or expired"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

## Get Async Response

> Polls for asynchronous video generation results. Poll with the \`taskId\` returned from POST /api/v1/videos until status is \`succeed\` or \`completed\`, then download from the provided URL.\
> \
> \*\*taskId and model rules:\*\*\
> \- If \`taskId\` includes a provider prefix (e.g. \`pol\_\` for Pollo), then \*\*model is optional\*\*.\
> \- If \`taskId\` has no provider prefix, then \*\*model is required\*\* and must match the model used in the original /videos request.\
> \
> \*\*Response shape varies by provider:\*\*\
> \- \*\*Pollo\*\* — per-generation \`id\`, \`createdDate\`, \`status\`, \`url\`, etc.\
> \- \*\*Kling\*\* — \`data.status: completed\`; generations have \`duration\` and \`url\`.\
> \- \*\*Veo (Google)\*\* — \`data.status: completed\`; generations have \`bytesBase64Encoded\`. Use \`fastrouter\_assets.urls\` for download.\
> \- \*\*Sora\*\* — OpenAI-style response with \`progress\`, \`status\`, etc.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Video","description":"Generate videos from text prompts or images using video generation models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/getAsyncResponse":{"post":{"operationId":"getAsyncResponse","tags":["Video"],"summary":"Get Async Response","description":"Polls for asynchronous video generation results. Poll with the `taskId` returned from POST /api/v1/videos until status is `succeed` or `completed`, then download from the provided URL.\n\n**taskId and model rules:**\n- If `taskId` includes a provider prefix (e.g. `pol_` for Pollo), then **model is optional**.\n- If `taskId` has no provider prefix, then **model is required** and must match the model used in the original /videos request.\n\n**Response shape varies by provider:**\n- **Pollo** — per-generation `id`, `createdDate`, `status`, `url`, etc.\n- **Kling** — `data.status: completed`; generations have `duration` and `url`.\n- **Veo (Google)** — `data.status: completed`; generations have `bytesBase64Encoded`. Use `fastrouter_assets.urls` for download.\n- **Sora** — OpenAI-style response with `progress`, `status`, etc.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["taskId"],"properties":{"taskId":{"type":"string","description":"The taskId returned from POST /api/v1/videos. When it includes a provider prefix (e.g. pol_ for Pollo), model can be omitted."},"model":{"type":"string","description":"The video model used in the original /videos request. Required when taskId has no provider prefix; optional when taskId has a provider prefix (e.g. pol_)."}}}}}},"responses":{"200":{"description":"Video generation status or completed video","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Pollo / Kling / Veo response (most video models)","properties":{"chat_id":{"type":"string"},"code":{"type":"string"},"message":{"type":"string"},"data":{"type":"object","properties":{"taskId":{"type":"string"},"status":{"type":"string","enum":["waiting","processing","succeed","completed","failed"],"description":"Overall status (Kling/Veo). Pollo uses per-generation status instead."},"generations":{"type":"array","items":{"type":"object","properties":{"id":{"type":"string","description":"Pollo"},"createdDate":{"type":"string","description":"Pollo"},"updatedDate":{"type":"string","description":"Pollo"},"status":{"type":"string","enum":["waiting","processing","succeed","failed"],"description":"Pollo"},"failMsg":{"type":"string","description":"Pollo"},"url":{"type":"string","format":"uri","description":"Pollo, Kling"},"mediaType":{"type":"string","description":"Pollo"},"duration":{"type":"integer","description":"Kling — duration in seconds"},"bytesBase64Encoded":{"type":"string","description":"Veo — base64 video bytes"}}}}}},"fastrouter_assets":{"type":"object","properties":{"status":{"type":"string"},"urls":{"type":"array","items":{"type":"string","format":"uri"}},"expires_at":{"type":"integer"},"cached_at":{"type":"integer"}}},"usage":{"type":"object"}}},{"type":"object","description":"Sora model response","properties":{"id":{"type":"string"},"chat_id":{"type":"string"},"object":{"type":"string"},"created_at":{"type":"integer"},"status":{"type":"string","enum":["queued","in_progress","completed","failed"]},"progress":{"type":"integer","description":"Completion percentage (0-100)"},"completed_at":{"type":"integer"},"model":{"type":"string"},"seconds":{"type":"string"},"size":{"type":"string"},"usage":{"type":"object"}}}]}}}},"400":{"description":"Bad Request - Invalid taskId or model"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - Invalid taskId or expired result"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Realtime

## Create Realtime Session (WebSocket)

> Establishes a WebSocket connection for real-time audio and text conversations using OpenAI's Realtime API models.\
> \
> \*\*Connection URL:\*\*\
> \`\`\`\
> wss\://api.fastrouter.ai/v1/realtime?model=MODEL\_ID\
> \`\`\`\
> \
> \*\*Authentication:\*\* Pass your FastRouter API key via \`Authorization: Bearer\` header during the WebSocket handshake.\
> \
> \*\*Required header:\*\* \`OpenAI-Beta: realtime=v1\`\
> \
> \*\*Supported models:\*\* \`openai/gpt-4o-realtime-preview-2024-12-17\` and other OpenAI realtime models.

````json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Realtime","description":"WebSocket-based real-time audio and text conversations using OpenAI's Realtime API models."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"BadRequestError":{"description":"Bad Request - The request is malformed. This could be due to missing parameters, invalid formats, or routing errors.\n\n**Note:** the 400 body shape is not uniform. Validation errors return the structured object below, but routing/model errors may instead return a plain string: `{ \"error\": \"<message>\" }`. A few endpoints (e.g. some file operations) may return a plain-text body rather than JSON.","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Structured validation error.","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"}}}}},{"type":"object","description":"Plain-string routing/model error.","properties":{"error":{"type":"string"}}}]}}}},"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}}}},"paths":{"/v1/realtime":{"get":{"operationId":"connectRealtime","tags":["Realtime"],"summary":"Create Realtime Session (WebSocket)","description":"Establishes a WebSocket connection for real-time audio and text conversations using OpenAI's Realtime API models.\n\n**Connection URL:**\n```\nwss://api.fastrouter.ai/v1/realtime?model=MODEL_ID\n```\n\n**Authentication:** Pass your FastRouter API key via `Authorization: Bearer` header during the WebSocket handshake.\n\n**Required header:** `OpenAI-Beta: realtime=v1`\n\n**Supported models:** `openai/gpt-4o-realtime-preview-2024-12-17` and other OpenAI realtime models.","parameters":[{"name":"model","in":"query","required":false,"schema":{"type":"string","default":"gpt-4o-realtime-preview-2024-12-17"},"description":"The realtime model to use."}],"responses":{"200":{"description":"WebSocket connection established. Communicate using OpenAI Realtime API event format."},"400":{"$ref":"#/components/responses/BadRequestError"},"401":{"$ref":"#/components/responses/UnauthorizedError"}}}}}}
````


# Moderations

## Content Moderation

> Classifies text for content policy violations using OpenAI's moderation models.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Moderation","description":"Check content for policy violations using AI moderation."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}}},"paths":{"/api/v1/moderations":{"post":{"operationId":"createModeration","tags":["Moderation"],"summary":"Content Moderation","description":"Classifies text for content policy violations using OpenAI's moderation models.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["input","model"],"properties":{"input":{"oneOf":[{"type":"string"},{"type":"array","items":{"type":"string"}}],"description":"Text to moderate"},"model":{"type":"string","description":"Moderation model in provider/model format."}}}}}},"responses":{"200":{"description":"Moderation complete","content":{"application/json":{"schema":{"type":"object","properties":{"id":{"type":"string"},"model":{"type":"string"},"results":{"type":"array","items":{"type":"object","properties":{"flagged":{"type":"boolean"},"categories":{"type":"object"},"category_scores":{"type":"object"}}}}}}}}}}}}}}
```


# List Models

## List Available Models

> Returns all available AI models along with their pricing, architecture, context length, supported parameters, and (where applicable) per-parameter allowed values. No authentication required.\
> \
> This is the source of truth for what each model accepts — use \`supported\_parameters\` and \`supported\_params\_details\` to programmatically discover the exact fields and accepted values for any model (chat, video, image, embedding, audio, etc.).

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Models","description":"List and retrieve available AI models with pricing and capability information."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[],"paths":{"/api/v1/models":{"get":{"operationId":"listModels","tags":["Models"],"summary":"List Available Models","description":"Returns all available AI models along with their pricing, architecture, context length, supported parameters, and (where applicable) per-parameter allowed values. No authentication required.\n\nThis is the source of truth for what each model accepts — use `supported_parameters` and `supported_params_details` to programmatically discover the exact fields and accepted values for any model (chat, video, image, embedding, audio, etc.).","parameters":[],"responses":{"200":{"description":"List of all available models","content":{"application/json":{"schema":{"type":"object","properties":{"data":{"type":"array","description":"Array of model objects","items":{"type":"object","properties":{"id":{"type":"string","description":"Model ID in provider/model format"},"name":{"type":"string","description":"Human-readable model name"},"description":{"type":"string","description":"Long-form description of the model"},"created":{"type":"integer","description":"Unix timestamp when the model was added"},"context_length":{"type":"integer","description":"Maximum context window size in tokens"},"architecture":{"type":"object","description":"Model architecture details","properties":{"modality":{"type":"string","description":"Combined modality string"},"input_modalities":{"type":"array","items":{"type":"string"},"description":"List of supported input modalities"},"output_modalities":{"type":"array","items":{"type":"string"},"description":"List of output modalities produced by the model"},"tokenizer":{"type":"string","description":"Tokenizer used by the model"}}},"pricing":{"type":"object","description":"Pricing details. All values are USD strings (per token unless otherwise noted). An empty string means the field is not applicable for this model.","properties":{"prompt":{"type":"string","description":"Cost per prompt token (USD)"},"completion":{"type":"string","description":"Cost per completion token (USD)"},"request":{"type":"string","description":"Cost per request"},"image":{"type":"string","description":"Cost per image"},"web_search":{"type":"string"},"citation":{"type":"string"},"reasoning":{"type":"string"},"duration":{"type":"string"},"internal_reasoning":{"type":"string"},"input_cache_read":{"type":"string"},"input_cache_write":{"type":"string"},"prompt_more_than_128k_input":{"type":"string"},"completion_more_than_128k_input":{"type":"string"},"prompt_more_than_200k_input":{"type":"string"},"completion_more_than_200k_input":{"type":"string"},"prompt_more_than_272k_input":{"type":"string"},"completion_more_than_272k_input":{"type":"string"},"web_search_per_1000_requests_low":{"type":"string"},"web_search_per_1000_requests_medium":{"type":"string"},"web_search_per_1000_requests_high":{"type":"string"},"web_search_per_1000_requests":{"type":"string"},"audio_output_per_minute":{"type":"string"},"audio_input":{"type":"string"},"audio_output":{"type":"string"},"cached_audio_input":{"type":"string"},"batchPrice":{"type":"object","description":"Per-token pricing for batch processing (if offered)"},"flex_service_tier_pricing":{"type":"object","description":"Flex service-tier pricing overrides (if offered)"},"videoCost":{"type":"array","description":"Per-length/resolution video pricing entries for video models. Null for non-video models."},"videoCostWithAudio":{"type":"array","description":"Per-length/resolution video pricing entries with audio for video models. Null when not applicable."},"imageCost":{"type":"array","description":"Per-size image pricing entries for image models. Null for non-image models."},"priceToShow":{"type":"object","description":"Pre-computed display prices used by the dashboard / docs"}}},"top_provider":{"type":"object","description":"Provider-level information for the model","properties":{"context_length":{"type":"integer"},"max_completion_tokens":{"type":"integer"},"is_moderated":{"type":"boolean"}}},"supported_parameters":{"type":"array","items":{"type":"string"},"description":"Names of parameters accepted by this model on its primary endpoint."},"supported_params_details":{"type":"object","description":"Per-parameter input format details (dataType, enum, min, max). Useful for validating requests before calling the endpoint. Only present for models that publish parameter schemas (typical for video / image / specialised models).","additionalProperties":{"type":"object","properties":{"inputFormat":{"type":"object","properties":{"dataType":{"type":"string","description":"Underlying data type (string, number, bool, ...)"},"enum":{"type":"array","description":"Allowed values (when restricted)"},"min":{"type":"number","description":"Minimum allowed value (numeric params)"},"max":{"type":"number","description":"Maximum allowed value (numeric params)"}}}}}},"models_extra_params":{"type":"object","description":"Extra model metadata","properties":{"category":{"type":"array","items":{"type":"string"},"description":"Optional category tags (e.g. Coding, Legal, Health). Can be null when no categories are assigned."}}},"is_active":{"type":"boolean","description":"Whether this model is currently active / available"},"creator":{"type":"string","description":"Original creator of the model"}}}}}}}}},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}},"components":{"responses":{"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}}}
```


# Models x Provider Breakdown

## Get Model with Provider Breakdown

> Returns the same model object as \*\*GET /api/v1/models\*\* for a single model \*\*plus\*\* a \`provider\_data\` array showing every provider that serves this model, with each provider's own \`provider\_model\_id\`, pricing, supported parameters (and their per-provider schemas via \`supported\_parameters\_v2\`), and provider-specific capabilities.\
> \
> Use this to figure out:\
> \- which providers serve a given model,\
> \- how a provider exposes the model (its \`provider\_model\_id\`, pricing, what parameters it accepts).

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Models","description":"List and retrieve available AI models with pricing and capability information."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[],"paths":{"/api/v1/modelProviderData":{"get":{"operationId":"getModelProviderData","tags":["Models"],"summary":"Get Model with Provider Breakdown","description":"Returns the same model object as **GET /api/v1/models** for a single model **plus** a `provider_data` array showing every provider that serves this model, with each provider's own `provider_model_id`, pricing, supported parameters (and their per-provider schemas via `supported_parameters_v2`), and provider-specific capabilities.\n\nUse this to figure out:\n- which providers serve a given model,\n- how a provider exposes the model (its `provider_model_id`, pricing, what parameters it accepts).","parameters":[{"name":"id","in":"query","required":true,"schema":{"type":"string"},"description":"The model ID in `provider/model` format."}],"responses":{"200":{"description":"Model details with per-provider breakdown","content":{"application/json":{"schema":{"type":"object","description":"Same shape as a single item from GET /api/v1/models, with an extra `provider_data` array.","properties":{"id":{"type":"string"},"name":{"type":"string"},"description":{"type":"string"},"created":{"type":"integer"},"context_length":{"type":"integer"},"architecture":{"type":"object","properties":{"modality":{"type":"string"},"input_modalities":{"type":"array","items":{"type":"string"}},"output_modalities":{"type":"array","items":{"type":"string"}},"tokenizer":{"type":"string"}}},"pricing":{"type":"object","description":"Aggregated/display pricing (see GET /api/v1/models for the full list of pricing fields)."},"top_provider":{"type":"object","properties":{"context_length":{"type":"integer"},"max_completion_tokens":{"type":"integer"},"is_moderated":{"type":"boolean"}}},"provider_data":{"type":"array","description":"One entry per provider that serves this model. Same item shape as GET /api/v1/providerModels.","items":{"$ref":"#/components/schemas/ProviderModelEntry"}},"supported_parameters":{"type":"array","items":{"type":"string"},"description":"Aggregated supported parameters across providers."},"models_extra_params":{"type":"object","properties":{"category":{"type":"array","items":{"type":"string"}}}},"is_active":{"type":"boolean"},"creator":{"type":"string"}}}}}},"400":{"$ref":"#/components/responses/BadRequestError"},"404":{"description":"Model not found"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{},"responses":{"BadRequestError":{"description":"Bad Request - The request is malformed. This could be due to missing parameters, invalid formats, or routing errors.\n\n**Note:** the 400 body shape is not uniform. Validation errors return the structured object below, but routing/model errors may instead return a plain string: `{ \"error\": \"<message>\" }`. A few endpoints (e.g. some file operations) may return a plain-text body rather than JSON.","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Structured validation error.","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"}}}}},{"type":"object","description":"Plain-string routing/model error.","properties":{"error":{"type":"string"}}}]}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}}}
```


# List Providers

## List Providers

> Returns the list of providers (e.g. OpenAI, Anthropic, Google AI Studio) available through FastRouter, with their machine-readable \`provider\_id\` and human-readable \`label\`.\
> \
> Use the returned \`provider\_id\` values with the \`provider.only\` / \`provider.order\` fields in chat/completions and other endpoints.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Models","description":"List and retrieve available AI models with pricing and capability information."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[],"paths":{"/api/v1/providers":{"get":{"operationId":"listProviders","tags":["Models"],"summary":"List Providers","description":"Returns the list of providers (e.g. OpenAI, Anthropic, Google AI Studio) available through FastRouter, with their machine-readable `provider_id` and human-readable `label`.\n\nUse the returned `provider_id` values with the `provider.only` / `provider.order` fields in chat/completions and other endpoints.","parameters":[],"responses":{"200":{"description":"List of providers","content":{"application/json":{"schema":{"type":"object","properties":{"data":{"type":"array","items":{"type":"object","properties":{"provider_id":{"type":"string"},"label":{"type":"string"}}}}}}}}},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}},"components":{"responses":{"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}}}
```


# Provider x Model Supported Parameters

## List Provider-Model Pairs

> Returns a flat array of every (provider, model) pair available on FastRouter. Each entry shows how a given provider exposes a model: the provider's \`provider\_model\_id\`, its own pricing, the parameters it accepts (with detailed input formats in \`supported\_parameters\_v2\`), and provider-specific capability flags in \`extra\_params\`.\
> \
> This is the most granular view — use it to filter by provider or to discover all the provider variants of a model. For a single model's full provider breakdown, use \*\*GET /api/v1/modelProviderData?id={modelId}\*\*.\
> \
> \> \*\*Tip:\*\* Inspect \`supported\_parameters\_v2\` on each entry to see exactly which parameters a provider accepts (with their data types, enums, and min/max ranges).

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Models","description":"List and retrieve available AI models with pricing and capability information."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[],"paths":{"/api/v1/providerModels":{"get":{"operationId":"listProviderModels","tags":["Models"],"summary":"List Provider-Model Pairs","description":"Returns a flat array of every (provider, model) pair available on FastRouter. Each entry shows how a given provider exposes a model: the provider's `provider_model_id`, its own pricing, the parameters it accepts (with detailed input formats in `supported_parameters_v2`), and provider-specific capability flags in `extra_params`.\n\nThis is the most granular view — use it to filter by provider or to discover all the provider variants of a model. For a single model's full provider breakdown, use **GET /api/v1/modelProviderData?id={modelId}**.\n\n> **Tip:** Inspect `supported_parameters_v2` on each entry to see exactly which parameters a provider accepts (with their data types, enums, and min/max ranges).","parameters":[],"responses":{"200":{"description":"Array of provider-model entries","content":{"application/json":{"schema":{"type":"array","items":{"$ref":"#/components/schemas/ProviderModelEntry"}}}}},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}},"components":{"schemas":{},"responses":{"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}}}
```


# Auto Router

## Get Model Selection

> Returns the optimal model recommendation for a given prompt without making the actual completion request. Uses FastRouter's intelligent routing engine.\
> \
> \*\*Available at both:\*\*\
> \- \`POST <https://api.fastrouter.ai/api/v1/model/selection\\`\\>
> \- \`POST <https://api.fastrouter.ai/v1/model/selection\\`\\>
> \
> Set model to \`fastrouter/auto\` for automatic selection.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Model Selection","description":"Get the optimal model recommendation for a given prompt without making the actual completion request. Uses FastRouter's intelligent routing engine."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"BadRequestError":{"description":"Bad Request - The request is malformed. This could be due to missing parameters, invalid formats, or routing errors.\n\n**Note:** the 400 body shape is not uniform. Validation errors return the structured object below, but routing/model errors may instead return a plain string: `{ \"error\": \"<message>\" }`. A few endpoints (e.g. some file operations) may return a plain-text body rather than JSON.","content":{"application/json":{"schema":{"oneOf":[{"type":"object","description":"Structured validation error.","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"}}}}},{"type":"object","description":"Plain-string routing/model error.","properties":{"error":{"type":"string"}}}]}}}},"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"InternalServerError":{"description":"Internal Error - Something went wrong on our side. Retry the request, and contact support if the issue persists.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/model/selection":{"post":{"operationId":"getModelSelection","tags":["Model Selection"],"summary":"Get Model Selection","description":"Returns the optimal model recommendation for a given prompt without making the actual completion request. Uses FastRouter's intelligent routing engine.\n\n**Available at both:**\n- `POST https://api.fastrouter.ai/api/v1/model/selection`\n- `POST https://api.fastrouter.ai/v1/model/selection`\n\nSet model to `fastrouter/auto` for automatic selection.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["model","messages"],"properties":{"model":{"type":"string","description":"Set to `fastrouter/auto` for auto selection, or provide a specific model ID."},"messages":{"type":"array","description":"Conversation messages. The routing engine analyzes these to determine the optimal model.","minItems":1,"items":{"type":"object","required":["role","content"],"properties":{"role":{"type":"string","enum":["system","user","assistant"]},"content":{"type":"string"}}}}}}}}},"responses":{"200":{"description":"Model selection result","content":{"application/json":{"schema":{"type":"object","properties":{"selected_model":{"type":"string","description":"The recommended model ID"},"assigned_tags":{"type":"array","items":{"type":"string"},"description":"Tags assigned to the prompt"},"top_candidates":{"type":"array","items":{"type":"string"},"description":"Top candidate models"},"selection_strategy":{"type":"string","enum":["auto","explicit"]},"error":{"type":"string","description":"Error message if selection encountered issues"}}}}}},"400":{"$ref":"#/components/responses/BadRequestError"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"500":{"$ref":"#/components/responses/InternalServerError"}}}}}}
```


# Batch Processing

## FastRouter Batch API

Batch processing allows you to send large volumes of requests efficiently at reduced costs. Upload a JSONL file containing your requests, trigger the batch, monitor its progress, and download the results — all through a simple four-step workflow.

> **Note:** Batch requests are processed asynchronously within a specified completion window, making them ideal for non-time-sensitive workloads such as bulk evaluations, data processing, and large-scale content generation.

***

### 1. File Upload (POST Request)

Upload your JSONL file to FastRouter to initiate the batch process. This returns a `file_id` for use in subsequent steps.

## Upload Batch File

> Uploads a JSONL file containing multiple batch requests. Each line must include custom\_id, provider, method, url, and body. Returns file\_id for use in /batches endpoint.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Batch Processing","description":"Process large volumes of requests asynchronously with 50% cost savings."},{"name":"Files","description":"Manage files for batch processing and other features."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/files":{"post":{"operationId":"uploadFile","tags":["Batch Processing","Files"],"summary":"Upload Batch File","description":"Uploads a JSONL file containing multiple batch requests. Each line must include custom_id, provider, method, url, and body. Returns file_id for use in /batches endpoint.","requestBody":{"required":true,"content":{"multipart/form-data":{"schema":{"type":"object","required":["file"],"properties":{"file":{"type":"string","format":"binary","description":"JSONL file containing batch requests. Each line must be a valid JSON object with custom_id, provider, method, url, and body fields."},"purpose":{"type":"string","enum":["batch"],"default":"batch","description":"File purpose (always 'batch' for batch processing)"}}}}}},"responses":{"200":{"description":"File uploaded successfully - use file_id in next step","content":{"application/json":{"schema":{"type":"object","properties":{"file_id":{"type":"string","description":"Unique file identifier - use this in batch creation"},"id":{"type":"string","description":"Same as file_id"},"filename":{"type":"string","description":"Original filename"},"num_rows":{"type":"integer","description":"Number of requests in the file"},"url":{"type":"string","format":"uri","description":"Storage URL for the uploaded file"},"object":{"type":"string"},"bytes":{"type":"integer","description":"File size in bytes"},"created_at":{"type":"integer","description":"Unix timestamp"},"purpose":{"type":"string"}}}}}},"400":{"description":"Bad Request - Invalid file format or content"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"413":{"description":"Payload Too Large - File exceeds size limit"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

### 2. Trigger Batch (POST Request)

Once the file is uploaded, trigger the batch processing by providing the `input_file_id`, `endpoint`, and `completion_window` (currently accepts only `"24h"` for a 24-hour processing window).

## Create Batch

> Creates a batch processing job with uploaded file\_id. Provides 50% cost savings for asynchronous processing. Supports /v1/chat/completions and /v1/embeddings endpoints. Completion window: 24h.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Batch Processing","description":"Process large volumes of requests asynchronously with 50% cost savings."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/batches":{"post":{"operationId":"createBatch","tags":["Batch Processing"],"summary":"Create Batch","description":"Creates a batch processing job with uploaded file_id. Provides 50% cost savings for asynchronous processing. Supports /v1/chat/completions and /v1/embeddings endpoints. Completion window: 24h.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["input_file_id","endpoint","completion_window"],"properties":{"input_file_id":{"type":"string","description":"The file_id returned from the file upload step. Must be a valid JSONL file with batch requests."},"endpoint":{"type":"string","enum":["/v1/chat/completions","/v1/embeddings"],"description":"API endpoint to process. Must match the request types in your JSONL file."},"completion_window":{"type":"string","enum":["24h"],"default":"24h","description":"Time window for batch completion. Currently only '24h' is supported."},"metadata":{"type":"object","description":"Optional metadata object for tracking or organization purposes"}}}}}},"responses":{"200":{"description":"Batch created successfully - save batch_id for status checks","content":{"application/json":{"schema":{"type":"object","properties":{"batch_id":{"type":"string","description":"Unique batch identifier - use this to check status"},"id":{"type":"string","description":"Same as batch_id"},"object":{"type":"string"},"endpoint":{"type":"string"},"status":{"type":"string","enum":["validating","in_progress","finalizing","completed","failed","cancelled","expired","validation_failed"],"description":"Current batch status"},"created_at":{"type":"integer","description":"Unix timestamp"},"input_file_id":{"type":"string","description":"Input file ID"},"completion_window":{"type":"string"}}}}}},"400":{"description":"Bad Request - Invalid file ID or parameters"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - File ID not found"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

### 3. Get Status (GET Request)

Check the status of your batch job using the `batch_id` returned from the trigger step. Statuses include `"in_progress"`, `"completed"`, `"failed"`, etc.

## Get Batch Status

> Retrieves batch processing status. Poll until status is \`completed\`, then use \`output\_file\_id\` to download results. Status values: \`validating\`, \`in\_progress\`, \`finalizing\`, \`completed\`, \`failed\`, \`cancelled\`, \`expired\`, \`validation\_failed\`.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Batch Processing","description":"Process large volumes of requests asynchronously with 50% cost savings."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/batches/{batch_id}":{"get":{"operationId":"retrieveBatch","tags":["Batch Processing"],"summary":"Get Batch Status","description":"Retrieves batch processing status. Poll until status is `completed`, then use `output_file_id` to download results. Status values: `validating`, `in_progress`, `finalizing`, `completed`, `failed`, `cancelled`, `expired`, `validation_failed`.","parameters":[{"name":"batch_id","in":"path","required":true,"schema":{"type":"string"},"description":"Batch ID returned from batch creation"}],"responses":{"200":{"description":"Batch status and details","content":{"application/json":{"schema":{"type":"object","properties":{"id":{"type":"string","description":"Batch ID"},"object":{"type":"string"},"endpoint":{"type":"string"},"model":{"type":"string","description":"Model used (if applicable)"},"errors":{"type":"object","nullable":true,"description":"Error information when the batch fails validation or processing. When `status` is `validation_failed`, contains `message` and `code: \"validation_failed\"`.","properties":{"message":{"type":"string","description":"Human-readable error message describing the validation failure."},"code":{"type":"string","enum":["validation_failed"],"description":"Error code. Set to `validation_failed` when the input file fails validation."}}},"input_file_id":{"type":"string","description":"Input file ID"},"completion_window":{"type":"string"},"status":{"type":"string","enum":["validating","in_progress","finalizing","completed","failed","cancelled","expired","validation_failed"],"description":"Current batch status. `validation_failed` means the uploaded input file failed validation before processing started."},"output_file_id":{"type":"string","nullable":true,"description":"Output file ID - use this with /files/{file_id}/content to download results (available when completed)"},"error_file_id":{"type":"string","nullable":true,"description":"Error file ID if some requests failed"},"created_at":{"type":"integer","description":"Batch creation timestamp"},"in_progress_at":{"type":"integer","description":"When batch started processing"},"expires_at":{"type":"integer","description":"Expiration timestamp"},"finalizing_at":{"type":"integer","description":"When batch entered finalizing state"},"completed_at":{"type":"integer","nullable":true,"description":"Completion timestamp (null if not completed)"},"failed_at":{"type":"integer","nullable":true,"description":"Failure timestamp (null if not failed)"},"expired_at":{"type":"integer","nullable":true},"cancelling_at":{"type":"integer","nullable":true},"cancelled_at":{"type":"integer","nullable":true},"request_counts":{"type":"object","description":"Request statistics","properties":{"total":{"type":"integer","description":"Total requests in batch"},"completed":{"type":"integer","description":"Successfully completed requests"},"failed":{"type":"integer","description":"Failed requests"}}},"usage":{"type":"object","description":"Token usage and cost information"},"metadata":{"type":"object","nullable":true,"description":"Custom metadata if provided"}}}}}},"400":{"description":"Bad Request - Invalid batch ID"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - Batch ID not found"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

### 4. Download File (GET Request)

Once the batch is completed (verify via the status endpoint), download the output JSONL file containing responses for each request.

## Download Batch Results

> Downloads the output JSONL file once batch processing is completed. Use the output\_file\_id from the batch status response. Each line contains the custom\_id, error (if any), and response with body and status\_code.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Batch Processing","description":"Process large volumes of requests asynchronously with 50% cost savings."},{"name":"Files","description":"Manage files for batch processing and other features."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/files/{file_id}/content":{"get":{"operationId":"getFileContent","tags":["Batch Processing","Files"],"summary":"Download Batch Results","description":"Downloads the output JSONL file once batch processing is completed. Use the output_file_id from the batch status response. Each line contains the custom_id, error (if any), and response with body and status_code.","parameters":[{"name":"file_id","in":"path","required":true,"schema":{"type":"string"},"description":"Output file ID from batch status response (e.g., batch-output-fr_batch-abc123abc123.jsonl)"}],"responses":{"200":{"description":"Batch results file in JSONL format. Each line contains a response for one request.","content":{"application/x-ndjson":{"schema":{"type":"string","description":"JSONL content with one result per line. Each line is a JSON object with custom_id, error (null if successful), and response object containing body and status_code."}}}},"400":{"description":"Bad Request - Invalid file ID format"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - File not found, not ready, or expired"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Generations

## Get Generation Details

> Retrieves detailed usage and performance metrics for a past generation request. Returns cost breakdown, model/provider info, token counts, latency, throughput, and BYOK status.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Generation Details","description":"Retrieve detailed information about API requests and responses."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.\n\nNote: the 401 error body uses `code`, `message`, `param`, and `type` (there is no `status` field), and `type` is `invalid_request_error`.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"param":{"type":"string","nullable":true},"code":{"type":"string"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/api/v1/generation":{"post":{"operationId":"getGenerationDetails","tags":["Generation Details"],"summary":"Get Generation Details","description":"Retrieves detailed usage and performance metrics for a past generation request. Returns cost breakdown, model/provider info, token counts, latency, throughput, and BYOK status.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["generation_id"],"properties":{"generation_id":{"type":"string","description":"The generation ID returned in the API response. Can be chat_id, request_id, or similar ID field from any FastRouter API endpoint response."}}}}}},"responses":{"200":{"description":"Generation details retrieved successfully","content":{"application/json":{"schema":{"type":"object","properties":{"id":{"type":"string","description":"Generation ID (same as requested)"},"model":{"type":"string","description":"Model used for this generation in provider/model format"},"provider_name":{"type":"string","description":"Provider that processed the request"},"api_key":{"type":"string","description":"Hashed API key used for the request (for auditing)"},"credits_used":{"type":"number","description":"Total credits consumed by this request"},"query_time":{"type":"string","format":"date-time","description":"ISO 8601 timestamp of when the request was made"},"avg_response_time_minutes_per_transaction":{"type":"number","description":"Average response time in minutes per transaction"},"avg_tokens_per_nano":{"type":"number","description":"Average tokens processed per nanosecond (throughput metric)"},"input_token_size":{"type":"integer","description":"Number of tokens in the input/prompt"},"output_token_size":{"type":"integer","description":"Number of tokens in the generated output/completion"},"is_byok":{"type":"boolean","description":"Whether this request used Bring Your Own Key (BYOK). True if customer's own API key was used."},"endpoint":{"type":"string","description":"API endpoint used for the request"},"created_at":{"type":"integer","description":"Unix timestamp of request creation"},"request":{"type":"object","description":"Full request body that was sent (if available)"},"response":{"type":"object","description":"Full response body that was returned (if available)"},"usage":{"type":"object","description":"Detailed token usage information","properties":{"prompt_tokens":{"type":"integer","description":"Tokens in the prompt"},"completion_tokens":{"type":"integer","description":"Tokens in the completion"},"total_tokens":{"type":"integer","description":"Total tokens used"},"cost":{"type":"number","description":"Cost in USD"}}},"timing":{"type":"object","description":"Timing information","properties":{"started_at":{"type":"integer","description":"Unix timestamp when processing started"},"completed_at":{"type":"integer","description":"Unix timestamp when processing completed"},"duration_ms":{"type":"integer","description":"Total duration in milliseconds"}}},"metadata":{"type":"object","description":"Additional metadata about the request"}}}}}},"400":{"description":"Bad Request - Invalid generation ID format"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - Generation ID not found or expired"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


# Error Codes

### Error Code Overview

FastRouter uses standard HTTP status codes to indicate errors during API requests. Below is a list of common errors you might encounter, along with tips to resolve them.

| Code | Meaning                           | Description                                                                                                |
| ---- | --------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| 400  | **Bad Request**                   | The request is malformed. This could be due to missing parameters, invalid formats, or CORS issues.        |
| 401  | **Invalid Credentials**           | Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.              |
| 402  | **Insufficient Credits**          | Your account or API key has run out of credits. Add more credits and retry the request.                    |
| 403  | **Moderation Blocked**            | The input was flagged by the model’s content moderation system. Modify the prompt and try again.           |
| 429  | **Rate Limited**                  | You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.                        |
| 500  | **Internal Error**                | Something went wrong on our side. Retry the request, and contact support if the issue persists.            |
| 502  | **Model Down / Invalid Response** | The selected model is currently unavailable or returned an invalid response. Retry or use fallback models. |


# Changelog

New updates and improvements

{% updates format="full" %}
{% update date="2026-07-01" %}

## Added

**Model Playground: Image & Video** — Added dedicated Image and Video Playgrounds to experiment with multimodal models directly from the browser. Generate images and videos, compare model outputs, and iterate without writing any code.
{% endupdate %}

{% update date="2026-06-25" %}

## Improved

**Prompt Library** — Optimize any saved prompt version directly from Prompt Library. Prompt Optimizations now creates a new version automatically, preserving the original while making it easy to review and promote improvements.

**Prompt Comparison with Samples** — Compare the original and optimized prompt side-by-side using sample inputs before promoting a new version. Quickly validate improvements and understand how prompt changes affect model outputs.

<https://docs.fastrouter.ai/prompt-library>
{% endupdate %}

{% update date="2026-06-18" %}

## Added

**BytePlus Provider** — Added support for BytePlus-hosted models. FastRouter automatically handles BytePlus' custom pricing flow, including providers that return pricing information asynchronously, ensuring accurate cost tracking and billing.

**New Video & Multimodal Models** — Added support for the latest image, video, and reasoning models, including x-ai/grok-imagine-video, GLM 5.2, Kimi Code 2.7, and Minimax M3.

<https://fastrouter.ai/models?order=newest><br>
{% endupdate %}

{% update date="2026-06-11" %}

## Added

**MCP Server Templates** — Added pre-configured templates for popular MCP servers, eliminating the need to manually enter server configuration values. Connect common tools in just a few clicks while retaining the flexibility to customize settings when needed.

<https://docs.fastrouter.ai/mcp-gateway>
{% endupdate %}

{% update date="2026-06-04" %}

## Added

**Prompt Library** — Write, store, version, and optimize prompts in one place and reference them by ID in API calls, so prompt changes ship without a code deploy. Mark any version as **Production** to serve it to all live requests, and roll back instantly by promoting an earlier version. Optimize — Refine a stored prompt with Prompt Optimizations and save the result as a tracked, optimized version, with **Compare** to diff versions before promoting. Variables — Insert `{{curly braces}}` placeholders in a prompt and fill them per request via the `variables` field.

<https://docs.fastrouter.ai/prompt-library><br>
{% endupdate %}

{% update date="2026-05-28" %}

## Added

**Free Models** (`:free`) — Append `:free` to a supported model ID (e.g. `sarvam/sarvam-105b:free`) to route requests at no cost, with the suffix stripped transparently before reaching the provider. Available to all orgs regardless of billing status. Per-model daily quota — 10 requests per org per day, tracked independently per model and reset daily at UTC midnight; paid orgs consume free quota rather than billing credits.

<https://docs.fastrouter.ai/explore-features/free-models-free>
{% endupdate %}

{% update date="2026-05-21" %}

## Added

**Support for non-Claude models via Anthropic Messages format** — Route Claude Code requests to OpenAI, DeepSeek, and other FastRouter-supported providers using the same Anthropic-compatible interface\
**Universal model access in Claude Code** — Launch Claude Code with any FastRouter-supported model using the `--model` flag, without changing tooling or workflows

<https://docs.fastrouter.ai/integrations/claude-code><br>
{% endupdate %}

{% update date="2026-05-14" %}

## Added

**Bring Your Own Keys (BYOK) for external providers** — Attach your own API credentials from supported LLM providers directly to FastRouter while preserving your negotiated pricing\
**Custom model provisioning** — Register fine-tuned or privately hosted models with custom endpoints, pricing metadata, and API compatibility mappings\
**Advanced endpoint configuration** — Override provider base URLs with OpenAI-, Anthropic-, or Gemini-compatible formats, plus support for custom authentication headers\
**Granular model enablement** — Enable or disable individual catalog models per integration and map custom models to provider-specific endpoints\
**Integrated routing visibility** — Reference integrations via Provider Slug across Virtual Models, Gateway Configs, and Activity Logs for full routing traceability\
<https://docs.fastrouter.ai/add-external-keys-byok><br>
{% endupdate %}

{% update date="2026-05-07" %}

## Added

**Video Evaluations for AI-generated content** — Automatically assess video outputs at scale using LLM-based judges, with scoring across motion fidelity, audio-visual sync, cinematic quality, and prompt adherence

**Seamless log-based dataset creation** — Import video generation logs directly from FastRouter activity with filtering, sampling, and zero manual uploads

**Unified evaluation infrastructure** — Use the same Custom Evaluations setup as text and image evals, including shared judge configuration, scoring rubrics, and dashboards

**Multimodal LLM judging** — Leverage capable video-aware models to evaluate outputs with structured reasoning across multiple quality dimensions

**Deep-dive result analysis** — Access per-video judge reasoning, aggregated performance metrics, and cost/latency insights in a single view

<https://docs.fastrouter.ai/video-evaluations><br>
{% endupdate %}

{% update date="2026-04-30" %}

## Added

**Flex Pricing for Vertex AI and Google AI Studio models** — Access supported models at up to **50% lower cost** by using provider Flex inference tiers, ideal for batch jobs, background workloads, and latency-tolerant applications

**Zero code-change activation** — Append `:flex` to any supported model ID (for example `google/gemini-3.1-pro-preview:flex`) while keeping the same API key, endpoint, and request payload

**Provider-native Flex routing** — FastRouter automatically routes requests to the provider’s discounted Flex tier, with support for provider pinning to ensure correct execution paths

**Built for async and cost-sensitive workloads** — Recommended for summarisation pipelines, data extraction, classification, eval runs, scheduled jobs, and large-scale preprocessing tasks where response speed is less critical

**Model Catalog Flex visibility** — View supported Flex-enabled models and pricing directly in the model catalog, with per-model availability across providers

<https://docs.fastrouter.ai/flex-pricing>
{% endupdate %}

{% update date="2026-04-23" %}

## Added

**Flex Pricing for OpenAI models** — Access supported models at up to **50% lower cost** by using provider Flex inference tiers, ideal for batch jobs, background workloads, and latency-tolerant applications

**Zero code-change activation** — Append `:flex` to any supported model ID (for example `openai/gpt-5.4-nano:flex`) while keeping the same API key, endpoint, and request payload

**Provider-native Flex routing** — FastRouter automatically routes requests to the provider’s discounted Flex tier, with support for provider pinning to ensure correct execution paths

**Built for async and cost-sensitive workloads** — Recommended for summarisation pipelines, data extraction, classification, eval runs, scheduled jobs, and large-scale preprocessing tasks where response speed is less critical

**Model Catalog Flex visibility** — View supported Flex-enabled models and pricing directly in the model catalog, with per-model availability across providers

<https://docs.fastrouter.ai/flex-pricing>
{% endupdate %}

{% update date="2026-04-13" %}

## Added

**Prompt Optimizations (GEPA-powered)** — Automatically improve system prompts using FastRouter’s Genetic-Pareto optimization engine with iterative reflection, mutation, and scoring

**Run prompt experiments from your own data** — Import datasets from files or Activity Logs, evaluate against custom metrics, and compare optimized prompts against baseline performance

**LLM-as-a-Judge evaluations** — Score prompts across metrics like Accuracy, Helpfulness, Safety, Completeness, or your own custom criteria using a shared evaluator model

**Optimization Insights** — Review improvement %, final scores, accepted iterations, and the full optimized prompt in a dedicated results view

<https://docs.fastrouter.ai/prompt-optimizations>
{% endupdate %}

{% update date="2026-04-03" %}

## Added

**MCP Gateway** — Register any MCP-compatible server (GitHub, Linear, Gmail, or your own APIs) and expose its tools to any model routed through FastRouter, with centralized credential management, project-level scoping, and selective tool exposure

**OAuth 2.0 & Static Header authentication for MCP servers** — Securely store and inject credentials server-side across all tool calls, with support for No Auth, Static Header, and full OAuth 2.0 authorization code flow

**Auto-execution mode** — Set `auto_execute_tools: true` to let FastRouter handle the complete tool-call loop and return a final text response directly, with a configurable `max_tool_rounds` cap (maximum 5)

<https://docs.fastrouter.ai/mcp-gateway>
{% endupdate %}
{% endupdates %}

{% updates format="full" %}
{% update date="2026-03-27" %}

## Added

**Priority Routing** — Route requests through models in a fixed priority order, with automatic sequential fallback for deterministic, predictable routing

**Category-Based Routing** — Direct requests to different model groups based on detected prompt category, with per-category sub-strategies and a configurable default fallback

<https://docs.fastrouter.ai/explore-features/virtual-model-aliases>
{% endupdate %}

{% update date="2026-03-20" %}

## Added

**Tracing (W3C `traceparent` support)** — Group multiple LLM API calls into a single trace with ordered spans

**Traces view in Activity** — Visualize execution timelines, latency, tokens, and cost across spans

<https://docs.fastrouter.ai/tracing>
{% endupdate %}
{% endupdates %}


# Coding Assistants

Use FastRouter as an OpenAI-compatible provider inside your coding assistant.   One key, 100+ models, full observability and cost control.

Every coding assistant below speaks the OpenAI-compatible API, so pointing it at FastRouter takes two fields: a base URL and a key. Once traffic routes through FastRouter you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, and Mistral behind one endpoint, including coding models like [Grok Code Fast 1](https://fastrouter.ai/models/x-ai/grok-code-fast-1). Swap models by changing one slug, no re-auth.
* **Observability** on every request: cost, tokens, latency, and which model ran, visible in real time in your [dashboard](https://dashboard.fastrouter.ai/).
* **Reliability** through automatic failover across providers, response caching, and intelligent routing.
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation.

The setup is the similar everywhere: paste your FastRouter key, set the base URL, enter a model slug in `provider/model-name` format. Pick your tool below for the exact steps.

<table data-view="cards"><thead><tr><th data-type="content-ref"></th></tr></thead><tbody><tr><td><a href="/pages/OQPVDFdjUsbyqpcH9bV8">/pages/OQPVDFdjUsbyqpcH9bV8</a></td></tr><tr><td><a href="/pages/I6Wtc0S5LZ65SCc44M1a">/pages/I6Wtc0S5LZ65SCc44M1a</a></td></tr><tr><td><a href="/pages/jeOz4ZJ63n7V6pttzuIb">/pages/jeOz4ZJ63n7V6pttzuIb</a></td></tr><tr><td><a href="/pages/PXHLlzvl99HdMuXSLzau">/pages/PXHLlzvl99HdMuXSLzau</a></td></tr><tr><td><a href="/pages/6FAB6VHuLRdOLkbtoeme">/pages/6FAB6VHuLRdOLkbtoeme</a></td></tr><tr><td><a href="/pages/LIdNJeVQxI079WUTRovj">/pages/LIdNJeVQxI079WUTRovj</a></td></tr><tr><td><a href="/pages/qyyUYvAoJMbl3weOjsuy">/pages/qyyUYvAoJMbl3weOjsuy</a></td></tr><tr><td><a href="/pages/kjXyTcEJLbiSkR6Us8iG">/pages/kjXyTcEJLbiSkR6Us8iG</a></td></tr><tr><td><a href="/pages/sf2PM8v12ObwTQ6ZZaot">/pages/sf2PM8v12ObwTQ6ZZaot</a></td></tr><tr><td><a href="/pages/2EKXkrHJ73MtSXjHMPV8">/pages/2EKXkrHJ73MtSXjHMPV8</a></td></tr></tbody></table>


# Aider

Track usage, control costs, and add guardrails to your Aider coding sessions

What is Aider?

[Aider](https://aider.chat/) is an AI pair programming tool that runs in your terminal. It edits code in your local git repository, makes commits, and works across multiple files—driven entirely by natural-language instructions. It connects to any OpenAI-compatible endpoint.

By routing Aider through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—including coding models like [Grok Code Fast 1](https://fastrouter.ai/models/x-ai/grok-code-fast-1)
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers configuring Aider to use FastRouter through its OpenAI-compatible settings.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Python 3.10 or higher, and a git repository to work in

***

#### Quick Start

**Step 1: Install Aider**

```bash
python -m pip install aider-install
aider-install
```

**Step 2: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

**Step 3: Point Aider at FastRouter**

Aider uses LiteLLM under the hood. Set FastRouter as the OpenAI-compatible endpoint with two environment variables. **Run these in your terminal—not at Aider's `>` prompt—before launching Aider:**

```bash
export OPENAI_API_BASE=https://api.fastrouter.ai/api/v1
export OPENAI_API_KEY=sk-add-your-key-here
echo $OPENAI_API_KEY   # confirm it prints before continuing
```

> **Note:** Environment variables only last for the current terminal session. For a setup that persists, see Persisting your credentials below.

**Step 4: Launch Aider with a FastRouter Model**

Prefix the FastRouter model slug with `openai/` so LiteLLM uses the OpenAI-compatible protocol:

```bash
cd /path/to/your/git/repo
aider --model openai/x-ai/grok-code-fast-1
```

> **Note:** The `openai/` prefix is **required**. It selects the OpenAI-compatible protocol; the rest (`x-ai/grok-code-fast-1`) is the FastRouter model slug. Without it, LiteLLM cannot determine the provider and fails with `LLM Provider NOT provided`. For an Anthropic model, use `--model openai/anthropic/claude-4.5-sonnet`.

If the folder isn't a git repository, Aider offers to create one—answer **Yes** (recommended, so it can track and undo changes), or run with `--no-git` to skip git entirely.

On first launch you'll see two harmless warnings: *"Unknown context window size and costs"* and *"Did you mean …"*. Both occur because Aider's local model database doesn't recognize FastRouter slugs. They don't affect anything—silence them with `--no-show-model-warnings`.

Aider starts in your repo and is ready for instructions. All requests route through FastRouter, and every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

**Persisting your credentials**

Because `export` only lasts for the current terminal session, the recommended setup is a `.env` file in your project root, which Aider loads automatically:

```bash
cat > .env <<'EOF'
OPENAI_API_BASE=https://api.fastrouter.ai/api/v1
OPENAI_API_KEY=sk-add-your-key-here
EOF
echo ".env" >> .gitignore   # never commit your key
```

With `.env` in place, just run `aider --model openai/x-ai/grok-code-fast-1`—no exports needed. Alternatively, add the two `export` lines to your shell profile (`~/.zshrc` or `~/.bashrc`) to apply them everywhere.

> **Security:** If you use a `.env` file, make sure it is git-ignored (the command above does this). Aider auto-ignores `.aider*` but **not** `.env`.

<figure><img src="/files/N0cPoasb6mFk336sYXNQ" alt=""><figcaption></figcaption></figure>

***

#### Use Aider with 100+ Models

FastRouter uses the `provider/model-name` format (after LiteLLM's `openai/` protocol prefix). Switch models with the `--model` flag:

```bash
# Anthropic Claude
aider --model openai/anthropic/claude-4.5-sonnet

# OpenAI
aider --model openai/openai/gpt-5.2
```

**Recommended models for Aider:**

* `x-ai/grok-code-fast-1` — fast and inexpensive; a good default for routine edits
* `anthropic/claude-4.5-sonnet` — highest quality for complex, multi-file changes
* `openai/gpt-5.2` — strong all-round alternative

A common Aider pattern is a strong "architect" model for planning plus a cheaper "editor" model for the mechanical edits. Both use the same FastRouter key:

```bash
aider --model openai/anthropic/claude-4.5-sonnet \
      --editor-model openai/x-ai/grok-code-fast-1
```

For large files, add `--edit-format diff` so Aider sends only the changed lines instead of rewriting whole files.

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```bash
aider --model openai/fastrouter/auto
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**I get `litellm.AuthenticationError: ... OPENAI_API_KEY ... must be set`. What's wrong?**

The environment variables aren't set in the terminal you launched Aider from—usually because you opened a new terminal (variables don't persist across sessions) or typed the `export` lines at Aider's `>` prompt instead of in the shell. Run the two `export` commands in your terminal *before* launching Aider, confirm with `echo $OPENAI_API_KEY`, or use the `.env` file so the key is always available.

**I get `litellm.BadRequestError: LLM Provider NOT provided`. What's wrong?**

You omitted the `openai/` protocol prefix. The model must be `openai/<fastrouter-slug>`—for example, `openai/x-ai/grok-code-fast-1`, not `x-ai/grok-code-fast-1`.

**Aider reports an unknown model or context-window warning. Is that a problem?**

No. Aider's local model database doesn't have metadata for FastRouter slugs, so it shows *"Unknown context window size and costs"* and *"Did you mean …"* and falls back to sane defaults. Everything works—suppress the warnings with `--no-show-model-warnings`.

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Switch models with `--model` at any time—no key changes needed.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Costs & Budgeting**

**Aider can send large repo context. How do I control cost?**

Set a budget and rate limit on the key, and use Aider's `/tokens` command to monitor context size. The Dashboard breaks down costs by project, key, model, and tag.

**Privacy & Security**

**Is my code sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# Claude Code

This guide walks you through setting up Claude Code to work with FastRouter.ai as your API provider.

#### Introduction

**Claude Code** is Anthropic's official CLI tool that brings Claude's AI capabilities directly to your terminal and development workflow. It enables you to interact with Claude for coding tasks, file operations, git workflows, and more—all from the command line.

**FastRouter.ai** is an intelligent AI routing platform that offers access to multiple model providers with a single API, offering cost reduction, improved reliability, and enhanced performance through smart routing algorithms.

By integrating **Claude Code** with **FastRouter.ai**, you gain a centralized and enterprise-ready way to operate Claude across your organization. FastRouter acts as the control plane for all usage—allowing you to securely manage and rotate user API keys, standardize access across teams, and eliminate the risks of scattered credentials.

In addition, FastRouter provides built-in observability and governance, giving you clear visibility into usage patterns, spend, and performance. You can enforce rate limits to prevent abuse, define role-based access controls to ensure the right people have the right level of access, and set budgets to keep costs predictable. Together, this integration lets you scale Claude Code safely, efficiently, and with full operational control—without slowing down developers.

#### Prerequisites

* Linux operating system
* Terminal access
* A FastRouter.ai API key ([Get one here](https://fastrouter.ai/))

#### Installation

**Step 1: Install Claude Code**

Run the following command in your terminal to install Claude Code on Linux:

```bash
curl -fsSL https://claude.ai/install.sh | bash
```

This will download and install the latest version of Claude Code on your system.

#### Configuration

**Step 2: Navigate to Your Project Directory**

Change to the directory where you want to use Claude Code:

```bash
cd /path/to/your/project
```

**Step 3: Set Environment Variables**

Run the following commands in your terminal to configure Claude Code to use FastRouter.ai:

```bash
export ANTHROPIC_BASE_URL="https://api.fastrouter.ai"
export ANTHROPIC_API_KEY=""
export ANTHROPIC_AUTH_TOKEN=[YOUR-FASTROUTER-API-KEY]
```

**Important Notes:**

* Replace `[YOUR-FASTROUTER-API-KEY]` with your actual FastRouter.ai API key
* Setting `ANTHROPIC_API_KEY=""` (empty string) is **required** for proper functionality
* These environment variables will only persist for your current terminal session

**Step 4: Persist Environment Variables (Optional)**

To make these settings permanent across terminal sessions, add them to your shell profile file.

**For Bash users** (`~/.bashrc` or `~/.bash_profile`):

```bash
echo 'export ANTHROPIC_BASE_URL="https://api.fastrouter.ai"' >> ~/.bashrc
echo 'export ANTHROPIC_API_KEY=""' >> ~/.bashrc
echo 'export ANTHROPIC_AUTH_TOKEN=[YOUR-FASTROUTER-API-KEY]' >> ~/.bashrc
source ~/.bashrc
```

**For Zsh users** (`~/.zshrc`):

```bash
echo 'export ANTHROPIC_BASE_URL="https://api.fastrouter.ai"' >> ~/.zshrc
echo 'export ANTHROPIC_API_KEY=""' >> ~/.zshrc
echo 'export ANTHROPIC_AUTH_TOKEN=[YOUR-FASTROUTER-API-KEY]' >> ~/.zshrc
source ~/.zshrc
```

**For Fish users** (`~/.config/fish/config.fish`):

```bash
echo 'set -x ANTHROPIC_BASE_URL "https://api.fastrouter.ai"' >> ~/.config/fish/config.fish
echo 'set -x ANTHROPIC_API_KEY ""' >> ~/.config/fish/config.fish
echo 'set -x ANTHROPIC_AUTH_TOKEN [YOUR-FASTROUTER-API-KEY]' >> ~/.config/fish/config.fish
source ~/.config/fish/config.fish
```

#### Verification

**Step 5: Launch Claude Code**

Start Claude Code by running:

```bash
claude
```

Claude Code should now be configured to route all API requests through FastRouter.ai.

***

#### Using Other Models via FastRouter

One of the key advantages of routing Claude Code through FastRouter is access to the **full range of models** supported by FastRouter—not just Anthropic's Claude. You can point Claude Code at any model available on FastRouter by passing the `--model` flag at startup.

**Syntax**

```bash
claude --model <provider-name>/<model-name>
```

**Examples**

**OpenAI GPT-5.4 Mini**

```bash
claude --model openai/gpt-5.4-mini
```

**DeepSeek V4 Pro**

```bash
claude --model deepseek/deepseek-v4-pro
```

**How It Works**

When you specify a model using the `--model` flag, FastRouter intercepts the request and routes it to the appropriate provider on your behalf. Your FastRouter API key handles authentication across all supported providers—no additional credentials needed.

> **Note:** Model availability depends on your FastRouter plan. Visit the [FastRouter model catalog](https://fastrouter.ai/) to see the full list of supported models and providers.

***

#### Troubleshooting

**Common Issues**

**Issue: Authentication errors**

* Verify your FastRouter.ai API key is correct
* Ensure `ANTHROPIC_API_KEY` is set to an empty string (`""`)
* Check that `ANTHROPIC_BASE_URL` is exactly `https://api.fastrouter.ai`

**Issue: Environment variables not persisting**

* Make sure you've added the exports to the correct shell profile file
* Run `source ~/.bashrc` (or appropriate file) to reload the configuration
* Verify the variables are set with: `echo $ANTHROPIC_BASE_URL`

**Issue: Claude Code not found after installation**

* Restart your terminal or open a new terminal window
* Check if the installation path is in your `PATH` variable
* Try running the installation command again

**Issue: Model not found or unsupported**

* Confirm the model name matches FastRouter's catalog exactly (e.g., `openai/gpt-5.4-mini`)
* Ensure your FastRouter plan includes access to the requested provider
* Check [FastRouter documentation](https://docs.fastrouter.ai/) for the current list of supported models

***


# Codex CLI

Integrating Codex CLI with FastRouter.

### What is Codex CLI

Codex CLI is OpenAI's open-source local coding agent that runs in your terminal. It supports multiple model providers, including FastRouter, allowing you to leverage FastRouter's unified API gateway, provider failover, unified observability, BYOK (Bring Your Own Key) support, and organizational controls while using Codex's agentic coding workflows.

### 1. Install Codex CLI

Start by installing Codex CLI on your local machine. Follow the [installation instructions](https://github.com/openai/codex) for your operating system.

```bash
npm install -g @openai/codex
# provide fastrouter api key by following steps 2 and 3
codex --version
```

### 2. Create a FastRouter account

Sign up or log in to your FastRouter account to access models through FastRouter's unified AI gateway.

### 3. Generate a FastRouter API key

1. Open the [FastRouter dashboard](https://dashboard.fastrouter.ai/).
2. Navigate to **API Keys** (keys page).
3. Create a new API key.
4. Copy and securely store your API key.

### 4. Configure Codex CLI to use FastRouter

#### 4.1. Edit the config file

Edit the `~/.codex/config.toml` file to connect to FastRouter:

```toml
model_provider = "fastrouter"
model_reasoning_effort = "high"
model = "openai/gpt-5.5"

[model_providers.fastrouter]
name = "fastrouter"
base_url = "https://api.fastrouter.ai/api/v1"
env_key = "FASTROUTER_API_KEY"
wire_api = "responses"
```

#### 4.2. Export your FastRouter API key

Export the API key in your terminal:

```bash
export FASTROUTER_API_KEY="sk-v1-****"
```

#### 4.3. Start Codex

```bash
cd /path/to/your/project
codex
```

### 5. Codex core settings reference

Codex supports project-level trust settings. Define project paths to control which directories and resources the agent can access.

```toml
[projects."/path/to/trusted/project"]
trust_level = "trusted"

[projects."/path/to/untrusted/project"]
trust_level = "untrusted"
```

* **trusted** — Agent can freely access and modify the project, including running commands and editing files.
* **untrusted** — Agent operates with limited permissions and requires additional approval for sensitive actions.

### 6. Alternative: install Codex CLI with FastRouter

#### One-line install script

In the commands below, replace `$API_KEY` with your actual FastRouter API key.

```bash
curl -fsSL https://fastrouter.ai/codex/install-fastrouter.sh | sh -s -- $API_KEY
```

Or download, inspect, then run:

```bash
curl -fsSL https://fastrouter.ai/codex/install-fastrouter.sh -o install-fastrouter.sh
less install-fastrouter.sh
chmod +x install-fastrouter.sh
./install-fastrouter.sh $API_KEY
```

#### Install Codex with the FastRouter skill file

Install using `skill.md`:

```bash
codex skills install https://fastrouter.ai/codex/skill.md
```

Or ask Codex to install FastRouter:

```
install fastrouter in codex with key <API_KEY>
```

### 7. Why use Codex CLI with FastRouter

Configure Codex CLI to use your FastRouter API key and endpoint. Once configured, you can access models through FastRouter and take advantage of:

**Unified API** — Access models from multiple AI providers through a single, consistent API interface.

**Provider failover & routing** — Automatically route requests to available providers and fail over during outages or rate limits.

**Unified observability** — Monitor requests, latency, costs, and usage across all providers from a centralized dashboard.

**Troubleshooting** — Debug API requests, authentication, routing, and provider responses faster with FastRouter's audit logs.

**BYOK (Bring Your Own Key)** — Use your own provider API keys while maintaining centralized routing and management.

**Organization controls** — Manage organizations, projects, keys, and API usage with enterprise-grade governance controls.

### 8. References

You can find the `skill.md` file attached below, or at <https://fastrouter.ai/codex/skill.md>.

***

````markdown
---
name: codex-cli-fastrouter
description: "Install and configure FastRouter (https://fastrouter.ai) as a custom OpenAI-compatible model provider in OpenAI Codex CLI. Use this skill whenever the user asks to add, set up, install, configure, or register fastrouter as a provider in codex, edit ~/.codex/config.toml, export FASTROUTER_API_KEY, or wants to use FastRouter models with Codex CLI. The skill takes exactly one input: the FastRouter API key."
version: 1.0.0
author: FastRouter
license: MIT
metadata:
  codex:
    tags:
      [
        fastrouter,
        codex,
        provider,
        setup,
        configuration,
        custom-provider,
        openai-compatible,
      ]
    homepage: https://fastrouter.ai
    docs: https://docs.fastrouter.ai
---

# FastRouter Setup for Codex CLI

Configures **fastrouter** as a custom OpenAI-compatible model provider in OpenAI Codex CLI. After setup, FastRouter models are invoked by setting `model_provider = "fastrouter"` and `model = "<slug>"` in `~/.codex/config.toml`, with auth flowing through the `FASTROUTER_API_KEY` env var.

## Inputs

This skill accepts **exactly one input**: the FastRouter API key. Everything else is hardcoded and must not be changed:

| Field        | Value                              |
| ------------ | ---------------------------------- |
| Provider key | `fastrouter`                       |
| Base URL     | `https://api.fastrouter.ai/api/v1` |
| Wire API     | `responses` (OpenAI Responses API) |
| Env var      | `FASTROUTER_API_KEY`               |
| Config file  | `~/.codex/config.toml`             |

## When to Use This Skill

Trigger this skill when the user asks any of:

- "install fastrouter for codex"
- "install fastrouter in codex with key <KEY>"
- "set up / configure / register fastrouter in codex"
- "add fastrouter as a provider in codex"
- "edit `~/.codex/config.toml` for fastrouter"
- "use fastrouter with codex cli"
- "point codex at fastrouter"

Anything similar referencing **codex** plus install / setup / configure / add / register / provider plus **fastrouter** should trigger it.

## Extracting the API Key (single-argument rule)

Treat every invocation as a single-argument call: `setup(api_key)`. Find the key in the user's message using these rules, in order:

1. If the message contains `key=<value>`, `--key <value>`, `--api-key <value>`, or `apiKey=<value>`, the value is the key.
2. Else, the **last whitespace-separated token in the message that looks like an API key** is the key. A token "looks like an API key" if it matches `^sk-v1-[A-Za-z0-9_\-]+$` (FastRouter keys always have the prefix `sk-v1-` followed by a random string of letters/digits/dashes/underscores).
3. Else, ask the user **once**: _"Please paste your FastRouter API key (just the key, nothing else)."_ Then treat their next message as the key verbatim, after stripping surrounding whitespace and quotes.

Examples of correct extraction:

| User message                                         | Extracted key     |
| ---------------------------------------------------- | ----------------- |
| `install fastrouter for codex with key sk-v1-abc123` | `sk-v1-abc123`    |
| `setup codex with fastrouter sk-v1-abc123`           | `sk-v1-abc123`    |
| `configure codex --api-key sk-v1-XYZ789`             | `sk-v1-XYZ789`    |
| `add fastrouter to codex key=sk-v1-9aBc...`          | `sk-v1-9aBc...`   |
| `set up codex with fastrouter`                       | (prompt the user) |

Once extracted, **never modify, paraphrase, or transform the key**. Do not strip characters you think are formatting. Do not lowercase. Do not echo the full key back to the user — only ever show it masked (`****` + last 4 chars) in confirmations.

## Redaction Check (do this before writing)

Some agent harnesses (Cursor, IDE extensions, CI loggers) may redact secrets in messages before the model sees them. After extracting the key, validate it. The key is **invalid for write** if any of:

- Equals `[REDACTED]`, `<redacted>`, `***`, `***REDACTED***`, or any string containing the substring `REDACTED` (case-insensitive)
- Does not start with the literal prefix `sk-v1-` (all real FastRouter keys begin with `sk-v1-`; anything else is either redacted, mistyped, or from a different provider)
- Has nothing after the `sk-v1-` prefix (i.e. the random portion is empty)
- Contains only `*` or `•` characters
- Is empty / whitespace-only

If the key fails validation due to redaction, **stop and tell the user**:

> "It looks like your environment is masking the API key before I can read it. To install fastrouter for Codex CLI, please run these in your terminal directly so the key never passes through the model:
>
> ```bash
> mkdir -p ~/.codex
> cat >> ~/.codex/config.toml <<'EOF'
> model_provider = "fastrouter"
> model = "openai/gpt-5.5"
>
> [model_providers.fastrouter]
> name = "fastrouter"
> base_url = "https://api.fastrouter.ai/api/v1"
> env_key = "FASTROUTER_API_KEY"
> wire_api = "responses"
> EOF
>
> export FASTROUTER_API_KEY="<YOUR_KEY>"
> echo 'export FASTROUTER_API_KEY="<YOUR_KEY>"' >> ~/.zshrc   # or ~/.bashrc
> ```

Do **not** proceed with a redacted-looking value — writing `[REDACTED]` as the API key silently breaks the provider with 401 errors that look unrelated to redaction.

## Procedure

Once you have a valid key, follow these steps in order.

### Step 1 — Ensure Codex CLI is installed

Run via the `terminal` tool:

```bash
codex --version
```

If `codex` is not found, install it first:

```bash
npm install -g @openai/codex
codex --version
```

### Step 2 — Ensure `~/.codex/config.toml` exists

```bash
mkdir -p ~/.codex
touch ~/.codex/config.toml
```

### Step 3 — Write the provider config

Codex stores configuration in TOML. Merge the following into `~/.codex/config.toml`. **Preserve existing keys** outside this block; do not clobber the file. If a top-level key already exists, update its value rather than duplicating.

Top-level keys (overwrite if present):

```toml
model_provider = "fastrouter"
model_reasoning_effort = "high"
model = "openai/gpt-5.5"
```

Provider table (must exist exactly as shown):

```toml
[model_providers.fastrouter]
name = "fastrouter"
base_url = "https://api.fastrouter.ai/api/v1"
env_key = "FASTROUTER_API_KEY"
wire_api = "responses"
```

The API key is **never** written to the TOML file. It always flows through the `FASTROUTER_API_KEY` env var, controlled by the `env_key = "FASTROUTER_API_KEY"` line above.

### Step 4 — Export the API key

Run via the `terminal` tool, using the extracted key **exactly as-is**:

```bash
export FASTROUTER_API_KEY="<USER_KEY>"
```

For persistence across shells, append the same line to the user's shell rc file:

```bash
echo 'export FASTROUTER_API_KEY="<USER_KEY>"' >> ~/.zshrc   # or ~/.bashrc
```

If unsure which shell rc to use, run `echo $SHELL` and pick the matching file.

### Step 5 — Verify

```bash
grep -A 4 '\[model_providers.fastrouter\]' ~/.codex/config.toml
[ -n "$FASTROUTER_API_KEY" ] && echo "FASTROUTER_API_KEY is set" || echo "FASTROUTER_API_KEY is NOT set"
```

Expected: all four fields (`name`, `base_url`, `env_key`, `wire_api`) appear under `[model_providers.fastrouter]`, and the env var prints `set`. **Never echo the actual value** of `FASTROUTER_API_KEY`.

### Step 6 — Report success

Reply to the user with usage examples. Show the API key **only masked**.

> ✓ FastRouter is configured for Codex CLI (`****<last 4 chars>`).
>
> Start Codex from any project:
>
> ```bash
> cd /path/to/your/project
> codex
> ```
>
> Switch models by editing `model = "..."` in `~/.codex/config.toml`. Browse available slugs at https://fastrouter.ai/models.

### Step 7 — Optional model wiring

If (and only if) the user named a specific model in their request, set it as `model` instead of the default `openai/gpt-5.5`:

```toml
model = "<that_model>"
```

Otherwise leave `model = "openai/gpt-5.5"` and `model_reasoning_effort = "high"` untouched.

## Pitfalls

- **Don't write the API key into `~/.codex/config.toml`.** Codex reads keys only from env vars, controlled by `env_key = "FASTROUTER_API_KEY"`. Storing the key in the TOML does nothing useful and risks committing it to git if the user version-controls dotfiles.
- **`model_provider` must match the table key exactly.** `model_provider = "fastrouter"` requires `[model_providers.fastrouter]` (same string). A mismatch produces `provider 'fastrouter' not found`.
- **`wire_api` must be `"responses"`, not `"chat"`.** FastRouter speaks the OpenAI Responses API. Setting `wire_api = "chat"` causes wire-format errors mid-stream.
- **Env var must be exported in the same shell that runs `codex`.** If the user starts `codex` from a fresh terminal, the var is gone unless persisted in `~/.zshrc` / `~/.bashrc`.
- **Config changes require restarting `codex`.** If the user is in an active Codex session, the new provider only becomes available after quitting and restarting.
- **Redacted keys silently corrupt config.** Always run the redaction check before writing. Writing `[REDACTED]` (or pasting a masked key into `export`) produces 401 errors later that look unrelated to redaction.
- **Never echo the full key back.** Only show it masked (last 4 chars). Some platforms log assistant output verbatim.

## Verification Checklist

Before reporting success, confirm:

- [ ] Exactly one key was extracted (Inputs rule)
- [ ] The key passed the redaction check (starts with `sk-v1-`, no `REDACTED` substring)
- [ ] `~/.codex/config.toml` exists and contains all four required fields under `[model_providers.fastrouter]`
- [ ] Top-level `model_provider = "fastrouter"` is set
- [ ] `FASTROUTER_API_KEY` is exported in the current shell (and persisted in shell rc if the user requested)
- [ ] You did NOT echo the raw API key — only masked form

## References

- Codex CLI: https://github.com/openai/codex
- FastRouter dashboard: https://dashboard.fastrouter.ai/
- FastRouter API base: `https://api.fastrouter.ai/api/v1`
- FastRouter models: https://fastrouter.ai/models
- FastRouter docs: https://docs.fastrouter.ai
````


# Cursor

Track usage, control costs, and add guardrails to your Cursor AI editor

#### What is Cursor?

[Cursor](https://cursor.com/) is a powerful AI-powered code editor built for pair-programming with AI. It provides chat, inline edits, and agentic coding workflows directly inside the editor, and supports custom OpenAI-compatible endpoints through its model settings.

By routing Cursor through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—including coding models like [Grok Code Fast 1](https://fastrouter.ai/models/x-ai/grok-code-fast-1)
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers configuring Cursor's OpenAI API settings to use FastRouter and adding models from the FastRouter catalog.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Cursor installed ([download](https://cursor.com/))

***

#### Quick Start

**Step 1: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

**Step 2: Configure the OpenAI Base URL**

Open **Cursor Settings**, go to the **Models** section, and in the OpenAI API key settings:

1. Paste your FastRouter API key into the **OpenAI API Key** field
2. Enable **Override OpenAI Base URL** and set it to:

```
https://api.fastrouter.ai/api/v1
```

<figure><img src="/files/bIQisnAFT0EpZ70ImtTw" alt=""><figcaption></figcaption></figure>

**Step 3: Add a Model**

Click **Add Model** and enter a FastRouter model slug, for example:

```
x-ai/grok-code-fast-1
```

<figure><img src="/files/eZpkihZlZvnmUFz68ME7" alt=""><figcaption></figcaption></figure>

**Step 4: Start Coding**

Select the model you added from Cursor's model picker and start a chat or edit. Requests now route through FastRouter, and every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

***

#### Use Cursor with 100+ Models

FastRouter uses the `provider/model-name` format. Add any model from the catalog the same way—click **Add Model** and enter the slug:

```
# Anthropic Claude
anthropic/claude-4.5-sonnet

# OpenAI
openai/gpt-5.2

# xAI Grok
x-ai/grok-4
```

Check the [FastRouter Model Catalog](https://fastrouter.ai/models) for available models and input modalities (text, image, file).

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost. Add this model identifier:

```
fastrouter/auto
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Add as many models as you like in Cursor's settings and switch between them from the model picker—no key changes needed.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**My requests fail after setup. What should I check?**

Verify the Base URL is exactly `https://api.fastrouter.ai/api/v1`, the API key is correct, and the model name matches a FastRouter catalog slug exactly (including the provider prefix). Also make sure Cursor is updated to the latest version.

**Costs & Budgeting**

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit.

**Privacy & Security**

**Is my code sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# Cline

Track usage, control costs, and add guardrails to your Cline coding agent

#### What is Cline?

[Cline](https://cline.bot/) is an AI coding assistant that integrates directly into your VS Code environment, providing autonomous coding capabilities—it can create and edit files, run commands, and complete multi-step tasks with your approval. It supports any OpenAI-compatible provider through its API settings.

By routing Cline through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—including coding models like [Grok Code Fast 1](https://fastrouter.ai/models/x-ai/grok-code-fast-1)
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers configuring the Cline VS Code extension to use FastRouter as an OpenAI-compatible provider.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* VS Code with the [Cline extension](https://marketplace.visualstudio.com/items?itemName=saoudrizwan.claude-dev) installed

***

#### Quick Start

**Step 1: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

**Step 2: Configure the API Provider**

Open **Cline Settings** in VS Code and configure:

1. **Select:**  Bring my own API key

<figure><img src="/files/F0fZwXyCx58gTkrDeXo0" alt=""><figcaption></figcaption></figure>

1. **API Provider:** select **OpenAI Compatible**
2. **Base URL:**

```
https://api.fastrouter.ai/api/v1
```

<figure><img src="/files/RyFrSX0bdPXXkz9sdRas" alt=""><figcaption></figcaption></figure>

3. **API Key:** paste your FastRouter API key
4. **Model ID:** enter a FastRouter model slug, for example:

```
x-ai/grok-code-fast-1
```

**Step 3: Start a Task**

Give Cline a task. All requests now route through FastRouter, and every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

<figure><img src="/files/JHNKxZUqdn2lxGI84LkR" alt=""><figcaption></figcaption></figure>

***

#### Use Cline with 100+ Models

FastRouter uses the `provider/model-name` format. Change the **Model ID** in Cline's settings to any catalog slug:

```
# Anthropic Claude
anthropic/claude-4.5-sonnet

# OpenAI
openai/gpt-5.2

# xAI Grok
x-ai/grok-4
```

Check the [FastRouter Model Catalog](https://fastrouter.ai/models) for available models and input modalities (text, image, file).

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost. Use this Model ID:

```
fastrouter/auto
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Change the Model ID in Cline's settings at any time—no key changes needed.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**My requests fail after setup. What should I check?**

Verify the Base URL is exactly `https://api.fastrouter.ai/api/v1`, the API key is correct, and the Model ID matches a FastRouter catalog slug exactly (including the provider prefix). Also make sure the Cline extension is updated to the latest version.

**Costs & Budgeting**

**Cline makes many requests per task. How do I keep costs under control?**

Set a budget and rate limit on the key you use with Cline. For team rollouts, create separate projects with project-scoped keys, and use Dynamic Tags for finer-grained attribution. The Dashboard breaks down costs by project, key, model, and tag.

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit.

**Privacy & Security**

**Is my code sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# Roo Code

Track usage, control costs, and add guardrails to your Roo Code coding agent

#### What is Roo Code?

[Roo Code](https://roocode.com/) is an AI coding assistant that integrates directly into your VS Code environment, providing autonomous coding capabilities through specialized modes for coding, architecture, and debugging. It supports any OpenAI-compatible provider through its API settings.

By routing Roo Code through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—including coding models like [Grok Code Fast 1](https://fastrouter.ai/models/x-ai/grok-code-fast-1)
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers configuring the Roo Code VS Code extension to use FastRouter as an OpenAI-compatible provider.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* VS Code with the [Roo Code extension](https://marketplace.visualstudio.com/items?itemName=RooVeterinaryInc.roo-cline) installed

***

#### Quick Start

**Step 1: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

**Step 2: Configure the API Provider**

Open **Roo Code Settings** in VS Code and configure:

<figure><img src="/files/n4987F6SYAKaDJ3X3gB2" alt=""><figcaption></figcaption></figure>

1. **API Provider:** select **OpenAI Compatible**
2. **Base URL:**

```
https://api.fastrouter.ai/api/v1
```

<figure><img src="/files/aNkXkBwfG1EmNNyYxzBh" alt=""><figcaption></figcaption></figure>

3. **API Key:** paste your FastRouter API key
4. **Model ID:** enter a FastRouter model slug, for example:

```
x-ai/grok-code-fast-1
```

**Step 3: Start a Task**

Give Roo Code a task in any of its modes. All requests now route through FastRouter, and every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

***

#### Use Roo Code with 100+ Models

FastRouter uses the `provider/model-name` format. Change the **Model ID** in Roo Code's settings to any catalog slug:

```
# Anthropic Claude
anthropic/claude-4.5-sonnet

# OpenAI
openai/gpt-5.2

# xAI Grok
x-ai/grok-4
```

Check the [FastRouter Model Catalog](https://fastrouter.ai/models) for available models and input modalities (text, image, file).

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost. Use this Model ID:

```
fastrouter/auto
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Change the Model ID in Roo Code's settings at any time—no key changes needed. You can also configure different models per Roo Code mode using configuration profiles.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**My requests fail after setup. What should I check?**

Verify the Base URL is exactly `https://api.fastrouter.ai/api/v1`, the API key is correct, and the Model ID matches a FastRouter catalog slug exactly (including the provider prefix). Also make sure the Roo Code extension is updated to the latest version.

**Costs & Budgeting**

**Roo Code makes many requests per task. How do I keep costs under control?**

Set a budget and rate limit on the key you use with Roo Code. For team rollouts, create separate projects with project-scoped keys, and use Dynamic Tags for finer-grained attribution. The Dashboard breaks down costs by project, key, model, and tag.

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit.

**Privacy & Security**

**Is my code sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# OpenCode

Track usage, control costs, and add guardrails to your OpenCode coding agent

### What is OpenCode?

[OpenCode](https://opencode.ai/) is an open-source coding agent that runs in your terminal, IDE, or desktop. It handles coding tasks, file operations, and multi-step workflows through natural language, and supports multiple model providers through its built-in provider settings.

By routing OpenCode through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers OpenCode desktop configuration with FastRouter, model selection, team governance, and production feature usage. It does not cover the OpenCode CLI or IDE plugins.

#### Prerequisites

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* OpenCode desktop app installed ([download](https://opencode.ai/download))

***

### Quick Start for OpenCode Desktop App

#### Step 1: Install OpenCode Desktop App

Download [OpenCode](https://opencode.ai/download) and install it on your system.

#### Step 2: Get Your FastRouter API Key

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

#### Step 3: Open OpenCode Settings

Launch the OpenCode desktop app and open **Settings**.

<img src="/files/g2RewhoFt2B47yIRfEOW" alt="" height="212" width="624">

#### Step 4: Go to Providers

In the Settings sidebar, select **Providers**.

<img src="/files/9rtIh4HwgOcyBjSwE8DG" alt="" height="252" width="624">

#### Step 5: Show More Providers

Click **Show more Providers** to expand the full provider list.

<img src="/files/hgLioren6mI1wrFEjk6u" alt="" height="185" width="624">

#### Step 6: Select FastRouter

Search for or scroll to **FastRouter** in the provider list and select it.

<img src="/files/h6EXSGm5Fonyv4Fgf4YZ" alt="" height="141" width="624">

#### Step 7: Add Your API Key

Paste the FastRouter API key from Step 2 and save. You should see a **Successfully added** confirmation.

<img src="/files/AsPDd8RIPKNU58k4ivkd" alt="" height="337" width="624">

#### Step 8: Enable FastRouter Models

To access FastRouter's full model catalog inside OpenCode:

1. Click the **model name** below the prompt box

<img src="/files/AEUE0lvzo2G2K1iLELb2" alt="" height="193" width="624">

2. Click **Manage Models**<br>

   <img src="/files/PErD7CqSk94px9N1i3Pf" alt="Manage Models" height="311" width="624">
3. Toggle on the FastRouter models you want to use<br>

   <img src="/files/L6vt0o1k9X2qXZwvEd1h" alt="" height="397" width="624">

&#x20; All enabled models are now available from the model picker. Requests route through FastRouter, and you can monitor usage in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

***

### Quick Start for OpenCode CLI

#### Step 1: Install OpenCode CLI

Download [OpenCode](https://opencode.ai/download) and install it on your system.

#### Step 2: Get Your FastRouter API Key

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

#### Step 3: Start OpenCode

```bash
cd /path/to/your/project
opencode
```

#### Step 4: Connect FastRouter

Inside OpenCode, run the `/connect` command and select **FastRouter** from the provider list:

```
/connect
```

<img src="/files/BnmNqxTMKlceBK0sOwVF" alt="" height="87" width="624">

1. Select **FastRouter**<br>

   <img src="/files/gpJuFZVu3zQGEk10CYL2" alt="" height="245" width="624">

2. Paste the API key from Step 2

   <img src="/files/NfDiWIzdvj4y2wdagNWJ" alt="" height="263" width="624">

3. Pick a model and start prompting<br>

   <img src="/files/5P4FdN3IPbSs8asD6omT" alt="" height="573" width="624">

#### Step 4 (Alternative): Configure via `opencode.json`

Instead of running `/connect`, you can declare the FastRouter provider in your `opencode.json` config file:

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "fastrouter": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "FastRouter",
      "options": {
        "baseURL": "https://api.fastrouter.ai/api/v1"
      },
      "models": {
        "anthropic/claude-sonnet-latest": {},
        "google/gemini-flash-latest": {}
      }
    }
  }
}
```

Set your API key via `/connect`, or by adding it to `~/.local/share/opencode/auth.json`:

```json
{
  "fastrouter": {
    "type": "api",
    "key": "sk-add-your-key-here"
  }
}
```

> **Note:** OpenCode stores credentials at `~/.local/share/opencode/auth.json` on Linux. macOS and Windows users should consult the OpenCode docs for their platform's auth path.

***

### Use OpenCode with 100+ Models

Switch between any of FastRouter's 100+ supported models from the OpenCode model picker (desktop) or by setting the `model` field / running `/model` (CLI). FastRouter uses the `provider/model-name` format:

```
openai/gpt-5.3-codex
```

#### Model Examples

Pick a different provider or model at any time:

```
# Anthropic Claude
anthropic/claude-sonnet-4.6

# Google Gemini
google/gemini-3-pro

# Mistral
mistralai/Mistral-Small-24B-Instruct-2501

# xAI Grok
x-ai/grok-4
```

No code changes or SDK swaps required. Select the model and OpenCode uses the new provider for the next request.

#### Automatic Model Selection

Let FastRouter pick the best model for each request based on query complexity, domain, and cost. Use this model identifier:

```
fastrouter/auto
```

FastRouter analyzes the input and routes to the most appropriate model from the available pool. This is the fastest way to get started without maintaining model preferences.

[Explore automatic provider selection](https://docs.fastrouter.ai/automatic-model-selection)

#### Cost-Optimized Routing with Sorting Slugs

Append a suffix to any model identifier to control provider selection:

```
# Route to the cheapest provider for this model
openai/gpt-5.3-codex:price

# Route to the fastest provider
openai/gpt-5.3-codex:throughput
```

[Configure cost and performance routing](https://docs.fastrouter.ai/provider-routing-strategies)

#### Flex Pricing

For batch-style coding tasks that tolerate higher latency, append `:flex` to any model to access up to 50% lower token costs:

```
openai/gpt-5.4-nano:flex
```

Flex routes your request to the provider's discounted inference tier. Same API key, same endpoint, same payload.

> **Note:** Flex is not recommended for interactive OpenCode sessions where you need low-latency responses. Use it for large refactoring jobs, codebase analysis, or documentation generation.

[Compare flex pricing options](https://docs.fastrouter.ai/explore-features/flex-pricing)

***

### FAQs

#### Configuration & Setup

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. The model is selected per request in OpenCode (model picker on desktop, `/model` or the `model` field in `opencode.json` on the CLI). You can switch models at any time without changing your key.

**Can I use OpenCode with my own provider keys (BYOK)?**

Yes. Set up an External Key integration in the FastRouter dashboard, then route OpenCode traffic through it. You retain your provider's pricing and rate limits while gaining FastRouter's routing and observability layer.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

#### Costs & Budgeting

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit. The developer receives an error response indicating the budget has been exceeded.

**How do I track OpenCode spending by team?**

Create separate projects for each team, issue project-scoped API keys, and use Dynamic Tags for finer-grained attribution. The Dashboard breaks down costs by project, key, model, and tag.

#### Performance & Reliability

**Does FastRouter add latency to OpenCode requests?**

FastRouter adds near-zero gateway overhead. For most workflows, this is negligible compared to model inference time.

#### Privacy & Security

**Is my code sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings.

***

### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# DeepSeek Reasonix CLI

Integrating DeepSeek Reasonix CLI with FastRouter.

### What is DeepSeek Reasonix CLI

DeepSeek Reasonix CLI is an open-source DeepSeek-native AI coding agent that runs in your terminal. It supports any DeepSeek-compatible model provider, including FastRouter, allowing you to leverage FastRouter's unified API gateway, provider failover, unified observability, BYOK (Bring Your Own Key) support, and organizational controls while using Reasonix's agentic coding workflows.

### Install DeepSeek Reasonix CLI

Start by installing Reasonix on your local machine. Follow the [installation instructions](https://github.com/esengine/DeepSeek-Reasonix) for your operating system.

**Install command**

```bash
npm install -g reasonix
```

### Create a FastRouter API key

Sign up or log in to your FastRouter account to access models through FastRouter's unified AI gateway.

1. Open the [FastRouter dashboard](https://dashboard.fastrouter.ai/).
2. Navigate to **API Keys** (keys page).
3. Create a new API key.
4. Copy and securely store your API key.

### Configure DeepSeek Reasonix CLI to use FastRouter

Edit the `~/.reasonix/config.json` file to connect to FastRouter:

```json
{
  "apiKey": "sk-v1-***",
  "baseUrl": "https://api.fastrouter.ai/api/v1",
  "model": "deepseek/deepseek-v3.2"
}
```

**Start DeepSeek Reasonix CLI**

```bash
reasonix code
```

### Alternative: install Reasonix CLI with FastRouter

#### One-line install script

In the commands below, replace `$API_KEY` with your actual FastRouter API key.

```bash
curl -fsSL https://fastrouter.ai/reasonix/install-fastrouter.sh | sh -s -- $API_KEY
```

Or download, inspect, then run:

```bash
curl -fsSL https://fastrouter.ai/reasonix/install-fastrouter.sh -o install-fastrouter.sh
less install-fastrouter.sh
chmod +x install-fastrouter.sh
./install-fastrouter.sh $API_KEY
```

#### Install Reasonix with the FastRouter skill file

Install using `skill.md`:

```bash
reasonix skills install https://fastrouter.ai/reasonix/skill.md
```

Or ask Reasonix to install FastRouter:

```
install fastrouter in reasonix with key <KEY>
```

### Why use DeepSeek Reasonix CLI with FastRouter

Configure DeepSeek Reasonix CLI to use your FastRouter API key and endpoint. Once configured, you can access models through FastRouter and take advantage of:

**Unified API** — Access models from multiple AI providers through a single, consistent API interface.

**Provider failover & routing** — Automatically route requests to available providers and fail over during outages or rate limits.

**Unified observability** — Monitor requests, latency, costs, and usage across all providers from a centralized dashboard.

**Troubleshooting** — Debug API requests, authentication, routing, and provider responses faster with FastRouter's audit logs.

**BYOK (Bring Your Own Key)** — Use your own provider API keys while maintaining centralized routing and management.

**Organization controls** — Manage organizations, projects, keys, and API usage with enterprise-grade governance controls.

### References

You can find the `skill.md` file attached below, or at <https://fastrouter.ai/reasonix/skill.md>.

***

````markdown
---
name: reasonix-cli-fastrouter
description: "Install and configure FastRouter (https://fastrouter.ai) as the model provider in DeepSeek Reasonix CLI. Use this skill whenever the user asks to add, set up, install, configure, or register fastrouter as a provider in reasonix, edit ~/.reasonix/config.json, point reasonix or deepseek-reasonix at FastRouter, or wants to use FastRouter models with DeepSeek Reasonix CLI. The skill takes exactly one input: the FastRouter API key."
version: 1.0.0
author: FastRouter
license: MIT
metadata:
  reasonix:
    tags:
      [
        fastrouter,
        reasonix,
        deepseek,
        provider,
        setup,
        configuration,
        custom-provider,
        openai-compatible,
      ]
    homepage: https://fastrouter.ai
    docs: https://docs.fastrouter.ai
---

# FastRouter Setup for DeepSeek Reasonix CLI

Configures **fastrouter** as the model provider in DeepSeek Reasonix CLI by writing `apiKey`, `baseUrl`, and `model` into `~/.reasonix/config.json`. After setup, FastRouter models are invoked when the user runs `reasonix code` from any project.

## Inputs

This skill accepts **exactly one input**: the FastRouter API key. Everything else is hardcoded and must not be changed:

| Field         | Value                              |
| ------------- | ---------------------------------- |
| Provider      | `fastrouter`                       |
| Base URL      | `https://api.fastrouter.ai/api/v1` |
| Default model | `deepseek/deepseek-v3.2`           |
| Config file   | `~/.reasonix/config.json`          |
| File mode     | `0600` (key is stored on disk)     |

## When to Use This Skill

Trigger this skill when the user asks any of:

- "install fastrouter for reasonix"
- "install fastrouter in reasonix with key <KEY>"
- "set up / configure / register fastrouter in reasonix"
- "add fastrouter as a provider in reasonix"
- "edit `~/.reasonix/config.json` for fastrouter"
- "use fastrouter with deepseek reasonix cli"
- "point reasonix at fastrouter"
- "configure deepseek reasonix with fastrouter"

Anything similar referencing **reasonix** (or "deepseek reasonix") plus install / setup / configure / add / register / provider plus **fastrouter** should trigger it.

## Extracting the API Key (single-argument rule)

Treat every invocation as a single-argument call: `setup(api_key)`. Find the key in the user's message using these rules, in order:

1. If the message contains `key=<value>`, `--key <value>`, `--api-key <value>`, or `apiKey=<value>`, the value is the key.
2. Else, the **last whitespace-separated token in the message that looks like an API key** is the key. A token "looks like an API key" if it matches `^sk-v1-[A-Za-z0-9_\-]+$` (FastRouter keys always have the prefix `sk-v1-` followed by a random string of letters/digits/dashes/underscores).
3. Else, ask the user **once**: _"Please paste your FastRouter API key (just the key, nothing else)."_ Then treat their next message as the key verbatim, after stripping surrounding whitespace and quotes.

Examples of correct extraction:

| User message                                            | Extracted key        |
| ------------------------------------------------------- | -------------------- |
| `install fastrouter for reasonix with key sk-v1-abc123` | `sk-v1-abc123`       |
| `setup reasonix with fastrouter sk-v1-abc123`           | `sk-v1-abc123`       |
| `configure reasonix --api-key sk-v1-XYZ789`             | `sk-v1-XYZ789`       |
| `add fastrouter to reasonix key=sk-v1-9aBc_def-456`     | `sk-v1-9aBc_def-456` |
| `set up reasonix with fastrouter`                       | (prompt the user)    |

Once extracted, **never modify, paraphrase, or transform the key**. Do not strip characters you think are formatting. Do not lowercase. Do not echo the full key back to the user — only ever show it masked (`****` + last 4 chars) in confirmations.

## Redaction Check (do this before writing)

Some agent harnesses (Cursor, IDE extensions, CI loggers) may redact secrets in messages before the model sees them. After extracting the key, validate it. The key is **invalid for write** if any of:

- Equals `[REDACTED]`, `<redacted>`, `***`, `***REDACTED***`, or any string containing the substring `REDACTED` (case-insensitive)
- Does not start with the literal prefix `sk-v1-` (all real FastRouter keys begin with `sk-v1-`; anything else is either redacted, mistyped, or from a different provider)
- Has nothing after the `sk-v1-` prefix (i.e. the random portion is empty)
- Contains only `*` or `•` characters
- Is empty / whitespace-only

If the key fails validation due to redaction, **stop and tell the user**:

> "It looks like your environment is masking the API key before I can read it. To install fastrouter for Reasonix, please run these in your terminal directly so the key never passes through the model:
>
> ```bash
> mkdir -p ~/.reasonix
> cat > ~/.reasonix/config.json <<'EOF'
> {
>   "apiKey": "<YOUR_KEY>",
>   "baseUrl": "https://api.fastrouter.ai/api/v1",
>   "model": "deepseek/deepseek-v3.2"
> }
> EOF
> chmod 600 ~/.reasonix/config.json
> ```
>
> If `~/.reasonix/config.json` already exists with other settings, edit it manually and merge the three keys above instead of overwriting."

Do **not** proceed with a redacted-looking value — writing `[REDACTED]` as `apiKey` silently breaks the provider with 401 errors that look unrelated to redaction.

## Procedure

Once you have a valid key, follow these steps in order.

### Step 1 — Ensure Reasonix CLI is installed

Run via the `terminal` tool:

```bash
reasonix --version
```

If `reasonix` is not found, install it first:

```bash
npm install -g reasonix
reasonix --version
```

Reference: https://github.com/esengine/DeepSeek-Reasonix

### Step 2 — Ensure `~/.reasonix/config.json` exists

```bash
mkdir -p ~/.reasonix
[ -f ~/.reasonix/config.json ] || echo '{}' > ~/.reasonix/config.json
```

If the user prefers an interactive setup, they can instead run `reasonix code` once and paste the API key when prompted — Reasonix's built-in wizard generates the file automatically. Either path produces the same end state, but only proceed past this step once `~/.reasonix/config.json` exists.

### Step 3 — Write the FastRouter config

Reasonix config is a flat JSON object. Merge these three keys, **preserving any other existing keys** in the file. Use the extracted key exactly as-is — never rename, lowercase, or transform it.

Required keys:

```json
{
  "apiKey": "<USER_KEY>",
  "baseUrl": "https://api.fastrouter.ai/api/v1",
  "model": "deepseek/deepseek-v3.2"
}
```

Safe merge with `jq` (preferred — preserves other keys):

```bash
KEY="<USER_KEY>"
jq --arg key "$KEY" \
   '. + {apiKey: $key, baseUrl: "https://api.fastrouter.ai/api/v1", model: "deepseek/deepseek-v3.2"}' \
   ~/.reasonix/config.json > ~/.reasonix/config.json.tmp \
   && mv ~/.reasonix/config.json.tmp ~/.reasonix/config.json
```

Fallback merge with Python (when `jq` is unavailable):

```bash
python3 - "<USER_KEY>" <<'PY'
import json, os, sys
key = sys.argv[1]
path = os.path.expanduser("~/.reasonix/config.json")
try:
    with open(path) as f: cfg = json.load(f)
except Exception:
    cfg = {}
cfg["apiKey"]  = key
cfg["baseUrl"] = "https://api.fastrouter.ai/api/v1"
cfg["model"]   = "deepseek/deepseek-v3.2"
with open(path, "w") as f: json.dump(cfg, f, indent=2)
PY
```

### Step 4 — Restrict file permissions

The API key is stored on disk in cleartext. Make the file owner-only:

```bash
chmod 600 ~/.reasonix/config.json
```

### Step 5 — Verify

```bash
ls -l ~/.reasonix/config.json
jq 'del(.apiKey)' ~/.reasonix/config.json
jq -r '.apiKey | "apiKey ends in: ****" + .[-4:]' ~/.reasonix/config.json
```

Expected:

- File mode is `-rw-------` (octal `0600`)
- `baseUrl` is `"https://api.fastrouter.ai/api/v1"` exactly
- `model` is `"deepseek/deepseek-v3.2"` (or the user-specified slug)
- Masked key suffix matches the last 4 chars the user provided

**Never `cat` or otherwise dump the full file** — the `apiKey` is in cleartext. Always use the `jq 'del(.apiKey)'` form when showing config.

### Step 6 — Report success

Reply to the user with usage examples. Show the API key **only masked**.

> ✓ FastRouter is configured for DeepSeek Reasonix CLI (`****<last 4 chars>`).
>
> Start Reasonix from any project:
>
> ```bash
> cd /path/to/your/project
> reasonix code
> ```
>
> Switch models by editing `model` in `~/.reasonix/config.json`. Browse available DeepSeek slugs at https://fastrouter.ai/models.

### Step 7 — Optional model wiring

If (and only if) the user named a specific model in their request, set it as `model` instead of the default `deepseek/deepseek-v3.2`:

```bash
jq --arg m "<that_model>" '.model = $m' \
   ~/.reasonix/config.json > ~/.reasonix/config.json.tmp \
   && mv ~/.reasonix/config.json.tmp ~/.reasonix/config.json
```

Otherwise leave `"model": "deepseek/deepseek-v3.2"` untouched.

## Pitfalls

- **The API key IS stored in the config file.** Reasonix reads `apiKey` directly from `~/.reasonix/config.json` in cleartext (no env-var indirection). Never commit this file to git or share it. Always `chmod 600` after writing. Add `.reasonix/` and `**/.reasonix/config.json` to global `~/.gitignore` if the user version-controls dotfiles.
- **Don't `cat ~/.reasonix/config.json`** when output may be streamed to logs, chat, or terminal recordings — the key is in cleartext. Use `jq 'del(.apiKey)'` for inspection.
- **Use merge, not overwrite.** A naive `echo '{...}' > ~/.reasonix/config.json` will wipe any other Reasonix settings the user has. Always merge via `jq` or the Python fallback.
- **Restart `reasonix` after changing the config.** An active session has the old config in memory; new values only apply to the next run.
- **Don't run `reasonix code` mid-config-write.** The interactive wizard may overwrite fields. Finish the procedure first, then launch.
- **Model slug must exist on FastRouter.** Use slugs from https://fastrouter.ai/models (DeepSeek-compatible models include `deepseek/deepseek-v3.2`, `deepseek/deepseek-r1`, etc.). A bad slug returns 404 from FastRouter, surfacing in Reasonix as a "model not found" error.
- **Redacted keys silently corrupt config.** Always run the redaction check before writing. Writing `[REDACTED]` (or pasting a masked key) as `apiKey` produces 401 errors later that look unrelated to redaction.
- **Never echo the full key back.** Only show it masked (last 4 chars). Some platforms log assistant output verbatim.

## Verification Checklist

Before reporting success, confirm:

- [ ] Exactly one key was extracted (Inputs rule)
- [ ] The key passed the redaction check (starts with `sk-v1-`, no `REDACTED` substring)
- [ ] `~/.reasonix/config.json` exists and contains all three required keys (`apiKey`, `baseUrl`, `model`)
- [ ] `baseUrl` equals `https://api.fastrouter.ai/api/v1` exactly
- [ ] File mode is `0600` (owner read/write only)
- [ ] Other pre-existing keys in the file were preserved (merge, not overwrite)
- [ ] You did NOT echo the raw API key — only masked form

## References

- DeepSeek Reasonix: https://github.com/esengine/DeepSeek-Reasonix
- FastRouter dashboard: https://dashboard.fastrouter.ai/
- FastRouter API base: `https://api.fastrouter.ai/api/v1`
- FastRouter models: https://fastrouter.ai/models
- FastRouter docs: https://docs.fastrouter.ai
````


# Kilo Code

Track usage, control costs, and add guardrails to your Kilo Code coding sessions

### What is Kilo Code?

[Kilo Code](https://kilocode.ai/) is an open-source AI coding agent for VS Code (with a CLI as well) that plans, writes, and fixes code across your project. It supports custom model providers, including any endpoint compatible with the OpenAI API standard—so it works with FastRouter out of the box.

By routing Kilo Code through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—including coding models like [Grok Code Fast 1](https://fastrouter.ai/models/x-ai/grok-code-fast-1)
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers adding FastRouter to Kilo Code as a custom OpenAI-compatible provider.

#### Prerequisites

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* VS Code with the [Kilo Code extension](https://kilocode.ai/) installed

***

### Quick Start

#### Step 1: Get Your FastRouter API Key

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

#### Step 2: Add FastRouter as a Custom Provider

1. In Kilo Code, open **Settings** (gear icon) and go to the **Providers** tab.

<figure><img src="/files/a4IcEP4RGROnxgGnh327" alt="" width="375"><figcaption></figcaption></figure>

2. Scroll to the bottom and click **Custom provider**.

3. Fill in the custom provider dialog:

4. **Provider ID:** `fastrouter`

* **Display name:** `FastRouter`
* **Provider API:** **OpenAI Compatible**
* **Base URL:** `https://api.fastrouter.ai/api/v1`
* **API key:** your FastRouter API key

<figure><img src="/files/ZcjpdM4RlXA7hxRB6VHZ" alt=""><figcaption></figcaption></figure>

#### Step 3: Select Your Models

Once the **Base URL** and **API key** are entered, Kilo Code queries FastRouter's models endpoint and presents a searchable model picker with the full catalog. Search with fuzzy matching (typing `grok` finds `x-ai/grok-code-fast-1`), select the models you want, and click **Submit**. The provider's models then appear in Kilo Code's model picker.

> **Note:** If auto-detection doesn't populate, enter FastRouter slugs manually in `provider/model-name` format—for example `x-ai/grok-code-fast-1`.

<figure><img src="/files/RbG4mCEOdcMOyFL6KqST" alt=""><figcaption></figcaption></figure>

#### Step 4: Start Coding

Pick a FastRouter model from the model picker and give Kilo Code a task. All requests route through FastRouter, and every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

***

### Use Kilo Code with 100+ Models

FastRouter uses the `provider/model-name` format. Add as many catalog models to the provider as you like and switch between them from the model picker:

```
x-ai/grok-code-fast-1
anthropic/claude-4.5-sonnet
openai/gpt-5.2
```

**Recommended models for Kilo Code:**

* `x-ai/grok-code-fast-1` — fast and inexpensive; a good default for routine edits
* `anthropic/claude-4.5-sonnet` — highest quality for complex, multi-file changes
* `openai/gpt-5.2` — strong all-round alternative

[Explore the full model catalog](https://fastrouter.ai/models)

#### Automatic Model Selection

Let FastRouter pick the best model for each request based on query complexity, domain, and cost by adding this slug to the provider's model list:

```
fastrouter/auto
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

### FAQs

#### Configuration & Setup

**The model picker doesn't auto-populate. What should I check?**

Confirm the **Base URL** is exactly `https://api.fastrouter.ai/api/v1` and the API key is correct—auto-detection queries FastRouter's models endpoint using both. If it still doesn't populate, add model IDs manually using full FastRouter slugs (including the provider prefix).

**I get "Model Not Found." What's wrong?**

The model ID doesn't match a FastRouter catalog slug. Use the full `provider/model-name` form—for example `x-ai/grok-code-fast-1`, not `grok-code-fast-1`.

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Add multiple models to the provider and switch from the model picker—no key changes needed.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

#### Costs & Budgeting

**Kilo Code can send large repo context. How do I control cost?**

Set a budget and rate limit on the key. The Dashboard breaks down costs by project, key, model, and tag.

#### Privacy & Security

**Is my code sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings.

***

### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# DeepSeek Reasonix CLI

Route DeepSeek Reasonix CLI through FastRouter for unified model access, observability, failover, and organizational controls.

### What is DeepSeek Reasonix CLI?

DeepSeek Reasonix CLI is an open-source, DeepSeek-native AI coding agent that runs in your terminal. It supports DeepSeek-compatible model providers, including FastRouter.

By routing DeepSeek Reasonix CLI through FastRouter, you get:

* **Unified API access** through a single, consistent endpoint
* **Provider failover and routing** for outages, rate limits, and provider availability
* **Unified observability** for requests, latency, costs, and usage
* **Bring Your Own Key (BYOK)** support for centralized provider-key management
* **Organization controls** for projects, keys, budgets, and usage governance

This guide covers installing DeepSeek Reasonix CLI, creating a FastRouter API key, configuring `~/.reasonix/config.json`, using the alternative installer, and installing the FastRouter Reasonix skill file. It does not cover general Reasonix usage outside FastRouter provider configuration.

#### Prerequisites

* Node.js and npm installed on your local machine
* A FastRouter account ([dashboard](https://dashboard.fastrouter.ai/))
* A FastRouter API key
* Terminal access to edit `~/.reasonix/config.json`

***

### Quick Start

#### Step 1: Install DeepSeek Reasonix CLI

Install Reasonix on your local machine. For platform-specific instructions, see the [DeepSeek Reasonix GitHub repository](https://github.com/esengine/DeepSeek-Reasonix).

```bash
npm install -g reasonix
```

#### Step 2: Create a FastRouter API Key

1. Sign up or log in to your FastRouter account.
2. Open the [FastRouter dashboard](https://dashboard.fastrouter.ai/).
3. Navigate to **API Keys**.
4. Create a new API key.
5. Copy and securely store your API key.

FastRouter API keys are only shown once. Store the key securely and do not commit it to source control.

#### Step 3: Configure DeepSeek Reasonix CLI to Use FastRouter

Edit `~/.reasonix/config.json` and add the FastRouter API key, base URL, and model.

```json
{
  "apiKey": "sk-v1-***",
  "baseUrl": "https://api.fastrouter.ai/api/v1",
  "model": "deepseek/deepseek-v3.2"
}
```

#### Step 4: Start DeepSeek Reasonix CLI

Start Reasonix from any project directory:

```bash
reasonix code
```

After configuration, Reasonix sends model requests through FastRouter.

***

### Alternative Install: Reasonix CLI with FastRouter

Use the FastRouter installer if you want a scripted setup. In the commands below, replace `$API_KEY` with your actual FastRouter API key.

#### One-Line Install Script

```bash
curl -fsSL https://fastrouter.ai/reasonix/install-fastrouter.sh | sh -s -- $API_KEY
```

#### Download, Inspect, Then Run

Use this option if you want to inspect the installer before running it.

```bash
curl -fsSL https://fastrouter.ai/reasonix/install-fastrouter.sh -o install-fastrouter.sh
```

```bash
less install-fastrouter.sh
```

```bash
chmod +x install-fastrouter.sh
```

```bash
./install-fastrouter.sh $API_KEY
```

***

### Install Reasonix with the FastRouter Skill File

Reasonix can install the FastRouter setup skill directly from the hosted skill file.

```bash
reasonix skills install https://fastrouter.ai/reasonix/skill.md
```

Then ask Reasonix to install FastRouter with your key:

```
install fastrouter in reasonix with key <KEY>
```

The skill file is available at <https://fastrouter.ai/reasonix/skill.md>.

***

### Use DeepSeek Reasonix CLI with FastRouter Models

Reasonix uses the `model` value in `~/.reasonix/config.json`. The default model in this guide is:

```
deepseek/deepseek-v3.2
```

To switch models, edit the `model` field in `~/.reasonix/config.json` and restart Reasonix. Use a valid FastRouter model slug from the [FastRouter models page](https://fastrouter.ai/models).

Example DeepSeek-compatible slugs include:

```
deepseek/deepseek-v3.2
deepseek/deepseek-r1
```

***

### Verify the Configuration

Check that Reasonix is installed:

```bash
reasonix --version
```

Check that `~/.reasonix/config.json` contains the FastRouter settings. Do not print the full file if it contains a real API key.

```bash
jq 'del(.apiKey)' ~/.reasonix/config.json
```

Expected values:

| Field     | Expected value                                   |
| --------- | ------------------------------------------------ |
| `baseUrl` | `https://api.fastrouter.ai/api/v1`               |
| `model`   | `deepseek/deepseek-v3.2`                         |
| `apiKey`  | A valid FastRouter key that starts with `sk-v1-` |

For security, restrict the config file to the file owner:

```bash
chmod 600 ~/.reasonix/config.json
```

***

### Why Use DeepSeek Reasonix CLI with FastRouter?

FastRouter centralizes access, routing, and governance for DeepSeek Reasonix CLI traffic.

#### Unified API

Access models from multiple AI providers through a single, consistent API interface.

#### Provider Failover & Routing

Automatically route requests to available providers and fail over during outages or rate limits.

#### Unified Observability

Monitor requests, latency, costs, and usage across all providers from a centralized dashboard.

#### Troubleshooting

Debug API requests, authentication, routing, and provider responses faster with FastRouter audit logs.

#### BYOK (Bring Your Own Key)

Use your own provider API keys while maintaining centralized routing and management.

#### Organization Controls

Manage organizations, projects, keys, and API usage with enterprise-grade governance controls.

***

### Troubleshooting

<table><thead><tr><th width="227.78125">Issue</th><th width="211.67578125">Cause</th><th width="300.625">Fix</th></tr></thead><tbody><tr><td><code>401</code> or authentication error</td><td>The API key is missing, invalid, or redacted.</td><td>Confirm <code>apiKey</code> starts with <code>sk-v1-</code> and replace masked values such as <code>***</code> or <code>[REDACTED]</code> with the real key.</td></tr><tr><td>Model not found</td><td>The <code>model</code> value is not a valid FastRouter slug.</td><td>Use a model slug from https://fastrouter.ai/models.</td></tr><tr><td>Reasonix still uses an old provider</td><td>An active Reasonix session loaded the previous config.</td><td>Restart <code>reasonix code</code> after editing <code>~/.reasonix/config.json</code>.</td></tr><tr><td>Config values disappeared</td><td>The config file was overwritten instead of merged.</td><td>Re-add the <code>apiKey</code>, <code>baseUrl</code>, and <code>model</code> keys while preserving other settings.</td></tr><tr><td>API key appears in logs</td><td>The full config file was printed or shared.</td><td>Rotate the API key in FastRouter, update <code>~/.reasonix/config.json</code>, and avoid using <code>cat ~/.reasonix/config.json</code>.</td></tr></tbody></table>

***

### Security Notes

* Reasonix stores `apiKey` in `~/.reasonix/config.json` in cleartext.
* Do not commit `~/.reasonix/config.json` to git.
* Use `chmod 600 ~/.reasonix/config.json` to restrict file access.
* Show masked API keys only. Do not echo the full key in logs, screenshots, or support messages.
* If you accidentally expose a key, rotate it in the FastRouter dashboard.

***

### FastRouter Reasonix Skill File Reference

You can find the `skill.md` file attached below, or at <https://fastrouter.ai/reasonix/skill.md>.

The skill configures **fastrouter** as the model provider in DeepSeek Reasonix CLI by writing `apiKey`, `baseUrl`, and `model` into `~/.reasonix/config.json`. After setup, FastRouter models are invoked when the user runs `reasonix code` from any project.

```yaml
---
name: reasonix-cli-fastrouter
description: "Install and configure FastRouter (https://fastrouter.ai) as the model provider in DeepSeek Reasonix CLI. Use this skill whenever the user asks to add, set up, install, configure, or register fastrouter as a provider in reasonix, edit ~/.reasonix/config.json, point reasonix or deepseek-reasonix at FastRouter, or wants to use FastRouter models with DeepSeek Reasonix CLI. The skill takes exactly one input: the FastRouter API key."
version: 1.0.0
author: FastRouter
license: MIT
metadata:
  reasonix:
    tags:
      [
        fastrouter,
        reasonix,
        deepseek,
        provider,
        setup,
        configuration,
        custom-provider,
        openai-compatible,
      ]
    homepage: https://fastrouter.ai
    docs: https://docs.fastrouter.ai
---
```

#### Inputs

This skill accepts **exactly one input**: the FastRouter API key. Everything else is hardcoded and must not be changed:

| Field         | Value                              |
| ------------- | ---------------------------------- |
| Provider      | `fastrouter`                       |
| Base URL      | `https://api.fastrouter.ai/api/v1` |
| Default model | `deepseek/deepseek-v3.2`           |
| Config file   | `~/.reasonix/config.json`          |
| File mode     | `0600` (key is stored on disk)     |

#### When to Use This Skill

Trigger this skill when the user asks any of:

* "install fastrouter for reasonix"
* "install fastrouter in reasonix with key "
* "set up / configure / register fastrouter in reasonix"
* "add fastrouter as a provider in reasonix"
* "edit `~/.reasonix/config.json` for fastrouter"
* "use fastrouter with deepseek reasonix cli"
* "point reasonix at fastrouter"
* "configure deepseek reasonix with fastrouter"

Anything similar referencing **reasonix** (or "deepseek reasonix") plus install / setup / configure / add / register / provider plus **fastrouter** should trigger it.

#### Extracting the API Key (single-argument rule)

Treat every invocation as a single-argument call: `setup(api_key)`. Find the key in the user's message using these rules, in order:

1. If the message contains `key=<value>`, `--key <value>`, `--api-key <value>`, or `apiKey=<value>`, the value is the key.
2. Else, the **last whitespace-separated token in the message that looks like an API key** is the key. A token "looks like an API key" if it matches `^sk-v1-[A-Za-z0-9_\-]+$` (FastRouter keys always have the prefix `sk-v1-` followed by a random string of letters/digits/dashes/underscores).
3. Else, ask the user **once**: *"Please paste your FastRouter API key (just the key, nothing else)."* Then treat their next message as the key verbatim, after stripping surrounding whitespace and quotes.

Examples of correct extraction:

| User message                                            | Extracted key        |
| ------------------------------------------------------- | -------------------- |
| `install fastrouter for reasonix with key sk-v1-abc123` | `sk-v1-abc123`       |
| `setup reasonix with fastrouter sk-v1-abc123`           | `sk-v1-abc123`       |
| `configure reasonix --api-key sk-v1-XYZ789`             | `sk-v1-XYZ789`       |
| `add fastrouter to reasonix key=sk-v1-9aBc_def-456`     | `sk-v1-9aBc_def-456` |
| `set up reasonix with fastrouter`                       | (prompt the user)    |

Once extracted, **never modify, paraphrase, or transform the key**. Do not strip characters you think are formatting. Do not lowercase. Do not echo the full key back to the user - only ever show it masked (`****` + last 4 chars) in confirmations.

#### Redaction Check (do this before writing)

Some agent harnesses (Cursor, IDE extensions, CI loggers) may redact secrets in messages before the model sees them. After extracting the key, validate it. The key is **invalid for write** if any of:

* Equals `[REDACTED]`, `<redacted>`, `***`, `***REDACTED***`, or any string containing the substring `REDACTED` (case-insensitive)
* Does not start with the literal prefix `sk-v1-` (all real FastRouter keys begin with `sk-v1-`; anything else is either redacted, mistyped, or from a different provider)
* Has nothing after the `sk-v1-` prefix (i.e. the random portion is empty)
* Contains only `*` or `•` characters
* Is empty / whitespace-only

If the key fails validation due to redaction, **stop and tell the user**:

> "It looks like your environment is masking the API key before I can read it. To install fastrouter for Reasonix, please run these in your terminal directly so the key never passes through the model:
>
> ```bash
> mkdir -p ~/.reasonix
> cat > ~/.reasonix/config.json <<'EOF'
> {
>   "apiKey": "<YOUR_KEY>",
>   "baseUrl": "https://api.fastrouter.ai/api/v1",
>   "model": "deepseek/deepseek-v3.2"
> }
> EOF
> chmod 600 ~/.reasonix/config.json
> ```
>
> If `~/.reasonix/config.json` already exists with other settings, edit it manually and merge the three keys above instead of overwriting."

Do **not** proceed with a redacted-looking value - writing `[REDACTED]` as `apiKey` silently breaks the provider with 401 errors that look unrelated to redaction.

#### Procedure

Once you have a valid key, follow these steps in order.

**Step 1: Ensure Reasonix CLI is installed**

Run via the `terminal` tool:

```bash
reasonix --version
```

If `reasonix` is not found, install it first:

```bash
npm install -g reasonix
reasonix --version
```

Reference: <https://github.com/esengine/DeepSeek-Reasonix>.

**Step 2: Ensure `~/.reasonix/config.json` exists**

```bash
mkdir -p ~/.reasonix
[ -f ~/.reasonix/config.json ] || echo '{}' > ~/.reasonix/config.json
```

If the user prefers an interactive setup, they can instead run `reasonix code` once and paste the API key when prompted — Reasonix's built-in wizard generates the file automatically. Either path produces the same end state, but only proceed past this step once `~/.reasonix/config.json` exists.

**Step 3: Write the FastRouter config**

Reasonix config is a flat JSON object. Merge these three keys, **preserving any other existing keys** in the file. Use the extracted key exactly as-is — never rename, lowercase, or transform it.

Required keys:

```json
{
  "apiKey": "<USER_KEY>",
  "baseUrl": "https://api.fastrouter.ai/api/v1",
  "model": "deepseek/deepseek-v3.2"
}
```

Safe merge with `jq` (preferred — preserves other keys):

```bash
KEY="<USER_KEY>"
jq --arg key "$KEY" \
   '. + {apiKey: $key, baseUrl: "https://api.fastrouter.ai/api/v1", model: "deepseek/deepseek-v3.2"}' \
   ~/.reasonix/config.json > ~/.reasonix/config.json.tmp \
   && mv ~/.reasonix/config.json.tmp ~/.reasonix/config.json
```

Fallback merge with Python (when `jq` is unavailable):

```bash
python3 - "<USER_KEY>" <<'PY'
import json, os, sys
key = sys.argv[1]
path = os.path.expanduser("~/.reasonix/config.json")
try:
    with open(path) as f: cfg = json.load(f)
except Exception:
    cfg = {}
cfg["apiKey"]  = key
cfg["baseUrl"] = "https://api.fastrouter.ai/api/v1"
cfg["model"]   = "deepseek/deepseek-v3.2"
with open(path, "w") as f: json.dump(cfg, f, indent=2)
PY
```

**Step 4: Restrict file permissions**

The API key is stored on disk in cleartext. Make the file owner-only:

```bash
chmod 600 ~/.reasonix/config.json
```

**Step 5: Verify**

```bash
ls -l ~/.reasonix/config.json
jq 'del(.apiKey)' ~/.reasonix/config.json
jq -r '.apiKey | "apiKey ends in: ****" + .[-4:]' ~/.reasonix/config.json
```

Expected:

* File mode is `-rw-------` (octal `0600`)
* `baseUrl` is `"https://api.fastrouter.ai/api/v1"` exactly
* `model` is `"deepseek/deepseek-v3.2"` (or the user-specified slug)
* Masked key suffix matches the last 4 chars the user provided

**Never `cat` or otherwise dump the full file** — the `apiKey` is in cleartext. Always use the `jq 'del(.apiKey)'` form when showing config.

**Step 6: Report success**

Reply to the user with usage examples. Show the API key **only masked**.

> ✓ FastRouter is configured for DeepSeek Reasonix CLI (`****<last 4 chars>`).
>
> Start Reasonix from any project:
>
> ```bash
> cd /path/to/your/project
> reasonix code
> ```
>
> Switch models by editing `model` in `~/.reasonix/config.json`. Browse available DeepSeek slugs at <https://fastrouter.ai/models>.

**Step 7: Optional model wiring**

If (and only if) the user named a specific model in their request, set it as `model` instead of the default `deepseek/deepseek-v3.2`:

```bash
jq --arg m "<that_model>" '.model = $m' \
   ~/.reasonix/config.json > ~/.reasonix/config.json.tmp \
   && mv ~/.reasonix/config.json.tmp ~/.reasonix/config.json
```

Otherwise leave `"model": "deepseek/deepseek-v3.2"` untouched.

#### Pitfalls

* **The API key IS stored in the config file.** Reasonix reads `apiKey` directly from `~/.reasonix/config.json` in cleartext (no env-var indirection). Never commit this file to git or share it. Always `chmod 600` after writing. Add `.reasonix/` and `**/.reasonix/config.json` to global `~/.gitignore` if the user version-controls dotfiles.
* **Don't `cat ~/.reasonix/config.json`** when output may be streamed to logs, chat, or terminal recordings — the key is in cleartext. Use `jq 'del(.apiKey)'` for inspection.
* **Use merge, not overwrite.** A naive `echo '{...}' > ~/.reasonix/config.json` will wipe any other Reasonix settings the user has. Always merge via `jq` or the Python fallback.
* **Restart `reasonix` after changing the config.** An active session has the old config in memory; new values only apply to the next run.
* **Don't run `reasonix code` mid-config-write.** The interactive wizard may overwrite fields. Finish the procedure first, then launch.
* **Model slug must exist on FastRouter.** Use slugs from <https://fastrouter.ai/models> (DeepSeek-compatible models include `deepseek/deepseek-v3.2`, `deepseek/deepseek-r1`, etc.). A bad slug returns 404 from FastRouter, surfacing in Reasonix as a "model not found" error.
* **Redacted keys silently corrupt config.** Always run the redaction check before writing. Writing `[REDACTED]` (or pasting a masked key) as `apiKey` produces 401 errors later that look unrelated to redaction.
* **Never echo the full key back.** Only show it masked (last 4 chars). Some platforms log assistant output verbatim.

#### Verification Checklist

Before reporting success, confirm:

* [ ] Exactly one key was extracted (Inputs rule)
* [ ] The key passed the redaction check (starts with `sk-v1-`, no `REDACTED` substring)
* [ ] `~/.reasonix/config.json` exists and contains all three required keys (`apiKey`, `baseUrl`, `model`)
* [ ] `baseUrl` equals `https://api.fastrouter.ai/api/v1` exactly
* [ ] File mode is `0600` (owner read/write only)
* [ ] Other pre-existing keys in the file were preserved (merge, not overwrite)
* [ ] You did NOT echo the raw API key - only masked form

***

### References

* DeepSeek Reasonix: <https://github.com/esengine/DeepSeek-Reasonix>
* FastRouter dashboard: <https://dashboard.fastrouter.ai/>
* FastRouter API base: `https://api.fastrouter.ai/api/v1`
* FastRouter models: <https://fastrouter.ai/models>
* FastRouter docs: <https://docs.fastrouter.ai>
* FastRouter Reasonix skill file: <https://fastrouter.ai/reasonix/skill.md>


# XCode

Track usage, control costs, and add guardrails to your Xcode AI coding sessions.

### What is Xcode?

[Xcode](https://developer.apple.com/xcode/) is Apple's IDE for building apps across Apple platforms. Its **Apple Intelligence** coding assistant (Xcode 26 and later) lets you add custom model providers under **Settings → Intelligence**, so you can chat with, and generate code from, models beyond the built-in options. Any provider that speaks an OpenAI-compatible API can be added—including FastRouter.

By routing Xcode through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—including coding models like [Grok Code Fast 1](https://fastrouter.ai/models/x-ai/grok-code-fast-1)
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers adding FastRouter as a model provider in Xcode's Intelligence settings.

#### Prerequisites

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Xcode 26 or later on a Mac with **Apple Intelligence** enabled

***

### Quick Start

#### Step 1: Enable Apple Intelligence

Go to **macOS Settings → Apple Intelligence & Siri** and make sure **Apple Intelligence** is turned on. This is required before Xcode can use any AI model providers.

<figure><img src="/files/uXO30ELOieQekDqY5lqV" alt=""><figcaption></figcaption></figure>

#### Step 2: Get Your FastRouter API Key

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

#### Step 3: Add FastRouter as a Model Provider

1. Open **Xcode → Settings → Intelligence**.

<figure><img src="/files/rp9NNWZehOL2Ea3UYj2o" alt=""><figcaption></figcaption></figure>

2. Click **Add a Model Provider** at the bottom of the page.

<figure><img src="/files/sMvYNeV7TjqdunEkHdFX" alt=""><figcaption></figcaption></figure>

3. In the dialog, enter the following details:
   * **URL:** `https://api.fastrouter.ai/api/`
   * **API Key Header:** `Authorization`
   * **API Key:** `Bearer sk-add-your-key-here` (include the literal `Bearer` prefix before your FastRouter key)
   * **Description:** `FastRouter` (or any name you prefer)

<figure><img src="/files/2JdvY7tbh69vagPSA6At" alt=""><figcaption></figcaption></figure>

5. Click Save to save the configuration.

> **Important:** Do not add `/v1` to the end of the endpoint. For Xcode provider setup, use `https://api.fastrouter.ai/api/`, not the direct API endpoint `https://api.fastrouter.ai/api/v1`.

> **Note:** Enter the API key with the `Bearer` prefix because the **API Key Header** is `Authorization`—Xcode sends the field's full value as the header, so it must be `Bearer <your-key>`.

<figure><img src="/files/ZEC7RjHcQHu1IJZVPFcV" alt=""><figcaption></figcaption></figure>

#### Step 4: Browse and Select Models

Once configured, select **FastRouter** in the provider list to see available models. If Xcode does not populate the list automatically, or you want a specific model, enter the FastRouter slug manually in `provider/model-name` format for example `x-ai/grok-code-fast-1`. Use **Favorites** to pin the models you use most; pinned models appear at the top for quick access.

<figure><img src="/files/RpcRV5VFxKTRjcSyYa2t" alt=""><figcaption></figcaption></figure>

#### Step 5: Start Using AI in Xcode

Open the chat interface and start working with your selected model. All requests route through FastRouter, and every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

<figure><img src="/files/F3dKT86IeqtjSbMgNOaF" alt=""><figcaption><p>XCode using Fable 5</p></figcaption></figure>

***

### Use Xcode with 100+ Models

FastRouter uses the `provider/model-name` format. Switch models from the chat model dropdown, or add FastRouter slugs to your Favorites:

```
# Anthropic Claude
anthropic/claude-4.5-sonnet

# OpenAI
openai/gpt-5.2
```

**Recommended models for Xcode:**

* `x-ai/grok-code-fast-1` — fast and inexpensive; a good default for routine edits
* `anthropic/claude-4.5-sonnet` — highest quality for complex, multi-file changes
* `openai/gpt-5.2` — strong all-round alternative

[Explore the full model catalog](https://fastrouter.ai/models)

#### Automatic Model Selection

Let FastRouter pick the best model for each request based on query complexity, domain, and cost by entering this slug as the model:

```
fastrouter/auto
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

### FAQs

#### Configuration & Setup

**No models appear after I add FastRouter. What should I check?**

Confirm the **URL** is exactly `https://api.fastrouter.ai/api/`, the **API Key Header** is `Authorization`, and the **API Key** field includes the `Bearer` prefix before your key. Do not add `/v1` to the Xcode provider URL. If the model list still doesn't populate, enter a known slug manually (for example `x-ai/grok-code-fast-1`) and verify your key has access to it.

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Switch models from the chat dropdown or add more Favorites—no key changes needed.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

#### Costs & Budgeting

**How do I keep spend predictable?**

Set a budget and rate limit on the key. The Dashboard breaks down costs by project, key, model, and tag.

#### Privacy & Security

**Is my code sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings.

***

### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# Agents

Route any Python agent framework through FastRouter. One key, 100+ models,   and full cost visibility across every agent call.

Agent workloads fan out into many LLM calls per run, so cost and reliability matter more here than anywhere else. Every framework below accepts an OpenAI-compatible client, which is all FastRouter needs. Point it at FastRouter and each agent call routes through one endpoint with:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, and Mistral behind one key. Assign a cheap model to triage agents and a frontier model to reasoning-heavy ones without juggling provider credentials.
* **Observability** on every request: cost, tokens, latency, and model per call, visible in your [dashboard](https://dashboard.fastrouter.ai/).
* **Reliability** through automatic failover across providers, response caching, and intelligent routing.
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation.

The pattern is the same everywhere: create an OpenAI-compatible client with FastRouter's base URL, pass it a model slug in `provider/model-name` format.

<table data-view="cards"><thead><tr><th></th><th data-type="content-ref"></th></tr></thead><tbody><tr><td></td><td><a href="/pages/mFDrx7EkmsNXzrYXi6XC">/pages/mFDrx7EkmsNXzrYXi6XC</a></td></tr><tr><td></td><td><a href="/pages/MGWE0sfoXFivjN7yWhk8">/pages/MGWE0sfoXFivjN7yWhk8</a></td></tr><tr><td></td><td><a href="/pages/P8qgyH8icOsEX0wmnzfn">/pages/P8qgyH8icOsEX0wmnzfn</a></td></tr><tr><td></td><td><a href="/pages/90mSHPYmV2HUxRXrsJZr">/pages/90mSHPYmV2HUxRXrsJZr</a></td></tr><tr><td></td><td><a href="/pages/Caay2cYNsREWz5Mdq5cL">/pages/Caay2cYNsREWz5Mdq5cL</a></td></tr></tbody></table>


# AutoGen

Track usage, control costs, and add guardrails to your AutoGen multi-agent apps

### What is AutoGen?

[AutoGen](https://microsoft.github.io/autogen/) is a framework from Microsoft for building multi-agent AI applications, where multiple conversational agents collaborate to solve tasks. It supports any OpenAI-compatible model through its client configuration.

By routing AutoGen through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—assign different models to different agents with one key
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting AutoGen (Python) to FastRouter using the OpenAI chat completion client.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Python 3.10 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once:

```bash
mkdir my_project
cd my_project
python -m venv .venv
```

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

```bash
source .venv/bin/activate
```

On Windows:

```bash
.venv\Scripts\activate
```

**Step 2: Install AutoGen**

```bash
pip install -U "autogen-agentchat" "autogen-ext[openai]"
```

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

```bash
export FASTROUTER_API_KEY=sk-add-your-key-here
```

**Step 4: Configure the Model Client for FastRouter**

AutoGen's `OpenAIChatCompletionClient` accepts a custom base URL. Save this as `autogen_example.py`:

```python
import os
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

model_client = OpenAIChatCompletionClient(
    model="openai/gpt-5.2",
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
    model_info={
        "vision": False,
        "function_calling": True,
        "json_output": True,
        "family": "unknown",
        "structured_output": True,
    },
)
import os
import asyncio
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient

model_client = OpenAIChatCompletionClient(
    model="openai/gpt-5.2",
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
    model_info={
        "vision": False,
        "function_calling": True,
        "json_output": True,
        "family": "unknown",
        "structured_output": True,
    },
)

agent = AssistantAgent("assistant", model_client=model_client)


async def main():
    result = await agent.run(task="Explain what an LLM gateway does in one sentence.")
    print(result.messages[-1].content)


if __name__ == "__main__":
    asyncio.run(main())
agent = AssistantAgent("assistant", model_client=model_client)


async def main():
    result = await agent.run(task="Explain what an LLM gateway does in one sentence.")
    print(result.messages[-1].content)


if __name__ == "__main__":
    asyncio.run(main())
```

> **Note:** For non-OpenAI model slugs, AutoGen requires the `model_info` block shown above so it knows the model's capabilities.

**Step 5: Run the Agent**

```bash
python autogen_example.py
```

<figure><img src="/files/asvSrU9H8kWbt6x8EF03" alt=""><figcaption></figcaption></figure>

The agent responds, and the request appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/) with token usage and cost.

***

#### Use AutoGen with 100+ Models

FastRouter uses the `provider/model-name` format. Switch providers by changing the model slug. Give each agent in a multi-agent team its own client—a fast model for worker agents and a frontier model for the orchestrator:

```python
model="anthropic/claude-4.5-sonnet"
```

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```python
model="fastrouter/auto"
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Each agent can have its own model client, all sharing one key.

**Why do I need the `model_info` block?**

AutoGen looks up model capabilities by name. Because FastRouter slugs aren't in AutoGen's built-in table, you declare capabilities explicitly with `model_info`.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Costs & Budgeting**

**Multi-agent conversations generate many calls. How do I control cost?**

Set budgets and rate limits on the key, and use Dynamic Tags to attribute spend per team or run. The Dashboard breaks down costs by project, key, model, and tag.

**Performance & Reliability**

**Does FastRouter add latency?**

FastRouter adds near-zero gateway overhead, negligible compared to model inference time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# Agno

Track usage, control costs, and add guardrails to your Agno agents

### What is Agno?

[Agno](https://www.agno.com/) (formerly Phidata) is a high-performance framework for building multi-agent systems with memory, knowledge, and tools. It is known for extremely fast agent instantiation and a clean, minimal API surface.

By routing Agno through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint, via Agno's built-in `OpenAILike` model class
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting Agno (Python) to FastRouter, running an agent, and adding tools.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Python 3.9 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once:

```bash
mkdir my_project
cd my_project
python -m venv .venv
```

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

```bash
source .venv/bin/activate
```

On Windows:

```bash
.venv\Scripts\activate
```

**Step 2: Install Agno**

```bash
pip install agno openai
```

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

```bash
export FASTROUTER_API_KEY=sk-add-your-key-here
```

**Step 4: Use the `OpenAILike` Model Class**

Agno ships an `OpenAILike` model class made for OpenAI-compatible gateways like FastRouter. Save this as `agno_example.py`:

```python
import os
from agno.agent import Agent
from agno.models.openai.like import OpenAILike

agent = Agent(
    model=OpenAILike(
        id="openai/gpt-5.2",
        base_url="https://api.fastrouter.ai/api/v1",
        api_key=os.environ["FASTROUTER_API_KEY"],
    ),
    markdown=True,
)

agent.print_response("Explain what an LLM gateway does in one sentence.")
```

**Step 5: Run the Agent**

```bash
python agno_example.py
```

Agno prints a formatted response panel in your terminal:

<figure><img src="/files/mFOD9psWeCSvm7FM2mCM" alt=""><figcaption></figcaption></figure>

The request appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/) with token usage and cost.

***

#### Adding Tools

The FastRouter-backed model works with Agno tools. Pass any Python function with a docstring and type hints:

```python
import os
from agno.agent import Agent
from agno.models.openai.like import OpenAILike


def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is 72°F and sunny."


agent = Agent(
    model=OpenAILike(
        id="openai/gpt-5.2",
        base_url="https://api.fastrouter.ai/api/v1",
        api_key=os.environ["FASTROUTER_API_KEY"],
    ),
    tools=[get_weather],
)

result = agent.run("What is the weather in Paris?")
print(result.content)
# The weather in Paris is currently 72°F and sunny.
```

***

#### Use Agno with 100+ Models

FastRouter uses the `provider/model-name` format. Change the `id` to any FastRouter model slug:

```python
model=OpenAILike(
    id="anthropic/claude-4.5-sonnet",
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
)
```

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```python
model=OpenAILike(
    id="fastrouter/auto",
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
)
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

**Flex Pricing for Background Agents**

Agents running scheduled or background jobs tolerate higher latency. Append `:flex` to any model to access up to 50% lower token costs:

```python
id="openai/gpt-5.2:flex"
```

> **Note:** Flex is not recommended for interactive agents where you need low-latency responses.

[Compare flex pricing options](https://docs.fastrouter.ai/explore-features/flex-pricing)

***

#### FAQs

**Configuration & Setup**

**I get `ModuleNotFoundError: No module named 'openai'`. What's missing?**

Agno's OpenAI-compatible models require the OpenAI SDK: `pip install openai`.

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Each `OpenAILike` instance selects its own model, so different agents can use different providers with one key.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Tool runs fail with an `Invalid type ... arguments` error from the API. What's wrong?**

Some model routes currently return tool-call `arguments` as a JSON object instead of the JSON-encoded string the OpenAI specification requires, which breaks the tool-call round trip. Plain chat responses are unaffected. If you hit this error, switch to a model verified for tool calling through FastRouter, such as `openai/gpt-5.2` or `anthropic/claude-4.5-sonnet`.

**Costs & Budgeting**

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit. The agent receives an error response indicating the budget has been exceeded.

**Performance & Reliability**

**Does FastRouter add latency to agent runs?**

FastRouter adds near-zero gateway overhead. For most workloads, this is negligible compared to model inference time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# CrewAI

Track usage, control costs, and add guardrails to your CrewAI multi-agent crews

### What is CrewAI?

[CrewAI](https://www.crewai.com/) is a leading open-source framework for orchestrating role-playing, autonomous AI agents. You define agents with roles, goals, and backstories, assign them tasks, and CrewAI coordinates them as a "crew" to accomplish multi-step workflows.

Multi-agent crews generate a high volume of LLM calls, which makes cost visibility and control critical. By routing CrewAI through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—assign different providers to different agents with one key
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting CrewAI (1.x) to FastRouter, running a crew, and assigning different models to different agents.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Python 3.10 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once. CrewAI requires Python 3.10–3.13:

```bash
mkdir my_project
cd my_project
python -m venv .venv
```

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

```bash
source .venv/bin/activate
```

On Windows:

```bash
.venv\Scripts\activate
```

**Step 2: Install CrewAI**

```bash
pip install crewai
```

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

```bash
export FASTROUTER_API_KEY=sk-add-your-key-here
```

**Step 4: Configure the CrewAI `LLM` to Use FastRouter**

Set `provider="openai"` so CrewAI uses its OpenAI-compatible client and passes the FastRouter model slug through verbatim. Save this as `crew_example.py`:

```python
import os
from crewai import Agent, Task, Crew, LLM

# provider="openai" makes CrewAI use the OpenAI-compatible protocol and
# pass the FastRouter model slug through verbatim
llm = LLM(
    model="openai/gpt-5.2",
    provider="openai",
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
)

researcher = Agent(
    role="Tech Researcher",
    goal="Summarize technical topics clearly",
    backstory="You are an experienced technology analyst.",
    llm=llm,
    verbose=True,
)

task = Task(
    description="Write a two-sentence summary of what an LLM gateway does.",
    expected_output="A two-sentence summary.",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task])
result = crew.kickoff()
print(result)
```

> **Note:** `provider="openai"` is required. Without it, CrewAI parses the model string itself and strips the provider prefix (sending `gpt-5.2` instead of `openai/gpt-5.2`), or—for non-OpenAI slugs—tries to use that provider's native SDK instead of FastRouter.

**Step 5: Run the Crew**

```bash
python crew_example.py
```

CrewAI prints the agent's progress (because `verbose=True`) followed by the final answer:

<figure><img src="/files/09xUnwXKi6Gs7oHhCfwj" alt=""><figcaption></figcaption></figure>

Every LLM call from every agent appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/) with token usage and cost.

***

#### Use CrewAI with 100+ Models

With `provider="openai"` set, any FastRouter slug works as-is—including non-OpenAI providers. A common pattern is to give cheap, fast models to high-volume worker agents and a frontier model to the planner, with no extra credentials:

```python
fast_llm = LLM(
    model="openai/gpt-4.1-nano",
    provider="openai",
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
)

smart_llm = LLM(
    model="anthropic/claude-4.5-sonnet",
    provider="openai",
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
)

worker = Agent(role="Researcher", goal="...", backstory="...", llm=fast_llm)
planner = Agent(role="Lead Strategist", goal="...", backstory="...", llm=smart_llm)
```

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```python
llm = LLM(
    model="fastrouter/auto",
    provider="openai",
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
)
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

**Flex Pricing for Unattended Crews**

Crews that run unattended—research pipelines, content generation, scheduled jobs—tolerate higher latency. Append `:flex` to any model to access up to 50% lower token costs:

```python
model="openai/gpt-5.2:flex"
```

> **Note:** Flex is not recommended for crews where you are waiting on the output interactively.

[Compare flex pricing options](https://docs.fastrouter.ai/explore-features/flex-pricing)

***

#### FAQs

**Configuration & Setup**

**I get `ImportError: Unable to initialize LLM with model '...'` mentioning LiteLLM. Do I need LiteLLM?**

No. This error means the model string didn't match a provider CrewAI recognizes natively. Add `provider="openai"` to your `LLM(...)` call and pass the FastRouter slug in `model`—no LiteLLM install needed.

**I get `400 - There is no available model provider that meets your routing requirements`. What's wrong?**

The model slug reaching FastRouter is malformed. Ensure `provider="openai"` is set and `model` is an exact FastRouter catalog slug (for example, `openai/gpt-5.2`).

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Each agent can have its own `LLM` instance with a different model, all sharing one key.

**Costs & Budgeting**

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit. The crew receives an error response indicating the budget has been exceeded.

**How do I track spending per crew or per team?**

Create separate projects for each team, issue project-scoped API keys, and use Dynamic Tags for finer-grained attribution. The Dashboard breaks down costs by project, key, model, and tag.

**Performance & Reliability**

**Does FastRouter add latency to crew runs?**

FastRouter adds near-zero gateway overhead. For multi-step crew executions, this is negligible compared to model inference time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# Langgraph

Track usage, control costs, and add guardrails to your LangGraph agents

### What is LangGraph?

[LangGraph](https://www.langchain.com/langgraph) is a framework for building stateful, multi-step agents as graphs. Built by the LangChain team, it adds durable state, cycles, branching, and human-in-the-loop control on top of LangChain's model interfaces.

By routing LangGraph through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting LangGraph (Python) to FastRouter using a `ChatOpenAI` model inside a prebuilt ReAct agent.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Python 3.10 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once:

```bash
mkdir my_project
cd my_project
python -m venv .venv
```

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

```bash
source .venv/bin/activate
```

On Windows:

```bash
.venv\Scripts\activate
```

**Step 2: Install LangGraph**

```bash
pip install langchain langgraph langchain-openai
```

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

```bash
export FASTROUTER_API_KEY=sk-add-your-key-here
```

**Step 4: Point the Model at FastRouter**

LangGraph uses LangChain chat models, so the standard `ChatOpenAI` class works once you override the base URL. Save this as `graph_example.py`:

```python
import os
from langchain_openai import ChatOpenAI
from langchain.agents import create_agent

llm = ChatOpenAI(
    base_url="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
    model="openai/gpt-5.2",
)


def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"The weather in {city} is 72°F and sunny."


agent = create_agent(llm, tools=[get_weather])

result = agent.invoke({"messages": [{"role": "user", "content": "What's the weather in San Francisco?"}]})
print(result["messages"][-1].content)
```

**Step 5: Run the Agent**

```bash
python graph_example.py
```

<figure><img src="/files/ivket4jd6KmH5NOw9EO0" alt=""><figcaption></figcaption></figure>

The agent calls the `get_weather` tool and responds with the weather. Every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

***

#### Use LangGraph with 100+ Models

FastRouter uses the `provider/model-name` format. Switch providers by changing the model slug—no new dependencies or credentials:

```python
# Anthropic Claude
model="anthropic/claude-4.5-sonnet"

# Google Gemini
model="google/gemini-3.1-pro-preview"
```

You can give each node in your graph a different model—a fast model for routing nodes and a frontier model for reasoning nodes.

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```python
model="fastrouter/auto"
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Each node can use its own `ChatOpenAI` instance with a different model, all sharing one key.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Costs & Budgeting**

**Graphs with cycles can make many model calls. How do I control cost?**

Set a budget and rate limit on the key, and use Dynamic Tags to attribute spend per graph or per run. The Dashboard breaks down costs by project, key, model, and tag.

**Performance & Reliability**

**Does FastRouter add latency to graph execution?**

FastRouter adds near-zero gateway overhead, negligible compared to model inference time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# OpenAI Agent SDK

Track usage, control costs, and add guardrails to your OpenAI Agents SDK applications

### What is the OpenAI Agents SDK?

The [OpenAI Agents SDK](https://openai.github.io/openai-agents-python/) is OpenAI's official framework for building agentic AI applications in Python. It provides a lightweight set of primitives—Agents, Tools, Handoffs, and Guardrails—for building multi-step, tool-using agents with minimal boilerplate.

By routing the Agents SDK through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—your agents are no longer locked to OpenAI models
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting the Python Agents SDK to FastRouter, running a tool-calling agent, and model selection. It does not cover the TypeScript SDK.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai/))
* Python 3.10 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once:

{% code overflow="wrap" %}

```
mkdir my_projectcd my_projectpython -m venv .venv
```

{% endcode %}

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

{% code overflow="wrap" %}

```
source .venv/bin/activate
```

{% endcode %}

On Windows:

{% code overflow="wrap" %}

```
.venv\Scripts\activate
```

{% endcode %}

**Step 2: Install the OpenAI Agents SDK**

{% code overflow="wrap" %}

```
pip install openai-agents
```

{% endcode %}

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai/)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

{% code overflow="wrap" %}

```
export FASTROUTER_API_KEY=sk-add-your-key-here
```

{% endcode %}

**Step 4: Point the SDK at FastRouter**

The Agents SDK accepts any OpenAI-compatible client. Create an `AsyncOpenAI` client with FastRouter's base URL and pass it to your agent's model. Save this as `agent_example.py`:

{% code overflow="wrap" %}

```
import osimport asynciofrom openai import AsyncOpenAIfrom agents import Agent, OpenAIChatCompletionsModel, Runner, function_tool, set_tracing_disabled
# Point the OpenAI client at FastRouterclient = AsyncOpenAI(    base_url="https://api.fastrouter.ai/api/v1",    api_key=os.environ["FASTROUTER_API_KEY"],)set_tracing_disabled(True)  # tracing uploads to OpenAI; disable when not using an OpenAI key

@function_tooldef get_weather(city: str) -> str:    """Get the current weather for a city."""    return f"The weather in {city} is 72°F and sunny."

agent = Agent(    name="Weather Assistant",    instructions="You are a helpful assistant. Use tools when needed.",    model=OpenAIChatCompletionsModel(model="openai/gpt-5.2", openai_client=client),    tools=[get_weather],)

async def main():    result = await Runner.run(agent, "What's the weather in San Francisco?")    print(result.final_output)

if __name__ == "__main__":    asyncio.run(main())
```

{% endcode %}

> **Note:** `set_tracing_disabled(True)` is recommended because the SDK's built-in tracing exports to OpenAI's platform and requires a native OpenAI key. All FastRouter requests remain fully visible in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

**Step 5: Run the Agent**

{% code overflow="wrap" %}

```
python agent_example.py
```

{% endcode %}

The agent calls the `get_weather` tool and responds:

\<img src="../.gitbook/assets/fastrouter-openai-agents-sdk-terminal.png" alt="OpenAI Agents SDK running a tool-calling agent through FastRouter" height="214" width="608">

Every request, token count, and cost now appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

***

#### Use the Agents SDK with 100+ Models

FastRouter uses the `provider/model-name` format. Switching your agent to a different provider is a one-line change:

{% code overflow="wrap" %}

```
# Anthropic Claudemodel=OpenAIChatCompletionsModel(model="anthropic/claude-4.5-sonnet", openai_client=client)
# DeepSeekmodel=OpenAIChatCompletionsModel(model="deepseek/deepseek-v4-pro", openai_client=client)
```

{% endcode %}

You can give each agent in a multi-agent workflow a different model—a fast, inexpensive model for triage agents and a frontier model for reasoning-heavy agents—all through the same client and key.

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

{% code overflow="wrap" %}

```
model=OpenAIChatCompletionsModel(model="fastrouter/auto", openai_client=client)
```

{% endcode %}

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

**Flex Pricing for Batch Agent Workloads**

Agent pipelines that run unattended—evaluation sweeps, data extraction, report generation—tolerate higher latency. Append `:flex` to any model to access up to 50% lower token costs:

{% code overflow="wrap" %}

```
model=OpenAIChatCompletionsModel(model="openai/gpt-5.2:flex", openai_client=client)
```

{% endcode %}

> **Note:** Flex is not recommended for interactive sessions where you need low-latency responses.

[Compare flex pricing options](https://docs.fastrouter.ai/explore-features/flex-pricing)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. The model is selected per agent in code, and you can switch models at any time without changing your key.

**Why do I see tracing errors or `OPENAI_API_KEY` warnings?**

The SDK's built-in tracing exports to OpenAI's platform, which requires a native OpenAI key. Call `set_tracing_disabled(True)` after creating your client. Your request telemetry is still captured in the FastRouter Dashboard.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Tool calls crash with `ValidationError: ... arguments — Input should be a valid string`. What's wrong?**

Some model routes currently return tool-call `arguments` as a JSON object instead of the JSON-encoded string the OpenAI specification requires, and the Agents SDK validates this field strictly. If you hit this error, switch to a model verified for tool calling through FastRouter, such as `openai/gpt-5.2` or `anthropic/claude-4.5-sonnet`.

**Costs & Budgeting**

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit. The agent receives an error response indicating the budget has been exceeded.

**Performance & Reliability**

**Does FastRouter add latency to agent runs?**

FastRouter adds near-zero gateway overhead. For multi-step agent runs, this is negligible compared to model inference time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# OpenClaw

This integration assumes you already have a valid chat running on OpenClaw with any supported provider.

***

#### Installation via ClawHub Skill

1. Install the latest version of the `fastrouter-setup` skill from ClawHub:

   ```
   openclaw skills install fastrouter-setup
   ```
2. Verify that the `fastrouter-setup` skill is present in your OpenClaw skills folder.
3. Restart your OpenClaw gateway.
4. In your chat, run:

   ```
   set up fastrouter with the api key sk-v1-xxxxxx
   ```

***

#### Reference

You can find the `skill.md` file:

* Attached below, or
* At: <https://fastrouter.ai/skills/openclaw/skill.md>

````xml
---
name: fastrouter-setup
description: "Add the FastRouter AI provider to OpenClaw with all available text and vision models, fetched live from the FastRouter API. Use when: (1) a user wants to set up FastRouter as a model provider, (2) a user says 'set up fastrouter', 'add fastrouter provider', or 'configure fastrouter' along with an API key, (3) a user wants to refresh/update their FastRouter model list. Triggers on phrases like 'set up fastrouter with API key sk-v1-xxxxx', 'add fastrouter provider', 'configure fastrouter', or 'update fastrouter models'."
---

# FastRouter Setup

Add the FastRouter AI provider to OpenClaw with all available text and vision models using only built-in tools (no script execution required).

## Inputs

- **API Key** (required): Starts with `sk-v1-` followed by a hex string. If not provided, ask for it.
- **Base URL** (optional): Defaults to `https://api.fastrouter.ai`

## Steps

### 1. Extract the API key

Parse the API key from the user's message. It starts with `sk-v1-`.

If no API key is found, ask the user for it. Do NOT proceed without one.

### 2. Fetch models

Use `web_fetch` to get the model list:

```
web_fetch url="https://api.fastrouter.ai/v1/models" extractMode="text"
```

### 3. Filter models

From the response JSON, keep only models where:
- `is_active` is true
- `architecture.output_modalities` includes "text"
- `architecture.input_modalities` includes "text" or "image"

For each qualifying model, extract:
- `id` — the model identifier
- `context_length` — context window size
- `top_provider.max_completion_tokens` — max output tokens (if 0 or missing, use min(context_length, 8192))
- Input types: list of "text" and/or "image" from input_modalities

### 4. Build provider config

Construct the provider object (do NOT include a "name" key — OpenClaw rejects it):

```json
{
  "baseUrl": "https://api.fastrouter.ai",
  "api": "openai-completions",
  "apiKey": "THE_API_KEY",
  "models": [
    {
      "id": "model/id",
      "name": "Display Name",
      "contextWindow": 128000,
      "maxTokens": 8192,
      "input": ["text", "image"],
      "cost": {"input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0},
      "reasoning": false
    }
  ]
}
```

### 5. Update openclaw.json

Use `read` to load `~/.openclaw/openclaw.json`.

Merge the provider into `models.providers.fastrouter`, preserving all other config.

Also add model references to `agents.defaults.models` — for each model, add:
```
"fastrouter/MODEL_ID": {}
```

Use `write` to save the updated config.

### 6. Restart gateway

```bash
openclaw gateway restart
```

This is the only step requiring user approval.

### 7. Report to user

Tell the user:
- How many models were added
- They can switch models with `/model fastrouter/MODEL_ID`
- Suggest popular models (claude, gpt, gemini, deepseek variants)

## Error Handling

- **API unreachable**: Tell the user the FastRouter API may be down, try again later
- **No qualifying models**: Warn that no text/image models were found
- **Config file missing**: Create the full structure from scratch
- **Invalid API key format**: Ask the user to double-check their key

## Notes

- Provider key is `fastrouter`
- Existing fastrouter config will be replaced with fresh model list
- All other providers and settings are preserved
- Cost is set to zero (FastRouter handles billing separately)
- Video-only and audio-only models are excluded

````


# Running Hermes Agent with FastRouter

***

**Hermes Agent** is an open-source, terminal-based AI agent built by [Nous Research](https://nousresearch.com). It brings a powerful, agentic assistant directly into your command line — capable of browsing the web, writing and running code, managing files, calling APIs, and orchestrating complex multi-step tasks, all without leaving your terminal.

**FastRouter** routes each call to the best backend model behind one OpenAI-compatible API. One key, every major model, unified billing.

***

## Prerequisites

* Hermes Agent installed and on your `PATH`. Verify with `hermes --version`. If you don't have it: [install Hermes](https://hermes-agent.nousresearch.com/docs).
* A FastRouter API key. Get one at [fastrouter.ai](https://fastrouter.ai).

## Three ways to install

Pick the one that matches where you're starting from. Please make sure you have hermes installed as listed in the prerequisites. Then chose any one of the below paths.

| Path                              | Best for                                                                   |
| --------------------------------- | -------------------------------------------------------------------------- |
| **A. One-line script**            | Fresh install . Fastest. FastRouter API key is the argument to the script. |
| **B. Manual `hermes config set`** | Explicit, CI-friendly, scriptable.                                         |
| **C. Skill inside Hermes chat**   | You already use Hermes with another provider.                              |

### A. One-line install script

In the below commands, replace <mark style="color:green;">$API\_KEY</mark> with your actual fastrouter api key.

```zsh
curl -fsSL https://fastrouter.ai/hermes/install-fastrouter.sh | sh -s -- $API_KEY
```

Or download, inspect, then run

```bash
curl -fsSL https://fastrouter.ai/hermes/install-fastrouter.sh -o install-fastrouter.sh
less install-fastrouter.sh
chmod +x install-fastrouter.sh
./install-fastrouter.sh $API_KEY
```

The script registers the provider, sets `fastrouter/auto` as your default model, enables cost display, and verifies the result.

## B. Manual hermes config set

```shellscript
# Required
hermes config set providers.fastrouter.base_url https://api.fastrouter.ai/api/v1
hermes config set providers.fastrouter.api_key $API_KEY

# Optional: make fastrouter/auto your default
hermes config set model.provider custom:fastrouter
hermes config set model.default fastrouter/auto

# Optional: show per-message cost
hermes config set display.show_cost true
```

Verify:

```shellscript
hermes config | grep -A 3 fastrouter
```

You should see both `base_url` and `api_key` under `providers.fastrouter`.

### C. Install FastRouter via Hermes Skill inside Hermes chat

Drive the FastRouter setup from a Hermes chat session. Install the skill once, then ask the agent to wire it up — and have it fetch and summarize available models on demand.

***

### 1. Install the Skill ( one time)

Run this once in the terminal to register the FastRouter skill with Hermes:

```bash
hermes skills install https://fastrouter.ai/hermes/skill.md
```

***

While installing, when prompted to pick a category of the skill , enter `gateway` or `router` or any other custom category that you want to put this skill into.

### 2. Ask Hermes to Install FastRouter

In a terminal, start a Hermes chat session by typing

`hermes`

Next, replace <mark style="color:green;">$API\_KEY</mark> with your fastrouter api key and type the following in the hermes chat

```
install fastrouter with key $API_KEY
```

The agent will load the skill, register FastRouter under `providers.fastrouter`, and confirm the setup. You can also ask it to list available models, group them by provider, or recommend one for your use case.

> #### <mark style="color:orange;">**Heads up — secret redaction. Hermes has an optional setting (**</mark><mark style="color:orange;">**`security.redact_secrets`**</mark><mark style="color:orange;">**) that masks API keys before the model sees them. It's off by default, so most users can ignore this. If you have it enabled, your key might be stripped before reaching the model and the skill won't be able to write it to config.**</mark>

***

### 3. Switch the Active Session to FastRouter

Custom providers are picked up at session start, so reset the session first:

```bash
/reset
```

Once the session restarts, open the model picker:

```bash
/model
```

Select `fastrouter` or `custom:fastrouter` from the provider list, then choose the FastRouter model you want to use. The session is now routing through FastRouter.

***

### Troubleshooting

* **Skill not triggering?** Type `/reload-skills` in the Hermes session, or restart the CLI.
* **Config changes not taking effect?** Custom providers load on session start. Run `/reset` in chat or relaunch `hermes`.
* **Script reports "hermes not found"?** Install Hermes first: [hermes-agent.nousresearch.com](https://hermes-agent.nousresearch.com/docs).
* **Getting invalid key error ?** check if the api key was redacted by hermes.

***

#### Reference

You can find the `skill.md` file:

* Attached below, or
* At: <https://fastrouter.ai/hermes/skill.md>

````xml
---
name: fastrouter-setup
description: "Install and configure FastRouter (https://fastrouter.ai) as a custom OpenAI-compatible provider in Hermes Agent. Use this skill whenever the user asks to add, set up, install, configure, or register fastrouter as a provider, or wants to use fastrouter models with Hermes. The skill takes exactly one input: the FastRouter API key."
version: 1.2.0
author: FastRouter
license: MIT
metadata:
  hermes:
    tags: [fastrouter, provider, setup, configuration, custom-provider, openai-compatible]
    homepage: https://fastrouter.ai
    docs: https://docs.fastrouter.ai
---

# FastRouter Setup for Hermes

Configures **fastrouter** as a custom OpenAI-compatible provider in Hermes
Agent. After setup, models can be invoked via `custom:fastrouter`.

## Inputs

This skill accepts **exactly one input**: the FastRouter API key.

Everything else is hardcoded and must not be changed:

| Field      | Value                                  |
|------------|----------------------------------------|
| Provider   | `fastrouter`                          |
| Base URL   | `https://api.fastrouter.ai/api/v1`     |
| API mode   | `chat_completions` (OpenAI-compatible) |
| Reference  | `custom:fastrouter`                   |

## When to Use This Skill

Trigger this skill when the user asks any of:

- "install fastrouter"
- "install fastrouter with key <KEY>"
- "install fastrouter sk-..."
- "add fastrouter as a provider"
- "set up / configure / register fastrouter in hermes"
- "use fastrouter with hermes"

Anything similar referencing **fastrouter** plus install / setup /
configure / add / register / provider should trigger it.

## Extracting the API Key (single-argument rule)

Treat every invocation as a single-argument call: `setup(api_key)`.

Find the key in the user's message using these rules, in order:

1. If the message contains `key=<value>`, `--key <value>`,
   `--api-key <value>`, or `apiKey=<value>`, the value is the key.
2. Else, the **last whitespace-separated token in the message that looks
   like an API key** is the key. A token "looks like an API key" if it
   matches `^[A-Za-z0-9_\-]{8,}$` and begins with one of: `sk-`, `fr-`,
   `fastrouter-`, or is otherwise clearly a credential (long random
   string of letters/digits/dashes/underscores).
3. Else, ask the user **once**: *"Please paste your FastRouter API key
   (just the key, nothing else)."* Then treat their next message as the
   key verbatim, after stripping surrounding whitespace and quotes.

Examples of correct extraction:

| User message                                  | Extracted key      |
|-----------------------------------------------|--------------------|
| `install fastrouter with key sk-abc123`      | `sk-abc123`        |
| `install fastrouter sk-abc123`               | `sk-abc123`        |
| `setup fastrouter --api-key fr-XYZ789`       | `fr-XYZ789`        |
| `add fastrouter key=fr-live-9aBc...`         | `fr-live-9aBc...`  |
| `install fastrouter`                         | (prompt the user)  |

Once extracted, **never modify, paraphrase, or transform the key**. Do
not strip characters you think are formatting. Do not lowercase. Do not
echo the full key back to the user — only ever show it masked
(`****` + last 4 chars) in confirmations.

## Redaction Check (do this before writing)

Hermes has an optional secret-redaction feature
(`security.redact_secrets`). It is **off by default**, but if the user
turned it on, the API key may arrive masked.

After extracting the key, validate it. The key is **invalid for write**
if any of:

- Equals `[REDACTED]`, `<redacted>`, `***`, `***REDACTED***`, or any
  string containing the substring `REDACTED` (case-insensitive)
- Is shorter than 8 characters AND does not match a known short test
  pattern the user clearly typed on purpose (e.g. the user literally
  wrote `sk-abc123` — accept that as a deliberate test value)
- Contains only `*` or `•` characters
- Is empty / whitespace-only

If the key fails validation due to redaction, **stop and tell the user**:

> "It looks like Hermes' secret redaction is masking your API key before
> I can read it. To install fastrouter, please run these in your
> terminal directly so the key never passes through the model:
>
> ```
> hermes config set providers.fastrouter.base_url https://api.fastrouter.ai/api/v1
> hermes config set providers.fastrouter.api_key <YOUR_KEY>
> ```
>
> Or temporarily disable redaction with
> `hermes config set security.redact_secrets false`, restart Hermes,
> ask me again, then re-enable redaction afterward."

Do **not** proceed with a redacted-looking value — writing `[REDACTED]`
as the API key silently breaks the provider with auth errors that look
unrelated to redaction.

## Procedure

Once you have a valid key, follow these steps in order.

### Step 1 — Write the provider config

Run these two commands via the `terminal` tool. Use the extracted key
exactly as-is.

```
hermes config set providers.fastrouter.base_url https://api.fastrouter.ai/api/v1
hermes config set providers.fastrouter.api_key <USER_KEY>
```

Both must exit 0. If either fails, surface the error and stop.

### Step 2 — Verify

```
hermes config | grep -A 3 fastrouter
```

Expected output should include both `base_url` and `api_key` under
`providers.fastrouter`.

### Step 3 — Report success

Reply to the user with usage examples. Show the API key **only masked**.

> ✓ FastRouter is configured (`****<last 4 chars>`).
>
> Use it for one-off calls:
> ```
> hermes chat --provider custom:fastrouter -m <model>
> ```
>
> Or set it as your default:
> ```
> hermes config set model.provider custom:fastrouter
> hermes config set model.default <model>
> ```
>
> Browse models at https://fastrouter.ai/models.

### Step 4 — Optional default-model wiring

If (and only if) the user named a specific model in their request, also
run:

```
hermes config set model.provider custom:fastrouter
hermes config set model.default <that_model>
```

Otherwise leave `model.provider` and `model.default` untouched.

## Pitfalls

- **Don't store the key in `.env`.** FastRouter lives under the
  top-level `providers:` dict in `config.yaml`, not as an aliased env
  var. If the user prefers env vars, use `key_env: FASTROUTER_API_KEY`
  instead of `api_key`, and have them export the var.
- **Two `providers:` keys exist** in some configs — one at top level and
  one nested under `model_catalog`. The `hermes config set
  providers.fastrouter.*` form always targets the top-level one. Don't
  hand-edit the wrong section.
- **Provider name is `custom:fastrouter`**, not just `fastrouter`.
  All custom providers are namespaced under `custom:` when referenced
  from the CLI, model picker, or `--provider` flag.
- **Config changes require a fresh session.** If the user is in an
  active Hermes chat when you run this skill, the new provider only
  becomes available after `/reset` or restart.
- **Redacted keys silently corrupt config.** Always run the redaction
  check before writing. Writing `[REDACTED]` as the API key produces
  auth errors later that look unrelated to redaction.
- **Never echo the full key back.** Only show it masked (last 4 chars).
  Some platforms log assistant output verbatim.

## Verification Checklist

Before reporting success, confirm:

- [ ] Exactly one key was extracted (Inputs rule)
- [ ] The key passed the redaction check
- [ ] Both `hermes config set` commands exited 0
- [ ] `hermes config` shows the `fastrouter` block with both fields
- [ ] You did NOT echo the raw API key — only masked form
````

*Need help?* [*FastRouter docs*](https://docs.fastrouter.ai) *·* [*Hermes docs*](https://hermes-agent.nousresearch.com/docs)


# Pydantic AI

Track usage, control costs, and add guardrails to your Pydantic AI agents

#### What is Pydantic AI?

[Pydantic AI](https://ai.pydantic.dev/) is an agent framework from the team behind Pydantic, designed to bring the "FastAPI feeling" to LLM development. Its signature strength is type-safe, validated structured outputs: you define a Pydantic model, and the agent guarantees the response matches it.

By routing Pydantic AI through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—run the same type-safe agent against any provider and compare structured-output reliability
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting Pydantic AI (Python) to FastRouter and running an agent with validated structured output.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Python 3.10 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once:

```bash
mkdir my_project
cd my_project
python -m venv .venv
```

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

```bash
source .venv/bin/activate
```

On Windows:

```bash
.venv\Scripts\activate
```

**Step 2: Install Pydantic AI**

```bash
pip install "pydantic-ai-slim[openai]"
```

(Or install the full package with `pip install pydantic-ai`.)

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

```bash
export FASTROUTER_API_KEY=sk-add-your-key-here
```

**Step 4: Point the OpenAI Provider at FastRouter**

Pydantic AI's `OpenAIProvider` accepts a custom base URL, which is all FastRouter needs. Save this as `pydantic_ai_example.py`:

```python
import os
from pydantic import BaseModel
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

model = OpenAIChatModel(
    "openai/gpt-5.2",
    provider=OpenAIProvider(
        base_url="https://api.fastrouter.ai/api/v1",
        api_key=os.environ["FASTROUTER_API_KEY"],
    ),
)


class CityInfo(BaseModel):
    city: str
    country: str
    population: int


agent = Agent(model, output_type=CityInfo)

result = agent.run_sync("Tell me about Tokyo.")
print(result.output)
```

**Step 5: Run the Agent**

```bash
python pydantic_ai_example.py
```

The agent returns a validated, typed `CityInfo` object:

<figure><img src="/files/ehSgW9Rw3vPaD690AO9W" alt=""><figcaption></figcaption></figure>

The request appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/) with full token and cost details.

***

#### Use Pydantic AI with 100+ Models

FastRouter uses the `provider/model-name` format. Switching providers is a one-line change to the model slug—useful for testing which model produces the most reliable structured outputs:

```python
model = OpenAIChatModel(
    "anthropic/claude-4.5-sonnet",
    provider=OpenAIProvider(
        base_url="https://api.fastrouter.ai/api/v1",
        api_key=os.environ["FASTROUTER_API_KEY"],
    ),
)
```

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```python
model = OpenAIChatModel(
    "fastrouter/auto",
    provider=OpenAIProvider(
        base_url="https://api.fastrouter.ai/api/v1",
        api_key=os.environ["FASTROUTER_API_KEY"],
    ),
)
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

**Flex Pricing for Batch Extraction**

Structured-output agents often run in batch—extracting records from documents, normalizing datasets, generating evaluations. Append `:flex` to any model to access up to 50% lower token costs on latency-tolerant jobs:

```python
"openai/gpt-5.2:flex"
```

> **Note:** Flex is not recommended for interactive agents where you need low-latency responses.

[Compare flex pricing options](https://docs.fastrouter.ai/explore-features/flex-pricing)

***

#### FAQs

**Configuration & Setup**

**I get an `ImportError` for `OpenAIChatModel` or `OpenAIProvider`. What's missing?**

Install the OpenAI extra: `pip install "pydantic-ai-slim[openai]"`.

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Each `OpenAIChatModel` instance selects its own model, and you can run several side by side.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Structured Outputs**

**My agent raises validation errors on structured output. How do I fix it?**

Try a more capable model—structured-output reliability varies by model. You can also simplify the output schema or add field descriptions to guide the model. Because FastRouter gives you every provider through one key, comparing models on your schema takes minutes.

**My agent crashes with `ValidationError: object — Input should be 'chat.completion'`. What's wrong?**

Some model routes currently return responses that deviate from the OpenAI chat-completions specification, and Pydantic AI validates responses strictly. If you hit this error, switch to a model verified to work with Pydantic AI through FastRouter, such as `openai/gpt-5.2` or `anthropic/claude-4.5-sonnet`.

**Performance & Reliability**

**Does FastRouter add latency to agent runs?**

FastRouter adds near-zero gateway overhead. For most workloads, this is negligible compared to model inference time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# App

Point ready-to-run agent apps and chat gateways at FastRouter. Install once, run a setup command, get 100+ models with full cost visibility.

These are standalone apps and gateways you run as-is, no framework code to write. Most install a FastRouter setup skill, then wire themselves up from a single natural-language command with your API key. Once connected, every request routes through FastRouter with:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, and Mistral behind one key. Switch models from the app's own picker.
* **Observability** on every request: cost, tokens, latency, and model per call, visible in your [dashboard](https://dashboard.fastrouter.ai/).
* **Reliability** through automatic failover across providers, response caching, and intelligent routing.
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation.

<table data-view="cards"><thead><tr><th></th><th data-type="content-ref"></th><th data-hidden data-card-cover data-type="image">Cover image</th></tr></thead><tbody><tr><td></td><td><a href="/pages/hJ1UlxGSFxGv9iu4gaQd">/pages/hJ1UlxGSFxGv9iu4gaQd</a></td><td></td></tr><tr><td></td><td><a href="/pages/IrV5IKv8ZivfFeqENYko">/pages/IrV5IKv8ZivfFeqENYko</a></td><td></td></tr><tr><td></td><td><a href="/pages/uhFYztEJncuvyp0egFTP">/pages/uhFYztEJncuvyp0egFTP</a></td><td></td></tr><tr><td></td><td><a href="/pages/0v4Te6Htd3scpH2kRAU7">/pages/0v4Te6Htd3scpH2kRAU7</a></td><td></td></tr><tr><td></td><td><a href="/pages/8pNF9q2p3RdmJieEGiQI">/pages/8pNF9q2p3RdmJieEGiQI</a></td><td></td></tr><tr><td></td><td><a href="/pages/KGlhuv8j0QbZz2w49eH6">/pages/KGlhuv8j0QbZz2w49eH6</a></td><td></td></tr></tbody></table>


# Instructor

Track usage, control costs, and add guardrails to your Instructor structured outputs

### What is Instructor?

[Instructor](https://python.useinstructor.com/) is a library for getting structured, validated outputs from LLMs. It patches the OpenAI client so you can request a Pydantic model as the response type and get back a validated object, with automatic retries on validation failure.

By routing Instructor through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—compare which model produces the most reliable structured outputs
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting Instructor (Python) to FastRouter by patching an OpenAI client pointed at FastRouter.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Python 3.9 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once:

```bash
mkdir my_project
cd my_project
python -m venv .venv
```

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

```bash
source .venv/bin/activate
```

On Windows:

```bash
.venv\Scripts\activate
```

**Step 2: Install Instructor**

```bash
pip install instructor
```

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

```bash
export FASTROUTER_API_KEY=sk-add-your-key-here
```

**Step 4: Patch an OpenAI Client Pointed at FastRouter**

Create a standard OpenAI client with FastRouter's base URL, then patch it with Instructor. Save this as `instructor_example.py`:

```python
import os
import instructor
from openai import OpenAI
from pydantic import BaseModel

client = instructor.from_openai(
    OpenAI(
        base_url="https://api.fastrouter.ai/api/v1",
        api_key=os.environ["FASTROUTER_API_KEY"],
    )
)


class CityInfo(BaseModel):
    city: str
    country: str
    population: int


result = client.chat.completions.create(
    model="openai/gpt-5.2",
    response_model=CityInfo,
    messages=[{"role": "user", "content": "Tell me about Tokyo."}],
)
print(result)
```

**Step 5: Run the Script**

```bash
python instructor_example.py
```

<figure><img src="/files/VpmT9hUWY0BN20N2SuR7" alt=""><figcaption></figcaption></figure>

You get back a validated `CityInfo` object. The request appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/) with token usage and cost.

***

#### Use Instructor with 100+ Models

FastRouter uses the `provider/model-name` format. Switch providers by changing the `model` argument—useful for finding which model produces the most reliable structured outputs:

```python
result = client.chat.completions.create(
    model="anthropic/claude-4.5-sonnet",
    response_model=CityInfo,
    messages=[{"role": "user", "content": "Tell me about Tokyo."}],
)
```

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```python
model="fastrouter/auto"
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Pass a different `model` on each call, all sharing one key.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Structured Outputs**

**My calls hit Instructor's retry limit. How do I fix it?**

Validation reliability varies by model. Try a more capable model, simplify the schema, or add field descriptions. Because FastRouter gives you every provider through one key, comparing models on your schema takes minutes.

**Performance & Reliability**

**Does FastRouter add latency?**

FastRouter adds near-zero gateway overhead, negligible compared to model inference time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# LlamaIndex

Track usage, control costs, and add guardrails to your LlamaIndex applications

#### What is LlamaIndex?

[LlamaIndex](https://www.llamaindex.ai/) is a leading data framework for building LLM applications over your own data. It provides ingestion, indexing, retrieval, and query primitives for retrieval-augmented generation (RAG) and agentic workflows.

By routing LlamaIndex through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting LlamaIndex (Python) to FastRouter using the `OpenAILike` LLM class.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* Python 3.9 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once:

```bash
mkdir my_project
cd my_project
python -m venv .venv
```

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

```bash
source .venv/bin/activate
```

On Windows:

```bash
.venv\Scripts\activate
```

**Step 2: Install LlamaIndex**

```bash
pip install llama-index-llms-openai-like
```

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

```bash
export FASTROUTER_API_KEY=sk-add-your-key-here
```

**Step 4: Point `OpenAILike` at FastRouter**

LlamaIndex's `OpenAILike` class targets any OpenAI-compatible endpoint. Save this as `llamaindex_example.py`:

```python
import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="openai/gpt-5.2",
    api_base="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
    is_chat_model=True,
)

response = llm.complete("Explain what an LLM gateway does in one sentence.")
print(response)
```

> **Note:** Set `is_chat_model=True` so LlamaIndex uses the chat completions endpoint.

**Step 5: Run the Script**

```bash
python llamaindex_example.py
```

<figure><img src="/files/kNiqsiwSCpyQTR7W1VPk" alt=""><figcaption></figcaption></figure>

The response prints to your terminal, and the request appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/) with token usage and cost.

***

#### Use LlamaIndex with 100+ Models

FastRouter uses the `provider/model-name` format. Switch providers by changing the model slug:

```python
llm = OpenAILike(
    model="anthropic/claude-4.5-sonnet",
    api_base="https://api.fastrouter.ai/api/v1",
    api_key=os.environ["FASTROUTER_API_KEY"],
    is_chat_model=True,
)
```

The same `llm` object plugs into LlamaIndex query engines, chat engines, and agents—set it as the default with `Settings.llm = llm`.

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```python
model="fastrouter/auto"
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. Create multiple `OpenAILike` instances with different models, all sharing one key.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Do I also need a separate embeddings model?**

RAG pipelines need an embeddings model in addition to the chat model. FastRouter supports embeddings through the same endpoint—see the [Embeddings API reference](https://docs.fastrouter.ai/api-reference/embeddings).

**Costs & Budgeting**

**How do I track RAG spending?**

Set budgets and rate limits on the key, and use Dynamic Tags to attribute spend per pipeline. The Dashboard breaks down costs by project, key, model, and tag.

**Performance & Reliability**

**Does FastRouter add latency to queries?**

FastRouter adds near-zero gateway overhead, negligible compared to model inference and retrieval time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# LangChain

Track usage, control costs, and add guardrails to your LangChain applications

### What is LangChain?

[LangChain](https://www.langchain.com/) is the most widely adopted framework for developing applications powered by large language models. It provides composable building blocks for chains, agents, retrieval (RAG), and tool calling.

By routing LangChain through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint, no separate provider SDKs or keys inside your LangChain code
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting LangChain (Python) to FastRouter via `ChatOpenAI`, plus streaming, tool calling, and model selection. It does not cover LangChain.js.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai/))
* Python 3.10 or higher

***

#### Quick Start

**Step 1: Create a Project and Virtual Environment**

You'll only need to do this once:

{% code overflow="wrap" %}

```
mkdir my_projectcd my_projectpython -m venv .venv
```

{% endcode %}

Activate the virtual environment. Do this every time you start a new terminal session.

On macOS or Linux:

{% code overflow="wrap" %}

```
source .venv/bin/activate
```

{% endcode %}

On Windows:

{% code overflow="wrap" %}

```
.venv\Scripts\activate
```

{% endcode %}

**Step 2: Install LangChain**

{% code overflow="wrap" %}

```
pip install langchain langchain-openai
```

{% endcode %}

**Step 3: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai/)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

Export it in your terminal:

{% code overflow="wrap" %}

```mdc
export FASTROUTER_API_KEY=sk-add-your-key-here
```

{% endcode %}

**Step 4: Point `ChatOpenAI` at FastRouter**

FastRouter is fully OpenAI-compatible, so the standard `ChatOpenAI` class works out of the box—just override the base URL. Save this as `langchain_example.py`:

{% code overflow="wrap" %}

```python
import osfrom langchain_openai import ChatOpenAI
llm = ChatOpenAI(    base_url="https://api.fastrouter.ai/api/v1",    api_key=os.environ["FASTROUTER_API_KEY"],    model="openai/gpt-5.2",)
response = llm.invoke("Explain what an LLM gateway does in one sentence.")print(response.content)
```

{% endcode %}

**Step 5: Run the Script**

{% code overflow="wrap" %}

```python
python langchain_example.py
```

{% endcode %}

<figure><img src="/files/MKDy4KFSWa3G3CGqAmHo" alt="The request appears immediately in your FastRouter Dashboard, with token usage and cost attribution."><figcaption><p>The request appears immediately in your <a href="https://dashboard.fastrouter.ai/">FastRouter Dashboard</a>, with token usage and cost attribution.</p></figcaption></figure>

***

#### Streaming and Tool Calling

Because `ChatOpenAI` is LangChain's standard chat model interface, the FastRouter-backed `llm` object drops into any LangChain feature. Both of these run through FastRouter unchanged:

{% code overflow="wrap" %}

```python
# Streamingfor chunk in llm.stream("Count to 3, digits only"):    print(chunk.content, end="", flush=True)
# Tool callingfrom langchain_core.tools import tool
@tooldef get_weather(city: str) -> str:    """Get the current weather for a city."""    return f"The weather in {city} is 72°F and sunny."
response = llm.bind_tools([get_weather]).invoke("What is the weather in Paris?")print(response.tool_calls)# [{'name': 'get_weather', 'args': {'city': 'Paris'}, ...}]
```

{% endcode %}

***

#### Use LangChain with 100+ Models

FastRouter uses the `provider/model-name` format. Switch providers by changing the model slug—no new dependencies or credentials:

{% code overflow="wrap" %}

```md
claude = ChatOpenAI(base_url="https://api.fastrouter.ai/api/v1",    api_key=os.environ["FASTROUTER_API_KEY"],    model="anthropic/claude-4.5-sonnet",)
gemini = ChatOpenAI(    base_url="https://api.fastrouter.ai/api/v1",    api_key=os.environ["FASTROUTER_API_KEY"],    model="google/gemini-3.1-pro-preview",)
```

{% endcode %}

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

{% code overflow="wrap" %}

```md
llm = ChatOpenAI(base_url="https://api.fastrouter.ai/api/v1",    api_key=os.environ["FASTROUTER_API_KEY"], model="fastrouter/auto",)
```

{% endcode %}

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

**Flex Pricing for Batch Workloads**

For RAG indexing, evaluation runs, or other batch chains that tolerate higher latency, append `:flex` to any model to access up to 50% lower token costs:

{% code overflow="wrap" %}

```
model="openai/gpt-5.2:flex"
```

{% endcode %}

> **Note:** Flex is not recommended for interactive chains where you need low-latency responses.

[Compare flex pricing options](https://docs.fastrouter.ai/explore-features/flex-pricing)

***

#### FAQs

**Configuration & Setup**

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. The model is selected per `ChatOpenAI` instance, and you can run several models side by side in one application.

**What if `OPENAI_API_KEY` is already set in my environment?**

The explicit `api_key=` argument takes precedence. Pass your FastRouter key directly to avoid accidentally routing to OpenAI.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**`bind_tools` raises `ValidationError: invalid_tool_calls.0.args — Input should be a valid string`. What's wrong?**

Some model routes currently return tool-call `arguments` as a JSON object instead of the JSON-encoded string the OpenAI specification requires. Plain `invoke` and `stream` calls are unaffected, but tool calling fails. If you hit this error, switch to a model verified for tool calling through FastRouter, such as `openai/gpt-5.2` or `anthropic/claude-4.5-sonnet`.

**Costs & Budgeting**

**How do I track LangChain spending by team or app?**

Create separate projects for each team, issue project-scoped API keys, and use Dynamic Tags for finer-grained attribution. The Dashboard breaks down costs by project, key, model, and tag.

**Performance & Reliability**

**Does FastRouter add latency to LangChain calls?**

FastRouter adds near-zero gateway overhead. For most chains, this is negligible compared to model inference time.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# n8n

Track usage, control costs, and add guardrails to your n8n AI workflows

### What is n8n?

[n8n](https://n8n.io/) is a workflow-automation platform that connects apps, APIs, and AI models into automated pipelines. With its visual node-based editor, you can build complex automations without writing code. The AI nodes are built on LangChain, and the OpenAI nodes accept a custom base URL—so they work with any OpenAI-compatible endpoint.

By routing n8n through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers pointing n8n's OpenAI credential at FastRouter, model selection, team governance, and production feature usage.

#### Prerequisites

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* An n8n instance (cloud or self-hosted)

***

### Quick Start

#### Step 1: Get Your FastRouter API Key

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

#### Step 2: Create an OpenAI Credential Pointing at FastRouter

1. In n8n, go to **Credentials** → **Add credential** and choose **OpenAI**.
2. Set **API Key** to your FastRouter API key.
3. Expand the credential options and set **Base URL** to:

   ```
   https://api.fastrouter.ai/api/v1
   ```
4. Save the credential.

#### Step 3: Add an AI Node and Select the Credential

1. Add an **AI Agent** node (or any node that uses a **Chat Model**) to your workflow.
2. Add a **Chat OpenAI** model sub-node and select the FastRouter credential you just created.
3. In the model field, enter a FastRouter model slug, for example `openai/gpt-5.2`.

> **Note:** Because FastRouter slugs aren't in n8n's built-in OpenAI model list, type the slug directly (for example `anthropic/claude-4.5-sonnet`) rather than relying on the dropdown.

#### Step 4: Run the Workflow

Execute the workflow. The request routes through FastRouter, and every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

<figure><img src="/files/SZa6VncvLVbHGtd4OZ2b" alt=""><figcaption></figcaption></figure>

***

### FAQs

#### Configuration & Setup

**The node fails with an authentication or "model not found" error. What should I check?**

Confirm the credential's **Base URL** is exactly `https://api.fastrouter.ai/api/v1`, the API key is correct, and the model field contains a valid FastRouter slug including its provider prefix (for example `openai/gpt-5.2`, not `gpt-5.2`).

**Can I use the same credential across many workflows?**

Yes. One FastRouter credential works for every OpenAI/AI node in your instance. Switch models per node without changing the credential.

**Can I use n8n with my own provider keys (BYOK)?**

Yes. Set up an External Key integration in the FastRouter dashboard, then route n8n traffic through it. You retain your provider's pricing and rate limits while gaining FastRouter's routing and observability layer.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

#### Costs & Budgeting

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit. The workflow receives an error response indicating the budget has been exceeded.

**How do I track n8n spending by workflow or team?**

Create separate projects for each team, issue project-scoped API keys per workflow, and use Dynamic Tags (via Code nodes) for finer-grained attribution. The Dashboard breaks down costs by project, key, model, and tag.

#### Performance & Reliability

**Does FastRouter add latency to a workflow run?**

FastRouter adds near-zero gateway overhead. For most workflows, this is negligible compared to model inference time.

#### Privacy & Security

**Is my data sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings.

***

### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# Open WebUI

Track usage, control costs, and add guardrails to your Open WebUI deployment

### What is Open WebUI?

[Open WebUI](https://openwebui.com/) is a self-hosted, ChatGPT-style web interface for chatting with large language models. It runs entirely on your own infrastructure and connects to any server or provider that implements the OpenAI-compatible API.

By routing Open WebUI through FastRouter, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint—your whole team picks from a shared catalog with one key
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation

This guide covers connecting Open WebUI to FastRouter by adding it as an OpenAI-compatible connection.

**Prerequisites**

* A FastRouter.ai account ([sign up](https://fastrouter.ai))
* A running Open WebUI instance with admin access ([installation guide](https://docs.openwebui.com/getting-started/))

***

#### Quick Start

**Step 1: Get Your FastRouter API Key**

1. Sign up or log in at [fastrouter.ai](https://fastrouter.ai)
2. Navigate to your project's **Keys** page
3. Click **Create User Key**
4. Copy the key immediately. FastRouter does not display the key again after creation.

**Step 2: Add FastRouter as a Connection**

1. Open Open WebUI in your browser.
2. Go to ⚙️ **Admin Settings** → **Connections** → **OpenAI**.

<figure><img src="/files/RlY84pcrnTujx4T1G5dW" alt=""><figcaption></figcaption></figure>

3. Click ➕ **Add Connection**.

<figure><img src="/files/tUqlzYt7mTRtXTtKrdtN" alt=""><figcaption></figcaption></figure>

3. Fill in:
   * **URL:** `https://api.fastrouter.ai/api/v1`
   * **API Key:** your FastRouter API key
4. Click **Save**.

<figure><img src="/files/JZS79avRzBJBe0ER5eSq" alt=""><figcaption></figcaption></figure>

**Step 3: Add the Models You Want**

FastRouter exposes a large catalog, so rather than loading every model, add the specific slugs your team needs to the **Model IDs (Filter)** allowlist:

1. In the connection settings, find **Model IDs (Filter)**.
2. Type a FastRouter model slug—for example `openai/gpt-5.2`—and click the **+** icon.
3. Repeat for any other models (e.g., `anthropic/claude-4.5-sonnet`, `x-ai/grok-code-fast-1`).
4. Click **Save**.

<figure><img src="/files/JZS79avRzBJBe0ER5eSq" alt=""><figcaption></figcaption></figure>

> **Note:** Open WebUI verifies a connection by calling the provider's `/models` endpoint. Even if verification is slow or returns a warning, chat completions still work—the **Model IDs (Filter)** allowlist guarantees the models you listed appear in the selector.

**Step 4: Start Chatting**

Select a FastRouter model from the model dropdown in a new chat and send a message. The request routes through FastRouter, and every request, token count, and cost appears in your [FastRouter Dashboard](https://dashboard.fastrouter.ai/).

<figure><img src="/files/9m5nNB2tbUodIBpghaXH" alt=""><figcaption></figcaption></figure>

***

#### Use Open WebUI with 100+ Models

FastRouter uses the `provider/model-name` format. Add any catalog slug to the connection's **Model IDs (Filter)** and it becomes selectable in the chat model dropdown:

```
anthropic/claude-4.5-sonnet
google/gemini-3.1-pro-preview
x-ai/grok-code-fast-1
```

[Explore the full model catalog](https://fastrouter.ai/models)

**Automatic Model Selection**

Let FastRouter pick the best model for each request based on query complexity, domain, and cost. Add this slug to your **Model IDs (Filter)** and select it like any other model:

```
fastrouter/auto
```

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

***

#### FAQs

**Configuration & Setup**

**Connection verification shows an error, but is the connection broken?**

Not necessarily. Verification calls the `/models` endpoint; if it is slow or returns a non-200, you'll see a warning, but chat completions still work. Make sure your models are listed in the **Model IDs (Filter)** allowlist, and they will appear in the selector regardless.

**Do I need a trailing slash on the URL?**

No. Use exactly `https://api.fastrouter.ai/api/v1` with no trailing slash.

**Can I run multiple connections at once?**

Yes. Each connection has a toggle to enable or disable it without deleting it, so you can keep FastRouter alongside other providers and switch as needed.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

**Costs & Budgeting**

**How do I control spend across a team using one shared instance?**

Set a budget and rate limit on the FastRouter key, and use Dynamic Tags to attribute spend. The Dashboard breaks down costs by project, key, model, and tag.

**Privacy & Security**

**Is my chat content sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your instance. Content logging can be disabled per key for sensitive workloads. See the **Disable Content Logging** option in key settings. For multi-user deployments, prefer a least-privilege key rather than an admin/master key.

***

#### Next Steps

* [Explore the full model catalog](https://fastrouter.ai/models)
* [Set up Fallback Models](https://docs.fastrouter.ai/fallback-models) for high availability
* [Configure Alerts](https://docs.fastrouter.ai/alerts) for spend and performance monitoring
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the Discord community](https://discord.gg/QfTgEtMyyU)


# Scalekit

Add per-user OAuth tool calling to your FastRouter agent with Scalekit. Gmail, GitHub, Slack, and more, with zero custom OAuth code.

### What is Scalekit?

Scalekit is an SDK that gives AI agents secure, per-user access to external services like Gmail, GitHub, and Slack. It handles OAuth flows, token storage, token refresh, tool discovery, and tool execution server-side. Your agent never touches raw access tokens.

By routing your agent through FastRouter with Scalekit, you get:

* **100+ models** from OpenAI, Anthropic, Google, xAI, Meta, Groq, Mistral, and more through one endpoint
* **Observability** for every request: cost, tokens, latency, and model selection tracked in real time
* **Reliability** through automatic failover across providers, response caching, and intelligent routing
* **Governance** with per-key budgets, rate limits, model restrictions, role-based access, and project isolation
* **Per-user OAuth** with zero custom OAuth code — Scalekit manages credentials for each connected account

This guide covers FastRouter + Scalekit integration, model selection, team governance, and production feature usage.

### Overview

FastRouter supports function calling out of the box. [Scalekit](https://docs.scalekit.com/agentkit) extends that with per-user OAuth tool access, so your agent can read Gmail, create GitHub issues, or post to Slack on behalf of individual users. You can choose from [100+ connectors](https://docs.scalekit.com/agentkit/connectors/).

The integration is one configuration change: point the OpenAI SDK's `baseURL` at FastRouter. Scalekit handles OAuth token storage, tool discovery, and tool execution. Your agent never touches raw access tokens.

**Sample repository:** [github.com/scalekit-developers/fastrouter-scalekit-demo](https://github.com/scalekit-developers/fastrouter-scalekit-demo)

***

### What you are building

**FastRouter as the LLM provider.** All chat completions go through FastRouter's OpenAI-compatible endpoint. Switch models by changing one environment variable.

**Scalekit for tool access.** `listScopedTools` returns per-user tool schemas ready to pass directly to FastRouter. `executeTool` runs each tool server-side and returns structured results.

**B2B OAuth without custom OAuth code.** Scalekit handles the OAuth flow, token storage, and refresh for each connected service. Your agent gets an auth link, waits for the user to authorize, and receives a verified, active connected account.

**Agentic loop.** The agent calls FastRouter, receives tool calls, executes them through Scalekit, and feeds results back, repeating until FastRouter returns a final answer.

***

### Prerequisites

* FastRouter account and API key ([sign up at fastrouter.ai](https://fastrouter.ai))
* Scalekit account with AgentKit enabled ([create one at app.scalekit.com](https://app.scalekit.com))
* At least one AgentKit connection configured (Gmail, GitHub, or Slack)
* Node.js 20 or later

***

### Clone and run the sample

#### 1. Clone the repository and install dependencies

```sh
git clone https://github.com/scalekit-developers/fastrouter-scalekit-demo
cd fastrouter-scalekit-demo
npm install
```

#### 2. Copy the example environment file and fill in your credentials

```sh
cp .env.example .env
```

Open `.env` and set these values:

```sh
# FastRouter — find your API key at fastrouter.ai/dashboard
FASTROUTER_API_KEY=sk-v1-...
FASTROUTER_BASE_URL=https://api.fastrouter.ai/api/v1
FASTROUTER_MODEL=openai/gpt-4o

# Scalekit — find these in your Scalekit dashboard under Developers → API credentials
SCALEKIT_ENVIRONMENT_URL=https://your-env.scalekit.dev
SCALEKIT_CLIENT_ID=your_client_id
SCALEKIT_CLIENT_SECRET=your_client_secret

# The AgentKit connection to use — must match a connection name in your dashboard
SCALEKIT_CONNECTION_NAME=gmail
SCALEKIT_IDENTIFIER=user_123
```

`SCALEKIT_CONNECTION_NAME` must match the exact connection name in your Scalekit dashboard under **AgentKit > Connections**.

`FASTROUTER_MODEL` accepts any model in the [FastRouter catalog](https://fastrouter.ai/models) that supports function calling.

#### 3. Run the agent

```sh
npm start
```

#### 4. Authorize the connection on first run

The agent prints an authorization link if the connected account is not yet active:

```
Authorization required.
Open this link and complete the flow:

https://your-env.scalekit.dev/magicLink/...

Waiting for callback on http://localhost:3000/callback ...
```

Open the link in your browser and complete the OAuth flow. The agent detects the callback automatically and continues.

After authorization, the agent loads tools, calls FastRouter, and prints a final answer:

```
Connected account is now active.
Loaded 17 scoped tools from Scalekit.
Model requested 1 tool call(s).

 Executing gmail_list_messages
  args: {"maxResults":5,"q":"is:unread"}

Final answer:

Here are your 5 most recent unread emails: ...
```

***

### How the agent works

Three pieces connect FastRouter to Scalekit tools.

#### 1. Initialize FastRouter using the OpenAI SDK

FastRouter's API is OpenAI-compatible. Point `baseURL` at FastRouter and pass your FastRouter API key:

```typescript
import OpenAI from 'openai';

const fastRouter = new OpenAI({
  apiKey: process.env.FASTROUTER_API_KEY,
  baseURL: process.env.FASTROUTER_BASE_URL ?? 'https://api.fastrouter.ai/api/v1',
});
```

No other FastRouter-specific setup is required. The standard `openai` package works as-is.

#### 2. B2B OAuth connects user accounts without custom token code

Scalekit handles the full OAuth flow. Your agent calls `getOrCreateConnectedAccount` to check whether the user's account is already connected, then calls `getAuthorizationLink` to get an auth URL if it isn't.

{% tabs %}
{% tab title="Node.js" %}

```typescript
import { ConnectorStatus } from '@scalekit-sdk/node/lib/pkg/grpc/scalekit/v1/connected_accounts/connected_accounts_pb.js';
import crypto from 'node:crypto';

const userVerifyUrl = 'http://localhost:3000/callback';

// Generate a random state value and store it (e.g. in a secure cookie or session)
// to validate on the OAuth callback and prevent CSRF / account mix-up attacks.
const state = crypto.randomUUID();

const { connectedAccount } = await scalekit.actions.getOrCreateConnectedAccount({
  connectionName: 'gmail',
  identifier: 'user_123',
  userVerifyUrl,
});

if (connectedAccount?.status !== ConnectorStatus.ACTIVE) {
  const { link } = await scalekit.actions.getAuthorizationLink({
    connectionName: 'gmail',
    identifier: 'user_123',
    userVerifyUrl,
    state,
  });
  // Show link to user, then wait for the browser redirect callback
}
```

{% endtab %}

{% tab title="Python" %}

```python
import secrets

user_verify_url = "http://localhost:3000/callback"

# Generate and store a state value (e.g. in a secure, HTTP-only cookie) for CSRF protection
state = secrets.token_urlsafe(32)

response = scalekit_client.actions.get_or_create_connected_account(
    connection_name="gmail",
    identifier="user_123",
    userVerifyUrl,
)

if response.connected_account.status != "ACTIVE":
    link_resp = scalekit_client.actions.get_authorization_link(
        connection_name="gmail",
        identifier="user_123",
        user_verify_url=user_verify_url,
        state=state,
    )
    # Show link_resp.link to the user
```

{% endtab %}
{% endtabs %}

`userVerifyUrl` is where Scalekit redirects the user's browser after the OAuth flow completes (a GET request with `auth_request_id` and `state` query parameters). The sample runs a minimal HTTP server on `localhost:3000` to catch that redirect, validate the `state` against the original value, extract the `auth_request_id`, and call `verifyConnectedAccountUser` to mark the account active:

{% tabs %}
{% tab title="Node.js" %}

```typescript
async function waitForCallback(port: number, expectedState: string): Promise<string> 
  return new Promise((resolve, reject) => {
    const server = http.createServer((req, res) => {
      const url = new URL(req.url ?? '/', `http://localhost:${port}`);
      const authRequestId = url.searchParams.get('auth_request_id');
      const returnedState = url.searchParams.get('state');

      res.writeHead(200, { 'Content-Type': 'text/html' });
      res.end('<html><body><h2>Authorization complete — return to your terminal.</h2></body></html>');
      server.close();

      if (authRequestId && returnedState === expectedState) {
        resolve(authRequestId);
      } else {
        reject(new Error('Invalid or missing auth_request_id or state in callback'));
      }
    });
    server.listen(port);
  });
}

const authRequestId = await waitForCallback(3000, state);
await scalekit.actions.verifyConnectedAccountUser({
  authRequestId,
  identifier: 'user_123',
});
```

{% endtab %}

{% tab title="Python" %}

```python
# In your web framework callback handler (e.g. FastAPI):
# 1. Validate that the "state" query param matches the value you stored earlier
# 2. Then exchange the auth_request_id (never trust identity from the URL alone)

result = scalekit_client.actions.verify_connected_account_user(
    auth_request_id=auth_request_id,
    identifier="user_123",
)
# redirect to result.post_user_verify_redirect_url
```

{% endtab %}
{% endtabs %}

{% hint style="info" %}
**Production callback endpoint:** In a production web app, replace `localhost:3000/callback` with your server's callback endpoint. Scalekit redirects the browser to it with `auth_request_id` and `state` query params. Your handler must validate the state before calling `verifyConnectedAccountUser` to complete account activation.
{% endhint %}

#### 3. Tool discovery returns schemas in FastRouter's expected format

`listScopedTools` returns only the tools the connected account has permission to use. Map each tool's `input_schema` to the `parameters` field FastRouter expects:

{% tabs %}
{% tab title="Node.js" %}

```typescript
const { tools } = await scalekit.tools.listScopedTools('user_123', {
  filter: { connectionNames: ['gmail'] },
  pageSize: 100,
});

const fastRouterTools = tools
  .map((t) => t.tool?.definition)
  .filter((def): def is NonNullable<typeof def> => Boolean(def?.name))
  .map((def) => ({
    type: 'function' as const,
    function: {
      name: String(def.name),
      description: String(def.description ?? ''),
      parameters: def.input_schema ?? { type: 'object', properties: {} },
    },
  }));
```

{% endtab %}

{% tab title="Python" %}

```python
from google.protobuf.json_format import MessageToDict

scoped_response, _ = scalekit_client.actions.tools.list_scoped_tools(
    identifier="user_123",
    filter={"connection_names": ["gmail"]},
)

fast_router_tools = [
    {
        "type": "function",
        "function": {
            "name": MessageToDict(tool.tool).get("definition", {}).get("name"),
            "description": MessageToDict(tool.tool).get("definition", {}).get("description", ""),
            "parameters": MessageToDict(tool.tool).get("definition", {}).get("input_schema", {}),
        },
    }
    for tool in scoped_response.tools
]
```

{% endtab %}
{% endtabs %}

FastRouter uses the same function-calling format as OpenAI. No additional schema transformation is needed.

#### 4. The agentic loop runs until the model stops requesting tools

Pass the tool list to FastRouter and execute each tool call through Scalekit until the model returns a response with no tool calls:

{% tabs %}
{% tab title="Node.js" %}

```typescript
const messages: OpenAI.ChatCompletionMessageParam[] = [
  { role: 'system', content: 'You are a helpful assistant. Use tools when they help. Do not invent tool results.' },
  { role: 'user', content: 'Fetch my last 5 unread emails and summarize them.' },
];

for (let turn = 0; turn < 8; turn++) {
  const response = await fastRouter.chat.completions.create({
    model: process.env.FASTROUTER_MODEL ?? 'openai/gpt-4o',
    messages,
    tools: fastRouterTools,
    tool_choice: 'auto',
  });

  const message = response.choices[0].message;
  messages.push(message);

  // No tool calls means a final answer
  if (!message.tool_calls?.length) {
    console.log(message.content);
    return;
  }

  // Execute each tool call and append the result
  for (const call of message.tool_calls) {
    const result = await scalekit.actions.executeTool({
      toolName: call.function.name,
      identifier: 'user_123',
      connector: 'gmail',
      toolInput: JSON.parse(call.function.arguments),
    });

    messages.push({
      role: 'tool',
      tool_call_id: call.id,
      content: JSON.stringify(result.data ?? {}),
    });
  }
}
```

{% endtab %}

{% tab title="Python" %}

```python
from openai import OpenAI

fast_router = OpenAI(
    api_key=os.environ["FASTROUTER_API_KEY"],
    base_url=os.environ.get("FASTROUTER_BASE_URL", "https://api.fastrouter.ai/api/v1"),
)

messages = [
    {"role": "system", "content": "You are a helpful assistant. Use tools when they help. Do not invent tool results."},
    {"role": "user", "content": "Fetch my last 5 unread emails and summarize them."},
]

for turn in range(8):
    response = fast_router.chat.completions.create(
        model=os.environ.get("FASTROUTER_MODEL", "openai/gpt-4o"),
        messages=messages,
        tools=fast_router_tools,
        tool_choice="auto",
    )

    message = response.choices[0].message
    messages.append(message)

    # No tool calls means a final answer
    if not message.tool_calls:
        print(message.content)
        break

    # Execute each tool call and append the result
    for call in message.tool_calls:
        result = scalekit_client.actions.execute_tool(
            tool_input=json.loads(call.function.arguments),
            tool_name=call.function.name,
            identifier="user_123",
            connection_name="gmail",
        )

        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result.data or {}),
        })
```

`executeTool` runs the tool server-side using the connected account's stored OAuth tokens. Your agent never handles raw access tokens.
{% endtab %}
{% endtabs %}

***

### Customize the agent

**Change the model.** Set `FASTROUTER_MODEL` in `.env` to any model in the [FastRouter catalog](https://fastrouter.ai/models) that supports function calling. The agent code stays identical regardless of which model you pick.

#### Model Examples

Switch to a different provider or model at any time:

```sh
# Anthropic Claude
FASTROUTER_MODEL=anthropic/claude-sonnet-4.6

# Google Gemini
FASTROUTER_MODEL=google/gemini-3-pro

# Mistral
FASTROUTER_MODEL=mistralai/Mistral-Small-24B-Instruct-2501

# xAI Grok
FASTROUTER_MODEL=x-ai/grok-4
```

No code changes or SDK swaps required. Update the model string and your agent uses the new provider.

#### Automatic Model Selection

Let FastRouter pick the best model for each request based on query complexity, domain, and cost:

```sh
FASTROUTER_MODEL=fastrouter/auto
```

FastRouter analyzes the input and routes to the most appropriate model from the available pool. This is the fastest way to get started without maintaining model preferences.

[Explore automatic model selection](https://docs.fastrouter.ai/automatic-model-selection)

#### Cost-Optimized Routing with Sorting Suffixes

Append a suffix to any model to control provider selection:

```sh
# Route to the cheapest provider for this model
FASTROUTER_MODEL=openai/gpt-5.3-codex:price

# Route to the fastest provider
FASTROUTER_MODEL=openai/gpt-5.3-codex:throughput
```

[Configure cost and performance routing](https://docs.fastrouter.ai/provider-routing-strategies)

#### Flex Pricing

For batch-style agent tasks that tolerate higher latency, append `:flex` to access up to 50% lower token costs:

```sh
FASTROUTER_MODEL=openai/gpt-5.4-nano:flex
```

Flex routes your request to the provider's discounted inference tier. Same API key, same endpoint, same payload.

> **Note:** Flex is not recommended for interactive agent sessions where you need low-latency responses. Use it for large batch processing, bulk email triage, or scheduled background tasks.

[Compare flex pricing options](https://docs.fastrouter.ai/explore-features/flex-pricing)

**Change the connection.** Set `SCALEKIT_CONNECTION_NAME` to any connection configured in your Scalekit dashboard:

| Value    | What it connects                    |
| -------- | ----------------------------------- |
| `gmail`  | Gmail read/send                     |
| `github` | Repositories, issues, pull requests |
| `slack`  | Channels, messages, users           |

**Change the prompt.** Pass a prompt as a CLI argument to override the default:

```sh
npm start "List all GitHub pull requests assigned to me"
```

Or set `USER_PROMPT` in `.env` to change the default.

**Support multiple connections.** Call `listScopedTools` with multiple connection names to give the model tools from all of them at once:

```typescript
const { tools } = await scalekit.tools.listScopedTools('user_123', {
  filter: { connectionNames: ['gmail', 'github', 'slack'] },
});
```

**Use FastRouter-specific features.** Since FastRouter is the LLM provider, you can layer on any FastRouter capability alongside Scalekit tools. For example, add tags for cost tracking, use virtual model aliases to route across model pools, or configure fallback models for resilience.

***

### FAQs

#### Configuration & Setup

**Can I use multiple models with the same API key?**

Yes. The API key controls access and budget. The model is set via `FASTROUTER_MODEL`. You can switch models at any time without changing your key.

**Can I use my own provider keys (BYOK)?**

Yes. Set up an External Key integration in the FastRouter dashboard, then route agent traffic through it. You retain your provider's pricing and rate limits while gaining FastRouter's routing and observability layer.

**Can I restrict a key to only use specific models?**

Yes. When creating or editing a key, use the **Select Models** setting to limit which models the key can access. FastRouter rejects requests to unauthorized models.

#### Costs & Budgeting

**What happens when a key exceeds its budget?**

FastRouter blocks further requests until the budget resets (if a reset interval is configured) or an admin increases the limit. The agent receives an error response indicating the budget has been exceeded.

**How do I track agent spending by team or connection?**

Create separate projects for each team, issue project-scoped API keys, and use Dynamic Tags for finer-grained attribution. The dashboard breaks down costs by project, key, model, and tag.

#### Performance & Reliability

**Does FastRouter add latency to requests?**

FastRouter adds near-zero gateway overhead. For most workflows, this is negligible compared to model inference time.

#### Privacy & Security

**Does Scalekit see my users' OAuth tokens?**

Scalekit stores OAuth tokens server-side. Your agent never handles raw access tokens. Scalekit uses them to execute tools on behalf of connected accounts.

**Is my request content sent to FastRouter's servers?**

FastRouter acts as a pass-through gateway. Requests are routed to the model provider and responses are returned to your client. Content logging can be disabled per key for sensitive workloads — see the **Disable Content Logging** option in key settings.

***

### Next steps

* [Sample repository](https://github.com/scalekit-developers/fastrouter-scalekit-demo) for the full working code
* [Explore the full model catalog](https://fastrouter.ai/models) at FastRouter
* [Run a Free Audit](https://fastrouter.ai/audit) on your existing LLM traffic to identify savings
* [Join the FastRouter Discord community](https://discord.gg/QfTgEtMyyU)

***