Custom Model Lists
Custom Model Lists make it easy to manage and optimize amongst multiple LLMs without changing your code.
Overview
Custom Model Lists in FastRouter.ai let you easily group several models and providers under a single virtual model name. You can reference these lists in your API calls by passing the list’s name to the model parameter. FastRouter will then automatically select and route requests to one of the configured models—according to the selection strategy you specify.
This feature maximizes flexibility, reliability, and efficiency when deploying LLMs across different providers or model versions.
How It Works
You can create a Model List by selecting models and providers, assigning weights (if needed), and choosing an automatic selection strategy. On each request, FastRouter will pick from the configured options automatically—and fall back to another model in the list if the primary one fails.
Use Cases:
A/B testing: Seamlessly test multiple models/providers.
Resilience: Automatic fallback if a provider or deployment is unavailable.
Optimization: Route requests for the best speed, price, or usage.
Selection Strategies
When configuring your Custom Model List, you can specify how FastRouter should select models for each request:
Available Strategies:
Random Shuffle Each request is sent to a randomly selected model in your list, according to the weights you assign. Useful for A/B tests or spreading traffic evenly.
Lowest Latency Automatically selects the model with the fastest current response time, ensuring minimum wait time for end-users.
Highest Throughput Picks the model/provider combination that can process requests at the highest rate, ideal for high-volume applications.
Lowest Usage Prioritizes models with the least usage, helping to balance load and prevent hitting usage or rate limits.
Lowest Price Routes requests to the least expensive model in the list, optimizing for cost savings.
Built-in Automatic Fallback
If a model/provider fails or is unavailable, FastRouter will transparently send the request to the next candidate in your list, according to your selected strategy—ensuring maximum reliability.
Getting Started
Go to the Custom Model Lists section in FastRouter.
Click Create Model List.
Enter a name for your list.
Select the models and providers you want to include and assign weights (for strategies that use them).
Choose a selection strategy.
Click Create.
You can then reference this Model List in your API calls by specifying its name in the model parameter.
Last updated