Observability Destinations
Own your observability data. Stream request traces from FastRouter straight into your own Kafka cluster.
Beta: Observability Destinations is currently available to select customers. To request access, contact: support@fastrouter.ai.
Overview
FastRouter records latency, tokens, cost, and the full request and response for every call you route through it. Observability let you keep that data in your own infrastructure: connect a Kafka cluster (or ClickHouse database - coming soon), and FastRouter streams each request trace to it.
Unlimited retention in your stack. Traces live in storage you control, for as long as your own policies allow.
Query with your own tooling. Analyze traces with the dashboards, warehouses, and pipelines your team already runs.
Credentials stay encrypted. Connection credentials are stored encrypted by FastRouter.
Common use cases:
Long-term audit and compliance by retaining every request and response beyond the Activity Log window
Custom analytics by joining LLM cost and latency data with your own product and business data
Real-time pipelines that feed traces into alerting, anomaly detection, or data quality jobs you already operate
Supported Destinations
Kafka
Each trace is produced as a message to a topic you choose.
ClickHouse: Coming Soon
Each trace is written to your ClickHouse database.
How It Works
You add a destination with its connection details and, optionally, filters.
Once the destination is saved, FastRouter forwards the trace for every request that matches your filters.
From there, retention, access control, and querying are governed entirely by your own infrastructure.
Traces continue to appear in the FastRouter Dashboard & Activity Log as usual. Destinations are an additional copy, not a replacement.
Prerequisites For Kafka:
One or more brokers reachable from FastRouter over the network
A listener configured for
SASL_SSLwith thePLAINmechanismAn existing topic for traces. FastRouter does not create topics.
A SASL user with permission to produce to that topic
Create a dedicated user for FastRouter with write access to the traces topic only. This keeps the credential's blast radius small and makes rotation straightforward.
Add a Kafka Destination
In the FastRouter dashboard, go to Observability and click Add Destination.
Under Destination Type, select Kafka.
Fill in the Kafka Connection and Authentication fields described below.
Optionally, set Filters to limit which traces are forwarded.
Click Test Connection, then Send Test Trace to verify the setup.
Click Create Destination. Traces start flowing once saved.
Kafka Connection
Bootstrap Servers
Yes
host:port of your Kafka broker. Separate multiple brokers with commas.
kafka-broker.example.com:9093
Topic
Yes
The topic traces are produced to. It must already exist.
fastrouter-traces
Listing more than one broker in Bootstrap Servers lets FastRouter discover your cluster even if one broker is temporarily unavailable.
Authentication
Security Protocol
Yes
How FastRouter connects and authenticates. See the options below.
SASL_SSL (PLAIN)
Username
Yes
The SASL username FastRouter authenticates with.
fastrouter-user
Password
Yes
The SASL password for that user. Stored encrypted. Use the eye icon to reveal it while typing.
—
Security protocol options:
SASL_SSL (PLAIN)
Credentials are sent over TLS using the PLAIN SASL mechanism.
Filters
Filters are optional and apply to every destination type. Leave both set to all to stream every request.
Project
All Projects
API Key
All API Keys
When filters are set, a trace is forwarded only if it matches both.
Test Before You Save
The Add Destination form includes two checks. Run both before clicking Create Destination.
Test Connection
FastRouter can reach your destination and authenticate with the credentials you entered.
Send Test Trace
End-to-end delivery. FastRouter sends a sample trace so you can confirm it arrives on your side.
To confirm the test trace arrived in Kafka, read from the topic with a consumer that uses the same kind of credentials:
Create a client.properties file:
Then consume from the topic:
Trace Format
Each exported trace is one record per request routed through FastRouter. Fields ending in Nanos are durations in nanoseconds.
If you use Tracing, the traceparent header you sent is included twice: once as the full value and once split into its components. You can use these fields to regroup multi-step workflows on your side.
Request
transactionId
Unique identifier for the transaction in FastRouter.
requestId
Unique identifier for the API request.
fr_task_id
FastRouter task ID for asynchronous jobs, such as image and video generation.
stream
Whether the request was a streaming request.
statusCode
HTTP status code returned for the request.
errorFromProvider
Error returned by the upstream provider, if the request failed.
storage_url
Location of the stored request and response payload.
tags
Dynamic tags attached to the request.
Organization & Access
orgId
ID of your FastRouter organization.
projectId
ID of the project the request belongs to.
apiKey
The FastRouter API key hash used for the request.
isByokKey
Whether the request was routed through your own provider key (BYOK).
Model & Routing
modelId
ID of the model that served the request.
provider
Provider that served the request.
autoRouter
Whether the request used Automatic Model Selection (fastrouter/auto).
strategyId
ID of the provider routing strategy applied.
strategyType
Type of provider routing strategy applied, such as lowest price or lowest latency.
service_tier_input
Service tier requested, such as flex.
service_tier_applied
Service tier actually applied by the provider.
Timing & Performance
queryTime
Timestamp of the request.
hour
Hour bucket of the request, useful for partitioning and aggregation.
requestReceivedAtRouter
Timestamp when FastRouter received the request.
responseFromRouter
Timestamp when FastRouter returned the response.
responseTimeNanos
Total response time, in nanoseconds.
timeToFirstTokenNanos
Time to first token, in nanoseconds.
timeToLastTokenNanos
Time to last token, in nanoseconds.
tokenPerMinute
Output throughput, in tokens per minute.
Token Usage
inputTokenSize
Total input tokens.
outputTokenSize
Total output tokens.
textInputTokens
Text input tokens.
textOutputTokens
Text output tokens.
audioInputTokens
Audio input tokens.
audioOutputTokens
Audio output tokens.
imageTokens
Image tokens.
videoTokens
Video tokens.
reasoningTokens
Reasoning tokens generated by the model.
input_tokens_cached
Input tokens read from the prompt cache.
output_tokens_cached
Output tokens served from cache.
cachedTextTokens
Cached text tokens.
cachedAudioTokens
Cached audio tokens.
Web Search
searchContextSize
Search context size used for web search, such as low, medium, or high.
citationTokens
Tokens consumed by citations in web search results.
Cost
creditsUsed
FastRouter credits charged for the request.
byokCost
Provider cost of the request when using your own key (BYOK).
Tracing & Session
input_user_id
User identifier supplied with the request.
input_session_id
Session identifier supplied with the request.
input_trace_id
Trace identifier supplied with the request.
trace_parent
Full traceparent header value.
trace_parent_version
Version field of the traceparent header (00).
trace_parent_trace_id
Trace ID from the traceparent header (32 hex characters).
trace_parent_parent_id
Caller span ID from the traceparent header (16 hex characters).
trace_parent_flags
Flags field of the traceparent header.
Prompt
prompt_id
ID of the Prompt Library prompt, if one was used.
prompt_version_id
Version of the Prompt Library prompt used.
prompt_type
Type of prompt.
prompt_content
Content of the prompt.
Prompt Compression
compressedRequest
Whether Prompt Compression was applied.
compressionEngine
Compression engine used.
compressionTokensBefore
Input tokens before compression.
compressionTokensAfter
Input tokens after compression.
compressionEstimatedCostSaved
Estimated cost saved by compression
Security & Data Handling
Credentials are encrypted. Destination credentials are stored encrypted by FastRouter.
Traffic is encrypted in transit. With
SASL_SSL, both credentials and trace payloads are sent over TLS.Traces contain full payloads. Prompts, completions, and any personal data inside them are forwarded as-is. Restrict access to your topic or table accordingly, and apply your own retention and deletion policies.
Use least privilege. Give the FastRouter user permission to write to the traces topic or table only.
Troubleshooting
Test Connection fails to connect
Confirm each host:port is correct and that the port is your SASL_SSL listener. Check that firewalls or security groups allow inbound connections from FastRouter.
Test Connection fails authentication
Verify the username and password, and that the PLAIN mechanism is enabled on the broker listener.
Send Test Trace fails
Make sure the topic exists (FastRouter does not auto-create it) and that the user has permission to write to it.
Test trace sent but not visible
Check your consumer is reading the right topic, from the earliest offset, with valid credentials.
No traces after saving
Confirm the destination was saved with Create Destination, and that your Project and API Key filters match the traffic you're sending.
Last updated