Batch Processing

Upload Batch File

post

Uploads a JSONL file containing multiple batch requests. Each line must include custom_id, provider, method, url, and body. Returns file_id for use in /batches endpoint.

Authorizations

bearerAuth

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Body

multipart/form-data

filestring · binaryRequired

JSONL file containing batch requests. Each line must be a valid JSON object with custom_id, provider, method, url, and body fields.

purposestring · enumOptional

File purpose (always 'batch' for batch processing)

Default: batchPossible values:

Responses

200

File uploaded successfully - use file_id in next step

application/json

file_idstringOptional

Unique file identifier - use this in batch creation

Example: file-abcdef12345

idstringOptional

Same as file_id

filenamestringOptional

Original filename

Example: sample.jsonl

num_rowsintegerOptional

Number of requests in the file

Example: 4

urlstring · uriOptional

Storage URL for the uploaded file

objectstringOptionalExample: file

bytesintegerOptional

File size in bytes

created_atintegerOptional

Unix timestamp

purposestringOptionalExample: batch

400

Bad Request - Invalid file format or content

401

Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.

application/json

413

Payload Too Large - File exceeds size limit

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

post

/v1/files

HTTPcURLJavaScriptPython

POST /v1/files HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: multipart/form-data
Accept: */*
Content-Length: 51

{
  "file": "(binary: sample.jsonl)",
  "purpose": "batch"
}

{
  "file_id": "file-abcdef12345",
  "filename": "sample.jsonl",
  "num_rows": 4,
  "url": "https://storage.googleapis.com/fastrouter-llm-batch-input-files-prod/file-abcdef12345"
}

Create Batch

post

Creates a batch processing job with uploaded file_id. Provides 50% cost savings for asynchronous processing. Supports /v1/chat/completions and /v1/embeddings endpoints. Completion window: 24h.

Authorizations

bearerAuth

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Body

application/json

input_file_idstringRequired

The file_id returned from the file upload step. Must be a valid JSONL file with batch requests.

Example: file-abcdef12345

endpointstring · enumRequired

API endpoint to process. Must match the request types in your JSONL file.

Example: /v1/chat/completionsPossible values:

completion_windowstring · enumRequired

Time window for batch completion. Currently only '24h' is supported.

Default: 24hPossible values:

metadataobjectOptional

Optional metadata object for tracking or organization purposes

Responses

200

Batch created successfully - save batch_id for status checks

application/json

batch_idstringOptional

Unique batch identifier - use this to check status

Example: fr_batch-abc123

idstringOptional

Same as batch_id

Example: fr_batch-abc123

objectstringOptionalExample: batch

endpointstringOptionalExample: /v1/chat/completions

statusstring · enumOptional

Current batch status

Possible values:

created_atintegerOptional

Unix timestamp

input_file_idstringOptional

Input file ID

completion_windowstringOptionalExample: 24h

400

Bad Request - Invalid file ID or parameters

401

Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.

application/json

404

Not Found - File ID not found

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

post

/v1/batches

HTTPcURLJavaScriptPython

POST /v1/batches HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 96

{
  "input_file_id": "file-abcdef12345",
  "endpoint": "/v1/chat/completions",
  "completion_window": "24h"
}

Chat Completions BatchEmbeddings Batch

{
  "batch_id": "fr_batch-abc123",
  "id": "fr_batch-abc123"
}

Get Batch Status

get

Retrieves batch processing status. Poll until status is completed, then use output_file_id to download results. Status values: validating, in_progress, finalizing, completed, failed, cancelled, expired.

Authorizations

bearerAuth

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Path parameters

batch_idstringRequired

Batch ID returned from batch creation

Example: fr_batch-abc123

Responses

200

Batch status and details

application/json

idstringOptional

Batch ID

Example: fr_batch-abc123

objectstringOptionalExample: batch

endpointstringOptionalExample: /v1/chat/completions

modelstringOptional

Model used (if applicable)

errorsobject · nullableOptional

Error information if batch failed

input_file_idstringOptional

Input file ID

completion_windowstringOptionalExample: 24h

statusstring · enumOptional

Current batch status

Possible values:

output_file_idstring · nullableOptional

Output file ID - use this with /files/{file_id}/content to download results (available when completed)

error_file_idstring · nullableOptional

Error file ID if some requests failed

created_atintegerOptional

Batch creation timestamp

in_progress_atintegerOptional

When batch started processing

expires_atintegerOptional

Expiration timestamp

finalizing_atintegerOptional

When batch entered finalizing state

completed_atinteger · nullableOptional

Completion timestamp (null if not completed)

failed_atinteger · nullableOptional

Failure timestamp (null if not failed)

expired_atinteger · nullableOptional

cancelling_atinteger · nullableOptional

cancelled_atinteger · nullableOptional

usageobjectOptional

Token usage and cost information

metadataobject · nullableOptional

Custom metadata if provided

400

Bad Request - Invalid batch ID

401

Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.

application/json

404

Not Found - Batch ID not found

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

get

/v1/batches/{batch_id}

HTTPcURLJavaScriptPython

GET /v1/batches/{batch_id} HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "id": "fr_batch-abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "model": "",
  "errors": null,
  "input_file_id": "file-123",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "batch-output-fr_batch-abc123abc123.jsonl",
  "error_file_id": null,
  "created_at": 1766036467,
  "in_progress_at": 0,
  "expires_at": 0,
  "finalizing_at": 0,
  "completed_at": 1766037007,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": null,
  "cancelled_at": null,
  "request_counts": {},
  "usage": {},
  "metadata": null
}

Download Batch Results

get

Downloads the output JSONL file once batch processing is completed. Use the output_file_id from the batch status response. Each line contains the custom_id, error (if any), and response with body and status_code.

Authorizations

bearerAuth

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Path parameters

file_idstringRequired

Output file ID from batch status response (e.g., batch-output-fr_batch-abc123abc123.jsonl)

Example: batch-output-fr_batch-abc123abc123.jsonl

Responses

200

Batch results file in JSONL format. Each line contains a response for one request.

application/x-ndjson

stringOptional

JSONL content with one result per line. Each line is a JSON object with custom_id, error (null if successful), and response object containing body and status_code.

400

Bad Request - Invalid file ID format

401

Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.

application/json

404

Not Found - File not found, not ready, or expired

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

get

/v1/files/{file_id}/content

HTTPcURLJavaScriptPython

GET /v1/files/{file_id}/content HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{"custom_id":"request-1","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 13 + 25 = 38","refusal":null,"role":"assistant"}}],"created":1766037005,"model":"claude-sonnet-4-5-20250929","object":"chat.completion","service_tier":"default","system_fingerprint":"anthropic_batch_transform"},"status_code":200}}
{"custom_id":"request-2","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 👋\n\nThe next number in the series 2, 4, 6, 8, 10 is **12**.\n\nThis is a simple arithmetic sequence where each number increases by 2:\n- 2 (+2) = 4\n- 4 (+2) = 6\n- 6 (+2) = 8\n- 8 (+2) = 10\n- 10 (+2) = **12**\n\nThe series consists of positive even numbers.","refusal":null,"role":"assistant"}}],"created":1766037005,"model":"claude-sonnet-4-5-20250929","object":"chat.completion","service_tier":"default","system_fingerprint":"anthropic_batch_transform"},"status_code":200}}
{"custom_id":"request-3","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 13 + 25 equals 38.","refusal":null,"role":"assistant"}}],"created":1766036499,"model":"gpt-4.1-nano-2025-04-14","object":"chat.completion","service_tier":"default","system_fingerprint":"fp_38343a2f8f"},"status_code":200}}
{"custom_id":"request-4","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"The next number in the series 2, 4, 6, 8, 10 is **12**. This sequence increases by 2 each time.","refusal":null,"role":"assistant"}}],"created":1766036547,"model":"gpt-4.1-2025-04-14","object":"chat.completion","service_tier":"default","system_fingerprint":"fp_1a2c4a5ede"},"status_code":200}}