Batch Processing

FastRouter Batch API

Batch processing allows you to send large volumes of requests efficiently at reduced costs. Upload a JSONL file containing your requests, trigger the batch, monitor its progress, and download the results — all through a simple four-step workflow.

Note: Batch requests are processed asynchronously within a specified completion window, making them ideal for non-time-sensitive workloads such as bulk evaluations, data processing, and large-scale content generation.

---

1. File Upload (POST Request)

Upload your JSONL file to FastRouter to initiate the batch process. This returns a file_id for use in subsequent steps.

Upload Batch File

post

Uploads a JSONL file containing multiple batch requests. Each line must include custom_id, provider, method, url, and body. Returns file_id for use in /batches endpoint.

Authorizations

bearerAuth

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Body

multipart/form-data

filestring · binaryRequired

JSONL file containing batch requests. Each line must be a valid JSON object with custom_id, provider, method, url, and body fields.

purposestring · enumOptional

File purpose (always 'batch' for batch processing)

Default: batchPossible values:

Responses

200

File uploaded successfully - use file_id in next step

application/json

file_idstringOptional

Unique file identifier - use this in batch creation

Example: file-abcdef12345

idstringOptional

Same as file_id

filenamestringOptional

Original filename

Example: sample.jsonl

num_rowsintegerOptional

Number of requests in the file

Example: 4

urlstring · uriOptional

Storage URL for the uploaded file

objectstringOptionalExample: file

bytesintegerOptional

File size in bytes

created_atintegerOptional

Unix timestamp

purposestringOptionalExample: batch

400

Bad Request - Invalid file format or content

401

Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.

Note: the 401 error body uses code, message, param, and type (there is no status field), and type is invalid_request_error.

application/json

413

Payload Too Large - File exceeds size limit

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

post/v1/files

HTTPcURLJavaScriptPython

POST /v1/files HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: multipart/form-data
Accept: */*
Content-Length: 51

{
  "file": "(binary: sample.jsonl)",
  "purpose": "batch"
}

{
  "file_id": "file-abcdef12345",
  "filename": "sample.jsonl",
  "num_rows": 4,
  "url": "https://storage.googleapis.com/fastrouter-llm-batch-input-files-prod/file-abcdef12345"
}

2. Trigger Batch (POST Request)

Once the file is uploaded, trigger the batch processing by providing the input_file_id, endpoint, and completion_window (currently accepts only "24h" for a 24-hour processing window).

Create Batch

post

Creates a batch processing job with uploaded file_id. Provides 50% cost savings for asynchronous processing. Supports /v1/chat/completions and /v1/embeddings endpoints. Completion window: 24h.

Authorizations

bearerAuth

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Body

application/json

input_file_idstringRequired

The file_id returned from the file upload step. Must be a valid JSONL file with batch requests.

Example: file-abcdef12345

endpointstring · enumRequired

API endpoint to process. Must match the request types in your JSONL file.

Example: /v1/chat/completionsPossible values:

completion_windowstring · enumRequired

Time window for batch completion. Currently only '24h' is supported.

Default: 24hPossible values:

metadataobjectOptional

Optional metadata object for tracking or organization purposes

Responses

200

Batch created successfully - save batch_id for status checks

application/json

batch_idstringOptional

Unique batch identifier - use this to check status

Example: fr_batch-abc123

idstringOptional

Same as batch_id

Example: fr_batch-abc123

objectstringOptionalExample: batch

endpointstringOptionalExample: /v1/chat/completions

statusstring · enumOptional

Current batch status

Possible values:

created_atintegerOptional

Unix timestamp

input_file_idstringOptional

Input file ID

completion_windowstringOptionalExample: 24h

400

Bad Request - Invalid file ID or parameters

401

Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.

Note: the 401 error body uses code, message, param, and type (there is no status field), and type is invalid_request_error.

application/json

404

Not Found - File ID not found

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

post/v1/batches

HTTPcURLJavaScriptPython

POST /v1/batches HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 96

{
  "input_file_id": "file-abcdef12345",
  "endpoint": "/v1/chat/completions",
  "completion_window": "24h"
}

Chat Completions BatchEmbeddings Batch

{
  "batch_id": "fr_batch-abc123",
  "id": "fr_batch-abc123"
}

3. Get Status (GET Request)

Check the status of your batch job using the batch_id returned from the trigger step. Statuses include "in_progress", "completed", "failed", etc.

Get Batch Status

get

Retrieves batch processing status. Poll until status is completed, then use output_file_id to download results. Status values: validating, in_progress, finalizing, completed, failed, cancelled, expired.

Authorizations

bearerAuth

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Path parameters

batch_idstringRequired

Batch ID returned from batch creation

Example: fr_batch-abc123

Responses

200

Batch status and details

application/json

idstringOptional

Batch ID

Example: fr_batch-abc123

objectstringOptionalExample: batch

endpointstringOptionalExample: /v1/chat/completions

modelstringOptional

Model used (if applicable)

errorsobject · nullableOptional

Error information if batch failed

input_file_idstringOptional

Input file ID

completion_windowstringOptionalExample: 24h

statusstring · enumOptional

Current batch status

Possible values:

output_file_idstring · nullableOptional

Output file ID - use this with /files/{file_id}/content to download results (available when completed)

error_file_idstring · nullableOptional

Error file ID if some requests failed

created_atintegerOptional

Batch creation timestamp

in_progress_atintegerOptional

When batch started processing

expires_atintegerOptional

Expiration timestamp

finalizing_atintegerOptional

When batch entered finalizing state

completed_atinteger · nullableOptional

Completion timestamp (null if not completed)

failed_atinteger · nullableOptional

Failure timestamp (null if not failed)

expired_atinteger · nullableOptional

cancelling_atinteger · nullableOptional

cancelled_atinteger · nullableOptional

usageobjectOptional

Token usage and cost information

metadataobject · nullableOptional

Custom metadata if provided

400

Bad Request - Invalid batch ID

401

Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.

Note: the 401 error body uses code, message, param, and type (there is no status field), and type is invalid_request_error.

application/json

404

Not Found - Batch ID not found

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

get/v1/batches/{batch_id}

HTTPcURLJavaScriptPython

GET /v1/batches/{batch_id} HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "id": "fr_batch-abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "model": "",
  "errors": null,
  "input_file_id": "file-123",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "batch-output-fr_batch-abc123abc123.jsonl",
  "error_file_id": null,
  "created_at": 1766036467,
  "in_progress_at": 0,
  "expires_at": 0,
  "finalizing_at": 0,
  "completed_at": 1766037007,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": null,
  "cancelled_at": null,
  "request_counts": {},
  "usage": {},
  "metadata": null
}

4. Download File (GET Request)

Once the batch is completed (verify via the status endpoint), download the output JSONL file containing responses for each request.

Download Batch Results

get

Downloads the output JSONL file once batch processing is completed. Use the output_file_id from the batch status response. Each line contains the custom_id, error (if any), and response with body and status_code.

Authorizations

bearerAuth

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Path parameters

file_idstringRequired

Output file ID from batch status response (e.g., batch-output-fr_batch-abc123abc123.jsonl)

Example: batch-output-fr_batch-abc123abc123.jsonl

Responses

200

Batch results file in JSONL format. Each line contains a response for one request.

application/x-ndjson

stringOptional

JSONL content with one result per line. Each line is a JSON object with custom_id, error (null if successful), and response object containing body and status_code.

400

Bad Request - Invalid file ID format

401

Invalid Credentials - Your API key is invalid, missing, or disabled. Check your credentials.

Note: the 401 error body uses code, message, param, and type (there is no status field), and type is invalid_request_error.

application/json

404

Not Found - File not found, not ready, or expired

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

get/v1/files/{file_id}/content

HTTPcURLJavaScriptPython

GET /v1/files/{file_id}/content HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{"custom_id":"request-1","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 13 + 25 = 38","refusal":null,"role":"assistant"}}],"created":1766037005,"model":"claude-sonnet-4-5-20250929","object":"chat.completion","service_tier":"default","system_fingerprint":"anthropic_batch_transform"},"status_code":200}}
{"custom_id":"request-2","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 👋\n\nThe next number in the series 2, 4, 6, 8, 10 is **12**.\n\nThis is a simple arithmetic sequence where each number increases by 2:\n- 2 (+2) = 4\n- 4 (+2) = 6\n- 6 (+2) = 8\n- 8 (+2) = 10\n- 10 (+2) = **12**\n\nThe series consists of positive even numbers.","refusal":null,"role":"assistant"}}],"created":1766037005,"model":"claude-sonnet-4-5-20250929","object":"chat.completion","service_tier":"default","system_fingerprint":"anthropic_batch_transform"},"status_code":200}}
{"custom_id":"request-3","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 13 + 25 equals 38.","refusal":null,"role":"assistant"}}],"created":1766036499,"model":"gpt-4.1-nano-2025-04-14","object":"chat.completion","service_tier":"default","system_fingerprint":"fp_38343a2f8f"},"status_code":200}}
{"custom_id":"request-4","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"The next number in the series 2, 4, 6, 8, 10 is **12**. This sequence increases by 2 each time.","refusal":null,"role":"assistant"}}],"created":1766036547,"model":"gpt-4.1-2025-04-14","object":"chat.completion","service_tier":"default","system_fingerprint":"fp_1a2c4a5ede"},"status_code":200}}