Batch Processing | FastRouter.AI Docs

Upload Batch File

post

Uploads a JSONL file containing multiple batch requests. Each line must include custom_id, provider, method, url, and body. Returns file_id for use in /batches endpoint.

Authorizations

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Body

filestring · binaryRequired

JSONL file containing batch requests. Each line must be a valid JSON object with custom_id, provider, method, url, and body fields.

purposestring · enumOptional

File purpose (always 'batch' for batch processing)

Default: batchPossible values:

Responses

200

File uploaded successfully - use file_id in next step

application/json

400

Bad Request - Invalid file format or content

401

Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.

application/json

413

Payload Too Large - File exceeds size limit

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

post

/files

POST /api/v1/files HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: multipart/form-data
Accept: */*
Content-Length: 51

{
  "file": "(binary: sample.jsonl)",
  "purpose": "batch"
}

{
  "file_id": "file-abcdef12345",
  "filename": "sample.jsonl",
  "num_rows": 4,
  "url": "https://storage.googleapis.com/fastrouter-llm-batch-input-files-prod/file-abcdef12345"
}

Create Batch

post

Creates a batch processing job with uploaded file_id. Provides 50% cost savings for asynchronous processing. Supports /v1/chat/completions and /v1/embeddings endpoints. Completion window: 24h.

Authorizations

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Body

input_file_idstringRequired

The file_id returned from the file upload step. Must be a valid JSONL file with batch requests.

Example: file-abcdef12345

endpointstring · enumRequired

API endpoint to process. Must match the request types in your JSONL file.

Example: /v1/chat/completionsPossible values:

completion_windowstring · enumRequired

Time window for batch completion. Currently only '24h' is supported.

Default: 24hPossible values:

metadataobjectOptional

Optional metadata object for tracking or organization purposes

Responses

200

Batch created successfully - save batch_id for status checks

application/json

400

Bad Request - Invalid file ID or parameters

401

Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.

application/json

404

Not Found - File ID not found

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

post

/batches

POST /api/v1/batches HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 96

{
  "input_file_id": "file-abcdef12345",
  "endpoint": "/v1/chat/completions",
  "completion_window": "24h"
}

{
  "batch_id": "fr_batch-abc123",
  "id": "fr_batch-abc123"
}

Get Batch Status

get

Retrieves batch processing status. Poll until status is completed, then use output_file_id to download results. Status values: validating, in_progress, finalizing, completed, failed, cancelled, expired.

Authorizations

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Path parameters

batch_idstringRequired

Batch ID returned from batch creation

Example: fr_batch-abc123

Responses

200

Batch status and details

application/json

400

Bad Request - Invalid batch ID

401

Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.

application/json

404

Not Found - Batch ID not found

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

get

/batches/{batch_id}

GET /api/v1/batches/{batch_id} HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "id": "fr_batch-abc123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "model": "",
  "errors": null,
  "input_file_id": "file-123",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "batch-output-fr_batch-abc123abc123.jsonl",
  "error_file_id": null,
  "created_at": 1766036467,
  "in_progress_at": 0,
  "expires_at": 0,
  "finalizing_at": 0,
  "completed_at": 1766037007,
  "failed_at": null,
  "expired_at": null,
  "cancelling_at": null,
  "cancelled_at": null,
  "request_counts": {},
  "usage": {},
  "metadata": null
}

Download Batch Results

get

Downloads the output JSONL file once batch processing is completed. Use the output_file_id from the batch status response. Each line contains the custom_id, error (if any), and response with body and status_code.

Authorizations

AuthorizationstringRequired

FastRouter API Key. Get yours at https://fastrouter.ai

Format: Authorization: Bearer YOUR_API_KEY

Path parameters

file_idstringRequired

Output file ID from batch status response (e.g., batch-output-fr_batch-abc123abc123.jsonl)

Example: batch-output-fr_batch-abc123abc123.jsonl

Responses

200

Batch results file in JSONL format. Each line contains a response for one request.

application/x-ndjson

Responsestring

JSONL content with one result per line. Each line is a JSON object with custom_id, error (null if successful), and response object containing body and status_code.

400

Bad Request - Invalid file ID format

401

Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.

application/json

404

Not Found - File not found, not ready, or expired

429

Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.

application/json

500

Internal Server Error

get

/files/{file_id}/content

GET /api/v1/files/{file_id}/content HTTP/1.1
Host: api.fastrouter.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{"custom_id":"request-1","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 13 + 25 = 38","refusal":null,"role":"assistant"}}],"created":1766037005,"model":"claude-sonnet-4-5-20250929","object":"chat.completion","service_tier":"default","system_fingerprint":"anthropic_batch_transform"},"status_code":200}}
{"custom_id":"request-2","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 👋\n\nThe next number in the series 2, 4, 6, 8, 10 is **12**.\n\nThis is a simple arithmetic sequence where each number increases by 2:\n- 2 (+2) = 4\n- 4 (+2) = 6\n- 6 (+2) = 8\n- 8 (+2) = 10\n- 10 (+2) = **12**\n\nThe series consists of positive even numbers.","refusal":null,"role":"assistant"}}],"created":1766037005,"model":"claude-sonnet-4-5-20250929","object":"chat.completion","service_tier":"default","system_fingerprint":"anthropic_batch_transform"},"status_code":200}}
{"custom_id":"request-3","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"Hello! 13 + 25 equals 38.","refusal":null,"role":"assistant"}}],"created":1766036499,"model":"gpt-4.1-nano-2025-04-14","object":"chat.completion","service_tier":"default","system_fingerprint":"fp_38343a2f8f"},"status_code":200}}
{"custom_id":"request-4","error":null,"response":{"body":{"choices":[{"finish_reason":"stop","index":0,"logprobs":null,"message":{"annotations":[],"content":"The next number in the series 2, 4, 6, 8, 10 is **12**. This sequence increases by 2 each time.","refusal":null,"role":"assistant"}}],"created":1766036547,"model":"gpt-4.1-2025-04-14","object":"chat.completion","service_tier":"default","system_fingerprint":"fp_1a2c4a5ede"},"status_code":200}}

hashtagUpload Batch File

hashtagCreate Batch

hashtagGet Batch Status

hashtagDownload Batch Results

Upload Batch File

Create Batch

Get Batch Status

Download Batch Results