# Batch Processing

## FastRouter Batch API

Batch processing allows you to send large volumes of requests efficiently at reduced costs. Upload a JSONL file containing your requests, trigger the batch, monitor its progress, and download the results — all through a simple four-step workflow.

> **Note:** Batch requests are processed asynchronously within a specified completion window, making them ideal for non-time-sensitive workloads such as bulk evaluations, data processing, and large-scale content generation.

***

### 1. File Upload (POST Request)

Upload your JSONL file to FastRouter to initiate the batch process. This returns a `file_id` for use in subsequent steps.

## Upload Batch File

> Uploads a JSONL file containing multiple batch requests. Each line must include custom\_id, provider, method, url, and body. Returns file\_id for use in /batches endpoint.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Batch Processing","description":"Process large volumes of requests asynchronously with 50% cost savings."},{"name":"Files","description":"Manage files for batch processing and other features."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/files":{"post":{"operationId":"uploadFile","tags":["Batch Processing","Files"],"summary":"Upload Batch File","description":"Uploads a JSONL file containing multiple batch requests. Each line must include custom_id, provider, method, url, and body. Returns file_id for use in /batches endpoint.","requestBody":{"required":true,"content":{"multipart/form-data":{"schema":{"type":"object","required":["file"],"properties":{"file":{"type":"string","format":"binary","description":"JSONL file containing batch requests. Each line must be a valid JSON object with custom_id, provider, method, url, and body fields."},"purpose":{"type":"string","enum":["batch"],"default":"batch","description":"File purpose (always 'batch' for batch processing)"}}}}}},"responses":{"200":{"description":"File uploaded successfully - use file_id in next step","content":{"application/json":{"schema":{"type":"object","properties":{"file_id":{"type":"string","description":"Unique file identifier - use this in batch creation"},"id":{"type":"string","description":"Same as file_id"},"filename":{"type":"string","description":"Original filename"},"num_rows":{"type":"integer","description":"Number of requests in the file"},"url":{"type":"string","format":"uri","description":"Storage URL for the uploaded file"},"object":{"type":"string"},"bytes":{"type":"integer","description":"File size in bytes"},"created_at":{"type":"integer","description":"Unix timestamp"},"purpose":{"type":"string"}}}}}},"400":{"description":"Bad Request - Invalid file format or content"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"413":{"description":"Payload Too Large - File exceeds size limit"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

### 2. Trigger Batch (POST Request)

Once the file is uploaded, trigger the batch processing by providing the `input_file_id`, `endpoint`, and `completion_window` (currently accepts only `"24h"` for a 24-hour processing window).

## Create Batch

> Creates a batch processing job with uploaded file\_id. Provides 50% cost savings for asynchronous processing. Supports /v1/chat/completions and /v1/embeddings endpoints. Completion window: 24h.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Batch Processing","description":"Process large volumes of requests asynchronously with 50% cost savings."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/batches":{"post":{"operationId":"createBatch","tags":["Batch Processing"],"summary":"Create Batch","description":"Creates a batch processing job with uploaded file_id. Provides 50% cost savings for asynchronous processing. Supports /v1/chat/completions and /v1/embeddings endpoints. Completion window: 24h.","requestBody":{"required":true,"content":{"application/json":{"schema":{"type":"object","required":["input_file_id","endpoint","completion_window"],"properties":{"input_file_id":{"type":"string","description":"The file_id returned from the file upload step. Must be a valid JSONL file with batch requests."},"endpoint":{"type":"string","enum":["/v1/chat/completions","/v1/embeddings"],"description":"API endpoint to process. Must match the request types in your JSONL file."},"completion_window":{"type":"string","enum":["24h"],"default":"24h","description":"Time window for batch completion. Currently only '24h' is supported."},"metadata":{"type":"object","description":"Optional metadata object for tracking or organization purposes"}}}}}},"responses":{"200":{"description":"Batch created successfully - save batch_id for status checks","content":{"application/json":{"schema":{"type":"object","properties":{"batch_id":{"type":"string","description":"Unique batch identifier - use this to check status"},"id":{"type":"string","description":"Same as batch_id"},"object":{"type":"string"},"endpoint":{"type":"string"},"status":{"type":"string","enum":["validating","in_progress","finalizing","completed","failed","cancelled","expired"],"description":"Current batch status"},"created_at":{"type":"integer","description":"Unix timestamp"},"input_file_id":{"type":"string","description":"Input file ID"},"completion_window":{"type":"string"}}}}}},"400":{"description":"Bad Request - Invalid file ID or parameters"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - File ID not found"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

### 3. Get Status (GET Request)

Check the status of your batch job using the `batch_id` returned from the trigger step. Statuses include `"in_progress"`, `"completed"`, `"failed"`, etc.

## Get Batch Status

> Retrieves batch processing status. Poll until status is completed, then use output\_file\_id to download results. Status values: validating, in\_progress, finalizing, completed, failed, cancelled, expired.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Batch Processing","description":"Process large volumes of requests asynchronously with 50% cost savings."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/batches/{batch_id}":{"get":{"operationId":"retrieveBatch","tags":["Batch Processing"],"summary":"Get Batch Status","description":"Retrieves batch processing status. Poll until status is completed, then use output_file_id to download results. Status values: validating, in_progress, finalizing, completed, failed, cancelled, expired.","parameters":[{"name":"batch_id","in":"path","required":true,"schema":{"type":"string"},"description":"Batch ID returned from batch creation"}],"responses":{"200":{"description":"Batch status and details","content":{"application/json":{"schema":{"type":"object","properties":{"id":{"type":"string","description":"Batch ID"},"object":{"type":"string"},"endpoint":{"type":"string"},"model":{"type":"string","description":"Model used (if applicable)"},"errors":{"type":"object","nullable":true,"description":"Error information if batch failed"},"input_file_id":{"type":"string","description":"Input file ID"},"completion_window":{"type":"string"},"status":{"type":"string","enum":["validating","in_progress","finalizing","completed","failed","cancelled","expired"],"description":"Current batch status"},"output_file_id":{"type":"string","nullable":true,"description":"Output file ID - use this with /files/{file_id}/content to download results (available when completed)"},"error_file_id":{"type":"string","nullable":true,"description":"Error file ID if some requests failed"},"created_at":{"type":"integer","description":"Batch creation timestamp"},"in_progress_at":{"type":"integer","description":"When batch started processing"},"expires_at":{"type":"integer","description":"Expiration timestamp"},"finalizing_at":{"type":"integer","description":"When batch entered finalizing state"},"completed_at":{"type":"integer","nullable":true,"description":"Completion timestamp (null if not completed)"},"failed_at":{"type":"integer","nullable":true,"description":"Failure timestamp (null if not failed)"},"expired_at":{"type":"integer","nullable":true},"cancelling_at":{"type":"integer","nullable":true},"cancelled_at":{"type":"integer","nullable":true},"request_counts":{"type":"object","description":"Request statistics","properties":{"total":{"type":"integer","description":"Total requests in batch"},"completed":{"type":"integer","description":"Successfully completed requests"},"failed":{"type":"integer","description":"Failed requests"}}},"usage":{"type":"object","description":"Token usage and cost information"},"metadata":{"type":"object","nullable":true,"description":"Custom metadata if provided"}}}}}},"400":{"description":"Bad Request - Invalid batch ID"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - Batch ID not found"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```

### 4. Download File (GET Request)

Once the batch is completed (verify via the status endpoint), download the output JSONL file containing responses for each request.

## Download Batch Results

> Downloads the output JSONL file once batch processing is completed. Use the output\_file\_id from the batch status response. Each line contains the custom\_id, error (if any), and response with body and status\_code.

```json
{"openapi":"3.1.0","info":{"title":"FastRouter API Reference","version":"1.0.0"},"tags":[{"name":"Batch Processing","description":"Process large volumes of requests asynchronously with 50% cost savings."},{"name":"Files","description":"Manage files for batch processing and other features."}],"servers":[{"url":"https://api.fastrouter.ai","description":"Production API"}],"security":[{"bearerAuth":[]}],"components":{"securitySchemes":{"bearerAuth":{"type":"http","scheme":"bearer","bearerFormat":"API Key","description":"FastRouter API Key. Get yours at https://fastrouter.ai\n\nFormat: `Authorization: Bearer YOUR_API_KEY`"}},"responses":{"UnauthorizedError":{"description":"Invalid Credentials - Your API key is invalid, disabled, or your OAuth session has expired. Check your credentials.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}},"RateLimitError":{"description":"Rate Limited - You have exceeded your request limits (TPM/RPM). Slow down or increase your limits.","content":{"application/json":{"schema":{"type":"object","properties":{"error":{"type":"object","properties":{"message":{"type":"string"},"type":{"type":"string"},"code":{"type":"string"},"status":{"type":"integer"}}}}}}}}}},"paths":{"/v1/files/{file_id}/content":{"get":{"operationId":"getFileContent","tags":["Batch Processing","Files"],"summary":"Download Batch Results","description":"Downloads the output JSONL file once batch processing is completed. Use the output_file_id from the batch status response. Each line contains the custom_id, error (if any), and response with body and status_code.","parameters":[{"name":"file_id","in":"path","required":true,"schema":{"type":"string"},"description":"Output file ID from batch status response (e.g., batch-output-fr_batch-abc123abc123.jsonl)"}],"responses":{"200":{"description":"Batch results file in JSONL format. Each line contains a response for one request.","content":{"application/x-ndjson":{"schema":{"type":"string","description":"JSONL content with one result per line. Each line is a JSON object with custom_id, error (null if successful), and response object containing body and status_code."}}}},"400":{"description":"Bad Request - Invalid file ID format"},"401":{"$ref":"#/components/responses/UnauthorizedError"},"404":{"description":"Not Found - File not found, not ready, or expired"},"429":{"$ref":"#/components/responses/RateLimitError"},"500":{"description":"Internal Server Error"}}}}}}
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fastrouter.ai/api-reference/batch-processing.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.