anonrouterdocs
ModelsChatStudioPrivacyQuickstart

Batch API Quickstart

Submit and retrieve asynchronous batches of inference requests

Design example

Visual prototype, not a supported AnonRouter endpoint

This page is here to demonstrate the documentation experience, including tabbed examples, API payloads, tables, callouts, and long-form navigation. The OpenRouter Batch API shown below is not currently part of AnonRouter's supported API surface.

The Batch API lets you submit many inference requests together and retrieve the results asynchronously. It is useful for work that does not need an immediate response, and it uses a 24-hour completion window so you can process requests without managing each call yourself.

The API supports several request shapes, including chat completions, Responses, Anthropic Messages, and embeddings. Requests are submitted as an inline JSON requests array.

Asynchronous results

A successful submission returns 202 Accepted with status: "validating". This means the request was persisted and queued for validation, not that every request has completed.

Submit a batch

Submit a batch with:

Endpoint
POST https://openrouter.ai/api/beta/batches

The request body has three required top-level fields:

FieldDescription
endpointThe API shape used by every request in the batch. Choose /v1/chat/completions, /v1/responses, /v1/messages, or /v1/embeddings.
modelAn OpenRouter model slug, such as openai/gpt-4o. This batch-level model is applied to every request.
requestsA non-empty array of { custom_id, body } items. custom_id must be unique within the batch, and body follows the selected endpoint.

Field order matters

Serialize endpoint and model before requests in the JSON body. The API stream-parses the request so it can accept very large arrays, and returns a 400 if requests appears first. Every example below uses the required order.

The batch-level model applies to every request. An individual request body can omit model to inherit it. If a request body specifies a model, it must match the batch-level model.

import json
import requests

response = requests.post(
    url="https://openrouter.ai/api/beta/batches",
    headers={
        "Authorization": "Bearer <OPENROUTER_API_KEY>",
        "Content-Type": "application/json",
    },
    data=json.dumps({
        "endpoint": "/v1/chat/completions",
        "model": "openai/gpt-4o",
        "requests": [
            {
                "custom_id": "req-0001",
                "body": {
                    "messages": [
                        {
                            "role": "user",
                            "content": "Summarize OpenRouter in one sentence."
                        }
                    ]
                }
            }
        ]
    })
)

print(response.json())

The response is a batch object with an ID you can use to check progress:

202 Accepted
{
  "id": "batch_123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "model": "openai/gpt-4o",
  "completion_window": "24h",
  "status": "validating",
  "created_at": 1782097200,
  "finalized_at": null,
  "request_counts": {
    "total": 1,
    "completed": 0,
    "failed": 0
  },
  "usage": null,
  "results": null,
  "error": null
}

The only supported completion window is 24h.

Poll for results

Use the batch ID to retrieve the current status:

Endpoint
GET https://openrouter.ai/api/beta/batches/:id

For example:

Shell
curl https://openrouter.ai/api/beta/batches/batch_123 \
  -H "Authorization: Bearer $OPENROUTER_API_KEY"

The status normally progresses through:

Status progression
validating → in_progress → finalizing → completed

Other possible statuses are failed, expired, cancelling, and cancelled. The terminal statuses are completed, failed, expired, and cancelled. Poll until the batch reaches a terminal status.

request_counts contains the total number of requests and the number that completed or failed:

request_counts
{
  "total": 100,
  "completed": 98,
  "failed": 2
}

While a batch is in progress, or has failed, expired, or been cancelled, results is null. When it completes, results are returned inline in the same response. Each result maps to an input using custom_id, and exactly one of response or error is populated.

Result item
{
  "id": "batch_req_123",
  "custom_id": "req-0001",
  "response": {
    "status_code": 200,
    "request_id": "request_123",
    "body": {
      "id": "gen-batch-1782097200-a1b2c3d4e5f6a7b8c9d0",
      "object": "chat.completion",
      "created": 1782097200,
      "model": "openai/gpt-4o",
      "choices": [
        {
          "index": 0,
          "message": {
            "role": "assistant",
            "content": "OpenRouter provides one API for many AI models."
          },
          "finish_reason": "stop"
        }
      ]
    }
  },
  "error": null
}

A completed batch response looks like this:

Completed batch
{
  "id": "batch_123",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "model": "openai/gpt-4o",
  "completion_window": "24h",
  "status": "completed",
  "created_at": 1782097200,
  "finalized_at": 1782100800,
  "request_counts": {
    "total": 1,
    "completed": 1,
    "failed": 0
  },
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 40,
    "total_tokens": 60,
    "cost": 0.00045,
    "is_byok": false
  },
  "results": [
    {
      "id": "batch_req_123",
      "custom_id": "req-0001",
      "response": {
        "status_code": 200,
        "request_id": "request_123",
        "body": {
          "id": "gen-batch-1782097200-a1b2c3d4e5f6a7b8c9d0",
          "object": "chat.completion",
          "model": "openai/gpt-4o",
          "choices": [
            {
              "index": 0,
              "message": {
                "role": "assistant",
                "content": "OpenRouter provides one API for many AI models."
              },
              "finish_reason": "stop"
            }
          ]
        }
      },
      "error": null
    }
  ],
  "error": null
}

Reporting issues

Each completed result's response.body.id is that request's generation ID. To flag a bad generation, copy that ID and submit it through the provider's generation feedback flow.

Keep the generation ID

Generation-level feedback is available for the chat completions, Responses, and Messages request shapes.

Use different API shapes

Set the top-level endpoint to choose the request shape used by every body in the batch:

  • Chat completions: /v1/chat/completions
  • Responses: /v1/responses
  • Anthropic Messages: /v1/messages

For example, an Anthropic Messages batch uses /v1/messages and puts the Messages-shaped request in each item's body:

Anthropic Messages batch
{
  "endpoint": "/v1/messages",
  "model": "anthropic/claude-3.5-sonnet",
  "requests": [
    {
      "custom_id": "req-1",
      "body": {
        "max_tokens": 32,
        "messages": [
          {
            "role": "user",
            "content": "Say hello."
          }
        ]
      }
    }
  ]
}

All requests in one batch use the same top-level endpoint. To mix API shapes, submit separate batches.

Embeddings

Embeddings are rolling out on providers that support them.

Set the top-level endpoint to /v1/embeddings and put the embeddings request in each item's body. Each body takes an input, which may be a string, an array of strings, a token array, or an array of token arrays.

Embeddings batch
{
  "endpoint": "/v1/embeddings",
  "model": "openai/text-embedding-3-small",
  "requests": [
    {
      "custom_id": "emb-0001",
      "body": {
        "input": [
          "The quick brown fox jumped over the lazy dog.",
          "Pack my box with five dozen liquor jugs."
        ]
      }
    },
    {
      "custom_id": "emb-0002",
      "body": {
        "input": "The quick brown fox jumped over the lazy dog."
      }
    }
  ]
}

Poll for results the same way as any other batch. Each result carries the standard embeddings response in its body, with one result per custom_id.

Embeddings results
[
  {
    "id": "batch_req_emb_1",
    "custom_id": "emb-0001",
    "response": {
      "status_code": 200,
      "request_id": "request_456",
      "body": {
        "object": "list",
        "data": [
          {
            "object": "embedding",
            "embedding": [0.0023064255, -0.009327292, 0.015797347],
            "index": 0
          },
          {
            "object": "embedding",
            "embedding": [-0.012282, 0.0034567, -0.0089123],
            "index": 1
          }
        ],
        "model": "openai/text-embedding-3-small",
        "usage": {
          "prompt_tokens": 18,
          "total_tokens": 18
        }
      }
    },
    "error": null
  }
]

Retention

Batch inputs and results are commonly retained as JSONL artifacts for a limited window. Download any results you need before the provider's retention window elapses.

API reference

The public HTTP surface for AnonRouter integrations.

On this page

Submit a batchPoll for resultsReporting issuesUse different API shapesEmbeddings