Batch APIs Compared: OpenAI, Anthropic and Gemini

All three major vendors sell the same deal. You submit a list of requests, collect the results within 24 hours and pay half the synchronous price. The vendors differ in their limits, their queue rules and the fate of a job that does not finish in time. Every price and limit below is as of October 2026, taken from the vendor page linked in its row.

A batch API takes work that nobody is waiting on and moves it to a queue the vendor drains when capacity is spare. The discount pays you for waiting. For a 500,000-document classification job, the saving runs from about $270 to $725 depending on the model. The arithmetic is further down.

How a batch job flows

The flow is the same everywhere. The one structural difference is where the input lives. OpenAI reads a file you uploaded first. Anthropic takes the request list inside the API call. Gemini accepts either an inline list under 20 MB or an uploaded JSONL file.

Results can come back out of order. Every request therefore carries an id you choose (custom_id at OpenAI and Anthropic, key at Gemini), and your parser matches on it.

OpenAI Batch API

Each batch holds up to 50,000 requests in one JSONL file of at most 200 MB. The only completion window is 24h. At the deadline, OpenAI cancels any unfinished requests and moves the batch to the expired status. It still bills the requests that completed. OpenAI deletes the output file 30 days after the batch completes.

Batch traffic draws from its own rate-limit pool. Each model has a cap on enqueued prompt tokens. Your Settings page shows the cap; the docs do not. You can create up to 2,000 batches per hour, and batches have no output-token limit.

One JSONL line is one request. The snippet is illustrative and wrapped for reading; in the file it stays on one line.

{"custom_id": "doc-000001", "method": "POST", "url": "/v1/chat/completions",
 "body": {"model": "gpt-6.1-terra",
          "messages": [{"role": "system", "content": "Classify the document as invoice, contract, memo or other. Reply with the label only."},
                       {"role": "user", "content": "<document text>"}],
          "max_completion_tokens": 50}}

batch_input_file = client.files.create(file=open("batchinput.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=batch_input_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)

OpenAI's GPT-6 Astra model page also lists Flex processing at the same price as batch. Flex takes requests one at a time instead of as a file. OpenRouter's listing shows the same Flex price for GPT-6.1 Terra.

Anthropic Message Batches

Anthropic caps a batch at 100,000 requests or 256 MB, whichever comes first. The docs say most batches complete within an hour. The hard limit is 24 hours, after which the batch expires. Results stay downloadable for 29 days after creation. All active models support batches, except in Fast mode. Each request needs max_tokens of at least 1.

Rate limits apply to the batch calls themselves and to the number of requests waiting in the queue. The rate-limits page prints them by tier: Start 1,000 batch requests per minute and 200,000 requests in the processing queue; Build 2,000 and 300,000; Scale 4,000 and 500,000. Batch usage does not count against the Messages API limits.

The request list goes inside the call. The snippet is illustrative and uses the model id from Anthropic's own example.

message_batch = client.messages.batches.create(
    requests=[
        {
            "custom_id": "doc-000001",
            "params": {
                "model": "claude-opus-5-5",
                "max_tokens": 50,
                "messages": [{"role": "user", "content": "<document text>"}],
            },
        },
        # ... up to 100,000 requests or 256 MB
    ]
)

Gemini Batch API

Gemini has two input modes. Inline requests go straight into the batches.create call and must total under 20 MB. Larger jobs upload a JSONL file of up to 2 GB. The rate-limits page adds limits of 20 GB of file storage and 100 concurrent batch jobs. The docs do not state a maximum request count. The turnaround target is 24 hours. A job still pending or running after 48 hours moves to JOB_STATE_EXPIRED. Google stores results for 6 weeks.

Gemini expresses rate limits as enqueued tokens per model per tier (see the next section). Embeddings have their own batch call, batches.create_embeddings.

Illustrative inline call, with the model id from Google's example:

inline_requests = [
    {"contents": [{"parts": [{"text": "Classify: <document text>"}], "role": "user"}]},
    # ... keep the whole list under 20 MB
]
inline_batch_job = client.batches.create(
    model="gemini-3.8-flash",
    src=inline_requests,
    config={"display_name": "classify-2026-10-31"},
)

A line of the file-based variant looks like {"key": "request-1", "request": {"contents": [{"parts": [{"text": "..."}]}]}}. Poll batch_job.state.name until it reaches a terminal state. Then read batch_job.dest.file_name and download it with client.files.download.

Which workloads fit

A batch queue suits anything that is produced in bulk and consumed later:

• Evaluations: run a test set against a new prompt or model overnight.
• Backfills: re-tag, re-summarise or re-extract an existing corpus after a schema change.
• Nightly classification: label the day's tickets, documents or transactions before the morning report.
• Embeddings generation: index a corpus; OpenAI and Gemini both expose embeddings in batch.
• Report generation: per-account or per-region summaries on a schedule.

Two kinds of work do not fit. The first is anything user-facing. The only promise is "within 24 hours", with nothing about the first minute. The second is anything with a deadline shorter than the window. A job that must finish by 09:00 cannot go into a queue that may legitimately take until tomorrow afternoon. Anthropic's "most batches completing within 1 hour" and Google's "much quicker in most cases" are observations. Neither vendor guarantees them.

The queue limits decide how much you can submit at once:

Gemini's Tier 1 cap of 3,000,000 enqueued tokens on 3.8 Flash is small. It holds roughly 2,500 documents of 1,200 tokens in the queue at a time. Tier 2 (after $100 paid and three days) raises it to 400,000,000. The Gemini free tier does not offer batch at all.

Beyond the big three, terms vary. Mistral's plan page says batch "reduces the price by 50%". xAI gives 20% on its listed Grok models. DeepSeek lists no batch discount and uses off-peak hours (50% of peak price) as its scheduling discount instead. Gemini also has service tiers for synchronous calls. Its flex tier bills at the batch price, and priority costs 1.8x standard.

Batch plus cache

The two discounts address different tokens. Batch halves the price of every token in the job, and caching cuts the price of a repeated prefix. Each vendor handles the combination differently.

• Anthropic: the docs state that the two discounts "can stack". Cache hits inside a batch are best-effort, because requests run concurrently. The docs report hit rates from 30% to 98% depending on traffic pattern. They recommend the 1-hour cache duration for batches with shared context. They also note that batches do not support max_tokens: 0 pre-warming.
• Gemini: batch requests support context caching through cached_content. For a cache hit in a batch, the docs say "you pay the standard context caching rates". The cached portion gets the caching price, and the rest of the request gets the batch price.
• OpenAI: the batch guide we read does not state how batch pricing interacts with cached input pricing.

For the structure of a cacheable prompt, see the prompt caching guide.

Classifying 500,000 documents

The job is to classify 500,000 documents with 1,200 input tokens and 50 output tokens each. Total input is 600,000,000 tokens (600M), and total output is 25,000,000 tokens (25M). The table uses one mid-tier model per vendor, with prices from the vendor pages. OpenAI's batch guide puts batch at 50% of the synchronous price, which gives $0.50 / $2.50 for GPT-6.1 Terra.

Output is about 17% of each bill here. The job sends 1,200 tokens in and gets 50 out, and all three models price output at five times input. That ratio is typical of classification and extraction. It is why batch pays off most on long-input, short-output jobs. Per the pricing page, Gemini 3.8 Flash's listed prices rise on 1 January 2027 to $1.50 / $7.50 (batch $0.75 / $3.75).

The job also has to clear the queue limits above. OpenAI needs ten batches of 50,000 requests. Anthropic needs five batches of 100,000. A Start-tier account there holds only 200,000 queued requests, so the job goes in three waves; Scale tier takes it in one. Gemini's Tier 2 enqueued cap for 3.8 Flash is 400M, below the job's 600M input tokens. The job then needs either two waves or Tier 3.

For the same comparison across more models, see the October 2026 pricing snapshot.

What to do next

• List your pipelines and mark the ones where nobody reads the output for at least a few hours. Those are batch candidates.
• Pick one of them, move it to a batch endpoint, and compare the invoice line for a week against the previous week.
• Check your queue limit before the first large submission; split into waves rather than hitting the cap mid-job.
• Put a shared prefix at the front of every request and use the 1-hour cache at Anthropic or cached_content at Gemini, then measure the hit rate.
• We can run this audit on a live pipeline; see the LLM cost audit checklist for what it covers.

Sources

• OpenAI Batch API guide
• OpenAI models
• Anthropic Message Batches
• Anthropic pricing
• Anthropic rate limits
• Gemini Batch API
• Gemini API pricing
• Gemini API rate limits
• xAI pricing
• Mistral pricing