The Real Cost of LLM Inference in 2026, Line by Line

Prices and model names in this article were updated on 31 October 2026.

"Per million tokens" is the unit every vendor uses, and it hides what a request costs. This article converts list prices into cost per request and per month for three common workloads. All prices are as of October 2026.

A million tokens is not a fixed amount of text. Anthropic states that Claude 4.7 and later models use a tokenizer that produces about 30% more tokens for the same text. Count tokens with each vendor's own tokenizer before you compare. If an average request is 2,000 tokens, one million tokens buys 500 requests.

The October 2026 price table

List prices per 1M tokens, standard tier, no caching, no batch discount:

Notes on the table. DeepSeek's off-peak rate is half the peak rate; peak hours are 01:00–04:00 and 06:00–10:00 UTC on weekdays. Mistral's plan page quotes Mistral Large at $0.50 / $1.50, while its docs price table shows $0.70 / $2.10; the table uses the docs figure. Groq lists no public price for Llama models ("Contact Sales"), so the Llama row uses OpenRouter's listing.

Cached input is much cheaper than fresh input. GPT-6 Astra charges $1 for cached input against $10 uncached. A cache read on Claude Opus 5.5 costs $0.20 and on Sonnet 5.5 $0.10, which is 0.05x the input price. Gemini 3.8 Flash charges $0.075 for cached input plus an hourly storage fee. Batch APIs from OpenAI, Anthropic and Google take 50% off both input and output. For the full grid with cached and batch columns, see the LLM API pricing snapshot for October 2026.

Why output tokens cost 3–8x more

Input tokens are processed in parallel during the prefill phase. The model reads the whole prompt in one pass and builds a KV cache of attention states. That work spreads well across GPU cores.

Output tokens are generated one at a time. Each new token needs a forward pass that depends on the token before it. This decode phase is limited by memory bandwidth, and it does not parallelise the way prefill does. The price gap follows the hardware cost.

The table above shows output at 5x input on every OpenAI and Anthropic model, and on Gemini 4 Argon and 3.8 Flash. The range across all rows runs from 3x (DeepSeek V4 Pro, Mistral Large 4) to 8.3x (Gemini 3.5 Flash-Lite).

Long answers are therefore the expensive part of a request. On Claude Sonnet 5.5, a 2,000-token answer costs $0.02 in output, and a 100-token answer costs $0.001. Add a 1,000-token prompt ($0.002) and the totals are $0.022 and $0.003, a 7x difference. If bullet points would do, asking for detailed prose costs real money.

What these numbers look like in practice

Customer support chatbot

Average request: 800 input tokens (system prompt, user message, history) and 200 output tokens.

How a row is computed, for GPT-6 Astra: 800 × $10 / 1M = $0.008 input, plus 200 × $50 / 1M = $0.010 output, so $0.018 per request.

The gap between GPT-6 Astra and Claude Haiku 5.5 is $1,782 a month at this volume. If a small model passes your quality bar for support replies, the flagship is hard to justify. Only your own evaluation set can tell you if it does. The small language models guide covers where small models hold up and where they fail.

Code review tool

Average request: 4,000 input tokens (code, context, instructions) and 1,500 output tokens.

Here the quality gap matters more. gpt-oss-120B on Groq runs 10K reviews for $15 a month, but it may miss issues a larger model would catch. Run an evaluation on your own diffs before you decide. For Gemini 4 Argon the date also matters. Google's introductory price ends on 30 November 2026, and the same workload doubles to $460 from 1 December.

Context window costs

Long context multiplies every other cost. Here is the input cost of putting a 50,000-token document into every request, before any output:

Very long prompts can move a request onto a different price row:

• OpenAI GPT-6 Astra: a request with more than 272K input tokens pays 2x input and 1.5x output for the whole request.
• Anthropic: Claude 4.6 and later models bill the full 1M context at standard rates. A 900K-token request pays the same per-token rate as a 9K one. Haiku 5.5 is the exception: prompts over 100K tokens pay $0.50 / $2.50.
• Google Gemini 3.1 Pro Preview: prompts over 200K tokens pay $4 / $18 instead of $2 / $12.

A 300K-token prompt therefore costs $6.00 in input on GPT-6 Astra (300,000 × $20 / 1M), $1.20 on Claude Opus 5.5 and $0.15 on Claude Haiku 5.5 at its long-prompt rate. If the same document goes into many requests, cache it. The cached rates above cut the repeat cost by 90–95%.

Where provider pricing diverges most

Reasoning tokens

Models that think before answering bill those thinking tokens as output. Anthropic's Claude Code docs say the default thinking budget "can be tens of thousands of tokens per request depending on the model". The same docs say thinking cannot be turned off on Claude Opus 5.5, Sonnet 5.5, Haiku 5.5 or the Fable models. Take an illustrative request on Opus 5.5 with 500 visible output tokens and 3,000 thinking tokens. Output is billed on 3,500 tokens, $0.07, against $0.01 for the visible part alone. DeepSeek bills thinking mode at the same per-token price, on total tokens.

Top-tier output

GPT-6 Astra and Claude Fable 5.1 both charge $50 per 1M output tokens. That is 2.5x Claude Opus 5.5 ($20) and 5x GPT-6.1 Sol or Claude Sonnet 5.5 ($10). Apps that produce long answers feel this most.

Rate limits on cheap hosts

Hosts of open models publish rate limits per model and per plan. Read them before you route bulk work to the cheapest row. A low price does not help if the host caps your tokens per minute.

Discounts for waiting

DeepSeek's off-peak hours halve the price. OpenAI, Anthropic and Google batch APIs take 50% off for jobs that can wait up to 24 hours. The batch API comparison covers limits and turnaround.

From tokens to a monthly budget

A team expects to process 1 billion tokens a month. If 60% is input and 40% is output, the monthly bill at list price is:

GPT-6 Astra costs 100x Claude Haiku 5.5 for the same token volume. Caching and batch discounts lower every row, but they do not close a gap of that size.

What to do next

• Measure your real input and output tokens per request for one week, split by feature.
• Price that mix against three or four rows of the table above, including one small model.
• Run your own evaluation set on the cheapest model that might pass, before you commit to a flagship.
• Check scheduled price changes (Gemini 4 Argon on 1 December 2026, Gemini 3.8 Flash on 1 January 2027) before you sign a budget. The price history shows how often these move.

Sources

• Claude pricing (platform.claude.com)
• OpenAI GPT-6 Astra model page
• OpenAI models overview
• Gemini API pricing
• DeepSeek API pricing
• Mistral pricing (docs)
• OpenRouter: Llama 3.3 70B Instruct
• Groq: gpt-oss-120b model page
• Gemini API models
• Claude Code: manage costs effectively