The Hidden Token Tax: Why AI Agents Cost More Than Expected

An agent demo shows one request and one response. Production logs show every call the agent made to get there. Each call re-sends the system prompt, the tool definitions and everything gathered so far. The worked example below puts one support ticket at about 6.5 times the single-call estimate.

Prices and model names in this article were updated on 31 October 2026.

What one "request" looks like

Take a customer support agent that can look up an account, check order history and search a knowledge base. Then it drafts a reply. The user sends one message and gets one answer, but the token counter sees five calls. Tool results stay in the conversation, so each call carries every result before it (account 380 tokens, orders 620, knowledge base 1,200, refund policy 600):

Call 1:  system_prompt (2,100t) + user_message (45t) + tool_defs (850t)                    = 2,995 input →  120 output (tool call: account lookup)
Call 2:  system_prompt (2,100t) + history (165t) + tool_defs (850t) + results (380t)        = 3,495 input →   85 output (tool call: order history)
Call 3:  system_prompt (2,100t) + history (250t) + tool_defs (850t) + results (1,000t)      = 4,200 input →   90 output (tool call: knowledge base)
Call 4:  system_prompt (2,100t) + history (340t) + tool_defs (850t) + results (2,200t)      = 5,490 input →   95 output (tool call: refund policy)
Call 5:  system_prompt (2,100t) + history (435t) + tool_defs (850t) + results (2,800t)      = 6,185 input →  380 output (final reply)

One ticket uses 22,365 input tokens and 770 output tokens in total. The token counts are illustrative; they were not measured.

At Claude Sonnet 5.5 list prices ($2 per million input tokens, $10 per million output tokens):

The real ticket costs about 6.5 times the demo estimate ($0.0524 / $0.0081). Nothing in the example is unusual. Five calls and four tools make a small agent.

Anthropic's pricing page points to one more effect under the token counts. Claude 4.7 and later models use a tokenizer that produces "approximately 30% more tokens for the same text." Token counts measured on an older model will understate the same prompts on a current one.

The four multipliers

1. System prompt repetition

Every call in the agent loop carries the full system prompt and tool definitions. The API is stateless, so the model remembers nothing between requests. The example's fixed prefix of 2,950 tokens is 66% of all input.

The fix is prompt caching. Anthropic's automatic caching is one extra field on the request, cache_control={"type": "ephemeral"}. The cache point then "moves forward automatically as conversations grow." On Claude Sonnet 5.5, a cache read costs $0.10 per million tokens, 5% of the normal $2 input price. A cache write costs $2.50 (five-minute lifetime). Prompts under 512 tokens are not cached on the 5.5 models.

A 25-call loop with a 2,000-token system prompt, on Sonnet 5.5:

The other two large vendors price cache reads in the same range. GPT-6 Astra bills $1 for cached input against $10 uncached. Gemini 4 Argon charges $0.10 against $2 until 30 November 2026; Google lists a $4 input price from 1 December 2026. The prompt caching guide covers the rules per vendor.

2. Tool-call response verbosity

Tool responses go back into context in full and stay there for every later call. The example's order history is 620 tokens, and the knowledge-base result is 1,200. Here is a customer record serialised as verbose JSON:

{
  "customer": {
    "id": "cust_8472947",
    "created_at": "2024-03-15T14:22:31.000Z",
    "updated_at": "2026-01-08T09:11:04.000Z",
    "email": "customer@example.com",
    "first_name": "Sarah",
    "last_name": "Chen",
    "subscription": {
      "plan": "pro",
      "status": "active",
      "billing_cycle": "monthly",
      "next_billing_date": "2026-08-01T00:00:00.000Z",
      ...
    }
  }
}

The agent needs only a few of these fields, so strip tool responses down to them. {"name": "Sarah Chen", "plan": "pro", "status": "active"} is a few dozen tokens, and every later call re-sends it at that size.

3. History growth

Each call carries all previous turns. History in the example grows from 45 tokens to 435, and tool results from 0 to 2,800. A longer task keeps adding to both.

There are three ways to limit the growth. Summarise history every N calls. Use a sliding window that drops old tool results once the agent has acted on them. Or store intermediate state outside the conversation and pass back only a short summary.

4. Loop overhead without termination logic

Agents without clear stop conditions run extra calls. An agent that checks "is this task complete?" with a separate LLM call pays for one more full-price request, with the whole context, to say "done". An explicit completion tool in the schema, or structured output with a finished: true flag, removes these dead-end calls. The budget guardrails article has code for this.

The real cost of a support operation

Suppose the support tool handles 50,000 tickets a month, at Sonnet 5.5 prices:

The cached row assumes a busy queue, so all five calls read the 2,950-token prefix from cache. The arithmetic is 14,750 × $0.10 / 1M + 7,615 × $2 / 1M + 770 × $10 / 1M = $0.0015 + $0.0152 + $0.0077. The uncached agent costs $2,217 a month more than the demo estimate, about $26,600 a year, for one feature.

Chatbot and agents side by side

Illustrative token counts, priced at October 2026 list prices with no caching:

Haiku 5.5 prices apply to prompts up to 100,000 tokens; longer prompts move to $0.50 / $2.50. Every call in these examples stays under that line. For other vendors' prices, see the October 2026 pricing snapshot.

Measuring cost per task

Count tokens on every call before changing anything. Anthropic's usage object reports uncached input, cache writes and cache reads in separate fields. Anthropic's caching docs say input_tokens "represents only the tokens that come after the last cache breakpoint". A cost tracker that reads only input_tokens therefore undercounts a cached agent.

from anthropic import Anthropic

client = Anthropic()

# Claude Sonnet 5.5 list prices, USD per 1M tokens, October 2026
PRICE = {"input": 2.00, "cache_write": 2.50, "cache_read": 0.10, "output": 10.00}
totals = {"input": 0, "cache_write": 0, "cache_read": 0, "output": 0}

def agent_step(system, messages, tools):
    response = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=1024,
        cache_control={"type": "ephemeral"},  # automatic prompt caching
        system=system,
        messages=messages,
        tools=tools,
    )
    u = response.usage
    totals["input"] += u.input_tokens
    totals["cache_write"] += u.cache_creation_input_tokens or 0
    totals["cache_read"] += u.cache_read_input_tokens or 0
    totals["output"] += u.output_tokens
    return response

# After the agent finishes:
cost = sum(totals[k] * PRICE[k] / 1_000_000 for k in PRICE)
print(f"Task cost: ${cost:.4f}")
print(f"Tokens: {totals}")

Run this on 100 representative tasks. Use the result to set a budget per task. Then make the agent loop stop when a task passes twice that figure. The LLM cost audit describes how to split the same numbers by feature and by token type.

What to fix first

1. Cache the system prompt and tool definitions. On Sonnet 5.5 and Opus 5.5, cache reads cost 5% of the input price.
2. Trim tool responses to the fields the agent uses.
3. Give the agent an explicit way to finish, so it does not spend a call deciding whether it is done.
4. Summarise or window long histories.
5. Send simple questions, such as opening hours, to a lookup or a small model instead of the agent.

Sources

• Claude pricing (platform.claude.com)
• Prompt caching (Claude Platform Docs)
• Claude Sonnet 5.5 model page
• OpenAI GPT-6 Astra model page
• Gemini API pricing
• Anthropic on X: prompt caching launch, 14 August 2024