Prompt caching is one of the highest-leverage cost optimizations available. It requires no model changes, no infrastructure, and often no prompt rewriting — just restructuring the order of content you're already sending.

The idea: when you send a long prompt, the provider caches the computed KV tensor states for the static prefix. On the next request with the same prefix, they skip the expensive prefill computation and load from cache. You pay a fraction of the normal input cost for the cached portion.

How Each Provider Implements It

Anthropic: Explicit Cache Breakpoints

Anthropic requires you to explicitly mark cache points using cache_control blocks. This gives you precise control but requires code changes.

from anthropic import Anthropic

client = Anthropic()

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": "You are an expert legal document analyst. Your job is to review contracts and identify key terms, obligations, and risk factors.",
            "cache_control": {"type": "ephemeral"}
        },
        {
            "type": "text",
            "text": f"<contract>\n{large_contract_text}\n</contract>",  # ~20,000 tokens
            "cache_control": {"type": "ephemeral"}
        }
    ],
    messages=[
        {"role": "user", "content": "What are the termination clauses in this contract?"}
    ]
)

# Check cache status in response
print(response.usage.cache_creation_input_tokens)  # tokens written to cache (first call)
print(response.usage.cache_read_input_tokens)      # tokens read from cache (subsequent calls)

Cache pricing on Claude: - Cache creation: 1.25x normal input cost (you pay a small premium to write the cache) - Cache reads: 0.10x normal input cost (90% discount on cached tokens) - Cache TTL: 5 minutes (ephemeral), extended on re-use

On a 20,000-token contract that gets queried 20 times: - Without caching: 20,000 × 20 = 400,000 tokens × $3.00/M = $1.20 - With caching: 20,000 (creation) × $3.75/M = $0.075, then 20,000 × 19 × $0.30/M = $0.114 - Total with caching: $0.189 vs $1.20 — 84% savings

OpenAI: Automatic Prefix Caching

OpenAI caches automatically for prompts over 1,024 tokens. No code changes needed. When the same prompt prefix is repeated, you get a 50% discount on those cached tokens automatically.

from openai import OpenAI

client = OpenAI()

# First call: full price, prefix cached automatically
# Subsequent calls with same prefix: 50% discount on cached portion
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": large_system_prompt},  # cached after first call
        {"role": "user", "content": specific_question}       # always fresh
    ]
)

# Check if caching was used
print(response.usage.prompt_tokens_details.cached_tokens)

GPT-4o cache pricing: 50% discount on cached input tokens ($1.25/M instead of $2.50/M).

Gemini: Explicit TTL-Based Caching

Gemini 2.5 Pro/Flash supports explicit context caching via the cacheContent API with a configurable TTL up to 1 hour.

import google.generativeai as genai
from google.generativeai import caching
import datetime

# Create cached content
cache = caching.CachedContent.create(
    model='models/gemini-2.5-flash',
    display_name='product_catalog',
    contents=[large_product_catalog_text],
    ttl=datetime.timedelta(hours=1)
)

# Use cached content in subsequent requests
model = genai.GenerativeModel.from_cached_content(cached_content=cache)
response = model.generate_content("What products are available in the 'enterprise' tier?")

Gemini charges a small storage fee per token per hour for cached content, then a reduced per-token rate for cache hits. At high query volume, this is highly cost-effective.

The Critical Rule: Static Content Must Come First

This is where most teams get caching wrong. KV caches are prefix-based — the provider caches from the beginning of the prompt up to the cache breakpoint. If your static content (system prompt, reference documents, examples) comes after dynamic content (user input, timestamps), the cache is never reused.

Wrong order (cache never hits):

[Dynamic: user name, timestamp]
[Static: large system prompt]
[Dynamic: user message]

Correct order (cache hits on static content):

[Static: system prompt]          ← cache_control breakpoint here
[Static: reference document]     ← cache_control breakpoint here
[Dynamic: conversation history]
[Dynamic: user message]

The ProjectDiscovery case study illustrates this: their initial implementation had a 7% cache hit rate because dynamic metadata was prepended to the system prompt. After restructuring to put all static content first, their cache hit rate jumped to 84% — dropping their monthly Claude bill from $12,400 to $2,100.

Structuring Prompts for Maximum Cache Hits

A well-structured cacheable prompt looks like this:

def build_cached_messages(
    static_instructions: str,
    reference_docs: list[str],
    conversation_history: list[dict],
    user_message: str
) -> list[dict]:

    system = [
        # Layer 1: Core instructions (rarely changes)
        {
            "type": "text",
            "text": static_instructions,
            "cache_control": {"type": "ephemeral"}
        }
    ]

    # Layer 2: Reference documents (changes per session, not per turn)
    for doc in reference_docs:
        system.append({
            "type": "text",
            "text": doc,
            "cache_control": {"type": "ephemeral"}
        })

    # Dynamic content goes in messages, not system
    messages = conversation_history + [
        {"role": "user", "content": user_message}
    ]

    return system, messages

You can have up to 4 cache breakpoints per request with Anthropic. Each adds a small overhead, so don't over-use them — mark the major static sections.

When Caching Pays Off Most

Multi-turn conversations with large system prompts. Every conversation turn is a new API call that re-sends the system prompt. A 3,000-token system prompt on a 15-turn conversation = 45,000 tokens of overhead. With caching, turns 2–15 pay 10% of that.

Document Q&A applications. Each user question re-sends the full document. Cache the document, pay full price once per session, and virtually nothing for subsequent questions.

RAG pipelines with static context. If your retrieval returns the same set of chunks for similar queries (which is common), caching the retrieved context across queries eliminates most of your input token costs.

High-volume classification with shared instructions. If you're classifying 100,000 items with the same instructions and few-shot examples, the instructions are paid for once and cached for the rest.

Monitoring Cache Performance

Track these metrics to validate caching is working:

# After each API call, log cache stats
def log_cache_stats(usage):
    total_input = usage.input_tokens
    cache_read = getattr(usage, 'cache_read_input_tokens', 0)
    cache_created = getattr(usage, 'cache_creation_input_tokens', 0)

    cache_hit_rate = cache_read / total_input if total_input > 0 else 0

    # Effective cost ratio vs no caching
    # cache_created costs 1.25x, cache_read costs 0.10x, fresh costs 1x
    fresh_tokens = total_input - cache_read - cache_created
    effective_cost = (fresh_tokens * 1.0) + (cache_created * 1.25) + (cache_read * 0.10)
    cost_ratio = effective_cost / total_input  # vs. 1.0 if no caching

    print(f"Cache hit rate: {cache_hit_rate:.1%}")
    print(f"Effective cost ratio: {cost_ratio:.2f}x vs no caching")

If your cache hit rate is under 50% on a workload that should benefit from caching, the prompt structure is almost certainly wrong — static content is following dynamic content somewhere in the request.