The Best AI Cost Control Tools in 2026: FinOps for LLMs

Prices and model names in this article were updated on 31 October 2026.

Most teams building with LLMs start with the provider's billing page. It shows total spend, but not which feature, user, prompt or model tier produced it.

LLM observability tools fill that gap. This article covers what each one does, how it connects to your code, and which metrics lead to action. Tool descriptions were checked against each vendor's documentation in October 2026. Plans and prices change often, so check each vendor's pricing page before you choose.

How the tools connect

There are three ways to get cost data out of LLM calls. A proxy sits between your app and the provider and logs every request. An SDK wraps your client and sends traces to a tracing service. OpenTelemetry instrumentation sends spans to whatever backend you already run.

The tools

Helicone

Helicone is an open-source observability platform (Apache 2.0) that can run as a proxy. Requests pass through Helicone, which logs them and forwards them to the provider. Basic setup needs no code changes beyond a base URL and a header.

Helicone's docs now recommend its newer AI Gateway and describe the per-provider proxy below as "maintained but no longer actively developed". The proxy still works with two lines of setup:

from openai import OpenAI

# Replace the base URL and add the Helicone header
client = OpenAI(
    api_key="your-openai-key",
    base_url="https://oai.helicone.ai/v1",
    default_headers={"Helicone-Auth": f"Bearer {helicone_key}"}
)

# Add metadata for cost attribution
response = client.chat.completions.create(
    model="gpt-6.1-sol",
    messages=[...],
    extra_headers={
        "Helicone-User-Id": user_id,
        "Helicone-Property-Feature": "report-generation",
        "Helicone-Property-Team": "analytics"
    }
)

The Helicone-User-Id and Helicone-Property-[Name] headers tag each request with a user and with any property you define, such as feature or team. Spend can then be grouped by them. A Helicone-RateLimit-Policy header sets rate limits, and Helicone-Cache-Enabled turns on response caching.

On the hosted version, all prompts and responses pass through Helicone's servers. Review its data processing terms before you send sensitive data, or self-host it. The open-source licence makes that possible.

LangSmith

LangSmith is LangChain's observability platform. It works through an SDK instead of a proxy. You wrap your client and mark functions with a decorator, and the SDK sends trace data to LangSmith.

# pip install -U langsmith openai
# export LANGSMITH_TRACING=true
# export LANGSMITH_API_KEY=<your-langsmith-api-key>
from openai import OpenAI
from langsmith import traceable
from langsmith.wrappers import wrap_openai

client = wrap_openai(OpenAI())

@traceable
def generate_support_reply(ticket: str) -> str:
    response = client.chat.completions.create(
        model="gpt-6.1-sol",
        messages=[{"role": "user", "content": ticket}]
    )
    return response.choices[0].message.content

LangSmith is strongest for multi-step chains and agents. It shows the full trace of each run, with nested spans for tools and model calls. You can see which step was slow and which was expensive. For a single-call application it is more than you need. On complex agent workflows, its traces show where cost problems sit.

PromptLayer

PromptLayer describes itself as "a platform for prompt engineering" that "tracks your LLM requests". It is simpler than LangSmith and centres on prompts: which version is live, what it costs and how it performs.

from promptlayer import PromptLayer

pl = PromptLayer(api_key="pl_xxxxx")
openai = pl.openai  # wrapped OpenAI client

response = openai.chat.completions.create(
    model="gpt-6.1-terra",
    messages=[...],
    pl_tags=["production", "support-v2"]  # tag for tracking
)

Tags on each request (pl_tags) group spend by prompt version or feature. Use PromptLayer if your main concern is the prompt lifecycle. For agent tracing, compare it with the SDK and OpenTelemetry options in this article before you choose.

Portkey

Portkey combines a gateway with observability; the AI load balancers article covers the routing side. On cost, its docs describe budget limits on providers and integrations. A cost-based limit "will automatically expire the key to prevent further usage and overspending" once a USD amount is reached. A token-based limit caps the number of tokens instead. Limits can reset weekly or monthly, and an email alert can go out when usage reaches a threshold.

Budget limits are not on every plan. Portkey states the feature "is currently available exclusively to Portkey Enterprise customers and select Pro users."

Phoenix by Arize

Phoenix is "Arize's open-source AI observability platform designed for experimentation, evaluation, and troubleshooting." It runs on your own machine or server, so no data has to leave your infrastructure. That matters for sensitive workloads.

pip install arize-phoenix openinference-instrumentation-openai openai

# Start a local Phoenix server
phoenix serve

from phoenix.otel import register

# Instruments the app based on the OpenInference packages installed
tracer_provider = register(project_name="my-llm-app", auto_instrument=True)

# OpenAI calls are now traced

Phoenix gives you traces and evaluations in a UI you host yourself, with no SaaS subscription. The trade-off is that you run and maintain the server. Phoenix centres on tracing and evaluation, so check that its views cover the spend reports you need before you rely on it for cost.

OpenLLMetry

OpenLLMetry, by Traceloop, is "a set of extensions built on top of OpenTelemetry", released under Apache 2.0. Its README lists tested destinations including Datadog, Grafana, Honeycomb, New Relic, Splunk, Sentry and any OpenTelemetry Collector.

# pip install traceloop-sdk
from traceloop.sdk import Traceloop

Traceloop.init()

# LLM calls made after init are traced and exported over OpenTelemetry

If you already run OpenTelemetry, OpenLLMetry is the cleanest integration. LLM traces land in the same Grafana or Datadog dashboards as your other service metrics. That suits engineering teams who want LLM cost next to application performance.

Metrics to track

Most teams install a tool and then look only at total cost. The metrics that lead to action are below. Set alert thresholds from your own first weeks of data.

Cache hit rate deserves its own dashboard. Providers report cached input tokens separately from uncached ones, and cached tokens are billed at a fraction of the input price. A drop in hit rate shows up on the bill.

Cost per successful outcome needs a definition of success for your application. It is also the metric that matters most. Here is a worked example. A change lifts the task success rate from 80% to 88% and raises cost per task from $0.010 to $0.012. Cost per success goes from $0.0125 ($0.010 / 0.80) to $0.0136 ($0.012 / 0.88). Quality went up and each success got 9% more expensive. Paying that is a product decision, and the numbers are now on the table.

Quick-start recommendation by team size

Do not over-instrument early. Pick one tool, get it shipping data, and act on what you learn before you add more.

What to do next

• Tag every LLM call with a feature name before you choose a tool; every tool above can group by it.
• Turn on the spend limits your provider already offers; the LLM cost audit shows how Anthropic's workspace spend limits work.
• If your team uses Claude Code, its built-in /usage command and OpenTelemetry export are covered in the Claude Code cost guide.
• Before traffic grows, read what changes in LLM costs at high volume.

Sources

• Helicone docs: OpenAI proxy integration
• Helicone docs: headers
• Helicone docs: open source
• LangSmith observability quickstart
• PromptLayer on PyPI
• Portkey docs: budget limits
• Arize Phoenix on PyPI
• Phoenix docs: OpenAI tracing
• OpenLLMetry README (Traceloop)
• LiteLLM docs: what is stored in the DB