LLM Cost Tracking Across OpenAI, Anthropic, and Vertex

The pattern is recognizable in every team that integrates LLMs into a product: the bill arrives, the bill is unexpected, the post-mortem reveals that nobody had visibility into per-request cost in real time. By the time you have a problem, you've already had a problem.

This page is the stack we'd ship for unified LLM cost visibility across OpenAI, Anthropic, and Google (Vertex / Gemini API), plus the alert and budget patterns that catch problems before the invoice does. The recommendations draw from Helicone's free tier, LiteLLM's gateway model, OpenLLMetry's OTEL approach, and the post-mortems we've read from teams that learned this the expensive way.

The four data points you need

For every LLM request in production, you need:

1. Provider (anthropic, openai, vertex, etc.)
2. Model (claude-opus-4-7, gpt-5.4, gemini-3.1-pro, ...)
3. Token counts (input + output, plus cache hit/miss for Anthropic and OpenAI)
4. Cost in USD (derived from token counts × per-provider rate)

The stack we'd ship

Three options, ordered by setup effort.

Option 1 — Helicone (SaaS, fastest setup)

from openai import OpenAI

client = OpenAI(     api_key=os.environ["OPENAI_API_KEY"],     base_url="https://oai.helicone.ai/v1",  # was https://api.openai.com/v1     default_headers={         "Helicone-Auth": f"Bearer {os.environ['HELICONE_KEY']}",     }, )

Same pattern for Anthropic (anthropic.helicone.ai), Vertex (gemini.helicone.ai). Helicone's dashboard shows requests, costs, per-model breakdowns, and supports custom properties for per-tenant or per-feature attribution.

What you get: 5 minutes of setup, real-time cost dashboards, alert rules, cache-hit visibility.

What you pay: $0 free tier, ~$50/mo for 100k requests, $250/mo for 1M requests. Your data goes through Helicone's proxy, which is a trust consideration for some teams.

Option 2 — LiteLLM self-hosted (gateway + tracking)

Deploy via Docker:

docker run -p 4000:4000 \
  -e LITELLM_MASTER_KEY=sk-... \
  -e DATABASE_URL=postgresql://... \
  ghcr.io/berriai/litellm:main-stable

Your app calls LiteLLM as if it were OpenAI:

from openai import OpenAI

client = OpenAI(     api_key="sk-your-litellm-key",     base_url="http://litellm-proxy:4000/v1", ) # Now call any model: model="claude-opus-4-7" or "gpt-5.4" or "gemini-3.1-pro"

What you get: full self-hosted observability, unified routing across providers, budget controls per virtual API key, no data leaves your network.

What you pay: software is free. You pay for Postgres + container hosting. Realistic ops cost: $50-200/month for a small team's traffic.

Budget alerts:

# Set a per-key daily budget
litellm-cli budget set --api-key sk-xxx --daily-usd 50

# When exceeded, LiteLLM rejects new requests and webhooks to your alert channel

Option 3 — OpenLLMetry (OTEL-native, plug into existing stack)

Setup:

from openllmetry import OpenLLMTracker

tracker = OpenLLMTracker() tracker.init()  # auto-instruments OpenAI, Anthropic, Google SDKs

What you get: native OTEL traces with gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.response.model, and computed cost attributes. Plugs into your existing dashboards.

What you pay: software is free. You pay for the OTEL backend you export to. Best when your team already runs Datadog / Grafana / Honeycomb — you get LLM observability without a separate tool.

The pricing tables (May 2026)

Per-million-token rates. Subject to vendor changes; always verify against the pricing page before alerting.

Per-feature attribution

The single highest-leverage practice for production cost tracking: tag every LLM request with the feature it serves. Then the dashboard can answer "which feature is responsible for the spike?"

Pattern in Helicone:

client.chat.completions.create(
    model="claude-opus-4-7",
    messages=[...],
    extra_headers={
        "Helicone-Property-Feature": "rag-query",
        "Helicone-Property-Tenant": tenant_id,
        "Helicone-Property-UserId": user_id,
    },
)

In LiteLLM, the same idea via metadata:

client.chat.completions.create(
    model="claude-opus-4-7",
    messages=[...],
    metadata={
        "feature": "rag-query",
        "tenant_id": tenant_id,
        "user_id": user_id,
    },
)

Now your dashboard can group by feature. When the bill jumps, you know where to look.

Budget alerts

Three tiers we set on every project:

LiteLLM has all three built in. Helicone has the first two. OpenLLMetry needs you to wire alerts in your OTEL backend (which most teams already have for non-LLM workloads).

The cross-provider comparison problem

A common request: "show me cost-per-task across providers so I can pick the cheapest." Three things make this harder than it sounds:

The comparison that actually works: cost-per-successful-task in production, measured for a week, per feature. Stop trying to do theoretical apples-to-apples; measure what your real workload costs on each provider, and switch based on data.

The cache discipline

The most actionable optimization in this space is improving prompt-cache hit rate. Specific to providers:

Your dashboard should show cache hit rate per model, per feature. If hit rate is below 60% on a request pattern that should be cacheable, that's where the next dollar is.

Where this fails

What to read next

• /topic/anthropic-prompt-caching — the caching playbook
• /topic/llm-gateway-decision — Vercel AI Gateway vs OpenRouter vs Portkey vs LiteLLM
• /topic/llm-evals — cost-vs-quality evaluation
• /topic/llm-as-judge — judge calls are themselves a cost line
• /blog/opus-4-7-tokenizer-tax — why per-token rates aren't the full story
• /blog/batch-plus-cache-95-percent-off — the Anthropic-specific discount stack
• /for/finops-ai-cost-tracker — the team that owns this

Sources

• Helicone. Pricing and free tier. YC W23.
• BerriAI. LiteLLM repository. 100+ model support, gateway + cost tracking.
• Traceloop. OpenLLMetry repository. OTEL-native LLM instrumentation.
• Anthropic. Pricing page. Per-model rates.
• Anthropic. Prompt caching docs. 10% cache-read rate.
• OpenAI. Pricing page.
• Google. Gemini pricing.
• HN 47301395. "Ask HN: How are you monitoring AI agents in production?". Tools cited: Helicone, LangSmith, Langfuse, Lava.
• Galileo. "State-of-AI-Evaluation 2026 report". 84.9% of teams hit a production incident within 6 months.