AI Agent Token Costs: Why Agents 10x Your Bill

The dominant narrative in 2025 and 2026 is that AI agents replace expensive human labor with cheap compute. A coding agent that costs a few dollars per task looks like a bargain next to a $150/hour engineer. A support agent that resolves tickets for pennies looks like a replacement for a $25/hour agent. The framing is so pervasive that it has become the default assumption in boardrooms and product reviews.

The logic is not wrong at the surface. Token prices have fallen dramatically. OpenAI's GPT-5.6 Luna lists output at $1.20 per million tokens, and Anthropic's Haiku 4.5 outputs at $5.00 per million tokens (OpenAI API pricing; Anthropic pricing). At those prices, even a verbose response feels free. The mistake is assuming that an agentic task produces one response.

Anthropic's own research distinguishes workflows from agents: workflows orchestrate LLMs through predefined code paths, while agents dynamically direct their own processes and tool usage. The key word is dynamically. An agent does not know in advance how many steps it will take. It plans, acts, observes, and replans. That loop is where the token bill compounds (Anthropic, “Building Effective Agents,” Dec 2024).

A single LLM call sends a prompt and receives a response. An agentic workflow sends the prompt, receives a plan, emits tool calls, reads tool results, possibly retries, and then synthesizes a final answer. Each of those phases is billable. Here are the four multipliers.

1. Tool calls re-inject context

When an agent calls a tool, the tool output is appended to the conversation and sent back to the model. If the agent calls a code interpreter, a search API, or a database, the result is often hundreds or thousands of tokens. Those tokens are billed as input on the next turn. A five-step agent with 500-token tool outputs adds 2,500 input tokens to the conversation before the final answer is even generated.

OpenAI charges for built-in tools separately: web search costs $10 per 1,000 calls, and the search content tokens are billed at model rates on top (OpenAI API pricing). The tool fee is small; the token re-injection is not.

2. Retries and self-correction burn output

Agents are designed to recover from errors. A coding agent that fails a test will re-run the loop. A research agent that finds a dead link will search again. Each retry generates a new plan and new output tokens. If your agent averages two attempts per task, you have already doubled the output cost before accounting for anything else.

This is not a bug; it is the feature. The value of an agent is persistence. But persistence is paid for in tokens. Anthropic notes that agents are ideal for open-ended problems where the number of steps is unpredictable, and that their autonomy means higher costs and the potential for compounding errors (Anthropic, Dec 2024).

3. Context windows fill with history

Most agent frameworks keep the full conversation history in the prompt. Every prior turn, every tool result, and every system reminder stays in context. As the conversation grows, so does the input token count on every subsequent call. A 128K context window sounds generous until you realize you are paying for the whole window on every turn.

The result is a subtle but relentless creep. An agent that starts with a 2,000-token prompt can easily carry 20,000 tokens of history by the end of a complex task. You are no longer paying for the question; you are paying for the entire conversation, repeatedly.

4. Reasoning models charge a premium

Reasoning models think before they answer, and that thinking is billed. OpenAI's o3 outputs at $8.00 per million tokens; o3-pro outputs at $80.00 per million tokens. Anthropic Claude Opus 5 outputs at $25.00 per million tokens (OpenAI; Anthropic). When an agent uses a reasoning model for planning and a smaller model for execution, the planning step alone can dominate the bill.

OpenAI explicitly describes o3 and o4-mini as models trained to reason about when and how to use tools, chaining multiple tool calls in a single task. That capability is powerful, but it is also expensive by design (OpenAI, “Introducing o3 and o4-mini,” April 2025).

The gap between a budget model and a reasoning/flagship model is 10–60x on output price. That gap matters because agents often use the expensive model for planning and the cheap model only for high-volume execution. The chart below shows output prices per million tokens for representative 2026 models.

The real cost blowup comes from combining these prices with the loop. Imagine a coding agent that uses o3-pro for planning ($80/MTok output) and generates 500 tokens of plan per turn. If the task takes five turns with two retries, the planning output alone is 3,500 tokens. At $80/MTok, that is $0.28 just for the planner's output. Add tool-result re-injection, final synthesis, and the execution model, and a single task can easily approach $0.50–$1.00. Scale that to 10,000 tasks per month and you are looking at $5,000–$10,000 for one feature.

For a deeper look at per-token pricing before agents enter the picture, see our LLM inference cost guide. For the broader AI infrastructure stack, including GPU waste, see State of AI Infrastructure Costs 2026.

The conventional cost-optimization advice is to use a cheaper model. That helps at the margin, but it misses the point. The dominant cost driver in agentic workflows is the number of turns, not the price per token. A workflow that takes ten turns on a cheap model can still outspend a workflow that takes one turn on an expensive model.

The better approach is to minimize autonomy. Use workflows with predefined steps wherever the task is well understood. Reserve true agents for open-ended problems where the path cannot be hardcoded. Anthropic's own guidance is to start with simple prompts and add multi-step agentic systems only when simpler solutions fall short (Anthropic, Dec 2024).

Route aggressively. Classify incoming requests and send simple ones to Haiku or GPT-5.6 Luna, keep Sonnet or GPT-5.6 Sol for medium complexity, and reserve Opus or o3-pro for the small fraction of tasks that actually need frontier reasoning. This is the same principle as using spot instances for fault-tolerant batch work and reserved instances for steady-state workloads: match the resource to the job.

Cap context growth explicitly. Set a maximum conversation length, summarize old turns, and evict tool outputs that are no longer needed. Every token you remove from the context window is a token you do not pay for on the next turn. This is the single most underappreciated cost lever in agent design.

If you are already running agentic features, here is a practical checklist to bring costs under control without killing the product.

1. Instrument token usage per turn. Most teams only track total API spend. Break it down by input tokens, output tokens, tool-call tokens, and retry count. You cannot optimize what you cannot see.
2. Set a per-task token budget. Hard-cap the number of turns and the context length. When the agent hits the cap, escalate to a human instead of letting it loop forever.
3. Cache stable context. If your system prompt and tool descriptions are static, use prompt caching. Anthropic charges roughly 10x less for cached reads than for fresh input; OpenAI offers similar discounts (Anthropic pricing).
4. Route by complexity. Build a lightweight classifier that sends at least 80% of requests to a small model. Only the exceptions should reach the frontier model.
5. Charge back agent costs to features. Make the product team see the cost of each agentic feature. When a feature costs $0.50 per task, the business model becomes a engineering decision.

For startups still deciding between foundation-model APIs and self-hosted GPUs, the break-even math is different but no less brutal. See self-hosting Llama vs. the OpenAI API for a worked example.

Not every agentic workflow is a cost disaster. A well-designed support agent that routes 90% of tickets to a small model and escalates the rest can be dramatically cheaper than a human team. The problem is assuming that autonomy is free.

There are also tasks where the agentic premium is worth it. Complex coding tasks, multi-step research, and open-ended analysis can justify the token spend because the alternative is senior engineer time. The mistake is applying agentic patterns to tasks that could be solved with a single LLM call or a deterministic workflow.

Finally, model prices change. The numbers in this post are list prices from early 2026. The ratios matter more than the absolute dollars. Reasoning models will likely remain premium, and agents will likely remain loop-heavy. Design for that reality.

Why do AI agents use more tokens than chatbots?

A chatbot usually answers in one turn. An agent plans, calls tools, reads results, retries, and synthesizes. Each phase sends tokens to the model and receives tokens back. A task that takes one turn for a chatbot can take five to ten turns for an agent.

How much more expensive is a reasoning model?

Reasoning models charge premium output rates. OpenAI's o3-pro outputs at $80 per million tokens, while budget models like GPT-5.6 Luna output at $1.20 per million tokens. That is roughly a 65x spread at list prices, before accounting for the longer thinking chains agents generate.

Is the answer just to use a cheaper model?

No. Cheaper models help, but the dominant cost driver is the number of turns. A ten-turn workflow on a cheap model can cost more than a one-turn workflow on an expensive model. The best optimization is to reduce autonomy for tasks that do not need it.

What is the fastest way to cut agent costs?

Cap context length and turn count. Summarize or drop old conversation history, cache static prompts, and route simple requests to small models. These three changes typically cut agent token spend by 40–60% without reducing capability on the tasks that matter.

AI agents are not a cheaper version of labor. They are a more flexible, more expensive version of automation. The cost is hidden in the loop: tool calls, retries, growing context, and reasoning premiums. A team that treats an agent like a single LLM call will be shocked by the invoice.

The teams that get this right will design for turns, not tokens. They will use workflows where the path is known, agents only where the path is unknown, and frontier models only for the exceptions. They will cap context, cache prompts, and route by complexity. The prize is not lower token prices; it is fewer tokens spent on the wrong work.