Reasoning Tokens Are the Invisible Line Item Blowing Up Your Bill

You gave a reasoning model a task. It returned a short answer. Behind that answer, the model may have spent thousands of tokens planning an approach, comparing alternatives, working through dead ends, and checking the result. You never see that internal work, but it can still show up on the bill.

Those hidden units of work are reasoning tokens. A reasoning model uses them before it produces the answer shown to the user. They can improve results on difficult tasks. They can also turn a short response into an unexpectedly expensive request.

That gap between the visible answer and the billed work is the thinking tax. It is not automatically waste. The tax may be worth paying for a hard coding problem or a careful plan. The problem is paying it without knowing when, where, or why.

What follows is a blueprint, not a description of a mature tool category. Most teams cannot manage reasoning cost this way today without combining provider usage data, application instrumentation, and their own evaluations. AI cost products, including ours, still have work to do before this becomes a standard workflow.

The response is not the receipt

Most teams still estimate AI cost with a simple mental model: input tokens plus the output they can see, multiplied by the published rates. Reasoning breaks that model.

Providers commonly count hidden reasoning as output usage. The user might receive 300 words while the model consumes thousands of internal tokens. The application looks efficient in a transcript and expensive on the invoice.

The amount is not fixed either. It can change with the prompt, task difficulty, model, reasoning setting, and provider behavior. That makes an average taken from a handful of calls a weak basis for a budget.

OpenAI and Anthropic can include a reasoning-token breakdown in the usage data returned with a response. If an application keeps only the answer text, that detail is lost. Preserving the reported usage makes it possible to tell later whether a cost increase came from the prompt, the visible answer, or the model's hidden reasoning.

A cheap rate can produce an expensive answer

Model comparisons often begin and end with the rate card. Model A charges less per million output tokens than Model B, so Model A must be cheaper. That conclusion only holds if both models consume roughly the same number of tokens for the work.

The Price Reversal paper tested that assumption across eight reasoning models and 12 tasks. Of its 336 model-pair-by-task comparisons, 106 (32 percent) showed a price reversal: the model with the lower listed price incurred the higher total cost. The reported reversal magnitude reached up to 28 times. For single-turn tasks, the authors attributed more than 95 percent of the cost difference among reversed model pairs to thinking tokens. In multi-turn agent tasks, interaction count and cached input also contributed substantially.

This is why a price table cannot choose a model for you. Rate is one part of the equation. The other part is volume, including work the user never sees.

A useful evaluation table looks like this:

Run the table on your own workload. A public benchmark cannot tell you how your prompts, tools, and success criteria behave.

Give reasoning a job description

Many applications turn on a reasoning model everywhere because it sounds safer. That is like assigning a senior investigator to alphabetize a spreadsheet.

Split work by difficulty. Classification, extraction, formatting, and simple rewriting often do not need deep reasoning. Multi-step analysis, difficult code changes, and ambiguous planning may benefit from it. The exact boundary depends on your quality target, but there should be a boundary.

A basic router can start with known workflow types rather than another model. Send invoice field extraction down the fast path. Send an unfamiliar debugging task down the reasoning path. Add a fallback only when the first attempt fails a clear check.

That last part matters. A router without an evaluation is just a guess that creates more calls.

Set a reasoning budget

Some APIs offer reasoning-effort settings such as low, medium, and high. Others let developers set a direct token budget. Either way, the setting should reflect the task instead of sending every request through the deepest reasoning available.

That does not require a new evaluation program for every workflow. If a recurring task becomes expensive, compare a lower reasoning setting on a few real examples. Keep it only if the results still meet the needs of that task.

Watch retries and agents

Reasoning cost compounds inside agent loops. One user action may ask a model to plan, call tools, inspect the result, correct an error, and try again. Each turn can carry the conversation history and create a fresh block of hidden thinking.

Track cost at both levels: per model call and per completed run. The call view helps engineers find a costly prompt. The run view tells the business what the task actually cost.

Unattended agent loops may need a cap so an error cannot trigger calls indefinitely, but a hard step limit is not a universal answer. Start by looking for repeated calls. If the same tool error triggers three expensive reasoning attempts, the tool interaction is probably the problem.

Also watch long context. Paying for a large prompt and a large reasoning trace at the same time can make a single workflow expensive from both ends.

Optimize for completed work

Reasoning tokens matter because they break the assumption that a short answer is a cheap answer. But minimizing reasoning is not the goal. A model that thinks longer and succeeds once may cost less than a cheaper model that fails and retries.

Evaluate reasoning models on your actual workload. Track total cost, success rate, latency, and retries for each completed task. If your provider exposes a reasoning-token breakdown, use it. If it does not, those broader signals can still reveal whether deeper reasoning is earning its cost.

The question is not “How do we eliminate the thinking tax?” It is “Where is the thinking tax worth paying?” Answer that per workflow, and reasoning becomes an intentional tradeoff instead of an invisible expense.

Sources

1. The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More, arXiv.
2. Reasoning models, OpenAI API documentation.
3. Extended thinking, Claude Platform documentation.