Reasoning Token

Programs like ChatGPT don’t process text letter by letter, but in small chunks: individual words or parts of words. These chunks are called tokens. A sentence with ten words typically consists of twelve to fifteen tokens. Newer programs of this kind first keep computing for themselves on difficult tasks before they answer. So they write a kind of internal notepad, and the tokens on this notepad are called reasoning tokens. In the end, the user only sees the finished answer, but pays for the notepad too.

Why the invisible notepad costs money

Providers bill the use of their AI services per token. Everything the model reads and everything it writes is charged. Reasoning tokens are part of what’s written, even though no one gets to see them. For a difficult math problem, the notepad can be ten times longer than the answer itself.

That explains a number that often appears in business reports: the operating costs of AI services have jumped sharply with the arrival of reasoning models. A request that used to generate two hundred tokens now easily generates several thousand. For companies building AI into their own software, that’s a real cost factor.

At the same time, this cost buys quality. On logic puzzles, programming tasks, and multi-step calculations, reasoning models are noticeably more reliable. For a simple question about the capital of Peru, on the other hand, the effort is pure waste. That’s why many providers now offer a slider to adjust the intensity of reasoning.

What happens on the internal notepad

The model always generates tokens the same way, namely one after another, each as the most probable continuation of the text so far. With reasoning tokens, only the goal is different. Instead of formulating a solution directly, the model breaks the task down into steps by itself. It tries approaches, discards them, and recalculates.

This ability is trained into the model. During training, the model receives feedback on whether its final results were correct. Reasoning paths that led to correct results are reinforced. This way it learns to take more time on difficult tasks, without anyone manually prescribing the intermediate steps.

A common misconception: that reasoning tokens are a record of what’s really going on inside the model. That’s not true. It’s ordinary text that sounds plausible and is usually helpful, but can also contain nonsense. Some providers therefore only show a shortened summary of the reasoning path instead of the actual tokens.

Where the count shows up in bills and products

You encounter reasoning tokens most directly in providers' billing. Anyone accessing a model through a programming interface sees a separate line there for the reasoning tokens consumed. Price lists, too, now distinguish between models with and without this additional overhead.

In everyday use, you notice them through the waiting time. When an AI “thinks” for a few seconds and displays intermediate steps while doing so, reasoning tokens are being generated right then. With some services, you can choose between a fast and a thorough mode, which controls exactly this consumption.

In news coverage about AI companies, reasoning tokens are also an argument in the debate about data centers. More reasoning per request means more graphics chips needed and more electricity. That’s why a very large share of the industry’s investment planning hinges on this small technical term.