Token consumption

Programs like ChatGPT don’t read text as letters, nor as whole words. They break it down into small chunks called tokens. A token is usually a syllable, a short word, or a word ending. As a rule of thumb: 100 words of English text correspond to roughly 130 to 150 such chunks. Token consumption is the number of these chunks that accumulate during a use. This includes both what you input and what the program answers.

Why providers bill in tokens

For an AI model, a token is the smallest unit of work. Each individual token means a computing step on expensive specialized chips. That’s why providers like OpenAI, Google, or Anthropic don’t bill per question, but per token. Typical prices are a few dollars per million tokens. That sounds tiny, but it adds up quickly.

An example makes it tangible. A company has 50,000 customer inquiries answered automatically every day. Each inquiry, including question and answer, consumes around 2,000 tokens. That’s 100 million tokens per day, so a noticeable cost block over a month. Whoever halves consumption halves the bill.

That’s why token consumption has also become a stock market metric. Google states in quarterly reports how many trillion tokens its systems process monthly. Rising numbers show demand, but also drive up spending on data centers. Analysts read from this whether AI businesses are becoming profitable.

What drives consumption up

The obvious factor is text length. Anyone attaching a 40-page PDF easily sends along 20,000 tokens. The less obvious factor is a chat’s memory. With every new message, the entire previous conversation history is sent along again, because the model doesn’t remember anything on its own. So a long conversation gets more expensive with each round than the previous one.

Then there’s the nature of the answer. Newer models with so-called reasoning generate a long internal chain of thought before the actual answer. This intermediate text is invisible to you, but is fully billed. A three-sentence answer can thus have cost thousands of tokens.

A common misconception: tokens are the same as words. For English text that’s roughly true, but not for German. Long compound words like “Krankenversicherungsbeitrag” are cut into multiple pieces. Program code, special characters, and emojis are also expensive because they break down into many small tokens.

Token consumption in everyday use

Anyone building an app with an AI feature sees consumption directly in the provider’s account. It states separately how many tokens were received and how many were generated. Output tokens usually cost several times as much as input tokens. Chat subscriptions also limit usage indirectly: when a service tells you that you’ve reached your limit, there’s often a token calculation behind it.

In news reports, the term comes up in the context of price cuts. Falling token prices are the main argument providers use to advertise new model versions. Conversely, reports about billions burned appear when users consume far more tokens than planned.

You can save with simple means. Start a fresh chat for a new topic instead of dragging along the old history. Attach only the relevant pages of a document. And choose a smaller model for simple tasks. This doesn’t change the token count, but significantly lowers the price per token.