Chain-of-Thought

Programs like ChatGPT assemble answers word by word. If you ask directly for the result, they often just guess quickly. Chain-of-Thought means: the program first writes down its calculation or reasoning path, step by step, and only states the result at the end. Roughly translated, it means “chain of thoughts.” The effect is the same as in a math exam: someone who only writes down the solution makes more mistakes than someone who notes the path.

Why this matters

The difference is measurable, not just cosmetic. On mathematical word problems, the accuracy rate rose in early experiments from around 18 to over 50 percent, simply by asking the model to show its path. The model itself was not changed. It merely used its computation time differently.

This has given rise to an entire generation of systems. Models like OpenAI o1, o3, or DeepSeek-R1 are called “reasoning models,” meaning models with a reasoning step. Before every answer, they think internally for a long time, sometimes spanning thousands of words. This costs money, since every one of these intermediate steps has to be computed. That’s why “thinking longer” is today both a selling point and a cost factor at the same time.

How it works

A language model only ever predicts the next word, based on everything that came before. Its own intermediate result thus becomes part of the basis for the next step. An intermediate step therefore relieves the memory: what has already been written down no longer needs to be kept in mind.

This can be triggered in three ways. First, through an instruction in the text, such as “Think step by step.” Second, through examples: you show the model a solved task along with its solution path, and it then imitates this pattern. Third, through training. Modern reasoning models are specifically trained to produce long solution paths and are rewarded when the correct result appears at the end.

One caveat is important here. The written-out path is not necessarily the path by which the model actually arrived at the answer. Studies show cases where the justification sounds plausible but was constructed after the fact. A chain of thought is therefore not proof, but a text that looks like a justification. It can also simply be wrong and still appear very convincing.

Where you encounter the term

In everyday use, you see the effect in chat interfaces. When it says “Thinking” or similar, and seconds to minutes pass, a chain of thought is running. Some providers show an abbreviated summary of it, others hide it because it is considered a trade secret. Anyone working with a chatbot themselves can force the effect by asking for the calculation path instead of just the number.

In business news, the term usually appears in connection with costs and data centers. Reasoning models consume a multiple of the computing power per request, which is driving demand for specialized chips. Experts speak of “test-time compute,” meaning computational effort at the moment of use. The idea behind it: performance no longer comes only from bigger models, but also from allowing a model to work longer. In audit reports from authorities and regulators, meanwhile, the visible chain of thought is discussed because it makes errors more traceable than a bare answer.