Context Window
The context window is the amount of text an AI language program can take in at once. Anything outside this window doesn't exist for the program at the moment it responds.
Programs like ChatGPT answer questions by reading text and writing matching text to continue it. In doing so, they can’t take in an unlimited amount of text at once. There is a fixed upper limit to how much text they can have in front of them at the same time while answering. This upper limit is called the context window. It encompasses everything: your question, the earlier messages in the conversation, attached documents, and the answer currently being generated. Whatever doesn’t fit falls out and is not taken into account when answering.
Why conversations eventually forget the beginning
The context window is the reason for a behavior that irritates many users. In a long chat, the program suddenly forgets what was discussed at the start. It isn’t distracted or inattentive. The old messages simply no longer fit in the window and get cut off.
A common misconception is that such a program learns permanently from every conversation. It normally doesn’t. Its knowledge comes from training that was completed long before your chat. Everything you tell it now only lives within the context window. Close the chat, and it’s gone, unless the product separately stores notes.
For companies, window size is therefore a hard criterion. Anyone who wants a 200-page contract reviewed needs a window that can fit 200 pages. Otherwise the text has to be broken into chunks, and connections spanning across chunk boundaries get lost.
Measured in tokens, not pages
The size isn’t given in words but in so-called tokens. A token is a text building block, usually a short word or a word syllable. As a rule of thumb: 1000 tokens correspond to roughly 750 English words. A window of 128,000 tokens thus roughly holds a thick paperback.
When answering, the program compares every text building block in the window with all the others. This is how it recognizes which words belong together. But this exact comparison is expensive: doubling the amount of text can quadruple the computational effort. That’s why large windows were technically difficult for a long time and are still billed at a higher rate today.
A large window also doesn’t guarantee good usage. In tests, models find information at the beginning and end of a long text more reliably than in the middle. Experts call this the lost-in-the-middle effect. A window of one million tokens therefore doesn’t mean that all details within it are equally present.
Where window size shows up in products
In product announcements, window size is one of the most important numbers. Early versions of ChatGPT worked with just a few thousand tokens. Today, providers like OpenAI, Anthropic, or Google cite figures ranging from 128,000 to over one million tokens. Such leaps are a key selling point.
In everyday use, you notice the difference with large tasks. Reviewing an entire coding project, summarizing a book, analyzing an hour-long conversation transcript: this only works with a large window. For short questions, on the other hand, the size doesn’t matter.
You shouldn’t confuse the context window with the memory some chat apps offer. This memory stores notes about you outside the window. Before each answer, matching notes are copied into it. So the window stays the same size; it’s just filled more cleverly. The same idea underlies systems that search through documents before answering and only include the relevant passages.