Context window
The context window is the maximum number of tokens a language model can take into account at once. It covers both the input and the output produced and so determines how much context a model has. Modern models reach context windows of up to around a million tokens.
Also known as: context window, context length, context size
What is a context window?
The context window describes the maximum amount of Tokenthat a Language model can process at once. Think of it as the model’s short-term memory: only what lies within this window can the model take into account for its answer.
Crucially, the context window covers both input and output. The prompt, documents supplied, the conversation so far and the answer the model produces all share the same available space. The larger the context window, the more information the model can keep in view at once.
How is the size of the context window measured?
A context window's size is given in tokens, a language model's smallest unit of processing. A context window of 128,000 tokens, for instance, means input and output together may not exceed that limit.
Since a token is generally shorter than a whole word, the token count gives a rough idea of the amount of text. Importantly, German texts tend to need more tokens per unit of content than English ones, so the same token limit means somewhat less text in German.
The context sizes available have grown greatly in recent years. While early models could process only a few thousand tokens, modern models reach context windows of up to around a million. Whole books, extensive code bases or long document collections can thus be processed in a single pass.
This development has considerably widened AI systems' practical usefulness. Tasks that used to be possible only through laborious splitting and calling a model several times can now be done in one step. Handling the space available deliberately nevertheless remains decisive, since not every task benefits from a context window filled to the brim.
Why is the context window so important?
The context window directly determines how much information a model has available. In longer conversations, analysis of extensive documents or complex tasks with a great deal of background, a large context window is a clear advantage. If there is not enough room, content has to be shortened or split, which can lose information.
At the same time, a larger context window does not automatically mean better results. Very long inputs raise costs and response times, and not every model uses information far back in the window equally reliably. A well-considered Context engineering, the deliberate selection and structuring of what you pass in, is therefore often worth more than simply maxing out the size.
If the context window is exceeded, the model cannot process the request in full. Earlier content then drops out of the window and is no longer taken into account in the answer.
Using the context window in practice
For professional use of AI, handling the context window deliberately is decisive. Rather than passing a model large amounts of data at random, it is worth selecting the most relevant information. Techniques such as retrieval-augmented generation help feed in only the fitting passages from large bodies of knowledge.
The order and structuring of the content within the context window matters just as much. Particularly relevant information should be clearly highlighted and sensibly placed so the model takes it into account reliably. In long conversations it can also make sense to summarise earlier content rather than carry it along in full, to save room in the window.
At Elisabit we design AI applications so context windows are used efficiently without driving costs and response times up unnecessarily. That way models always have the right context and still work economically.
Frequently asked questions
What happens when the context window is exceeded?
If input and output together exceed the context window's limit, the model cannot take all the content into account. Earlier or surplus information drops out of the window and is ignored in the answer, which can mean information is lost.
Is the context window measured in words or tokens?
The context window is measured in tokens, a language model's smallest unit of processing. Since a token is usually shorter than a whole word, a given token count corresponds to a somewhat smaller number of words, especially in German texts.
Is a larger context window always better?
Not necessarily. A large context window does allow extensive content to be processed but also raises costs and response times. Selecting the relevant information deliberately is often more effective than simply maxing out the size.
Does the output count towards the context window?
Yes. The context window covers both the input and the output the model produces. Both share the same available space, which is why a very long answer leaves less room for the input.
Related terms
A token is the smallest unit a language model processes — the pieces text is broken into for processing.
Context engineering shapes the information an LLM receives so that its answers become more precise and more reliable.
An LLM is an AI language model that understands and produces text by predicting the most likely next word.
RAG combines a language model with the retrieval of relevant information from external knowledge sources before answering.
Prompt engineering is the craft of phrasing AI instructions so that language models return better results.
Put AI to work for your business?
We help you integrate artificial intelligence into your processes, your marketing and your website — strategically and securely.
Your contact
Stefan
I look forward to hearing about your project and finding the best solution together.
