| An LLM context window is the model’s working memory for a single request. Token limits determine how much it can hold, how accurately it handles long inputs, and how much each query costs. Frontier models converged on 1-million-token windows in 2026. Information placed in the middle of a long context loses 10 to 25% recall accuracy compared to content at the start or end. Filling the same 1-million-token window costs between $0.09 and $10.00 depending on the model. RAG remains more cost-effective than large-context loading for most workloads. |
An LLM context window is the maximum number of tokens a model can process in a single request, and token limits shape three outcomes that matter most: how much the model remembers within a session, how accurately it handles long inputs, and how much each query costs. Context windows grew from 32,000 tokens in 2023 to 1 million tokens across frontier models in 2026, yet larger windows introduce quality and cost tradeoffs that raw size numbers do not capture. This article covers what context windows are, how tokens work, what degrades at scale, and how to work around the constraints. The Large Language Models subcategory on Universalnest.com maps the full technical picture across every article in this cluster.
What an LLM Context Window Is: The Model’s Complete Working Memory
An LLM context window is the total number of tokens a model can consider in a single request. It holds system instructions, conversation history, retrieved documents, user input, and the model’s own response all at once, with every element drawing from the same shared budget.
LLMs are stateless by design. They carry no memory between separate conversations. Everything the model uses during a session comes from one of two sources: the parameters baked into the model during training, or whatever is present in the current context window. When the session ends, the context is gone.
The working memory analogy maps directly: a human keeping track of instructions, earlier conversation points, and a new question holds all of it in active thought simultaneously. The model does the same but only within the token budget.
The fastest ways to consume context budget are system prompts (the instructions you give the model at the start of a session), accumulated conversation history (every prior turn in both directions), pasted documents, and the model’s own output, which also counts against the limit. When the window fills, the model does not issue a warning. It silently drops the oldest content and continues working with what remains.
How Tokens Work and What They Mean for Context Window Capacity
Tokens are the basic units an LLM processes, averaging 3 to 4 characters each, which means 1,000 English words equal approximately 1,300 tokens and a 128,000-token context window holds around 96,000 words, the length of a full novel.
Before entering the model, text converts into tokens through a tokenizer using Byte-Pair Encoding, which breaks text into subword units. A common word like “run” becomes a single token. A less common word like “tokenization” may split into two or three. Code, numbers, and non-English languages tokenise at lower efficiency, using more tokens per word than standard English prose.
As a practical rule: 75 English words equal approximately 100 tokens. Code typically runs 20 to 30% higher in token count than the same amount of English text.
| Context Window Size | Word Equivalent | Real-World Example |
| 16,000 tokens | ~12,000 words | A detailed technical report or long blog series |
| 128,000 tokens | ~96,000 words | A full novel or mid-sized software codebase |
| 1,000,000 tokens | ~750,000 words | 1,500 pages or an entire software repository |
| 10,000,000 tokens | ~7,500,000 words | 15,000 pages; a large archival library section |
Context Window Sizes Across Major LLMs in 2026: From 128K to 10 Million Tokens
The 1-million-token context window became the frontier standard in 2026, with GPT-5.5, Gemini 3.1 Pro, Claude Opus 4.8, and Claude Sonnet 5 all converging at that capacity. Llama 4 Scout holds the largest advertised window at 10 million tokens, though effective recall degrades significantly beyond 5 to 6 million tokens in independent benchmark testing.
In 2023, 32,000 tokens counted as long. By mid-2026, four tiers define the landscape: short (under 32K for on-device models), medium (128K to 256K for cost-sensitive enterprise tasks), large (1M for frontier closed-source models), and mega (10M for Llama 4 Scout’s open-weight architecture).
One distinction applies to every model in this table: advertised context and effective context are not the same number. GPT-5.4’s 1-million-token window delivers approximately 200,000 tokens of reliable recall for most production tasks. The remaining capacity exists on paper but comes with measurable accuracy degradation. Claude eliminated its long-context surcharge in March 2026, making flat-rate 1M pricing the new standard across its frontier range.
Context window capacity is one of five deployment factors in the open vs closed LLM selection decision. How the open vs closed LLM decision changes when context window capacity is a deployment constraint is covered in the previous article in this subcategory.
| Tier | Token Range | Key Models | Primary Use Case |
| Short | Under 32K | On-device models (Gemini Nano), small local builds | Chat, quick Q&A, code completion |
| Medium | 128K to 256K | DeepSeek V4 Flash (128K), Claude Haiku (128K) | Enterprise RAG, multi-document analysis |
| Large (Frontier) | 1M | GPT-5.5, Gemini 3.1 Pro, Claude Opus 4.8, Claude Sonnet 5 | Full-book processing, repository analysis |
| Mega | 10M | Llama 4 Scout (open-weight) | Archival retrieval, research use cases |
What Happens at the Context Limit: The Lost in the Middle Problem and Attention Dilution
When a context window fills, the model does not distribute attention equally across all tokens. A documented phenomenon known as the “lost in the middle” problem shows that information placed in the centre of a long context suffers 10 to 25% accuracy loss compared to information positioned at the start or end.
Liu et al. (2023) established this with research showing LLM attention follows a U-shaped curve: strong recall at the beginning and end, noticeably weaker in the middle. The NoLiMa benchmark from LMU Munich and Adobe Research, presented at ICML 2025, tested genuine recall by removing literal keyword matches between questions and answers. Without surface-level pattern matching to fall back on, 11 out of 13 tested LLMs dropped below 50% of their baseline accuracy scores at just 32,000 tokens. GPT-4o fell from 99.3% to 69.7%.
Attention dilution compounds this problem as context grows. A relevant 500-token passage in a 10,000-token context receives approximately 5% of the model’s total attention budget. In a 1-million-token context, the same passage receives approximately 0.05%. The signal-to-noise ratio deteriorates as the window expands.
Context rot describes the systematic performance decline that follows: a plateau phase where quality holds reasonably well, followed by a collapse when context density crosses a threshold. The practical consequence is consistent across model families: a curated 50,000-token context containing only relevant material outperforms a 500,000-token context where 90% of the content is loosely related.
65% of enterprise AI failures in 2025 were attributed to context drift or memory loss during multi-step reasoning tasks, making effective context management a production-critical discipline, not an optional optimisation.
How Context Window Size Controls LLM Cost: The Quadratic Scaling Problem
Context window usage determines cost because APIs bill per token and attention computation scales quadratically. Doubling the context from 64,000 to 128,000 tokens quadruples the processing work, which is why long-context requests cost more and respond more slowly than short ones at the same output length.
The underlying mechanism is the transformer’s self-attention layer. Every token attends to every other token in the window. A 10,000-token context requires 100 million pairwise comparisons. A 100,000-token context requires 10 billion. GPU memory becomes the bottleneck as this computation requires constant data movement between fast, limited SRAM and slower, larger HBM memory.
In practice, 78% of production LLM requests use under 16,000 tokens. Most teams never approach frontier context limits in day-to-day operation. The cost difference between providers for filling the same context budget is large enough to drive model selection decisions independently of performance considerations.
| Model | Cost to Fill 1M Input Tokens | Context Window |
| DeepSeek V4 Flash | $0.09 | 128K |
| Gemini 3.1 Pro | ~$0.31 | 1M |
| GPT-5.4 | ~$2.50 | 1M |
| Claude Opus 4.8 | ~$3.00 | 1M |
| Claude Fable 5 | $10.00 | 1M |
Input token costs only. Output costs add on top. Pricing reflects June to July 2026 API rates. Verify at the model’s documentation page before production decisions.
Strategic caching, compaction of earlier context, and context engineering techniques can reduce inference costs by 50 to 90% compared to naive full-window loading on every request.
How to Work Beyond Context Window Limits: RAG, Chunking, and Context Engineering
RAG remains more cost-effective than filling a large context window for most production workloads because it retrieves only the relevant portion of a knowledge base per request, rather than loading the entire dataset into context on every call.
Retrieval-Augmented Generation stores source documents in a vector database and fetches only the chunks most relevant to the current query. Context stays small, precise, and free from attention dilution. The retrieved content is placed where attention is strongest, the start of the context, which directly addresses the lost-in-the-middle risk that plagues naive long-context loading.
Chunking and sliding window: break long documents into overlapping segments that each fit within a manageable context budget. Process segments sequentially or in parallel, then merge outputs. This approach works well for summarisation and extraction tasks where cross-segment reasoning is not required.
Summarisation and compaction: compress earlier conversation history before passing it forward in a long agent session. Compacting at turn 10 in a 100-turn workflow saves more than the per-turn cost reduction suggests, because the compressed context carries through every subsequent request.
Context engineering: the placement of content within the window matters more than the total volume. Place the most critical instructions at the start, the user’s current question at the end, and lowest-priority material in the middle where attention is weakest. This applies to RAG pipelines, agent systems, and conversational applications equally.
The specific case where multi-file codebases exceed even large context windows, and which LLMs handle this best, is covered in the Large Language Models subcategory analysis of how context window size determines which LLMs perform best on multi-file coding and agentic engineering tasks.
Context window capacity defines what a model can hold in a single request. What goes into that window, where it is placed, and how much of it is genuinely relevant determines the quality of what comes out. Context engineering is now a production discipline as significant as prompt design, and the tools developers use to manage context connect directly to how AI products are built and priced.
Context Windows and the AI Tools That Depend on Them
AI productivity tools built on large language models inherit every context window constraint of their underlying model. A tool built on a 128,000-token model cannot hold an entire enterprise document in working memory the way one built on a 1-million-token model can. This difference shows up in how tools handle long document sessions, extended conversations, and multi-source research tasks.
The AI Tools and Productivity subcategory on Universalnest.com covers how these differences play out across the tools developers and businesses use daily, including how long-context capabilities change the way AI tools handle multi-document content creation workflows.
LLM Context Windows Within the Full AI Technology Landscape
Context windows are one technical layer in how large language models function. They sit alongside training data, model architecture, benchmark performance, and deployment cost as the variables that determine what any LLM can reliably do in production.
LLMs and the technology reshaping every industry tracked across AI News and Tech in 2026 maps every dimension of how these systems are built, compared, and deployed across all 12 subcategories on Universalnest.com.
What LLM Context Window Size Actually Determines in 2026
Context window size no longer separates frontier models in 2026. GPT-5.5, Gemini 3.1 Pro, Claude Opus 4.8, and Claude Sonnet 5 all ship at 1 million tokens. What still differentiates them is effective recall accuracy at that scale, cost per token, and reasoning quality within the window.
The more productive questions for any developer or business are whether the content entering the context is positioned correctly for maximum recall, whether RAG serves the workload at lower cost than full-window loading, and whether token consumption is actively monitored. 78% of production requests never exceed 16,000 tokens, which means the 1-million-token race is largely irrelevant to most real workloads.
Universalnest.com covers the complete Large Language Models subcategory, from how these models are trained and benchmarked to how they are selected, priced, and deployed at scale across every use case developers and businesses face in 2026.
Frequently Asked Questions
What is an LLM context window in simple terms?
An LLM context window is the model’s working memory for a single request. It holds system instructions, conversation history, documents, user input, and the model’s response simultaneously. When the token limit is reached, the oldest content is silently dropped. The model has no memory outside this window.
How many words fit in a 128K token context window?
A 128,000-token context window holds approximately 96,000 English words, equivalent to a full-length novel. As a practical rule, 75 English words equal roughly 100 tokens, so 1,000 words equal approximately 1,300 tokens. Code and non-English text use more tokens per word than standard prose.
Why do larger context windows cost more per request?
Attention computation scales quadratically with context length. Doubling the context from 64,000 to 128,000 tokens quadruples the processing work. APIs bill per token, and the additional computation raises both the cost and latency of each request. Longer context is inherently more expensive to process.
What is the lost in the middle problem in LLMs?
The lost in the middle problem is the finding that LLMs recall information at the start and end of a context more accurately than information in the middle. Research shows 10 to 25% accuracy degradation for middle-positioned content. The NoLiMa benchmark (ICML 2025) confirmed this across 13 models, with GPT-4o dropping from 99.3% to 69.7% at 32K tokens.
What is the difference between advertised and effective context length?
Advertised context length is the maximum token count a model accepts. Effective context length is how much of that window the model reliably uses without recall degradation. GPT-5.4’s 1-million-token window delivers approximately 200,000 tokens of reliable recall for most tasks. Every model’s effective context falls below its advertised maximum.
Is RAG still useful when a model has a 1-million-token context window?
RAG remains useful even with million-token windows because loading an entire knowledge base into context on every request is expensive and introduces attention dilution. RAG retrieves only the relevant portion per query, keeping context focused and costs 50 to 90% lower than filling a large window with mixed-relevance material on each call.
How can I reduce context window costs without changing models?
Prompt caching avoids re-billing identical system prompt content across requests. Summarisation compresses earlier conversation history before passing it forward. Context engineering places critical information at the start and the user’s question at the end. Combined, these three techniques can reduce LLM inference costs by 50 to 90% compared to naive full-window loading.
