13 min read
Long chats with ChatGPT or Claude tend to fail the same way. A constraint you set at the start quietly stops being applied, the model cannot find “that file I mentioned earlier,” and eventually the conversation refuses to continue. A model has no storage that holds a conversation between messages. It has a working surface that everything has to fit on, and the size of that surface is the context window.
Sizes here are counted in tokens, the chunks of text a model reads and bills by. A token runs roughly three to four characters of English, so a 500-word page lands somewhere near 650 tokens.
This piece collects what the official documentation from Anthropic, OpenAI, and Google said on November 4, 2026: how big the windows are, what fills them, where the rates change, and why accuracy still falls off in long conversations.
What a context window is
Anthropic’s documentation separates the context window from the training corpus and calls it “working memory”[1]. Training data is what the model learned before it ever met you. The window holds what it can see while writing the current reply.
The model rereads the whole conversation from the beginning every time you send a message. On your tenth question, the model reads every exchange from the first through the ninth along with the new question. Anthropic’s docs describe the same mechanic: as turns advance, each user message and response accumulates in the window, and previous turns are preserved intact[1].
“The AI forgot” therefore covers two different situations. In one, the conversation outgrew the window and the earlier part really is gone. In the other, the earlier part is still sitting in the window and the model failed to use it. The second is more common, and it calls for a different fix.
Current window sizes
Every figure in this table comes from the three vendors’ own model documentation[2][3][4].
| Model | Context window | Input per request | Output per request |
|---|---|---|---|
| Claude Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1 | 1,000,000 | window minus whatever is reserved for the reply | 128,000 |
| GPT-6 Astra | 1,050,000 | 922,000 | 128,000 |
| Gemini 4 Argon | 1,048,576 (input and output share it) | window minus the output setting | 65,536 by default, raisable toward the full window |
All four current Claude models sit at 1,000,000 tokens on the API, and that is the default with no special opt-in required[2]. GPT-6 Astra has a 1,050,000-token window but lists maximum input separately at 922,000 tokens[3]. Gemini’s documentation is the most explicit about the overlap: the Gemini 4 Argon page calls 1,048,576 a “unified input and output token limit,” with default output at 65,536 tokens and an option called long decode that raises the output ceiling toward the full window[4]. Raising the output ceiling leaves less room for input.
Window size and input limit are two different numbers. The reply draws on the same window, so part of it is always held back for what the model is about to write. Anthropic’s help center states that a portion of the context window is always reserved for Claude’s response, so the longest possible conversation is slightly smaller than the full window[5]. Astra listing 922,000 against a 1,050,000-token window is the same arithmetic made explicit.
For scale, Anthropic describes 200,000 tokens as roughly 500 pages of English text or more[6].
The same model gets a different window depending on where you use it
Anthropic publishes window sizes per product for paid plans. The table below centers on the models whose number changes depending on where you are working[6].
| Model | Chatting with Claude | Claude Code | Claude Cowork |
|---|---|---|---|
| Opus 5.5 | 1M | 1M | 1M |
| Sonnet 5.5 | 1M | 1M | 1M |
| Sonnet 4.6 | 500K | 1M | 200K |
| Opus 4.6 | 500K | 1M | 200K |
| Haiku 5.5 | 1M | 1M | 500K |
| Fable 5 | 500K | 1M | 1M |
Sonnet 4.6 gives you 500K in chat, 1M in Claude Code, and 200K in Cowork. Fable 5 in that table is the previous-generation model; Fable 5.1 is 1M across all three. Earlier-generation models can also need extra setup to reach their largest window: the docs note that Sonnet 4.6’s 1M window requires picking claude-sonnet-4-6[1m] from the model menu, with usage credits turned on[6]. Models absent from Anthropic’s table default to 200K.
Usage limits and length limits are also separate things in Anthropic’s documentation. Usage limits cap how much you can do across a five-hour session and across a week. Length limits govern how long any single conversation can get. A conversation blocked for length will not unblock when a usage window resets, so the fix there is a new chat[5].
ChatGPT’s per-plan window sizes are not in this piece because the primary sources were unreachable, which the closing section explains. The Astra figures above come from the developer documentation, which is reachable.
Your messages are the small part of what fills the window
Everything in the request occupies the window: the system prompt (standing instructions the service inserts before your first message), every message including tool results and attached images and documents, the tool definitions (the written specs telling the model when and how to call each tool), and the model’s own output including extended thinking, the private reasoning some models do before answering[1].
Tools and connectors are the item most people underestimate. Anthropic’s help center calls them token-intensive and recommends turning off the ones you are not using[5], because the spec describing each tool enters the window whether or not the tool ever gets called. Prompt caching, where a provider stores a repeated chunk of input so it does not have to be reprocessed, has a similar catch. Cached content still occupies the window; caching changes what you pay for those tokens while they keep taking up the same space[1].
Media has separate ceilings. A 1M-token model accepts up to 600 images or PDF pages per request, and a 200K-token model accepts 100. Send enough large documents and you hit that limit before the token limit[1].
Every turn re-bills the entire conversation
The accumulation shows up directly on the bill. A conversation with 300-token questions, 700-token answers, and a 2,000-token system prompt, run out to 30 turns and priced at the published rates, comes out like this[7].
Conversation length after 30 turns: 32,000 tokens
Input tokens billed across 30 turns: 504,000 tokens
Ratio: 15.8x
turn 1 request input: 2,300 tokens
turn 10 request input: 11,300 tokens
turn 30 request input: 31,300 tokens
Claude Opus 5.5: $2.44
GPT-6 Astra: $6.09

The conversation holds 32,000 tokens of text and 504,000 tokens of input were billed, so the model read the same material about 15.8 times[7]. The thirtieth question carries more than thirteen times the input of the first, which is why later questions in a thread cost more than early ones.
Attaching a long document and asking about it repeatedly widens that gap by another order of magnitude. A 50,000-token report attached once, followed by 20 questions, produces these figures[7].
Attached document: 50,000 tokens (attached once)
Input tokens billed: 1,236,000 tokens
Re-sent because of the doc: 1,000,000 tokens
Same 20 turns without it: 236,000 tokens
Claude Opus 5.5: $5.22 / at cache-hit rates: $0.53
GPT-6 Astra: $13.06 / at cache-hit rates: $1.94
The document went in once and was billed twenty times. Because the same content keeps reentering the request, a cache hit cuts the cost to between a seventh and a tenth. Opus 5.5 reads cached tokens at $0.20 per million against a $4 input rate, a twentieth of the price[8].
Ordinary back-and-forth does not come close to filling a 1M window. At 300-token questions and 700-token answers, reserving 128,000 tokens for the reply, the limit arrives around turn 870[7]. Attached documents, tool results, and thinking tokens are what fill these windows.
Rates change at 100K and 272K tokens, inside the window
GPT-6 Astra charges 2x on input and cache rates and 1.5x on output, applied to the entire request, once input passes 272,000 tokens[3]. Claude Haiku 5.5 moves from $0.10 to $0.50 per million input tokens, and $0.50 to $2.50 on output, above 100,000 input tokens[8]. Running the same calculator on either side of those lines gives[7]:
GPT-6 Astra, 270,000 input tokens: $2.70
GPT-6 Astra, 273,000 input tokens: $5.46 <- 2x rate
Claude Haiku 5.5, 99,000 tokens: $0.0099
Claude Haiku 5.5, 101,000 tokens: $0.0505 <- 5x rate

Three thousand tokens double the Astra bill, and two thousand tokens quintuple the Haiku bill. Working only from “the window is 1M” is how people cross these lines without noticing.
A bigger window is not uniformly reliable
Anthropic’s documentation has a name for the case where earlier material is still in the window and the model fails to use it. It calls this context rot: as token count grows, the model’s ability to accurately recall information from that context decreases. The same page adds that more context is not automatically better[1].
Anthropic’s engineering blog traces the effect to the architecture. In a transformer, the network design behind current language models, every token can be compared against every other token, so n tokens produce n² pairs to track, and tracking all of them becomes harder as the context lengthens. Models also train mostly on shorter passages, which leaves them with less practice at connections that span a long document. The blog frames this as an “attention budget” that each new token draws down. It also stresses that this is a gradient rather than a sharp drop: models stay capable at long context but lose precision on retrieving specific information and reasoning across distant parts of it[9].
The broadest public measurement is Chroma’s July 2025 technical report. Chroma, a company that builds search infrastructure for AI applications, held task difficulty constant and varied only input length across 18 models, finding that models do not use their context uniformly and grow less reliable as input grows[10]. That evaluation covered the GPT-4.1, Claude 4, and Gemini 2.5 generation, so its specific numbers should not be carried over to today’s models.
A 2026 follow-up looks at current ones. Xia and colleagues tested four flagship models on three long-horizon search benchmarks and found that models give up or produce uncertain wrong answers well before the window is exhausted. They name these inconclusive search failures and found them common even on easy and moderately difficult queries, which were queries the models were otherwise capable of solving[11].
The split runs along task type. Pulling one stated fact out of a long document holds up well at length. Counting or comparing items scattered through a long document loses accuracy as the document grows. Attaching a long file, asking for “all the ones that match,” and getting a list with gaps is the second case.
What happens when the window fills
In the Claude chat interface, paid plans with code execution enabled get automatic context management. As a conversation approaches the limit, Claude summarizes earlier messages to make room, and the conversation usually continues without interruption. The full chat history is preserved so Claude can still reference it, and the “organizing its thoughts” message during a long chat signals that this summarizing is under way. Conversations that trigger it consume more of your usage limit[6]. With code execution turned off, it does not run at all.
The API behavior is spelled out more precisely. If the input alone exceeds the window, the request is rejected outright with a 400 error, the standard code for a malformed request. On Claude 4.5-generation models and later, a request whose input plus requested output exceeds the window is still accepted, and generation stops with a model_context_window_exceeded signal if it reaches the limit[1]. OpenAI ships compaction as a documented server-side feature: past a threshold you set, the server replaces earlier context with a single compacted item and carries that into the next turn. The documentation notes that this item is opaque and not intended to be human-interpretable[12].
Compaction trades detail for room. Anthropic’s engineering blog says the art of it lies in choosing what to keep against what to discard, and that overly aggressive compaction loses subtle context whose importance only becomes apparent later[9]. Constraints set early in a chat going missing after it gets summarized is that trade showing up in practice.
What to do about it
- Start a new conversation when the topic changes. Continuing is more convenient, but the entire prior thread is resent on every turn, which raises both the bill and the error rate.
- Put reference documents in a Project rather than attaching them to a chat. Anthropic notes that Projects use retrieval-augmented generation, which searches your documents and loads only the passages relevant to each question instead of the whole file[5]. This pays off most when several conversations need the same material.
- Turn off connectors and tools you are not using. Their specs occupy the window even when they go unused[5].
- Keep project instructions short and put task-specific direction in the chat itself, which is also the official recommendation[5].
- Turn off extended thinking and lower the effort level when a task does not need them. Thinking tokens draw on both the window and the bill[5].
- Write important conclusions and constraints down outside the conversation, because automatic summarization can drop the details.
- Break up “find and count all of these” requests over long material into sections and total them yourself. That is the task type that degrades first.
- On the API, check whether you are crossing 272,000 tokens on GPT-6 Astra, which doubles the input rate, or 100,000 tokens on Claude Haiku 5.5, which quintuples it[3][8].
What this piece did not verify
No model was run for this article, so the degradation described above was not reproduced here. Every claim about accuracy comes from vendor documentation, the Chroma report, and the Xia paper. ChatGPT’s per-plan window sizes are omitted because openai.com and the OpenAI help center both block automated access, leaving no primary source to check, and third-party write-ups disagreed with one another. The cost figures are arithmetic applied to published rates and assumed conversation lengths, so a real bill will differ with cache hit rates and thinking token usage.
Window sizes and rates change often. These figures are current as of November 4, 2026, and the footnote links carry whatever has replaced them.
Notes
- Anthropic, Context windows, Claude Platform Docs (accessed 2026-11-04). Working memory framing, turn accumulation, what counts toward the window, context rot, the 600 image/PDF page ceiling, cached prefixes occupying the window, 400 errors and `model_context_window_exceeded`. Link ↩
- Anthropic, Models overview, Claude Platform Docs (accessed 2026-11-04). Fable 5.1, Opus 5.5, Sonnet 5.5, and Haiku 5.5 at a 1M-token window with 128K max output. Link ↩
- OpenAI, GPT-6 Astra, API documentation (accessed 2026-11-04). 1,050,000-token window, 922,000 maximum input tokens, 128,000 max output, $10 input and $1 cached input and $50 output per million tokens, and 2x input/cache plus 1.5x output above 272K input tokens. Link ↩
- Google, Gemini 4 Argon, Gemini API documentation (accessed 2026-11-04). 1,048,576 tokens as a unified input and output limit, 65,536 default output, up to 1,048,576 with long decode. Gemini 3.8 Flash lists input 1,048,576 and output 65,536 separately. Link ↩
- Anthropic, How do usage and length limits work?, Claude Help Center (accessed 2026-11-04). Usage versus length limits, the reserved response portion, Projects using retrieval-augmented generation, keeping instructions concise, extended thinking and effort settings, and the token cost of tools and connectors. Link ↩
- Anthropic, How large is the context window on paid Claude plans?, Claude Help Center (accessed 2026-11-04). Per-product window table, the 200K default and the 500-page figure, the `[1m]` model selection and usage credit requirement for Sonnet 4.6, automatic context management and its usage cost. Link ↩
- AI Signal calculation (2026-11-04), applying the published rates above to the turn structures described in the text to derive per-turn input, re-billed attachment volume, turns to fill the window, and the amounts on either side of each pricing threshold. ↩
- Anthropic, Pricing, Claude Platform Docs (accessed 2026-11-04). Opus 5.5 at $4 input, $20 output, $0.20 cache hits per million tokens; Haiku 5.5 at $0.10 input up to 100,000 tokens and $0.50 above it, with output moving from $0.50 to $2.50. Link ↩
- Anthropic, Effective context engineering for AI agents, engineering blog (accessed 2026-11-04). Attention budget, n² pairwise relationships, training distribution skewed toward shorter sequences, gradient rather than cliff, and the difficulty of choosing what compaction discards. Link ↩
- Hong, Troynikov, and Huber, Context Rot: How Increasing Input Tokens Impacts LLM Performance, Chroma technical report (2025-07-14). Task difficulty held constant while input length varied, 18 models evaluated, covering GPT-4.1, Claude 4, and Gemini 2.5. Link ↩
- Xia, Wang, Huang, and Liu, Diagnosing and Mitigating Context Rot in Long-horizon Search, arXiv:2606.29718 (submitted 2026-06-29, v3 2026-10-25). Four flagship models across three benchmarks, inconclusive search failures where models quit or answer uncertainly before exhausting the window, common even on easy and moderate queries. Link ↩
- OpenAI, Compaction, API documentation (accessed 2026-11-04). Server-side compaction past `compact_threshold`, with the compaction item described as opaque and not intended to be human-interpretable. Link ↩
Source: https://docs.claude.com/en/docs/build-with-claude/context-windows