Context windows: why an AI forgets the start of a long chat

The context window is the amount of text a model can hold in view for a single request, measured in tokens. It is not memory in any lasting sense. Every time you press send, the entire conversation is assembled and handed to the model afresh: the hidden instructions the product author wrote, every message you and the assistant have exchanged, any documents you attached, plus room reserved for the reply. If the total is under the limit, the model can see all of it. If it is over, something has to go, and what goes is usually the beginning of your conversation.

That single mechanism explains the two complaints people have about long chats: the model forgets things it clearly knew ten minutes ago, and it gets vaguer and less obedient the longer a session runs.

What is actually inside the window

Picture a fixed size container that gets refilled from empty on every turn:

The reserved output space is worth noticing. If a model advertises a 128,000 token window and reserves 8,000 for its answer, your usable input is 120,000. Some products split these into separate limits, so a large input window does not always mean a long answer is possible. The largest windows also tend to sit on the paid tiers, which is part of what a subscription actually buys you.

To convert any of this into pages, use the rough English ratio of 750 words per 1,000 tokens covered in what tokens are. A 128,000 token window is roughly a 300 page book. A million token window is a small shelf.

Failure one: the oldest turns fall off

When a conversation outgrows the window, something has to be discarded. Chat products usually do this quietly. The most common approaches are dropping the earliest messages entirely, or replacing the early portion with a compressed summary produced by the model itself.

Either way the loss is real. If you set out a careful brief in message one and are on message forty, that brief may simply no longer exist for the model. It is not being lazy or contradicting itself on purpose. It is answering a question it can see, with context it no longer has.

A summary is better than a hard cut, but summaries lose exactly what summaries always lose: specifics. Your instruction "always write amounts in dollars with a note that the same applies in pounds and euros" becomes "the user prefers currency formatted consistently", which is not the same instruction at all.

Failure two: it fits but gets ignored

The subtler problem is that fitting inside the window does not mean getting equal attention. Across many models, information placed at the very start or the very end of a long input is used more reliably than information sitting in the middle. Researchers describe this as a lost in the middle effect, and you can feel it in practice: paste 60 pages, ask about something on page 30, and you are more likely to get a vague or wrong answer than if you asked about page 1 or page 60.

This is why advertised window sizes should be read as capacity, not as guaranteed recall. A model that can accept a million tokens can genuinely process them. Whether it can reliably find one clause buried at the 400,000 token mark is a different question, and the honest answer is that accuracy degrades as the haystack grows.

Seven habits that keep a long session accurate

1. Put your instructions at the end of a long input, after the document, not before it. The last thing in the window gets the most reliable attention.
2. Restate hard constraints every five or six turns. One line is enough: "reminder: US spelling, under 400 words, no bullet lists."
3. Start a new chat whenever the task changes. A fresh window is free and removes forty turns of irrelevant history.
4. Attach the section, not the whole file. Ten relevant pages beat two hundred pages containing them, and doing that selection automatically over a whole library is what a retrieval setup is for.
5. Keep your own brief outside the chat, in a text file, and paste it in when you open a new session.
6. Ask the model to produce a handover summary before you start a fresh session: "list the decisions and constraints we have agreed so far." Then check it and paste it into the new chat.
7. For repeated multi step work, break it into separate short chats rather than one long one, with the output of each becoming the input of the next.

That last habit is the basis of prompt chaining, which is worth learning if you do the same long job regularly.

Memory features are not the same thing

Several assistants now offer a memory that persists across sessions, storing facts about you such as your job, your writing preferences or your projects. This is not a larger context window. It is a separate store, and the way it works is that selected notes get injected back into the window at the start of each new chat. That is useful, but it means the stored notes compete for the same space, and anything not stored is still gone. What gets saved, and how to review or delete it, is covered in AI memory and chat history.

What to check first

If a long chat has gone wrong, do not argue with it. Check three things in order: how many turns you are in, whether your original instruction is still visible in the scrollback, and whether the key material is at the start, middle or end of what you sent. Then open a fresh chat, paste the constraints and the relevant extract, and ask again in one well built prompt. The answer usually improves immediately, which tells you the model was never confused, just short of context.