Stage 4 · What usually goes wrong
Context rot
Why does it get worse in long conversations?
In very long conversations the model pays less attention to what's in the middle, mixes old instructions with new ones and performs worse, even before it reaches the limit of the window.
An image to remember it
A desk buried in papers: everything is there, but you can't find anything.
The misunderstanding
More context is always better. Relevant context is better. Extra context distracts.
What changes in practice
Short, focused conversations. When one gets long, ask for a summary and start a new conversation with it.
