How to Fix Your Context

Summary

Mitigating & Avoiding Context Failures Following up on our earlier post, “How Long Contexts Fail”, let’s run through the ways we can mitigate or avoid these failures entirely. But before we do, let’s briefly recap some of the ways long contexts can fail: Context Poisoning: When a hallucination or other error makes it into the context, where it is repeatedly referenced. Context Distraction: When a context grows so long that the model over-focuses on the context, neglecting what it learned during training. Context Confusion: When superfluous information in the context is used by the model to generate a low-quality response. Context Clash: When you accrue new information and tools in your context that conflicts with other information in the prompt. Everything here is about information management. Everything in the context influences the response. We’re back to the old programming adage of, “Garbage in, garbage out.” Thankfully, there’s plenty of options for dealing with the issues above. Context Management Tactics RAG Retrieval-Augmented Generation (RAG) is the act of selectively adding relevant information to help the LLM generate a better response. So much has been written about RAG, we’re not going to cover it today beyond saying: it’s very much alive. Every time a model ups the context window ante, a new “RAG is Dead” debate is born. The last significant event was when Llama 4 Scout landed with a 10 million token window. At that size it’s really tempting to think, “Screw it, throw it all in,” and call it a day. But, as we covered last time: if you treat your context like a junk drawer, the junk will influence your response. If you want to learn more, here’s a new course that looks great. Tool Loadout is the act of selecting only relevant tool definitions to add to your context. The term “loadout” is a gaming term that refers to the specific combination of abilities, weapons, and equipment you select before a level, match, or round. Usually, your loadout is tailored...