What the long-context papers actually measured, how far that evidence stretches to a coding agent session, and which habits (clearing, compaction, subagents) it really supports.
Contents · 11 sections
Claude Code's docs say the newest Claude models run with a 1M-token window by default. The same docs' best-practices guide tells you to "use /clear frequently between tasks". A vendor that just gave you a million tokens is telling you not to use them. Those two statements only fit together if a bigger window is not the same thing as a bigger working memory.
The name for the gap is "context rot", and it gets cited far more often than it gets examined. It's used to justify every session habit going: clear often, compact early, push everything into subagents, keep instruction files short. Some of those habits rest on published measurements. At least one rests on a vendor heuristic with no measurement behind it. And the measurements that do exist were made on question answering and synthetic retrieval, not on the thing you actually run: an agent session where tool output, diffs, failed attempts and your corrections pile up in one transcript.
So this piece keeps the two apart. First the long-context papers and their numbers, then the closest coding and conversation studies, then how Claude Code spends and recovers its window, and then which session practices the evidence supports and how strongly. It ends with a method you can run on your own transcripts, because the measurement that would settle the question for coding agents hasn't been published. It's the second part of a series that started with the coding agent as a system, where context was one of four levers.
A bigger window is not a bigger working memory
Anthropic's context engineering post (September 2025) is where most people pick up the term. It says needle-in-a-haystack studies "uncovered the concept of context rot: as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases", and that "this characteristic emerges across all models", though "some models exhibit more gentle degradation than others".
The post gives two causes. The first is architectural: a transformer lets every token attend to every other, which "results in n² pairwise relationships for n tokens", and as length grows the model's "ability to capture these pairwise relationships gets stretched thin". The second is training data. Shorter sequences are more common in training, so models have "less experience with, and fewer specialized parameters for, context-wide dependencies", and techniques like position encoding interpolation stretch a model to longer inputs "with some degradation in token position understanding".
The part of that post I find most useful is a single phrase: these factors "create a performance gradient rather than a hard cliff". There's no token count where the model falls over. There's a slope, and the post's prescription follows from it: find "the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome", and treat context as "a finite resource with diminishing marginal returns", an "attention budget" that every token draws down.
Claude Code's docs turn that into user-facing guidance. The best-practices page opens with the claim that "most best practices are based on one constraint: Claude's context window fills up fast, and performance degrades as it fills", and adds that when the window is getting full, "Claude may start 'forgetting' earlier instructions or making more mistakes". How Claude Code works gives the mechanical version: as the window approaches its limit, Claude Code "clears older tool outputs first, then summarizes the conversation if needed", and "detailed instructions from early in the conversation may be lost".
So the vendor position is clear and consistent: degradation is real, it's gradual, and it's why you manage sessions. None of those documents cites a number for how much, or at what length, for a coding session. For numbers, you have to go to the papers.
Lost in the Middle showed that position matters, not only length
The paper everyone cites first is Liu et al., "Lost in the Middle". The arXiv record says it was "accepted for publication" in TACL in 2023; the published version is in Transactions of the ACL, volume 12 (2024), pages 157-173, DOI 10.1162/tacl_a_00638. Cite the 2024 version; the arXiv comment says 2023 because that's when the paper was accepted.
The setup is deliberately plain. There are two tasks. In multi-document question answering, the model gets a question and 10, 20 or 30 retrieved passages, exactly one of which contains the answer, and the authors move that passage from first to last. In synthetic key-value retrieval, the model gets a JSON object of random UUID keys and values and has to return the value for one key. The main experiments used the long-context models of mid-2023: MPT-30B-Instruct, LongChat-13B (16K), GPT-3.5-Turbo and its 16K variant, and Claude-1.3 and its 100K variant.
The headline result is the U-shaped curve. The abstract says performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models". The most quoted detail is how bad the middle can get. GPT-3.5-Turbo answers 56.1% of the questions with no documents at all (closed-book, from its own weights, per Table 1 of the TACL paper). In the worst case, with 20 or 30 documents and the answer in the middle, its accuracy fell below that closed-book number. The model did worse with the answer in front of it than with no documents at all.
Two other findings matter more for agents than the U-shape itself. First, the extended-context variants weren't better at using context. When the input fit in both windows, the paper says performance between GPT-3.5-Turbo and GPT-3.5-Turbo (16K) "is nearly identical", which is its evidence that a longer window and better use of the window are different properties. Second, the obvious prompt trick only half worked. Putting the query both before and after the data ("query-aware contextualization") made key-value retrieval near-perfect, but on multi-document QA it "minimally affects performance trends". Restating the question helps a lookup; it doesn't fix reasoning over a crowded context.
For your session, it says less than people assume. The models are three years old. The tasks are retrieval tasks where the relevant passage is a well-formed paragraph among other well-formed paragraphs. A transcript where the relevant fact is one line of npm test output surrounded by thousands of lines of other output is a different shape of haystack. What I take from Liu et al. is narrower and more durable: where a fact sits in the window can matter as much as whether it's there at all, and a larger window doesn't fix that by itself.
Even easy retrieval degrades with length: NoLiMa and RULER
The obvious objection to Lost in the Middle is age. By 2024, models were scoring close to perfectly on the standard needle-in-a-haystack test (both papers below say so, and NoLiMa calls NIAH-based results "near-saturated"), so maybe the problem went away. Two later papers say it didn't; the standard test had stopped measuring it.
NoLiMa removes the shortcut
NoLiMa (Modarressi et al., accepted at ICML 2025 per its arXiv comment, v3 dated 9 July 2025) starts from a specific criticism of needle tests: the question and the needle usually share words, so the model can find the needle by literal matching. NoLiMa writes needles and questions with minimal lexical overlap. In the paper's example, the needle says a character "lives next to the Semper Opera House" and the question asks which character has been to Dresden, so the model has to know that the opera house is in Dresden rather than match a string.
The authors evaluated 13 models that claim at least 128K tokens of context, at lengths from 250 tokens to 32K; each model's "base score" averages, over all question-needle pairs, the best of its results at the three shortest lengths (250, 500 and 1K). The abstract's summary is the one that gets quoted: "At 32K, for instance, 11 models drop below 50% of their strong short-length baselines", and GPT-4o, one of the best, went "from an almost-perfect baseline of 99.3% to 69.7%". Note what 32K is. It's a small fraction of the windows these models advertise, and Claude Code's best-practices guide says a single debugging session or codebase exploration "might generate and consume tens of thousands of tokens".
The full table is more useful than the summary. These are rows from Table 3 of the v3 full text, scores in percent:
| Model | Claimed | Effective | Base | 1K | 4K | 16K | 32K |
|---|---|---|---|---|---|---|---|
| GPT-4o | 128K | 8K | 99.3 | 98.1 | 95.7 | 81.6 | 69.7 |
| Llama 3.3 70B | 128K | 2K | 97.3 | 94.2 | 81.5 | 59.5 | 42.7 |
| Gemini 1.5 Pro | 2M | 2K | 92.6 | 86.4 | 75.4 | 55.5 | 48.2 |
| Claude 3.5 Sonnet | 200K | 4K | 87.5 | 85.4 | 77.6 | 45.7 | 29.8 |
| GPT-4o mini | 128K | <1K | 84.8 | 67.7 | 44.2 | 20.6 | 13.7 |
NoLiMa's "effective length" is the longest tested length at which a model stays above 85% of its own base score. By that measure, a model that advertises two million tokens was effective to 2K on this task, and GPT-4o mini didn't clear the bar even at 1K. The appendix extends the test. Table 10 adds newer models and longer haystacks: GPT-4.1, which claims 1M tokens, had an effective length of 16K and scored 64.7% at 128K against a base of 97.0%; GPT-4o fell to 56.0% at 128K; Gemini 2.0 Flash fell to 16.4%.
Two more NoLiMa details carry over to agents. The paper reports a lost-in-the-middle dip at 32K when it varies needle placement, so the position effect survived into 2025 models. And the drop got worse with two-hop questions, where the model has to chain two associations, which is closer to "remember the convention, then apply it to this file" than a single lookup is.
Fig. 1 replots five of those rows, normalized to each model's base score so the slopes can be compared. The right-hand panel is empty on purpose. It's where a curve for coding-agent transcripts would go, and as far as I could find, nobody has published one.
RULER shows that a perfect needle score hides the drop
RULER (Hsieh et al., COLM 2024) comes at the same problem from a different side. Instead of making the needle harder to recognize, it adds task types: several needles at once, needles with multiple keys or values, multi-hop variable tracing, aggregation tasks that ask for the most frequent words across the whole context, and question answering. Thirteen tasks in all, at lengths from 4K to 128K, with 500 examples per length.
Its abstract is blunt: "Despite achieving nearly perfect accuracy in the vanilla NIAH test, almost all models exhibit large performance drops as the context length increases", and "while these models all claim context sizes of 32K tokens or greater, only half of them can maintain satisfactory performance at the length of 32K". RULER's threshold for "satisfactory" is fixed rather than relative: Llama 2 7B's average score at 4K, which is 85.6%.
When I read Table 3 of the v3 full text, I counted 10 of the 17 models with an effective length of 32K or more, which is more than half. The first version, from April 2024, evaluated ten models and its abstract didn't have that sentence; I couldn't work out which count the current sentence refers to. It doesn't change the thesis: almost every model falls below the threshold before its claimed length. It's a reminder to read the table rather than the abstract. A few rows show how uneven the slope is: GPT-4 went from 96.6 at 4K to 81.2 at 128K; Llama 3.1 70B from 96.5 to 66.6; Qwen2 72B held 94.1 at 32K and then fell to 53.7 at 128K, its claimed length. Gemini 1.5 Pro was the exception, at 94.4 at 128K.
Put the two papers together and you get the claim I'm comfortable stating as fact: on retrieval and reasoning tasks over long inputs, accuracy falls as input grows, it falls well before the advertised window, and the standard needle test is too easy to show it. What neither paper tells you is the size of the effect on a coding agent. For that, the evidence thins out fast.
The closest evidence to a coding session is still not a coding session
All three papers so far use question answering or synthetic tasks. Two newer sources get closer to what an agent does, and both are worth reading for what they don't measure as much as for what they do.
LongCodeBench stuffs a repository into the prompt
LongCodeBench (Rando et al., an arXiv preprint, v3 dated 22 October 2025, with no venue listed) builds two coding tasks from real GitHub issues: LongCodeQA, multiple-choice questions about a repository, and LongSWE-Bench, generating a patch for a real bug. Context lengths are bucketed at 32K, 64K, 128K, 256K, 512K and 1M. For the bug-fixing task, the prompt is a selection of files, the ones the fix needs plus random others, sized to hit the target length, and there are 100 instances per bucket.
The headline number is that Claude 3.5 Sonnet's accuracy on LongSWE-Bench "drops from 29% to 3% when increasing context length from 32K to 256K". That's the most coding-specific long-context number I know of, and it points the same way as everything above.
It's still not a session. The input is a static block of source files, assembled in one shot, with no tool calls, no history, no failed attempts. The paper's own appendix shows how much the filler matters: on a smaller set where the extra files were chosen by BM25 relevance instead of at random, the authors' logistic-regression trend between length and Claude 3.5 Sonnet's solve rate had a p-value of 0.025, above their 0.01 threshold, while the random-file version came in at 0.005. They read that as a variance trade-off, not as evidence the effect disappears, and an agent that reads files because they look relevant is, to my mind, closer to the BM25 case than the random one. Each length bucket also has its own 100 instances, so length isn't the only thing changing between buckets. And the models tested are a generation behind what you're likely running. I'd treat it as evidence that code doesn't escape the effect, not as a measurement of how big the effect is in an agent loop.
Chroma's LongMemEval test uses a conversation as the haystack
The other source is Chroma's July 2025 technical report, Context Rot: How Increasing Input Tokens Impacts LLM Performance. It's not peer-reviewed, and Chroma sells retrieval infrastructure, which is a product whose value goes up if long context is unreliable. I'm citing it because its design is relevant and its method is published, not because it settles anything.
The report evaluates 18 models, including Claude 4, GPT-4.1, Gemini 2.5 and Qwen3. The relevant experiment uses LongMemEval: a chat history between a user and an assistant, and a question about part of it. Each prompt comes in two versions, a "full" one averaging about 113K tokens of mostly irrelevant history, and a "focused" one of about 300 tokens holding only the relevant parts. The report says that "across all models, we see significantly higher performance on focused prompts compared to full prompts". That's the closest published analog I found to the question you care about: same task, same model, with and without the accumulated history. One detail cuts against reading it as a pure accuracy loss: the report says the Claude models showed "the most pronounced gap" between the two, "largely driven by abstentions", with Opus 4 and Sonnet 4 declining to answer under ambiguity.
Two of the report's other findings are odd enough to remember. Models did better when the haystack's sentences were shuffled than when they kept their logical flow, across all 18 models. And the failure modes differed by vendor: in the distractor experiments, the report says Claude Sonnet 4 and Opus 4 "tend to abstain when uncertain", while GPT models were more likely to give "confident but incorrect responses". For an agent, those are very different failures. An abstention shows up as a question to you. A confident wrong answer shows up as a plausible diff.
The direction is settled; the size is not
Here's where I come down. The direction of the effect is well supported: across retrieval, reasoning, code QA and conversational recall, more irrelevant context means lower accuracy, and position within the window matters. The size of the effect for a real coding session, with interleaved tool calls and outputs, at the lengths agents actually reach, on current frontier models, has not been measured in anything I could find. Every practice below should be read with that in mind. They're reasonable responses to a real effect of unknown magnitude.
A harness spends the window in a fixed order
If you can't control how the model degrades, you can control what's in the window when it does. That's the harness's job, and Claude Code documents its own behavior in more detail than most tools. All of what follows was checked against the docs on 18 October 2026, when the latest release in the changelog was 2.1.302 (16 October 2026). These details are version-gated and move fast.
Compaction runs at a model-dependent threshold you can lower
Compaction is triggered by a threshold, and the threshold depends on the model. The model configuration page says that by default Claude Code compacts when the conversation reaches the model's context limit, with exceptions. Models running with a native 1M window, which "include the Fable models, Sonnet 5 and later, Haiku 5.5, and Opus 4.7 and later", compact "at about 967K tokens by default". Sonnet 4.6 and Opus 4.6 without extended context compact at the 200K boundary, and so does any native-1M model if you set CLAUDE_CODE_DISABLE_1M_CONTEXT=1.
You can move the threshold. /autocompact takes a value from 100K to 1M tokens (/autocompact 500k, for example), saved per model since v2.1.288; there's also an autoCompactWindow setting, a --autocompact launch flag, and a CLAUDE_CODE_AUTO_COMPACT_WINDOW environment variable that overrides the rest. Since v2.1.296, the subagent docs say a subagent's frontmatter can set its own autoCompactWindow to make that subagent compact earlier.
{
"autoCompactWindow": 300000
}On a native-1M model, that makes the session compact at 300K tokens instead of about 967K. It applies to every model, with three exceptions the docs spell out: a window you saved for one model with /autocompact wins over it, CLAUDE_CODE_AUTO_COMPACT_WINDOW wins over everything, and the value is capped at the model's own window, so a 200K model still compacts at 200K. I'm showing it because it's the most direct lever the research points at, not because 300K is a measured optimum. Nobody has published one, and the right number depends on the model and the work.
There's also a newer trigger that isn't about the window at all. The settings reference describes idle compaction, added in v2.1.299: in an interactive terminal session on the one-hour prompt cache, a conversation of at least 200,000 tokens (the default threshold) left idle for about 54 minutes gets compacted "shortly before its prompt cache expires". The changelog entry gives the reason: "so your next message doesn't reprocess it all". That's about cost, not quality, but it means a long session can come back from lunch already summarized.
Old tool output is cleared before anything is summarized
Summarizing isn't the first step. How Claude Code works says the harness "clears older tool outputs first, then summarizes the conversation if needed". That ordering matches Anthropic's general advice: the context engineering post calls tool result clearing "one of the safest lightest touch forms of compaction", reasoning that once a tool result is deep in the history, the agent rarely needs to see the raw output again.
If you build your own agent on the API, the same mechanism is exposed as context editing. It's in beta behind the context-management-2025-06-27 header. The clear_tool_uses_20250919 strategy clears the oldest tool results once the prompt passes a trigger, by default 100,000 input tokens, keeps the 3 most recent tool use/result pairs by default, and replaces each cleared result with placeholder text so the model knows something was removed.
import Anthropic from '@anthropic-ai/sdk';
const client = new Anthropic(); // reads ANTHROPIC_API_KEY from the env
export async function nextTurn(
messages: Anthropic.Beta.BetaMessageParam[],
tools: Anthropic.Beta.BetaToolUnion[],
) {
return client.beta.messages.create({
model: process.env.MODEL!, // pin the exact model ID you evaluate
max_tokens: 4096,
tools,
messages,
betas: ['context-management-2025-06-27'],
context_management: {
edits: [
{
type: 'clear_tool_uses_20250919',
// Start clearing well before the window is full.
trigger: { type: 'input_tokens', value: 60000 },
keep: { type: 'tool_uses', value: 5 },
// Clearing breaks the prompt cache; make it worth it.
clear_at_least: { type: 'input_tokens', value: 10000 },
// Never clear results the agent must keep re-reading.
exclude_tools: ['read_project_rules'],
},
],
},
});
}That example type-checks against @anthropic-ai/sdk 0.135.0, the latest on npm as of 18 October 2026, and uses the option names from the context-editing docs. Run against a local stub of the API, it sends the context-management-2025-06-27 beta header and the context_management block as written; I haven't run it against the real API. The clear_at_least line matters more than it looks. The docs say clearing "invalidates cached prompt prefixes when content is cleared" and tell you to "clear enough tokens to make the cache invalidation worthwhile", so a strategy that clears a few hundred tokens on every turn can cost more than it saves. read_project_rules is a placeholder for whatever tool in your agent returns content it must not lose.
What survives compaction depends on how it got into the window
When summarization does run, the question is what comes back. The context window page has a table that I think every team running Claude Code should read once, because it decides where your rules should live. Condensed:
| Content | After compaction |
|---|---|
| System prompt and output style | Still apply |
| Project-root CLAUDE.md and unscoped rules | Re-injected from disk |
| Auto memory | Re-injected from disk |
| The plan written in plan mode | Re-injected from disk |
Rules with paths: frontmatter |
Reloaded on demand |
| Nested CLAUDE.md in subdirectories | Reloaded on demand |
| Files Claude read or edited | Up to five re-read, most recently modified first |
| Invoked skill bodies | Re-injected, 5,000 tokens per skill, 25,000 total |
| Context that hooks added earlier | Summarized with the rest |
SessionStart hooks matching compact |
Run again, output added |
Three details from the prose under that table matter in practice. A re-read file "over 5,000 tokens comes back as a path reference without its content". Skill bodies over the cap are truncated, and "truncation keeps the start of the file", so the docs advise putting "the most important instructions near the top of SKILL.md"; once the 25,000-token total is exceeded, the oldest invoked skills are dropped. And path-scoped rules and nested CLAUDE.md files "load into message history when Claude reads, writes, or edits their trigger file, so compaction summarizes them away with everything else". They come back only when a matching file is touched again.
Fig. 2 plays that sequence. Notice what kind of content gets summarized: everything that arrived through the conversation, which includes your corrections, a rule you typed in chat, and every path-scoped rule that loaded because of a file read. The interactive timeline on the context window page says the summary keeps "your requests and intent, key technical concepts, files examined or modified with important code snippets, errors and how they were fixed, pending tasks, and current work", and that "full tool outputs and intermediate reasoning are gone". A summary of a correction is not the correction.
Each session practice rests on different evidence
With the research and the harness behavior in hand, the vendor guidance sorts itself into tiers. Here's each practice, what it's justified by, and how strong that justification is.
/clear between unrelated tasks: well motivated
The best-practices guide names the "kitchen sink session" as a failure pattern: you start one task, ask something unrelated, go back to the first, and the "context is full of irrelevant information". The fix is /clear between unrelated tasks.
That's the practice with the most support. It's what the Chroma LongMemEval comparison measures most directly (focused versus full history on the same question), and it's consistent with every paper above: irrelevant tokens lower accuracy on the relevant ones. It's also cheap to undo if you were wrong, because the important things for the next task (code, tests, CLAUDE.md) are on disk, not in the conversation.
/clear after two failed corrections: a heuristic
The same guide says: "If you've corrected Claude more than twice on the same issue in one session, the context is cluttered with failed approaches", and in its failure-patterns list, "After two failed corrections, /clear and write a better initial prompt incorporating what you learned." It adds that "a clean session with a better prompt almost always outperforms a long session with accumulated corrections".
The reasoning is sound. Failed attempts are plausible, on-topic distractors, and the Chroma report found that distractors hurt unevenly and that their impact shows "more prominently as input length increases". But the number two is a heuristic. I couldn't find a published measurement behind it, and the guide doesn't cite one. If your sessions recover after three corrections, you're not contradicting any evidence.
Compaction: necessary, lossy, and tunable
Compaction is the harness's answer to running out of room, and the context engineering post is candid about its cost: "overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later." Its advice to people building compaction is to tune the prompt "on complex agent traces", maximizing recall first and then improving precision.
As a Claude Code user you get two levers on the summary. How Claude Code works says you can add a "Compact Instructions" section to CLAUDE.md, or run /compact with a focus. The best-practices page gives an example instruction: "When compacting, always preserve the full list of modified files and any test commands".
## Compact Instructions
When compacting, always preserve:
- the full list of files modified in this session
- the exact test command for each package touched
- any decision the user made that overrides a rule in this fileThat file is read from disk after compaction, so the instruction itself survives. Whether the summarizer follows it every time is a separate question, and one you can test with the method later in this piece.
Two version notes show that compaction is a piece of software that changes, not a fixed operation. The changelog for 2.1.293 (7 October 2026) lists a fix for "Claude sometimes treating its own last actions before a context compaction as done after it, and retracting or redoing finished work", which is a compaction failure mode in plain words. And per the context window page, since v2.1.198 the summarization request inherits the session's extended-thinking setting, which changes how the summary is produced.
Subagents: good isolation, at a price
The best-practices guide's most specific advice for context is to "use subagents to keep research out of" the main conversation, because "subagents run in separate context windows and report back summaries". The context engineering post puts a number on the summary: a subagent might use "tens of thousands of tokens or more" exploring, but returns "a condensed, distilled summary of its work (often 1,000-2,000 tokens)". That's a description of typical behavior, not a measured distribution.
Fig. 3 shows the shape of the trade. The subagent docs are specific about what isolation means: a non-fork subagent "doesn't see your conversation history, the skills you've already invoked, or the files Claude has already read". It starts from its own system prompt, the delegation message Claude writes, the CLAUDE.md hierarchy, a git status snapshot, and any skills preloaded in its definition. The built-in Explore and Plan agents skip both the CLAUDE.md files and the git status, though a subdirectory's CLAUDE.md and matching path-scoped rules still load when they work with files there. A fork is the exception: it "inherits the parent conversation instead of starting fresh".
That isolation is both the benefit and the cost. The docs list the trade-off themselves under choosing between subagents and the main conversation: stay in the main conversation when "multiple phases share significant context" or when "latency matters", because a subagent that isn't a fork "starts fresh and may need time to gather context". The subagent page also notes that each subagent "sends its own requests, which count toward the same usage limits". So a subagent buys a clean main thread with duplicated exploration, extra latency, and whatever the delegation message failed to mention. The evidence for the benefit is the same evidence as for /clear; the cost is documented by the vendor and real.
Resets versus compaction: a design account
The last practice is the most interesting and the least tested. Anthropic's harness design post (March 2026) describes "context anxiety", where some models "begin wrapping up work prematurely as they approach what they believe is their context limit". Its answer was a context reset: "clearing the context window entirely and starting a fresh agent, combined with a structured handoff that carries the previous agent's state and the next steps".
The post draws the distinction carefully. Compaction summarizes "in place so the same agent can keep going", which "preserves continuity" but "doesn't give the agent a clean slate, which means context anxiety can still persist". In earlier testing, it says, Sonnet 4.5 showed context anxiety "strongly enough that compaction alone wasn't sufficient", so resets "became essential". Then the model changed. "Opus 4.5 largely removed that behavior on its own", the author writes, so the later harness dropped resets entirely and ran "as one continuous session across the whole build", with the Agent SDK's automatic compaction handling growth.
This is the best illustration I know of the point from the first part of this series: a harness component encodes an assumption about the model, and the assumption expires. It's also a single engineer's account of building one harness, not a controlled comparison of resets and compaction. I'd take two things from it: resets with a written handoff are a legitimate tool when a model misbehaves near its limit, and you should re-test whether you still need them every time the model changes.
Protocols and formats already budget the window
The research has reached the specs, too. Several of the formats agents use were designed so that content doesn't have to sit in the window all the time, and knowing which knobs exist helps you use them.
The Agent Skills specification builds progressive disclosure into the format. At startup, only the metadata (the name and description fields, "~100 tokens") loads for each skill. The full SKILL.md body, with "< 5000 tokens recommended", loads when the skill is activated. Files under scripts/, references/ or assets/ load "only when required". The spec also says to "keep your main SKILL.md under 500 lines". That design spends tokens in proportion to need, which is the attention-budget idea turned into a file layout. Combined with Claude Code's 5,000-token re-injection cap and its "keeps the start of the file" truncation, it also tells you how to write a skill: the essential instructions first, the reference material in separate files.
The Model Context Protocol's 2026-07-28 tools spec lets a tool return a resource_link instead of the content itself: "a URI that can be subscribed to or fetched by the client". A server that returns a link to a 40,000-line log instead of the log has handed the decision about what enters the window back to the client. The resources spec says resources are "application-driven, with host applications determining how to incorporate context based on their needs", from explicit user selection to automatic inclusion. The protocol doesn't decide your budget; it gives the host the means to have one.
The OpenSSF guide to instructions for AI code assistants makes the same argument from the security side. Instructions "should be kept concise, specific, and actionable", and "the goal is to influence the AI's behaviour without overwhelming it". The guide says its recommendations are "based on expert opinion and various recommendations in the literature", so it's guidance from a standards body rather than a measurement, but it's notable that the advice converges.
None of this is evidence that budgeting works. It's evidence that the people who build the formats believe the effect is real enough to design around, and it's where your levers are.
You can measure the curve on your own transcripts
Since nobody has published a degradation curve for coding-agent transcripts, the most useful thing I can offer is a method for producing one. I haven't run it; what follows is the design and a harness, and I'm not reporting any results.
The idea is a needle test where the haystack is real agent work. Take a session from your own repository and keep its genuine tool output: file reads, test runs, grep results, diffs. Plant one project-specific fact that the model can't guess, such as a non-default test command ("integration tests run with make itest SHARD=3, never npm test"). Then ask for a task that only goes right if the model uses the fact.
The grid has three variables:
- Depth: plant the fact at 10%, 50% and 90% of the way through the transcript. This tests the Lost in the Middle position effect on interleaved tool output.
- Fill: pad the session to 25%, 50% and 75% of your auto-compact window with more genuine output from the same repository. This tests the length effect.
- Compaction: run every cell once as is, and once after compacting. In Claude Code that's
/compactin an interactive session; on the API, use your own summarization step. This separates position and length effects from summarization loss, which the survival table says will hit a fact that arrived through conversation.
Run at least 10 trials per cell, and preferably 20, and record two things: whether the task used the fact (a hit) and the input token count. Ten trials give wide intervals. Using the Wilson interval from the earlier piece in this series, 9 hits out of 10 is 0.60 to 0.98 and 4 out of 10 is 0.17 to 0.69, so even a 90%-versus-40% gap still overlaps at 10 trials; at 20 trials, 18 against 8 gives 0.70 to 0.97 against 0.22 to 0.61, and it doesn't. Keep the model ID and harness version fixed and written down for the whole grid.
Here's the core of a harness that does the API side of that, without compaction. It replays captured tool output as conversation turns, plants the fact at a chosen depth, and scores a single answer. Its task is the easiest version, a direct question about the fact; a request like "run the integration tests for this change" is the harder and more realistic variant, and needs a grader that checks the command the model proposes:
import Anthropic from '@anthropic-ai/sdk';
import { readFile } from 'node:fs/promises';
const client = new Anthropic();
const model = process.env.MODEL!; // e.g. the exact ID you run daily
// transcript.json: an array of real tool outputs you captured,
// e.g. [{ "cmd": "cat src/db.ts", "output": "..." }, ...]
type Step = { cmd: string; output: string };
const FACT =
'Team rule: integration tests run with `make itest SHARD=3`, ' +
'never `npm test`.';
const TASK =
'What exact command should I run for the integration tests? ' +
'Reply with the command only.';
function build(steps: Step[], depth: number): Anthropic.MessageParam[] {
const at = Math.round(steps.length * depth);
const turns: Anthropic.MessageParam[] = [];
steps.forEach((s, i) => {
const note = i === at ? `\n\n${FACT}` : '';
turns.push({ role: 'user', content: `$ ${s.cmd}\n${s.output}${note}` });
turns.push({ role: 'assistant', content: 'Noted.' });
});
turns.push({ role: 'user', content: TASK });
return turns;
}
const steps: Step[] = JSON.parse(await readFile('transcript.json', 'utf8'));
for (const depth of [0.1, 0.5, 0.9]) {
const messages = build(steps, depth);
const { input_tokens } = await client.messages.countTokens({
model,
messages,
});
let hits = 0;
for (let trial = 0; trial < 10; trial++) {
const res = await client.messages.create({
model,
max_tokens: 200,
messages,
});
const text = res.content.map((b) => (b.type === 'text' ? b.text : ''));
if (text.join('').includes('make itest SHARD=3')) hits++;
}
console.log({ depth, input_tokens, hits, trials: 10 });
}It runs on Node 20 or later as an ES module (for the top-level await), with the SDK installed and ANTHROPIC_API_KEY and MODEL set. countTokens takes the same message list as a create call; the docs call its result an estimate that can differ from the actual input "by a small amount", so log res.usage.input_tokens from a real call if you need the exact figure. I checked that the script type-checks against SDK 0.135.0 and runs end to end against a local stub of the API; I haven't run it against a real model. To vary fill, cut or extend transcript.json; to vary depth beyond three points, change the array.
The harness has limits. Replaying tool output as user turns with a stock "Noted." reply is an approximation of a transcript, not a replay of one; a real session has tool-use blocks, the model's own reasoning, and its own earlier mistakes, and the Chroma result about coherent versus shuffled haystacks suggests structure matters. The closest you can get to the real thing is to run the grid inside Claude Code itself: build each session interactively, plant the fact in a message at the right point, and ask the question at the end, with and without /compact. That's slower, but it measures the system you actually use, harness included.
Keep durable rules on disk and exploration out of the main thread
These recommendations are scoped to the evidence above. Where a practice rests on a heuristic, I say so.
If a rule must survive a long session, put it in the project-root CLAUDE.md or an unscoped rule, not a path-scoped rule or a nested CLAUDE.md. The context window page is explicit: root files and unscoped rules are re-injected from disk after compaction, while path-scoped rules and nested CLAUDE.md files load into message history, "so compaction summarizes them away with everything else" until a matching file is touched again. The page's own advice is to "drop the paths: frontmatter or move it to the project-root CLAUDE.md". Here's the fragile version, then the fix:
---
paths:
- "db/migrations/**/*.sql"
---
<!-- Fragile in long sessions: this loads when Claude reads or edits
a migration, and the next compaction summarizes it away. Until a
migration file is touched again, schema decisions run without it. -->
- Every migration must be reversible: write the down step.<!-- Fixed: an unscoped rule in the root file is re-read from disk
after every compaction. Keep only rules that must always hold. -->
## Database
- Every migration in db/migrations must be reversible: write the
down step.The cost of the fix is that the rule is now in context in every session, including the ones that never touch the database. That's the budget trade in miniature. The memory docs list the three ways an instruction goes missing after /compact: it was given only in conversation, it lives in a nested CLAUDE.md that hasn't reloaded, or it's a path-scoped rule that hasn't matched a file since. There's also a middle option between "always loaded" and "loaded on demand": the hooks guide shows a SessionStart hook with a compact matcher that re-injects text after every compaction:
{
"hooks": {
"SessionStart": [
{
"matcher": "compact",
"hooks": [
{
"type": "command",
"command": "cat \"$CLAUDE_PROJECT_DIR\"/.claude/after-compact.md"
}
]
}
]
}
}Send high-volume reads to subagents, and keep the main thread for decisions. The best-practices guide, the context engineering post and the subagent docs all recommend this, and it applies the strongest finding (irrelevant context hurts) at the point where most irrelevant context enters a session: exploration. Restate any rule the subagent must follow in your delegation prompt, which is the docs' own advice: a non-fork subagent won't see your conversation, and the built-in Explore and Plan agents start without your CLAUDE.md files. Don't use subagents for work that needs back-and-forth with you; the docs say so, and the duplicated exploration costs real tokens.
/clear between unrelated tasks, every time. This has the clearest support and the smallest downside. If you're nervous about losing state, write it to a file first; the context engineering post's "structured note-taking" pattern exists for this.
Treat "clear after two failed corrections" as a starting point, not a rule. The reasoning is good, the number isn't measured. If you're on a newer model and sessions recover, adjust.
Lower the auto-compact window if your sessions routinely run past a few hundred thousand tokens, and compact with a focus before a long new task. The papers say degradation starts well below a million tokens, and a 1M window postpones compaction without removing that slope. I'd rather compact at a size I chose, with /compact focus on ..., than at 967K with a summary that has to guess what mattered; the context window page makes the same point, that a focused compact "keeps what you choose instead of what the automatic pass guesses is important". I can't give you a measured best threshold; the method above is how you'd find yours.
Put the important part of every skill first. Re-injection truncates from the end at 5,000 tokens, and the Agent Skills spec already wants the body under 5,000 tokens and 500 lines.
If you build your own agent, clear old tool results before you summarize, and gate it on a minimum. The API's context editing does the former; clear_at_least keeps you from trading the prompt cache for a few hundred tokens.
Re-test any context workaround when the model changes. The harness design post's resets were essential for one model and unnecessary for the next. Whatever you add to fight context rot is an assumption with an expiry date.
What we don't know
How fast accuracy decays in coding-agent transcripts. I found no peer-reviewed study that measures it. Lost in the Middle, NoLiMa and RULER use question answering and synthetic retrieval. LongCodeBench, a preprint, uses static repository dumps with different instances in each length bucket. Chroma's LongMemEval comparison, a vendor technical report, uses conversations but not tool use. A study that runs a fixed coding task at controlled transcript lengths on current models, with interleaved tool output, would settle it; the method above is a sketch of one.
Whether the middle-position dip holds for interleaved tool output. Lost in the Middle found it for passages, and NoLiMa saw it at 32K for book text. Agent transcripts have a different structure: many short, dissimilar chunks, and the Chroma report found that structure itself changes results. Nobody has published the position curve for a transcript.
Whether current 1M-context frontier models show the same shape at matching lengths. The longest systematic results I could verify stop at 128K for most models (RULER, and NoLiMa's appendix) or use models a generation or more old (LongCodeBench's Claude 3.5 Sonnet). Vendor statements that degradation "emerges across all models" are plausible and consistent with the papers, but they come without published curves for the models you run today.
Where the "two failed corrections" threshold came from. The Claude Code best-practices guide states it without a citation, and I couldn't find a measurement behind it. A comparison of recovery rates after one, two and three corrections would answer it.
How much compaction loses, and what. Anthropic says compaction can drop "subtle but critical context", and the changelog shows compaction bugs being fixed as recently as October 2026, but there's no published measurement of what fraction of facts survive a Claude Code summary, or whether "Compact Instructions" reliably changes that. The compaction arm of the demo grid would give you a number for your own setup.
What RULER's "only half" sentence counts. The abstract says half of the 17 models are satisfactory at 32K; the v3 table, as I read it, has 10. The authors' repository or a corrected revision would settle which is right. It doesn't change the direction of the result.
Whether resets beat compaction on current models. The harness design post's evidence is one engineer's experience across two model versions. No controlled comparison of context resets with structured handoffs against automatic compaction has been published that I could find.
References
- Effective context engineering for AI agents. Anthropic>. Accessed .
- Best practices for Claude Code. Anthropic>. Accessed .
- How Claude Code works. Anthropic>. Accessed .
- Explore the context window. Anthropic>. Accessed .
- Model configuration (Claude Code). Anthropic>. Accessed .
- Create custom subagents. Anthropic>. Accessed .
- Manage Claude's memory (CLAUDE.md and rules). Anthropic>. Accessed .
- Hooks guide. Anthropic>. Accessed .
- Settings reference (Claude Code). Anthropic>. Accessed .
- Claude Code changelog. Anthropic>. Accessed .
- Context editing. Anthropic>. Accessed .
- Token counting. Anthropic>. Accessed .
- Harness design for long-running application development. Anthropic>. Accessed .
- Lost in the Middle: How Language Models Use Long Contexts (arXiv v3). Liu et al., arXiv>. Accessed .
- Lost in the Middle: How Language Models Use Long Contexts (TACL vol. 12, 2024, pp. 157-173). ACL Anthology / MIT Press>. Accessed .
- NoLiMa: Long-Context Evaluation Beyond Literal Matching (ICML 2025). Modarressi et al., arXiv>. Accessed .
- NoLiMa full text, v3 (Tables 3 and 10). Modarressi et al., arXiv>. Accessed .
- RULER: What's the Real Context Size of Your Long-Context Language Models? (COLM 2024). Hsieh et al., arXiv>. Accessed .
- RULER full text, v3 (Table 3). Hsieh et al., arXiv>. Accessed .
- LongCodeBench: Evaluating Coding LLMs at 1M Context Windows (preprint, v3). Rando et al., arXiv>. Accessed .
- Context Rot: How Increasing Input Tokens Impacts LLM Performance (technical report). Chroma>. Accessed .
- Agent Skills specification. Agent Skills>. Accessed .
- Security-Focused Guide for AI Code Assistant Instructions. OpenSSF Best Practices and AI/ML Working Groups>. Accessed .
- Tools - Model Context Protocol specification 2026-07-28. Model Context Protocol>. Accessed .
- Resources - Model Context Protocol specification 2026-07-28. Model Context Protocol>. Accessed .
- @anthropic-ai/sdk package metadata. npm registry>. Accessed .