Why your AI instruction files are making the output worse

You wrote a CLAUDE.md around 6 months ago. Since then, every time an output came back wrong, you added a rule. Then another one. Now the file is 3,000 words, nobody has deleted a line from it, and you have no idea which parts are doing any work.

I checked the research behind this after watching Boris Cherny, the creator of Claude Code, explain his team's method. When Opus 5 shipped in July 2026, more than 80% of Claude Code's system prompt was deleted, and the model performed better without the scaffolding. The studies underneath that decision are more useful than the headline, and one number I gave in the video was wrong. The correction is at the bottom.

What an ablation is, and why it applies to your prompt file

Ablation is a research discipline, not a tidy-up. Delete the entire system prompt, use the product, watch where the model stumbles, and add instructions back one line at a time, only when the model repeatedly fails on the same specific issue. Never add a line speculatively, because that instruction gets read on every single use.

Cherny's advice for people who are not building agents is blunter. In the Y Combinator interview released on 28 July 2026, he suggested deleting your CLAUDE.md files, custom skills and hooks every 6 months and rebuilding from scratch.

Here is a detail the video did not have. Each team at Anthropic keeps a CLAUDE.md in git, and as of January 2026 Cherny said theirs runs at about 2,500 tokens. The person who built the tool keeps his instruction file small. Most of us keep ours large because we never deleted anything.

Worth sharpening one thing from the video: the 80% figure refers to the system prompt, not the whole codebase.

Why long instruction files lower accuracy

The cost is not only tokens. In July 2025, Chroma Research published a study by Kelly Hong, Anton Troynikov and Jeff Huber testing 18 frontier models, and found accuracy drops well before the context window fills. A 200,000 token window can show serious accuracy loss at 50,000 tokens of input. The models tested included GPT-4.1, Claude Opus 4 and Gemini 2.5 Pro, so this is not a story about weak old models.

Position matters as much as length. Liu et al., published through Stanford in TACL 2024, found a U-shaped curve: models attend well to the start and end of the context and poorly to the middle, with accuracy dropping by more than 30% when the relevant document sat in positions 5 to 15 rather than first or last.

Read that against your own file. The rule you care about most, the one you added in month 3 about spacing, is sitting in the middle of a 3,000 word document. That is the worst seat in the room.

What happens when you hand a model 15 rules at once

Designers do not write prompts. We write red lines. Spacing, hex values, states, the order of operations. We do it to protect taste, and it feels responsible.

Every red line is a constraint, and constraints do not stack for free. The RECAST paper from May 2025 found that models struggle once explicit requirements pass roughly 10. The MOSAIC benchmark, published January 2026, shows pronounced drops in prompt-level compliance beyond about 15 constraints.

And the SEQUOR benchmark, May 2026, measured what happens over a longer session: with a single constraint, accuracy fell by more than 11% between the first and last turn. With several constraints at once, the drop passed 38%. When constraints were introduced one at a time as the conversation went on, losses went above 40%.

That last number describes exactly how most of us work. We add a rule, then another, mid session, because the last output was off.

The replacement is 3 parts. Describe the task. Describe the guardrails. Describe the exit criteria. Then leave it alone and come back.

Verification is the part designers already train for

Cherny's own framing is that the skill now is giving the model a task that looks slightly too hard, then making it possible for the model to verify its own work, and that verification is the single thing most people get wrong.

There is a research reason this works. Checking an answer is an easier job than producing one, which is why verifier-based systems punch above their weight. The Weaver paper from Stanford, June 2025, got Llama 3.3 70B up to o3-mini level accuracy, 87.7% on average, by pairing a cheaper generator with an ensemble of verifiers.

Designers have a real advantage here, and it is measurable. Eye-tracking work across expert domains shows medical diagnosticians fixate on abnormalities faster than novices and dwell longer on the odd region, and fingerprint examiners search efficiently for the features that carry information while showing no such advantage on scrambled images. Expertise shows up as faster detection of the thing that is wrong, and only inside your own domain.

You have done thousands of reps comparing 2 states of the same screen. That is the skill the model needs borrowed.

The protocol, step by step

1. Archive first. Copy the whole folder, date it, and leave it alone. You are running an experiment, not deleting 6 months of work.
2. Hand the file to the newest model and ask 2 questions. Which lines are foundational, and which ones are correcting behaviour you already have? Then ask for the token count before and after. Mine came back between 40% and 70% shorter.
3. Delete. Work a full week on the short version, on real jobs, not test prompts.
4. Add back one line at a time, only where the same failure appears twice.
5. Write exit criteria into every brief. Compare the output against the reference screenshot. Check every spacing value against the scale. Check every colour against a variable and flag anything hard coded. Screenshot both versions and report what changed.
6. Run the comparison yourself. One small UI task, 2 sessions. Session A gets 10 ordered steps. Session B gets task, guardrails, exit criteria. Look at the 2 results side by side.
7. Diary it. If nothing broke in that week, those lines were never doing anything.

One correction from the video

I said a codebase was rewritten from one language to another in 11 weeks. It was 11 days. Cherny described Opus 5 rewriting the Bun runtime from Zig to Rust in 11 days using thousands of parallel agents.

The 2 week number belongs to a different example. His own prompt asked Claude to rewrite the Electron desktop app in Swift, run the Electron version in a Mac virtual machine, screenshot it, compare pixel by pixel, and not stop until it matched. At the time of the interview it had been running 14 or 15 days and was still going.

I would rather correct that here than let it travel.

The part I keep thinking about is not the deleting. It is what happens if I delete a week's worth of rules and nothing at all breaks. That would mean I have been carrying instructions I never tested, feeling organised about it, and paying for them on every single prompt. I would rather find that out on purpose.