Inference
Context Rot
The decline in LLM output quality as the number of input tokens grows, even on simple tasks that fit well inside the context window.
In depth
The decline in LLM output quality as the number of input tokens grows, even on simple tasks that fit well inside the context window. Chroma's 2025 report popularized the term after testing 18 models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, and found performance grew less reliable with length, with distractors making it worse. The practical lesson is that a large window is capacity, not a guarantee, so curating what goes into context still matters.
Primary source: research.trychroma.com
Example
In practice, developers reach for Context Rot when they need the capability described above as part of an AI feature or workflow.
Go deeper at Developers Digest
Hands-on guides, comparisons, and tutorials that cover Inference.
FAQ
Common questions
What is Context Rot?
The decline in LLM output quality as the number of input tokens grows, even on simple tasks that fit well inside the context window.
Why does Context Rot matter for AI developers?
Context Rot sits in the Inference part of the AI stack. Understanding it helps you make better decisions when building, debugging, and shipping AI features.
Where can I learn more about Context Rot?
Developers Digest publishes tutorials and videos that cover Inference topics including Context Rot. Check the blog and YouTube channel for hands-on walkthroughs.
Related
Related terms
Inference
Input vs Output Token Pricing
The split pricing model where LLM APIs charge one rate per million tokens you send (input) and a higher rate per million tokens the model generates (output).
Inference
Context Window
The maximum amount of text (measured in tokens) that a model can process in a single request.
Inference
Chat Template
The formatting rule that turns a list of role and content messages into the exact token sequence a chat model was trained on.
Put this concept to work
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever