AgentChaos: Fault Injection Shows Agent Robustness Is a Systems Problem, Not a Model Problem

Every agent system depends on LLM APIs for every single response, and those APIs fail in the real world: 5xx errors, token-limit truncation, garbled output, malformed tool calls. A new paper from Sun Yat-sen University, Singapore Management University, and Monash University argues that most teams discover how their agents handle these failures the hard way - in production. AgentChaos is their answer: a chaos engineering framework that injects faults into live agent systems at the HTTP layer, without touching source code.

The paper (arXiv 2608.06790, accepted at ASE 2026) is the first systematic, runtime fault injection study of agent systems, and its headline result is uncomfortable: every system tested degrades under fault injection, pass@1 drops by up to 50 percentage points, and robustness is determined by system architecture, not model choice.

What AgentChaos does#

Because all agent systems reach LLMs through the same HTTP interface, AgentChaos injects faults at that shared layer. A proxy sits between the agent and the model API, intercepts responses at runtime, and modifies them before they reach the agent. No source code changes, no offline patching of prompts, no reimplementation.

The framework defines a fault taxonomy with three families across two target fields (content and tool call):

• Crash faults: server errors and timeouts, the visible failures
• Omission faults: truncated responses and empty content, which return valid HTTP 200 responses and bypass most error handling
• Value faults: corrupted content and schema violations in tool call arguments

Each injection is verified against its trigger condition, and tasks where the fault never actually fired are filtered out. That filter matters: earlier approaches that skip it understate fault impact by mixing unaffected tasks into the denominator.

The evaluation#

AgentChaos ran 65 fault configurations across five agent systems (AutoGen, MAD, MapCoder, EvoMAC, Mini-SE) and seven benchmarks, from HumanEval and MBPP to MMLU-Pro, MATH-500, and SWE-bench Pro. Each system was reimplemented on Google's ADK with unified tool interfaces, and all four backbone models were swapped in: Claude Sonnet 4.5, GPT-5.2, DeepSeek-V3.2, and Seed-1.8.

Two findings stand out.

First, every system degrades, with pass@1 dropping up to 50 percentage points under injection. Second - the paper's sharpest result - the robustness ranking of the systems is nearly identical no matter which LLM is behind them. Swap GPT-5.2 for Claude Sonnet 4.5 and the relative order of the systems does not change. Robustness is a property of how the system processes responses, not which model generates them. Replacing the model alone is unlikely to fix these weaknesses.

Severe is not the same as harmful#

The taxonomy exposes a trap: the most severe-looking faults are not the most damaging. Omission faults - truncation and empty responses - degrade performance about as much as crashes on most systems, while looking far less alarming. On MAD, empty content caused a 38.46% pass@1 drop, close to the 37.5% of explicit error content and well above the 22.33% of timeouts. Crash faults trigger error handling and automatic retries; omission faults come back as valid HTTP 200s and slip through silently.

The diagnosis problem is worse. Omission faults are also the hardest to detect after the fact: rule-based diagnosis identifies truncation with only 4.3% accuracy, and LLM-based diagnosis reaches just 34.41%. A truncated output looks like weak model output in execution traces, so developers misattribute the failure to model capability and upgrade the model instead of fixing the fault handling. Overall, existing fault diagnosis methods score below 53% on fault type and below 56% on fault step.

Architecture decides#

The architecture results give agent builders something actionable. MapCoder, the pipeline system evaluated, is the most vulnerable: a single fault at its first stage drops pass@1 by up to 83.87%, because each stage consumes the previous stage's output and propagates the fault downstream. The iterative systems were the most robust - later rounds can observe and correct errors from earlier ones. The mechanism is structural, which means it should carry over to other systems with the same shape.

What developers should take from this#

The paper's practical advice maps directly onto production agent code:

• Validate every response, not just errors. Check finish_reason, verify code syntax completeness, and confirm tool call arguments match the expected schema after every call. Omission faults pass 200-level checks by design.
• Log structured metadata per call: token usage relative to the limit, finish_reason, and response length. Truncation becomes detectable in later analysis instead of looking like a weak model.
• Treat pipelines as chains of trust. Add stage-level output validation in pipeline systems, and consider iterative refinement so later stages can recover.
• Stop blaming the model. If a swap from one frontier model to another does not change your failure profile, the fault handling in your code is the problem. This is the agent reliability cliff in measured form: the success rate of a multi-step chain collapses long before the model itself becomes the bottleneck.

AgentChaos fits a growing theme in agent research: evals that freeze the repository measure the model, but production agents need tests that break the runtime. The ORCA-bench result made the same point from the SRE side - agents look better in frozen environments than in live systems. AgentChaos supplies the fault injection half of that story, and its proxy-layer design means the framework can be dropped into a staging environment without forking the agent framework you use. The code is open on GitHub.

The honest takeaway: your agent's failure handling is a systems property, it degrades under realistic API faults, and you will not discover that by benchmarking against a clean backend. Chaos engineering is standard practice for distributed systems; AgentChaos is the argument that agent systems have reached the same stage of maturity.

Continue Reading#

• Agentic AI Reliability Is a Systems Problem
• The Agent Reliability Cliff: Why Your 10-Step Chain Only Succeeds 20% of the Time
• Agent Evals Need Baseline Receipts
• 12-Factor Agents: Production Principles for Reliable AI Agents
• Claude API Reliability: Error Handling Best Practices

Sources#

• AgentChaos: Chaos Engineering for Agent Systems via Programmatic Fault Injection - arXiv
• AgentChaos source code - GitHub