Flat Score, Amplified Failures: The Error Budget Hides Tool-Calling Damage in Quantized Agents

Authors: Jiwon Jang, Kisu Yang, Heuiseok Lim, Hyunwoo Park

arXiv ID: 2607.27275

Problem: Post-training quantization to 4-bit weights is widely reported as nearly lossless, and that claim now matters most for multi-turn, tool-calling agents, the workloads that actually get shipped on quantized local models. Aggregate benchmark scores do not say whether quantization damages the agentic process, because a task-reward metric can absorb a few extra errors before a run is judged failed.

Key Methodology:

Eight cells: two open-weight model families in dense and MoE variants, two domains (telecom, retail), 456 episodes each, at 16-, 8- and 4-bit weights
Standard task-reward metric plus per-channel error-rate analysis over the full trajectory logs
Equivalence testing on the noisiest cell and a shrinking-error-budget experiment to test the masking account

Key Results:

On the standard metric quantization looks free: no cell shows a score change that survives multiple-comparison correction, and in the cell with the largest process damage equivalence testing bounds the change within +/-7.5 points
The process tells a different story: quantization amplifies the failure the model already exhibited at full precision (tool-name hallucination in telecom, same directional trend in retail entity errors) by up to 2.5x in volume (+17.6 points per task) while creating essentially no new failure types (rank correlation >= 0.94 across precisions, 0.18% novel events)
The score stays flat because the benchmark's ten-error budget absorbs the extra failures; shrinking the budget to two errors re-exposes a score gap of 17 points, and only in the one cell where quantization added error volume
A targeted error-repair prompt removes the damage exactly and only where it lives, across five telecom models at every precision

What it means for developers: "Lossless" quantization claims are scoped to aggregate pass rates, not to agent reliability. Anyone running quantized models in tool-calling or agent loops should measure per-channel error rates (which failure type the model already had) and success under a tight error budget, since those are the diagnostics that expose precision loss. The failure set is identical across precisions, so quantization is not a new failure class: it is an amplifier of existing ones, and a targeted prompt repair on the amplified channel can recover most of the damage. Expect benchmark authors to start reporting error-budget curves next to task reward.

Paper: arXiv:2607.27275