Amazon Swapped Sonnet 4.6 for a Decision Model: What It Does and Doesn't Replace

Last updated: November 11, 2026

Amazon engineers replaced Claude Sonnet 4.6 with a small decision model called Strands Decider on an internal ticket-labeling job, and the Strands team reports the same quality at 98% lower cost. That is a large number, and it is also a narrow result: it applies to a closed choice among 20 labels, not to agents in general. If you are searching for model routing and cheaper alternatives to a frontier model on every call, this is the clearest worked example so far of where the cheapest tier is not a smaller LLM at all.

What the primary source says#

The Strands Agents write-up (published November 9, 2026, by two Amazon engineers and the Strands team) describes a ticket system that sorted tickets into 20 categories using Sonnet 4.6, a long system prompt and JSON output. They swapped in the E4B build of Strands Decider, a Gemma 4-based decision model, served on a SageMaker ml.g5.xlarge instance.

The numbers below are all vendor-reported by the Strands team, for one internal workload:

Sonnet was priced at $3 per million input and $15 per million output tokens; the decision model at $1.408 per instance-hour. At roughly 640 ms per request one instance handles about 5,600 tickets an hour, which is $250 per million tickets at full use. They budget about $500 per million with burst headroom, hence "about 44 times cheaper." Production Lambda duration fell from an average of 3,312 ms to 748 ms.

Two details matter if you try to reproduce this. The labels' descriptions were longer than the model's 4,096-token window, so each of the 20 labels was rewritten as a one- or two-sentence criterion and sent as the options of one choice question plus a "none" option. And on long tickets the model drops the oldest comments to fit. The authors also found that JSON key order changed output probabilities by up to 0.19, so they now send sorted keys.

What a decision model actually is#

Decision-only models are a new class, which we covered in TypeSafe Jev. The launch post for Strands Decider 2B (October 1, 2026) explains the design: take a pretrained LLM, remove the language-model head so it cannot generate text, and add a small pointer head that scores each offered option. You get a probability per option and a reliability score, which frontier LLM APIs do not give you. The authors are explicit about the trade: it is "significantly worse at solving complex problems than reasoning models" and unsuited to coding, chatbots and summarization.

Its pitch is the rote decisions inside an agent: model routing, tool selection, guardrails and policy classification. The 2B model's median decision latency was around 115 ms on an RTX 3090 (vendor-reported).

The angle: the cheap tier is often not an LLM#

Most of the cost conversation in agents is about picking a cheaper LLM for subagents, for example Haiku 5.5 versus GPT-6 Luna or Meta's $0.10 Muse Ember. This case argues for a tier below that: if the output is a pick from a known list, you are paying for generation you do not use. The 170 output tokens per ticket and the 6,500-token prompt were mostly there to coax a frontier model into a rigid format.

Who wins: teams with high-volume, closed-choice steps - triage, intent routing, label assignment, "is this tool call grounded" checks. Who does not: anything with open-ended output, a fast-changing label set, or context that will not fit in a few thousand tokens. The authors say the same.

What to do Monday#

Skip the model for a moment and copy the rollout. Amazon read a three-way setting from AWS AppConfig on every message:

• BEDROCK: the LLM labels everything, the new model is not called.
• DUAL: both run in parallel, the LLM result is applied, and the two are compared.
• DECIDER: only the new model runs.

Their own summary: running both on live traffic and judging every disagreement taught them more than an offline eval, and rolling back was a 45-second config change. The same pattern works for any cheap-model swap, as covered in our guide to model routing as an orchestration layer and automatic routing to control spend.

To try the model yourself, the launch post gives a one-line install and a verified CLI example:

pip install strands-decider

strands-decider ask StrandsAgents/strands-decider-2B-hobson-v19 \
  --state "Help! My payouts have been failing for 3 days! " \
  --choice "Which team should handle this?=billing,sales,retail"

The documented output ranks billing highest at confidence 0.768. Treat the 2B model as a starting point; Amazon's production result used the larger E4B build.

What people are actually saying#

The Hacker News thread (about 600 points) leans skeptical in an informed way. Several commenters pointed out that classification never needed an LLM in the first place, and that a TF-IDF plus SVM pipeline is fast and cheap, though one noted such a system fails in ways that are unfixable inside it. Others countered that LLM labeling is still very useful for proofs of concept, for generating training data and for active-learning loops. One commenter made the sharpest point: expensive models can be deployed instantly and used to generate training data until you swap in your own cheap model, and the quoted training time ignores the data collection and labeling work. A related thread of replies argued that small specialized models beating a general one is the Bitter Lesson working as intended rather than against it. The counter-case to take seriously: this is one internal workload reported by the vendor, with no public eval set.

FAQ#

What is a decision model?#

A model that takes state and a question with fixed options and returns a probability per option plus a confidence score. It cannot write free text.

How much cheaper was it?#

Vendor-reported: about $0.0005 per ticket versus $0.0222 for Sonnet 4.6 at list prices, roughly 44x.

Can it replace my agent's main model?#

No. It fits closed choices at high volume. Open-ended tasks still need an LLM.

Sources#

Continue Reading#

• TypeSafe Jev: the first decision-only model class - the model class Strands Decider belongs to
• AI model routing as an orchestration layer - how routing decisions get made in an agent stack
• Factory router and automatic model routing - routing to control spend
• Cheap subagents, visible work - where the low-cost tier belongs in a harness
• Haiku 5.5 or GPT-6 Luna as a subagent - the LLM-tier alternative to a decision model
• Meta Muse Ember - another cheap-worker option and what separates it
• AI API Cost Calculator - estimate per-request and monthly cost