How to Cut Your LLM API Bill in 2026: Prompt Caching, Batch and Flex Tiers Compared Across 15 Providers

Last updated: October 18, 2026

Short answer: cache first, batch second, and only count on both together where the provider says they stack. On a workload where every request repeats a 20,000-token system prompt, caching alone cut the daily bill by roughly 78-80% on all four models we priced. Batch takes another 50% off at most providers, but Moonshot gives 40%, xAI gives 20% on four listed models, and Together discounts only listed models. Anthropic, Fireworks and xAI say batch and caching stack. Vertex AI, Groq and Alibaba Cloud say they do not, Gemini bills batch cache hits at the normal cache rate, and the other providers do not say either way.

Every number below comes from two datasets we keep: the cost levers directory (caching, batch and service-tier mechanics for 15 providers, each with a verbatim quote from the provider's own docs) and the per-model prices behind the models pricing table. Values the directory marks as derived are called derived here too.

The worked example#

The workload: 10,000 requests a day, each with the same 20,000-token system prompt, 500 tokens of fresh user input and 500 tokens of output. Per day that is 200M shared prompt tokens, 5M fresh input tokens and 5M output tokens.

Assumptions, labelled so you can swap in your own:

• A1. Even traffic. Requests are spread over 24 hours (about seven a minute), so a cache is never idle long enough to expire.
• A2. One write per TTL window. We charge a full 20,000-token cache write once per TTL window: every 5 minutes on Anthropic (288 writes a day) and every 30 minutes on OpenAI (48 a day). This is deliberately conservative. Both providers say a hit refreshes the cache lifetime, so steady traffic could write far less often.
• A3. All remaining requests hit. Automatic caches (Gemini implicit, xAI) do not guarantee hits, so treat their cached figures as best case.
• A4. Gemini uses an explicit cache kept for 24 hours, billed at the storage rate of $0.50 per 1M tokens per hour (a promotional rate through December 31, 2026). Our data records no per-write fee for Gemini, so the one-time cache creation is not counted.
• A5. Short-context rates only. 20,500 input tokens is under GPT-6 Luna's 272K and Grok 4.3's 200K long-context thresholds. No regional surcharges, tool fees or Fast mode.

Rates used, per 1M tokens, from our model facts:

Gemini 3.8 Flash prices apply through December 31, 2026 and double from January 1, 2027, per Google's pricing page.

The arithmetic#

Notes on the rows that need one:

• Sonnet 5.5 batch + cache applies Anthropic's cache multipliers to the batch base price: 1.25x of $1 for writes and 0.05x of $1 for reads. Anthropic's caching docs say "These multipliers stack with other pricing modifiers, including the Batch API discount and data residency." With the 1-hour TTL instead (24 writes a day at 2x input), the cached figure is $81.87 and batch plus cache is $40.94.
• GPT-6 Luna batch + cache is left blank because OpenAI's batch docs do not say. Flex does say: "Tokens are priced at Batch API rates, with additional discounts from prompt caching." If every Flex column is half of Standard, as it is on the pricing page, Flex plus cache comes to about $2.56 a day. That figure is derived, and it assumes the cache write is halved too, which our data does not state directly.
• Gemini batch + cache follows Google's rule that a batch cache hit pays "the standard context caching rates", so the 200M cached tokens stay at $0.075. Our data does not say whether an explicit cache can be used from a batch job; with implicit caching and every request hitting, the figure is $26.25 with no storage line.
• Grok 4.3 batch + cache uses $0.16, which is the $0.20 cached rate less 20%. That is derived from xAI's statement that the batch discount "applies to all token types"; our model facts do not list a batch cached rate.

What the table says#

Three things fall out. Caching does more than batch on any workload dominated by a repeated prefix, because it discounts 200M of the 210M daily tokens by 84-95% (95% on Sonnet 5.5, whose cache read is 0.05x; 84% on Grok 4.3). Batch is a flat cut on everything, so it matters most when output or fresh input dominates. And the stacking rules decide the last step: on Sonnet 5.5, batch halves the cached bill again, while on Gemini batch only touches the 10M uncached tokens.

Lever 1: prompt caching#

Automatic or explicit. Eight of the 15 providers cache automatically with no code change: Mistral, xAI, DeepSeek, Moonshot, Z.ai, Groq, Together and Fireworks. Six offer both modes: OpenAI, Google, Vertex AI, Bedrock, Azure and Alibaba Cloud. Anthropic is the only one where nothing is cached until you add cache_control, either once at the top level for automatic breakpoint placement or on individual blocks. The difference matters for the bill: automatic caches are usually best effort. Alibaba says its implicit cache "cannot be disabled" but hits are not guaranteed, Bedrock calls implicit caching "best effort", and DeepSeek says its cache "does not guarantee a 100% cache hit rate". Explicit caches are how you get a predictable rate.

TTLs. Anthropic's default is 5 minutes, "refreshed for no additional cost each time the cached content is used", with a 1-hour option. OpenAI's GPT-5.6 and later keep a prefix "for 30 minutes after its most recent write or reuse". Groq's cache "expires after 2 hours without use". Gemini's explicit cache "defaults to 1 hour" with "no minimum or maximum bounds". Alibaba's explicit cache lasts 5 minutes and resets on a hit. At the loose end, DeepSeek clears unused cache "within a few hours to a few days", Fireworks says "several minutes" up to "several hours", xAI entries "can be evicted at any time", and Mistral does not document a TTL.

Minimum prompt length. Short prompts are never cached, often silently. Mistral starts at 64 tokens, Groq at 128 to 1,024 depending on model, Anthropic at 512 for the Claude 5.x family (up to 4,096 on older models), and OpenAI, Azure and Alibaba at 1,024. Gemini 3.8 Flash needs 4,096, and on Vertex AI the same model needs 6,144 and supports implicit caching only. Anthropic warns that shorter prompts are "processed without caching, and no error is returned."

Write premiums. Anthropic charges 1.25x input for a 5-minute write and 2x for a 1-hour write, and says caching "pays off after one cache read" at 5 minutes. OpenAI charges 1.25x on GPT-5.6 and later and nothing extra before that. Alibaba's explicit writes are "typically 125%" of input. Moonshot's kimi-k3 bills a 5-minute write at the input price and a 1-hour write at 2x (derived from its pricing table), while its K2 models have no write charge. Vertex AI bills cache creation "at the standard input token price", and Groq caching comes with "no additional fees". Gemini's explicit cache replaces the write fee with hourly storage.

One disagreement to know about. Z.ai's caching guide says hits are "usually 50% of standard price", while its pricing table lists cached input at roughly 15-20% of input (a derived ratio). Our Z.ai page treats the table as authoritative. For setup code per provider, see our prompt caching comparison.

Lever 2: batch#

Nine providers give a flat 50% on batch: OpenAI, Anthropic, Google, Vertex AI, Azure, Mistral, Alibaba Cloud, Groq and Fireworks. Bedrock gives 50% on "select foundation models". The exceptions:

• Moonshot: 40%. "Batch API inference costs are 60% of the standard model price", and only kimi-k2.7-code and kimi-k2.6 are supported. See Moonshot's cost levers.
• xAI: 20% on listed models. The discount table lists grok-4.3 and three grok-4.20-0309 variants at 20%, and "Models not listed above have no batch discount." See xAI's cost levers.
• Together: listed models only. The headline is "up to 50% off", but "Models not listed run at standard rates", and the list currently holds only openai/whisper-large-v3. See Together's cost levers.
• DeepSeek and Z.ai have no batch API in the pages we read.

The windows differ more than the discounts. Most are 24 hours, and Anthropic says "most batches completing within 1 hour". Moonshot accepts 12 hours to 7 days, Groq 24 hours to 7 days, Alibaba 24 to 336 hours, and Fireworks lets you pick 12, 24, 48 or 72 hours. Gemini jobs that run past 48 hours expire with no results. Our Claude Batch API guide covers the production side.

Lever 3: Flex and Priority tiers#

Flex is batch pricing for synchronous calls that can wait. OpenAI Flex is "priced at Batch API rates" (50%, derived from the batch guide and the pricing page) and returns a 429 Resource Unavailable that is not charged when capacity runs out. Google's Flex is "a 50% cost reduction" with a 1 to 15 minute latency target and "no server-side fallback". Vertex AI Flex PayGo, Bedrock Flex and Azure Flex (preview) are also 50%. Groq's Flex is the odd one: same price as on-demand, with 10x rate limits, so it is a throughput lever rather than a discount.

Priority goes the other way. Google charges "75-100% more", Bedrock "a 75% premium", Mistral "1.75x", xAI "a 2x premium", and Fireworks between 1.2x and 1.5x on headline models (derived). OpenAI's Fast mode and Anthropic's Fast mode are 2x on the models where they run (2x for the GPT-6 family is derived from the pricing tables, as is Anthropic's), and OpenAI Ultrafast is 6x (derived). Anthropic's Priority Tier "capacity commitments are no longer available for purchase". Vertex AI and Azure say Priority costs more but do not state the multiplier on the pages we read. The useful detail for budgeting: Google, OpenAI, Vertex AI and Azure all say requests downgraded to standard bill at the standard rate, and xAI bills the priority rate only when the response confirms it.

Lever 4: off-peak (DeepSeek)#

DeepSeek is the only provider in the set with time-of-day pricing: "Off-peak rates are half of the peak rates." Peak is 01:00-04:00 and 06:00-10:00 UTC, Monday to Friday, excluding Chinese public holidays; everything else, including weekends, is off-peak. Run the same workload on DeepSeek V4.1 Flash ($0.30 input, $1.20 output, $0.006 cache hit at peak) and it costs $67.50 a day uncached at peak, $8.70 with cache hits at peak, and $4.35 with cache hits off-peak, again assuming every shared prefix hits. There is no batch API, so off-peak scheduling is DeepSeek's batch. More on the schedule in our DeepSeek V4.1 Flash cache and peak hours post.

Does batch stack with caching?#

This is the question the providers answer least consistently.

Yes, three providers say so:

• Anthropic: "Batch API and prompt caching discounts can be combined." (pricing)
• Fireworks: "Batch jobs automatically use prompt caching for additional 50% cost savings on cached tokens." (batch inference)
• xAI: "The batch discount applies to all token types." (batch API)

No, three providers say so:

• Vertex AI: "However, the discounts for cache and batch don't stack. The 90% cache hit discount takes precedence over the batch discount." (batch inference)
• Groq: "Note: The batch discount does not stack with prompt caching discounts. All batch tokens are billed at the 50% batch rate regardless of cache status." (batch)
• Alibaba Cloud: batch "also does not support features such as context cache." (batch interface)

At the cache rate: Gemini API: "If a request in your batch results in a cache hit, you pay the standard context caching rates." (batch API)

Not stated: OpenAI, Bedrock, Azure, Mistral, Moonshot and Together. DeepSeek and Z.ai have no batch to stack. Until a provider says so, price a batch job as if cached tokens get no extra discount.

Checklist#

1. Put the stable part of the prompt first and the variable part last. Every provider here caches a prefix.
2. Check the minimum cacheable length for your model. Below it you pay full price with no error.
3. On Anthropic, add cache_control. Nothing is cached without it.
4. Match the TTL to your traffic gaps. A 5-minute cache on bursty hourly traffic pays the write premium again and again.
5. Use a routing hint where offered (prompt_cache_key on Mistral, x-grok-conv-id on xAI, session affinity on Fireworks) to raise hit rates on automatic caches.
6. Move anything that can wait a day to batch, and check your model is on the discount list at Bedrock, Moonshot, xAI and Together.
7. For calls that can wait minutes but not hours, try Flex at OpenAI, Google, Vertex AI, Bedrock or Azure.
8. Only budget batch plus cache as stacked on Anthropic, Fireworks and xAI.
9. Watch for surcharges: OpenAI adds 10% for regional processing on models released on or after March 5, 2026, and Anthropic's US-only inference is 1.1x on Claude 4.6 and later.
10. Re-check promotional prices. Gemini 3.8 Flash doubles on January 1, 2027.

For a provider-by-provider view, the cost levers directory has a page each, for example Anthropic, OpenAI, Google, Vertex AI, Groq, Fireworks, Alibaba Cloud and DeepSeek. If your prompt is long enough that context limits matter, the context window table shows where long-context pricing starts to bite.

FAQ#

What is the cheapest way to reduce LLM API costs?#

If your requests share a long prefix, prompt caching. In our example it cut the bill by 78-80% on all four models before any other change. Batch is the next step for work that can wait.

Is the batch discount always 50%?#

No. It is 50% at most providers, but Moonshot gives 40%, xAI 20% on four listed models, and Together only on listed models. DeepSeek and Z.ai have no batch API.

Can I combine batch and prompt caching?#

Anthropic, Fireworks and xAI say yes. Vertex AI, Groq and Alibaba Cloud say no. Gemini bills batch cache hits at the standard cache rate. The rest do not say.

Is Flex the same as batch?#

On price, mostly: OpenAI, Google, Vertex AI, Bedrock and Azure price Flex at 50% off. The difference is the shape. Flex is a normal synchronous request that may be slow or rejected, while batch is a job you poll for. Groq's Flex is not a discount at all.

Why did my cache not hit?#

The usual causes are a prompt under the model's minimum, a prefix that changes between requests, a TTL that lapsed between calls, or a best-effort automatic cache that evicted the entry. On Anthropic, a missing cache_control means nothing was cached.

Continue Reading#

• Prompt Caching Across AI Providers: Prices, TTLs and Setup - the setup code for each provider's cache.
• Claude Batch API: Cutting Async Workload Costs In Half - the batch lever in production.
• LLM API Pricing, October 2026: Claude vs OpenAI vs Gemini - the list prices this post discounts from.
• Why Price Per 1M Tokens Is a Misleading Metric for LLM Costs - why cost per task beats cost per token.
• DeepSeek 4.1 Flash as a Daily Coding Model: Cache, Peak Hours and Limits - the off-peak lever in daily use.

Sources#

Discount mechanics come from our cost levers directory, where every quote was string-matched against the live page on October 18, 2026. Per-model prices come from our model facts, read from each provider's pricing page on October 17, 2026. Provider pages quoted:

• Anthropic: prompt caching, pricing, batch processing, fast mode, service tiers
• OpenAI: prompt caching, batch, flex processing, fast mode, ultrafast mode, pricing
• Google Gemini API: caching, batch API, flex inference, priority inference, pricing
• Vertex AI: context cache overview, batch inference, Flex PayGo, Priority PayGo
• Amazon Bedrock: prompt caching, batch inference, service tiers
• Azure OpenAI: prompt caching, batch, flex processing
• Mistral: prompt caching, batch processing, priority tier
• xAI: prompt caching, batch API, priority processing, pricing
• DeepSeek: context caching, pricing
• Moonshot AI: chat pricing, context caching, batch pricing
• Z.ai: cache guide, pricing
• Alibaba Cloud Model Studio: context cache, batch interface
• Groq: prompt caching, batch, flex processing
• Together AI: batch overview, serverless overview
• Fireworks AI: prompt caching, batch inference, serverless pricing

Prices verified October 18, 2026 (discount mechanics read from the provider docs that day; per-model rates as recorded in our model facts on October 17, 2026).