How to cut your Together AI API bill

Mode

Automatic: repeated prefixes are cached with no code change

Serverless: automatic, prefix-based cached-input billing on select chat models only (those with a Cached input price). Dedicated endpoints: prompt caching on by default, scoped to your replicas.

Cache lifetime

Best-effort and short-lived; no configurable retention window

Minimum prompt

Not stated

Cache write cost

Not stated

Cache read cost

Steep discount on supported models (per-model Cached input price)

Storage cost

Not stated

How to enable

Automatic; no header, parameter or toggle.

From the docs

“Automatic: There is no header, parameter, or account toggle to enable it.”

“Best-effort and short-lived: The serverless cache is shared across the fleet and entries are evicted as traffic shifts, so cache hits aren't guaranteed and there's no configurable retention window.”

“Limited to supported models: Only models with a value in the Cached input pricing column on Chat models support cached input billing.”

“Select serverless chat models bill cached input tokens at a steep discount.”

Discount

50% off select models

'Up to 50% off', but only listed discounted models get 50% (currently only openai/whisper-large-v3); unlisted models run at standard rates

Window

24h completion window, fixed, best-effort (can exceed it)

Limits

Up to 50,000 requests per batch; 100 MB per input file; 10 MB per line; 30B tokens enqueued per model; separate rate-limit pool; files retained 7 days

Endpoint

Batch API with JSONL lines targeting /v1/chat/completions (audio: /v1/audio/transcriptions, /v1/audio/translations)

Stacks with caching

Not stated: the provider's pages do not say

From the docs

“You get up to 50% off serverless rates and a separate rate limit pool, in exchange for a job-shaped (rather than request-shaped) workflow.”

“Selected serverless models run at 50% off batch rates:”

“Models not listed run at standard rates.”

“Completion window defaults to `24h` and cannot be changed. It is a best-effort target.”

“Up to 50,000 requests per batch.”

“Up to 100 MB per input file.”

“Batch jobs can also run against dedicated model inference, but the discount does not apply to dedicated model inference usage.”

Sources: docs.together.ai/docs/inference/batch/overview, docs.together.ai/docs/serverless/overview

Service tiers on the pages checked: Not stated

• Provisioned throughput (committed throughput, via sales)

  “For committed throughput and reliability guarantees, contact sales about provisioned throughput.”

• Dedicated endpoints (reserved hardware)

  “If you need direct control over reserved hardware, use a dedicated endpoint.”

This site does not track per-model Together AI prices. Use the sources below for the current rate card.