Inference
Rate Limits (RPM and TPM)
Caps on how much traffic an organization can send to a model API, measured in requests per minute (RPM) and tokens per minute (TPM).
In depth
Caps on how much traffic an organization can send to a model API, measured in requests per minute (RPM) and tokens per minute (TPM). Anthropic splits the token cap into input tokens per minute (ITPM) and output tokens per minute (OTPM), refills capacity continuously with a token bucket rather than resetting each minute, and does not count cached input reads toward ITPM on most models. Exceeding any limit returns HTTP 429, usually with a retry-after header, and limits rise with your usage tier.
Primary source: platform.claude.com
Example
In practice, developers reach for Rate Limits (RPM and TPM) when they need the capability described above as part of an AI feature or workflow.
Go deeper at Developers Digest
Hands-on guides, comparisons, and tutorials that cover Inference.
FAQ
Common questions
What is Rate Limits (RPM and TPM)?
Caps on how much traffic an organization can send to a model API, measured in requests per minute (RPM) and tokens per minute (TPM).
Why does Rate Limits (RPM and TPM) matter for AI developers?
Rate Limits (RPM and TPM) sits in the Inference part of the AI stack. Understanding it helps you make better decisions when building, debugging, and shipping AI features.
Where can I learn more about Rate Limits (RPM and TPM)?
Developers Digest publishes tutorials and videos that cover Inference topics including Rate Limits (RPM and TPM). Check the blog and YouTube channel for hands-on walkthroughs.
Related
Related terms
Inference
Input vs Output Token Pricing
The split pricing model where LLM APIs charge one rate per million tokens you send (input) and a higher rate per million tokens the model generates (output).
Inference
Reasoning Effort
An API setting that tells a reasoning model how much work to put into a response, trading answer quality against latency and token cost.
Inference
Top-K / Top-P Sampling
Two methods for controlling the randomness of model output during token generation.
Put this concept to work
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever