Inference
Time to First Token (TTFT)
The time between sending a request and receiving the first output token.
In depth
The time between sending a request and receiving the first output token. It generally includes request queueing, the prefill pass over the prompt, and network latency, so long prompts and busy servers both push it up. TTFT is what makes an interface feel responsive, and it is measured separately from generation speed because a model can be slow to start and fast once it streams, or the reverse.
Primary source: docs.nvidia.com
Example
In practice, developers reach for Time to First Token (TTFT) when they need the capability described above as part of an AI feature or workflow.
Go deeper at Developers Digest
Hands-on guides, comparisons, and tutorials that cover Inference.
FAQ
Common questions
What is Time to First Token (TTFT)?
The time between sending a request and receiving the first output token.
Why does Time to First Token (TTFT) matter for AI developers?
Time to First Token (TTFT) sits in the Inference part of the AI stack. Understanding it helps you make better decisions when building, debugging, and shipping AI features.
Where can I learn more about Time to First Token (TTFT)?
Developers Digest publishes tutorials and videos that cover Inference topics including Time to First Token (TTFT). Check the blog and YouTube channel for hands-on walkthroughs.
Related
Related terms
Inference
Top-K / Top-P Sampling
Two methods for controlling the randomness of model output during token generation.
Inference
Prefill
The first phase of LLM inference, where the model processes the entire input prompt in parallel and builds the KV cache before producing any output.
Inference
Tokens per Second
A throughput measure of how many output tokens a model or serving system produces each second.
Put this concept to work
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever