Inference
Chat Template
The formatting rule that turns a list of role and content messages into the exact token sequence a chat model was trained on.
In depth
The formatting rule that turns a list of role and content messages into the exact token sequence a chat model was trained on. Models use different control tokens. Mistral wraps user turns in [INST] and [/INST], while Zephyr uses <|user|> and <|assistant|>, and using the wrong format can badly hurt output quality. In Hugging Face Transformers the template ships with the tokenizer and is applied with tokenizer.apply_chat_template. Its add_generation_prompt option appends the tokens that cue the assistant's reply.
Primary source: huggingface.co
Example
In practice, developers reach for Chat Template when they need the capability described above as part of an AI feature or workflow.
Go deeper at Developers Digest
Hands-on guides, comparisons, and tutorials that cover Inference.
FAQ
Common questions
What is Chat Template?
The formatting rule that turns a list of role and content messages into the exact token sequence a chat model was trained on.
Why does Chat Template matter for AI developers?
Chat Template sits in the Inference part of the AI stack. Understanding it helps you make better decisions when building, debugging, and shipping AI features.
Where can I learn more about Chat Template?
Developers Digest publishes tutorials and videos that cover Inference topics including Chat Template. Check the blog and YouTube channel for hands-on walkthroughs.
Related
Related terms
Inference
Constrained Decoding
A generation technique that restricts which tokens a model may pick at each step so the output is guaranteed to match a format such as a regular expression, a grammar, or a JSON Schema.
Inference
Tokenizer
The component that converts raw text into tokens (and back) for a language model.
Inference
Continued Pretraining
Running more next-token-prediction training on an already pretrained model using a large unlabeled corpus from a target domain, such as code, math, medicine, or a new language.
Put this concept to work
Get Smarter About AI Dev
New tutorials, open-source projects, and deep dives on coding agents - delivered weekly.
One email per weekReal code, not theoryFree forever