Prompt Caching

In depth

A provider feature that caches the processing of static prompt prefixes so repeated requests with the same system prompt or context pay the computation cost only once. Subsequent requests that share the cached prefix skip re-processing, resulting in lower latency and reduced costs (typically 90% cheaper for cached tokens). Prompt caching is especially valuable for RAG systems, agent loops, and any application that sends the same large context repeatedly.