OpenAI has launched an improved prompt caching system for its GPT-6 family that delivers higher cache hit rates by default, the company said on September 22, 2026.
GPT-6 enables persistent agents to work for hours on complex tasks, from refactoring codebases to producing well-researched documents and presentations. The applications behind these agents make a series of API requests that build on one another, often carrying forward the same instructions, tool definitions and context from earlier turns. OpenAI caches that shared context to reuse computation across requests, reducing response times and giving developers discounts of up to 90% on cached input tokens.
Under the new system, OpenAI said it now gives cache discounts for eligible shared prefixes reused within a 30-minute window. It is also introducing new tools to help developers monitor cache performance, diagnose misses and choose how much of a prompt to cache.
"OpenAI's prompt caching plays a critical role in helping GitHub Copilot deliver fast, efficient experiences at scale," said Mario Rodriguez, Chief Product Officer. "Over the past several months, we've reduced by more than 50% the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models, relative to our previous baseline."
The new Prompt Caching Dashboard shows how much of an application's input is served from cache, allowing developers to track hit rates over time and compare cached and uncached tokens. When an unexpected cache miss occurs, a prompt caching diagnostics tool lets developers compare a request with a recent response to identify changes to the model, tools, settings or input that prevented reuse.
Developers can also use explicit cache breakpoints to choose which prompt prefixes to reuse. On GPT-6 models, reasoning effort can now be changed between responses without breaking cache by appending a configuration_update while leaving request-level reasoning effort unchanged. OpenAI also recommends keeping tool definitions, schemas and ordering stable, using allowed_tools or setting tool_choice to none instead of removing definitions, and prewarming the cache to reduce latency.
"Working with OpenAI, we moved our session agents to explicit cache breakpoints," said Eugene Mikhantyev, AI Engineer. "In under a week, cache hit rates on our evaluations rose from 83% to 91%."
Full Article: Better prompt caching for GPT-6