By Greg Nowak. Updated 10 September 2026.
If your AI workflow sends the same instructions or reference material repeatedly, prompt caching can reduce the cost of processing them. But creating a cache entry can carry a premium. The business question is whether enough later requests actually use what you paid to store.
That matters for customer support assistants, document processing and agency automations with long instructions or several tool calls. A workflow can produce perfectly acceptable answers while quietly rewriting a large prompt on every request. Before spending time tuning it, establish which repeated work accounts for a meaningful share of your bill.
When does a cache write pay for itself?
OpenAI lists GPT-5.6 Sol cache writes at 1.25 times the ordinary input rate, with cached reads at one tenth of that rate. The write price is the total rate for those tokens, not an extra 1.25-times surcharge. See the official model pricing.
For an illustrative prefix that would normally cost one unit to process, writing it once and reading it once costs 1.35 units. Processing it twice without caching costs two units. One complete reuse therefore covers the write premium. Two different prefixes, each written and never reused, cost 2.50 units.
This calculation excludes changing input, output and tool charges. It explains the opportunity, but your decision should use the cost of a completed business task, including retries and failed attempts. A cheaper input bill is useful only if the workflow still does its job.
Start with the workflows worth investigating
Choose one recurring workflow with meaningful volume or long prompts. For an agency, keep client and workflow totals separate so a busy account cannot hide waste elsewhere. Use this checklist to decide what to inspect first.
| What you observe | What to investigate | Practical next step |
|---|---|---|
| Many writes, few reads | Changing content inside the cached prefix | Compare consecutive rendered requests and locate the first difference. |
| Good reuse during tests, poor reuse in production | Request timing, routing and client-specific variations | Replay realistic traffic intervals and group results by client. |
| Reuse drops after a deployment | Changed instructions, tool definitions or document order | Compare prompt versions and repeat the cost baseline. |
| High cache hit rate, disappointing savings | Output, retries or tools dominate total cost | Measure the full cost per successful task. |
| Large reference document used once | No likely follow-up request | Test whether a smaller input or avoiding an unnecessary write costs less. |
Inspect the request your application actually sends
A prompt template can look stable while the assembled request changes constantly. Inspect it after variables, retrieved documents, tool definitions and conversation history have been added.
For example, an assistant might use a fixed policy document across several questions. Adding a fresh timestamp before that document changes the beginning of every request. Keep necessary changing information later, and give the policy snapshot an explicit version.
Generate a fingerprint of each candidate prefix and compare requests expected to share it. When fingerprints differ, inspect the first changed block. Use this to find accidental variation; do not remove information the task needs simply to improve a metric.
Keep static instructions, examples and tool definitions consistently ordered. Anthropic’s cache hierarchy runs through tools, system content and messages, so changing an earlier layer invalidates subsequent layers. Its prompt caching documentation explains these dependencies.
Choose cache boundaries deliberately
For GPT-5.6, the minimum cacheable prefix is 1,024 visible input tokens. Implicit mode can check earlier eligible message boundaries; it does not match every arbitrary shared prefix. Where changing content causes waste, mark the stable content block with prompt_cache_breakpoint: {"mode": "explicit"} and set prompt_cache_options.mode to "explicit".
Explicit mode requires explicit breakpoints: selecting the mode alone creates no writes. Keep prompt_cache_key consistent within the intended customer grouping. The default and only supported prompt_cache_options.ttl value is "30m", a minimum lifetime refreshed by reuse. These details are covered in OpenAI’s implementation guide.
Apply provider-specific settings. Anthropic offers five-minute writes at 1.25 times base input pricing and one-hour writes at twice base pricing. Choose the longer window when realistic request spacing justifies it.
Google’s Gemini guide distinguishes implicit caching from explicit cache objects: the Interactions API supports implicit caching only; explicit caching requires the generateContent API. Model minimums also differ. On Amazon Bedrock, check the selected model’s checkpoint and lifetime requirements.
Measure a sequence, including the awkward parts
Capture the provider’s usage fields alongside workflow, client, prompt version, latency and outcome:
- OpenAI Responses:
usage.input_tokens_details.cached_tokensandusage.input_tokens_details.cache_write_tokens. - Anthropic:
cache_read_input_tokensandcache_creation_input_tokenswithinusage. - Bedrock Converse:
cacheReadInputTokensandcacheWriteInputTokens.
Normalise these carefully. Anthropic’s input_tokens and Bedrock Converse’s inputTokens exclude cache reads and writes; add those categories to calculate total input. OpenAI Responses includes them in input_tokens.
Replay a cold start, repeated requests, tool turns, a pause, a document refresh and a prompt deployment. Compare total cost and response times, then check the answers against the same acceptance criteria. Retain the measurements after launch and assign someone to review changes when a prompt or model is updated.
Make the audit a manageable piece of work
A useful first audit should leave your team with a baseline, identified causes of wasted writes, tested request changes and a clear recommendation on what to deploy. Start with one workflow and expand when the evidence supports it.
If you want help connecting the API details to operational costs, Greg can help scope the audit and coordinate implementation with your team. Get in touch with Greg with the workflow you want to improve, the provider it uses and the cost or latency problem you are seeing.
Related on GrN.dk
- A Voice Agent Is Only Ready When the Human Handoff Works
- AI automations need a spend dashboard before the first runaway bill
- Before Your Support Bot Learns the Help Center, Test Whether It Can Forget
Need help with this kind of work?
Discuss your AI workflow with Greg Get in touch with Greg.
