By Greg Nowak. Updated 10 November 2026.
Prompt caching can make a recurring AI workflow cheaper and faster. But when your provider charges a premium to write the cache, storing content that nobody reuses can increase the bill. The deciding factors are what stays unchanged, where the reusable content ends and when the next request arrives.
For business owners and operations leads, the useful measure is cost per completed job. For agencies, measure it per client too: a busy account can hide inefficient caching across quieter accounts. Start with one recurring workflow, such as support triage or document processing, and establish whether caching earns its place.
When does a cache write pay for itself?
Separate three prices: ordinary input, cache writes and cache reads. A write rate generally replaces ordinary input billing for those tokens; check your provider’s accounting before adding the two together.
As checked on 10 November 2026, GPT-5.6 Sol’s published pricing lists ordinary input at $4 per million tokens and cached input at $0.40. Its promotional pricing is available until 30 November 2026. Prompts above 272,000 input tokens attract higher rates. For a 30-minute cache lifetime, OpenAI’s caching documentation specifies a write rate of 1.25 times ordinary input.
Under those conditions, writing a 10,000-token prefix costs $0.05. One complete cached reuse costs $0.004. Together, that is $0.054, compared with $0.08 for processing the prefix twice without caching. One full reuse covers the write premium. Writing two different prefixes once each costs $0.10 and produces no read savings.
This example excludes changing input, outputs and tools. Partial hits, expired entries and longer cache lifetimes need their own calculation. Use your actual model, endpoint and billing tier.
Start with a baseline you can explain
Collect a representative sample covering busy bursts and quiet periods. Record the client, workflow, model, prompt version, request timing, token usage, latency and outcome. Give each business task an identifier so its retries and tool calls stay connected.
Use the observations to choose the investigation:
| What you see | What to investigate | Practical next step |
|---|---|---|
| Many writes, few reads | Changing prefixes or unsuitable boundaries | Compare assembled requests and locate the first difference. |
| Good test results, poor production reuse | Request spacing and client variations | Replay realistic intervals and separate client totals. |
| Reuse drops after deployment | Changed instructions, tools or document order | Compare prompt versions and repeat the baseline. |
| High hit rate, little financial benefit | Outputs, tools or retries dominate costs | Calculate total spend per successful task. |
| Large documents are used once | No realistic reuse opportunity | Test smaller inputs or avoid unnecessary writes where supported. |
Inspect the request your application actually sends
A stable template can still produce a changing request. Inspect the assembled input after variables, retrieved documents, tool definitions and conversation history have been added.
Consider a support assistant with a fixed policy document and a changing customer question. A fresh timestamp before the policy changes the prefix on every call. Put necessary changing information after reusable material, keep document ordering consistent and version the policy snapshot.
Group candidate prefixes using fingerprints, then investigate unexpected differences. Avoid collecting customer content unnecessarily. Keep context that improves the answer: cutting useful instructions to improve a caching metric can increase mistakes and manual review.
Match cache settings to the provider and endpoint
OpenAI Responses, GPT-5.6 and later: use prompt_cache_options: {"mode": "explicit"} and place prompt_cache_breakpoint: {"mode": "explicit"} on the supported content block ending the stable prefix. Keep that boundary in subsequent requests. Explicit mode without breakpoints creates no writes. The minimum is 1,024 visible input tokens. The current guide supports prompt_cache_options.ttl values of "30m" and "12h", with write rates of 1.25 and 2.5 times ordinary input respectively. Choose from observed request gaps; longer coverage carries a higher upfront cost.
Anthropic: place cache_control: {"type": "ephemeral"} on the last stable block for an explicit boundary. Its documentation also supports automatic caching through a top-level cache_control. Five-minute writes cost 1.25 times base input; one-hour writes cost twice base input. Read rates vary by model. The prefix hierarchy is tools, system, then messages, so earlier changes invalidate later cached content.
Amazon Bedrock: check the selected model and API in AWS’s caching documentation. Support, minimum lengths and lifetimes vary. Even eligible identical requests do not guarantee a hit.
Keep reads, writes and ordinary input separate
Retain the response usage fields when calculating costs:
- OpenAI:
usage.input_tokens_details.cached_tokensandcache_write_tokens. Both are included inusage.input_tokens; subtract them to obtain ordinary input. When mixing lifetimes, use the write breakdown to apply each rate. - Anthropic:
usage.cache_read_input_tokensandusage.cache_creation_input_tokensare separate from ordinaryusage.input_tokens. - Bedrock Converse:
usage.cacheReadInputTokensandusage.cacheWriteInputTokensare separate from ordinaryusage.inputTokens.
Price each category separately, then add outputs and tools. Copying one provider’s formula into another integration can double-count tokens or leave charges out.
Test the change against completed work
Replay a cold start, repeated calls, tool turns, a realistic pause, a document refresh and a prompt deployment. Apply the same acceptance criteria before and after. Divide total spend, including failed attempts, by successful tasks, and check whether users wait less.
Leave the team with a baseline, tested changes, a rollback option and someone responsible for reviewing costs after prompt or model changes. If savings are small, keep the implementation simple.
Greg can help connect API behaviour to operating costs and coordinate changes across your team or agency. Get in touch with Greg with the workflow, provider and cost or latency problem you want to address.
Related on GrN.dk
- Is Prompt Caching Actually Lowering Your AI API Bill?
- AI automations need a spend dashboard before the first runaway bill
- Before Your Support Bot Learns the Help Center, Make Sure It Can Forget
Need help with this kind of work?
Discuss your AI workflow with Greg Get in touch with Greg.