Cloudflare AI Gateway: Put LLM Spend Limits in the Request Path

Illustrated infographic summarizing: Cloudflare AI Gateway Puts LLM Budgets in the Request Path

By Greg Nowak. Updated 12 November 2026.

An AI feature can stay within its expected request volume and still become expensive. Longer conversations, larger documents and a switch to a premium model can all change the bill. For a business owner or operations lead, the useful question is: who can stop the spending, and what happens to the service when they do?

Cloudflare AI Gateway puts spend controls between your application and its model providers. That gives teams running several products, automations or agency integrations a place to enforce budgets. The implementation deserves as much attention as the budget amount.

What a spend limit actually controls

A rate limit caps request volume. A spend limit tracks accumulated model cost within a fixed or rolling window. Cloudflare checks applicable rules before forwarding a request and returns 429 when a budget is exhausted. Spend limits support Unified Billing and BYOK requests for models with known pricing.

There is room for overshoot: usage is recorded after completion, and enforcement is eventually consistent. Concurrent calls can therefore exceed the configured amount. Costs are estimates; reconcile them with provider billing. These boundaries are documented in Cloudflare’s spend-limit guidance.

For planning, leave headroom below the amount you are prepared to spend. Long outputs and parallel jobs deserve particular attention when testing that margin.

Give each budget an owner and a failure plan

Start with the decisions people need to make. Who pays for this workflow? Who can increase its allowance? Can it wait until tomorrow? A company-wide ceiling helps, but it should sit alongside controls that reflect how the business operates.

Workload Suggested budget scope Response to exhaustion
Optional internal experiments Team allowance Pause and ask the owner to review
Premium customer assistance Premium model allowance Use a tested cheaper model
Customer-specific processing Separate tenant allowances Queue work or apply the agreed service limit
Shared production services Overall gateway ceiling Preserve a usable non-AI journey
Choose the service response alongside the budget, before enabling enforcement.

Cloudflare’s filter mode targets a particular dimension value; split mode gives each value its own budget. For example, filter on environment=prod or split on tenant_id. Avoid splitting by several dimensions without checking the consequence: separate tenant-and-model buckets do not represent one combined allowance per tenant.

Make ownership visible in every request

Agree a small metadata contract across applications and agencies. Useful fields include team, app, workflow, environment and tenant_id. Document permitted values so that support and customer-support do not accidentally become separate accounting categories.

Cloudflare saves up to five custom entries per request, with string, number or boolean values; extra entries are ignored. Keys beginning with cf. are reserved. Access-protected custom domains can also add a verified cf.user_id, potentially displacing the fifth custom entry. See the custom metadata documentation.

Set budget identifiers in trusted application code. A customer should not be able to choose another tenant’s allowance by changing a request field. Also decide how your integration handles missing identifiers before relying on per-tenant controls.

Prove the request path with one workflow

Begin with a non-critical workflow and a dedicated test gateway. This request selects a gateway named budget-test and attaches ownership metadata:

curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \
  -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  -H "cf-aig-gateway-id: budget-test" \
  -H 'cf-aig-metadata: {"team":"support","app":"helpdesk","environment":"test"}' \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-4.1","messages":[{"role":"user","content":"Hello"}],"max_tokens":128}'

Create the named gateway first and set the account ID and token in your environment. These REST endpoints require Account > Workers AI > Read; a token with only AI Gateway configuration permissions will fail. The gateway header makes the destination explicit instead of relying on the account default. Both details appear in the REST API documentation.

Confirm that requests, usage and metadata appear under the intended gateway before adding enforcement. Check the scheduled jobs and background workers too: controls only cover traffic that actually passes through them.

Choose credentials and fallback behavior deliberately

Unified Billing charges supported model usage through Cloudflare. BYOK stores provider credentials in Cloudflare’s Secrets Store. On Unified Billing endpoints such as /ai/v1/chat/completions, the provider’s stored default key is consulted; without it, requests fall through to Unified Billing. Record this behavior in your handover so a credential change does not unexpectedly change who bills you. Cloudflare explains the distinction in its BYOK documentation.

For customer-facing work, dynamic routes can apply budget nodes and select fallbacks. They require gateway authentication and stored provider keys. As of this update, routes support the newer /ai/v1/chat/completions endpoint, as well as Workers bindings and the older compatible endpoint. Use a model value such as dynamic/support and select the gateway containing that route. Anthropic Messages and /ai/run do not support dynamic routes; see the current routing requirements.

Test fallback quality on representative tasks. Check structured output, tool calls and response time before accepting a cheaper model. A fallback must also fit within broader spending controls.

Roll out with a clear handover

Observe representative usage, set initial allowances, then deliberately exhaust a small test budget. Verify what customers see, whether queued work resumes correctly and whether retries stop. Repeatedly retrying an exhausted budget will not restore service.

For an agency handover, document the budget owner, change authority, credential owner, reset window and recovery procedure. Review blocked requests alongside spending: a low bill can also mean useful work has stopped.

If your model calls are spread across products, scripts and suppliers, Greg can help map the traffic, agree ownership and coordinate a controlled rollout. Discuss your AI Gateway setup with Greg.

Related on GrN.dk

Need help with this kind of work?

Discuss your AI Gateway setup with Greg Get in touch with Greg.

Sources

Latest articles

Check whether your contact form rejects missing or invalid Turnstile tokens, handles failed checks for genuine users and delivers accepted enquiries to your team.

Compare Bing citations, identifiable AI visits and verified enquiries to see which pages deserve attention before you increase your AI search marketing budget.

Teamtailor is phasing out Riminder on 21 November 2026. Check your parser settings, contact fields and integrations, and make sure you have a process for deleting applicant data.

WordPress jobs running late? Learn how to separate scheduling problems, task errors and a growing backlog, then choose the right scheduler or runner.

Before changing an AI workflow, check that it still does the job. Use realistic examples, clear release criteria and a comparison with the version running today.

Sending an invoice does not mean it has arrived. Understand delivery statuses, the system requirements coming in 2027, and how focused AI support can help finance teams handle errors and follow-up.

Before scaling AI content, check the pages you already have. Verify claims, resolve overlapping coverage, and give WordPress reviewers control over publication.

Build a refund approval workflow in n8n and Stripe that shows reviewers the exact action, expires old approvals and tracks refunds through to completion.

A timeout can hide a successful job creation. Use stable operation IDs, destination API rules and a clear recovery process to prevent duplicate work.

Does your ChatGPT subscription cover website translation? Understand API credits, access and a WordPress workflow that turns approved Danish content into translation drafts.