By Greg Nowak. Updated 4 November 2026.
A customer submits an enquiry. Your system saves it, then asks AI to classify it. The AI request fails, and the page says “Please try again.” The customer submits twice. Your team now has duplicate records and someone wondering whether you received their message.
This is the practical problem behind AI API reliability: customers need to know what happened to their work. Whether you run a business, manage operations, or deliver integrations for clients, agree on the saved state, the recovery plan, and the customer message before adding another retry loop.
What has actually finished?
Separate the customer’s main action from the AI step. A saved enquiry can be acknowledged while classification waits. An assistant that failed to produce an answer cannot claim completion just because it received the question.
For each workflow, record whether the task was received, is processing, completed, or needs attention. Keep a stable task reference across attempts. Build the confirmation message from that record rather than from whether the latest API call succeeded.
For example, “Your enquiry has been received. Reference 1234” is appropriate after a confirmed save. “We’ll email your result” requires a saved task and a working delivery route. These are different promises.
Match the recovery to the failure
OpenAI’s official error documentation distinguishes temporary rate limits from exhausted credits and spend or usage limits. Several return 429. Inspect the error body, including error.code, before deciding to retry.
| Recorded condition | Application response | Example customer message |
|---|---|---|
| Enquiry saved; AI enrichment delayed | Keep the submission and defer enrichment. | “Your enquiry has been received.” |
Temporary 429, including slow_down |
Reduce traffic and retry within the agreed budget. | “This is taking longer than expected.” |
503 with server_is_overloaded |
Wait before a bounded retry; offer a fallback when the deadline passes. | “We couldn’t complete your answer. Please try again later.” |
| Credits, billing, credentials, or configuration need attention | Stop automatic retries and alert the responsible person. | “This service is currently unavailable. Please contact us directly.” |
| Timeout after a possible external action | Check the outcome before replaying the action. | “We’re checking whether your request completed.” |
Set a page deadline and a task deadline
The page deadline determines when someone stops waiting at a spinner. The task deadline determines how long saved work may continue trying. Choose both around the service you are offering: an interactive answer and an overnight classification job have different tolerances.
OpenAI’s rate-limit guidance says to respect a valid Retry-After as a minimum wait and add a small random delay. Without a valid hint, use increasing delays with jitter. Bound the number of attempts and total retry time; failed requests also consume per-minute limits.
If the required wait exceeds the remaining budget, defer work that can safely wait or end with a clear next step. An individual attempt timing out does not establish a deadline for the whole workflow. For streaming answers, handle interruptions explicitly: errors can arrive after output begins, and automatic replay may repeat content the customer already saw.
Count retries across the whole integration
An SDK, an application wrapper, and a queue worker may each retry. Amazon’s reliability guidance explains how retries at several layers multiply traffic. Assign responsibility for the retry policy and count all attempts together.
The official OpenAI Python library reference documents two automatic retries for eligible errors by default. If your application manages retries, OpenAI(max_retries=0) disables SDK retries. Otherwise, include them in your budget. Check the installed version and configure timeouts deliberately; a client timeout setting alone does not cap the entire business task.
Prevent duplicate actions and keep unfinished work visible
A timeout does not prove that nothing happened. Amazon’s guidance warns that an operation may have produced side effects before the caller stopped waiting. This matters when AI workflows create CRM records, send emails, or trigger other systems.
Use the stable task reference to associate attempts with one customer action. Where a destination supports idempotency keys, reuse the same key for retries of that action. Otherwise, record completion and check the destination before replaying uncertain work. A disabled submit button helps with repeated clicks; it does not protect against worker retries.
For deferred tasks, provide an operations view showing age, attempts, last error, and owner. After the retry limit, route failures to a review queue, often called a dead letter queue. Check your queue provider’s retention rules and arrange alerts before messages expire. Someone needs responsibility for resolving or closing each exception.
A practical review for an existing AI integration
Follow one customer submission through storage, AI processing, external actions, and confirmation. Then test a temporary rate limit, an overload response, an account error, a broken stream, and a timeout after an external action. Include a task that exhausts its retries.
The useful acceptance test is whether support can explain what happened and give the correct next step from the task record. For an agency handover, document who receives alerts and who can safely restart work.
If your integration leaves those answers unclear, discuss it with Greg. A focused review can map failure states, agree retry limits, and define customer messages and recovery steps your team can maintain.
Related on GrN.dk
- AI automations need a spend dashboard before the first runaway bill
- OpenAI Computer Use: What Browser Agents Need Beyond the Demo
- A Voice Agent Is Only Ready When the Human Handoff Works
Need help with this kind of work?
Discuss your AI integration with Greg Get in touch with Greg.
