Background AI Tasks Need Queues, Not Longer API Calls

Illustrated infographic summarizing: Background AI Tasks Need Queues, Not Just Longer API Calls

By Greg Nowak. Updated 19 October 2026.

A longer timeout may get an AI demo through a slow request. Once that request becomes a customer report, CRM update or publishing workflow, the team needs more: a reliable way to track the work, control what happens next and recover when something fails.

Background AI tasks need a persistent job record and a worker that can resume processing. A queue holds work until a worker can pick it up. Together, these let people close their browser without losing the task—and give operations staff somewhere to investigate it.

Background mode finishes the AI request, not the business job

Suppose an agency generates a client report, sends it for approval and uploads the approved version to a portal. The AI response can succeed while the upload fails. Calling the whole job “completed” at that point gives everyone the wrong answer.

OpenAI’s background mode documentation shows how to start an asynchronous Responses API request:

curl https://api.openai.com/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-6-astra",
    "input": "Prepare the monthly variance review.",
    "background": true
  }'

Save the returned response ID against your business job. Retrieve it with GET /v1/responses/{response_id}; poll while its status is queued or in_progress. Check the final status before using the output: leaving those states does not guarantee success.

Plan result collection around retention. OpenAI currently documents roughly ten minutes of temporary disk storage for background execution, including ZDR projects using store=false. For Modified Abuse Monitoring projects, retaining background responses beyond that window requires explicit store=true. Confirm the settings and data terms for your project; a delayed webhook should not be your only recovery mechanism.

Create a job people can understand and recover

Before calling the model, save who requested the work, the client or workspace, the input version, the intended action and whether approval is required. Add the provider response ID, attempt count, timestamps and external record IDs as processing progresses.

Use business states such as queued, running, awaiting_approval, applying, completed, failed and cancelled. “Draft ready for review” is more useful to an account manager than a provider’s technical status.

Close the gap between saving and scheduling. If a job is saved but enqueueing fails, it must still be discoverable. A database-backed worker can claim pending jobs directly; with a separate queue, use a durable dispatch record saved in the same transaction as the job, then retry dispatch until acknowledged.

How much orchestration does the workflow need?

Choose around the consequences of failure and the team’s ability to support the system.

Workflow Starting point Essential control
Internal draft with no external update Database job table and worker Saved output, clear status and safe retry
One CRM, CMS or helpdesk update Persistent jobs plus a managed queue when needed Duplicate protection and external record ID
Several dependent steps or long approval waits Durable workflow engine Recorded progress, deadlines and recovery per step
Customer communication or financial action Appropriate foundation above, with approval Permission check, audit trail and rollback procedure
More consequential actions need stronger controls, even at low volume.

A managed queue helps separate incoming requests from processing capacity. A durable workflow engine becomes useful when coordinating several steps and waits would otherwise require substantial custom recovery code. Temporal’s workflow documentation, for example, describes persisted execution state and recovery through recorded event history. External actions still need duplicate protection.

Make retries safe before making them automatic

Amazon SQS documents at-least-once delivery: a standard queue can deliver a message again. Design workers so repeating a job does not repeat its business effect. This property is called idempotency.

Assign a stable key to the intended action, such as one approved report version uploaded to one client portal. Reuse it on retries, enforce uniqueness locally and pass it to the destination API where supported.

A local “done” flag alone cannot resolve a timeout after the destination has accepted an update. Check the destination using the key or a recorded reference before retrying. If you cannot establish what happened, route the job to a person rather than guessing.

Retry temporary outages with backoff and limits on attempts, elapsed time and spend. Invalid input and revoked permissions need correction. Resume from the failed step so a portal outage does not trigger another paid report generation.

Keep webhooks short and cancellation honest

OpenAI’s webhook guidance recommends signature verification and quick successful responses. Verify using the official SDK helper, deduplicate with webhook-id, persist receipt and schedule processing durably before returning 2xx. Failed or slow deliveries may be retried for up to 72 hours. Delivery deduplication does not replace the business action key.

When cancelling, record the request internally and cancel the provider response if possible. Workers must check cancellation before consequential writes. Define what happens if cancellation races with an update already underway: stopping processing cannot unsend an email or undo a committed change.

Agree the handover before launch

Give the operating team a view of failed jobs, stalled work and pending approvals, with an owner and a safe next action. For agency delivery, agree who monitors it after handover. Recheck permissions before applying results and keep secrets and unnecessary personal data out of logs.

Before release, demonstrate recovery from a worker restart, duplicate delivery, destination timeout and late cancellation. If those outcomes are unclear, the workflow needs another pass.

Greg can help map one workflow into a delivery brief covering job states, approvals, integrations and recovery ownership. Discuss your AI workflow with Greg to establish what a supportable first release needs.

Related on GrN.dk

Need help with this kind of work?

Discuss your AI workflow with Greg Get in touch with Greg.

Sources

Latest articles

Plan your PHP upgrade around checkout, enquiries and daily operations, with practical checks for compatibility, staging, deployment and rollback before support ends.

Claude Sonnet 4.5 has a tentative November 30 retirement date. Find affected workflows, check compatibility and test a supported replacement before switching.

Before closing an Apache patch ticket, check what is running, which fixes the package contains, and whether the updated service passes configuration and application checks.

Make monthly reporting easier to repeat with AI for Excel, Python and consistent checks. Get imports, error lists and reconciliation under control so the numbers can be verified.

A supplier PDF can influence more than an AI summary. Follow its path to a CRM update and see where validation, tool permissions and destination controls matter.

Check whether AI shopping channels show the right product, variant, price and availability, then trace discrepancies from your catalog through to checkout.

Add approved hours from Clockify to draft invoices in Dinero. AI writes service descriptions, while fixed rules govern amounts, approvals and reruns.

A blanket AI restriction can limit search discovery. Compare Google and Bing controls, agree content permissions, and check what your CMS actually delivers.

Measure AI automation costs per accepted result, including retries and human review. Check caching, batch processing and spending controls before scaling.

A 502 after maintenance can interrupt enquiries or checkout. Trace the NGINX-to-PHP connection, diagnose the failure, and check the fix survives a restart.