Background AI Tasks Need Queues—not Just Longer API Calls

By Greg Nowak. Updated 17 August 2026.

A long-running AI request can now survive beyond a browser session or short HTTP timeout. That is useful for document reviews, research, record enrichment, report preparation, support triage, and other work that may take minutes rather than seconds.

But asynchronous execution does not make the surrounding business process reliable. The model can finish successfully while the CRM is unavailable, an approval is still pending, or the same completion event is delivered twice. A user can also cancel the work after an external update has already happened. Production systems therefore need two separate concepts: the AI response and the operational job responsible for using it.

Background mode fixes one failure mode

OpenAI’s Responses API supports background execution by setting background to true. The application can poll the response while its status is queued or in_progress, receive a completion webhook, and cancel an in-flight response. A current minimal request looks like this:

curl https://api.openai.com/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{
    "model": "gpt-5.6",
    "input": "Prepare the monthly variance review.",
    "background": true
  }'

This prevents a complex model task from depending on one uninterrupted client connection. It does not remember whether finance approved the result, whether a customer was notified, or whether an update was already written to another system.

Data handling also deserves an explicit check during procurement. OpenAI’s current documentation says background requests can use store=false; for Zero Data Retention projects, response data is temporarily stored for roughly ten minutes to support asynchronous execution and polling. Confirm that behavior against the organisation’s own contractual and compliance requirements before sending sensitive material.

Give the business process its own job record

Create an internal job before calling the model. At minimum, record a job ID, job type, requesting user, permission context, input version, intended side effect, idempotency key, AI response ID, retry count, timestamps, and external record IDs.

The job should have business-readable states such as queued, running, awaiting_approval, applying, completed, failed, and cancelled. Do not copy the provider’s response status directly into the interface: “response completed” may only mean that a draft is ready for review.

Task pattern	Minimum control	Sensible foundation
Private draft returned to one user	Persistent job record and recoverable status	Database-backed worker
One CRM, helpdesk, or CMS update	Idempotent write, bounded retry, audit record	Managed queue and worker
Several APIs or long waiting periods	Step-level state, timeouts, retries, compensation	Durable workflow engine
Customer-facing or irreversible action	Human approval and tightly scoped credentials	Workflow with an approval gate

Choose the lightest orchestration layer that still controls the consequences of failure.

Make webhook handling deliberately boring

A webhook endpoint should verify the signature using the raw request body, deduplicate the event, persist enough information to investigate it, enqueue any substantial work, and return a successful response quickly. OpenAI currently retries unsuccessful or slow deliveries for up to 72 hours with exponential backoff. Its documentation also warns that duplicate events can occur and recommends using the webhook-id header for deduplication.

That delivery ID prevents the same event from being accepted twice. A separate business idempotency key should prevent the intended action from happening twice. Before writing a CRM note, publishing content, or sending a notification, check whether that side effect has already been committed. Store the external system’s resulting ID in the same transaction where possible.

If the job is cancelled, a late completion webhook should be recorded without reviving it. Cancelling the OpenAI response is idempotent, but cancellation cannot undo an email, invoice, publication, or third-party update that your application has already made.

Queue, worker, or workflow engine?

A database job table and worker are often enough for one model request followed by one controlled action. This keeps the first version understandable and makes good use of infrastructure the team may already operate.

A managed message queue is appropriate when the producer and worker need independent availability or scaling. Remember that many standard queues use at-least-once delivery. Amazon SQS, for example, explicitly tells consumers to be idempotent because the same message can occasionally arrive again.

A durable workflow engine earns its place when work has several dependent steps, waits for people or external events, or may remain open for days. Temporal is one option; its documented workflow executions persist state through failures, can wait for signals or activities, and expose running, paused, completed, failed, cancelled, and timed-out states. Equivalent cloud workflow services may fit better if the organisation is already committed to a particular platform.

A practical production checklist

Snapshot the input so a later edit cannot silently change work already underway.
Separate generated output from permission to apply it.
Define which errors are retryable and set retry and cost limits.
Use idempotency at event, job, and external-write levels.
Expose failed jobs and pending approvals in an operator view.
Log state transitions without placing secrets or unnecessary personal data in logs.
Decide what cancellation and compensation mean before launch.

The prompt is only one part of this design. Ownership, permissions, state transitions, exception handling, and the final business handoff usually require more project work than the initial API call.

Start with the action that must not happen twice

For a first production release, choose one workflow and identify its most consequential side effect. Build the job states, approval rule, idempotency check, and recovery view around that action before adding more autonomy.

If your AI prototype now needs to touch CRM, support, billing, or publishing systems, Greg can help turn the technical idea into a controlled delivery plan. Discuss the workflow and its handoffs without committing to a larger build first.

Related on GrN.dk

Need help with this kind of work?

Plan your AI workflow with Greg Get in touch with Greg.

Sources

Last modified

2026-08-17