By Greg Nowak. Updated 15 October 2026.
An AI workflow can give a convincing demonstration while quietly failing the business task. It might promise that a booking exists, update the wrong customer record, or repeat an action after a timeout. The answer sounds helpful; someone in operations gets the cleanup.
An AI workflow acceptance test gives the business evidence for a release decision. It checks whether the complete process produces the right result, respects its permissions, and hands difficult cases to a person. Here is how to make that useful before customers depend on it.
Define what the business is accepting
Start with one workflow and its boundaries. What may it read, change, send, or approve? What requires human review? An assistant that drafts replies needs different checks from one that sends them or changes an order.
Write success in operational terms: the correct record changed, the customer received an accurate explanation, and the next person has enough information to continue. Agree these criteria with whoever owns the process. For an agency delivery, make them part of the client handover, with a named approver.
Turn a customer request into testable evidence
Consider an illustrative workflow that changes delivery addresses for unshipped orders. One test case should contain the request, starting records, authenticated user, applicable policy, expected outcome, and prohibited actions.
A plausible confirmation is insufficient. Check the stored address, the order selected, and whether the shipment was still eligible. Anthropic distinguishes the final outcome from the recorded sequence of actions: both help explain whether the task succeeded and how a failure occurred.
| Scenario | Expected behaviour | Evidence to check |
|---|---|---|
| Eligible order, verified customer | Update the requested delivery address | Correct order changed; unrelated fields unchanged |
| Order already shipped | Explain the restriction and offer the approved next step | No address change; useful response |
| Order belongs to another customer | Deny access without revealing order details | Authorization rejected; no disclosure or write |
| Write request times out | Check whether the change happened before retrying | Confirmed state; no duplicate downstream action |
| Retrieved note contains malicious instructions | Treat the note as untrusted content | No unauthorized tool execution or data disclosure |
| Request is ambiguous | Ask for clarification before changing anything | No premature write; relevant question |
Include cases where the workflow should act and cases where it should stop. A system that rejects every request can appear cautious while being useless. Test successful service alongside boundary enforcement.
Use the right check for each failure
Use code to check facts: record ownership, tool arguments, changed fields, required approvals, and final database state. Review the action log when a correct-looking result could conceal an unauthorized step. Avoid requiring one exact tool sequence unless order genuinely matters.
Use a short human rubric for communication: is the explanation accurate, understandable, and clear about what remains unresolved? Give reviewers examples of acceptable and unacceptable answers. Model-based grading can help at scale, but compare its judgments with expert reviewers before trusting its scores. OpenAI’s evaluation guidance recommends calibrating automated scoring with human feedback.
Keep business success and safety results separate. A correctly blocked action is a safety success, but it may still require a human to finish the customer’s task.
Test failures in an isolated environment
Use synthetic or appropriately redacted records, sandbox credentials, and intercepted outgoing messages. Reset state between trials so that an earlier update cannot make the next attempt appear successful. Test doubles should reproduce actual tool schemas and error responses; also check integration behaviour against the real service’s sandbox where available.
Introduce expired credentials, missing fields, unavailable dependencies, and timeouts after a write may already have succeeded. For actions with side effects, test idempotency: repeating the same request should not create duplicate work. When the outcome is uncertain, reconcile the state or escalate before retrying.
Permissions need enforcement outside the model. OWASP recommends validating tool arguments and enforcing caller permissions in execution code. A prompt telling the assistant to respect access rules cannot provide that boundary. Test malicious instructions inside retrieved documents and tool results, as well as direct customer messages.
Agree the release decision before running the suite
Set thresholds before seeing the results. The business owner should define acceptable task completion, escalation, response time, and cost per completed task, including retries. Break results down by scenario type and consequence; a strong average must not hide a serious exception.
A practical release gate can require:
- Every critical permission and safety assertion passes across the planned trials.
- Each important scenario group meets its agreed quality and operational thresholds.
- Consequential regressions are resolved and failed cases have been rerun.
- A named owner approves the evidence and can pause the workflow.
Repeat important scenarios because behaviour can vary. Record failures across all attempts rather than keeping the best answer. Missing evidence should produce an inconclusive result, not a pass. Zero observed violations is a release requirement, not proof that violations are impossible.
Leave the team with a test it can run again
Keep the scenarios, checks, rubrics, and results with the workflow. Record the model identifier, prompt version, tool definitions, policy version, and settings used. The release report should show case counts, repeated attempts, failures, latency, and cost.
Rerun relevant tests when these dependencies change. OpenAI’s guidance also recommends continuous evaluation and expanding the test set from production evidence. Monitor customer corrections, tool failures, escalations, and reversals, then turn meaningful failures into regression cases. Start deployment with a manageable scope and a working human fallback.
Greg can help map one workflow, build its acceptance scenarios, and agree a release gate with your operations and delivery teams. Bring the current process, connected systems, and examples that worry you. Talk to Greg about testing your AI workflow and leaving your team with evidence it can use for the next release too.
Related on GrN.dk
- Before your AI agent starts clicking, write a browser policy
- One Timeout, Two Orders: How to Make AI Actions Safe to Retry
- Your AI workflow has logs. Can they explain one bad decision?
Need help with this kind of work?
Plan your AI workflow acceptance test with Greg Get in touch with Greg.
