By Greg Nowak. Last updated 2026-07-08.
AI integrations are no longer just helping someone write a first draft. They are being wired into practical jobs: update a CRM record, create a support ticket, prepare invoice data, route a lead, publish metadata, notify a team.
That is where the risk changes. A weak paragraph usually gets noticed by a person. A bad field inside a JSON object can move quietly through an API workflow and only show up later as a messy record, a wrong status, or an automation that did exactly what the software allowed it to do.
The underlying pattern is now normal across the main AI platforms. OpenAI documents structured outputs for schema-constrained responses. Anthropic describes tool use as a loop where Claude requests a tool call and the application executes it. Google documents function calling in the Gemini API. JSON Schema gives teams a standard way to describe fields, types, required properties, and constraints. Pydantic makes that practical for Python validation.
So the useful question is not whether the model can return JSON. It is what your system does when the JSON is almost right.
The handoff is where the workflow usually gets fragile
Structured outputs are valuable because software can consume them. A support classifier might return a category, urgency, summary, and customer ID. A content workflow might return a title, slug, meta description, tags, and publishing status. A sales assistant might return a company name, contact role, next action, and due date.
That predictability is also why small errors matter. A missing field, an unsupported enum, a duplicated tag, a string where the billing system expects a number, or a customer ID mapped to the wrong account is not just an AI quality problem. It becomes an integration problem.
Anthropic's tool-use model is a useful reminder: the model can ask for a tool call, but the application executes it and returns the result. Google frames function calling in a similar way. OpenAI's structured outputs approach comes at the same issue from the response-format side: define the shape you expect, then ask the model to fit it.
Across providers, the direction is clear enough. This work belongs inside normal engineering discipline. The model proposes structured data. The application still owns the business action.
Valid JSON can still be wrong
Parsing is only the first gate. If the integration stops at whether the payload is syntactically valid JSON, it is too thin for production work.
A payload can parse cleanly and still be commercially unsafe. A status can be spelled correctly but not accepted by the receiving system. A date can be present but ambiguous. A required field can exist but be empty. A currency amount can arrive as text. A tag list can include duplicates. A record ID can point to the wrong customer.
JSON Schema helps because it turns expectations into a contract: objects, properties, required fields, types, arrays, and constraints. For AI workflows, that contract sits between the model and the systems that consume its output.
| Checkpoint | What to test | Business reason |
|---|---|---|
| Syntax | Can the response be parsed as JSON? | Stops obvious breakage before business logic runs. |
| Schema | Are required fields, types, objects, and arrays present? | Keeps malformed payloads out of CRMs, CMSs, queues, and finance tools. |
| Business rules | Are IDs, enum values, dates, amounts, and statuses acceptable? | Catches data that is well formed but operationally wrong. |
| Execution boundary | Does the app approve tool calls before running them? | Keeps real-world actions under application control. |
| Failure path | Are bad outputs retried, logged, or sent for review? | Makes errors visible before they become quiet data quality problems. |
Design schemas for operations, not demos
A demo schema often proves that a model can produce a tidy object. A production schema should protect the workflow it feeds.
That means using required fields only where the downstream process truly requires them. It means using enums where the receiving system has a fixed list of allowed values. It means deciding, in advance, how null and optional values should behave.
Free text deserves special care. A summary can vary. A payment status, ticket priority, product SKU, or publication state usually should not. Those fields need tighter boundaries because they trigger business logic.
OpenAI's structured outputs guide is relevant here because the schema is part of the API interaction, not a side note. That is the right mental model. The schema is not just documentation beside the integration. It is part of the handoff between the model and the software.
Still, schemas do not replace application validation. They describe the expected shape. Your application still has to decide what happens when validation fails, when a tool call should not run, or when a field looks valid but does not make sense in context.
Validate before the write, not after the cleanup
For Python teams, Pydantic is a practical way to turn typed models into validation logic and JSON Schema. Its documentation connects model definitions with JSON Schema, which helps when the same object shape has to live in prompts, API contracts, and application code.
A stronger workflow defines a Pydantic model for the AI-generated object, validates the response, rejects or retries invalid payloads, and logs the exact validation error. That gives the team a repeatable checkpoint before the object leaves the AI boundary.
PHP teams should apply the same discipline, even with different tooling. Decode the JSON. Validate required fields. Check accepted values. Normalize dates and numbers. Only then call WordPress, Drupal, a CRM API, a billing platform, or a support system.
The library choice matters less than the rule: AI-generated objects should not get write access to business software just because they look plausible.
Tool calling needs a controlled loop
Tool calling can make an integration feel more capable, but the application still decides what happens. Anthropic's documentation describes a flow where the assistant requests a tool, the application runs it, and the result goes back to the model. Gemini function calling follows the same broad pattern: structured output can be used by the application to invoke code.
That architecture is useful because execution remains in your software. It is also where controls belong.
Before a tool call runs, the application should check whether the tool name is allowed, whether the arguments match the schema, whether the user or workflow has permission, and whether the action needs confirmation. Creating a draft and sending an invoice should not share the same approval path. Summarizing a ticket and closing a ticket should not be treated as equivalent just because both are represented as JSON.
This is the point where many AI experiments become ordinary software projects. The model can prepare the structured request. The application must enforce the operating rules.
Retries help, but they need a ceiling
When a model returns invalid JSON or misses the schema, a retry can be reasonable. The application can pass back the validation error and ask for a corrected payload.
But retries should not become an invisible loop that hides weak integration design. A production workflow needs clear limits: how many retries are allowed, which errors are retryable, which failures need human review, and where failed payloads are logged.
If the same enum, date format, or nested object fails repeatedly, that is useful information. It may point to a loose schema, an unclear instruction, a weak example, or a downstream mapping that needs to be made more explicit.
Logging is not paperwork in this context. It is how the team learns whether the integration is stable enough to trust.
Where Greg can help
For companies already testing AI inside operational workflows, the most useful next step is often not another prototype. It is hardening the handoff between AI output and business software.
A focused engagement can start by mapping every place an AI response becomes structured data: CMS publishing fields, support categories, CRM updates, product metadata, invoice preparation, internal notifications, or task creation. Each handoff needs a schema, validation layer, error path, retry policy, and logging strategy.
The implementation work is concrete: define JSON Schemas for expected outputs, add Python or PHP validation, create sample valid and invalid payloads, build retry and exception handling, and make failed outputs visible before they touch customers, records, or revenue operations.
The aim is not a more dramatic AI demo. It is a safer automation system. Treat AI-generated JSON as untrusted input, validate it like any other external payload, and keep the final business action under application control.
Related on GrN.dk
- When Google can call the business, your local data stops being cosmetic
- Background AI Tasks Need Queues, Not Just Longer API Calls
- ChatGPT apps need a permissions map before they touch company data
Need help with this kind of work?
Harden your AI integration before it touches production data Get in touch with Greg.