Chatbot Transcripts Quietly Became a Retention and Redaction Problem
By Greg Nowak. Last updated 2026-07-21.
To the customer, a chatbot conversation happens in one place. Behind the interface, the same text may pass through an application database, a model provider, an observability platform, analytics, security logs and a support system. Each stop can create another copy, with different access controls, retention periods and deletion behaviour.
Most teams do not set out to build a transcript archive. It happens one sensible decision at a time: tracing is added to diagnose failures, messages are stored for continuity, events are exported for analytics, and support staff get access to investigate complaints. The individual choices may be justified. The combined result is a distributed information-management problem that nobody fully owns.
One conversation, several records
Chat messages are less predictable than form fields. A customer trying to explain a problem might include a name, email address, account details, health information, internal business context or even a credential. With tool-using agents, the exposure grows: file reads, shell output and tool responses can all end up in the transcript sent to a provider.
A July 2026 paper on preventing oversharing in agent transcripts describes leakage involving file paths, email addresses and API keys. It also shows why simple placeholder replacement is fragile. A sensitive value might sit inside a longer string, recur across several turns or resemble an innocent value. The researchers tested a local boundary that rewrites sensitive structural values while leaving enough context for the model to do its job.
The implementation is specific, but the operational lesson is widely useful: redact data before it leaves the trusted application environment. Cleaning up the version shown in an admin screen is too late if the original text has already reached the provider, a tracing event or a raw application log.
Retention is not one setting
A statement such as “we keep chats for 30 days” sounds reassuring, but it is rarely a complete description. The real answer can change with the provider, product, model, workspace configuration and record type.
The OpenAI Compliance Platform documentation, for example, says the Compliance Logs Platform retains data for 30 days. Customers that need a longer period must continuously download the logs and retain them under their own policies. The documentation also separates deletion from internal audit and security retention: an item deleted through the API is removed from production search and retrieval indexes, while internal retention may continue for up to 30 days after the request. Authenticated API requests are also logged for security and compliance.
Anthropic’s June 2026 policy for covered models makes the same underlying point in a different way. It introduced 30-day retention for prompts and outputs from designated covered models, including certain organisations otherwise using zero-data-retention configurations. The applicable policy depends on the model, workspace and access route. Automatic deletion may also have exceptions when data is flagged by safety systems or must be retained for legal reasons.
Those policies are not interchangeable, and neither accounts for the copies inside your own systems. A useful retention register therefore needs to describe the live data flow, not repeat a single line from a provider page.
| Destination | Keep | Remove or transform | Evidence to collect |
|---|---|---|---|
| Model request and provider record | Only the context needed to answer or complete the task | Unneeded personal data, credentials, tokens and local system details | Provider setting, documented exceptions and a dated deletion test |
| Application database | Conversation state the product genuinely requires | Expired messages and fields without a defined purpose | Deletion-job result followed by a lookup confirming absence |
| Tracing and error monitoring | Request ID, timing, status and safe diagnostic metadata | Raw prompts, responses, headers and tool output by default | Expiry check covering active storage and exported events |
| Analytics or support system | Aggregated measures or the minimum context needed for a case | Full transcripts when summaries or pseudonymous identifiers will do | Connector test confirming downstream deletion or expiry |
Log the event, not every word
The OWASP Logging Cheat Sheet gives teams a sound design principle: capture enough information for the intended monitoring and analysis, but do not assume that means capturing the full content. Many chatbot reliability problems can be investigated with an interaction ID, timestamp, component, event type, outcome, latency and error category. The customer’s complete message does not need to appear in every trace.
OWASP says access tokens, passwords, connection strings, encryption keys and sensitive personal data should generally be removed, masked, sanitised, hashed or encrypted instead of being recorded directly. File paths, internal network details, names, phone numbers and email addresses may also need special handling. When identity is not required, deletion, scrambling or pseudonymisation can reduce exposure.
In practice, the work breaks down into five controls:
- Collect less. Define the fields actually needed for the response, debugging, analytics and support.
- Redact at the boundary. Detect secrets and sensitive identifiers before invoking either the model provider or a logging handler.
- Separate operational data from content. A dashboard that needs latency and failure rates should not automatically expose full conversations.
- Limit and record access. Give transcript access only to the roles that need it, and log access to sensitive records.
- Expire every copy. Run retention and deletion processes against application storage, analytics, tracing and support systems. Deleting the original record will not necessarily affect the rest.
Redaction also needs product-level testing. Useful cases include credentials buried in ordinary sentences, personal data split across turns, structured tool output, attachments or retrieved content, harmless values that resemble secrets, and failures in the logging path itself. OWASP recommends covering logging in code review, application testing and security verification, including checks for access control problems and unwanted side effects.
A deletion response is only the first check
If the chatbot database returns a successful deletion response, you know one operation worked. You still do not know what happened to a queued analytics event, a linked support case, a tracing platform or provider-side records subject to separate rules.
A more dependable workflow starts with a stable conversation identifier and a register of every downstream destination. For each destination, record whether data can be deleted directly, must be allowed to expire or falls under a documented exception. An automated job can then issue the available deletion requests, capture the results and perform follow-up checks. Any exception or failed check should remain visible instead of being silently counted as success.
This register cannot be a one-off document. Provider policies and interfaces change. OpenAI’s documentation, for instance, records a 2026 move from an older stateful route to a newer conversation-log system. Review the data flow on a schedule, and again whenever the team changes a model, plan, workspace setting or integration.
Build transcript controls into the service
The sensible target is not zero logging. A chatbot still needs enough operational information to support customers, diagnose failures and improve reliability. The aim is to keep that value without accumulating an unmanaged archive of customer conversations.
Greg can help make the controls concrete: trace where transcripts travel, decide which fields each system genuinely needs, put redaction ahead of model and logging calls, automate retention and deletion, tighten access, and test whether deletion reaches the full pipeline. That leaves the service supportable while replacing assumptions with controls the team can inspect and verify.
Related on GrN.dk
- Background AI Tasks Need Queues, Not Just Longer API Calls
- OpenAI Is Retiring Agent Builder: Save the Workflow, Not Just Prompts
- When AI writes JSON, one bad field can break the workflow
Need help with this kind of work?
Map your chatbot data flow Get in touch with Greg.