What Happens to Your Data Inside AI Systems: Retention, Deletion and the Copies Nobody Tracks

Every AI feature an enterprise switches on creates new copies of company data: the prompt, the output, the application log, the observability trace, the embedding in a vector store, the fine-tuning set, the cached response and the agent's memory. Most corporate retention schedules cover none of them. The model provider's terms are usually the best-governed link. OpenAI's API keeps abuse-monitoring logs for up to 30 days by default; Anthropic deletes API inputs and outputs within 30 days; Google caches Gemini inputs and outputs for 24 hours unless caching is turned off. The harder problem is the copies the organization creates itself, which no vendor contract governs and no records manager has catalogued. The position here: treat every AI copy as a record with an owner and a retention period, classify derived copies like their source, and test deletion rather than assume it. The cost is that short retention removes evidence engineers and investigators want later, and that tradeoff is the decision executives actually own.

Key Takeaways

• A single AI assistant query can leave data in several places across the organization's own systems, the model provider, the application vendor and observability tools. Retention policy has to be set per copy, because each copy has a different default.
• Enterprise terms from the major providers now exclude training on business data by default, but "not used for training" is a different promise from "not retained". Abuse monitoring, chat history and stored application state each keep data for their own periods.
• Zero data retention is a contract term that can change. In June 2026 Anthropic required 30-day retention for its most capable models, overriding existing zero-retention agreements for most customers (with an interim zero-retention exception for some), and both Anthropic and OpenAI responded to enterprise pushback with new arrangements within three months.
• Deletion is hardest for derived copies. Research shows text can be largely reconstructed from embeddings, and fine-tuned weights cannot be edited to remove one person's record. The practical answer is to keep personal data out of these copies in the first place.
• AI logs are discoverable. A federal court ordered OpenAI to preserve deleted ChatGPT logs for months in 2025, and a February 2026 ruling held that a defendant's chatbot conversations were not privileged. Legal holds must name AI systems explicitly.

The Copies an AI Feature Creates

Each AI exchange leaves a trail, and each copy is governed by a different team, contract and default.

Prompts and outputs

The prompt includes whatever the user typed plus whatever the application added: retrieved documents, customer records, system instructions. The output can echo all of it. Both typically sit in the application's chat history and in the model provider's systems for some period.

Logs and traces

Engineering teams log AI calls for debugging, cost and evaluation, often with the full prompt and response, and observability platforms hold their own copy. Datadog's LLM Observability documentation states that traces are retained for 15 days by default, extendable to 90 with an add-on, and that prompts in its prompt registry are kept for three years. LangSmith keeps traces for 14 days at base retention but automatically upgrades a trace to 400 days when someone adds feedback or queues it for annotation, and traces copied into datasets are not subject to either limit. A routine quality review can turn a two-week copy into a permanent one. The operational case for these tools is set out in LLM observability and monitoring, though rarely with the retention case.

Embeddings in a vector store

Retrieval systems convert documents into numerical vectors so a model can find relevant passages. The vector store is a second copy of the indexed corpus, usually stored with the original text chunks. When a source document is deleted or its permissions change, the vector store does not know unless someone built that synchronisation.

Fine-tuning sets and fine-tuned weights

A fine-tuning job needs a training file of example prompts and answers drawn from real company data, uploaded and stored until deleted. The resulting weights carry the imprint of that data and cannot be edited.

Caches

Caches store recent inputs and outputs to cut cost and latency, as covered in LLM caching strategies. Providers cache too: Google Cloud's zero data retention documentation, updated October 2026, states that Gemini inputs and outputs are cached in memory with a 24-hour time to live at the project level unless an administrator disables caching.

Agent memory

Agents increasingly keep memory across sessions: conversation summaries, preferences, facts learned from tasks. It accumulates exactly the personal detail privacy law cares about. The design choices are covered in agent memory in production.

The data-copy table

What the Major Providers' Enterprise Terms Say

The terms below were checked in October 2026; they change often and should be re-verified at contract time.

OpenAI

OpenAI's API documentation states that API data has not been used for training by default since March 1, 2023, and that abuse-monitoring logs are retained for up to 30 days; eligible customers can apply for zero data retention or modified abuse monitoring. Conversations, assistants, threads, vector stores and files are stored "until deleted". For ChatGPT Enterprise and Business, OpenAI states that business content is not used for training by default, chats are saved until a user deletes them, and, per OpenAI's data governance guidance, Enterprise workspace owners can set custom retention with a 90-day minimum.

Anthropic

Anthropic's privacy center, updated July 2026, states that API inputs and outputs are deleted within 30 days, that deleted Claude for Work conversations are removed from back-end systems within 30 days, and that content flagged for usage policy violations can be kept for up to two years, with safety classification scores kept for up to seven. Enterprise plans can set custom retention with a 30-day minimum. The same page limits any research use of an organization's data to anonymised data, and only where the contract permits it.

Microsoft

Microsoft's Foundry documentation, updated June 2026, states that prompts and completions for models sold by Azure, including Azure OpenAI, are not used to train foundation models, and that stored data for features such as the Responses API can be deleted by the customer at any time. Flagged content may get human review unless a customer is approved for modified abuse monitoring. For Microsoft 365 Copilot, Purview documentation explains that prompts and responses are stored in a hidden folder in each user's Exchange mailbox, where retention policies and eDiscovery apply; Microsoft's own example shows a delete-after-one-day policy can take up to 16 days to purge a message.

Google

Google's Workspace documentation, updated October 2026, lets administrators auto-delete Gemini conversations after about 3 months, 18 months or 3 years, or never, and Google states that Workspace content is not used to train models outside the customer's domain without permission. On Google Cloud, zero retention for Gemini requires disabling the 24-hour cache, obtaining an abuse-monitoring logging exception where it applies, and avoiding grounding with Google Search, whose derived queries are stored for up to three days.

Zero retention is a contract term, not a property of the technology

The most important development of 2026 is that zero retention became negotiable again. Anthropic's support documentation states that from June 9, 2026, its "Covered Models", currently Claude Fable 5, Fable 5.1, Mythos 5 and Mythos 5.1, require 30-day retention of prompts and outputs for misuse detection, overriding zero data retention agreements, including those through Amazon Bedrock, Google Cloud and Microsoft Azure. Fortune reported on October 5, 2026 that the industry reaction was strongly negative.

Both labs then offered a different design. OpenAI announced Private Safety Processing on August 19, 2026, detecting misuse across interactions without retaining prompts, in preview for zero-retention deployments from September. Anthropic announced Enterprise Frontier Safeguards on September 1, 2026: under a program rolling out in phases from later this fall, prompts and outputs are still retained for misuse review, but in the customer's own cloud storage, for eligible customers who request access. Anthropic's support page, updated September 5, 2026, now says organizations notified as eligible under the program can use Fable 5 and Fable 5.1 under zero retention.

The lesson for procurement: the retention tier is tied to the model, not the vendor. A zero-retention agreement can be lost by upgrading models, so contracts should require notice and a right to stay on prior terms.

Why Deletion Is Hard

Deleting a file is solved. Deleting a fact from an AI system is not, because several copies are derived rather than stored.

Embeddings are not anonymised text

Embeddings look like meaningless numbers, so teams treat them as low-risk. A 2023 EMNLP paper by Morris, Kuleshov, Shmatikov and Rush reconstructed 92 percent of 32-token text inputs exactly from their embeddings, and the OWASP Top 10 for LLM Applications 2025 lists embedding inversion as a risk. A vector store built from HR files is an HR data store and should carry the same classification.

Fine-tuned weights cannot be edited

There is no reliable way today to remove one record's influence from a trained model short of retraining without it. An IAPP analysis published in June 2026 argued that for individual erasure requests the defensible practice is to delete the source data and exclude the person from future training, while noting that this has not been tested in enforcement. The European Data Protection Board's Opinion 28/2024, adopted December 17, 2024, held that a model trained on personal data cannot automatically be treated as anonymous; anonymity must be shown case by case. A model that memorises customer records may itself be personal data.

The deletion cascade

A single erasure request touches every copy in the table. A customer's email in a support ticket may also sit in the ticket's embedding, the trace of the agent that answered it, an engineer's evaluation dataset, the agent's memory and the provider's abuse log. Removing the ticket alone is partial deletion, and without a map the organization cannot prove otherwise.

Legal Holds and E-Discovery Reach AI Logs

Courts treat AI conversations as evidence, and a legal hold overrides every deletion setting above.

The clearest example is the New York Times copyright case against OpenAI. In May 2025 a federal magistrate judge ordered OpenAI to preserve output logs that would otherwise have been deleted; OpenAI said ChatGPT Enterprise, Edu and zero-retention API customers were not covered. The obligation ended on September 26, 2025, under an order filed October 9, 2025, though already-preserved logs and accounts the Times flagged stayed held. On November 7, 2025 the court ordered production of 20 million de-identified ChatGPT logs, affirmed by the district judge on January 5, 2026.

Separately, in United States v. Heppner, Judge Jed Rakoff of the Southern District of New York ruled from the bench on February 10, 2026, with a written opinion on February 17, that a defendant's conversations with a generative AI tool were not protected by attorney-client privilege or work product. Law firms including Orrick and Greenberg Traurig have since published guidance warning that employee use of AI tools for legal questions can waive privilege and create discovery risk.

Inside a Microsoft tenant, a litigation or eDiscovery hold suspends permanent deletion of Copilot interactions regardless of the retention policy. That is the right design, but it only helps if legal teams know where AI interactions live. Hold notices should name AI assistants, agent platforms, observability tools and vector stores explicitly, and AI incident playbooks should do the same.

Privacy Rights: Erasure Requests Meet AI Systems

Under Article 17 of the GDPR, individuals can require erasure of their personal data without undue delay where grounds apply, and Article 12 sets a one-month response period, extendable in some cases. California's CCPA gives consumers a comparable right to delete personal information a business collected from them, subject to exceptions; its updated regulations took effect January 1, 2026, with automated decision-making requirements applying from January 1, 2027, according to the California Privacy Protection Agency.

None of these laws exempts AI copies. An erasure request reaches the vector store and the trace archive as much as the customer database, and a copy held in another jurisdiction is still in scope, as covered in AI data residency. Regulated firms also face the tension between erasure and record-keeping obligations described in data privacy in financial services.

The Governance Answer

The fix is ordinary records management applied to a new class of records, in four steps.

1. Build an AI data map

For every AI feature in production, list each copy from the table, where it lives, who owns it, which contract governs it and what data enters it. Expect to find copies nobody approved, such as debug logs holding full prompts.

2. Set retention per copy type

Set a period for each copy, not one for "AI": short for caches and traces, tied to the source for embeddings, tied to business need for chat history, never indefinite by default. Where evaluation needs history, keep a curated, redacted dataset rather than raw traces.

3. Negotiate vendor terms up front

Terms are easiest to change before signature. Each contract should answer these in writing.

Vendor-terms checklist

• Is customer data used for training or product improvement, and is that off by default for this tier?
• What is retained for abuse monitoring, for how long, and who can review it?
• Is zero retention or modified monitoring available, for which models, and does it survive a model upgrade?
• Which features store state (threads, files, memory, vector stores) and what are their expiry options?
• How long after deletion is data purged from back-ups, and will the vendor certify it?
• Which subprocessors, including observability and safety vendors, receive copies?
• How will the vendor respond to a legal hold or a third-party subpoena for customer prompts, and will it notify the customer?
• What notice is given before retention terms change, and can the customer stay on prior terms?
• Where is each copy stored geographically, and can that be pinned?

4. Test deletion

Seed a synthetic record, a fictitious customer with a unique string, into each AI pipeline, issue a deletion request through the normal process, and search every copy in the data map for the string after the stated purge period. It turns a policy assumption into evidence and finds the vector store nobody wired to the deletion workflow.

A Decision Framework: How Much Retention Each Use Case Can Carry

The position, and its tradeoff

The recommended default is short retention everywhere the organization controls it, derived copies classified like their source, and personal data kept out of fine-tuning entirely. Vendor retention should be negotiated before go-live and tied to named models.

The tradeoff is real. Short trace retention means engineers cannot replay last quarter's failure. Short chat history weakens the assistant and removes material investigations sometimes need. And where frontier providers require retention, a strict stance may mean a less capable model for sensitive work. The alternative, keeping everything because storage is cheap, carries a cost that appears later: in discovery, in a breach, or in an erasure request that cannot be honoured. AI governance should make this choice deliberately rather than inherit vendor defaults.

Frequently Asked Questions

Does ChatGPT Enterprise store company data?

Yes. ChatGPT Enterprise keeps conversations until a user deletes them, unless workspace owners set a custom retention policy, which OpenAI's admin guidance says has a 90-day minimum. OpenAI states that Enterprise and Business content is not used to train its models by default. Deleted conversations are removed from OpenAI's systems within 30 days unless the law requires otherwise.

Can data be deleted from an AI model after it was used for fine-tuning?

Not selectively. Training files can be deleted, but the fine-tuned weights retain the data's influence and cannot be edited to remove one record. The options are deleting the model, retraining without the data, or arguing the person's influence is negligible, which is untested in enforcement. The safer course is to keep personal data out of fine-tuning sets.

Can AI chat logs be used in litigation?

Yes. Courts ordered preservation and then production of ChatGPT logs in the New York Times case against OpenAI, and in February 2026 a federal judge held in United States v. Heppner that a defendant's chatbot conversations were not privileged. Organizations should assume AI interactions are discoverable and include them in legal holds.

What is zero data retention, and is it guaranteed?

Zero data retention is a contractual arrangement under which a provider does not store prompts and outputs beyond processing them. It is not permanent: in June 2026 Anthropic required 30-day retention for its most capable models, overriding existing agreements for most customers, with an interim zero-retention exception for eligible customers until its replacement program is ready. Contracts should tie zero retention to named models and require notice before terms change.

The Bottom Line

The question executives usually ask is whether the vendor trains on their data. Every major provider now answers that well for business tiers; it is the wrong place to stop. The larger exposure is the copies the organization makes itself, in logs, traces, vector stores, evaluation sets and agent memory, which no contract governs and no retention schedule names.

Organizations that handle this well treat AI copies as records: mapped, owned, given a retention period, classified like their sources, covered by model-specific vendor terms, and proven deleted with a test record. The rest will learn what their AI systems kept when a court, a regulator or a customer asks, which is the most expensive time to learn it.