# Configuring Zero Data Retention for LLM APIs

[Skip to content](#lm-inhoud)Network/[NL](/en/zero-data-retention-api-configuraties)EN[Hubhub.llmnet.nlCompare models on task, language, cost and license.](https://hub.llmnet.nl/en/)[Communitycommunity.llmnet.nlPrompt techniques, patterns and system prompts.](https://community.llmnet.nl/en/)[APIapi.llmnet.nlLLMs in production: rate limits, routing, structured output.](https://api.llmnet.nl/en/)[Consultancyconsultancy.llmnet.nlRolling out AI in an organization, pilot to production.](https://consultancy.llmnet.nl/en/)[Newsnieuws.llmnet.nlAI developments, explained for the Netherlands.](https://nieuws.llmnet.nl/en/)[Benchmarkbenchmark.llmnet.nlMeasure AI quality yourself, on your own tasks.](https://benchmark.llmnet.nl/en/)[Careersvacatures.llmnet.nlAI roles, salaries and career paths in the Netherlands.](https://vacatures.llmnet.nl/en/)[Learnleren.llmnet.nlAI concepts in plain language, beginner to builder.](https://leren.llmnet.nl/en/)[Guidegids.llmnet.nlRun AI privately on your own Mac, PC, NAS or home server.](https://gids.llmnet.nl/en/)[Directorydirectory.llmnet.nlMapping the AI ecosystem: tools, models, companies.](https://directory.llmnet.nl/en/)[Radarradar.llmnet.nlSignals from X, research and communities for indie developers.](https://radar.llmnet.nl/en/)[Appsapps.llmnet.nlReviews of AI apps and open-source repos, with tips for builders.](https://apps.llmnet.nl/en/)[llmnet.nl — main site](https://llmnet.nl/en/)[](https://x.com/intent/post?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fzero-data-retention-api-configuraties&text=Configuring%20Zero%20Data%20Retention%20for%20LLM%20APIs)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fzero-data-retention-api-configuraties)[](https://www.reddit.com/submit?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fzero-data-retention-api-configuraties&title=Configuring%20Zero%20Data%20Retention%20for%20LLM%20APIs)[](#)[](https://x.com/intent/post?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fzero-data-retention-api-configuraties&text=Configuring%20Zero%20Data%20Retention%20for%20LLM%20APIs)[](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fzero-data-retention-api-configuraties)[](https://www.reddit.com/submit?url=https%3A%2F%2Fapi.llmnet.nl%2Fen%2Fzero-data-retention-api-configuraties&title=Configuring%20Zero%20Data%20Retention%20for%20LLM%20APIs)[](#)

# Configuring Zero Data Retention for LLM APIs

By Ivo Donker — compiled with AI support (Claude & Gemini) · Last updated: 6 August 2026

When integrating large language models (LLMs) into business applications, handling sensitive data is a critical factor. When organizations use external API providers, prompts and the generated responses are sent over the public internet to infrastructure managed by third parties. To prevent this data from being stored persistently, many API providers offer the option of Zero Data Retention (ZDR). This article analyzes what ZDR technically entails, how the different storage layers relate to each other, and how to implement and verify these configurations within your own software architecture.

## What zero data retention promises (and what it doesn't)

The term Zero Data Retention suggests that no data is ever retained at any point. In practice, this definition is more nuanced and specifically pertains to the persistence of the payload (the prompt and the completion) on physical storage media after the API transaction completes. It's important to understand the technical boundaries of this promise in order to identify risks via an [AI risk assessment (DPIA)](https://consultancy.llmnet.nl/en/ai-risicoanalyse-dpia).

When a request is sent to an LLM API under a ZDR agreement, the following happens to the payload:

 
- In-memory processing: The prompt is read into the working memory (RAM or VRAM) of the processing servers to execute the transformer computations. During processing, the text exists in an active state in memory.
 
- No disk storage: Once the API response has been fully generated and sent back to the client, the payload is cleared from volatile memory. No writing occurs to the provider's database systems, transactional logs, or backup media.
 
- Metadata exception: ZDR almost never applies to the request's metadata. Data such as the timestamp of the call, the API key used, the number of tokens processed, the client's IP address, and the specific model type are still logged for operational purposes.

 Important: ZDR prevents data at rest from being stored at the provider after the request, but it doesn't protect the data against interception during transport (data in transit). For this, transport encryption (TLS) and network security remain the primary mechanisms.

## The distinction between the three retention layers

When analyzing the privacy policy and API documentation of LLM providers, three different forms of data retention are often conflated. For a correct configuration, these layers must be assessed strictly separately:

 
 
 Retention layer | 
 Purpose of storage | 
 Typical retention period | 
 ZDR status | 
 

 
 
 
 Model training | 
 Improving and training future model versions. | 
 Permanent (unless deleted) | 
 Disabled by default on commercial APIs. | 
 

 
 Abuse monitoring | 
 Detecting misuse, illegal content, or policy violations. | 
 Often 14 to 30 days | 
 Must be explicitly turned off (ZDR opt-in). | 
 

 
 Debugging & Support | 
 Troubleshooting and incident support by engineers. | 
 A few hours to days | 
 Fully disabled for payload under ZDR. | 
 

 

### 1. Use of input for model training

For most commercial API plans (as opposed to free consumer interfaces), input data is not used to train models by default. This is the minimal baseline. However, this doesn't mean the data isn't stored; it only means the data doesn't end up in the training pipeline.

### 2. Retention for abuse monitoring

This is the most common reason data still gets stored temporarily. Providers keep prompts and completions to be able to check afterward whether misuse occurred (such as generating harmful code or disinformation). ZDR is only truly active once this abuse monitoring has been disabled for your API keys or account. This often requires a specific approval or contractual addendum.

### 3. Retention for debugging and support

When an API call fails with an HTTP 5xx error, systems often store the exact payload to let developers reproduce the error. Under a ZDR configuration, this form of logging must be blocked for the payload. In that case, error logs may only contain non-sensitive metadata.

## Default settings versus explicit configuration

Assuming an API connection is secure by default can lead to data leaks. Default settings vary widely between providers. There's also an important technical distinction between account-level settings and per-request configurations.

At the account or organization level a provider can enforce ZDR for all API keys under that account. This is often done based on an approved request form or an enterprise contract. The advantage is that individual developers within the organization don't need to include specific parameters to activate ZDR. The policy is enforced centrally at the provider's API gateway.

At the request level (per API call) some systems offer parameters or HTTP headers to suppress data storage. This provides flexibility when only part of the application works with sensitive data. The risk, however, is that a coding error or omitting the parameter in a new microservice results in data still being stored persistently. This can be checked against the [GDPR privacy checklist](https://gids.llmnet.nl/en/avg-privacy-checklist) for AI systems.

## What's typically still retained: the impact on the DPIA

When performing a data protection impact assessment, it's not enough to state that ZDR is enabled. The metadata that falls outside the ZDR scope must be mapped out precisely. This data can, after all, still contain personal data or be traceable to individuals.

The following metadata is generally still retained by API providers:

 
- IP addresses and routing information: The network origin of the request. This is relevant if the API is called directly from the client application (for example, a mobile app) rather than via your own backend.
 
- Token counts (input and output): Needed for billing and volume calculations. Although the tokens themselves (the text) are deleted, the number (for example prompt_tokens: 142, completion_tokens: 85) is retained.
 
- Rate-limit counters and timestamps: To combat misuse and overload (DDoS), the provider tracks how many requests a specific API key makes per minute or day.
 
- Billing records: Usage data linked to the account identifier for administrative purposes.

If metadata is retained, the DPIA must document how long it's kept and how the provider protects this data against unauthorized access.

## Consequences for stateful features and advanced API functionality

ZDR is technically relatively simple for stateless, one-off text generations. However, as soon as you use more complex API features that depend on state, conflicts arise with the zero-data-retention promise. The following features require extra attention:

### Server-side conversation state (Assistant APIs)

Some providers offer Assistant APIs where the conversation history (the 'threads') is stored on the provider's servers. This lets developers avoid resending the context with every call. Under an active ZDR policy, these stateful endpoints are often unusable, or the state must be fully managed on the client side (in your own database).

### File uploads (RAG and fine-tuning)

Features where files (such as PDFs or JSONL training files) are uploaded to the provider for indexing or fine-tuning are inherently at odds with ZDR. These files must be stored persistently so the model can read them during runtime. In these scenarios, ZDR often applies only to the final query, not to the uploaded reference files.

### Prompt caching

To reduce cost and latency, many APIs offer prompt caching. Here, the system prompt or a large document is temporarily cached on the provider's servers so that subsequent requests are processed faster. Although this cache is often ephemeral (expiring after a few hours), it does mean the prompt remains stored during that time. This must be explicitly weighed against ZDR requirements.

### Asynchronous batch processing

With batch APIs, you send a large number of requests at once, which the provider then processes within 24 hours. The input and generated results must be stored on the provider's servers during this window until the client retrieves the results. ZDR here is often only active after the results have been successfully downloaded, or after a fixed retention period has elapsed (for example, 7 days).

## Enforcing ZDR in your own gateway

Rather than blindly trusting the external provider's configuration, it's architecturally safer to set up a filtering and control layer within your own infrastructure. This is especially relevant when you decide to [self-host an LLM gateway](https://api.llmnet.nl/en/llm-gateway-zelf-hosten). The gateway acts as a proxy between the internal applications and the external APIs.

Within this gateway, the following mechanisms can be implemented to enforce data protection:

// Conceptuele weergave van payload-sanitatie in een eigen gateway
function preprocessRequest(request) {
 // 1. Normaliseer het verzoek en verwijder niet-toegestane velden
 const sanitizedBody = filterFields(request.body, ALLOW_LIST);
 
 // 2. Strip potentiële persoonsgegevens met regex of PII-detectie
 sanitizedBody.prompt = stripPII(sanitizedBody.prompt);
 
 // 3. Dwing ZDR headers of parameters af
 sanitizedBody.extra_headers = {
 ...sanitizedBody.extra_headers,
 "X-Disable-Abuse-Monitoring": "true"
 };
 
 return sanitizedBody;
}

The gateway should operate on the basis of an allow-list of API parameters instead of a deny-list. This prevents new parameters introduced by the provider (which might activate data storage) from accidentally being passed through to the external API. The gateway can also analyze outgoing requests for sensitive patterns (such as national ID numbers or credit card data) and pseudonymize these before they leave the infrastructure.

## The risk of your own logging

A common pitfall is that an organization puts a lot of effort into configuring ZDR with the LLM provider, only to then store the prompts and completions unencrypted in its own application logs, tracing systems, or APM tools (Application Performance Monitoring). This merely shifts the data-leak risk from the provider to your own infrastructure.

To prevent this, logging and observability systems must be configured so that payloads are systematically excluded from storage, as described in the guide on [observability and logging](https://api.llmnet.nl/en/observability-en-logging). This requires concrete measures:

 
- Separation of metadata and payload: Do log the status code, response time, and token count, but never the content of the messagesarray or the completiontext.
 
- Masking in error reports: In application exceptions, the framework may be inclined to include the full HTTP request body in the stack trace. Make sure the HTTP client is configured to mask sensitive headers and body content in error logs.
 
- Access control on debug logs: If debug logging must be temporarily enabled in production, set up automatic retention limits (for example, a maximum of 24 hours) and strict role-based access control (RBAC) for these logs.

## Region and processing location versus zero data retention

It's a common misconception that ZDR and data residency are synonyms. ZDR only guarantees that data isn't stored persistently. It says nothing about where the data is processed in memory.

If an organization is bound by European legislation, processing of personal data is often required to take place within the European Economic Area (EEA). An API call to a provider with ZDR enabled can still be routed to servers in the United States for the in-memory computations. To operate compliantly, both properties must be configured independently:

 
- Data residency: Configure the API client to connect to specific regional endpoints (for example eu-west-1.api.provider.com) to guarantee that the data doesn't cross the geographic boundary during transport and temporary processing.
 
- ZDR: Make sure the ZDR settings are active on those specific regional endpoints to prevent persistent storage after processing.

## Verification and contractual documentation

A technical configuration is legally and operationally worthless without the right contractual basis. The technical ZDR setting must be directly tied to the legal agreements in the provider's [AI contracts and SLAs](https://consultancy.llmnet.nl/en/ai-contracten-en-sla) .

When reviewing contracts and data processing agreements (DPAs), pay attention to the following aspects:

 
- Right to audit: Does the provider offer the option of external audits (such as SOC 2 Type II reports) that specifically validate the operation of the ZDR pipelines?
 
- Change management: How does the provider inform you about changes to the API terms or default settings? API version changes (for example, from v1 to v2) can sometimes cause earlier opt-out settings for abuse monitoring to lapse or need to be requested again.
 
- Subprocessors: Check whether the provider uses subprocessors to process the API requests, and whether the ZDR obligations are also imposed on these parties.

## Practical checklist for go-live

Before production data actually flows through an LLM API integration, the technical implementation should be verified against this checklist:

 
- [ ] Contractual confirmation: Has the data processing agreement been signed, and has the ZDR clause (including exclusion of model training and abuse monitoring) been formally confirmed for the relevant account IDs?
 
- [ ] Endpoint verification: Do all API calls in the production code point to the correct regional and ZDR-specific endpoints?
 
- [ ] Header and parameter check: If ZDR is configured at the request level: is there a unit test or integration test that verifies the required parameters (such as specific headers or flags) are actually present in every outgoing HTTP request?
 
- [ ] Feature restrictions: Are stateful features (such as assistants or server-side threads) disabled or migrated to client-side alternatives?
 
- [ ] Gateway security: Does your own API gateway filter out unknown or unsafe parameters and enforce an allow-list?
 
- [ ] Local log sanitization: Has it been manually verified that no prompt or completion data ends up in your own application logs, error reporting systems, and APM dashboards? This is essential for compliance with your internal policy on [audit logging and compliance](https://api.llmnet.nl/en/audit-logging-en-compliance).
 
- [ ] Periodic review cycle: Is there a process in place to revalidate the retention settings with every major update to the API client or the provider's API?

## Further reading

 
- [Audit logging and compliance for API integrations](https://api.llmnet.nl/en/audit-logging-en-compliance)
 
- [Observability and logging in complex LLM environments](https://api.llmnet.nl/en/observability-en-logging)
 
- [Step-by-step plan: Self-hosting an LLM gateway](https://api.llmnet.nl/en/llm-gateway-zelf-hosten)
 
- [AI risk assessment and conducting a DPIA](https://consultancy.llmnet.nl/en/ai-risicoanalyse-dpia)
 
- [AI contracts and SLAs: What to look out for](https://consultancy.llmnet.nl/en/ai-contracten-en-sla)
 
- [The GDPR privacy checklist for AI applications](https://gids.llmnet.nl/en/avg-privacy-checklist)

llmnet.nl - LLM aggregation and API integration