A privacy checklist for LLM inference in Europe

Privacy work is strongest when it is specific and repeatable. Treat prompts and outputs as potentially personal data, encrypt them, limit access, and avoid storing more than the workload needs. A regional deployment may help with latency or infrastructure location, but it does not by itself establish where every request, log, metric, backup, support record, or supporting service is processed.

• Residency and location:
  • Choose a region that aligns with your compliance needs.
  • Document the complete request path, including supporting services and remote access; an endpoint region alone does not establish residency.

Try Compute today

Run a self-managed inference server on Compute when your team wants to operate the serving stack. Choose from locations currently available for the workload, then confirm the full data path and contractual responsibilities before using the deployment for a residency requirement.

Introduction to LLM inference

LLM inference is when computers use large language models to understand and create human-like text. It's the tech that runs your chatbots, translation tools, and automated writing helpers. Customer support systems rely on it too. When you're using LLM inference in your organization, data protection becomes crucial—especially with sensitive information. You need clear policies about how long to keep data, how to protect it, and when to delete it safely. European Union regulations demand this. GDPR's core data processing principles apply to each stage of an LLM's lifecycle, from training to deployment. Build strong data protection into every step of your LLM process. This reduces risk and shows you handle sensitive data responsibly. However, the 'black-box' nature of LLMs complicates the ability to explain how personal data influences their outputs, making compliance with data subject rights difficult. The right of access under GDPR allows individuals to know if their data is being processed, but the complex structure of LLMs complicates this further. Additionally, LLMs can perpetuate biases or produce inaccurate outputs, which can violate principles of fair processing under GDPR.

Core principles (keep it simple)

• Data minimization. Collect only what you need to serve the request.
• Purpose limitation. Define each use of prompts and outputs and establish a valid lawful basis. Consent or contract wording is not blanket permission for unrelated uses.
• Storage limitation. Keep logs and traces for the shortest useful period.
• Security by default. Enforce TLS, scoped keys, and least‑privilege access.

• Putting these principles into practice is essential for effective data privacy and compliance. Organizations must conduct Data Protection Impact Assessments (DPIAs) before implementing LLMs that may pose high risks to individuals' rights. Additionally, organizations need to conduct risk assessments to identify privacy risks throughout the AI development lifecycle. AI technologies increase privacy risks by enhancing data collection and analysis.

Review Compute for your workload

Check current hardware, locations, pricing and deployment requirements before choosing an instance.

Try Compute now

Residency and location

• Choose a deployment region that meets the workload’s requirements, then verify routing, storage, backups, support access and subprocessors. An EU endpoint does not guarantee EU-only processing.
• Document data flows (client → gateway → inference → storage).
• Assess cross-border backups and access against the full processing arrangement, including any applicable transfer requirements; a contract alone does not settle them.

Logging and retention

• Log counts and timings, not raw text. Prefer: prompt_tokens, output_tokens, TTFT, TPS, error codes. Only retain data that is necessary for operational purposes.
• If you must log text for debugging, sample sparingly, redact, and store separately with tighter controls. Retain records only when required for debugging, and ensure secure storing of such records.
• Set documented retention periods by data type and purpose, with deletion controls. A 7–30-day period is an operational example, not a GDPR rule or safe harbor. Review it when processing or legal requirements change.
• Tag logs by region and environment; keep EU logs in EU storage. Tag logs to track when and how data was collected. AI models can inadvertently expose sensitive data, leading to accidental data leakage.

Data subject rights (DSRs)

• Build a process to locate relevant records and assess access or erasure requests. Check applicable exceptions and response deadlines. Deleting inference logs does not by itself remove information learned during model training.
• Use restricted request identifiers where needed for investigation. Hashed user IDs may still be personal data; pseudonymization is not anonymization.
• Document who approves deletions and how long they take.

Inputs, outputs, and redaction

• Treat prompts and outputs as personal data unless proven otherwise.
• Redact obvious PII before storage; avoid pasting secrets into prompts.
• Blocklist dangerous patterns (access keys, card numbers) at the gateway when possible.
• Train staff not to publish customer prompts without authorization and a valid legal basis. Prefer synthetic or genuinely anonymized examples; assess re-identification risk instead of assuming those labels remove all obligations.

Sensitive data handling

You're handling sensitive data when you deploy LLM inference systems, and that's a big responsibility. These models process personally identifiable information, confidential business records, and other sensitive data that need strong protection. You'll want to put strict safeguards in place. Encrypt your data when it's stored and when it moves between systems. Set up detailed access controls so only the right people can see what they need. Use secure storage that you can trust. Here's what's crucial: create clear rules for how long you keep different types of sensitive data. Define specific timeframes, then delete that data securely when you no longer need it. Sensitive information is increasingly collected to create and fine-tune AI and machine learning systems. LLMs can memorize personal information from training data, which raises privacy risks. These controls can reduce risk, but do not establish compliance on their own. Assess the applicable legal basis and any additional conditions for special-category personal data.

Access and keys

• Use per‑service API keys with least privilege and rotation, implementing a secure system for managing access and keys.
• Restrict SSH/Jupyter access to named engineers, with MFA and short‑lived credentials.
• Keep an allowlist for admin ports; keep inference to HTTPS only.
• Store keys in a secrets manager; never in code or chat logs. Data exfiltration from AI applications poses a significant privacy risk if sensitive data is targeted by attackers.

Subprocessors and contracts

• Identify each vendor’s actual role. Use an Article 28-compliant agreement where a vendor processes personal data on your behalf; a DPA is not a substitute for assessing that role.
• For transfers outside the EEA, identify the applicable Chapter V route, such as an adequacy decision or appropriate safeguards. SCCs are one option, not the only one.
• Track subprocessors and meet the applicable authorization, information and change-notification duties. Do not treat a public list as a universal substitute for those obligations.

Risk assessments

You need regular risk assessments when you're using LLM inference. They're vital. These checks help you spot and fix threats to your data privacy and security before they become problems. Look for weak spots like data breaches, unauthorized access, and gaps where your data retention policies don't quite work. Review how you keep records. Make sure retention periods match what the law requires and what your business needs. Can you access records when you need them? Can you delete them? You should be able to do both. Conducting audits is essential to understand personal data processed by LLMs and to ensure compliance with data minimization. When you identify risks step by step and put targeted security measures in place, you'll strengthen how well you meet compliance requirements. You'll reduce the chance of incidents happening. Your data retention practices will stay effective and current.

Transparency and lawful basis

Explain what personal data you process, why, for how long, and with whom it is shared. Document the appropriate lawful basis for each purpose. Consent is one basis; it is not required for every processing activity. Where you rely on consent, make sure it meets the GDPR conditions and can be withdrawn.

Incident response

• Define what a privacy incident is for your LLM stack, and dedicate appropriate resources to incident response and compliance.
• Keep a 24/7 escalation path and run one tabletop per quarter.
• Pre‑draft customer notifications and regulator checklists to save time.
• After incidents, shorten retention or add gate checks where failures occurred.

Try Compute today

Use Compute for a self-managed vLLM deployment when the available location and operating model fit the workload. Set output caps and define logging deliberately. Confirm request handling, retention, subprocessors, and support access separately.

Data Protection Officer

Assess whether a Data Protection Officer is required under Article 37 and applicable national rules. Mandatory cases include relevant public bodies and core activities involving large-scale regular monitoring or large-scale special-category or criminal-offense data processing. LLM use alone does not decide the question. A DPO advises and monitors independently; the organization remains accountable.

A practical GDPR playbook for LLM inference

Build the review around the actual data path: what enters the system, what is stored, who can access it, how long each record is kept, which supporting services are involved, and what the contract says. A regional deployment can be one control, but GDPR responsibility depends on the complete processing arrangement.

A robust data retention policy is essential for both businesses and consumers, as it addresses privacy concerns and supports compliance with applicable privacy requirements. The European Commission plays a significant role in shaping regulation, such as the GDPR, which sets strict requirements for data handling and retention. Factors like business requirements, legal mandates, and risk analysis all influence decision making around enterprise data retention, requiring ongoing analysis to balance operational needs with regulatory obligations. Effective management of enterprise data helps businesses meet compliance standards and protect consumers' privacy rights.

Retention of internet data, including metadata and online activities, raises additional privacy concerns due to the involvement of national authorities, security services, and the criminal justice system in surveillance and law enforcement. For example, medical treatment data, such as patient records and photos, may be subject to GDPR requirements, and improper use in AI training datasets can lead to significant privacy concerns for individuals.

FAQ

Is running the endpoint in the EU enough for GDPR?

No. Residency helps, but you still need lawful basis, minimization, security controls, retention limits, and a DSR process.

Are prompts personal data?

Often yes. Prompts can include names, emails, or free‑text that identifies someone. Handle as personal data unless you are certain they do not.

Can we train or fine‑tune on customer prompts?

Do not reuse customer prompts for training solely because the terms mention it or an opt-out exists. Assess the lawful basis, purpose compatibility, transparency and any special-category conditions for that separate use.

How long should we keep logs?

Keep personal data only for the period justified by its purpose and applicable obligations. Document that period and apply deletion controls; GDPR does not prescribe one default number of days for LLM logs.

Do we need SCCs if everything stays in the EU?

Assess whether a Chapter V transfer occurs, including remote access by another organization outside the EEA. Physical EU hosting alone does not answer that question. Where a transfer occurs, follow an applicable transfer route; SCCs are not always the required route.

How do we handle right to erasure with streamed logs?

Keep only the identifiers needed for a documented purpose, protect any mapping, and treat linkable hashed IDs as personal data. Assess erasure requests and applicable exceptions, then delete records where required and document the outcome.

Do inference vendors act as processors or controllers?

A vendor may be a processor for work performed on your instructions and a controller for its own purposes. Assess each processing activity and the actual controller and processor roles, not only the contract label.

Is this legal advice?

No. It is practical guidance for engineers. Work with counsel for your specific obligations.

‍