GDPR-compliant AI automation: how to scope an AI agent that touches customer data
Lawful basis, minimisation, where the model runs, processor agreements, retention, human review, DPIAs, AI disclosure and subject access requests, with a checklist.
By Thoufic, founder of Flange ·
Start with what the agent actually does
Most GDPR problems with AI projects are not about the model. They come from nobody writing down what data the system touches. Before you choose a provider or write a prompt, map the flow for the one process you want to automate: what personal data comes in, where it is sent, who can see it, what the system decides, and how long each piece is kept. A support chatbot answering questions about opening hours touches very little. An AI agent that reads customer emails, looks up orders, issues refunds and writes notes into a CRM touches a lot.
That map is the basis for every decision below, and it is the first thing a data protection officer or lawyer will ask for. This guide is written for businesses in the EU and the UK. It is practical guidance, not legal advice: take your own legal advice, and have whoever is responsible for data protection in your organisation sign off the design.
1. Pick a lawful basis for each purpose
You need a lawful basis under Article 6 for each purpose, not one for the project as a whole. The common ones for AI agents:
- Contract. Handling a customer's own request: tracking their order, rebooking their appointment, answering their support ticket.
- Legitimate interests. Qualifying inbound leads, routing enquiries, summarising calls for your team. Write a short legitimate interests assessment showing the processing is necessary and does not override the person's interests.
- Consent. Usually for electronic marketing, and for anything people would not reasonably expect.
If the agent may handle health data, such as a clinic's booking line, or any other special category data, you also need an Article 9 condition. That raises the bar for everything else in this list.
2. Minimise what the model sees
An agent only needs the fields required for the task in front of it. Don't send the whole customer record to the model when the task needs a name and an order number. Practical rules:
- Fetch data from your systems per task, rather than loading entire databases into the AI's context.
- Strip or mask fields the model does not need, such as dates of birth, full addresses or card numbers.
- Never let the agent collect payment card details in chat or on a call. Send a secure payment link.
- For a knowledge assistant built on your documents, keep personal files out of the index unless the use case truly needs them, and respect existing access rights.
3. Decide where the model runs and where data is stored
You have three broad options, and the right one depends on how sensitive the data is.
- A major provider's API with EU or UK data processing. Several providers offer regional processing and settings that stop your data being used for training. Check what the contract actually says, including how long prompts and outputs are retained for abuse monitoring.
- A model hosted in an EU or UK cloud region under your own cloud account, so data stays in infrastructure you already control.
- An open-weights model you host yourself, for the most sensitive cases. More control, more work to run and keep up to date.
If any data leaves the EEA or the UK, you need a transfer mechanism: an adequacy decision (such as the EU-US Data Privacy Framework for certified US companies, or the UK's extension of it), or standard contractual clauses with a transfer risk assessment. Remember the whole chain: the model provider, the telephony provider for a voice agent, the transcription service and your hosting. We agree where data is stored on the first call, before anything is built.
4. Sign data processing agreements with every processor
Under Article 28, each company that processes personal data on your behalf needs a written agreement with you. For a typical AI agent that includes the model provider, the hosting provider, any speech-to-text or voice provider, the messaging or telephony platform, and the team that builds and maintains the system if they can access production data. Most large providers publish a standard data processing agreement; read the sub-processor list as well, because that is where your data may travel next.
5. Set retention for transcripts and logs
AI systems create a lot of new personal data: chat transcripts, call recordings, summaries, extracted fields, scores and debug logs. Decide how long each kind is kept and delete it automatically.
- Call recordings are often the most sensitive. Keep them for the shortest period you can justify.
- Summaries written into your CRM follow your normal customer-record retention.
- Debug and prompt logs should be short-lived and access-controlled, with personal data masked where possible.
6. Keep a person on decisions that matter
Under Article 22 of the EU GDPR, people have the right not to be subject to a decision based solely on automated processing that has legal or similarly significant effects on them, with limited exceptions that require safeguards: the right to a human review, to express their view and to contest the decision. Rejecting a credit application, a tenancy or a job applicant are the classic examples. Answering questions or booking appointments are not.
The UK has moved on from the original text. The Data (Use and Access) Act 2025 replaced Article 22 in the UK GDPR with new Articles 22A to 22D, which allow more automated decision-making but keep safeguards: telling people about the decision, letting them make representations and contest it, and offering human intervention. The stricter rules remain where special category data is involved.
The design answer is the same in both places: let the agent gather information, recommend and prepare, and route anything significant to a person who can change the outcome. That is also simply good practice.
7. Do a DPIA when the risk is high
Article 35 requires a data protection impact assessment where processing is likely to result in a high risk. AI projects often tick several of the warning signs regulators list: new technology, processing at scale, special category data, systematic monitoring, or automated decisions with significant effects. If you tick two or more, assume a DPIA is needed. Even when it isn't strictly required, a short one is the cleanest way to record your decisions on everything in this guide.
8. Tell people they are talking to an AI
Update your privacy notice to describe the AI processing, under Articles 13 and 14. Separately, say at the start of every chat or call that the person is talking to an AI assistant and how to reach a person. In the EU this is now a legal duty in its own right: the AI Act's transparency rules for systems that interact with people have applied since 2 August 2026. Our guide to the EU AI Act for small businesses covers what that means for a chatbot or voice agent.
9. Make AI logs answerable to subject access requests
Transcripts, recordings, summaries and lead scores are personal data, so they fall within a subject access request under Article 15, which you normally have one month to answer. Erasure requests reach them too. Build for this from the start:
- Tag every transcript, recording and log entry with a customer or contact identifier.
- Give your team one place to search everything the AI system holds about a person, and export it.
- Make deletion work across every store, including the vector index of a knowledge assistant.
A practical checklist
- Data flow map for the process: inputs, systems, outputs, people, retention.
- Lawful basis per purpose, plus an Article 9 condition if special category data is possible.
- Legitimate interests assessment where you rely on legitimate interests.
- Field-level minimisation: what the model sees and what it never sees.
- Hosting and model location agreed, with transfer mechanisms for anything leaving the EEA or UK.
- Data processing agreements and sub-processor lists for every provider in the chain.
- Retention periods for recordings, transcripts, summaries and logs, with automatic deletion.
- Human review for any decision with legal or similarly significant effects.
- DPIA completed or a recorded reason why one isn't needed.
- Privacy notice updated, and AI disclosure at the start of every conversation.
- Subject access, correction and erasure tested end to end before go-live.
- An owner for the system after launch, and a review date.
Mistakes to avoid
- Testing on real customer data before the paperwork exists. Use synthetic or anonymised samples until the agreements are signed.
- Forgetting the voice and messaging layer. The model provider is not the only processor. Telephony, transcription and messaging platforms all count.
- Keeping everything forever "for training". If you have no specific plan for the data, you have no reason to keep it.
- Letting the agent decide what a person should. Refusals, eligibility and anything with a significant effect belong with your team.
How we approach it
We build agents, support assistants and knowledge assistants for businesses in the EU and UK, and data handling is part of scoping, not an afterthought. We agree where data is stored on the first call, design for minimisation and human review, and give your data protection lead what they need to sign off. We don't make compliance promises on your behalf; that decision stays with you and your advisers. See how we work with businesses in the European Union and in the United Kingdom. The full text of the GDPR and the ICO's UK GDPR guidance are the primary sources.
FAQ
Common questions
Can we use a US-based AI model provider under GDPR?
Often, yes, with the right safeguards: a data processing agreement, a transfer mechanism such as the EU-US Data Privacy Framework for certified companies or standard contractual clauses, and settings that limit retention and training use. Some providers also offer EU or UK data processing. Take your own legal advice.
Do we need a DPIA for an AI chatbot?
Not always. A chatbot that answers general questions with little personal data may not need one. If the system processes special category data, works at scale, monitors people or makes significant automated decisions, assume you do. A short DPIA is good practice either way.
Does Article 22 stop us using AI to make decisions about customers?
It restricts decisions based solely on automated processing that have legal or similarly significant effects, such as refusing credit or a tenancy. Answering questions and booking appointments are not in scope. In the UK, Articles 22A to 22D now apply instead, with similar safeguards.
Are chat transcripts and call recordings covered by subject access requests?
Yes. Transcripts, recordings, summaries and scores about a person are personal data. Tag them by customer so they can be found, exported and deleted within the deadline.
Do we have to tell people they are talking to an AI?
You should always say so, and in the EU the AI Act has required it for systems that interact with people since 2 August 2026, unless it is already obvious. Your privacy notice should also describe the AI processing.