Question 12 of 53
Is our training data compliant with global privacy laws?
By Cody Maxwell · AI Governance Institute · February 2026 · Updated September 2026
Check rights to training data and identify personal information in datasets. Review GDPR and EU AI Act data obligations.
If you only do 3 things, do this:
- 1.Before using any dataset for training, document the source, the original collection purpose, and whether training use is compatible with that purpose under GDPR's purpose limitation principle.
- 2.Check every training dataset for personal information: datasets that look anonymized often contain combinations of details, such as zip code, birth date, and job title, that can be pieced together to identify people.
- 3.Retain records of every dataset used to train each model version. You will need this when a data subject makes an erasure request or a regulator conducts an AI inquiry.
The Situation
Who this is for: Data science teams, privacy officers, and legal counsel involved in AI model development
When you need this: Before initiating any AI training run, or when auditing existing models for privacy compliance
The Decision
Do we have a lawful basis to use this data for this training purpose, and what are our obligations to data subjects whose data was used?
The Steps
- 1Inventory every dataset used in model training or fine-tuning, including its source and original collection context
- 2For each dataset, document the legal basis for collection and assess whether training use is compatible with that purpose
- 3Assess the dataset for personally identifiable information (PII): direct identifiers like names, combinations of details that could identify someone (quasi-identifiers), and sensitive categories of data
- 4Apply data minimization: remove unnecessary fields, group or summarize data where possible, and consider adding statistical noise so individuals cannot be singled out (differential privacy)
- 5Document your approach to data subject rights (erasure, access) for training data, including whether and how you can remove a person's data from an already-trained model (machine unlearning)
- 6Retain provenance records for each training dataset as part of the model's documentation
The Artifacts
- —Training data provenance record template (source, collection basis, training compatibility assessment, PII assessment)
- —PII assessment methodology (direct identifiers, quasi-identifiers, sensitive categories)
- —Data minimization checklist for training datasets
- —Data subject rights response template for training data
- —EU AI Act training data governance checklist (for high-risk systems)
The Output
A documented legal basis for each training dataset, a PII assessment on file, data minimization measures applied, and provenance records retained with the model.
The right to use data for training is not implied
Collecting data lawfully and using it to train an AI model are two different things. GDPR's purpose limitation principle requires that personal data be collected for specified, explicit, and legitimate purposes and not processed in a manner incompatible with those purposes. If you collected customer data to provide a service, using it to train a model that optimizes a different process may not be compatible with the original purpose without a fresh legal basis.
Before using any dataset for AI training, document the source of the data, the legal basis under which it was collected, whether the subjects were informed of or consented to this use, and whether the intended training use is compatible with that legal basis. This analysis should be completed before training begins, not after.
PII identification and minimization
Datasets that appear anonymized can often be traced back to individuals when combined with other data or analyzed by powerful models. Before using any dataset for training, assess it for personal information, covering direct identifiers (names, email addresses, account numbers), quasi-identifiers (combinations of attributes that could identify individuals), and sensitive categories (health, financial, racial or ethnic origin, political opinion).
Apply data minimization before training: remove fields that are not necessary for the model's intended function, aggregate or generalize where precision is not required, and consider differential privacy techniques, which add statistical noise so no single person can be picked out, for datasets with a high risk of re-identification. The EU AI Act requires that training data for high-risk AI systems be subject to appropriate data governance practices, including examination for biases and relevance.
Ongoing obligations after training
Privacy obligations do not end when training is complete. Data subject rights, including the right to erasure, apply to personal data used in training. Removing one person's data from a model that has already been trained (machine unlearning) remains technically difficult for large models. Even so, organizations should document their approach to data subject requests involving training data and be prepared to justify their position to regulators.
Retain records of the datasets used to train each model version, including data provenance, processing steps applied, and the legal basis for use. This documentation is required by the EU AI Act for high-risk systems and is increasingly expected by data protection authorities conducting AI-related inquiries.
Turn this guidance into an implementation plan
Get the free Excel tracker for all 132 governance controls. Score maturity, assign owners, and set deadlines, including this playbook's 4 related controls.
- 132 controls in Excel
- Score maturity and assign owners
- Track deadlines and regulation coverage
Includes AI Governance Weekly every Thursday. Unsubscribe anytime.
Governance Controls
Operational controls that implement the guidance in this playbook.
Related frameworks
Recent Coverage
News and developments relevant to this playbook topic.