Sensitive Data Handling in AI Pipelines
Added May 2026
Prevent personally identifiable information, credentials, health data, and other sensitive content from entering AI models, prompts, or logs inappropriately.
Objective
Reduce the risk of data exposure through AI systems by enforcing data classification and handling requirements at every stage where data enters, moves through, or leaves an AI system.
Maturity Levels
Initial
No controls prevent sensitive data from entering AI prompts or being stored in logs.
Developing
Engineers are aware of sensitive data risks but handling controls are inconsistent.
Defined
Documented data handling rules specify which data categories may be used in AI inputs, with technical enforcement where feasible.
Managed
Sensitive data flows through AI systems are mapped and reviewed; violations are tracked.
Optimizing
Automated data classification and redaction are applied in real time before data reaches AI models.
Evidence Requirements
What an auditor or assessor would expect to see for this control.
- —AI data flow map documenting what data enters prompts, AI models, logs, and training pipelines at each stage
- —PII detection and redaction configuration with sample test results demonstrating effectiveness
- —Vendor contract or business associate agreement (BAA) confirming data use and retention restrictions with each AI provider
- —Log scrubbing configuration and sample scrubbed log entries demonstrating sensitive content removal
- —Data Protection Officer (DPO) or Privacy Officer approval records for any exceptions permitting sensitive data in AI training or prompts
Implementation Notes
Key steps
- Map your AI data flows: what data enters prompts, what gets logged, what is used for fine-tuning (further training a model on your own data)? Most organizations underestimate how much personally identifiable information (PII) flows through these channels.
- Apply automatic PII detection and redaction before data is sent to an AI model, especially in retrieval-augmented generation (RAG) pipelines, which feed your internal documents to the model.
- Review your vendor contracts: most AI providers state that prompts may be used for model improvement; ensure sensitive data is not inadvertently being shared.
- Implement logging policies that scrub sensitive content from AI logs while preserving sufficient context for security monitoring.
Example Implementation
Healthcare company using a RAG pipeline to assist clinicians with documentation
Sensitive Data Handling Policy: Clinical Documentation RAG Pipeline
Data classification in scope: PHI, patient identifiers, diagnosis codes, treatment records
Pipeline controls:
| Stage | Control | Implementation |
|---|---|---|
| Document ingestion | PII detection | Automated scan with Microsoft Presidio before indexing |
| Prompt construction | Field-level redaction | Patient name replaced with [PATIENT], DOB with [DATE] in prompts sent to external model API |
| Model API calls | Vendor data use | BAA in place with model provider; zero-retention API tier selected |
| Output logging | Log scrubbing | PII detector runs on all logged outputs; flagged content stored in restricted PHI log store |
| Fine-tuning | Training data review | PHI explicitly prohibited in fine-tuning datasets; DPO sign-off required before any training run |
Exception process: Any case where unredacted PHI must be sent to external model requires written Privacy Officer approval and is logged separately