Privacy by design for AI begins before model training and continues after deployment. AI systems can copy, transform, infer, retain, retrieve, and expose personal information in ways that are difficult to see from a traditional application data-flow diagram. Treating privacy as a launch-time checklist misses the architectural decisions that determine what data enters the system, what the model can memorize or infer, what users can retrieve, and how information is removed when the purpose ends.<\/p>\n
IAPP\u2019s AIGP body of knowledge explicitly includes privacy-law intersections and governance of data collection and use during AI development and deployment. The IAPP AIGP<\/a> includes privacy-law and data-governance decisions across AI development and deployment. AI lifecycle governance<\/a> covers the broader lifecycle, while privacy engineering narrows the problem to data collection, use, retention, access, provenance, and deletion choices.<\/p>\n Before collecting data for an AI use case, define the purpose precisely enough to evaluate necessity. \u201cImprove the model\u201d or \u201cenable personalization\u201d is too vague to justify every available field. Identify the decision or service the system supports, which data elements are needed, and which uses are outside scope.<\/p>\n Purpose should also constrain later reuse. A dataset gathered for fraud detection may not automatically be appropriate for employee performance scoring or marketing. AI teams often discover new capabilities after data is centralized; privacy by design requires a governance step before expanding use rather than treating technical possibility as permission.<\/p>\n Minimization asks whether the system needs each field, granularity, history window, identifier, and data source. Removing unnecessary information reduces breach impact, inference risk, bias pathways, retention burden, and the number of controls that must remain effective. It can also simplify model behavior by removing noisy or proxy variables.<\/p>\n Minimization does not mean using the smallest dataset regardless of performance. It means using the least personal data reasonably necessary for the defined purpose and documenting trade-offs. Where full identifiers are not required, pseudonymization, aggregation, tokenization, or separate lookup services can reduce direct exposure while preserving utility.<\/p>\n AI systems may combine first-party records, licensed datasets, public content, synthetic data, user prompts, feedback, and third-party corpora. Privacy teams need to know where data came from, what rights or notices apply, how it was transformed, and which model or index consumed it. Without provenance, it becomes difficult to answer access, deletion, correction, licensing, or incident questions.<\/p>\n Lineage should extend beyond model training. Evaluation datasets, fine-tuning data, retrieval indexes, prompt logs, and monitoring samples may contain personal information too. A design that protects the training warehouse but leaves prompt telemetry indefinitely retained has only moved the privacy risk.<\/p>\n Health, biometric, financial, location, children\u2019s, employment, identity, or other sensitive data may trigger stronger legal and ethical expectations. Even when a sensitive field is removed, models can infer sensitive characteristics from proxies. The assessment should therefore consider both direct collection and inference capability.<\/p>\n Controls can include exclusion, separate processing, stronger access, explicit consent where appropriate, limited retention, differential review, output restrictions, and testing for proxy effects. The choice should follow the use case and applicable law. \u201cThe model never sees the sensitive column\u201d is not enough if correlated inputs reproduce the same information.<\/p>\n Traditional systems store records in databases that can be located and deleted. Models can encode patterns from training data, and generative systems may reproduce unusual or sensitive content under certain prompts. Risk depends on model type, dataset, training method, access, and adversarial effort, but privacy design should consider whether information can be extracted even after the source record is removed.<\/p>\n Techniques such as deduplication, filtering, privacy-preserving training, access control, rate limiting, red-team testing, and output monitoring can reduce exposure. Governance should also define what deletion can realistically mean for the system and whether retraining, index deletion, or other remediation is required.<\/p>\n RAG systems can reduce some model-knowledge problems by retrieving enterprise sources at inference time, but they introduce document-level access and query-logging concerns. The retrieval layer must enforce authorization so a user cannot retrieve documents merely because the model can technically index them.<\/p>\nPurpose limitation should determine which data is allowed into the system<\/h2>\n
Data minimization reduces privacy and model risk together<\/h2>\n
Provenance should follow data into training, evaluation, and retrieval<\/h2>\n
Sensitive attributes require explicit handling decisions<\/h2>\n
Model memorization and extraction create a different privacy boundary<\/h2>\n
Retrieval systems need privacy controls on documents and queries<\/h2>\n