What is differential privacy?
Differential privacy is a mathematical guarantee that the output of a computation, such as a published statistic or a trained model, changes almost imperceptibly whether or not any single person’s record is included. Because the result barely depends on any one individual, nobody can learn much about that individual from it, regardless of what other information they hold. That last point separates it from older anonymization, which failed repeatedly when stripped records were matched against outside data.
How differential privacy works
The mechanism adds carefully calibrated random noise. For a published count or average, noise is added to the result. For model training, noise is added to the gradient updates during each step, a method known as differentially private stochastic gradient descent. The amount of noise is governed by a parameter called epsilon, the privacy budget. A smaller epsilon means stronger privacy and less accurate results. Every query against the data spends part of the budget, and the guarantee holds only while the total stays within the agreed limit.
The U.S. Census Bureau used differential privacy to protect its published population tables, and large technology companies use it when collecting usage statistics from devices. In business it appears when statistics derived from sensitive records must be published or shared, and when a model trained on personal data will be handed to a partner or exposed to many users.
Differential privacy in practice
A county government publishes monthly statistics on social service usage by neighborhood. In small neighborhoods, a count of three cases could identify specific households. The data team applies differential privacy to every published cell, documents the epsilon it chose, and tracks the cumulative budget across releases. Counts for large areas barely change, while small-area counts carry visible noise, and the publication says so to keep analysts from over-reading them.
What to watch for
- Choosing epsilon is a policy decision for leadership and counsel, since it trades privacy against accuracy.
- The budget is cumulative, so an open-ended stream of queries eventually exhausts the guarantee.
- Accuracy suffers most for small groups and rare events, which are often the groups a fairness review cares about.
- A vendor claim of differential privacy without a stated epsilon and mechanism carries no information.
- Differential privacy protects individuals rather than trade secrets or aggregate business facts, so access control is still required.