How do we measure and mitigate algorithmic bias?

Algorithmic bias is not a single phenomenon. It can manifest as disparate treatment (the model uses a protected characteristic as an input), disparate impact (the model produces systematically different outcomes for protected groups even without using the characteristic directly), or intersectional bias (the model performs differently for individuals who belong to multiple protected groups simultaneously).

Before testing, define what fairness means for your specific use case. Several mathematically precise fairness definitions exist, and they are often mutually incompatible. Demographic parity requires equal selection rates across groups. Equalized odds requires that the system be equally accurate for each group: it should correctly approve qualified people, and wrongly approve unqualified ones, at the same rates across groups. Individual fairness requires that similar individuals be treated similarly. Choosing a definition involves value judgments about what kind of error is most harmful, and that choice should be made explicitly, not left to default.

The four-fifths rule (also called the 80% rule) from the Equal Employment Opportunity Commission's (EEOC) Uniform Guidelines provides a widely accepted starting point for employment contexts: if the selection rate for any group is less than 80% of the rate for the most selected group, adverse impact is indicated. This is a screening tool, not a legal standard, but it provides a defensible threshold for triggering further investigation.

For models that make yes/no decisions, measure errors separately for each protected group. Compare how often each group is wrongly approved, how often each is wrongly rejected, and overall accuracy. Significant differences in error rates across groups indicate bias that may produce discriminatory outcomes even if overall accuracy is high. For scoring models used in credit or risk assessment, compare score distributions and cutoff outcomes across groups.

Bias remediation options depend on where the bias originates. Bias in the training data may be addressed by rebalancing it: adding examples, removing duplicates, or giving underrepresented groups more weight. Other techniques build fairness rules into the model while it is being trained. A third option adjusts the model's outputs after the fact to meet the chosen fairness definition.

Document every bias finding and every remediation step. Record the metrics before and after remediation, the technique applied, and the rationale for choosing it. Bias testing and remediation records should be retained as part of the model's audit trail and reviewed whenever the model or its deployment context changes materially.