Question 13 of 34
How do we measure and mitigate algorithmic bias?
Published by AI Governance Institute · Practical Governance for Enterprise AI
Standardized metrics for testing whether a model unfairly discriminates against protected groups, and processes for remediation when bias is found.
If you only do 3 things, do this:
- 1.Choose your fairness definition before you start testing — demographic parity, equalized odds, and individual fairness are mathematically incompatible. This is a values decision, not a technical one.
- 2.Run the four-fifths rule as a first-pass screen on every system that makes decisions about people. If any group's selection rate is below 80% of the top group, investigate further.
- 3.Document before-and-after metrics for every remediation step. Without these records you can't defend your testing if challenged.
The Situation
Who this is for: Data science, ML engineering, and compliance teams responsible for fairness testing
When you need this: Before deploying any AI system that makes decisions about individuals, and periodically after deployment
The Decision
Are we measuring the right kind of fairness for our specific use case, and are our remediation records defensible?
The Steps
- 1Define the fairness criteria for the specific use case (demographic parity, equalized odds, individual fairness, or a combination)
- 2Identify all relevant protected characteristics and intersectional combinations to test
- 3Run the four-fifths rule as a screening test on selection rates by group
- 4Compute group-specific confusion matrices for classifiers; compare score distributions for scoring models
- 5If bias is detected, diagnose the source: training data, proxy variables, or model design
- 6Select and apply remediation technique(s); re-test; document before-and-after metrics
- 7Schedule periodic re-testing on the same cadence as model reviews
The Artifacts
- —Fairness definition selection worksheet (use case criteria → recommended fairness metric)
- —Disparate impact analysis template (selection rates, four-fifths calculator)
- —Confusion matrix by subgroup template
- —Bias remediation decision tree (bias source → technique options)
- —Bias testing audit trail (findings, remediation, before/after metrics, re-test date)
The Output
Documented fairness definitions for every decision-making AI system, pre-deployment bias assessments on file, and a re-testing schedule aligned with model review cadence.
Defining bias in measurable terms
Algorithmic bias is not a single phenomenon. It can manifest as disparate treatment (the model uses a protected characteristic as an input), disparate impact (the model produces systematically different outcomes for protected groups even without using the characteristic directly), or intersectional bias (the model performs differently for individuals who belong to multiple protected groups simultaneously).
Before testing, define what fairness means for your specific use case. Several mathematically precise fairness definitions exist, and they are often mutually incompatible. Demographic parity requires equal selection rates across groups. Equalized odds requires equal true positive and false positive rates. Individual fairness requires that similar individuals be treated similarly. Choosing a definition involves value judgments about what kind of error is most harmful, and that choice should be made explicitly, not left to default.
Standardized testing metrics
The four-fifths rule (also called the 80% rule) from EEOC Uniform Guidelines provides a widely accepted starting point for employment contexts: if the selection rate for any group is less than 80% of the rate for the most selected group, adverse impact is indicated. This is a screening tool, not a legal standard, but it provides a defensible threshold for triggering further investigation.
For binary classification models, compute confusion matrices separately for each protected group and compare false positive rates, false negative rates, and overall accuracy. Significant differences in error rates across groups indicate bias that may produce discriminatory outcomes even if overall accuracy is high. For scoring models used in credit or risk assessment, compare score distributions and cutoff outcomes across groups.
Remediation and documentation
Bias remediation options depend on where the bias originates. Training data bias may be addressed by resampling, reweighting, or augmenting the dataset. In-processing techniques modify the learning algorithm to incorporate fairness constraints during training. Post-processing techniques adjust model outputs after prediction to achieve fairness criteria.
Document every bias finding and every remediation step. Record the metrics before and after remediation, the technique applied, and the rationale for choosing it. Bias testing and remediation records should be retained as part of the model's audit trail and reviewed whenever the model or its deployment context changes materially.
Governance Controls
Operational controls that implement the guidance in this playbook.