TLDR:
Algorithmic bias refers to systematic errors in AI system outputs that produce unfair or discriminatory results, typically disadvantaging protected groups defined by characteristics like race, gender, age, or disability. Bias is a major legal, ethical, and operational concern across AI deployments, particularly in hiring, lending, healthcare, and criminal justice.
Sources of Bias
Algorithmic bias enters AI systems through several pathways: training data bias (historical data reflecting past discrimination), sampling bias (underrepresentation of certain groups), labeling bias (subjective labels reflecting annotator prejudice), proxy variables (seemingly neutral features that correlate with protected attributes—e.g., zip code as a proxy for race), measurement bias (the same outcome measured differently across groups), and deployment bias (a model trained for one population deployed in another). Bias can compound across the AI pipeline from data collection through deployment.
Detection and Mitigation
Bias detection uses statistical fairness metrics: demographic parity (equal positive prediction rates across groups), equalized odds (equal true positive and false positive rates), calibration (predicted probabilities match observed outcomes equally across groups), and individual fairness (similar individuals receive similar predictions). These metrics often conflict mathematically, requiring trade-offs. Mitigation techniques include rebalancing training data, fairness-aware learning algorithms, post-processing model outputs, and ongoing monitoring with disaggregated metrics.
Legal and Regulatory Obligations
Multiple legal frameworks impose obligations to address algorithmic bias: in the US, Title VII (employment), the Fair Housing Act, the Equal Credit Opportunity Act, and Section 1557 (healthcare) prohibit discriminatory outcomes; the EEOC has issued specific guidance on AI hiring tools; the EU AI Act classifies many bias-relevant systems as high-risk; the GDPR grants rights to challenge automated decisions. Failures have produced significant litigation—the EEOC’s first AI hiring discrimination suit settled in 2023, and class actions are increasingly common against algorithmic decision systems.