AI Bias & Fairness Assessment

Purpose

A Bias & Fairness Assessment is a structured evaluation of an AI system's potential to produce unequal outcomes across demographic groups. Where the AI Impact Assessment covers seven impact dimensions broadly, the Bias & Fairness Assessment goes deep on one: it identifies which protected characteristics are at risk, selects the fairness metrics appropriate for the decision type, runs pre-deployment tests against those metrics, records the results, and establishes an ongoing monitoring commitment to detect fairness drift after deployment.

This is not a checklist of fairness principles — it is an operational testing and monitoring document. The assessment records what was tested, what data was used, what the results showed, and what the organization decided to do with those results. That record is what makes fairness governance defensible to regulators, auditors, and affected communities.

The assessment is universal — relevant to any AI system that makes or informs decisions about people, regardless of PM methodology. The depth of the assessment should scale with the system's risk level and the consequentiality of its decisions.

Where It Fits in Your Document Pack

Fairness concerns identified in the impact assessment — demographic disparities in outputs, proxy variables for protected characteristics, gaps in training data representativeness — become the scope for this assessment. The assessment then feeds directly into the TEVV Plan (which metrics to test and at what threshold), the Risk Register (unfair outcome risk entries), the Human Oversight Plan (which outputs require human review when fairness indicators trigger), and the Monitoring Plan (which fairness metrics to track post-deployment).

For systems where the Impact Assessment determines fairness impact is low or not applicable, a lightweight version of this assessment is still valuable: it documents that the scoping decision was made deliberately, not overlooked.

What This Template Covers

• Assessment identification: assessor, date, version, linked AI system, and system type
• Fairness scope decision: which protected characteristics are in scope, which are out of scope, and the documented rationale for each scoping decision
• Data & testing environment: demographic data availability, holdout set details, and known constraints that affect testing coverage
• Fairness metric selection: up to three primary metrics (e.g., demographic parity, equalized odds, calibration by group) with threshold definitions and justification for each choice
• Pre-deployment test results: a structured matrix recording test outcomes per characteristic per metric, with pass/fail status and anomaly notes
• Findings & recommendations: overall fairness assessment, specific findings requiring action, and a formal deployment decision (deploy / conditional deploy / hold)
• Post-deployment monitoring plan: which fairness metrics to track, monitoring frequency, responsible role, and the escalation threshold that triggers reassessment
• Review and sign-off with three-tier approval: lead assessor, PM/system owner, governance/compliance
• Revision history with trigger column — records what prompted each assessment update (model retrain, data source change, regulatory development, monitoring alert)
• Completion guidance page with field-by-field instructions including metric selection guidance and threshold calibration by decision type

Framework Alignment

This template is designed to support compliance with the following frameworks:

• EU AI Act Art. 10 — Data governance requirements for high-risk AI systems, including examination of training data for possible biases and requirements to address data gaps affecting protected groups
• EU AI Act Art. 9 — Risk management obligations to identify and analyze known and foreseeable risks, including fairness risks, and document mitigation measures
• NIST AI RMF — MEASURE 2.5 & 2.7 — AI bias testing as part of the MEASURE function; documentation of fairness evaluation results and ongoing monitoring commitments
• ISO/IEC 42001 Clause 8.5 — Fairness and non-discrimination as an AI management system objective, with documented controls and monitoring
• IEEE P7003 — Algorithmic bias considerations framework: scope definition, metric selection, testing approach, and audit trail
• OECD AI Principles — Inclusive growth and human-centred values: AI systems should not discriminate against individuals or groups

Download