Fairness testing is not a single metric. Insurance and credit models can create unequal outcomes through direct features, proxies, data gaps, pricing segmentation, threshold choices or unequal access to review. A responsible control framework tests the full customer journey.
Define the outcome first
Specify what is being tested: approval, price, limit, referral, claim triage, fraud alert or service level. A model can have similar average accuracy across groups while producing materially different false-positive or false-negative outcomes.
Test proxies and missingness
Geography, occupation, transaction patterns, language and device data can act as proxies for protected characteristics. Missing data can also be unevenly distributed. Compare input quality, model performance, outcomes, overrides and complaints across relevant populations.
Make remediation operational
When a disparity is found, record whether the cause is data, feature design, calibration, threshold, process or human review. Potential responses include removing a feature, changing a threshold, adding a referral route, improving data collection or limiting use. Document the trade-off and obtain accountable approval.
The NIST AI RMF supports a structured approach to fairness, transparency and accountability. Legal assessment must be tailored to the jurisdiction and product.