Predictive accuracy is only one part of AI model validation. A banking model can score well in development and still be unsafe because its data is unstable, its output is unexplained, its limitations are hidden or its use has expanded beyond the approved purpose.
Validate the full model lifecycle
Independent validation should examine conceptual soundness, data lineage, feature construction, performance, calibration, stability, fairness, explainability, implementation and monitoring. For generative systems, add prompt controls, retrieval sources, hallucination testing, confidentiality and output review.
Test uncertainty and drift
Use time-based validation, out-of-sample tests and stress scenarios. Monitor population stability, missingness, input shifts, outcome drift and override patterns. Where labels arrive late, define interim indicators and a plan for eventual outcome validation.
Set use limitations
A model should have explicit boundaries: approved population, geography, products, data sources, decision role, threshold range and human-review requirements. A model that is valid for triage may not be valid for an adverse credit decision.
The PRA model-risk principles and NIST AI RMF support a lifecycle approach. Quantica Risk Modelling recommends reporting residual uncertainty in language a risk committee can act on.