AI changes the profile, not the need for governance
AI systems can introduce opacity, rapid change, broad data dependence and outputs that vary with context. These characteristics require additional controls, but they do not remove the relevance of model inventories, materiality, validation, change control and accountable use.
The starting point remains the decision and potential harm. A simple model used in a critical automated process may warrant more control than a complex model used only for research.
Define the system boundary
The governed object includes training and reference data, prompts, retrieval, model version, application logic, guardrails, human review and downstream actions. Validating only the foundation model leaves much of the actual risk outside scope.
Third-party components require evidence on service changes, data handling, availability and exit options. Contractual assurance does not replace independent assessment of the use case.
Test outcomes and controls
Validation should assess accuracy, stability, bias, explainability appropriate to the decision, security and failure modes. Testing should include realistic edge cases and attempts to elicit unsafe or unsupported outputs.
Where human review is a key control, its effectiveness must be tested. Reviewers need sufficient information, time and authority; a nominal approval click is not meaningful mitigation.
Monitor a changing system
Monitoring should cover input drift, output quality, override, incidents, user behaviour and downstream impact. Material provider updates or changes in retrieval data may require revalidation even when application code is unchanged.
Senior reporting should connect technical measures to business consequences and risk appetite. That keeps AI governance anchored in accountable decision-making.