AIGO — AI Governance Operating Framework
AI Monitoring Procedure
Version: 0.1Status: Draft
Working Name: AIGO
Full Name: AI Governance Operating Framework
Document Identifier: AIGO-PROC-009
1. Purpose
This procedure defines the process for establishing, operating, reviewing, and improving monitoring of AI systems throughout their lifecycle. The procedure ensures that AI systems remain observable and that material changes in performance, risk, controls, compliance, security, privacy, safety, and operational behavior are identified and addressed.2. Scope
This procedure applies to AI systems within the organization’s AIGO governance scope. It may apply to:- AI models;
- AI applications;
- generative AI systems;
- AI agents;
- AI-enabled business processes;
- third-party AI services;
- AI data pipelines;
- supporting infrastructure; and
- AI governance controls.
3. Objectives
The objectives of AI monitoring are to:- detect material changes in AI system behavior;
- identify emerging risks;
- monitor performance;
- monitor control effectiveness;
- identify incidents;
- support human oversight;
- identify compliance concerns;
- provide evidence for governance decisions;
- trigger reassessment where necessary; and
- support continuous improvement.
4. Monitoring Principles
AI monitoring should be:- risk-based;
- proportionate;
- continuous where appropriate;
- measurable;
- actionable;
- documented;
- traceable;
- timely; and
- integrated with governance processes.
5. Monitoring Responsibility
The AI system owner is accountable for ensuring that required monitoring is established and maintained. Monitoring responsibilities may also involve:- AI governance;
- risk management;
- security;
- privacy;
- compliance;
- technical operations;
- control owners;
- business owners; and
- assurance functions.
6. Monitoring Requirements
Monitoring requirements should be established before operational deployment where appropriate. Requirements should identify:- what is monitored;
- why it is monitored;
- monitoring frequency;
- measurement method;
- thresholds;
- responsible owner;
- escalation requirements; and
- evidence requirements.
7. Monitoring Scope
The monitoring scope should be based on:- AI system purpose;
- risk classification;
- system criticality;
- autonomy;
- data;
- users;
- affected stakeholders;
- dependencies;
- controls; and
- applicable requirements.
8. Monitoring Categories
Monitoring may include:- performance;
- reliability;
- accuracy;
- data quality;
- model behavior;
- security;
- privacy;
- fairness;
- safety;
- human oversight;
- controls;
- incidents;
- compliance; and
- operational resilience.
9. Performance Monitoring
AI system performance should be monitored against defined expectations. Performance indicators may include:- accuracy;
- precision;
- recall;
- error rate;
- response quality;
- latency;
- availability;
- throughput; and
- task completion.
10. Reliability Monitoring
Reliability monitoring should identify:- service failures;
- model failures;
- repeated errors;
- downtime;
- degraded performance;
- dependency failures;
- unexpected interruptions; and
- operational instability.
11. Data Monitoring
Where data materially affects AI behavior, monitoring should consider:- data quality;
- completeness;
- consistency;
- availability;
- distribution changes;
- data drift;
- unexpected values;
- data pipeline failures; and
- unauthorized changes.
12. Model Monitoring
Where applicable, model monitoring should consider:- model performance;
- model drift;
- data drift;
- concept drift;
- output changes;
- version changes;
- unexpected behavior;
- degradation; and
- validation results.
13. Generative AI Monitoring
Generative AI systems should be monitored for relevant risks including:- hallucination;
- harmful outputs;
- inappropriate content;
- prompt injection;
- data leakage;
- policy violations;
- unsafe recommendations;
- output quality;
- misuse; and
- unexpected behavior.
14. AI Agent Monitoring
AI agents should be monitored for:- actions taken;
- tools used;
- permissions exercised;
- transactions initiated;
- failed actions;
- unauthorized actions;
- escalation events;
- human interventions; and
- deviations from intended behavior.
15. Human Oversight Monitoring
Where human oversight is required, monitoring should consider:- intervention frequency;
- override frequency;
- response time;
- reviewer workload;
- missed interventions;
- escalation events;
- reviewer competence; and
- indicators of excessive reliance on automated outputs.
16. Security Monitoring
Security monitoring may include:- unauthorized access;
- suspicious activity;
- prompt injection;
- adversarial activity;
- model abuse;
- data exfiltration;
- credential misuse;
- anomalous behavior; and
- security incidents.
17. Privacy Monitoring
Privacy monitoring may include:- unauthorized data use;
- unexpected data disclosure;
- inappropriate retention;
- access violations;
- profiling concerns;
- privacy incidents;
- unauthorized processing; and
- changes in data handling.
18. Fairness Monitoring
Where relevant, monitoring should assess whether AI outcomes remain appropriately fair. Indicators may include:- performance differences;
- outcome differences;
- error-rate differences;
- demographic disparities; and
- changes in affected populations.
19. Safety Monitoring
Safety-relevant AI systems should be monitored for:- unsafe outputs;
- unsafe actions;
- near misses;
- failures;
- safety-control activation;
- emergency events; and
- safety incidents.
20. Compliance Monitoring
Monitoring should identify whether the AI system continues to comply with:- applicable laws;
- regulations;
- internal policies;
- governance requirements;
- contractual requirements;
- approval conditions; and
- other applicable obligations.
21. Control Monitoring
Controls should be monitored to determine whether they continue to operate as intended. Monitoring may include:- control execution;
- control failures;
- exceptions;
- overdue actions;
- evidence availability;
- testing results; and
- changes in control effectiveness.
22. Monitoring Indicators
Each monitored AI system should have appropriate indicators. Indicators should be:- relevant;
- measurable;
- understandable;
- actionable;
- traceable; and
- proportionate to risk.
23. Thresholds
Where appropriate, monitoring indicators should have defined thresholds. Thresholds may define:- acceptable range;
- warning level;
- critical level;
- escalation point; and
- required response.
24. Alerts
Monitoring mechanisms should generate alerts when defined thresholds are exceeded. Alerts should identify, where applicable:- indicator;
- observed value;
- threshold;
- date and time;
- affected AI system;
- severity; and
- required action.
25. Monitoring Frequency
Monitoring frequency should be determined according to:- classification;
- risk;
- system criticality;
- change rate;
- incident history;
- regulatory requirements;
- operational requirements; and
- available monitoring capability.
26. Monitoring Methods
Monitoring may be performed through:- automated monitoring;
- manual review;
- periodic assessment;
- statistical analysis;
- sampling;
- user feedback;
- control testing;
- audits; and
- assurance activities.
27. Automated Monitoring
Automated monitoring may be used where continuous or high-frequency observation is required. Automated monitoring should be designed to:- detect defined conditions;
- generate reliable alerts;
- minimize unnecessary alerts;
- preserve relevant evidence; and
- support escalation.
28. Manual Monitoring
Manual monitoring may be appropriate where:- automated detection is insufficient;
- expert judgment is required;
- the system is low volume;
- the monitored outcome is qualitative; or
- regulatory or governance requirements require human review.
29. Monitoring Dashboard
Where appropriate, monitoring information should be presented through dashboards. Dashboards may provide:- system status;
- risk status;
- control status;
- performance;
- incidents;
- alerts;
- trends;
- exceptions; and
- outstanding actions.
30. Monitoring Records
Monitoring records should be maintained where required. Records may include:- measurements;
- logs;
- alerts;
- investigations;
- decisions;
- incidents;
- reviews;
- remediation; and
- evidence.
31. Monitoring Evidence
Monitoring evidence should be sufficient to demonstrate that required monitoring occurred. Evidence may include:- monitoring reports;
- system logs;
- dashboards;
- test results;
- review records;
- alert records;
- investigation records; and
- approval records.
32. Monitoring Review
Monitoring results should be reviewed by responsible personnel. The review should determine whether:- indicators remain appropriate;
- thresholds remain appropriate;
- risks have changed;
- controls remain effective;
- incidents have occurred;
- trends require attention; and
- reassessment is required.
33. Trend Analysis
Monitoring should consider trends rather than isolated events where appropriate. Trend analysis may identify:- gradual performance degradation;
- increasing error rates;
- recurring incidents;
- increasing control failures;
- changing user behavior;
- changing data characteristics; and
- emerging risks.
34. Anomaly Detection
Where appropriate, anomaly detection should be used to identify unexpected behavior. Anomalies should be investigated according to their significance. An anomaly does not necessarily constitute an incident, but material anomalies should be assessed for potential risk and incident implications.35. User Feedback
Relevant user feedback should be considered as a monitoring input. Feedback may identify:- unexpected behavior;
- poor outputs;
- harmful outputs;
- usability problems;
- bias concerns;
- safety concerns;
- reliability problems; and
- changes in user expectations.
36. Monitoring and Incident Management
Monitoring should integrate with the organization’s AI incident management process. Monitoring events may trigger:- incident investigation;
- escalation;
- containment;
- remediation;
- reporting; and
- reassessment.
37. Monitoring and Risk Management
Monitoring results should feed into AI risk management. Material monitoring findings should trigger risk review where appropriate. Risk indicators should be reviewed when monitoring demonstrates changes in:- likelihood;
- impact;
- affected populations;
- controls;
- operating conditions; or
- system behavior.
38. Monitoring and Change Management
Material changes to an AI system should be assessed for monitoring implications. Changes may require:- new indicators;
- changed thresholds;
- increased monitoring frequency;
- new alerts;
- additional testing; or
- revised monitoring responsibilities.
39. Monitoring and Control Management
Monitoring should provide evidence about control effectiveness where appropriate. Control failures should be:- recorded;
- assessed;
- escalated where required;
- remediated; and
- verified.
40. Monitoring and Assurance
Monitoring information should support assurance activities where appropriate. Assurance functions may use monitoring evidence to assess:- control effectiveness;
- risk management;
- governance compliance;
- operational performance; and
- ongoing conformity.
41. Escalation
Monitoring findings should be escalated when:- thresholds are exceeded;
- risk exceeds tolerance;
- controls fail;
- incidents occur;
- performance materially degrades;
- unauthorized behavior is detected; or
- governance requirements may no longer be satisfied.
42. Monitoring Response
The appropriate response should be proportionate to the finding. Possible responses include:- continued monitoring;
- increased monitoring;
- investigation;
- corrective action;
- risk reassessment;
- control reassessment;
- change management;
- incident management;
- system restriction; or
- system suspension.
43. Monitoring Exceptions
Monitoring exceptions should be documented. An exception record should identify:- affected system;
- requirement;
- reason;
- duration;
- risk;
- compensating measures;
- owner; and
- approval.
44. Monitoring Failures
A failure of required monitoring should itself be assessed. The assessment should determine:- why monitoring failed;
- duration;
- affected period;
- potential blind spots;
- risk implications;
- incident implications; and
- corrective actions.
45. Monitoring Calibration
Monitoring thresholds and indicators should be periodically reviewed to ensure they remain meaningful. Calibration may consider:- historical performance;
- incidents;
- false positives;
- false negatives;
- system changes;
- risk changes; and
- operational experience.
46. Monitoring Quality
Monitoring mechanisms should themselves be subject to appropriate quality controls. Quality considerations may include:- accuracy;
- completeness;
- reliability;
- timeliness;
- availability;
- traceability; and
- resistance to manipulation.
47. Third-Party Monitoring
Third-party AI services should be monitored according to available information and contractual arrangements. Monitoring may include:- service availability;
- provider incidents;
- model changes;
- performance;
- security;
- privacy;
- compliance;
- service-level performance; and
- provider communications.
48. Monitoring During High-Risk Operations
AI systems performing high-risk or critical functions may require enhanced monitoring. Enhanced monitoring may include:- increased frequency;
- additional human review;
- expanded indicators;
- tighter thresholds;
- additional logging;
- real-time alerts; and
- enhanced escalation.
49. Monitoring During Change
AI systems undergoing material change should receive appropriate monitoring before, during, and after implementation. Post-change monitoring should determine whether:- expected performance was achieved;
- unexpected behavior occurred;
- controls remain effective;
- risk changed; and
- further action is required.
50. Monitoring During Incident Recovery
Monitoring should be enhanced where an AI system is recovering from an incident. Enhanced monitoring should remain in place until the responsible authority determines that normal monitoring is appropriate.51. Monitoring Reporting
The AI governance function should receive monitoring reports appropriate to the organization’s governance model. Reporting may include:- system performance;
- risk indicators;
- control status;
- incidents;
- alerts;
- exceptions;
- trends;
- unresolved findings; and
- required actions.
52. Monitoring Metrics
Organizations may establish monitoring metrics. Examples include:- monitoring coverage;
- threshold breaches;
- alert volume;
- unresolved alerts;
- incident detection rate;
- false-positive rate;
- monitoring failures;
- control failures; and
- overdue monitoring actions.
53. Monitoring Review Frequency
Monitoring results should be reviewed at a frequency proportionate to risk. Review frequency may include:- continuous;
- daily;
- weekly;
- monthly;
- quarterly; or
- event-driven review.
54. Monitoring Responsibilities
AI System Owner- ensure monitoring requirements are established;
- ensure monitoring is operational;
- review material findings;
- initiate reassessment where required; and
- ensure corrective actions are completed.
- define governance expectations;
- oversee monitoring compliance;
- review material findings;
- coordinate escalation; and
- maintain governance reporting.
- operate monitoring mechanisms;
- maintain monitoring infrastructure;
- investigate technical findings; and
- preserve monitoring evidence.
- review monitoring information relevant to their responsibilities;
- assess material findings;
- support escalation; and
- recommend corrective actions where required.
55. Monitoring Workflow
The standard workflow should be:- Define monitoring requirements.
- Identify indicators.
- Define thresholds.
- Assign monitoring ownership.
- Implement monitoring mechanisms.
- Begin monitoring.
- Collect monitoring data.
- Review results.
- Detect anomalies or threshold breaches.
- Assess significance.
- Escalate where required.
- Initiate incident, risk, change, or control processes where applicable.
- Implement corrective actions.
- Verify corrective actions.
- Update monitoring requirements.
- Report material findings.
- Maintain monitoring evidence.
56. Continuous Improvement
The monitoring process should be improved based on:- incidents;
- near misses;
- monitoring failures;
- assurance findings;
- audit findings;
- changes in AI behavior;
- user feedback;
- regulatory developments; and
- operational experience.
57. Procedure Review
This procedure should be reviewed periodically and when material changes occur. Review triggers may include:- significant incidents;
- changes to AIGO requirements;
- regulatory developments;
- assurance findings;
- material changes to AI systems;
- changes in monitoring technology; and
- implementation experience.
58. Procedure Status
Document: AIGO AI Monitoring Procedure Version: 0.1 Status: Draft Working Name: AIGO Full Name: AI Governance Operating Framework Document Identifier:AIGO-PROC-009
Document Type: Operational Procedure
This procedure establishes the operational process for monitoring AI systems and identifying changes requiring governance action throughout the AI governance lifecycle.