AIGO — AI Monitoring Example
AIGO — AI Governance Operating Framework
Version: 0.1 Status: Draft Working Name: AIGO Full Name: AI Governance Operating Framework Document Identifier:AIGO-EXAMPLE-009
Document Type: Implementation Example
Example Type: AI Monitoring
1. Purpose
This document provides an illustrative example of how an organization can establish, operate, evaluate, escalate, and improve AI monitoring using the AIGO AI Governance Operating Framework. The example demonstrates how monitoring connects:- AI system operation;
- lifecycle stage;
- risk;
- controls;
- performance;
- fairness;
- security;
- privacy;
- human oversight;
- incidents;
- evidence;
- assurance;
- continual improvement.
2. Example Organization
For this example, the organization is ExampleCorp, a fictional organization implementing AIGO. The organization operates an AI-enabled recruitment-support system.3. AI System
System Name: Candidate Assessment Assistant AI System ID:AI-HR-001
Business Function: Human Resources
Classification: Class 3 — Enhanced Governance
Lifecycle Stage: Operate
System Owner: HR AI System Owner
Model Owner: AI/ML Engineering Lead
Risk Owner: Enterprise Risk Manager
4. Monitoring Scenario
ExampleCorp operates the Candidate Assessment Assistant in production. The organization establishes a monitoring framework to determine whether the AI system continues to operate within approved:- performance thresholds;
- risk tolerances;
- fairness thresholds;
- security requirements;
- privacy requirements;
- operational parameters;
- control expectations.
- investigation;
- corrective action;
- change management;
- incident management;
- risk reassessment;
- suspension;
- rollback;
- retirement.
5. Monitoring Objective
The monitoring process is designed to:- detect deviations;
- identify emerging risks;
- assess control performance;
- identify model degradation;
- detect data drift;
- detect fairness deterioration;
- support human oversight;
- generate evidence;
- trigger escalation;
- support continual improvement.
6. Monitoring Principle
AI monitoring should not focus exclusively on technical model performance. Effective AI monitoring should consider the broader governance environment.7. Monitoring Lifecycle
8. Monitoring Scope
ExampleCorp monitors:- system availability;
- model performance;
- input data quality;
- data drift;
- output distribution;
- fairness;
- human overrides;
- security indicators;
- privacy indicators;
- incidents;
- control effectiveness;
- user feedback;
- operational thresholds.
9. Monitoring Categories
10. Monitoring Architecture
11. Monitoring Plan
Monitoring Plan ID:MON-PLAN-AI-001
System: AI-HR-001
Owner: AI System Owner
Review Frequency: Monthly
Enhanced Monitoring: Triggered when required
12. Monitoring Indicators
ExampleCorp establishes a set of key AI monitoring indicators.13. Performance Monitoring
The organization monitors:- accuracy;
- precision;
- recall;
- error rates;
- response time;
- failure rates;
- output quality.
14. Performance Thresholds
Example:
Thresholds must be approved for the specific AI system.
15. Data Quality Monitoring
Data monitoring evaluates:- completeness;
- accuracy;
- consistency;
- validity;
- timeliness;
- distribution;
- unexpected values.
16. Data Quality Indicators
17. Data Drift
Data drift occurs when the characteristics of operational data change relative to the data used to develop or validate the AI system. ExampleCorp monitors:- feature distribution;
- category frequency;
- statistical distribution;
- input volume;
- subgroup composition.
18. Data Drift Response
19. Model Drift
Model drift may occur when model performance changes over time. ExampleCorp evaluates:- performance degradation;
- changing data relationships;
- changing user behavior;
- changing operating conditions.
20. Fairness Monitoring
The organization monitors relevant fairness indicators according to the system’s intended use and applicable requirements. Potential indicators include:- group performance;
- error-rate differences;
- selection-rate differences;
- false-positive differences;
- false-negative differences.
21. Fairness Thresholds
Example:
Thresholds should be determined based on the system, context, applicable requirements, and organizational risk criteria.
22. Fairness Alert Example
Monitoring detects an unexpected increase in recommendation disparity. Alert ID:ALERT-2026-022
Severity: High
Status: Under Investigation
Potential Risk: R-001 — Biased Recommendations
23. Human Oversight Monitoring
ExampleCorp monitors:- human override rates;
- override patterns;
- rejected AI recommendations;
- user complaints;
- escalation frequency;
- situations where users cannot effectively challenge AI outputs.
24. Human Override Indicator
An unusual increase in human overrides may indicate:- model degradation;
- changing operating conditions;
- poor user trust;
- inappropriate recommendations;
- changes in workflow;
- inadequate model fit.
25. Security Monitoring
Security monitoring includes:- unauthorized access;
- authentication failures;
- unusual API activity;
- suspicious model interactions;
- dependency vulnerabilities;
- data-access anomalies;
- model endpoint attacks.
26. Privacy Monitoring
Where applicable, monitoring includes:- unauthorized data access;
- inappropriate data use;
- unexpected data retention;
- sensitive-data exposure;
- privacy incidents;
- data-processing deviations.
27. Operational Monitoring
Operational monitoring includes:- availability;
- processing failures;
- integration failures;
- queue failures;
- system latency;
- resource utilization;
- deployment failures.
28. Governance Monitoring
Governance monitoring evaluates whether required governance activities remain operational. Examples include:- risk reviews completed;
- control assessments completed;
- approvals current;
- monitoring performed;
- incidents recorded;
- evidence retained;
- reviews performed on schedule.
29. Control Monitoring
AIGO controls should be monitored for both:- design effectiveness;
- operating effectiveness.
30. Risk Monitoring
Risk indicators are monitored against approved risk thresholds. Example:31. Monitoring Threshold Model
32. Alert Severity
33. Alert Management
Every material alert should have:- alert identifier;
- timestamp;
- source;
- indicator;
- threshold;
- observed value;
- severity;
- owner;
- investigation status;
- resolution;
- evidence.
34. Example Monitoring Alert
Alert ID:ALERT-2026-022
Indicator: Fairness disparity
Expected Range: Approved threshold
Observed: Threshold exceeded
Severity: High
System: AI-HR-001
Risk: R-001
Status: Investigating
35. Monitoring Investigation
The investigation determines:- whether the alert is genuine;
- whether the metric is reliable;
- whether the threshold remains appropriate;
- whether affected stakeholders exist;
- whether risk has changed;
- whether a control failed;
- whether an incident should be opened.
36. False Positive Handling
A monitoring alert may be a false positive. The organization should document:- why the alert occurred;
- why it was determined to be a false positive;
- whether the threshold was appropriate;
- whether monitoring logic should be changed.
37. Monitoring Escalation
38. Incident Trigger
Monitoring may trigger the AI Incident Management Procedure when:- a material threshold is exceeded;
- potential harm is identified;
- unauthorized operation occurs;
- security compromise is suspected;
- fairness deterioration is material;
- system behavior becomes unpredictable;
- control failure is identified.
39. Change Trigger
Monitoring may also trigger the AI Change Management Procedure. Examples:- persistent model degradation;
- significant data drift;
- recurring fairness issues;
- outdated monitoring thresholds;
- changes in operating context.
40. Risk Reassessment Trigger
Monitoring should trigger risk reassessment when:- risk indicators materially change;
- new risks emerge;
- controls become ineffective;
- system context changes;
- incidents reveal previously unknown risk.
41. Monitoring Evidence
Evidence should include:- monitoring logs;
- dashboards;
- metric results;
- threshold evaluations;
- alerts;
- investigations;
- actions;
- approvals;
- reports;
- review records.
42. Monitoring Evidence Chain
43. Monitoring Dashboard
A governance dashboard may contain:44. Monitoring Review
The AI System Owner reviews monitoring results periodically. The review determines:- whether the system remains within approved parameters;
- whether risks remain acceptable;
- whether controls remain effective;
- whether changes are required;
- whether incidents occurred;
- whether evidence is complete.
45. Monthly Monitoring Review
Review ID:MON-REV-001
Review Period: Month 1
System: AI-HR-001
Overall Status: Acceptable
Open Alerts: 1
Open Incidents: 0
Risk Status: Within approved tolerance
46. Monitoring Report
A monitoring report should summarize:- monitoring period;
- system status;
- key indicators;
- threshold breaches;
- incidents;
- changes;
- risks;
- controls;
- corrective actions;
- recommendations.
47. Monitoring Report Example
48. Enhanced Monitoring
Enhanced monitoring may be initiated after:- material change;
- serious incident;
- significant risk increase;
- control failure;
- major model update;
- new deployment;
- unexpected behavior.
- defined duration;
- defined indicators;
- defined thresholds;
- assigned ownership;
- documented exit criteria.
49. Enhanced Monitoring Example
Following a material model update, ExampleCorp establishes: Duration: 30 days Fairness monitoring: Daily Performance monitoring: Daily Data drift: Continuous Incident review: Immediate Governance report: Weekly50. Monitoring Frequency
Monitoring frequency should reflect risk.
The exact frequency should be determined for each AI system.
51. Monitoring Ownership
52. Monitoring Independence
Where practical, monitoring should include appropriate independence. Critical monitoring results should not be suppressed, modified, or ignored by individuals whose performance is directly affected by the result.53. Monitoring and Human Oversight
Monitoring should support human decision-making rather than replace governance judgment. Automated alerts identify conditions requiring attention. Authorized personnel remain responsible for interpreting material findings and determining appropriate action.54. Monitoring and Decision-Making
55. Monitoring and Assurance
Monitoring provides operational evidence. Assurance evaluates whether the monitoring system itself is:- appropriate;
- complete;
- reliable;
- effective;
- properly governed.
56. Monitoring and Continual Improvement
Monitoring results should feed the improvement cycle.57. Monitoring Failure
Monitoring itself may fail. Examples include:- missing telemetry;
- incorrect thresholds;
- unavailable dashboards;
- incomplete data;
- false negatives;
- excessive false positives;
- monitoring disabled without authorization.
58. Monitoring Control Failure Example
The organization discovers that fairness monitoring stopped for three days because a monitoring pipeline failed. The system continued operating. The organization:- records the monitoring failure;
- assesses the monitoring gap;
- evaluates potential impact;
- restores monitoring;
- performs retrospective analysis;
- reassesses risk;
- determines whether an incident occurred.
59. Monitoring Recovery
Recovery includes:- restoring telemetry;
- validating monitoring logic;
- reconstructing missing evidence where possible;
- assessing the unmonitored period;
- reviewing affected outputs;
- documenting limitations.
60. Monitoring Closure
A monitoring alert or monitoring issue may be closed when:- the condition is understood;
- required action is complete;
- risk is assessed;
- evidence is retained;
- monitoring is restored;
- required approval is obtained.
61. Monitoring Checklist
- Monitoring scope defined
- Indicators defined
- Thresholds approved
- Monitoring owner assigned
- Data sources identified
- Monitoring implemented
- Alerts configured
- Escalation criteria defined
- Risk indicators monitored
- Performance monitored
- Data quality monitored
- Drift monitored
- Fairness monitored where applicable
- Security monitored
- Privacy monitored where applicable
- Human oversight monitored
- Controls monitored
- Evidence retained
- Reviews completed
- Issues escalated
- Continual improvement actions recorded
62. Monitoring Traceability
A complete monitoring record should connect:63. Monitoring and Lifecycle Management
Monitoring operates continuously during the operational lifecycle.- change;
- retraining;
- reclassification;
- risk treatment;
- incident management;
- system suspension;
- retirement.
64. Monitoring and AI Change Management
Monitoring can identify conditions that justify an AI change. Examples:- performance degradation;
- model drift;
- changing data;
- user feedback;
- fairness deterioration.
65. Monitoring and AI Incident Management
Monitoring can be the primary detection mechanism for AI incidents. A material monitoring threshold breach should be assessed to determine whether incident-management procedures are required.66. Monitoring and Risk Acceptance
Where monitoring identifies a known condition that remains within approved tolerance, risk may continue to be accepted. Where risk exceeds approved tolerance, the appropriate risk treatment or escalation process should be initiated.67. Monitoring and Retirement
Monitoring may provide evidence that an AI system should be retired. Examples:- persistent unacceptable performance;
- unresolved material risk;
- obsolete purpose;
- inadequate controls;
- inability to maintain required monitoring;
- unacceptable operating cost or risk.
68. Relationship to AIGO Procedures
This example should be implemented through the applicable AIGO procedures, particularly:- AI Monitoring Procedure;
- AI Governance Procedure;
- AI Risk Assessment Procedure;
- AI Classification Procedure;
- AI Control Assessment Procedure;
- AI Change Management Procedure;
- AI Incident Management Procedure;
- AI Assurance Procedure;
- AI Approval Procedure;
- Continuous Improvement Procedure;
- AI Retirement Procedure.
69. Relationship to AIGO Controls
The monitoring example demonstrates interaction between:- monitoring controls;
- performance controls;
- risk controls;
- fairness controls;
- human oversight controls;
- security controls;
- privacy controls;
- incident controls;
- change controls;
- assurance controls.
70. Relationship to ISO/IEC 42001
AI monitoring can support an AI management system by providing operational performance information, risk indicators, control evidence, evaluation inputs, incident signals, and continual-improvement information. Applicable ISO/IEC 42001 requirements should be evaluated separately by the implementing organization.71. Relationship to NIST AI RMF
The monitoring process can support activities associated with:72. Complete Monitoring Model
73. Key Governance Principles
73.1 Monitoring Must Be Risk-Based
Higher-risk systems generally require stronger monitoring.73.2 Monitoring Must Be Continuous Where Necessary
Critical indicators may require continuous or near-real-time monitoring.73.3 Thresholds Must Be Defined
Monitoring without defined decision thresholds may not provide effective governance.73.4 Alerts Must Lead to Action
A threshold breach should have a defined response.73.5 Monitoring Evidence Must Be Preserved
Material monitoring results should remain traceable.73.6 Monitoring Must Include Governance Indicators
Technical performance alone does not demonstrate responsible AI operation.73.7 Monitoring Must Support Human Oversight
Humans remain responsible for material governance decisions.73.8 Monitoring Must Feed Improvement
Monitoring results should influence risk, controls, procedures, and system decisions.74. Final Monitoring Decision
ExampleCorp determines that the Candidate Assessment Assistant remains within its approved operating parameters. One moderate data-drift indicator remains under enhanced observation. No material incident has been identified. The system remains approved for operation subject to continued monitoring.75. Document Status
Document: AIGO — AI Monitoring Example Version: 0.1 Status: Draft Working Name: AIGO Full Name: AI Governance Operating Framework Document Identifier:AIGO-EXAMPLE-009
Document Type: Implementation Example
Example Type: AI Monitoring
This document provides an illustrative example of how AI monitoring can be designed, operated, evaluated, escalated, evidenced, and integrated with risk management, incident management, change management, assurance, and continual improvement.
76. End of Example Document
AIGO — AI Monitoring Example Document ID:AIGO-EXAMPLE-009
Version: 0.1
Status: Draft
End of Document