title: “AI Monitoring” description: “Monitor AI systems, risks, controls, governance conditions, and operational performance throughout the AI lifecycle.”
AI Monitoring
Monitoring provides ongoing visibility into AI system operation, risk conditions, control performance, and the effectiveness of governance arrangements. Within AIGO, monitoring is a continuous lifecycle activity. It provides information that can trigger investigation, risk reassessment, control changes, incident management, change management, approval review, assurance, and continual improvement. Monitoring therefore connects AI system operation with ongoing governance.Purpose
Monitoring helps organizations determine whether an AI system continues to operate within its approved governance conditions. Monitoring can identify: * changes in system behavior * performance degradation * emerging risks * control weaknesses * incidents * changes in operating conditions * deviations from intended use * changes in stakeholder impact * failures to satisfy governance requirements * changes in applicable requirements * assurance findings * opportunities for improvement Monitoring should be proportionate to the AI system’s characteristics, classification, risk, and governance requirements.Monitoring objectives
A monitoring program should establish what the organization needs to observe and why. Objectives may include: * detecting material changes * identifying emerging risks * verifying control operation * detecting incidents * verifying approval conditions * monitoring system performance * monitoring operational conditions * identifying governance deviations * supporting assurance * generating evidence * supporting continual improvement Monitoring objectives should be connected to the risks and governance conditions they are intended to address.What should be monitored?
Monitoring scope depends on the AI system and its governance context. Potential monitoring areas include:AI system performance
Organizations may monitor: * accuracy * reliability * availability * performance degradation * unexpected behavior * output quality * model drift * data drift * relevant system-specific performance indicators The appropriate measures depend on the AI system and intended use.Risk indicators
Monitoring may track indicators associated with known or emerging risks. Examples include: * risk thresholds * risk events * changes in exposure * recurring failure patterns * affected stakeholder signals * emerging risk indicators * changes in residual risk Monitoring should focus on indicators that provide meaningful information for governance decisions.Control performance
Monitoring may provide evidence about whether controls continue to operate as intended. Examples include: * control execution * control failures * control exceptions * overdue control activities * evidence availability * recurring deficiencies * control performance indicators Monitoring does not replace formal control assessment where formal assessment is required. See Control Assessment.Governance conditions
Monitoring may verify whether the AI system continues to operate within approved conditions. These conditions may include: * approved intended use * deployment boundaries * human oversight requirements * approval conditions * access restrictions * operating constraints * required controls * required monitoring thresholds A deviation may require escalation or governance review.Incidents
Monitoring is an important mechanism for incident detection. Monitoring signals may identify: * abnormal system behavior * significant failures * harmful outputs * security events * privacy events * control failures * unexpected use * other conditions requiring investigation Not every monitoring signal constitutes an incident. Organizations should establish criteria for determining when a monitoring signal enters the formal incident-management process. See AI Incident Management.Changes
Monitoring may identify changes that were not previously recorded through the organization’s change process. Examples include: * changes in system behavior * changes in operating environment * changes in data * changes in stakeholder impact * changes in applicable requirements * changes in risk exposure Potentially material changes should enter the appropriate change-management process. See AI Change Management.Monitoring frequency
Monitoring frequency should be determined according to the governance context. Factors may include: * system classification * risk level * system criticality * rate of change * operational environment * affected stakeholders * applicable requirements * historical incidents * control performance * assurance findings Some indicators may require continuous or near-real-time monitoring. Others may be appropriate for periodic review. The organization should document the rationale for significant monitoring frequencies and thresholds.Monitoring thresholds
Monitoring should define meaningful thresholds where threshold-based response is appropriate. A threshold may indicate that: * normal operation continues * additional investigation is required * a control exception has occurred * a risk should be reassessed * an incident should be opened * approval should be reviewed * the system should be restricted or suspended Thresholds should be appropriate to the AI system and should not be treated as universally applicable values.Monitoring results
Monitoring produces observations, measurements, signals, findings, or other governance information. Results may include: * normal operation * threshold warning * threshold breach * control exception * emerging risk * incident signal * governance deviation * required corrective action * required reassessment Monitoring results should be retained as evidence where required.Monitoring workflow
A practical AIGO monitoring workflow is: 1. Define monitoring objectives 2. Identify indicators 3. Define thresholds and response criteria 4. Collect monitoring information 5. Evaluate monitoring results 6. Record significant observations 7. Escalate where required 8. Investigate exceptions 9. Trigger governance action where necessary 10. Verify corrective actions 11. Review monitoring effectiveness 12. Improve the monitoring program The workflow may operate continuously, periodically, or through a combination of both.1. Define monitoring objectives
The organization should determine what needs to be monitored and what governance decision the monitoring is intended to support. Each important monitoring objective should have a clear relationship to one or more of: * AI system characteristics * risks * controls * governance requirements * approval conditions * operational requirements * assurance objectives This prevents monitoring from becoming a collection of metrics without a defined governance purpose.2. Identify indicators
Indicators should provide useful information about the condition being monitored. Indicators may be: * quantitative * qualitative * event-based * threshold-based * trend-based * control-based * risk-based Examples include: * performance measures * error rates * drift indicators * incident counts * control exceptions * risk indicators * approval-condition breaches * unresolved findings Indicators should be selected according to the system and governance objective.3. Define thresholds and response criteria
Where appropriate, organizations should define what constitutes: * normal condition * warning condition * significant deviation * critical condition The response associated with each condition should be clear. For example: indicator → threshold → response → owner → evidence This makes monitoring actionable rather than purely observational.4. Collect monitoring information
Monitoring information may come from: * AI system telemetry * application logs * model evaluation * operational metrics * user reports * control records * incident records * change records * assurance activities * governance reviews * external information Collection should follow applicable privacy, security, confidentiality, and retention requirements.5. Evaluate monitoring results
Monitoring results should be evaluated against the defined objectives, indicators, and thresholds. Evaluation may identify: * normal conditions * trends * anomalies * threshold breaches * control deficiencies * emerging risks * incidents * governance deviations Machine-readable evaluation can support this activity where appropriate. However, automated evaluation should not be assumed to replace human governance judgment where interpretation or contextual assessment is required.6. Record significant observations
Significant monitoring results should be recorded in an appropriate governance record. Records may include: * observation * date and time * affected AI system * indicator * observed value or condition * threshold * assessment * responsible owner * required action * related risk * related control * related incident or change * evidence This creates traceability from monitoring activity to governance action.7. Escalate where required
Monitoring results should be escalated according to defined criteria. Escalation may be required when: * a critical threshold is exceeded * a significant risk emerges * an approval condition is violated * a control fails * an incident is detected * system behavior materially changes * stakeholder impact increases * applicable requirements may no longer be satisfied Escalation responsibilities should be defined in the organization’s governance operating model.8. Investigate exceptions
Not every exception has the same significance. Investigation should determine: * what happened * whether the observation is valid * whether the condition is temporary or persistent * whether an incident exists * whether risk has changed * whether controls remain effective * whether a material change has occurred The investigation outcome should determine the appropriate governance response.9. Trigger governance action
Monitoring results may trigger: * investigation * incident management * risk reassessment * control changes * control assessment * corrective action * change management * approval review * escalation * assurance * system restriction * suspension * retirement * continual improvement A representative relationship is: monitoring → signal → assessment → governance action The appropriate action should be proportionate to the significance of the monitoring result.Monitoring and risk management
Monitoring provides ongoing information about whether risk conditions remain within acceptable boundaries. A representative relationship is: risk → indicator → monitoring → observation → reassessment Monitoring should not be treated as a replacement for periodic or event-triggered risk assessment. Material changes or significant monitoring results may require formal reassessment. See Risk Assessment.Monitoring and controls
Monitoring can provide evidence about control operation. The relationship can be represented as: control → monitoring → evidence → assessment Monitoring may identify that: * a control is operating effectively * a control is not operating as expected * evidence is missing * an exception has occurred * remediation is required Formal control assessment should be performed where required by the applicable governance process. See Control Assessment.Monitoring and incidents
Monitoring is a major source of incident detection. A representative relationship is: monitoring → signal → incident → investigation → corrective action The monitoring program should therefore define criteria for escalating monitoring observations into incident management. See AI Incident Management.Monitoring and change management
Monitoring can identify changes that require formal change management. For example, persistent changes in system behavior may indicate: * model drift * data drift * environmental change * changed stakeholder impact * changed risk exposure Where the change is material, it should enter the change-management process. See AI Change Management.Monitoring and approval
Monitoring can verify whether an AI system continues to operate within approved conditions. A monitoring result may therefore trigger an approval review where: * approval conditions are breached * risk changes materially * controls become ineffective * intended use changes * significant incidents occur * assurance conclusions are no longer valid See AI Governance Approval.Monitoring and assurance
Monitoring provides evidence that can support assurance activities. Assurance may independently evaluate whether monitoring: * is appropriately designed * covers relevant risks * uses appropriate indicators * applies suitable thresholds * produces reliable evidence * results in appropriate governance responses Monitoring findings may therefore become inputs to assurance, while assurance findings may result in changes to the monitoring program. See AI Assurance.Monitoring and continual improvement
Monitoring results should contribute to continual improvement. The organization should periodically evaluate whether its monitoring program remains effective. Improvement may involve: * adding indicators * removing ineffective indicators * changing thresholds * changing monitoring frequency * improving data quality * improving escalation * improving evidence collection * integrating additional systems * improving governance workflows See Continuous Improvement.Machine-readable representation
AIGO provides a machine-readable Monitoring Schema for representing governed monitoring records. The monitoring structure can be related to: * AI systems * risks * controls * assessments * incidents * changes * approvals * assurance * evidence * improvementsCanonical schema
schemas/06-monitoring/06-AIGO-Monitoring-Schema-v0.1.json
Schema documentation
schemas/06-monitoring/06-AIGO-Monitoring-Schema-Documentation-v0.1.md
Machine-readable monitoring records can support:
* monitoring platforms
* governance systems
* automated alerts
* evidence management
* traceability
* evaluation
* assurance
* lifecycle management
Monitoring artifacts
A practical monitoring implementation may produce: 1. monitoring objectives 2. indicator definitions 3. threshold definitions 4. monitoring plans 5. monitoring observations 6. monitoring results 7. alerts 8. exception records 9. evidence records 10. incident records where required 11. change records where required 12. risk reassessments where required 13. control assessments where required 14. approval reviews where required 15. assurance findings 16. improvement actions Organizations may combine records where appropriate, provided that required traceability is maintained.Implementation resources
The canonical procedure is:guidance/02-procedures/09-AIGO-AI-Monitoring-Procedure-v0.1.md
The corresponding implementation template is:
guidance/03-templates/09-AIGO-AI-Monitoring-Template-v0.1.md
The machine-readable structure is:
schemas/06-monitoring/06-AIGO-Monitoring-Schema-v0.1.json