Skip to main content

AIGO — Monitoring Schema Documentation

1. Document Purpose

This document describes the machine-readable AIGO Monitoring Schema. The schema provides the structured representation of AI monitoring arrangements within the AIGO AI Governance Operating Framework. It is intended to support:
  • monitoring-plan definition;
  • monitoring objectives;
  • indicators and measurements;
  • thresholds;
  • alerts;
  • monitoring frequency;
  • monitoring ownership;
  • data sources;
  • automated and manual monitoring;
  • trend analysis;
  • escalation;
  • monitoring responses;
  • enhanced monitoring;
  • risk monitoring;
  • control monitoring;
  • model and system performance monitoring;
  • stakeholder and affected-person monitoring;
  • compliance monitoring;
  • incident detection;
  • monitoring review; and
  • monitoring closure.
The JSON schema defines structural requirements. This documentation explains the intended meaning, governance context, validation expectations, and traceability model for monitoring records.

2. Schema Information


3. Scope

The Monitoring Schema applies to ongoing or event-driven monitoring of AI systems and associated governance arrangements. Monitoring may cover:
  • AI system performance;
  • model performance;
  • data quality;
  • data drift;
  • model drift;
  • system reliability;
  • availability;
  • resilience;
  • safety;
  • security;
  • privacy;
  • fairness;
  • human oversight;
  • control effectiveness;
  • risk indicators;
  • compliance;
  • stakeholder impact;
  • operational outcomes;
  • supplier performance; and
  • governance performance.
Monitoring should be proportionate to system classification, risk, intended purpose, affected-person impact, and applicable requirements.

4. Object Model

The primary object represented by this schema is:
A Monitoring record represents an approved arrangement for observing defined AI-related conditions over time or in response to specified events. The Monitoring record should remain distinct from:
  • the AI System record;
  • the risk record;
  • the control record;
  • individual measurements;
  • incidents;
  • changes;
  • assessments;
  • assurance activities; and
  • evidence.
Related records should normally be connected through stable identifiers.

5. Core Identity

A Monitoring record should contain stable identity information. Key fields include:
  • id;
  • objectType;
  • objectVersion;
  • schemaVersion;
  • monitoring title or purpose;
  • monitoring owner;
  • status;
  • scope; and
  • monitoring objectives.
Example:
The monitoring identifier should remain stable throughout the monitoring arrangement unless a materially distinct monitoring arrangement is established.

6. Monitoring Purpose

A monitoring arrangement should clearly establish why monitoring is required. Objectives may include:
  • detecting deterioration;
  • verifying control operation;
  • detecting emerging risks;
  • detecting incidents;
  • confirming compliance;
  • tracking model performance;
  • detecting drift;
  • verifying approval conditions;
  • monitoring affected-person impacts;
  • monitoring supplier performance; or
  • confirming continued suitability of an AI system.
Monitoring objectives should be linked to the governance requirement or risk that they support.

7. Monitoring Scope

The scope should clearly identify what is monitored. Scope may include:
  • AI system;
  • model;
  • model version;
  • data source;
  • process;
  • control;
  • risk;
  • stakeholder group;
  • jurisdiction;
  • environment;
  • lifecycle stage;
  • supplier; or
  • governance function.
Scope should identify important inclusions and exclusions.

8. Monitoring Owner

Each monitoring arrangement should have an accountable owner. The monitoring owner is responsible for ensuring that:
  • indicators remain appropriate;
  • data sources remain reliable;
  • thresholds are maintained;
  • monitoring occurs at the required frequency;
  • alerts are reviewed;
  • responses are initiated;
  • escalations occur when required;
  • monitoring evidence is retained; and
  • the monitoring arrangement is periodically reviewed.
Operational measurement may be performed by a separate team.

9. Monitoring Design

Monitoring should be designed around a defined logic:
The design should be capable of demonstrating how an observed condition leads to an appropriate governance response.

10. Monitoring Indicators

An indicator represents a defined measurable or observable condition. Examples include:
  • accuracy;
  • error rate;
  • false-positive rate;
  • false-negative rate;
  • latency;
  • availability;
  • uptime;
  • data-quality score;
  • drift metric;
  • fairness metric;
  • complaint rate;
  • override rate;
  • incident count;
  • control-failure rate;
  • risk-level movement; or
  • supplier service performance.
Indicators should have clear definitions and measurement methods.

11. Indicator Definition

Each important indicator should identify, where applicable:
  • name;
  • definition;
  • purpose;
  • unit;
  • measurement method;
  • data source;
  • owner;
  • frequency;
  • threshold;
  • target;
  • escalation condition; and
  • response.
An indicator that cannot be interpreted consistently is unlikely to provide reliable governance value.

12. Baselines and Targets

Monitoring may use:
  • baseline values;
  • target values;
  • expected ranges;
  • control limits;
  • thresholds; or
  • benchmark comparisons.
The basis for the baseline or target should be documented where it materially affects decisions. Changes to baseline or target values should be governed where they could materially change risk visibility.

13. Thresholds

Thresholds determine when a monitoring result requires attention. Threshold levels may include:
The exact schema enumeration is authoritative. Thresholds should specify:
  • condition;
  • measurement basis;
  • action;
  • responsible role;
  • escalation path; and
  • response timeframe where applicable.

14. Threshold Governance

Thresholds should be reviewed when:
  • risk changes;
  • system behavior changes;
  • the model changes;
  • controls change;
  • incidents occur;
  • performance deteriorates;
  • operating context changes;
  • regulatory requirements change; or
  • monitoring experience demonstrates that thresholds are ineffective.
Thresholds should not be changed solely to reduce the number of alerts without documented justification.

15. Monitoring Frequency

Monitoring frequency should be proportionate to:
  • system classification;
  • risk level;
  • potential severity of harm;
  • rate of change;
  • operating environment;
  • data availability;
  • control criticality; and
  • regulatory requirements.
Possible frequencies include:
  • continuous;
  • real time;
  • per transaction;
  • hourly;
  • daily;
  • weekly;
  • monthly;
  • quarterly;
  • annually;
  • event triggered; or
  • lifecycle triggered.
The monitoring arrangement should specify the applicable frequency.

16. Automated Monitoring

Automated monitoring may be appropriate where:
  • measurements are available electronically;
  • rapid detection is required;
  • high monitoring frequency is necessary;
  • manual observation is impractical; or
  • system behavior must be continuously observed.
Automated monitoring should still be subject to:
  • configuration control;
  • testing;
  • access control;
  • logging;
  • validation;
  • exception handling; and
  • periodic review.
Automation does not remove accountability.

17. Manual Monitoring

Manual monitoring may be appropriate for:
  • contextual judgment;
  • affected-person feedback;
  • human oversight;
  • stakeholder signals;
  • qualitative risk indicators;
  • governance decisions; and
  • conditions that cannot reliably be automated.
Manual monitoring should define:
  • reviewer;
  • frequency;
  • method;
  • evidence;
  • escalation; and
  • response.

18. Hybrid Monitoring

Many AIGO monitoring arrangements will combine automated measurement with human review. A typical pattern is:
This can provide both speed and contextual judgment.

19. Monitoring Data Sources

Monitoring data sources may include:
  • AI system logs;
  • application logs;
  • model telemetry;
  • data pipelines;
  • operational databases;
  • user feedback;
  • complaints;
  • incident systems;
  • security systems;
  • privacy systems;
  • supplier reports;
  • control assessments;
  • assurance results; and
  • external sources.
The source should be sufficiently reliable and appropriate for the indicator.

20. Data Quality

Monitoring quality depends on data quality. The monitoring arrangement should consider:
  • completeness;
  • accuracy;
  • timeliness;
  • consistency;
  • integrity;
  • representativeness;
  • availability;
  • data transformation; and
  • measurement error.
Material monitoring conclusions should not rely on demonstrably unreliable data without appropriate qualification.

21. Monitoring Confidence

Where appropriate, monitoring should identify confidence or reliability considerations. These may include:
  • data-quality limitations;
  • sampling limitations;
  • measurement uncertainty;
  • model uncertainty;
  • system outages;
  • incomplete coverage;
  • delayed data; and
  • external-source limitations.
Material limitations should be recorded and considered when interpreting results.

22. Monitoring of AI Performance

AI-system monitoring may include:
  • accuracy;
  • precision;
  • recall;
  • error rates;
  • calibration;
  • latency;
  • throughput;
  • availability;
  • robustness;
  • reliability;
  • user outcomes; and
  • other system-specific measures.
Performance indicators should reflect the AI system’s intended purpose. A performance metric is useful only when it is connected to a meaningful governance objective.

23. Model Drift

Where applicable, monitoring should identify model drift. Drift may include:
  • data drift;
  • concept drift;
  • performance drift;
  • distribution shift;
  • behavior changes;
  • demographic performance changes; or
  • changes in error patterns.
Drift thresholds should be defined before operational monitoring where practical. Material drift may require:
  • investigation;
  • enhanced monitoring;
  • model validation;
  • risk reassessment;
  • change management;
  • retraining;
  • restriction; or
  • suspension.

24. Data Drift

Monitoring may identify changes in input data. Indicators may include:
  • distribution changes;
  • missingness;
  • outliers;
  • category changes;
  • source changes;
  • schema changes;
  • volume changes; or
  • quality degradation.
Data drift may affect model performance and therefore should be connected to risk and change processes where material.

25. Fairness Monitoring

Where fairness is relevant, monitoring may include:
  • outcome disparity;
  • error-rate differences;
  • approval or rejection rates;
  • group-level performance;
  • representation;
  • complaints;
  • affected-person feedback; and
  • changes in fairness indicators.
Fairness thresholds should reflect the organization’s approved methodology and applicable legal context.

26. Safety Monitoring

Safety-relevant systems may require monitoring of:
  • unsafe outputs;
  • system failures;
  • near misses;
  • override frequency;
  • human interventions;
  • critical alarms;
  • fallback activation;
  • safety incidents; and
  • other safety indicators.
Safety-related monitoring should have clear escalation requirements.

27. Security Monitoring

Security monitoring may include:
  • unauthorized access;
  • anomalous activity;
  • attack indicators;
  • vulnerabilities;
  • model extraction signals;
  • prompt injection indicators;
  • data exfiltration;
  • credential misuse;
  • suspicious system behavior; and
  • security incidents.
Security monitoring should integrate with the organization’s security operations where applicable.

28. Privacy Monitoring

Privacy monitoring may include:
  • unauthorized data access;
  • data retention;
  • data deletion;
  • inappropriate data usage;
  • privacy complaints;
  • sensitive-data handling;
  • unauthorized transfers;
  • access violations; and
  • privacy incidents.
Privacy monitoring requirements should remain aligned with applicable privacy governance processes.

29. Human Oversight Monitoring

Monitoring may evaluate the effectiveness of human oversight. Indicators may include:
  • review completion rate;
  • override rate;
  • escalation rate;
  • review quality;
  • response time;
  • intervention frequency;
  • unreviewed decision rate;
  • training status; and
  • competence indicators.
Unexpected changes may indicate:
  • automation bias;
  • reduced vigilance;
  • inappropriate workload;
  • insufficient staffing; or
  • control degradation.

30. Risk Monitoring

Monitoring may track risk-related indicators including:
  • residual risk;
  • risk-level changes;
  • treatment progress;
  • control failures;
  • accepted-risk exposure;
  • emerging risks;
  • incident trends; and
  • risks exceeding tolerance.
Where monitoring indicates a material risk change, the Risk record should be reassessed in accordance with the relevant procedure.

31. Control Monitoring

Controls may themselves be monitored. Examples include:
  • control execution;
  • exception rates;
  • control failures;
  • control review status;
  • evidence availability;
  • threshold breaches; and
  • remediation status.
Control monitoring may identify the need for:
  • control assessment;
  • corrective action;
  • risk reassessment;
  • assurance; or
  • change management.

32. Approval Condition Monitoring

Approval conditions may require specific monitoring. For example:
This allows conditional approvals to remain actively governed rather than becoming static records.

33. Monitoring Alerts

An alert should identify an observed condition requiring review or action. An alert may include:
  • alert identifier;
  • indicator;
  • observed value;
  • threshold;
  • timestamp;
  • affected AI system;
  • severity;
  • owner;
  • response;
  • escalation; and
  • closure.
Alerts may lead to incident, change, risk, control, assurance, or improvement records.

34. Monitoring Response

A monitoring response should identify the action taken after an alert or material observation. Possible responses include:
  • investigate;
  • continue monitoring;
  • increase monitoring;
  • correct;
  • restrict;
  • suspend;
  • escalate;
  • initiate incident management;
  • initiate change management;
  • reassess risk;
  • reassess control;
  • seek approval;
  • initiate assurance; or
  • retire the affected system where authorized.
The response should be proportionate to the observed condition.

35. Escalation

Monitoring should identify when escalation is required. Escalation may be triggered by:
  • critical threshold;
  • repeated warnings;
  • rapid deterioration;
  • safety concern;
  • security concern;
  • privacy concern;
  • fairness concern;
  • regulatory concern;
  • significant affected-person impact;
  • control failure; or
  • other predefined governance conditions.
The monitoring record should identify the escalation authority where applicable.

36. Enhanced Monitoring

Enhanced monitoring may be established when ordinary monitoring is insufficient. Triggers may include:
  • material change;
  • significant incident;
  • high-risk deployment;
  • conditional approval;
  • deteriorating performance;
  • control failure;
  • emerging risk;
  • regulatory concern; or
  • management direction.
Enhanced monitoring should define:
  • reason;
  • duration;
  • indicators;
  • thresholds;
  • owner;
  • review frequency; and
  • exit criteria.

37. Monitoring and Incident Management

Monitoring is often an early detection mechanism for incidents. The recommended relationship is:
An alert is not automatically an incident. The organization should apply its incident classification criteria.

38. Monitoring and Change Management

Monitoring results may indicate that a change is required. Examples include:
  • model update;
  • data pipeline change;
  • control modification;
  • threshold adjustment;
  • infrastructure change;
  • monitoring enhancement;
  • workflow change; or
  • intended-purpose review.
Material changes should follow the AIGO Change process.

39. Monitoring and Risk Reassessment

Monitoring should feed risk management. A material change in an indicator may require:
  • risk reassessment;
  • new risk identification;
  • change to likelihood;
  • change to impact;
  • treatment adjustment;
  • risk acceptance review; or
  • escalation.
The Monitoring record should reference resulting risk actions where applicable.

40. Monitoring and Assurance

Assurance may examine whether monitoring:
  • is appropriately designed;
  • covers material risks;
  • uses reliable data;
  • has suitable thresholds;
  • responds consistently;
  • escalates appropriately; and
  • produces sufficient evidence.
Assurance findings should remain independent records.

41. Monitoring Evidence

Monitoring should produce appropriate evidence. Evidence may include:
  • measurements;
  • logs;
  • dashboards;
  • reports;
  • alerts;
  • review records;
  • configuration;
  • analysis;
  • decisions;
  • incident tickets; and
  • remediation records.
Evidence should be retained according to applicable requirements. The Evidence Schema provides the machine-readable evidence structure.

42. Monitoring Data Integrity

The monitoring arrangement should protect the integrity of monitoring data. Controls may include:
  • immutable logs;
  • access controls;
  • timestamps;
  • validation;
  • checksums or hashes;
  • source authentication;
  • controlled configuration;
  • audit trails; and
  • restricted administrative access.
Monitoring conclusions should not rely on data whose integrity cannot reasonably be established where integrity is material.

43. Monitoring Review

Monitoring arrangements should be periodically reviewed. The review should consider:
  • indicator relevance;
  • indicator coverage;
  • thresholds;
  • response effectiveness;
  • data quality;
  • false alerts;
  • missed events;
  • emerging risks;
  • changes in AI behavior;
  • stakeholder feedback; and
  • regulatory developments.
Monitoring that no longer supports a meaningful governance objective should be changed or retired.

44. Monitoring Limitations

Monitoring limitations should be documented. Examples include:
  • incomplete data;
  • delayed measurements;
  • sampling limitations;
  • unavailable systems;
  • supplier limitations;
  • measurement uncertainty;
  • insufficient historical baseline;
  • insufficient group representation; or
  • technical limitations.
Material limitations should be considered when making governance decisions.

45. Monitoring Availability

Critical monitoring should have appropriate resilience. Considerations may include:
  • availability;
  • redundancy;
  • backup;
  • failure detection;
  • recovery;
  • offline evidence preservation;
  • monitoring-of-monitoring; and
  • escalation when monitoring itself fails.
Failure of critical monitoring may itself require governance escalation.

46. Monitoring of the Monitoring System

Where monitoring is relied upon for critical governance decisions, the organization should consider monitoring the monitoring mechanism itself. This may include:
  • data pipeline health;
  • sensor or telemetry health;
  • alert delivery;
  • monitoring job completion;
  • dashboard availability;
  • threshold configuration;
  • time synchronization; and
  • evidence capture.
This helps detect silent monitoring failures.

47. Monitoring and Third Parties

Third-party AI services may require monitoring of:
  • service availability;
  • service-level performance;
  • model or service changes;
  • vendor notifications;
  • security events;
  • privacy events;
  • contractual compliance;
  • supplier incidents; and
  • service degradation.
The organization should establish contractual requirements for supplier monitoring where necessary.

48. Monitoring Reporting

Monitoring records may support governance reporting. Typical reporting may include:
  • indicator status;
  • threshold breaches;
  • alert trends;
  • system performance;
  • risk movement;
  • control performance;
  • incidents;
  • enhanced monitoring;
  • unresolved alerts;
  • monitoring gaps; and
  • monitoring effectiveness.
Reports should identify the relevant period and source.

49. Monitoring Metrics

Organizations may establish monitoring governance metrics such as: Metrics should have defined formulas and owners.

50. Monitoring Closure

A monitoring arrangement may be closed when:
  • the AI system is retired;
  • the underlying risk no longer exists;
  • the monitored control is retired;
  • monitoring is replaced;
  • the monitoring requirement is transferred; or
  • an approved governance decision authorizes closure.
Closure should preserve historical monitoring records as required. Monitoring should not be deleted merely because the system is no longer operational.

51. Monitoring Validation Requirements

A valid Monitoring record should satisfy:

Structural Validation

The JSON document must validate against: 06-AIGO-Monitoring-Schema-v0.1.json

Objective Validation

The monitoring objective should be defined.

Indicator Validation

Required indicators should have clear definitions.

Data-Source Validation

Material indicators should reference appropriate data sources.

Threshold Validation

Thresholds should be defined where threshold-based responses are required.

Owner Validation

Monitoring should have accountable ownership.

Frequency Validation

Required monitoring frequency should be defined.

Escalation Validation

Material conditions should have appropriate escalation pathways.

Evidence Validation

Monitoring results should produce or reference appropriate evidence.

Reference Validation

Related systems, risks, controls, incidents, changes, assurance records, and approvals should resolve where required.

52. Schema Limitations

JSON Schema cannot independently determine:
  • whether an indicator is meaningful;
  • whether a threshold is appropriate;
  • whether monitoring data is accurate;
  • whether a monitoring system actually executed;
  • whether alerts were properly interpreted;
  • whether a response was sufficient;
  • whether monitoring provides adequate risk visibility; or
  • whether the organization has achieved effective monitoring.
These matters require operational controls, qualified review, evidence, testing, and assurance.

53. Relationship to Templates

The Monitoring Schema corresponds primarily to: guidance/03-templates/09-AIGO-AI-Monitoring-Template-v0.1.md It may also support monitoring records generated by:
  • AI system governance;
  • risk management;
  • control management;
  • incident management;
  • change management;
  • approval conditions;
  • assurance;
  • management review; and
  • continuous improvement.
The Markdown template supports human operation. The JSON schema provides the machine-readable representation.

54. Relationship to Procedures

The schema should be used with: guidance/02-procedures/09-AIGO-AI-Monitoring-Procedure-v0.1.md and, where applicable:
  • AI governance;
  • risk assessment;
  • control assessment;
  • approval;
  • incident management;
  • change management;
  • assurance;
  • risk acceptance;
  • continuous improvement; and
  • retirement procedures.
The procedures define how monitoring is established, operated, reviewed, escalated, and closed.

55. Framework Traceability


56. Monitoring Traceability Model

The recommended traceability chain is:
This allows the organization to demonstrate how operational signals are converted into governance action.

57. Example Record

A conceptual Monitoring record may look like:
The actual record must conform to the authoritative JSON schema.

58. Schema Registry Relationship

This schema is registered in: schemas/00-AIGO-Schema-Registry-v0.1.json The registry should maintain:
  • schema identifier;
  • object type;
  • version;
  • controlled path;
  • status;
  • dependencies;
  • referenced schemas;
  • owner;
  • documentation path; and
  • traceability metadata.

59. Change and Version Management

Changes to the Monitoring Schema should be managed through AIGO change management. Potential impacts should be assessed against:
  • monitoring plans;
  • indicators;
  • thresholds;
  • alert workflows;
  • incident management;
  • risk management;
  • control monitoring;
  • approval conditions;
  • assurance;
  • evidence;
  • reporting;
  • procedures;
  • templates; and
  • validation tooling.
Breaking changes should include migration guidance where appropriate.

60. Future Extensions

Future versions may support:
  • real-time event schemas;
  • streaming metrics;
  • formal time-series structures;
  • statistical process-control metadata;
  • anomaly detection metadata;
  • automated threshold management;
  • monitoring confidence scores;
  • machine-readable alert rules;
  • monitoring dependency graphs;
  • monitoring resilience profiles; and
  • machine-readable escalation policies.
Extensions should preserve stable indicator identity and clear traceability to governance objectives.

61. Document Control


62. Document Status

Document: AIGO — Monitoring Schema Documentation Version: 0.1 Status: Draft Working Name: AIGO Full Name: AI Governance Operating Framework Document Identifier: AIGO-SCHEMA-DOC-006 Document Type: Schema Documentation This document provides the human-readable interpretation, governance context, validation expectations, and traceability guidance for the AIGO Monitoring Schema. End of Document