AIGO — Monitoring Schema Documentation
1. Document Purpose
This document describes the machine-readable AIGO Monitoring Schema. The schema provides the structured representation of AI monitoring arrangements within the AIGO AI Governance Operating Framework. It is intended to support:- monitoring-plan definition;
- monitoring objectives;
- indicators and measurements;
- thresholds;
- alerts;
- monitoring frequency;
- monitoring ownership;
- data sources;
- automated and manual monitoring;
- trend analysis;
- escalation;
- monitoring responses;
- enhanced monitoring;
- risk monitoring;
- control monitoring;
- model and system performance monitoring;
- stakeholder and affected-person monitoring;
- compliance monitoring;
- incident detection;
- monitoring review; and
- monitoring closure.
2. Schema Information
3. Scope
The Monitoring Schema applies to ongoing or event-driven monitoring of AI systems and associated governance arrangements. Monitoring may cover:- AI system performance;
- model performance;
- data quality;
- data drift;
- model drift;
- system reliability;
- availability;
- resilience;
- safety;
- security;
- privacy;
- fairness;
- human oversight;
- control effectiveness;
- risk indicators;
- compliance;
- stakeholder impact;
- operational outcomes;
- supplier performance; and
- governance performance.
4. Object Model
The primary object represented by this schema is:- the AI System record;
- the risk record;
- the control record;
- individual measurements;
- incidents;
- changes;
- assessments;
- assurance activities; and
- evidence.
5. Core Identity
A Monitoring record should contain stable identity information. Key fields include:id;objectType;objectVersion;schemaVersion;- monitoring title or purpose;
- monitoring owner;
- status;
- scope; and
- monitoring objectives.
6. Monitoring Purpose
A monitoring arrangement should clearly establish why monitoring is required. Objectives may include:- detecting deterioration;
- verifying control operation;
- detecting emerging risks;
- detecting incidents;
- confirming compliance;
- tracking model performance;
- detecting drift;
- verifying approval conditions;
- monitoring affected-person impacts;
- monitoring supplier performance; or
- confirming continued suitability of an AI system.
7. Monitoring Scope
The scope should clearly identify what is monitored. Scope may include:- AI system;
- model;
- model version;
- data source;
- process;
- control;
- risk;
- stakeholder group;
- jurisdiction;
- environment;
- lifecycle stage;
- supplier; or
- governance function.
8. Monitoring Owner
Each monitoring arrangement should have an accountable owner. The monitoring owner is responsible for ensuring that:- indicators remain appropriate;
- data sources remain reliable;
- thresholds are maintained;
- monitoring occurs at the required frequency;
- alerts are reviewed;
- responses are initiated;
- escalations occur when required;
- monitoring evidence is retained; and
- the monitoring arrangement is periodically reviewed.
9. Monitoring Design
Monitoring should be designed around a defined logic:10. Monitoring Indicators
An indicator represents a defined measurable or observable condition. Examples include:- accuracy;
- error rate;
- false-positive rate;
- false-negative rate;
- latency;
- availability;
- uptime;
- data-quality score;
- drift metric;
- fairness metric;
- complaint rate;
- override rate;
- incident count;
- control-failure rate;
- risk-level movement; or
- supplier service performance.
11. Indicator Definition
Each important indicator should identify, where applicable:- name;
- definition;
- purpose;
- unit;
- measurement method;
- data source;
- owner;
- frequency;
- threshold;
- target;
- escalation condition; and
- response.
12. Baselines and Targets
Monitoring may use:- baseline values;
- target values;
- expected ranges;
- control limits;
- thresholds; or
- benchmark comparisons.
13. Thresholds
Thresholds determine when a monitoring result requires attention. Threshold levels may include:- condition;
- measurement basis;
- action;
- responsible role;
- escalation path; and
- response timeframe where applicable.
14. Threshold Governance
Thresholds should be reviewed when:- risk changes;
- system behavior changes;
- the model changes;
- controls change;
- incidents occur;
- performance deteriorates;
- operating context changes;
- regulatory requirements change; or
- monitoring experience demonstrates that thresholds are ineffective.
15. Monitoring Frequency
Monitoring frequency should be proportionate to:- system classification;
- risk level;
- potential severity of harm;
- rate of change;
- operating environment;
- data availability;
- control criticality; and
- regulatory requirements.
- continuous;
- real time;
- per transaction;
- hourly;
- daily;
- weekly;
- monthly;
- quarterly;
- annually;
- event triggered; or
- lifecycle triggered.
16. Automated Monitoring
Automated monitoring may be appropriate where:- measurements are available electronically;
- rapid detection is required;
- high monitoring frequency is necessary;
- manual observation is impractical; or
- system behavior must be continuously observed.
- configuration control;
- testing;
- access control;
- logging;
- validation;
- exception handling; and
- periodic review.
17. Manual Monitoring
Manual monitoring may be appropriate for:- contextual judgment;
- affected-person feedback;
- human oversight;
- stakeholder signals;
- qualitative risk indicators;
- governance decisions; and
- conditions that cannot reliably be automated.
- reviewer;
- frequency;
- method;
- evidence;
- escalation; and
- response.
18. Hybrid Monitoring
Many AIGO monitoring arrangements will combine automated measurement with human review. A typical pattern is:19. Monitoring Data Sources
Monitoring data sources may include:- AI system logs;
- application logs;
- model telemetry;
- data pipelines;
- operational databases;
- user feedback;
- complaints;
- incident systems;
- security systems;
- privacy systems;
- supplier reports;
- control assessments;
- assurance results; and
- external sources.
20. Data Quality
Monitoring quality depends on data quality. The monitoring arrangement should consider:- completeness;
- accuracy;
- timeliness;
- consistency;
- integrity;
- representativeness;
- availability;
- data transformation; and
- measurement error.
21. Monitoring Confidence
Where appropriate, monitoring should identify confidence or reliability considerations. These may include:- data-quality limitations;
- sampling limitations;
- measurement uncertainty;
- model uncertainty;
- system outages;
- incomplete coverage;
- delayed data; and
- external-source limitations.
22. Monitoring of AI Performance
AI-system monitoring may include:- accuracy;
- precision;
- recall;
- error rates;
- calibration;
- latency;
- throughput;
- availability;
- robustness;
- reliability;
- user outcomes; and
- other system-specific measures.
23. Model Drift
Where applicable, monitoring should identify model drift. Drift may include:- data drift;
- concept drift;
- performance drift;
- distribution shift;
- behavior changes;
- demographic performance changes; or
- changes in error patterns.
- investigation;
- enhanced monitoring;
- model validation;
- risk reassessment;
- change management;
- retraining;
- restriction; or
- suspension.
24. Data Drift
Monitoring may identify changes in input data. Indicators may include:- distribution changes;
- missingness;
- outliers;
- category changes;
- source changes;
- schema changes;
- volume changes; or
- quality degradation.
25. Fairness Monitoring
Where fairness is relevant, monitoring may include:- outcome disparity;
- error-rate differences;
- approval or rejection rates;
- group-level performance;
- representation;
- complaints;
- affected-person feedback; and
- changes in fairness indicators.
26. Safety Monitoring
Safety-relevant systems may require monitoring of:- unsafe outputs;
- system failures;
- near misses;
- override frequency;
- human interventions;
- critical alarms;
- fallback activation;
- safety incidents; and
- other safety indicators.
27. Security Monitoring
Security monitoring may include:- unauthorized access;
- anomalous activity;
- attack indicators;
- vulnerabilities;
- model extraction signals;
- prompt injection indicators;
- data exfiltration;
- credential misuse;
- suspicious system behavior; and
- security incidents.
28. Privacy Monitoring
Privacy monitoring may include:- unauthorized data access;
- data retention;
- data deletion;
- inappropriate data usage;
- privacy complaints;
- sensitive-data handling;
- unauthorized transfers;
- access violations; and
- privacy incidents.
29. Human Oversight Monitoring
Monitoring may evaluate the effectiveness of human oversight. Indicators may include:- review completion rate;
- override rate;
- escalation rate;
- review quality;
- response time;
- intervention frequency;
- unreviewed decision rate;
- training status; and
- competence indicators.
- automation bias;
- reduced vigilance;
- inappropriate workload;
- insufficient staffing; or
- control degradation.
30. Risk Monitoring
Monitoring may track risk-related indicators including:- residual risk;
- risk-level changes;
- treatment progress;
- control failures;
- accepted-risk exposure;
- emerging risks;
- incident trends; and
- risks exceeding tolerance.
31. Control Monitoring
Controls may themselves be monitored. Examples include:- control execution;
- exception rates;
- control failures;
- control review status;
- evidence availability;
- threshold breaches; and
- remediation status.
- control assessment;
- corrective action;
- risk reassessment;
- assurance; or
- change management.
32. Approval Condition Monitoring
Approval conditions may require specific monitoring. For example:33. Monitoring Alerts
An alert should identify an observed condition requiring review or action. An alert may include:- alert identifier;
- indicator;
- observed value;
- threshold;
- timestamp;
- affected AI system;
- severity;
- owner;
- response;
- escalation; and
- closure.
34. Monitoring Response
A monitoring response should identify the action taken after an alert or material observation. Possible responses include:- investigate;
- continue monitoring;
- increase monitoring;
- correct;
- restrict;
- suspend;
- escalate;
- initiate incident management;
- initiate change management;
- reassess risk;
- reassess control;
- seek approval;
- initiate assurance; or
- retire the affected system where authorized.
35. Escalation
Monitoring should identify when escalation is required. Escalation may be triggered by:- critical threshold;
- repeated warnings;
- rapid deterioration;
- safety concern;
- security concern;
- privacy concern;
- fairness concern;
- regulatory concern;
- significant affected-person impact;
- control failure; or
- other predefined governance conditions.
36. Enhanced Monitoring
Enhanced monitoring may be established when ordinary monitoring is insufficient. Triggers may include:- material change;
- significant incident;
- high-risk deployment;
- conditional approval;
- deteriorating performance;
- control failure;
- emerging risk;
- regulatory concern; or
- management direction.
- reason;
- duration;
- indicators;
- thresholds;
- owner;
- review frequency; and
- exit criteria.
37. Monitoring and Incident Management
Monitoring is often an early detection mechanism for incidents. The recommended relationship is:38. Monitoring and Change Management
Monitoring results may indicate that a change is required. Examples include:- model update;
- data pipeline change;
- control modification;
- threshold adjustment;
- infrastructure change;
- monitoring enhancement;
- workflow change; or
- intended-purpose review.
39. Monitoring and Risk Reassessment
Monitoring should feed risk management. A material change in an indicator may require:- risk reassessment;
- new risk identification;
- change to likelihood;
- change to impact;
- treatment adjustment;
- risk acceptance review; or
- escalation.
40. Monitoring and Assurance
Assurance may examine whether monitoring:- is appropriately designed;
- covers material risks;
- uses reliable data;
- has suitable thresholds;
- responds consistently;
- escalates appropriately; and
- produces sufficient evidence.
41. Monitoring Evidence
Monitoring should produce appropriate evidence. Evidence may include:- measurements;
- logs;
- dashboards;
- reports;
- alerts;
- review records;
- configuration;
- analysis;
- decisions;
- incident tickets; and
- remediation records.
42. Monitoring Data Integrity
The monitoring arrangement should protect the integrity of monitoring data. Controls may include:- immutable logs;
- access controls;
- timestamps;
- validation;
- checksums or hashes;
- source authentication;
- controlled configuration;
- audit trails; and
- restricted administrative access.
43. Monitoring Review
Monitoring arrangements should be periodically reviewed. The review should consider:- indicator relevance;
- indicator coverage;
- thresholds;
- response effectiveness;
- data quality;
- false alerts;
- missed events;
- emerging risks;
- changes in AI behavior;
- stakeholder feedback; and
- regulatory developments.
44. Monitoring Limitations
Monitoring limitations should be documented. Examples include:- incomplete data;
- delayed measurements;
- sampling limitations;
- unavailable systems;
- supplier limitations;
- measurement uncertainty;
- insufficient historical baseline;
- insufficient group representation; or
- technical limitations.
45. Monitoring Availability
Critical monitoring should have appropriate resilience. Considerations may include:- availability;
- redundancy;
- backup;
- failure detection;
- recovery;
- offline evidence preservation;
- monitoring-of-monitoring; and
- escalation when monitoring itself fails.
46. Monitoring of the Monitoring System
Where monitoring is relied upon for critical governance decisions, the organization should consider monitoring the monitoring mechanism itself. This may include:- data pipeline health;
- sensor or telemetry health;
- alert delivery;
- monitoring job completion;
- dashboard availability;
- threshold configuration;
- time synchronization; and
- evidence capture.
47. Monitoring and Third Parties
Third-party AI services may require monitoring of:- service availability;
- service-level performance;
- model or service changes;
- vendor notifications;
- security events;
- privacy events;
- contractual compliance;
- supplier incidents; and
- service degradation.
48. Monitoring Reporting
Monitoring records may support governance reporting. Typical reporting may include:- indicator status;
- threshold breaches;
- alert trends;
- system performance;
- risk movement;
- control performance;
- incidents;
- enhanced monitoring;
- unresolved alerts;
- monitoring gaps; and
- monitoring effectiveness.
49. Monitoring Metrics
Organizations may establish monitoring governance metrics such as:
Metrics should have defined formulas and owners.
50. Monitoring Closure
A monitoring arrangement may be closed when:- the AI system is retired;
- the underlying risk no longer exists;
- the monitored control is retired;
- monitoring is replaced;
- the monitoring requirement is transferred; or
- an approved governance decision authorizes closure.
51. Monitoring Validation Requirements
A valid Monitoring record should satisfy:Structural Validation
The JSON document must validate against:06-AIGO-Monitoring-Schema-v0.1.json
Objective Validation
The monitoring objective should be defined.Indicator Validation
Required indicators should have clear definitions.Data-Source Validation
Material indicators should reference appropriate data sources.Threshold Validation
Thresholds should be defined where threshold-based responses are required.Owner Validation
Monitoring should have accountable ownership.Frequency Validation
Required monitoring frequency should be defined.Escalation Validation
Material conditions should have appropriate escalation pathways.Evidence Validation
Monitoring results should produce or reference appropriate evidence.Reference Validation
Related systems, risks, controls, incidents, changes, assurance records, and approvals should resolve where required.52. Schema Limitations
JSON Schema cannot independently determine:- whether an indicator is meaningful;
- whether a threshold is appropriate;
- whether monitoring data is accurate;
- whether a monitoring system actually executed;
- whether alerts were properly interpreted;
- whether a response was sufficient;
- whether monitoring provides adequate risk visibility; or
- whether the organization has achieved effective monitoring.
53. Relationship to Templates
The Monitoring Schema corresponds primarily to:guidance/03-templates/09-AIGO-AI-Monitoring-Template-v0.1.md
It may also support monitoring records generated by:
- AI system governance;
- risk management;
- control management;
- incident management;
- change management;
- approval conditions;
- assurance;
- management review; and
- continuous improvement.
54. Relationship to Procedures
The schema should be used with:guidance/02-procedures/09-AIGO-AI-Monitoring-Procedure-v0.1.md
and, where applicable:
- AI governance;
- risk assessment;
- control assessment;
- approval;
- incident management;
- change management;
- assurance;
- risk acceptance;
- continuous improvement; and
- retirement procedures.
55. Framework Traceability
56. Monitoring Traceability Model
The recommended traceability chain is:57. Example Record
A conceptual Monitoring record may look like:58. Schema Registry Relationship
This schema is registered in:schemas/00-AIGO-Schema-Registry-v0.1.json
The registry should maintain:
- schema identifier;
- object type;
- version;
- controlled path;
- status;
- dependencies;
- referenced schemas;
- owner;
- documentation path; and
- traceability metadata.
59. Change and Version Management
Changes to the Monitoring Schema should be managed through AIGO change management. Potential impacts should be assessed against:- monitoring plans;
- indicators;
- thresholds;
- alert workflows;
- incident management;
- risk management;
- control monitoring;
- approval conditions;
- assurance;
- evidence;
- reporting;
- procedures;
- templates; and
- validation tooling.
60. Future Extensions
Future versions may support:- real-time event schemas;
- streaming metrics;
- formal time-series structures;
- statistical process-control metadata;
- anomaly detection metadata;
- automated threshold management;
- monitoring confidence scores;
- machine-readable alert rules;
- monitoring dependency graphs;
- monitoring resilience profiles; and
- machine-readable escalation policies.
61. Document Control
62. Document Status
Document: AIGO — Monitoring Schema Documentation Version: 0.1 Status: Draft Working Name: AIGO Full Name: AI Governance Operating Framework Document Identifier:AIGO-SCHEMA-DOC-006
Document Type: Schema Documentation
This document provides the human-readable interpretation, governance context, validation expectations, and traceability guidance for the AIGO Monitoring Schema.
End of Document