CRS v0.1: Calibration Reporting Standard
Version 0.1 (draft), last changed 2026-11-10.
Executive Summary
Autonomous AI agents may make consequential financial decisions: credit approval, trading, fraud flagging, portfolio management. None of the five public documents checked for this specification (listed in section 1) defines a calibration reporting format for such agents.
The Calibration Reporting Standard (CRS) is one proposal to address that gap. It defines a 13-field record that a deploying institution compiles for each agent and an independent attester completes, and a rule that derives a compliance state from the calibration rate and the expected calibration error (ECE). CRS is a measurement and disclosure format, designed as one input to an institution's own performance monitoring.
Separation of execution and accountability. Agents act in real time; the CRS record is compiled at the end of the reporting period.
Portability across jurisdictions. One record carries jurisdiction flags (F13) for each listed regulatory jurisdiction under which the attestation is issued.
Independence of the attestation function. The deployer does not attest to its own calibration; an independent third-party attester (F11) completes the record. This is the author's design choice, not derived from any regulation.
CRS v0.1 is a draft specification published by VACRA.AI for review and comment. All rights are reserved and no licence is granted at this time. Patent pending (two PCT international applications, filed 2026). No attestation or compliance-state service is offered.
1. The Accountability Gap: SR 26-2, the FCA Mills Review, the MAS FEAT Principles and AI Risk Management Guidelines, and the EU AI Act
The public documents below were checked for this specification. None defines a calibration reporting format for autonomous AI agents.
SR 26-2 (US Federal Reserve, FDIC and OCC, April 2026) states in footnote 3: "Generative AI and agentic AI models are novel and rapidly evolving. As such, they are not within the scope of this guidance." It adds that a banking organization's risk management and governance practices should guide governance of such tools.
The FCA Mills Review (July 2026) recommended (priority recommendation 5) that the FCA consider leading "a trusted framework for AI agent participation in financial services, clarifying how agents can be authorised, identified and held accountable." It concerns authorisation, identification and accountability, not calibration reporting.
MAS FEAT Principles (2018) set out fairness, ethics, accountability and transparency principles for AI in Singapore's financial sector and define no structured attestation format.
MAS Guidelines on Artificial Intelligence Risk Management (7 October 2026) say that, in ongoing monitoring of deployed AI, a financial institution should consider defining key metrics to be monitored and acceptable performance thresholds for each metric (paragraph 5.23(a)), and that AI use cases assessed as having high risk materiality be subject to regular re-validations by independent parties (paragraph 5.24). Sections 3 and 4 may be met from 7 October 2027 and Sections 5 and 6 by 7 October 2028 (paragraph 1.8). They do not define a calibration reporting format.
EU AI Act (Regulation (EU) 2024/1689) requires high-risk AI systems to allow automatic recording of events (Article 12) and, under Article 21(2), requires providers, on a reasoned request, to give competent authorities access to the automatically generated logs to the extent those logs are under their control. It defines no calibration threshold and no calibration reporting format (EUR-Lex).
CRS is one author's proposal for such a format.
2. CRS Specification
2.1 Architecture
CRS separates the accountability layer from the execution layer. Agent decisions and CRS attestation operate on independent timescales.
CRS v0.1 specifies the record only. How and when an institution gathers the underlying decision and outcome data is outside v0.1.
2.2 Three-Pillar Structure
CRS organizes its 13 fields across three accountability pillars.
Pillar 1: Attribution (F1-F4) answers: which agent, operating under whose authority, over what period?
Pillar 2: Disclosure (F5-F10) answers: was the agent calibrated, what were its error rates, and were failures disclosed?
Pillar 3: Governance (F11-F13) answers: who independently reviewed this record, when, and for which jurisdictions?
2.3 Pillar 1: Attribution (F1-F4)
In this draft, MUST, MANDATORY and similar words describe the proposed format only. No person or authority is obliged by them.
| Field | Name | Description | Requirement |
|---|---|---|---|
| F1 | agent_did | W3C DID uniquely identifying the autonomous AI agent; binds this attestation record to an agent identifier | MANDATORY |
| F2 | reporting_period_start | Start date of the calibration measurement period (ISO 8601) | MANDATORY; preferred period 90 calendar days |
| F3 | reporting_period_end | End date of the calibration measurement period (ISO 8601); must be after F2 | MANDATORY |
| F4 | system_scope | Structured sub-record: (a) decision_categories (list of authorized financial decision categories); (b) deployer_did (W3C DID of deploying institution); (c) authorization_chain_reference (hash or URI pointing to board resolution authorizing the agent) | MANDATORY |
2.4 Pillar 2: Disclosure (F5-F10)
| Field | Name | Threshold | Description |
|---|---|---|---|
| F5 | calibration_rate (CR) | CR ≥ 0.80 | Proportion of probabilistic predictions where stated confidence matches observed outcome frequency (±5pp tolerance), over all qualifying decisions in the reporting period |
| F6 | false_positive_rate (FPR) | No threshold in v0.1 | Proportion of negative-class instances incorrectly classified as positive |
| F7 | expected_calibration_error (ECE) | ECE < 0.05 (with CR ≥ 0.80) = COMPLIANT ECE > 0.10 = MANDATORY_RECALIBRATION | Probability-weighted mean absolute difference between predicted confidence and observed accuracy across equal-width bins, per Guo et al. (2017) |
| F8 | human_review_rate | ≥ 0.001 (1 in 1,000 minimum) | Proportion of autonomous decisions reviewed by a qualified human decision-maker; the minimum is the author's proposal |
| F9 | systematic_failure_disclosures | Failures disclosed in the record | Structured array of decision categories where CR < 0.80 or ECE > 0.10 occurred; empty array = no failures; each entry records decision category, failure type, observed CR, observed ECE |
| F10 | reattestation_event_log | MANDATORY | Structured array of material model updates triggering re-attestation since prior record; empty = no events; each entry records trigger condition, update description, re-attestation date |
The F5 and F7 values are the author's proposal; they are not derived from any regulation. A record reports the achieved figures whether or not they meet the proposed values. F6 carries no threshold in v0.1; it is recorded and not threshold-gated.
The F8 floor of 0.001 is an author-chosen minimum for recording purposes; it is not a statement about adequate human oversight.
2.5 Pillar 3: Governance (F11-F13)
| Field | Name | Description | Requirement |
|---|---|---|---|
| F11 | attester_did | W3C DID of the independent third party; MUST be independent from the deploying institution in F4(b) | MANDATORY; independent of the deployer |
| F12 | attestation_date | Date the attester completed the record (ISO 8601) | MANDATORY |
| F13 | regulatory_jurisdiction | Array identifying the regulatory jurisdiction(s) under which this attestation is issued; enables a single record to serve multiple jurisdictions for cross-border agents. Enumerated values: ADGM_FSRA, MAS, FCA, FSB, FRB_FDIC_OCC, ESMA, HKMA, APRA, OSFI. These values are identifiers only; listing a body does not mean it has reviewed, recognised or accepted CRS. | Enumerated |
Attester qualification. CRS v0.1 does not specify how an attester is selected or qualified. To the author's knowledge, no attester has completed a CRS record. Whether an attester is qualified is a matter for the party relying on the record.
2.6 Compliance State Machine
Open items in v0.1. v0.1 leaves several computation details unspecified, and does not say which model changes count as material for F10. Two attesters could compute different values from the same data.
A compliance state is a label computed by the rule below. It is not an assessment of safety, fitness or regulatory compliance. F8 and F9 are disclosure fields and do not change the state.
| State | Condition | Proposed action |
|---|---|---|
| COMPLIANT | CR ≥ 0.80 AND ECE < 0.05 | None under the draft |
| REQUIRES_REMEDIATION | CR < 0.80 OR (ECE ≥ 0.05 AND ECE ≤ 0.10), provided ECE ≤ 0.10 | Re-attestation within 30 calendar days |
| MANDATORY_RECALIBRATION | ECE > 0.10 | Re-attestation within 72 hours |
States are ordered MANDATORY_RECALIBRATION > REQUIRES_REMEDIATION > COMPLIANT, and the determination is the most severe state whose condition holds. The 30-day and 72-hour windows are the author's proposed response windows, not obligations imposed by any authority.
3. Scope
CRS v0.1 is designed for AI agents in financial services. Sector-specific mappings are not part of v0.1.
A COMPLIANT record is something a deployer could present when asked for evidence of calibration. To the author's knowledge no insurer or supervisor has adopted CRS. A CRS record is not evidence of insurability, regulatory compliance or safety.
High-frequency agents. The reporting period aggregates CR and ECE over all qualifying decisions in the period, whatever the decision frequency.
4. Regulatory Context (author's mapping)
CRS does not replace any existing regulatory framework. It is an independent proposal for a calibration measurement and disclosure format. The mapping below is the author's own.
| Jurisdiction | Author's observation | Fields the author considers related (not reviewed by any regulator) |
|---|---|---|
| US SR 26-2: Supervisory Guidance on Model Risk Management (17 April 2026) |
Agentic AI explicitly excluded (Footnote 3)
|
F1–F4 attribution; F5–F10 disclosure; F13 FRB_FDIC_OCC (the Federal Reserve Board, the FDIC and the OCC, the three issuers of SR 26-2) |
| UK Mills Review (July 2026) |
Recommendation 5: the FCA to consider a framework for how agents are authorised, identified and held accountable
Recommendation 5 does not define calibration metrics
|
F4 authorization chain; F5–F7 calibration; F11 independent attester |
| EU: AI Act Regulation (EU) 2024/1689 |
Article 12: high-risk AI systems must technically allow automatic recording of events
Article 15(3): the levels of accuracy and the relevant accuracy metrics of high-risk AI systems shall be declared in the accompanying instructions of use. The text does not use the word calibration and defines no calibration threshold or calibration reporting format.
|
None. CRS does not implement Article 12 or Article 15(3). |
ISO/IEC 42001:2023 is a management system standard for AI. The author considers that a CRS record could be used as one input to an organisation's own monitoring and evaluation of AI. CRS is not an ISO artefact.
Multi-Jurisdiction Deployment. CRS v0.1 applies the same draft thresholds to every F13 jurisdiction listed in a record. Comments on the thresholds are welcome at contact@vacra.ai.
5. Deployment
5.1 Roles and Responsibilities
Deployer: The institution that operates the autonomous AI agent. Responsible for generating the CRS record at end of each reporting period, populating F1-F10 from agent monitoring data, and engaging an independent attester to complete F11-F13. Selection and qualification of attesters are outside v0.1 (see section 2.5).
Attester: The independent third party identified by F11. Completes the record (F11 attester DID, F12 attestation date).
Regulator: The supervisory authority identified by F13. A regulator could receive the CRS record.
5.2 Example record (illustrative)
Example CRS Record (COMPLIANT)
{
"agent_did": "did:web:bank.example.com:agents:credit-agent-vq7x",
"reporting_period_start": "2026-07-01",
"reporting_period_end": "2026-09-28",
"system_scope": {
"decision_categories": ["credit_origination", "credit_limit_adjustment"],
"deployer_did": "did:web:bank.example.com",
"authorization_chain_reference":
"https://bank.example.com/governance/board-resolution-2026-Q2-rev-kx4.pdf"
},
"calibration_rate": 0.87,
"false_positive_rate": 0.042,
"expected_calibration_error": 0.032,
"human_review_rate": 0.003,
"systematic_failure_disclosures": [],
"reattestation_event_log": [],
"attester_did": "did:web:auditor.example.org:attesters:attester-m3zt",
"attestation_date": "2026-10-02",
"regulatory_jurisdiction": ["FRB_FDIC_OCC", "FCA"]
}This illustrative record (constructed values, not measured data) evaluates to COMPLIANT: CR = 0.87 (≥ 0.80) and ECE = 0.032 (< 0.05). The empty systematic_failure_disclosures array reports that no decision category had CR below 0.80 or ECE above 0.10 during the period.
6. How to Engage
Comments and reviews on CRS v0.1 are welcome at contact@vacra.ai, including comments on any jurisdiction row in Section 4. Comments are treated as non-confidential, and the author may use them without obligation or compensation. Please do not send confidential information or proposals that you regard as your own invention. The JSON Schema is available for review on request. A position paper proposing CRS was sent to the FSB, MAS, FCA, HKMA and ADGM in 2026; sending it does not imply review, acceptance or endorsement.
CRS v0.1 is authored and maintained by VACRA.AI, an independent project of the author. The text of this page is copyright E Chan, all rights reserved. VACRA.AI and the author have a commercial interest in CRS adoption.
Glossary
| Term | Definition |
|---|---|
| Attestation | Independent third-party review of a deployer's CRS record, recorded in F11 to F13 |
| Calibration | The property whereby an agent's stated confidence probabilities match observed outcome frequencies; a well-calibrated agent that claims 80% confidence is correct 80% of the time |
| CR (Calibration Rate) | CRS Field F5; proportion of predictions where stated confidence matches observed outcome frequency within a 5 percentage point tolerance |
| ECE | Expected Calibration Error; CRS Field F7; probability-weighted mean absolute difference between predicted confidence and observed accuracy; lower is better; 0 = perfect calibration |
| SR 26-2 | US Federal Reserve, FDIC, and OCC guidance on model risk management (April 2026); explicitly excludes agentic AI from scope |
References
Specification Resources
Post-Consultation Position Paper: Attribution Infrastructure for Autonomous Financial Agents. (2026). SSRN Working Paper 7511058. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7511058 The SSRN paper sets out an earlier formulation of the 13 fields and proposed thresholds. Where it differs from this page, this page is the current draft.
CRS Schema (JSON Schema, draft 2020-12): available on request from contact@vacra.ai
Literature on calibration measurement (context; these works do not set the CRS thresholds)
Guo et al. (2017). On Calibration of Modern Neural Networks.
Regulatory Sources
Board of Governors of the Federal Reserve System, FDIC, and OCC (2026, April 17). SR 26-2: Supervisory guidance on model risk management. Note: SR 26-2 explicitly excludes generative and agentic AI models from scope (Footnote 3).
Mills, S. (2026, July). AI and the future of retail financial services (Mills Review). Financial Conduct Authority. fca.org.uk
Monetary Authority of Singapore (2018). Principles to Promote Fairness, Ethics, Accountability and Transparency (FEAT) in the Use of Artificial Intelligence and Data Analytics in Singapore's Financial Sector.
Monetary Authority of Singapore (2026, 7 October). Guidelines on Artificial Intelligence Risk Management.
European Parliament and Council (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act). Application dates differ by obligation; check the current text.