Turn evaluation runs into exportable regulatory audit packages.

Export Annex IV sections, GMLP summaries or model-governance packs. Edit the manifest and watch the hash break.

annex-iv-technical-documentation.pdf

Generated from run_7f3c2a · 2026-11-04

PDF

2(g)Validation and testing procedures, metrics, signed test logsrun_7f3c2a, 4 suites, 9,420 cases
2(e)Human oversight measures assessedescalation assertions, 312 cases
3Capabilities and limitations in performanceper-slice accuracy, stress-variant report
4Appropriateness of performance metricsrubric v3 rationale, judge κ 0.87
6Relevant changes through the lifecyclePR #482 to #511 change log

manifest.json

{
  "run_id": "run_7f3c2a",
  "created_at": "2026-11-04T02:14:09Z",
  "agent": {
    "name": "prior-auth",
    "commit": "c91d02e",
    "model": "pinned@2026-10-01"
  },
  "suites": [
    "healthcare/clinical-prior-auth-v2@2.3.0",
    "prior-auth-internal@1.0.0"
  ],
  "judges": "ensemble.prior-auth.v3",
  "results": {
    "cases": 9420,
    "critical_pass_rate": 0.998,
    "judge_mean": 0.95
  }
}

signedcomputing

nowcomputing

Verifying

Maximum fine for breaching high-risk AI system obligations (worldwide annual turnover, whichever is higher). Prohibited practices reach €35M or 7%.: €15M or 3%; Maximum fine for breaching high-risk AI system obligations (worldwide annual turnover, whichever is higher). Prohibited practices reach €35M or 7%.; Regulation (EU) 2024/1689 (EU AI Act), Article 99
Creditors must state the specific principal reasons for adverse action, including when a complex model made the decision; a generic checklist reason that does not reflect the real factors is not enough.: Specific reasons; Creditors must state the specific principal reasons for adverse action, including when a complex model made the decision; a generic checklist reason that does not reflect the real factors is not enough.; ECOA and Regulation B, 12 CFR 1002.9(b)(2) and Official Interpretations (CFPB Circulars 2022-03 and 2023-03, withdrawn May 2025, applied this to algorithms)
A tribunal held an airline responsible for incorrect fare information its customer-service chatbot gave a passenger.: Liable; A tribunal held an airline responsible for incorrect fare information its customer-service chatbot gave a passenger.; Moffatt v. Air Canada, 2024 BCCRT 149

EU AI Act Annex IV technical documentation, section by section.

What each Annex IV point asks for, the evidence an GONATURELLE run supplies, and what remains the provider's to write. As of November 2026, Annex III high-risk obligations apply from 2 December 2027 under the Digital Omnibus amendments.

Annex IV point	Requirement	GONATURELLE evidence	Completed by
2(e)	Assessment of the human oversight measures needed under Article 14	Escalation and hand-off assertions, override rates per case slice	Shared
2(g)	Validation and testing procedures, data used, metrics for accuracy, robustness and discriminatory impact, dated and signed test logs and reports	Suite versions, case lineage, assertion and judge results, signed run manifests	GONATURELLE evidence
3	Capabilities and limitations in performance, including accuracy for specific groups	Per-slice results by population, language and edge-case category	Shared
4	Appropriateness of the performance metrics	Rubric rationale, judge agreement with experts (kappa, AC1)	GONATURELLE evidence
5	Risk management system under Article 9	Failure clusters and the regression cases added for each	Provider, with evidence
6	Relevant changes made through the lifecycle	Gate history: every prompt, model and tool change with its run	GONATURELLE evidence
7, 8	Harmonised standards applied; EU declaration of conformity	Not produced by GONATURELLE	Provider
9	Post-market performance evaluation system	Nightly suite trends and drift alerts as monitoring inputs	Shared

FDA SaMD validation summaries aligned to GMLP.

The Good Machine Learning Practice guiding principles published by FDA, Health Canada and the UK MHRA in October 2021 describe what reviewers expect to see. The export maps your runs to the principles evaluation can evidence.

Dr. Meera Iyer and Nandini Joshi review every template.

P3Data sets are representative of the intended patient populationSlice coverage report: age bands, sex, payer type, care setting, language
P4Training data sets are independent of test setsHold-out lineage: hashes proving evaluation cases never entered prompt or fine-tune sets
P5Reference datasets are based upon best available methodsTwo credentialed labels per case, adjudication log, agreement statistics
P7Focus on the performance of the human-AI teamEscalation assertions and reviewer-override analysis
P8Testing demonstrates performance during clinically relevant conditionsStress variants: typos, conflicting labs, code-mixed input, missing fields
P10Deployed models are monitored and re-training risks managedChange history per release, inputs for a PCCP modification protocol

Adverse action, FINRA and model-risk summaries for financial services.

Bank model-risk summaries (SR 26-2, formerly SR 11-7)

Conceptual soundness, ongoing monitoring and outcomes analysis sections, populated from rubric design notes, weekly suite trends and back-tests against expert labels. SR 26-2 and OCC Bulletin 2026-13 replaced SR 11-7 in April 2026 and exclude generative and agentic AI from their scope, so many banks apply these practices to agents as internal policy. Written for second-line validators.

Regulation B adverse action evidence

Tests that each stated principal reason is specific and matches the factors that drove the decision, as Regulation B (12 CFR 1002.9) requires even when a complex model made the decision. CFPB Circulars 2022-03 and 2023-03, which applied this to algorithms, were withdrawn in May 2025; the regulation itself still applies. Checklist-style reasons fail the rubric.

FINRA suitability and communications

Rule 2111 suitability cases and communications checks, organized for supervisory review. FINRA Regulatory Notice 24-09 reminds firms that existing rules apply to generative AI.

Algorithmic fairness test summaries

Paired counterfactual cases that vary protected-class proxies while holding credit factors fixed, with disparity metrics per decision type and the cases behind each number.

SHA-256 signatures on every dataset, prompt and model reference.

Each gated run writes a manifest covering case hashes, suite versions, prompt files, the model identifier and judge configuration. The digest is signed with a per-tenant key and stored append-only, so an auditor can verify a report months later without trusting the dashboard.

datasets/*.jsonl → per-case sha256
prompts/*.md → file sha256
manifest.json → signed digest

verify.shShell

# Export the evidence bundle for a run
gonaturelle export run_7f3c2a --format annex-iv --out evidence/

# Recompute the manifest digest locally
sha256sum evidence/manifest.json
# 3b7e0c...  evidence/manifest.json

# Verify the signature with the tenant public key
gonaturelle verify evidence/manifest.json \
  --signature evidence/manifest.sig \
  --public-key gonaturelle-tenant.pub
# OK  run_7f3c2a  signed 2026-11-04T02:14:09Z  9,420 cases

GONATURELLE is an automated testing and software evaluation platform. GONATURELLE test suites and compliance reports do not constitute legal certification, regulatory approval or medical device clearance. Customers remain responsible for model validation decisions and regulatory filings. Nothing on this page is legal, regulatory or medical advice.

Frequently asked questions

Does an GONATURELLE report make our system compliant?

No. Reports document the testing you performed and the evidence it produced. GONATURELLE does not issue government certifications, legal compliance certificates or FDA clearance, and your organization remains accountable for conformity assessment, deployment decisions and regulatory filings. Have qualified counsel and regulatory staff review every export.

When do EU AI Act high-risk obligations apply?

Regulation (EU) 2024/1689 originally set 2 August 2026 for Annex III high-risk obligations. Under the Digital Omnibus amendments that entered into force in July 2026, Annex III obligations apply from 2 December 2027 and obligations for high-risk AI in products covered by Annex I from 2 August 2028. That is the position as of November 2026; confirm the current consolidated text before relying on a date.

What exactly is signed in a run manifest?

The manifest lists the agent commit and model version, every suite version and case hash, judge configuration, assertion results and aggregate metrics. GONATURELLE computes a SHA-256 digest over the canonical JSON and signs it with a per-tenant key. Anyone with the public key can recompute the digest and verify that nothing changed after the run.

Can exports be customized to our internal templates?

Yes. Enterprise plans include custom export templates mapped to your model inventory, validation report format and committee packs. Every template still pulls only from signed manifests, so the numbers in the document trace back to a verifiable run.

Hand your auditors evidence they can verify themselves.

Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.

Start free trial Book a technical demo