EU AI Act Annex IV for engineers

Why engineers should read Annex IV themselves

Regulation (EU) 2024/1689, the EU AI Act, requires providers of high-risk AI systems to draw up technical documentation before the system is placed on the market or put into service (Article 11), and Annex IV says what that documentation must contain. Legal and compliance teams own the obligation, but most of the content is engineering material: how the system is built, what data it was trained and tested on, how it was validated, what metrics were used and what the test logs say. If engineers first meet Annex IV a month before a conformity assessment, the team ends up reconstructing evidence from memory and old dashboards.

This note is a practitioner's reading of the annex for the people who will generate that evidence. It is not legal advice, and it does not replace your own counsel's reading of the regulation, the harmonised standards as they are published, or guidance from the AI Office.

Where the timeline stands, as of November 2026

The Act entered into force on 1 August 2024. Prohibited practices under Article 5 have applied since 2 February 2025, and the general-purpose AI model obligations since 2 August 2025. The Digital Omnibus on AI, in force since late July 2026, moved the application date for high-risk systems listed in Annex III (such as creditworthiness assessment and certain insurance pricing uses) to 2 December 2027, and for AI systems that are safety components of products covered by Annex I legislation, which includes medical devices, to 2 August 2028. These dates have moved once already; confirm them against the Official Journal text before you plan a release around them.

The extra time is useful for one reason: technical documentation describes the system's history, not just its current state. Annex IV asks for test logs, a description of changes made through the lifecycle and the validation procedures used. Teams that start generating that record now will have a year of real evidence when the obligations apply. Teams that wait will have a snapshot. The penalty ceiling for breaching high-risk obligations is up to EUR 15 million or 3% of worldwide annual turnover, whichever is higher (Article 99); the EUR 35 million or 7% tier applies to prohibited practices, not to high-risk documentation failures.

The nine points of Annex IV, mapped to artifacts

Annex IV has nine numbered points. Below is how each one maps to something an engineering team can produce and version. Points 2 and 4 carry most of the testing evidence.

• 1. General description: intended purpose, provider, versions, how the system interacts with hardware or other software, and the forms in which it is placed on the market. Artifact: a system card per release, generated from config.
• 2. Detailed description of the elements and the development process: design choices, architecture, data requirements and provenance, human oversight measures, predetermined changes, and under point 2(g) the validation and testing procedures, data, metrics and test logs and reports, dated and signed by the responsible persons. Artifact: dataset versions, eval suite definitions and signed run manifests.
• 3. Monitoring, functioning and control: capabilities and limitations, expected accuracy for specific groups, foreseeable unintended outcomes. Artifact: per-slice results and documented known failure modes.
• 4. Appropriateness of the performance metrics. Artifact: a written rationale for each metric and threshold, including why judge scores are trusted (agreement with experts).
• 5. The risk management system under Article 9. Artifact: the risk register, linked to the test cases that probe each risk.
• 6. Relevant changes made through the lifecycle. Artifact: a release change log with eval deltas per change.
• 7. Harmonised standards or other specifications applied.
• 8. A copy of the EU declaration of conformity.
• 9. The post-market monitoring plan under Article 72. Artifact: the production sampling and re-evaluation schedule.

Point 2(g) is a CI requirement in disguise

Read point 2(g) slowly and it describes a test pipeline: the validation and testing procedures used, information about the validation and testing data and their main characteristics, the metrics used to measure accuracy and resilience and potentially discriminatory impacts, and test logs and all test reports dated and signed by the responsible persons, including for predetermined changes. A notebook someone ran last quarter does not meet that description well. A pipeline that runs a versioned golden set on every release candidate, records the exact model, prompt and dataset versions, and emits a signed manifest does.

The practical requirements that follow are simple to state. Every run must be reproducible from its manifest: dataset version and hash, agent commit, model identifier, judge versions and calibration file, thresholds. Every manifest must be tamper-evident. And each run must be attributable to a named responsible person who approved the release, which in practice means the approval step in your release process signs or countersigns the manifest.

from gonaturelle import EvalSuite

from agents.underwriting import agent

suite = EvalSuite.load('insurance/life-health-pricing-v1', version='1.4.0')
results = suite.evaluate(agent=agent, max_concurrency=10, mode='full')

results.assert_tool_schema_valid()
results.assert_metric('judge.rationale_faithfulness', min=0.92)
for slice_name in ['age_band', 'sex', 'member_state']:
    results.slice_by(slice_name).assert_metric('accuracy', max_gap=0.03)

# Point 2(g): dated, signed test log tied to exact versions
results.export_manifest('annex_iv/2g_validation_run.json', sign=True, approver='head-of-model-risk')
results.export_report('annex_iv/2g_test_report.pdf', sections=['2g', '3', '4', '6'])

What signing and retention actually require

"Dated and signed" is not satisfied by a PDF with a typed name in the footer. At minimum, compute a cryptographic hash over the manifest and the raw results it references, record who approved it and when, and store the signature somewhere the engineering team cannot silently rewrite. SHA-256 hashes plus a signing key held by the release approver, with the signed manifests written to write-once storage, is a reasonable baseline. If you already sign build artifacts, reuse that infrastructure rather than inventing a second one.

Retention is the part teams underestimate. Article 18 requires the provider to keep the technical documentation at the disposal of national authorities for 10 years after the system is placed on the market or put into service. Article 19 separately requires keeping automatically generated logs under the provider's control for at least six months. Ten years outlives most eval tools, most model versions and many companies' storage choices, so keep manifests and reports in an open format (JSON plus PDF), keep the referenced datasets or their hashes, and make sure an export does not depend on your eval vendor still existing.

What testing evidence cannot do for you

Two boundaries are worth stating plainly. First, an evaluation platform produces evidence; it does not certify conformity. Conformity assessment, the declaration of conformity and CE marking are the provider's responsibility, and for Annex I products they run through the sectoral procedure with a notified body. GONATURELLE generates the Annex IV testing sections and signed manifests; it does not issue certificates, and no vendor can make you compliant on your behalf. Second, documentation of a weak test is still documentation of a weak test. Reviewers will ask why your metrics are appropriate (point 4), and "our LLM judge said so" is not an answer unless you can show its agreement with qualified human reviewers.

If you are 12 to 24 months from your applicable date, the most valuable thing to do this quarter is to make every release produce a signed manifest automatically, even before the content is complete. History cannot be backfilled. Book a technical demo if you want to walk through an Annex IV export generated from your own runs with our regulatory science team.