Catalog of AI agent failure modes in regulated industries
How agents fail in clinical, lending, legal and claims workflows, each entry written so you can turn it into a test.
- Omission28
- Jurisdiction & currency24
- Hallucination21
- Disclosure & reasoning17
- Tool-call schema14
- Sequence violation12
- Privacy leakage12
- Loop & runaway10
- Bias & fairness9
- Adversarial input8
- of multi-step agent failures in the catalog were tool-argument schema or parameter-boundary violations, not reasoning breakdowns.
- 32.8%
- of multi-step agent failures in the catalog were tool-argument schema or parameter-boundary violations, not reasoning breakdowns.
- GONATURELLE Failure Modes Catalog analysis, 2026
- Hallucination rate measured in leading AI legal research tools on a preregistered query set.
- 17% to 33%
- Hallucination rate measured in leading AI legal research tools on a preregistered query set.
- Magesh et al., Stanford RegLab and HAI, "Hallucination-Free?" (2024)
- Sanctions imposed on attorneys who filed a brief citing cases fabricated by a chatbot.
- $5,000
- Sanctions imposed on attorneys who filed a brief citing cases fabricated by a chatbot.
- Mata v. Avianca, Inc., No. 22-cv-1461 (S.D.N.Y. June 22, 2023)
Search the catalog
Detailed entries open a full write-up with a worked trajectory, root causes, detection strategy and assertion code. Compact entries expand in place.
Most of these never show up in a final-answer eval.
An identity check skipped at step two, a dose passed as a string, a citation no retrieval step returned: the final message can still read well. The catalog is organised around where the failure happens in the trajectory so you can pick the right assertion type, deterministic where possible and judged only where the rubric needs a domain expert.
How trajectory assertions workFrequently asked questions
What counts as a failure mode in this catalog?
A repeatable way an AI agent produces a harmful or non-compliant result in a regulated workflow, described tightly enough to write a test for it. Each entry names the vertical, a category (hallucination, omission, tool-call schema, sequence violation and so on) and a severity based on the worst realistic consequence, not on how often it happens.
How is severity assigned?
Critical means the failure can directly harm a patient or consumer, move money, or breach a statutory duty without a human noticing. High means a material compliance or financial error that a downstream human review would usually catch. Medium means degraded quality, cost or latency that becomes a compliance problem only at scale.
Where do the entries come from?
From failures GONATURELLE's clinical, legal and regulatory reviewers documented while building golden datasets, from public court and enforcement records such as Mata v. Avianca and Moffatt v. Air Canada, and from patterns in published research. Customer incidents are only included when fully de-identified and generalized.
Can I search the catalog from a link?
Yes. Add ?q= to the URL, for example https://gonaturelle.si/resources/failure-modes-catalog?q=citation, and the catalog opens filtered to that term. The search box keeps the URL in sync so you can share a filtered view.
The Regulated Agent
A monthly deep dive on AI evaluations, safety benchmarks and regulatory enforcement. One email, first Tuesday of the month.
Stop testing regulated agents on generic prompts.
Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.