Insurance · Omission

FNOL required field drop

A first notice of loss agent records a claim with required or decision-driving fields missing or defaulted, usually injuries, passengers or third parties mentioned in passing. Downstream triage, reserving and benefits handling all key off fields that are wrong from the first minute.

Get the assertion Back to catalog

High severityOmissionInsurance

Scenario

An auto insurer's intake agent takes FNOL by chat and voice. Tools: get_policy(policy_id), create_fnol(payload) with a JSON schema that includes loss_datetime, location, vehicles, parties, injuries_reported, injured_parties, police_report, and narrative, then assign_queue(claim_id, queue). The fictional caller, 'Priya Halvorsen-Ade', was rear-ended at a stop light.

Input

Caller: Someone rear-ended me at the light on Elm and 5th about an hour ago. My bumper's crushed. I'm okay, just my neck's a bit stiff. My daughter was in the back in her car seat, she's fine I think, just crying. The other guy gave me his insurance card and the police came.

How the trajectory goes wrong

The caller reported neck stiffness and a child passenger who was crying after a rear-end collision. Intake guidelines generally record any reported symptom as a possible injury and capture every occupant, because soft tissue symptoms often develop over the following days and injury flags drive bodily injury and medical payments handling. Routing to a property-only track delays contact from an injury adjuster. The payload passed schema validation because the fields were present with valid values; they were simply wrong. Police attendance was mentioned but no report number was requested.

Root causes

Extraction that resolves hedged statements ('I'm okay', 'I think') to false instead of recording the symptom.
Schema validation that checks types and presence, not consistency with the narrative.
No required follow-up questions for fields with conditional triggers (police attended, minors present).
Golden FNOL transcripts that state injuries plainly, with no hedging or minimization.

trace.jsonl5 stepsgate failed

1
Step 1: get_policy confirms active coverage and the listed vehicle.
2
Step 2: The agent extracts loss location, time and other driver's insurer correctly.
3
Step 3: It reads 'I'm okay' and 'she's fine I think' and sets injuries_reported=false and injured_parties=[]. The daughter is not added as a passenger.
4
Step 4: police_report is set to null because no report number was given, with no follow-up question.
5
Step 5: create_fnol succeeds against the schema, and assign_queue routes the claim to the property-damage-only fast track.

AssertionError step 5: The caller reported neck stiffness and a child passenger who was crying after a rear-end collision. Intake guidelines generally record any reported symptom as a possible injury and capture every occupant, because soft tissue symptoms often develop over the following days and injury flags drive bodily injury and medical payments handling. Routing to a property-only track delays contact from an injury adjuster. The payload passed schema validation because the fields were present with valid values; they were simply wrong. Police attendance was mentioned but no report number was requested.

How GONATURELLE detects it

Narrative-to-field consistency

GONATURELLE compares the structured payload with adjuster-labelled facts from the transcript. Any reported symptom with injuries_reported=false, or any mentioned occupant missing from parties, fails the case.

Conditional follow-ups

Assertions require a follow-up question when a trigger appears: police attended without a report number, minors present, or a third party mentioned without contact details.

Routing outcome

The assigned queue is checked against the label; injury-indicated claims must not route to property-only tracks.

test_fnol_completeness.pyPython

from gonaturelle import EvalSuite, assertions as A

suite = EvalSuite.load('insurance/auto-fnol-intake-v3')
results = suite.evaluate(agent=fnol_agent, max_concurrency=10)
results.assert_tool_schema_valid()

results.assert_all([
    A.required_fields('create_fnol', ['loss_datetime', 'location', 'parties', 'injuries_reported']),
    A.field_matches('create_fnol', 'args.payload.injuries_reported', key='case.labels.injury_indicated'),
    A.covers_all(source=('case', 'labels.occupants'), target=('create_fnol', 'args.payload.parties[*].name')),
    A.asks_follow_up(when='case.triggers', topics=['police_report_number', 'minor_condition']),
    A.never_outputs_label('pd_fast_track', when='case.labels.injury_indicated'),
])

Mitigations

01Record symptoms verbatim and set injury indicators from any symptom, leaving severity to adjusters.
02Add semantic validation that compares payload fields with the narrative before submission.
03Script conditional follow-up questions for police, minors and third parties.
04Include minimization and hedging variants in every FNOL golden case.

Regulations this maps to

Related failure modes

Questions about this failure

The payload passed schema validation. Why is it a failure?

Schema validation proves the shape is right. FNOL failures are mostly about content: a valid false where the truth is true. That needs labelled transcripts and consistency checks.

Is the caller real?

No. The caller and the transcript are synthetic, written by former adjusters from common intake patterns for evaluation only.

Scenarios use fictional people and synthetic records for evaluation only. This entry is not legal or financial advice.

Explore the 2026 Vertical AI Reliability Index: failure rates across 12 regulated domainsSchema adherence, trajectory completion, domain factuality and adversarial robustness for 48 models. Free to cite.View the index

Stop testing regulated agents on generic prompts.

Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.

Start free trial Book a technical demo