State claim deadline miscalculation
A claims workflow agent computes a regulatory deadline with the wrong state's rule, the wrong trigger event, or calendar days where the statute counts business days. The diary entry looks authoritative, so adjusters plan around a date that is already late.
A multistate property carrier uses an agent to set diary dates on new claims. Tools: get_claim(claim_id) returning loss state and document receipt dates, get_state_rules(state, rule_type), business_days_add(start, n, state), and set_diary(claim_id, task, due_date). The fictional claim is a Texas homeowners water damage claim. The last document the carrier requested was received on Monday, 2 November 2026.
Set the diary date for the accept-or-reject decision on this claim.
How the trajectory goes wrong
Under the Texas Prompt Payment of Claims Act (Texas Insurance Code section 542.056), an insurer generally must accept or reject a claim in writing within 15 business days after receiving all items, statements and forms it requested, with up to 45 additional days if it notifies the claimant of the reasons it needs more time. Fifteen business days from 2 November 2026 falls around 23 November, depending on how the carrier's calendar treats intervening holidays, nearly three weeks before the diary date. Missed prompt payment deadlines can expose the carrier to statutory interest and attorney's fees under section 542.060. Two errors compound: the wrong state, then calendar arithmetic instead of business days.
Root causes
- State parameters taken from conversation memory instead of the current claim record.
- Rules returned as plain numbers without units (calendar versus business days) or trigger events.
- Date arithmetic done by the model instead of a calendar-aware tool.
- Golden sets concentrated in one or two states.
- 1Step 1: get_claim returns loss_state='TX' and all_items_received='2026-11-02'.
- 2Step 2: get_state_rules is called with state='CA' because the agent copied the state from a previous claim in its working memory.
- 3Step 3: The rule returned is California's 40-day period to accept or deny after receiving proof of claim.
- 4Step 4: The agent adds 40 calendar days and sets the diary for 12 December 2026.
- 5Step 5: set_diary succeeds, and the adjuster's queue shows the claim as not urgent.
How GONATURELLE detects it
Hallucinated argument check
no_hallucinated_args requires that the state passed to get_state_rules equal the loss_state returned by get_claim in the same trajectory.
Deadline recomputation
GONATURELLE recomputes each due date from the trigger date with a business-day calendar per state and compares it with the diary. Any late diary fails; early diaries are reported but allowed.
State coverage slices
Results are sliced by state and rule type so a regression in one state's logic is visible even when overall accuracy stays high.
from gonaturelle import EvalSuite, TrajectoryTester, assertions as A
suite = EvalSuite.load('insurance/claims-deadlines-multistate-v2')
results = suite.evaluate(agent=diary_agent, max_concurrency=10)
results.assert_tool_schema_valid()
TrajectoryTester(results).assert_order('get_claim', 'get_state_rules', 'set_diary')
results.assert_all([
A.no_hallucinated_args('get_state_rules', fields=['state'], source=('get_claim', 'output.loss_state')),
A.calls('business_days_add', when="rule.unit == 'business_days'"),
A.param_within('set_diary', 'due_date', max='case.labels.statutory_due_date'),
])
results.slice_by('loss_state').assert_metric('deadline_accuracy', min=0.98, min_support=20)Mitigations
- 01Pass loss state from the claim record directly into rule lookups; never from conversation memory.
- 02Return rules as structured objects with trigger event, count, unit and extension conditions.
- 03Do all date arithmetic in a calendar service that knows each state's business days.
- 04Build deadline cases for every state you write in, including catastrophe extensions.
Regulations this maps to
- NAIC AI BulletinNAIC Model Bulletin on the Use of Artificial Intelligence Systems by Insurers (adopted 4 December 2023), Sections 3 and 4
- Colorado AI ActSB 24-205 (2024), amended by SB25B-004 (2025); repealed and reenacted by SB 26-189 (2026)
- NIST AI RMFNIST AI 100-1 (January 2023), MEASURE function; NIST AI 600-1 (July 2024)
Questions about this failure
Do the deadlines on this page apply to every claim?
No. Texas and California rules have conditions, extensions and exceptions, including catastrophe extensions, and other states differ. The example shows the testing method. Confirm deadlines with counsel; this is not legal advice.
Why allow early diary dates?
An early diary is conservative and harmless for compliance. The assertion fails only diaries later than the statutory date, while reporting early ones so adjusters are not flooded with false urgency.
Scenarios use fictional people and synthetic records for evaluation only. This entry is not legal or financial advice.
Explore the 2026 Vertical AI Reliability Index: failure rates across 12 regulated domainsSchema adherence, trajectory completion, domain factuality and adversarial robustness for 48 models. Free to cite.View the indexStop testing regulated agents on generic prompts.
Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.