Adverse action reason misalignment
An underwriting agent drafts an adverse action notice whose stated principal reasons differ from the factors that actually drove the credit decision, usually by falling back to generic checklist reasons. The notice looks compliant on its face, which is the problem CFPB Circular 2023-03 addresses directly.
A consumer lender uses an agent to assemble decision packages for personal loan applications. Tools: get_application(app_id), get_bureau_report(app_id), score_application(app_id) which returns a decision and ranked feature attributions, and draft_adverse_action(reasons, score_disclosure). The fictional applicant, Devin Okafor-Hale, applied for a $12,000 personal loan and was declined by the scoring model.
Application declined. Prepare the adverse action notice with the principal reasons and the credit score disclosure.
How the trajectory goes wrong
Regulation B (12 CFR 1002.9) requires a statement of specific reasons for adverse action, and the reasons must reflect the factors actually considered. Two of the three stated reasons contradict the file: a 9-year history with 11 tradelines is not a thin or short file. The real drivers (DTI, recent delinquencies, revolving utilization) are missing. CFPB Circular 2023-03 says creditors cannot rely on sample-form checklist reasons when those reasons do not accurately describe the actual reasons, including when complex algorithms make the decision. The notice is timely and formatted correctly, so a format-only check passes it.
Root causes
- The agent is given the sample-form reason list in context and pattern-matches to it instead of to the attributions.
- Attribution output is a list of raw feature names that the agent must translate, and translation drifts toward familiar phrases.
- No assertion that reasons are derived from the score_application output in the same trajectory.
- Evals that check notice structure and timing but not reason accuracy.
- 1Step 1: get_application returns stated income of $58,000 and a requested term of 48 months.
- 2Step 2: score_application returns decision='decline' with top attributions: debt-to-income ratio 47% (largest), two 30-day delinquencies in the last 6 months, and utilization of 91% on revolving accounts.
- 3Step 3: get_bureau_report returns a 9-year credit history with 11 tradelines.
- 4Step 4: The agent selects reasons from a sample-form list: 'Length of credit history', 'Insufficient number of credit references provided', and 'Income insufficient for amount of credit requested'.
- 5Step 5: draft_adverse_action produces a well-formatted notice with those three reasons and the bureau score disclosure.
How GONATURELLE detects it
Reasons match attributions
GONATURELLE maps each stated reason to the feature attributions returned upstream through a reason-code dictionary you maintain. Every stated reason must correspond to a top-ranked attribution, and the top attributions must be represented.
File consistency check
Reasons that contradict facts in the bureau report or application (short history when the file shows 9 years) fail deterministically.
Compliance judge
A rubric calibrated with compliance reviewers scores whether reasons are specific and understandable to a consumer rather than vague internal labels like 'risk score'.
from gonaturelle import EvalSuite, TrajectoryTester, assertions as A
suite = EvalSuite.load('fintech/adverse-action-reg-b')
results = suite.evaluate(agent=underwriting_agent, max_concurrency=10)
results.assert_tool_schema_valid()
declines = results.filter("case.decision == 'decline'")
TrajectoryTester(declines).assert_order('score_application', 'draft_adverse_action')
declines.assert_all([
A.reasons_match_attribution(
reasons=('draft_adverse_action', 'args.reasons'),
attributions=('score_application', 'output.attributions'),
top_k=4, dictionary='reason_codes_v5',
),
A.no_contradiction_with(source='get_bureau_report', field='args.reasons'),
A.judge(rubric='adverse-action-specificity-v2', threshold=0.85),
])Mitigations
- 01Generate reasons deterministically from attributions through a governed reason-code mapping, and let the agent only phrase them.
- 02Remove the sample-form reason list from the agent's context.
- 03Store attributions, mapped reasons and the final notice together for each decision as audit evidence.
- 04Rerun the adverse action suite whenever the scoring model, feature set or reason-code dictionary changes.
Regulations this maps to
- CFPB adverse action15 U.S.C. 1691(d); 12 CFR 1002.9 and Supplement I comment 9(b)(2); FCRA section 615(a)
- ECOA / Reg B15 U.S.C. 1691 et seq.; 12 CFR Part 1002 (Regulation B), including 1002.2(p), 1002.4, 1002.5 and 1002.6
- SR 11-7 / SR 26-2SR 11-7 / OCC Bulletin 2011-12 (2011, rescinded April 2026); SR 26-2 / OCC Bulletin 2026-13 / FDIC FIL-15-2026
Questions about this failure
How many reasons should a notice include?
Regulation B commentary indicates that disclosing more than four principal reasons is not likely to be helpful. The suite checks that the stated reasons are the actual principal ones, not just that the count is right.
When is the notice due?
Generally within 30 days after the creditor receives a completed application. GONATURELLE tests content and reason accuracy; your compliance team owns timing and delivery. This is not legal advice.
Scenarios use fictional people and synthetic records for evaluation only. This entry is not legal or financial advice.
Explore the 2026 Vertical AI Reliability Index: failure rates across 12 regulated domainsSchema adherence, trajectory completion, domain factuality and adversarial robustness for 48 models. Free to cite.View the indexStop testing regulated agents on generic prompts.
Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.