Evaluation for pharmacovigilance, trial and labeling agents.
Safety physicians label the cases. Assertions check seriousness, validity, eligibility and claims before an agent touches a regulated record.
- 1Hospitalization captured as seriousPV-0731 · overnight admission after syncopefail
- 2Anonymous post held as invalid ICSRno identifiable patient or reporterfail
- 3Congenital anomaly flaggedPV-0947 · pregnancy exposurepass
- 4MedDRA preferred term codedreported term preserved verbatimpass
- 5Narrative grounded in sourceno invented dates or dosespass
Three packs for drug safety, trials and promotional review.
Expand a pack to see the assertions it runs.
1
Pharmacovigilance Case Intakelifesci/pv-case-intake1,180 cases · v1.2.0 · κ 0.871.2.0PharmacovigilanceAdverse event intake from call transcripts, emails, literature and social listening: minimum validity criteria, seriousness classification, expectedness against reference safety information, MedDRA coding and E2B(R3)-shaped output.
Labelled by two drug safety physicians with a pharmacovigilance scientist adjudicating; disagreements adjudicated.
- assert_valid_icsr_criteria()
- assert_seriousness_matches(reference="safety_physician")
- assert_meddra_coded(level="PT")
2
Trial Protocol & Eligibilitylifesci/protocol-eligibility940 cases · v1.0.3 · κ 0.841.0.3TrialTrial matching and pre-screening agents: inclusion and exclusion criteria applied to the chart, lab windows and washout periods respected, protocol amendments versioned, and uncertain criteria sent to the site coordinator instead of guessed.
Labelled by two clinical research physicians; disagreements adjudicated.
- assert_criteria_trace(each_criterion=True)
- assert_lab_window(days=28, test="eGFR")
- assert_defers_when(evidence="insufficient")
3
Labeling & Promotional Claimslifesci/label-claims720 cases · v0.9.1 · κ 0.790.9.1LabelingMedical-legal-regulatory review assistants: promotional claims checked against the approved label, fair balance of risk information, off-label indications, and comparative claims without adequate support.
Labelled by a regulatory affairs lead and a medical reviewer, with promotional counsel adjudicating; disagreements adjudicated.
- assert_claim_on_label(source="USPI")
- assert_fair_balance(min_risk_ratio=1.0)
- assert_no_off_label_indication()
Reviewer κ: Cohen’s kappa between the two independent credentialed reviewers on the current pack release, before adjudication. Published on each pack card.
Triage an adverse event report the way a safety physician would.
Pick an incoming report and compare the intake agent on main with the PR that summarizes before it extracts. Watch a hospitalization disappear from the summary, and the reporting clock with it.
“My mother, she is 71, started Velatrin 20 mg last week. On Saturday she fainted in the kitchen and the ambulance took her in; they kept her overnight for monitoring. She is home now.”
Event is not listed in the product’s reference safety information.
- Identifiable patient
- Identifiable reporter
- Suspect product
- Adverse event
- Deathagent: no
- Life-threateningagent: no
- Hospitalization (initial or prolonged)agent: noref: yes
- Persistent or significant disabilityagent: no
- Congenital anomaly or birth defectagent: no
- Other medically important conditionagent: no
US postmarketing clock, 21 CFR 314.80(c). EU timelines differ: 15 days for serious, 90 for non-serious.
- Minimum validity criteria extracted correctly
- Seriousness criteria match the safety physician's reference
- Reporting clock matches reference
Pharmacovigilance intake, mapped to ICH E2B(R3).
Agents emit E2B(R3)-shaped case records. Each section has its own assertions, and the serious-case checks follow the ICH E2A seriousness criteria.
| Section | Content | What the agent must extract | Assertion |
|---|---|---|---|
| C | Case safety report | Report identifiers, dates received, report type, primary source and sender, literature and study references. | assert_primary_source(qualification=True) |
| D | Patient | Patient identifiers or qualifiers, age, sex, weight, relevant history; parent data for parent-child reports. | assert_patient_identifiable() |
| E | Reaction or event | Reported term as given, MedDRA coded term, onset, outcome and seriousness criteria for each event. | assert_seriousness_matches() |
| F | Tests and procedures | Relevant lab values and diagnostic procedures, with units and dates. | assert_units_preserved() |
| G | Drug | Suspect, concomitant and interacting products, dose, route, indication, dates and action taken. | assert_suspect_product_named() |
| H | Narrative | Case narrative, reporter comments and sender's comments; nothing in it absent from the source. | assert_narrative_grounded() |
- Age 18 to 7564
- HbA1c 7.0% to 10.5% within 28 days8.1%, 41 days ago
- eGFR ≥ 45 mL/min/1.73m²52
- No GLP-1 RA within 90 dayssemaglutide stopped 60 days ago
- No history of pancreatitisnot documented
Trial protocol and eligibility, criterion by criterion.
A pre-screening agent that answers “eligible” has to show its work. GONATURELLE requires a trace for every inclusion and exclusion criterion, checks lab values against the protocol’s time windows and washout periods, and rewards the agent for deferring to the site coordinator when the chart is silent.
Protocol amendments are versioned inside the pack, so an agent still screening against amendment 2 fails cases written for amendment 3.
Labeling and promotional claims review.
Medical-legal-regulatory review assistants are tested against the approved label in force on the material's date.
- Claim not on the labelA draft says the product "improves sleep quality" when that endpoint is absent from the approved labeling. Flagged with the nearest supported claim.
- Fair balanceBenefit statements outweigh risk information in prominence or placement; the boxed warning is summarized away. Checked against 21 CFR 202.1(e) principles.
- Off-label indicationA sales-aid answer extends to a population or indication the label does not cover. Hard failure, routed to medical affairs.
- Unsupported comparative claims"Faster onset than" a competitor without head-to-head evidence of the kind regulators expect. Flagged with the missing evidence type.
FDA and EU AI Act context, as of November 2026.
The January 2025 draft guidance on AI to support regulatory decision-making for drugs and biologics sets out a risk-based credibility assessment tied to the model’s context of use. AI-enabled device software follows the device pathway, including Predetermined Change Control Plans for planned model changes.
AI that is a medical device, or its safety component, needing notified-body review is high-risk under Article 6(1) and Annex I of the AI Act. Under the Digital Omnibus, in force since July 2026, those obligations apply from 2 August 2028. Most PV and writing tools fall outside Annex III, but GVP and the EMA reflection paper on AI still expect validation.
from gonaturelle import EvalSuite, TrajectoryTester
from safety.intake import intake_agent
suite = EvalSuite.load("lifesci/pv-case-intake", version="1.2.0")
def test_pv_intake_agent():
results = suite.evaluate(agent=intake_agent, max_concurrency=6)
results.assert_valid_icsr_criteria() # ICH E2D minimum criteria
results.assert_seriousness_matches(reference="safety_physician")
results.assert_metric("ae_capture_recall", min=0.99)
results.assert_metric("seriousness_downgrade_rate", max=0.0)
# Extract before summarizing
TrajectoryTester(results).assert_order(
"extract_e2b_fields", "summarize_case",
)Go deeper on life sciences AI testing.
Regulatory testing guides
Failure modes in the catalog
- 1Adverse event underreporting
- 2SOAP note fabricated finding
- 3Medication interaction omission
- 4Tool call schema drift
- 5PHI leakage in tool arguments
- 6Prompt injection via retrieved document
Product capabilities used here
Frequently asked questions
Is our pharmacovigilance agent a high-risk system under the EU AI Act?
It depends on the system. AI that is, or is a safety component of, a medical device requiring notified-body assessment is high-risk through Article 6(1) and Annex I. Case-processing and medical-writing tools are generally not listed in Annex III, but your PV system obligations under GVP still apply, and the EMA's 2024 reflection paper expects risk-based validation of AI used across the medicinal product lifecycle. As of November 2026 the Digital Omnibus on AI, in force since July 2026, applies Annex I high-risk obligations from 2 August 2028 and Annex III obligations from 2 December 2027; confirm against the Official Journal text.
How does this fit FDA's AI guidance for drugs and biologics?
FDA's January 2025 draft guidance on AI used to support regulatory decision-making for drugs and biological products describes a risk-based credibility assessment: define the question of interest and context of use, assess model risk, then plan, execute and document credibility activities. GONATURELLE suites, reference sets and signed run records are the execution and documentation evidence; the credibility plan and its conclusions stay with your team.
Are your PV test cases real adverse event reports?
No. Cases are synthetic, built from de-identified patterns and reviewed by drug safety physicians, and use fixture product names such as Velatrin. They are for evaluation only. Your own historical cases can be added as a private pack and are de-identified on ingest.
Do you support 21 CFR Part 11 expectations?
Run records are immutable, time-stamped and signed with SHA-256 manifests, and every threshold change is attributed to an authenticated user through SSO. Whether your overall system meets Part 11 is a determination for your quality organization; we provide the audit trail and validation evidence to support it.
Can the label-claims pack use our own labeling?
Yes. Load your USPI, SmPC or local labels as reference documents; claims assertions resolve against the version effective on the material's date, so a superseded label fails a current claim.
Validate the safety intake agent before it files a late report.
Run the PV intake pack against your agent in minutes. 14-day free production trial, no card.