Skip to content
GONATURELLE
LlamaIndex logoAgent framework

GONATURELLE for LlamaIndex

Evaluate LlamaIndex FunctionAgent and AgentWorkflow runs with every tool call, hand-off and retrieved node captured from the workflow event stream.

Start 14-day trialSetup steps
AgentWorkflow event stream
tests/test_coverage_workflow.py
from llama_index.core.agent.workflow import AgentWorkflow, FunctionAgent
from llama_index.llms.openai import OpenAI
from gonaturelle import EvalSuite, TrajectoryTester, assertions as A
from gonaturelle.integrations.llamaindex import LlamaIndexAdapter

from app.tools import policy_search, determine_coverage

coverage_agent = FunctionAgent(
    name='coverage',
    description='Interprets coverage and exclusions for a claim.',
    tools=[policy_search, determine_coverage],
    llm=OpenAI(model='gpt-4.1'),
    system_prompt='Quote only policy language returned by policy_search.',
)
workflow = AgentWorkflow(agents=[coverage_agent], root_agent='coverage')
agent = LlamaIndexAdapter(workflow, agent='coverage-workflow')

def test_coverage():
    results = EvalSuite.load('insurance/coverage-exclusions').evaluate(agent=agent, max_concurrency=6)
    t = TrajectoryTester(results)
    t.assert_order('policy_search', 'determine_coverage')
    results.check(A.cites_only(source='policy_search'))

LlamaIndex agents are built on workflows: a FunctionAgent or ReActAgent runs inside an AgentWorkflow, and the handler returned by run() streams typed events as the agent works. AgentInput, AgentOutput, ToolCall and ToolCallResult events describe exactly what happened, in order, including which agent in a multi-agent workflow made each call and when control was handed off.

The GONATURELLE adapter reads that stream. For each case it calls workflow.run with the case input, iterates handler.stream_events(), and converts tool calls and results into trajectory steps with their tool_kwargs and outputs. When a tool wraps a query engine or retriever, the source nodes and their metadata are kept as evidence, which lets grounding assertions check that a coverage determination or a cited statute actually appears in the retrieved text rather than in the model's memory.

This matters most for retrieval-heavy agents in law and insurance, where the answer is only as good as the documents behind it. A citation-verification agent can reach the right holding from the wrong case; an exclusion-interpretation agent can quote an endorsement that was never retrieved. Trajectory assertions on retrieval and tool steps catch both. Tested against llama-index-core 0.12 and later.

What you can test with LlamaIndex.

Multi-agent hand-off checks

In an AgentWorkflow with several agents, every step carries the current agent name. Assertions can require that only the coverage agent calls determine_coverage, or that the intake agent hands off before any payment tool is touched.

Grounding against retrieved nodes

Source nodes returned by query engine tools are stored with their ids and text. A.cites_only() fails an answer that quotes policy language or case law absent from those nodes.

Tool kwargs validation

FunctionTool schemas generated from Python signatures are used for schema checks, and suite-level bounds catch values such as a deductible outside the declarations page range.

Workflow state and context

Values the agents write to the workflow Context are captured at each step, so state assertions work for shared memory such as a claim status or a verified-identity flag.

Set up in 5 steps.

Most teams have a first suite running against LlamaIndex in under an hour, then add the CI gate.

Get an API key
  1. 01

    Install gonaturelle with llama-index-core and the LLM integration you use, such as llama-index-llms-openai.

  2. 02

    Export GONATURELLE_API_KEY locally and in your CI secrets store.

  3. 03

    Build your FunctionAgent or AgentWorkflow as usual and wrap it with LlamaIndexAdapter, which handles run(), event streaming and the final response.

  4. 04

    Load a pack such as legal/citation-verification or insurance/coverage-exclusions and call evaluate with the adapter.

  5. 05

    Add retrieval-grounding and tool-order assertions, then enable the fast-path gate in CI.

LlamaIndex integration FAQ

Does this work with the older QueryPipeline or AgentRunner APIs?

The adapter targets the workflow-based agents (FunctionAgent, ReActAgent, AgentWorkflow) that LlamaIndex now recommends. Older agent classes can be evaluated through the generic trace decorator, with tool calls recorded by wrapping each tool function.

Can we evaluate a plain RAG query engine without an agent?

Yes. Pass a function that calls query_engine.query and returns the response; GONATURELLE records the retrieval as a single step with its source nodes, and the same grounding judges and citation assertions apply.

Is retrieved document text stored?

Retrieved text is stored with the run so graders and reviewers can see the evidence, under the same redaction rules and retention settings as other case data. On Enterprise VPC deployments it never leaves your cloud account.

Related integrations

All integrations

Stop testing regulated agents on generic prompts.

Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.

Start free trialBook a technical demo