Skip to content
GONATURELLE
AWS Bedrock logoModel provider

GONATURELLE for AWS Bedrock

Evaluate agents built on the Amazon Bedrock Converse API inside your AWS account, with toolUse and toolResult blocks captured and judges served from Bedrock in-region.

bedrock-runtime Converse
app/bedrock_agent.py
import os
import boto3
from gonaturelle.integrations.bedrock import BedrockConverseLoop

from app.tools import TOOL_SPECS, call_tool  # [{'toolSpec': {'name', 'description', 'inputSchema': {'json': {...}}}}]

client = boto3.client('bedrock-runtime', region_name='us-east-1')
MODEL_ID = os.environ['UNDERWRITING_MODEL_ID']

def run_agent(user_input: str) -> str:
    loop = BedrockConverseLoop(client, agent='underwriting')
    messages = [{'role': 'user', 'content': [{'text': user_input}]}]
    for _ in range(8):
        response = loop.converse(modelId=MODEL_ID, messages=messages, toolConfig={'tools': TOOL_SPECS})
        message = response['output']['message']
        messages.append(message)
        if response['stopReason'] != 'tool_use':
            return ''.join(c.get('text', '') for c in message['content'])
        results = []
        for content in message['content']:
            if 'toolUse' in content:
                use = content['toolUse']
                output = call_tool(use['name'], use['input'])
                results.append(loop.tool_result(use['toolUseId'], {'json': output}))
        messages.append({'role': 'user', 'content': results})
    raise RuntimeError('tool loop exceeded 8 turns')

The Bedrock Converse API gives one request and response shape across the models Bedrock hosts. Tools are declared in toolConfig as toolSpec entries with an inputSchema, the model signals a tool request with stopReason set to tool_use and a toolUse content block carrying a toolUseId, name and input, and your code replies with a toolResult block. For regulated teams the attraction is that inference stays inside their AWS account boundary, governed by IAM and logged by CloudTrail.

GONATURELLE follows that boundary. In an Enterprise single-tenant VPC deployment on AWS, sandboxes, the evaluation workers and storage run in your account, the agent under test calls Bedrock through your VPC endpoints, and judges that need a frontier model can call Bedrock models in the same region. Nothing about the evaluation requires data to leave the account. The BedrockConverseLoop wrapper records every toolUse block and the toolResult you return, including error status, as ordered trajectory steps.

Because the Converse API normalizes tool calling across model families, it is also a convenient way to compare candidates for the same agent. Run one suite against two model IDs with identical prompts and tools, and the trajectory diff shows where the models choose different tools or arguments, which tells you more than a single aggregate score before you switch a production agent.

What you can test with AWS Bedrock.

In-account evaluation

With VPC deployment, agent calls, judge calls and run storage all stay in your AWS account and region, using IAM roles you control.

toolUse and toolResult capture

toolUseId pairing, inputs and toolResult status are recorded per step, so error handling and argument correctness are assertable.

Model candidate comparison

Run the same pack against several Bedrock model IDs and compare trajectories case by case before choosing a model for a regulated workflow.

Guardrails as a recorded step

If your agent applies an Amazon Bedrock Guardrail through guardrailConfig, intervention results are recorded, so suites can check that the guardrail fires on the cases it should.

Set up in 5 steps.

Most teams have a first suite running against AWS Bedrock in under an hour, then add the CI gate.

Get an API key
  1. 01

    Install gonaturelle and boto3 in the agent's environment and confirm the IAM role has bedrock:InvokeModel for the model IDs you use.

  2. 02

    Export GONATURELLE_API_KEY, or for VPC deployments point the SDK at your in-account GONATURELLE endpoint.

  3. 03

    Wrap the bedrock-runtime client with BedrockConverseLoop and call loop.converse where you called client.converse.

  4. 04

    Return tool outputs through loop.tool_result so each toolResult is recorded and paired with its toolUseId.

  5. 05

    Evaluate on a suite, add assertions, and run the CI gate on runners that can reach Bedrock.

AWS Bedrock integration FAQ

Do you support Bedrock Agents (the managed agent service)?

Yes, through invoke_agent with tracing enabled: the trace events Bedrock returns are converted into trajectory steps. Agents built directly on the Converse API, shown here, give more precise step data and are what we recommend for gated suites.

Can judges run entirely on Bedrock?

Yes. In AWS VPC deployments you can restrict every model-based judge to Bedrock model IDs in your region, so no evaluation traffic goes to any other provider.

Which regions are supported?

Any AWS region where your chosen Bedrock models are available. The multi-tenant GONATURELLE cloud runs in the US and India; single-tenant VPC deployments run in the region you choose.

Stop testing regulated agents on generic prompts.

Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.