Skip to content
GONATURELLE
Semantic Kernel logoAgent framework

GONATURELLE for Semantic Kernel

Record every kernel function call, its arguments and its result through Semantic Kernel's function invocation filters, then assert on them like any other trajectory.

Start 14-day trialSetup steps
FUNCTION_INVOCATION filter
tests/test_servicing_kernel.py
from semantic_kernel import Kernel
from semantic_kernel.agents import ChatCompletionAgent
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from gonaturelle import EvalSuite, TrajectoryTester
from gonaturelle.integrations.semantic_kernel import KernelCapture

from app.plugins import IdentityPlugin, AccountsPlugin

kernel = Kernel()
kernel.add_service(OpenAIChatCompletion(ai_model_id='gpt-4.1'))
kernel.add_plugin(IdentityPlugin(), plugin_name='identity')
kernel.add_plugin(AccountsPlugin(), plugin_name='accounts')
capture = KernelCapture(agent='servicing-sk')
capture.install(kernel)  # registers a FUNCTION_INVOCATION filter

agent = ChatCompletionAgent(kernel=kernel, name='Servicing', instructions='Verify identity before any account lookup.')

async def run_case(case):
    response = await agent.get_response(messages=case.input)
    return str(response.content)

def test_servicing():
    results = EvalSuite.load('fintech/servicing-balances').evaluate(agent=run_case, max_concurrency=8)
    t = TrajectoryTester(results)
    t.assert_order('identity.verify', 'accounts.get_balance')
    t.assert_never_calls('accounts.transfer')

Semantic Kernel exposes AI capabilities as plugins: classes whose methods are marked with the kernel_function decorator, registered on a Kernel and called by the model through automatic function calling. ChatCompletionAgent and the other agent types build on the same kernel, which makes the kernel the single place every tool call passes through.

Semantic Kernel also gives you a clean, supported interception point for exactly this purpose: filters. GONATURELLE registers a FUNCTION_INVOCATION filter on the kernel used during evaluation. The filter receives a FunctionInvocationContext with the plugin name, function name and KernelArguments, calls next(context) to let the function run, then records the FunctionResult. Because the filter wraps execution, GONATURELLE sees the arguments the model chose and the value the function actually returned, in order, with no change to the plugins themselves.

Enterprises in banking and insurance often standardize on Semantic Kernel because it sits naturally in .NET and Azure estates. The Python filter shown here has a direct .NET equivalent through IFunctionInvocationFilter, and the GONATURELLE TypeScript and Python SDKs accept trajectories from either. Microsoft has announced Microsoft Agent Framework as the successor that combines Semantic Kernel and AutoGen ideas; suites written today carry over unchanged if you migrate.

What you can test with Semantic Kernel.

Plugin-level permissions

Steps are labelled plugin.function, so assertions can forbid whole plugins in a context, for example no PaymentsPlugin calls in a servicing agent that should only read balances.

Auto function calling loops

Automatic function calling can chain several calls in one turn. Each one passes through the filter, and cycle detection bounds repeated calls before they run up cost.

Argument and result capture

KernelArguments are recorded before execution and FunctionResult values after, which gives hallucinated-argument detection both the inputs and the upstream values it needs.

.NET and Python parity

The same suite can evaluate a Python prototype and the .NET production service, with trajectories compared side by side to confirm the port behaves identically.

Set up in 5 steps.

Most teams have a first suite running against Semantic Kernel in under an hour, then add the CI gate.

Get an API key
  1. 01

    Install gonaturelle and semantic-kernel in the evaluation environment.

  2. 02

    Export GONATURELLE_API_KEY and configure your chat completion service (OpenAI, Azure OpenAI or another connector).

  3. 03

    Create the kernel used by your agent and call capture.install(kernel), which registers the function invocation filter.

  4. 04

    Pass the suite a function that sends case.input to your ChatCompletionAgent and returns the response content.

  5. 05

    Add plugin and order assertions, then gate pull requests in CI.

Semantic Kernel integration FAQ

What does the filter look like if we want to write it ourselves?

It is an async function decorated with kernel.filter(FilterTypes.FUNCTION_INVOCATION) that takes the context and next, records context.function and context.arguments, awaits next(context), then records context.result. KernelCapture does this plus redaction, ordering and upload.

Does the filter slow down production?

Install it only on the kernel used in evaluation runs. In production you can leave it out entirely, or keep it with sampling if you want to promote real traces into test cases.

Can we evaluate .NET agents?

Yes. Implement IFunctionInvocationFilter in .NET to emit steps over OpenTelemetry or the GONATURELLE ingest API, and run suites from the CLI. The trajectory format and assertions are the same as for Python.

Related integrations

All integrations

Stop testing regulated agents on generic prompts.

Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.

Start free trialBook a technical demo