Skip to content
GONATURELLE
GitLab CI logoCI/CD

GONATURELLE for GitLab CI

Run GONATURELLE gates from .gitlab-ci.yml with rules for push and merge request pipelines, JUnit reports in the merge request widget and a merge request note.

Start 14-day trialSetup steps
.gitlab-ci.yml include
.gitlab-ci.yml
stages: [test]

.gonaturelle:
  stage: test
  image: python:3.12-slim
  before_script:
    - pip install gonaturelle -r requirements.txt
  artifacts:
    when: always
    reports:
      junit: reports/gonaturelle.xml

gonaturelle-fast-path:
  extends: .gonaturelle
  rules:
    - if: $CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH
  script:
    - gonaturelle run --suite gonaturelle.yml --mode fast-path --junit reports/gonaturelle.xml

gonaturelle-full:
  extends: .gonaturelle
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
  script:
    - >
      gonaturelle run --suite gonaturelle.yml --mode full
      --baseline $CI_MERGE_REQUEST_TARGET_BRANCH_NAME
      --junit reports/gonaturelle.xml --mr-note

GitLab CI pipelines are defined in .gitlab-ci.yml, and the rules keyword controls when each job runs. GONATURELLE uses that to split the gate: a fast-path job runs on branch pushes, and a full-suite job runs in merge request pipelines where CI_PIPELINE_SOURCE is merge_request_event, comparing the source branch with CI_MERGE_REQUEST_TARGET_BRANCH_NAME as the baseline.

The gonaturelle CLI writes a JUnit XML report that GitLab reads through artifacts:reports:junit, so failed cases appear in the merge request's test report widget alongside your unit tests, each with the failing assertion as the failure message. With the --mr-note flag, the CLI also posts or updates a single merge request note with the regression table, judge deltas and token cost. That note uses a project access token stored as a masked CI/CD variable, separate from the GONATURELLE API key.

Teams on GitLab often run self-managed instances and their own runners inside a restricted network, which pairs well with GONATURELLE's single-tenant VPC and air-gapped Kubernetes deployments. Point the CLI at your in-network GONATURELLE endpoint with GONATURELLE_API_URL and no evaluation traffic leaves your environment. To enforce the gate, enable "Pipelines must succeed" in the project's merge request settings.

What you can test with GitLab CI.

Rules-based fast path and full suite

Two jobs share one template through extends; rules route push pipelines to the fast path and merge request pipelines to the full suite against the target branch.

Test report widget

JUnit output lists each failed case and assertion in the merge request widget, and GitLab tracks newly failing tests compared with the target branch.

Merge request note

One note per merge request with regressed cases, score deltas and cost, updated on each pipeline rather than appended.

Self-managed and air-gapped

Runners on your network can target an in-network GONATURELLE deployment, which suits self-managed GitLab instances in banks and health systems.

Set up in 5 steps.

Most teams have a first suite running against GitLab CI in under an hour, then add the CI gate.

Get an API key
  1. 01

    Add GONATURELLE_API_KEY as a masked and protected CI/CD variable, plus GITLAB_NOTE_TOKEN if you want merge request notes.

  2. 02

    Commit gonaturelle.yml with pinned suites, assertions and thresholds.

  3. 03

    Add the two jobs below to .gitlab-ci.yml, or include them from a shared template project.

  4. 04

    Open a merge request that changes a prompt and confirm the test report widget and note appear.

  5. 05

    Enable "Pipelines must succeed" in Settings, Merge requests so a failing gate blocks the merge.

GitLab CI integration FAQ

Why do pushes and merge requests run different jobs?

Running the full suite on every push is slow and spends eval runs on work in progress. The fast path gives quick feedback on each push; the full suite runs once the change is proposed for merge, where the comparison with the target branch matters.

Can we use GitLab's merge request pipelines and push pipelines together without duplicates?

Yes. The rules shown exclude pushes to the default branch and only run the full job for merge_request_event. If you also use workflow:rules to avoid duplicate pipelines, keep merge_request_event enabled.

Does this work on GitLab.com and self-managed?

Both. The jobs use only standard keywords (rules, extends, artifacts:reports:junit), and the CLI needs outbound access only to your GONATURELLE endpoint, which can be inside your network.

Related integrations

All integrations

Stop testing regulated agents on generic prompts.

Run your agent against 100 industry-standard clinical or financial edge cases in 5 minutes. 14-day free production trial, no card.

Start free trialBook a technical demo