CI: gate PRs with Tripwire

Tripwire is designed to be the check that gates a pull request. Two surfaces share the same engine and the same exit codes:

• The GitHub Action (action.yml) — drop-in PR gating with artifacts and auto-filed issues.
• The CLI (tripwire run) — the underlying gate, usable from any CI system or terminal.

Exit codes (the gate)

The CLI's exit code is the gate. It's the same everywhere:

Code	Meaning
0	All cases passed.
1	At least one case failed (an assertion didn't hold).
2	At least one case is broken (the run couldn't be carried out).
3	Spec / config error (bad path, no suites matched, missing API key).

Broken (2) outranks failed (1) outranks ok (0). A non-zero exit fails the job and blocks the PR.

The GitHub Action

Add one step to a workflow. The action sets up Python + uv, installs the cross-platform backend runtime, installs Chromium, runs your suites headless, writes reports, and emits counts as outputs.

name: Tripwire E2E

on:
  pull_request:

concurrency:
  group: tripwire-${{ github.ref }}
  cancel-in-progress: true

jobs:
  e2e:
    name: Tripwire E2E (gate)
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Run Tripwire
        id: tripwire
        uses: tripwire-e2e/tripwire@v1     # inside the Tripwire repo itself, use `uses: ./`
        with:
          suite: "examples/suites/*.tripwire.yaml"
          file-issues: "github"
          anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }}

      - name: Summary
        if: ${{ always() }}
        run: |
          echo "Passed:  ${{ steps.tripwire.outputs.passed }}"
          echo "Failed:  ${{ steps.tripwire.outputs.failed }}"
          echo "Broken:  ${{ steps.tripwire.outputs.broken }}"
          echo "Skipped: ${{ steps.tripwire.outputs.skipped }}"

A ready-to-copy version lives at examples/ci/example-tripwire.yml — drop it into your own project's .github/workflows/ and set the ANTHROPIC_API_KEY secret. Referencing tripwire-e2e/tripwire@v1 needs a published v1 release and read access to the action's repo (same org, or public).

Inputs

Input	Default	Purpose
suite	suites/*.tripwire.yaml	Path or glob of suites to run.
file-issues	github	Tracker provider to file failures into, or '' / none to disable.
anthropic-api-key	(required)	The Anthropic key. Pass secrets.ANTHROPIC_API_KEY.
out	tripwire-artifacts	Directory for report.json / junit.xml / report.html.
python-version	3.12	Python to set up.
working-directory	backend	Path to the Tripwire backend (contains app/cli.py).
upload-artifacts	true	Upload the report directory as a workflow artifact.

Outputs

passed · failed · broken · skipped — the per-status case counts, scraped from the CLI's summary line. Use them in a later step (e.g. a Slack ping or a PR comment).

What the action injects for issue filing

When file-issues is enabled, the action passes these to the run so filed issues are rich and deduped:

GITHUB_TOKEN: ${{ github.token }}
GITHUB_REPO:  ${{ github.repository }}
GIT_SHA:      ${{ github.sha }}
CI_RUN_URL:   ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

The issue body carries the run URL and commit, and re-runs comment on the existing issue instead of duplicating. See Filing issues.

The CLI directly

The same gate, from any CI system or your terminal. See the full CLI reference.

cd backend
ANTHROPIC_API_KEY=sk-ant-... python -m app.cli run "../suites/*.tripwire.yaml" \
  --file-issues github \
  --out artifacts
echo "exit: $?"     # 0 / 1 / 2 / 3

• run <path-or-glob> runs one or more suites; a bare path or a ** glob both work.
• --file-issues <provider> files failures (needs the matching env — GITHUB_TOKEN, GITHUB_REPO, …). Comma-separate to file to several at once.
• --out <dir> writes report.json, junit.xml, and report.html. Multiple suites land in per-suite subdirs.
• --headed runs with a visible browser (default is headless — required for CI).

The CLI prints a per-suite summary, a line per non-passing case, a TOTAL line, and a machine-readable ::tripwire-output:: passed=… failed=… broken=… skipped=… line the Action scrapes for its outputs.

JUnit reports

Every run writes a junit.xml alongside the JSON and HTML reports. Point your CI's test-reporter at it to get per-case pass/fail in the PR's checks UI. With the GitHub Action, the whole report directory is uploaded as the tripwire-report artifact (when upload-artifacts is true).

Notes for runners

• The default Playwright path is cross-platform — Ubuntu runners are fine. The Action installs the exact non-mac runtime deps (skipping the macOS-only pyobjc-framework-quartz, whose import is lazy and never reached on the Playwright path) and playwright install --with-deps chromium.
• Use a concurrency group (as above) to cancel superseded runs on rapid pushes.
• Keep ANTHROPIC_API_KEY in repo/organization secrets, never in the workflow file.

Next: Filing issues · Connecting server logs