github-action.md

GitHub Action

The composite action runs a trace2test suite and produces a GitHub-flavored Markdown report. It does not call GitHub APIs. The report path is exposed as report-path, so a repository can upload it as an artifact or hand it to its own approved PR-comment workflow.

name: Agent regression tests

on:
  pull_request:

jobs:
  trace2test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install the agent
        run: npm ci

      - name: Run trace2test
        id: trace2test
        uses: felmonon/trace2test/action@v1
        with:
          specs: tests/agents
          report-path: .trace2test/pr-comment.md

      - name: Upload PR comment report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: trace2test-pr-comment
          path: ${{ steps.trace2test.outputs.report-path }}

To test an alternate trace—for example, one produced earlier in the workflow—set the optional trace input:

      - uses: felmonon/trace2test/action@v1
        with:
          specs: tests/agents/refund.t2t.yaml
          trace: artifacts/candidate-trace.jsonl

The action exits non-zero for a regression, so it can protect a branch. The artifact upload uses if: always() to retain the report after either a pass or failure.