Triage WPT updates with the wpt-update skill in CI

[npaun]

The WPT updater now has an agent triage each bump with the
wpt-update skill and opens a pull request with the result.

The agent reads untrusted upstream content, so its job holds no
write token, DevProd PAT, or remote cache key. Its commands run
in Claude Code's sandbox, which cannot read the AI Gateway token
and which Bazel is configured to trust for TLS.

The agent's commits are exported as patches. A separate job
recreates the bump from the recorded WPT version, applies the
patches on top, and rejects any that touch paths outside the WPT
update's scope. If the triage fails, the pull request contains
only the bump. Automerge applies only when the triage made no
changes.

update_wpt.py can now pin a given version or print the latest.
The skill now treats patches/wpt as in scope for every update.

[ask-bonk]

Since last review: 1 resolved, 0 still open, 1 new.
LGTM!

The workflow now isolates runner-generated files before creating the update PR.

Other findings

• [SUGGESTION] .github/workflows/update-wpt.yml:292-307: create-pull-request@v8 already stashes tracked and untracked files after it captures the workflow-created commits. Keeping this separate stash leaves redundant state in the repository; let the action restrict the paths it stages when it finds an uncommitted change instead.

        - name: Open pull request
          id: create-pr
          uses: peter-evans/create-pull-request@v8
          with:
            # A manual run from another branch gets its own pull request branch, leaving main's alone.
            branch: automatic-update-wpt${{ github.ref_name != 'main' && format('-{0}', github.ref_name) || '' }}
            title: "Update WPT"
            token: ${{ secrets.DEVPROD_PAT }}
            author: "Workers DevProd <workers-devprod@cloudflare.com>"
            committer: "Workers DevProd <workers-devprod@cloudflare.com>"
            body-path: ${{ runner.temp }}/pr-body.md
            delete-branch: true
            add-paths: |
              build/deps/shared_deps.jsonc
              build/deps/gen
  

Not re-run: tests, docs (no author changes in their files since the last review)

---

_{Reviewed commit: 5c443f6e · github run}

[logan-gatlin]

This has been a good exercise in learning about how the security model needs to work for AI automation. Clearly the AI reviewer needs more guidance, and frankly humans will need help as well. If I hadn't just finished working on an AI automation I would have had a lot of difficulty understanding the threat model here. I don't have any more blocking concerns

[clanky-the-clanker]

🛑 Significant concerns — 2 BLOCKING security items. The triage agent can write to runner.temp, which holds GITHUB_ENV and GITHUB_PATH, so code it plants runs unsandboxed in later steps. It can also plant a report.md symlink that leaks the AI Gateway token into the public PR body. Both break the isolation this PR depends on.

Findings: 2 blocking · 1 warning · 1 question · 1 info · 1 suggestion

_{The decision is advisory: this comment never approves, blocks or requests changes on the pull request.}

🔧 Fix with your agent

Paste the following into your coding agent to investigate and fix these issues.

Fix the following code review issues found in pull request #7731.

Before making changes, review each issue and present a brief summary table:
- For each finding, state whether you agree, disagree, or need clarification
- If you disagree (e.g., the fix would require disproportionate changes
  for minimal ROI, or the finding is factually incorrect), explain why
- If you need to clarify something before deciding, ask those questions
- Then share your plan for which issues to tackle and in what order

## BLOCKING (2)

### [Security] allowWrite includes runner.temp, which holds GITHUB_ENV, GITHUB_PATH and GITHUB_OUTPUT (Claude's Edit and Write tools are also outside the Bash sandbox). The agent can make later steps run its code outside the sandbox: they hold the Actions runtime token and can read settings.json with the gateway token.
- **File:** .github/workflows/update-wpt.yml:118
- **Issue:** allowWrite includes runner.temp, which holds GITHUB_ENV, GITHUB_PATH and GITHUB_OUTPUT (Claude's Edit and Write tools are also outside the Bash sandbox). The agent can make later steps run its code outside the sandbox: they hold the Actions runtime token and can read settings.json with the gateway token.

### [Security] The agent can make report.md a symlink to ~/.config/wpt-triage/settings.json. upload-artifact runs outside the sandbox and follows the link, and the pull-request job puts the file into the public PR body, leaking the gateway token. Validate report.md is a regular file and copy it to a fresh directory; write commits.mbox there as well.
- **File:** .github/workflows/update-wpt.yml:214
- **Issue:** The agent can make report.md a symlink to ~/.config/wpt-triage/settings.json. upload-artifact runs outside the sandbox and follows the link, and the pull-request job puts the file into the public PR body, leaking the gateway token. Validate report.md is a regular file and copy it to a fresh directory; write commits.mbox there as well.

## WARNING (1)

### [Security] The Bazel server started in 'Build WPT tests' keeps running outside the sandbox during triage. If the agent's bazel client reuses it, build actions the agent adds could read settings.json. Run `bazel shutdown` before triage.
- **File:** .github/workflows/update-wpt.yml:78
- **Issue:** The Bazel server started in 'Build WPT tests' keeps running outside the sandbox during triage. If the agent's bazel client reuses it, build actions the agent adds could read settings.json. Run `bazel shutdown` before triage.

## INFO (1)

### [Code Quality] @anthropic-ai/claude-code is installed without a pinned version in a job that holds the gateway token, and the sandbox settings schema depends on the version.
- **File:** .github/workflows/update-wpt.yml:145
- **Issue:** @anthropic-ai/claude-code is installed without a pinned version in a job that holds the gateway token, and the sandbox settings schema depends on the version.

## SUGGESTION (1)

### [Documentation] Document update_wpt.py's new VERSION argument and --latest option in the skill's bump instructions.
- **File:** .opencode/skills/wpt-update/SKILL.md:60
- **Issue:** Document update_wpt.py's new VERSION argument and --latest option in the skill's bump instructions.

## QUESTIONS (1)

### [Code Quality] Is `claude-opus-5-5` a valid model name on the gateway? continue-on-error would hide a wrong name behind bump-only PRs.
- **File:** .github/workflows/update-wpt.yml:200
- **Question:** Is `claude-opus-5-5` a valid model name on the gateway? continue-on-error would hide a wrong name behind bump-only PRs.

---

🔒 Security: 🛑 3 issues

The isolation around the triage agent can be bypassed: the paths it can write are later read and run without the sandbox, and the warm Bazel server may run its actions outside the sandbox too.

Recommendation: 🔴 Serious Concerns

• [BLOCKING] allowWrite includes runner.temp, which holds GITHUB_ENV, GITHUB_PATH and GITHUB_OUTPUT (Claude's Edit and Write tools are also outside the Bash sandbox). The agent can make later steps run its code outside the sandbox: they hold the Actions runtime token and can read settings.json with the gateway token. (.github/workflows/update-wpt.yml:118)
• [BLOCKING] The agent can make report.md a symlink to ~/.config/wpt-triage/settings.json. upload-artifact runs outside the sandbox and follows the link, and the pull-request job puts the file into the public PR body, leaking the gateway token. Validate report.md is a regular file and copy it to a fresh directory; write commits.mbox there as well. (.github/workflows/update-wpt.yml:214)
• [WARNING] The Bazel server started in 'Build WPT tests' keeps running outside the sandbox during triage. If the agent's bazel client reuses it, build actions the agent adds could read settings.json. Run bazel shutdown before triage. (.github/workflows/update-wpt.yml:78)

View full review

1. The agent can write to runner.temp, which holds the runner's command files (.github/workflows/update-wpt.yml:118, :189, :205-214) — BLOCKING

• filesystem.allowWrite includes ${{ runner.temp }}. On hosted runners that directory contains _runner_file_commands/ (set_env_*, add_path_*, set_output_*), so the agent's sandboxed Bash can append to GITHUB_ENV or GITHUB_PATH.
• Claude's own Edit and Write tools run outside the Bash sandbox under bypassPermissions. The only deny rules cover ~/.config/wpt-triage and ~/.claude, so those tools can write that directory too.
• If the agent appends NODE_OPTIONS=--require …, BASH_ENV or LD_PRELOAD, or prepends a directory to PATH, its code runs outside the sandbox in Export commits and in actions/upload-artifact.
  • actions/upload-artifact receives the Actions runtime token, which can write to the Actions cache. The Linux release job's bazel-disk-cache-* keys live in that cache, so they could be poisoned.
  • The planted code can also read ~/.config/wpt-triage/settings.json, which holds the gateway token.
• Writing to set_output_* can also change the updated or version outputs that the pull-request job relies on.
• Fix:
  • Give the agent an output directory outside runner.temp (for example ~/wpt-triage-out) and drop runner.temp from allowWrite.
  • Delete settings.json and ca.key in the triage step itself, right after claude exits.
  • Consider moving the export and upload into a step or job that does not inherit an environment the agent could have changed.

2. A report.md symlink can publish the gateway token in the PR body (.github/workflows/update-wpt.yml:191, :209-214, :288-289) — BLOCKING

• The agent decides what goes into $RUNNER_TEMP/wpt-update. denyRead stops it from reading settings.json, but not from creating a symlink to it: ln -s ~/.config/wpt-triage/settings.json …/report.md.
• upload-artifact runs outside the sandbox and resolves symlinks by default. The pull-request job then cats report.md into a PR body published with DEVPROD_PAT in a public repository, which would expose cf-aig-authorization: Bearer ….
• commits.mbox has the same weakness: > writes through a symlink the agent may have planted there.
• Fix: in the update job, reject a report.md that is not a regular file, cap its size, and copy it into a fresh directory for upload. Write commits.mbox into that fresh directory as well.
• The report is also agent-written markdown, so consider removing @mentions from it.

3. The Bazel server started before the agent runs is outside the sandbox (.github/workflows/update-wpt.yml:74-78) — WARNING

• bazel build //src/wpt/... starts a Bazel server outside the sandbox, and that server is still running during triage.
• If the agent's sandboxed bazel client connects to it, build actions such as a genrule the agent adds under src/wpt run as that server, outside Claude's sandbox. They could read settings.json.
• Whether the client can reach the server through the sandbox's network isolation was not verified here.
• Fix: run bazel shutdown before the triage step, which keeps the warm disk cache but stops the server.

Checked and found fine

• The path and mode allowlist in Apply commits correctly rejects symlinks, submodules and paths outside the allowed set.
• Automerge only applies when the triage made no changes.

The PR's own description says the agent's commands cannot read the gateway token. Items 1 and 2 show two ways they can.

---

📋 Code Quality: ❓ 2 questions

The bump and patch-verification logic look sound. Two minor points remain: confirm the model name, and pin the CLI version. The isolation findings were moved to the security section.

Recommendation: 💬 Optional Nits

• [QUESTION] Is claude-opus-5-5 a valid model name on the gateway? continue-on-error would hide a wrong name behind bump-only PRs. (.github/workflows/update-wpt.yml:200)
• [INFO] @​anthropic-ai/claude-code is installed without a pinned version in a job that holds the gateway token, and the sandbox settings schema depends on the version. (.github/workflows/update-wpt.yml:145)

View full review

• Model name (.github/workflows/update-wpt.yml:200): please confirm that claude-opus-5-5 is a valid model name on the gateway. Because of continue-on-error: true, a wrong name would quietly turn every run into a PR with only the bump.
• Unpinned CLI (.github/workflows/update-wpt.yml:145): npm install --global @anthropic-ai/claude-code installs whatever version is latest. The job holds the gateway token, and the sandbox settings it uses (tlsTerminate, credentials.envVars) depend on the CLI version, so pin one.

The Bazel server, post-agent step and report.md concerns are deduplicated into the security section.

---

📝 Documentation: 💬 1 comment

The skill doesn't mention the updater's new VERSION and --latest options.

Recommendation: 💬 Optional Nits

• [SUGGESTION] Document update_wpt.py's new VERSION argument and --latest option in the skill's bump instructions. (.opencode/skills/wpt-update/SKILL.md:60)

View full review

• .opencode/skills/wpt-update/SKILL.md:60-63 only shows running the updater with no arguments. Mention the new VERSION argument and the --latest option so readers know how to pin a specific release or print the latest tag.

---

✅ All other checks passed

📖 AGENTS.md: ✅ LGTM

No AGENTS.md concerns.

---

⚡ Performance: ✅ LGTM

No performance concerns.

---

_{Reviewed commit: 5c443f6e}