[bolichen97]

Problem / Motivation

Goal: Redaction tests that only need "some credential" use a fake value the redactor treats as a secret, not an AWS access key ID.

About 430 test files use the AWS documentation key ID as their stand-in for a credential. The redaction design in #22552 moves key IDs to the Identifier tier, which is never redacted. Once that lands, every one of those tests would stop exercising redaction at all.

Why it matters

Without this swap, the key-ID change would have to rewrite these tests in the same PR as the security change, which buries the real behaviour change under thousands of mechanical lines. Done first, the swap is reviewable on its own, and the follow-up PR carries only the tests whose expected result really changes.

Not a goal

• Changing what is redacted. No product code changes here.
• Tests that check key-ID behaviour itself; those change with the key-ID PR.
• The other half of the files, which is in the first PR of this pair.

What changed (motivation → approach → change)

Tests used a key ID to mean "a credential", so a tier change for key IDs would silently empty them. Each such test now uses a fake token that is a secret under every tier: a GitLab-shaped glpat- fake with the same 16-character body, a Slack-shaped xoxb- fake where the test depends on a 20-character length or on a body that stops at _, and a labelled fake secret key where the test reads a labelled value.

Where a test cut the fake at a fixed offset ([4:], a padding length, a split index), the offset moves with the longer prefix so the test still exercises the same boundary. A few pinned digests of input-echoing transcripts are updated for the new input text. website/src/test/sanitizeExfiltrationUrls.test.ts takes the Slack-shaped fake, which the frontend mirror also redacts.

This is the second of two test-only PRs; together they cover every such test. The two touch disjoint files and each passes on main alone.

Backwards compatibility

Compatible: only tests change; nothing a user runs is touched.

Removes nothing: every test keeps its assertion; only the fake input changes.

Tests

Every changed test passes on origin/main with the swap, run file by file (about 12,400 tests). The only reds in that run also fail on unmodified main on this host (test_design_tweak_backend.py, test_design_tweak_devproc.py: the host refuses the preview with 403; test_prompt_override_encoding.py: a host binary check), and CI runs them in its own environment.

No test is added, so no determinism proof applies.

Manual verification

N/A - test-only change; the suite is the verification.

Related Issues

Part of #22552

Checklist

• At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
• Existing tests pass and new tests added for new functionality
• Self-review completed; code follows project style guidelines
• Documentation updated (if applicable)
• No secrets, credentials, or internal references in the diff

[github-actions]

First Principles Review (Opus 5.5) — ⏭️ skipped

Revision af4e42be8289021f47d9920686fda3099719a449 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

[github-actions]

UX Review (Opus 5.5) — ✅ PASS

UX-level review of af4e42be8289021f47d9920686fda3099719a449 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

UX-Verdict: PASS

Nothing to check. Every changed file is a test file, so the diff adds or changes no user-visible control and needs no screenshot.

[UX-REVIEWED] af4e42b

Description read

[DESCRIPTION-READ] 8d4dc988629494ef23294d9eee5dadcd919d01044d7bca1ed4a7f0c8c4034576

That sha256 names everything this verdict read: the PR description, plus the 3 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

[github-actions]

Design Review (Opus 5.5) — ✅ PASS

Design-level review of af4e42be8289021f47d9920686fda3099719a449 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Test-only fixture swap with no product code or user-visible surface; design issues: none.

[DESIGN-REVIEWED] af4e42b

Description read

[DESCRIPTION-READ] 80c728bc5c49231f15249bc6c415f7bf0097feb9f10a4232658c240237a0ac91

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

[github-actions]

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of af4e42be8289021f47d9920686fda3099719a449 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] af4e42b

False positive or not applicable? A repository writer can comment:
/ai-review override gpt af4e42be8289021f47d9920686fda3099719a449: <one-sentence reason>

[github-actions]

Opus 5.5 Review — ✅ no blocking findings

Reviewed af4e42be8289021f47d9920686fda3099719a449 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] af4e42b

Verdict parsed from the review's SHA-scoped output markers for commit af4e42be8289021f47d9920686fda3099719a449.

False positive or not applicable? A repository writer can comment:
/ai-review override fable af4e42be8289021f47d9920686fda3099719a449: <one-sentence reason>

[bolichen97]

• span=b0a70c4ebb90 fixed: split offset in the markup-split fixture

The split kept the old 4-letter prefix offset, so the fixture read glpat-**I**-IOSF... and joined to a different token than KEY.
Now KEY[7:] (prefix glpat- plus the bolded I), so the payload reassembles exactly to KEY at all three sites: test/test_whatsapp_renderer.py:75, :118 and test/test_whatsapp_turn_renderer.py:24.
Fixed in af4e42b; both files pass.

[bolichen97]

Superseded by #22887, which carries this change together with the rest of #22552.