Repository navigation
test: use a secret-shaped fake credential in redaction tests (1/2) - #22736
bolichen97 wants to merge 1 commit into
Conversation
UX Review (Opus 5.5) — ⏭️ skippedRevision |
Design Review (Opus 5.5) — ✅ PASSDesign-level review of Design-Verdict: PASS Nothing blocks: this is a test-only swap of fake credentials ahead of the key-ID tier change, and it leaves product code untouched. [DESIGN-REVIEWED] 7fb5855 Description read[DESCRIPTION-READ] 589f60dccaf3b374882f9d00a79b61f3a77f76644b121c9ec5f37bd78af9b14d That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with |
First Principles Review (Opus 5.5) — ✅ PASSPremise-level review of First-Principles-Verdict: PASS This is a test-only change of input values: it adds no product surface, defaults or riders, and the reason for it is recorded in #22552. What this change shipsInventory (3 items) — 3 justifiedIntent: keep the redaction tests testing redaction after #22552 stops redacting AWS key IDs. This is a FIX done ahead of that change, in tests only.
[FIRST-PRINCIPLES-REVIEWED] 7fb5855 Description read[DESCRIPTION-READ] 275860c85c6b8104fe9696225446d052ea53b8d1df26cae20b856137b01a4c3e That sha256 names the PR description this verdict read. Recompute it with |
Opus 5.5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
GPT 6.1 Review — ✅ no blocking findingsGPT 6.1 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
d958560 to
b7eda21
Compare
b7eda21 to
8c3dcb5
Compare
8c3dcb5 to
142abe5
Compare
Tests that use an AWS access key ID as their stand-in for "a credential" now use a fake token the redactor treats as a secret (a GitLab- or Slack-shaped fake, or a labelled fake secret key). No product code changes; every test passes on main before and after.
142abe5 to
7fb5855
Compare
Problem / Motivation
Goal: Redaction tests that only need "some credential" use a fake value the redactor treats as a secret, not an AWS access key ID.
About 430 test files use the AWS documentation key ID as their stand-in for a credential. The redaction design in #22552 moves key IDs to the Identifier tier, which is never redacted. Once that lands, every one of those tests would stop exercising redaction at all.
Why it matters
Without this swap, the key-ID change would have to rewrite these tests in the same PR as the security change, which buries the real behaviour change under thousands of mechanical lines. Done first, the swap is reviewable on its own, and the follow-up PR carries only the tests whose expected result really changes.
Not a goal
What changed (motivation → approach → change)
Tests used a key ID to mean "a credential", so a tier change for key IDs would silently empty them. Each such test now uses a fake token that is a secret under every tier: a GitLab-shaped
glpat-fake with the same 16-character body, a Slack-shapedxoxb-fake where the test depends on a 20-character length or on a body that stops at_, and a labelled fake secret key where the test reads a labelled value.Where a test cut the fake at a fixed offset (
[4:], a padding length, a split index), the offset moves with the longer prefix so the test still exercises the same boundary. The shared fixture insrc/kiro_crew/testing/fake_computer_use.pyis test-only (no runtime caller) and takes the same swap.This is the first of two test-only PRs; together they cover every such test.
Backwards compatibility
Compatible: only tests and a test-only fixture module change; nothing a user runs is touched.
Removes nothing: every test keeps its assertion; only the fake input changes.
Tests
Every changed test passes on
origin/mainwith the swap, run file by file (about 15,600 tests). The only reds in that run also fail on unmodified main on this host (test_design_tweak_backend.py,test_design_tweak_devproc.py: the host refuses the preview with 403;test_prompt_override_encoding.py: a host binary check), and CI runs them in its own environment.No test is added, so no determinism proof applies.
Manual verification
N/A - test-only change; the suite is the verification.
Related Issues
Part of #22552
Checklist
feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)