[leonlaiyc]

Problem / Motivation

Goal: the backend tests that read a repository file as text decode it as UTF-8 on every host, so they pass or fail on what the file says rather than on the host's code page.

On a Windows host whose ANSI code page is cp950 (zh-TW), 24 tests fail before they assert anything:

UnicodeDecodeError: 'cp950' codec can't decode byte 0xe2 in position 33: illegal multibyte sequence

Each one reads a UTF-8 file from the repository (a source module, a CloudFormation template, a JSON fixture, a shell script, pyproject.toml) with a bare read_text() or open(), which decodes with the host's code page. The first —, → or … in the file is not valid cp950. On a cp1252 host, the code page GitHub's Windows runners use, the same bytes decode without an error into other characters, so CI stays green while the test reads something other than the file.

Why it matters

A contributor on a CJK-locale Windows machine cannot get a green local run of these tests, and scripts/local-gate.py reports them red on diffs that never touched them. D5 of the determinism contract already rules out depending on the host's locale; these reads do.

Not a goal

• Product code. This changes no src/ file; locale decoding there is check_subprocess_encoding.py's territory and the reads it covers.
• The other bare read_text() calls in test/ (about 1,500 on main). Almost all of them read ASCII files and decode the same on every host. Only the reads measured to fail are changed.
• A lint rule that refuses a bare read_text() in tests.

What changed (motivation → approach → change)

• Symptom: 24 tests raise UnicodeDecodeError: 'cp950' on a cp950 host.
• Root cause: each reads a non-ASCII UTF-8 file without naming the codec.
• How the set was found: an AST scan for a bare read_text() / text-mode open() in a test module that names a repository file holding non-ASCII bytes gave 97 candidate files. All 97 were run on a cp950 host, and these are the ones that failed with the decode error.
• Change: each failing read names encoding="utf-8", at 24 sites in 21 test files. The four open() sites in test_deploy_round29_fixes.py and test_deploy_web_iam.py hand the stream to yaml_helpers.load_with, which is where the traceback lands.
• docs/system-specs/common/testing-conventions.md D5 gets one sentence: a test that reads a repository file as text names the codec, and why.

Backwards compatibility

Compatible: test-only, plus one documentation sentence. The files are UTF-8, so on a UTF-8 host every read returns the same text it did before.

Removes nothing: no assertion, test or file is removed or narrowed.

Tests

• The 24 failing tests, on a cp950 Windows host (Python 3.13.5): 24 failed on main, 24 passed with this change.
• The 21 changed files in full: on Python 3.12.13, 1,003 passed and 2 failed; on 3.13.5, 1,001 passed and 4 failed. Every remaining failure also fails on main on this host: two need symlink-creation privilege (WinError 1314, test_agent_panel_store's symlinked-directory tests), and two in test_pod_provision look for uv on the 3.13 interpreter only.
• test/test_flake_pattern_ratchet.py: 60 passed. black (the 13 non-baselined files), isort, flake8, check_comment_history, check_brand_name, check_file_line_cap and scripts/docs_lint.py pass.
• A trial merge with the open PRs that touch these files (feat(browser): import cookies into the agent's browser from the dashboard #23332, test(chat-slack): give weak tests real checks and merge twin cases #23743, fix(browser): let the native panel browse without playwright-cli #21627, fix(dev-fleet): stamp the revision an artifact was built from #22686, fix(members): record select_crew's routing decision through the gateway #23457, fix(security): classify redaction by what a value can do (awaiting design approval) #22887) is clean. refactor: retire crew triggers, session_color, route_crew, select_crew #19274 and feat(chat): /rewind [N] forks the chat from N turns back #17774 already conflict with main, and this head adds no conflicting file.

Proof: not provable -- test-only change

Manual verification

N/A — test-only change; the before/after run on the cp950 host above is the check.

Related Issues

Fixes #23774

Checklist

• At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
• Existing tests pass and new tests added for new functionality
• Self-review completed; code follows project style guidelines
• Documentation updated (if applicable)
• No secrets, credentials, or internal references in the diff

Contribution License Agreement

🤖 Generated with Claude Code

[github-actions]

First Principles Review (Opus 5.5, fork) — ⏭️ skipped

Revision 832bc73d74927aa562ff901372202e8a56632e14 ships no reviewable capability (docs, tests or generated files only), so there is nothing to inventory. Does not block merge.

[github-actions]

UX Review (Opus 5.5, fork) — ⏭️ skipped

Revision 832bc73d74927aa562ff901372202e8a56632e14 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

[github-actions]

Design Review (Opus 5.5, fork) — ✅ PASS

Design-level review of 832bc73d74927aa562ff901372202e8a56632e14 via the fork AI-review pipeline — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing to check. The change only touches tests, it was verified on the failing host, and it is easy to revert.

[DESIGN-REVIEWED] 832bc73

Description read

[DESCRIPTION-READ] f2437466c2a7d0d8a743c79546647b74d2e147b68939b13aa81cf6c1f06d7305

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

[github-actions]

Opus 5.5 Review (fork) — ✅ no blocking findings

Reviewed 832bc73d74927aa562ff901372202e8a56632e14 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 832bc73

[github-actions]

GPT 6.1 Review (fork) — ✅ no blocking findings

Reviewed 832bc73d74927aa562ff901372202e8a56632e14 via the fork AI-review pipeline; updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 832bc73

[iamwhatever]

Approve at 832bc73.

Each change names encoding="utf-8" on a test's read of a repository file and changes nothing else, so on a UTF-8 host every read returns the same text and on a cp950 host it stops failing before the assertion. The one-sentence D5 addition states the rule (testing-conventions.md#L83-L87). AI lanes pass and every check is green. I read CI results and did not run tests.

— crew-pr-reviewer (r124)