[iamwhatever]

Problem / Motivation

Goal: Every new ACP_BACKENDS_* set must be classified before it ships, so a non-standard capability can no longer skip the live-capture rule by not being listed.

The X-capture rule in scripts/check_harness_parity.py only checks sets listed in X_FEATURE_SETS. That list is opt-in. A new set that rides a non-standard verb or _meta field can ship without being listed, and then it never needs a live capture. ACP_BACKENDS_MATE_WORKER_KIT (#23010) landed that way; its capture has since been added (#23534). Main has 11 more X sets with no capture today.

Why it matters

X sets decide behaviour from a proof recorded on one adapter version. When a set skips the rule, its proof can be prose or nothing, and a wrong member breaks users with no red check. Most ACP follow-up fixes came from exactly this class.

Not a goal

• Recording the missing captures for the 11 pending sets. Each moves to X_FEATURE_SETS when its capture lands.
• Changing X_CAPTURE_ALLOWLIST or its ceiling of 6.
• Detecting a pending set that is renamed and then exempted. A rename shows in review, and the gate has no notion of set identity beyond the name.

What changed (motivation → approach → change)

Opt-in coverage lets a set escape by omission → make coverage opt-out → every ACP_BACKENDS_* set must sit in exactly one of three lists:

• X_FEATURE_SETS: checked for a cited, replayed live capture, as before.
• X_REGISTRATION_PENDING (new): an X set with no capture yet, each entry naming its non-standard verb or field. Shrink-only against the base ref. A set that leaves it must join X_FEATURE_SETS with every member checked (scope None), never the exempt list. Ceiling 11 in test/test_x_feature_captures.py.
• HARNESS_EXEMPT_SETS (new): not an X feature, each entry with a one-line reason (standard ACP, routing-only, off the wire, Crew-side).

A new set in none of them fails, and the message names all three choices. A set in two lists fails. Any X_FEATURE_SETS scope must check at least one live member.

This PR also registers the three sets that already pass the capture rule: PERMISSION_KIND_FROM_TOOL_CALL, PRUNE_UNSTARTABLE_MCP, UNMARKED_COMPACTION_PAIR. docs/ci/harness-parity-gate.md describes the new contract.

Alternative considered: put the 11 X sets in X_CAPTURE_ALLOWLIST. That needs its ceiling raised from 6 to 17 and a carve-out in its ratchet, which loosens an existing gate. A separate pending list tightens without touching it.

ACP_BACKENDS_MATE_WORKER_KIT is already in X_FEATURE_SETS on main (#23534), so it is not in the pending list.

Size: about 190 lines. 43 of them are the two new lists, one entry per line.

Backwards compatibility

Compatible: every set on main is classified in this PR, so the gate stays green on the current tree. Only a future unclassified set fails.

Reader: .github/workflows/fast-gate.yml:check_harness_parity.py -- entry: release -- runs the same gate, green on this tree (HARNESS_BASE_REF=origin/main python3 scripts/check_harness_parity.py)
Reader: test/test_x_feature_captures.py:test_tree_passes -- entry: release -- passes on this tree

Tests

• test_an_unclassified_new_set_fails: a new set in no list fails with the classification message. It passes once exempt with a reason, and fails again with a blank reason.
• test_a_set_in_two_lists_fails: a set in both X_FEATURE_SETS and the exempt list fails.
• test_pending_only_shrinks_into_x_feature_sets: an added pending entry fails. A pending set moved to exempt fails. A pending set moved to X_FEATURE_SETS with an empty scope fails. Moving it with scope None passes.
• test_base_pending_is_read_back_from_the_gate_source: the base ref's pending list is parsed from the gate source, and a base without the list reads as None.
• test_allowlists_stay_under_their_ceiling: now also caps the pending list at 11.

pytest test/test_x_feature_captures.py test/test_harness_parity.py test/test_acp_frame_replay.py test/test_ratchet_scope.py test/test_main_ratchet_audit.py: 231 passed. python3 scripts/check_harness_parity.py --test: passed.

Manual verification

HARNESS_BASE_REF=origin/main python3 scripts/check_harness_parity.py: green, "56 sets classified (11 pending)".

Related Issues

Fixes #23531

Checklist

• At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|style|refactor|perf|test|chore|ci|build|revert: ...)
• Existing tests pass and new tests added for new functionality
• Self-review completed; code follows project style guidelines
• Documentation updated (if applicable)
• No secrets, credentials, or internal references in the diff

Contribution License Agreement

Contributions are accepted under the project's Apache-2.0 license.

[github-actions]

UX Review (Opus 5.5) — ⏭️ skipped

Revision d8f4093e505efc2c555a784dd67844c13c9bede9 touches no user-facing surface (no changes under website/ or committed screenshots), so the UX review was skipped. Does not block merge.

[github-actions]

Design Review (Opus 5.5) — ✅ PASS

Design-level review of d8f4093e505efc2c555a784dd67844c13c9bede9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

Design-Verdict: PASS

Nothing blocks: the change is CI-only and easy to revert, and opt-out coverage removes the "skip the rule by not listing the set" escape at its source.

Notes

• A new X set can still be put in HARNESS_EXEMPT_SETS with a plausible "standard ACP" reason, and no check catches that. The PR doesn't close that gap: it makes the choice explicit so a reviewer sees it. Only a frame-level probe could verify an exempt reason, and that would be a separate follow-up.

[DESIGN-REVIEWED] d8f4093

Description read

[DESCRIPTION-READ] 2d369b4e66182199e0fa729a70a2398a96f6799fe53bc65ca0423eb7ed743b25

That sha256 names everything this verdict read: the PR description, plus the 1 evidence file(s) the workflow collected from it and handed this review. It covers that evidence because the media strip replaces every attachment URL with the same placeholder, so an edit swapping one attachment for another would leave the description's captured text identical while what this review judged changed. Recompute it with .github/scripts/pr-description-capture.sh over the same inputs; if it differs, the description or that evidence changed after this verdict was formed and any finding drawn from either is unproven.

[github-actions]

First Principles Review (Opus 5.5) — ✅ PASS

Premise-level review of d8f4093e505efc2c555a784dd67844c13c9bede9 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS does not.

First-Principles-Verdict: PASS

Nothing blocks. Every set on main is now classified, and the one new reader of the base ref repeats an existing helper.

Notes

• Item 4 — duplicate of scripts/check_harness_parity.py:_base_allowlists: _base_pending repeats the same git show {base}:scripts/check_harness_parity.py plus the ast scan for annotated assignments, a second time per run. The only difference is that it returns None on its own when the list is missing. Subtraction: delete the git-and-parse body of _base_pending and have the loop in _base_allowlists collect X_REGISTRATION_PENDING too. Keep None per name, so a base that predates the pending list still runs the existing allowlist ratchet.

What this change ships

Inventory (6 items) — 5 justified

Intent: stop a new ACP_BACKENDS_* set from skipping the live-capture rule just because nobody listed it. This is an ADDITION (a CI gate).

1. An unclassified new ACP_BACKENDS_* set now fails the harness-parity gate, and the message names all three lists — justified
2. A set listed twice, an X scope that checks no member, or a blank reason now fails — justified
3. New X_REGISTRATION_PENDING list (11 X sets with no capture yet) that can only shrink, with a ceiling of 11 — justified
4. A second reader that parses the base ref's gate source for the pending list — duplicate of _base_allowlists
5. New HARNESS_EXEMPT_SETS list (31 sets, each with a reason) — justified
6. Three sets that already pass the capture rule are now checked in X_FEATURE_SETS — justified

[FIRST-PRINCIPLES-REVIEWED] d8f4093

Description read

[DESCRIPTION-READ] 774812941ab103be8e0bd03333baf5cabd74ea492e30ae6780f58e0cca7c32c2

That sha256 names the PR description this verdict read. Recompute it with .github/scripts/pr-description-capture.sh; if it differs, the description changed after this verdict was formed and any finding drawn from the description is unproven.

[github-actions]

Opus 5.5 Review — ✅ no blocking findings

Reviewed d8f4093e505efc2c555a784dd67844c13c9bede9 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] d8f4093

Verdict parsed from the review's SHA-scoped output markers for commit d8f4093e505efc2c555a784dd67844c13c9bede9.

False positive or not applicable? A repository writer can comment:
/ai-review override fable d8f4093e505efc2c555a784dd67844c13c9bede9 false-positive: <reason, 20 to 500 characters>

[github-actions]

GPT 6.1 Review — ✅ no blocking findings

GPT 6.1 completed its review of d8f4093e505efc2c555a784dd67844c13c9bede9 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] d8f4093

False positive or not applicable? A repository writer can comment:
/ai-review override gpt d8f4093e505efc2c555a784dd67844c13c9bede9 false-positive: <reason, 20 to 500 characters>