[finding] A matching-rule gate cannot detect its own regression on a clean tree — --self-test is the only instrument, and nothing says so

Measured while building check:auth-mount-ledger (#10534 follow-up 4, PR #11149). Filed unassigned — recording, not claiming.

The measurement

PR #11149's ablation E mutated the new gate's reconcile to credit mounts by substring instead of exact METHOD /full/wire/path equality — reintroducing, deliberately, the exact defect #10534's own census fell into (/admin/sso/register scored as accounted-for on the strength of /admin/sso/register-saml).

Result:

production run, clean tree  ->  exit 0   (GREEN)
--self-test                 ->  exit 1   "a strict-prefix mount was CREDITED to its longer sibling"

The gate's own real run could not see that its matching rule had been broken. On a clean tree there is no unaccounted mount to mis-credit, so a looser rule produces an identical green. Only the self-test, which supplies the adversarial input, reddens.

Why this generalises past one gate

The property is not about check:auth-mount-ledger. It follows from the shape of any gate whose defect class is the matching rule itself rather than the population it scans:

a gate that counts things detects a wrong count on the real tree;
a gate that decides whether two strings correspond only exhibits a loosened rule when an input exists that the loose and strict rules classify differently — and a clean tree, by definition, contains no such input.

So for this family, --self-test is not a convenience or a CI nicety. It is the only instrument watching the rule. The production verdict is structurally incapable of covering it.

The consequence worth writing down

A future seat optimising CI time, or "simplifying" a self-test that looks redundant beside a green production run, would be removing the only coverage of the rule — and every signal available to them would say the gate still works. The production run would stay green through the entire regression.

This is the same shape as two things this repo has already ruled on, from the other direction:

the const-hoist evasion (2026-08-21) — an instrument made to read green with no change to the fact it measures;
auth-mount coverage — follow-ups 1/2/4 LANDED; remaining scope is follow-up 3 (sdk bridge) + the #10975 ledger row (body below is the original census, now historical) #10534's own leg 1 reading "5 undocumented" when the truth was 6 — a matching rule too loose, whose output looked like a clean measurement.

Here the instrument and the fact come apart in the third way: the instrument is correct, the fact is unobservable to it, and the only thing that closes the gap is a test nobody is required to keep.

Candidate population — NOT yet measured

check:auth-mount-ledger is one instance and it states the property in its own header. The question this card exists to answer is which other gates are in the family. Named as candidates by PR #11149's report, unverified:

check:route-envelope
the other error-code / vocabulary gates that classify by pattern (check:dispatcher-error-vocabulary, check-error-code-casing)

⚠️ That list is a starting point, not a census. check-error-code-casing is a suggestive case rather than a confirmed one: #10658 records it printing ✓ no lowercase error codes in 4346 scanned file(s) while two ObjectStack-authored lowercase codes shipped, because a literal reached through an || fallback matched none of its four recognizers. That is the same "matching rule too loose, production run green" signature — but whether its self-test would have caught it is exactly the thing this card would have to measure rather than assume.

Possible directions (not a recommendation — this wants grading)

Enumerate the family, then require --self-test in CI for every member, so removing one is a visible act rather than an invisible one.
A convention pin: for gates in this family, assert that the self-test contains at least one case the production run cannot produce (i.e. the self-test's adversarial inputs are not a subset of what a clean tree offers). Mechanically checkable in principle; the classifier for "in the family" is the hard part.
Documentation only — state the property in the gate-authoring guidance and rely on authors reading it. Cheapest; and this repo has repeatedly measured that a boundary living only in prose is a boundary that erodes (AuthManager trusts localhost wildcards for CSRF in production whenever trustedOrigins arrives empty — measured #10366's "development convenience" comment being the day's clearest example).

Refs

PR #11149 (where E was measured) · #10534 (the census whose leg 1 fell into this class) · #10658 (check-error-code-casing's partial read reported as complete) · the 2026-08-21 const-hoist ruling

Triage: lands in scripts/ gate conventions → domain:devx, pm:queue, type Task. Scope for the dev: direction 1 of the card — enumerate the matching-rule-gate family (candidates listed are a starting point, measure membership rather than assume), then require --self-test in CI for every member so removing one becomes a visible act. Direction 2 (the convention pin) may fall out of the enumeration but is not required; direction 3 alone is insufficient per the card's own precedent. Strengthening-only — no gate weakening is authorized here.

Generated by Claude Code

Claim: PM dispatch (devx seat)

Session: session_015ahemw8RcTgqtxrj15PEZx
Branch: claude/issue-11150-matching-rule-selftest-family

Unblocked as of a few minutes ago: this card's lint.yml holder, PR #11905, has landed (3b94a2ff2, located by (#11905) in its subject; control (#11868) → e75e34381 in the same read).

Co-tenant, and why it does not block

PR #11908 (governed draft, #10855) also adds ~21 lines to .github/workflows/lint.yml. I checked whether that is a hard block and it is not:

check:single-claim-paths declares exactly one path — .objectui-sha — and lint.yml appears 0 times in scripts/check-single-claim-paths.mjs (control: single-claim appears 13 times in the same file, so the grep discriminates). The gate is much narrower than its name suggests; it will not red this PR.

So the collision is an ordinary merge, and the ordering should be: this PR lands first, #11908 absorbs the merge. #11908 is blocked indefinitely on decision #11910 (the AGENTS.md line ceiling) and already has to merge main anyway now that #11905 landed — so letting it freeze a live workflow file would be the wrong trade.

⚠️ For the dev: a bad conflict resolution in lint.yml silently drops a CI step. If this branch goes dirty on that file, resolve by reading both sides' steps and keeping both — never by taking one side wholesale.

Tier: opus. Derived live — node scripts/pm/dispatch-gates.mjs --tier scripts/check-role-word.mjs .github/workflows/lint.yml reports no path-derived mandate, so sonnet is the floor and this is a judgment call: triage's scope says "measure membership rather than assume", and deriving which gates are matching-rule gates is the load-bearing half — a family enumerated by guess would wire --self-test for the wrong set and read as complete.

Generated by Claude Code

{
  "issue": 11150,
  "status": "done",
  "branch": "claude/issue-11150-matching-rule-selftest-family",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/11918",
  "premise_still_valid": true,
  "summary": "Enumerated the matching-rule-gate family by measurement rather than by the card's candidate list, then shipped scripts/check-self-test-wired.mjs (wired into lint.yml) which requires every script CI runs whose code ships a --self-test to have that self-test run by CI. The criterion: a gate is in the family iff its clean-tree verdict is a fixed point of predicate weakening (green means the finding set is empty; weakening can only shrink that set; the empty set is the fixed point), and it LEAVES the family exactly when the rule has a must-be-reached witness in the tree, which turns emptiness into an equality. Four ablations decided membership and two of the card's three named candidates changed status: check-error-code-casing is confirmed in the family (answering the card's explicit open question - only its self-test would have caught the loosening), check-dispatcher-error-vocabulary is FALSIFIED (its stale-row ledger reddens the production run), and check-route-envelope is in the family for one rule with NO instrument at all (filed as #11920). Because two rules inside one file land on opposite sides, no static classifier decides membership, so enforcement runs over the mechanically decidable superset rather than a named list - which is what ruling 4 warns a guessed family would get wrong. Enumeration found five members whose --self-test no workflow ran with the flag; measured one at a time, four were already covered by another route (recorded as a shrink-only, evidence-re-checked SELF_TEST_RUN_OTHERWISE ledger) and one was genuinely unrun (render-release-coverage-anchor.mjs, now wired in release-coverage-patrol.yml). Direction 2 deliberately not delivered - the classifier it needs is the one the measurement shows is undecidable; direction 3 is nowhere delivered alone.",
  "tests": "All gate runs on the final commit fbe397b22 (worktree /home/user/objectstack-11150). Gate families derived live with `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack`, no hand-written path list - it took the change set from the merge base itself ('3 path(s) vs merge base 1f6d04703', three-dot semantics) and reported 24 matched families, already discovering this PR's own gate. Every exit code captured before any pipe (redirect-then-capture), every verdict quoted from the gate's own printed line. ALL GREEN: check:nul-bytes 'OK (scanned 6637 text file(s) ... no NUL bytes)'; check:agent-test-spelling '0 violations - 365 file(s)'; check:entry-guard '155 scripts/ file(s) - every entry guard goes through invoked-as.mjs'; check:parse-guard '154 scripts/ file(s)'; check:node-version 'OK (33 setup-node step(s) across 27 workflow(s))'; check:pnpm-acquisition; check:pnpm-filter-targets '135/168 --filter occurrence(s)'; check:required-contexts '6 required context name(s) pinned'; check:shard-attestation; check:workflow-status-functions '27 workflow file(s), 50 job(s)'; check:cross-package-test-inputs 'All 109 self-test cases passed.'; check:type-check-coverage 'OK - 65/78 workspace packages type-checked'; check-aggregator-roster; check-ci-filter-parity; check-cross-package-test-inputs; check-release-section-coverage (advisory); check-required-contexts; check-shard-attestation; check-step-collectors --self-test '120 assertions, 3 live block(s) driven under a real bash -e'; check-step-collectors '341 run: steps across 27 workflow(s); 3 step(s) run 2+ independent self-tests, all through a collector' (unchanged count - both blocks added here name ONE script, the tolerated <gate> --self-test + <gate> shape); check-whole-set-label-write; docs-audit/check-drift-comment '56 cases pass'; pm/ci-failure --self-test; render-release-coverage-anchor --self-test '10 assertions pass'. NEW GATE: `node scripts/check-self-test-wired.mjs --self-test` exit 0 '4 live ledger row(s) verified, plus the comment mask, the right boundary, alias resolution and both audit directions'; `node scripts/check-self-test-wired.mjs` exit 0 'every one of the 127 script(s) CI runs that ship a --self-test has that self-test run by CI. scope: 172 file(s) under scripts/, 136 carrying --self-test in code (comments masked); 127 of those are run by 27 workflow(s); 123 through the flag, 4 through a recorded route.' ABLATIONS (5 total, each mutation confirmed on disk before any reading - removed-anchor count 1->0 AND injected-marker count 0->1, each under a restore trap, each restore confirmed by an empty `git diff --stat`; no build/dist step applies, these are plain node scripts): (A) check-error-code-casing local-fallback recognizer -> never-matching: production exit 0 GREEN, --self-test exit 1 RED with 4 cases 'expected 1 hit(s), got 0' => IN FAMILY, and this answers the card's open question. (B) check-route-envelope MODULES[file] -> basename fallback: production exit 0, --self-test exit 0 => in family AND uninstrumented. (C) POSITIVE CONTROL for B, same gate different mutation - discovery convention narrowed: production exit 1 RED => the harness can redden this gate, so B's double-green is a property of the rule. (D) check-dispatcher-error-vocabulary objlittemplate recognizer -> never-matching: production exit 1 RED '[stale-row] ... declares APPROVAL_*_FAILED at packages/rest/src/rest-server.ts (objlittemplate) but the scan no longer finds it' => NOT in family, candidate falsified. (E) check-auth-mount-ledger exact accounting -> strict-prefix credit, re-measured on current main rather than inherited: production exit 0 GREEN (still printing 'accounting is EXACT on METHOD path'), --self-test exit 1 RED 'a strict-prefix mount was CREDITED to its longer sibling' => reproduces PR #11149. REVERSE VERIFICATION of the new gate, from the committed state: deleted `node scripts/check-step-collectors.mjs --self-test` from its lint.yml step (run-line anchor 1->0, marker 0->1) => gate exit 1, '[self-test-not-run] scripts/check-step-collectors.mjs'; restored, git diff --stat empty. TWO DECLARED NARROWINGS. (1) `pnpm lint` run narrowed with all three evidence pieces: population read from eslint.config.mjs itself via isPathIgnored/calculateConfigForFile - 1 of 3 changed paths is in eslint's population, the two .yml files report ignored=true with zero configured rules, and that path has exactly 2 active rules; count read from --format json - results.length=1, errors 0, warnings 0; invariance is the config's OWN recorded property ('never enables type-aware linting (no parserOptions.project, no typed @typescript-eslint rules) for ANY file'), so no cross-file program exists for this diff to move. Positive control in the same run on the same real file path: two planted violations each produced exactly 1 error where the real file produced 0 (an earlier control attempt on a synthetic path produced 0 and was discarded as non-discriminating). (2) check:type-check-debt NOT run - its --re-measure leg refuses without the built workspace closure ('56 workspace dependenc(ies) ... have no built type entry point on disk') and asks for a full turbo build behind a contended shared lock; its sibling check:type-check-coverage passes, and this diff adds no package, tsconfig or typecheck script, so it cannot move that reading. CO-TENANCY: checked against origin/main bb62d4d08 - no commit between the merge base and origin/main touches lint.yml or release-coverage-patrol.yml, and `git merge-tree` reports 0 conflicted hunks, so this branch never went dirty on lint.yml and no step count is in question. skip-changeset applied and READ BACK: labels are now ci/cd, size/l, skip-changeset (whole-set write via the MCP issue_write union - the additive labels endpoint was unavailable, $GITHUB_TOKEN answers 401 Bad credentials from this seat; declared here rather than left implicit).",
  "open_questions": [],
  "out_of_scope_findings": [
    "filed as #11920: check-route-envelope's declared-vs-discovered module correspondence (MODULES[file]) has no instrument - loosening it to a basename fallback moves neither the production run nor --self-test, while a control mutation on the same gate reddens production; needs one --self-test case driving audit()'s reconciliation over a synthetic disagreeing pair"
  ]
}

Generated by Claude Code

os-dev-report

(Supersedes the comment above: its HTML-comment marker did not survive GitHub's body sanitizer, and neither did an angle-bracketed fragment inside the tests string. This one carries the marker as literal text and spells the fragment out. The JSON is otherwise identical.)

{
  "issue": 11150,
  "status": "done",
  "branch": "claude/issue-11150-matching-rule-selftest-family",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/11918",
  "premise_still_valid": true,
  "summary": "Enumerated the matching-rule-gate family by measurement rather than by the card's candidate list, then shipped scripts/check-self-test-wired.mjs (wired into lint.yml) which requires every script CI runs whose code ships a --self-test to have that self-test run by CI. The criterion: a gate is in the family iff its clean-tree verdict is a fixed point of predicate weakening (green means the finding set is empty; weakening can only shrink that set; the empty set is the fixed point), and it LEAVES the family exactly when the rule has a must-be-reached witness in the tree, which turns emptiness into an equality. Four ablations decided membership and two of the card's three named candidates changed status: check-error-code-casing is confirmed in the family (answering the card's explicit open question - only its self-test would have caught the loosening), check-dispatcher-error-vocabulary is FALSIFIED (its stale-row ledger reddens the production run), and check-route-envelope is in the family for one rule with NO instrument at all (filed as #11920). Because two rules inside one file land on opposite sides, no static classifier decides membership, so enforcement runs over the mechanically decidable superset rather than a named list - which is what ruling 4 warns a guessed family would get wrong. Enumeration found five members whose --self-test no workflow ran with the flag; measured one at a time, four were already covered by another route (recorded as a shrink-only, evidence-re-checked SELF_TEST_RUN_OTHERWISE ledger) and one was genuinely unrun (render-release-coverage-anchor.mjs, now wired in release-coverage-patrol.yml). Direction 2 deliberately not delivered - the classifier it needs is the one the measurement shows is undecidable; direction 3 is nowhere delivered alone.",
  "tests": "All gate runs on the final commit fbe397b22 (worktree /home/user/objectstack-11150). Gate families derived live with `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack`, no hand-written path list - it took the change set from the merge base itself ('3 path(s) vs merge base 1f6d04703', three-dot semantics) and reported 24 matched families, already discovering this PR's own gate. Every exit code captured before any pipe (redirect-then-capture), every verdict quoted from the gate's own printed line. ALL GREEN: check:nul-bytes 'OK (scanned 6637 text file(s) ... no NUL bytes)'; check:agent-test-spelling '0 violations - 365 file(s)'; check:entry-guard '155 scripts/ file(s) - every entry guard goes through invoked-as.mjs'; check:parse-guard '154 scripts/ file(s)'; check:node-version 'OK (33 setup-node step(s) across 27 workflow(s))'; check:pnpm-acquisition; check:pnpm-filter-targets '135/168 --filter occurrence(s)'; check:required-contexts '6 required context name(s) pinned'; check:shard-attestation; check:workflow-status-functions '27 workflow file(s), 50 job(s)'; check:cross-package-test-inputs 'All 109 self-test cases passed.'; check:type-check-coverage 'OK - 65/78 workspace packages type-checked'; check-aggregator-roster; check-ci-filter-parity; check-cross-package-test-inputs; check-release-section-coverage (advisory); check-required-contexts; check-shard-attestation; check-step-collectors --self-test '120 assertions, 3 live block(s) driven under a real bash -e'; check-step-collectors '341 run: steps across 27 workflow(s); 3 step(s) run 2+ independent self-tests, all through a collector' (unchanged count - both blocks added here name ONE script, the tolerated SCRIPT --self-test then SCRIPT shape). check-whole-set-label-write; docs-audit/check-drift-comment '56 cases pass'; pm/ci-failure --self-test; render-release-coverage-anchor --self-test '10 assertions pass'. NEW GATE: `node scripts/check-self-test-wired.mjs --self-test` exit 0 '4 live ledger row(s) verified, plus the comment mask, the right boundary, alias resolution and both audit directions'; `node scripts/check-self-test-wired.mjs` exit 0 'every one of the 127 script(s) CI runs that ship a --self-test has that self-test run by CI. scope: 172 file(s) under scripts/, 136 carrying --self-test in code (comments masked); 127 of those are run by 27 workflow(s); 123 through the flag, 4 through a recorded route.' ABLATIONS (5 total, each mutation confirmed on disk before any reading - removed-anchor count 1 to 0 AND injected-marker count 0 to 1, each under a restore trap, each restore confirmed by an empty `git diff --stat`; no build/dist step applies, these are plain node scripts): (A) check-error-code-casing local-fallback recognizer made never-matching: production exit 0 GREEN, --self-test exit 1 RED with 4 cases 'expected 1 hit(s), got 0' means IN FAMILY, and this answers the card's open question. (B) check-route-envelope MODULES[file] loosened to a basename fallback: production exit 0, --self-test exit 0 means in family AND uninstrumented. (C) POSITIVE CONTROL for B, same gate different mutation - discovery convention narrowed: production exit 1 RED, so the harness can redden this gate and B's double-green is a property of the rule. (D) check-dispatcher-error-vocabulary objlittemplate recognizer made never-matching: production exit 1 RED '[stale-row] ... declares APPROVAL_*_FAILED at packages/rest/src/rest-server.ts (objlittemplate) but the scan no longer finds it' means NOT in family, candidate falsified. (E) check-auth-mount-ledger exact accounting loosened to strict-prefix credit, re-measured on current main rather than inherited: production exit 0 GREEN (still printing 'accounting is EXACT on METHOD path'), --self-test exit 1 RED 'a strict-prefix mount was CREDITED to its longer sibling', reproducing PR #11149. One earlier ablation attempt was a syntax-error no-op and was discarded rather than read. REVERSE VERIFICATION of the new gate, from the committed state: deleted `node scripts/check-step-collectors.mjs --self-test` from its lint.yml step (run-line anchor 1 to 0, marker 0 to 1) giving gate exit 1, '[self-test-not-run] scripts/check-step-collectors.mjs'; restored, git diff --stat empty. TWO DECLARED NARROWINGS. (1) `pnpm lint` run narrowed with all three evidence pieces: population read from eslint.config.mjs itself via isPathIgnored/calculateConfigForFile - 1 of 3 changed paths is in eslint's population, the two .yml files report ignored=true with zero configured rules, and that path has exactly 2 active rules; count read from --format json - results.length=1, errors 0, warnings 0; invariance is the config's OWN recorded property ('never enables type-aware linting (no parserOptions.project, no typed @typescript-eslint rules) for ANY file'), so no cross-file program exists for this diff to move. Positive control in the same run on the same real file path: two planted violations each produced exactly 1 error where the real file produced 0 (an earlier control attempt on a synthetic path produced 0 and was discarded as non-discriminating). (2) check:type-check-debt NOT run - its --re-measure leg refuses without the built workspace closure ('56 workspace dependenc(ies) ... have no built type entry point on disk') and asks for a full turbo build behind a contended shared lock; its sibling check:type-check-coverage passes, and this diff adds no package, tsconfig or typecheck script, so it cannot move that reading. CO-TENANCY: checked against origin/main bb62d4d08 - no commit between the merge base and origin/main touches lint.yml or release-coverage-patrol.yml, and `git merge-tree` reports 0 conflicted hunks, so this branch never went dirty on lint.yml and no step count is in question. skip-changeset applied and READ BACK: labels are now ci/cd, size/l, skip-changeset (whole-set write via the MCP issue_write union - the additive labels endpoint was unavailable, $GITHUB_TOKEN answers 401 Bad credentials from this seat; declared here rather than left implicit). CI not awaited, per the standing contract: report is delivered at draft-PR time.",
  "open_questions": [],
  "out_of_scope_findings": [
    "filed as #11920: check-route-envelope's declared-vs-discovered module correspondence (MODULES[file]) has no instrument - loosening it to a basename fallback moves neither the production run nor --self-test, while a control mutation on the same gate reddens production; needs one --self-test case driving audit()'s reconciliation over a synthetic disagreeing pair"
  ]
}

Generated by Claude Code

Navigation Menu

Repository navigation

[finding] A matching-rule gate cannot detect its own regression on a clean tree — `--self-test` is the only instrument, and nothing says so #11150

The measurement

Why this generalises past one gate

The consequence worth writing down

Candidate population — NOT yet measured

Possible directions (not a recommendation — this wants grading)

Refs

Co-tenant, and why it does not block

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[finding] A matching-rule gate cannot detect its own regression on a clean tree — --self-test is the only instrument, and nothing says so #11150

Description

The measurement

Why this generalises past one gate

The consequence worth writing down

Candidate population — NOT yet measured

Possible directions (not a recommendation — this wants grading)

Refs

Activity

os-zhuang commented on Aug 23, 2026

os-steve commented on Aug 25, 2026

Co-tenant, and why it does not block

os-steve commented on Aug 25, 2026

os-steve commented on Aug 25, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

[finding] A matching-rule gate cannot detect its own regression on a clean tree — `--self-test` is the only instrument, and nothing says so #11150