Repository navigation
fix: stop the gate reporting a leak when no benchmark exists - #21
Merged
Merged
Conversation
With no worker receipt, the Stage 17 gate's security dimension said "retrieval benchmark reported a cross-tenant leak". The benchmark was missing, not leaking: `undefined !== 0` held. The reason now distinguishes the two cases: - missing benchmark: "cross-tenant isolation is unproven: the retrieval benchmark is missing." - counted leaks: "retrieval benchmark reported N cross-tenant leak(s)." Both remain a critical FAIL, so the gate stays fail-closed. Only the reason text changes. The C0.4 golden corpus caught the change in exactly one case, GENESISRAG17-RECEIPTS. It is re-baselined; the transcript diff is this reason text only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a batch has no worker receipt, the Stage 17 quality gate gave this security reason:
That was wrong. The benchmark was missing, and no leak had been measured. The check was
crossTenantLeaks !== 0, which is true when the value isundefined. This was noted in the corpus v2 report.The reason text (
packages/gks-core/src/pipeline.mjs) now distinguishes the two cases:cross-tenant isolation is unproven: the retrieval benchmark is missing.crossTenantLeaks: N, where N > 0retrieval benchmark reported N cross-tenant leak(s).crossTenantLeaks: 0What stays the same:
FAILin both cases, so the gate still blocks publication when isolation cannot be proven.verdict,allowPublicationandcriticalare unchanged in every case. Only the reason string changes.benchmark, so the missing-benchmark case only happens when there is no receipt at all.Compatibility: a consumer that matches the old reason string exactly will no longer match. Nothing in this repository does.
Golden corpus
This PR is the first behaviour change since the C0.4 corpus became replayable (#20). Before re-baselining,
npm run check:corpusfailed in exactly one case,C0.4-GENESISRAG17-RECEIPTS, at the security reason. After re-baselining, the diff ofexpected/*.jsonis only this reason text, in two places: the gate response and the Stage 17 evidence row that stores the verdict. The registry hash for that case and the manifest'srecordedFromare updated to match.Test plan
tests/contract/pipeline-genesisrag17.test.mjs:{ FAIL, critical, ["cross-tenant isolation is unproven: …"] };crossTenantLeaks: 2. It asserts the verdict isFAIL, the security reason names 2 leaks, and the retrieval dimension still flags it.npm test: vitest 289 passed and 2 skipped (the MSP suites needMSP_REPO_ROOT); security 12/12.npm run check:c0gives PASS 24 / NOT_RUN 1.check:baselineholds, sincepipeline.mjsis not a hashed input.check:corpusreplays all 24 cases.🤖 Generated with Claude Code