qa(checklist): clauses for three round-3 defects outside the ledger, plus two RUNNER rules — a pin scores only the layer it drives, and an off-clause probe budget

docs/qa/platform-checklist/RUNNER.md

@@ -110,10 +110,49 @@ selector handed you has a verdict; planned ids are listed, not chased.
    and cite its output as the evidence; drive the browser only for what the pin doesn't
    cover. The reverse also holds: when a sweep hand-proves something repeatedly,
    propose promoting it to a permanent test and set `automated` in a revision.
+
+   **A pin scores only the layer it drives.** Before a pin scores a clause whose oracle
+   is a public door's answer (an HTTP status, an error envelope, a row the door left
+   behind), name the layer the pin reaches — a **plugin unit** (one plugin's hooks or
+   middleware over a stub), the **engine** (a real ObjectQL engine and driver, no HTTP),
+   or **HTTP** (a booted server answering the door) — and write that layer in the
+   clause's evidence. Read it off the pin's own harness, never off its file name or its
+   `automated.ref` prose: a stub terminal standing in for the engine means "lands" is
+   "the middleware called `next()`", and nothing below it ran. A pin that stops above
+   the layer the clause's oracle sits at scores that clause **`partial`**, naming the
+   missing layer, and the live leg is driven, or recorded `blocked` with its fixture
+   cited. It never scores `pass`. The instance that made this a rule is
+   `access-security.no-active-org-session-semantics`: the `group` CONTROL of
+   `no-active-organization-write-refusal.test.ts` "lands" a write by a member with no
+   active organization at the plugin-security layer, two rounds scored the walled legs
+   from it while a walled boot was refused, and on a full walled `group` boot the engine
+   answers that same write with a 500 (#24784).
 7. **Verify pass for high-stakes claims.** For P0 `fail`s and any finding that would
    ship or block a release: a second, independent agent re-derives the verdict from the
    captured evidence alone (not from the first agent's narrative) before it is acted
    on. Disagreement → re-run the item.
+8. **Keep an off-clause probe budget.** Clauses only ask what someone already thought
+   to ask. Every run keeps time AFTER its clauses have verdicts for probing the doors its
+   selector covers past those clauses — on the same boot, as the same personas. Probe
+   where a clause stops:
+   - **boundary values** on the inputs the clauses drive: the empty string, a missing
+     id, an unknown id;
+   - **the opposite severity or posture** of a path a clause asserts: the `warning`
+     twin of an `error` rule, `group` beside `isolated`;
+   - **recipients or readers outside the asserted scope**: who else a notification,
+     share or read reaches.
+
+   A probe is not a clause, so it scores nothing and no item verdict moves on it. A
+   finding that passes the filing gate is extracted like any other product defect (see
+   "Extraction obligation" below), and its card carries
+   `QA-source: #<run record> · <area.item> · observed beside the clauses`, naming the
+   item whose doors were probed. A probe that finds a gap the ledger should have asked
+   about is also a checklist-accuracy finding for the wave's anchor card: the clause it
+   was missing. The run record states how long the probes took and what they covered
+   (doors, personas, the boundary classes tried), and says so when the budget was
+   spent on nothing. A record that omits the probe line reads as a run that skipped it.
+   Round 3 of the 18.0 sweep (QA run #24779) found #24781 and #24782 this way, beside
+   items whose clauses all passed.
 
 ### Environment facts the runner should not re-derive
 
@@ -717,7 +756,8 @@ The in-environment JSON scratch (RUNNER shape, never committed):
 
 The issue body is: env fingerprint · scope (selector + per-item `revision`) · the
 per-clause verdict table (text oracle evidence) · a reproduction rule per `fail` (rule 2's
-carve-out excepted) · derived item verdicts + fixture gaps. The durable,
+carve-out excepted) · derived item verdicts + fixture gaps · the off-clause probe line
+(rule 8: how long, which doors and boundary classes, what it found). The durable,
 version-controlled truth is still the checklist under `areas/`; a run is a dated assertion
 about one build, and it lives in its issue, not the tree.
 

docs/qa/platform-checklist/areas/access-security.json

@@ -2352,7 +2352,7 @@
       "title": "A session with no active organization is a LEGAL state with fail-closed semantics: tenant-scoped reads resolve to nothing, tenant-scoped writes are refused loudly naming the missing org, and nothing is ever NULL-stamped",
       "since": "v17",
       "status": "active",
-      "revision": 6,
+      "revision": 7,
       "priority": "P0",
       "surface": "api",
       "personas": [
@@ -2371,13 +2371,14 @@
           "`invite-only` is no longer a fixture gap: the walled boot must declare a membership policy anyway (RUNNER.md 'A walled boot is a fixture of its own', prerequisite 3), and declaring `invite-only` gives knownGaps[0]'s producer (1). SSO-JIT-pending still has no stock fixture: that producer is blocked(fixture) unless the run configures SSO.",
           "requires[0]'s tenancy-walled deployment is a walled boot. On 18.0.0 os serve — and os dev / os start, which boot through it — boots OS_TENANCY_POSTURE=group|isolated (and the legacy OS_MULTI_ORG_ENABLED=true): 18.x step 4 (#24580) deleted the walled boot refusal (serve.ts walledPostureRefusal) before 18.0.0, and of the published prereleases only 18.0.0-next.5 carries it. The walled boot is provisioned per RUNNER.md's environment-facts entry 'A walled boot is a fixture of its own': the multi-org runtime declared by the app (no example app declares it), OS_PLATFORM_OWNER_EMAIL, and a declared OS_AUTH_MEMBERSHIP_POLICY; each one absent refuses an os dev boot naming itself, while a bare os serve in production mode judges none of them until OS_AUTH_SECRET is set (that entry). The item stays blocked (fixture, #9334) because every clause is a walled leg and the stock showcase provisions no walled boot, not for want of a producer (knownGaps[0]); its pin (automated.ref) boots the posture in-process.",
           "acceptance[4] has an open question for a user who holds NO membership at all (producer (1) of knownGaps[0]): its login/logout rows in sys_audit_log carry tenant_id null, because no organization exists to stamp and ADR-0123 D3's ordering fix 'must not manufacture a membership that policy says must not exist'. QA run #24783 read those rows only as the platform operator (GET /api/v1/data/sys_audit_log filtered by user_id answered 200), while the user itself gets 403 on sys_audit_log, on both postures. ADR-0123 does not state whether that is the intended answer: D3 forbids a NULL-tenanted row under this state 'in any subsystem' and pays the audit ledger by ordering, which binds nobody under invite-only. The clause stands as written. Until the question is ruled (raised by QA run #24783's finding F3 for wave anchor #24579; no card carries it yet), score acceptance[4] on that user from what is measured (the raw rows, their tenant_id and the readers that see them) and name this question beside the verdict",
-          "On `group`, producer (2), a member with no active organization, gets 500 {\"code\": \"ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED\"} on a tenant-scoped create, where `isolated` and a member-less caller answer the acceptance[1] 403 naming the missing organization; nothing is written (QA run #24783: acceptance[1], acceptance[5], acceptance[6] and negative[3] FAIL on `group` only). That is the open product card #24784. The clauses stand as written: score them against it until it lands, and score the member-less caller's write separately"
+          "On `group`, producer (2), a member with no active organization, gets 500 {\"code\": \"ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED\"} on a tenant-scoped create, where `isolated` and a member-less caller answer the acceptance[1] 403 naming the missing organization; nothing is written (QA run #24783: acceptance[1], acceptance[5], acceptance[6] and negative[3] FAIL on `group` only). That is the open product card #24784. The clauses stand as written: score them against it until it lands, and score the member-less caller's write separately",
+          "acceptance[7] (the walled `group` leg for a member with no active organization) stays red until #24784 lands: on a walled group boot at 580fd4ed (QA run #24783, re-derived under RUNNER rule 7) the create answers 500 ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED and writes no row. It is scored only on a live walled group boot; a run without one records it blocked(fixture) citing RUNNER.md's 'A walled boot is a fixture of its own' — ⛔ never from automated.ref, whose group CONTROL stops at the plugin-security layer (RUNNER rule 6, a pin scores only the layer it drives). When #24784's fix merges, re-run the clause and drop this line"
         ]
       },
       "blocked": { "by": "fixture", "ref": "#9334" },
       "automated": {
         "kind": "unit",
-        "ref": "packages/plugins/plugin-security/src/no-active-organization-write-refusal.test.ts — pins every clause of this item at unit level, INCLUDING a control asserting that the `single` posture has no wall, i.e. the test itself agrees the stock showcase configuration makes the item vacuous"
+        "ref": "packages/plugins/plugin-security/src/no-active-organization-write-refusal.test.ts — pins every clause of this item at unit level, INCLUDING a control asserting that the `single` posture has no wall, i.e. the test itself agrees the stock showcase configuration makes the item vacuous. The layer it drives is a plugin unit: SecurityPlugin's middleware over a stub terminal that stands in for the engine, so a case that 'lands' there means the middleware called next(), and the engine's own insert path never runs. Its `group` describe's CONTROL 'a NON-EMPTY membership set lands, even with no active organization' therefore proves the plugin-security wall lets that write through and nothing past it: on a full walled group boot the engine refuses the same write with 500 ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED (#24784). Under RUNNER rule 6 the pin scores a clause whose oracle is the HTTP answer partial at most, naming the engine and HTTP layers as missing, and it cannot score acceptance[7] at all"
       },
       "steps": [
         "establish the state and PROVE it: obtain a session whose `activeOrganizationId` is null, and cite the evidence (session row / session endpoint payload) — do not infer the state from the symptom you are about to test",
@@ -2387,7 +2388,8 @@
         "query the underlying table as admin for rows whose tenant column is NULL, before and after the write attempt",
         "repeat the read and the write against a NON-tenant-scoped object to bound the blast radius",
         "sign in and out with the state in force, then inspect the audit ledger rows the session produced (plugin-audit auth-session-audit login/logout)",
-        "run the same read+write as the control admin persona to show the refusal is state-derived, not a blanket denial"
+        "run the same read+write as the control admin persona to show the refusal is state-derived, not a blanket denial",
+        "walled `group` leg (acceptance[7]), on a walled boot with OS_TENANCY_POSTURE=group provisioned per RUNNER.md's 'A walled boot is a fixture of its own': a member of organization A (non-empty membership set) sets A active with POST /api/v1/auth/organization/set-active, creates a showcase_account and a showcase_project in A, then clears it with POST /api/v1/auth/organization/set-active {organizationId: null} and proves the state (GET /api/v1/auth/get-session reads session.activeOrganizationId null). As that member, POST /api/v1/data/showcase_task {title, project: the project in A}; take the admin-side count of showcase_task rows whose organization_id is NULL or empty before and after, and read back any row the create produced"
       ],
       "acceptance": [
         {
@@ -2431,6 +2433,12 @@
           "oracle": "api",
           "verify": "the run record states each subsystem's observed behavior side by side; any disagreement is the finding, whichever way it leans",
           "evidence": "the three-way comparison table"
+        },
+        {
+          "clause": "walled `group` leg: a member whose membership set is NON-EMPTY and whose session has no active organization never gets a 5xx on a tenant-scoped create, and nothing is stamped with a NULL organization_id. Which answer `group` gives — the write lands in a member organization, or a 4xx refusal — is #24784's to decide; this clause asserts only those two facts. Red until #24784 lands",
+          "oracle": "api",
+          "verify": "step-9 create on the live walled group boot: status is NOT 5xx (500 ERR_SYSTEM_WRITE_ORGANIZATION_REQUIRED, logged '[REST] Unhandled error: SystemWriteOrganizationRequiredError', is the #24784 shape); the admin-side NULL-or-empty organization_id count for showcase_task is identical before and after; and if the create answered 2xx, the row it produced carries a real organization_id the member belongs to. Score it only from that live boot: the automated.ref pin's group CONTROL stops above the engine (RUNNER rule 6), so it cannot score this clause",
+          "evidence": "the get-session payload + the create response, quoted + both NULL counts + the produced row's organization_id when the create landed"
         }
       ],
       "negative": [
@@ -2453,7 +2461,8 @@
         "PR #8471 (ADR-0123 + enforcement)",
         "@objectstack/plugin-security security-plugin.ts step 3.7 + tenant-layer.ts (computeTenantLayer0Filter, RLS_DENY_FILTER)",
         "@objectstack/plugin-auth auth-manager.ts composeDatabaseHooks + reconcile-membership.ts",
-        "@objectstack/plugin-audit auth-session-audit.ts (login/logout rows)"
+        "@objectstack/plugin-audit auth-session-audit.ts (login/logout rows)",
+        "QA-source: #24783 · access-security.no-active-org-session-semantics · acceptance[1] (the walled group leg; #24784)"
       ],
       "history": [
         {
@@ -2486,6 +2495,12 @@
           "date": "2026-10-20",
           "change": "producers re-measured and acceptance[0] aligned with its governing text — QA run #24783 (finding F3), round 11 of wave anchor #24579. personas[0]–[1], fixtures.requires[1] and knownGaps[0]–[1] now name the two durable producers on a walled boot (a user the operator creates under invite-only, membershipCreated:false; a member that sets its active organization to null). The signup race is closed by ADR-0123 D3's session-seam settle, and the removed-member path revokes or re-points the session (QA run #24783 measured 401). acceptance[0]'s clause and verify scope 'resolve to nothing' to a caller with no organization scope, which ADR-0123 D2 defines under `group` as an empty membership set, ADR-0105 D2 letting a `group` member's reads span its memberships. acceptance[4] is unchanged: ADR-0123 states no answer for a member-less user's NULL-tenanted audit rows, so knownGaps[3] names the question. knownGaps[4] names #24784, the `group` write 500. knownGaps[2]: the prerequisite sentence holds through os dev only (a bare production os serve needs OS_AUTH_SECRET first, RUNNER.md), and the blocked marker's reason is restated (every clause is a walled leg) because a producer is no longer missing; the marker itself is unchanged",
           "ref": "#24801"
+        },
+        {
+          "revision": 7,
+          "date": "2026-10-20",
+          "change": "acceptance[7], steps[8] and knownGaps[5] added, and automated.ref now names the layer its pin drives — for #24784, which the walled pass of round 3 (QA run #24783, 580fd4ed) found. Rounds 1 and 2 scored the walled legs from the pin because a walled boot was refused then; the pin's group CONTROL 'lands' a write by a member with no active organization at the plugin-security layer, and on a full walled group boot the engine refuses that write with a 500. The new clause asserts only no 5xx and no NULL stamping (which answer group gives is #24784's to decide) and stays red until #24784's fix lands. Merged onto revision 6 (round 11 of wave anchor #24579, #24801), whose knownGaps[0] producer (2) is the one steps[8] drives and whose knownGaps[4] names the same card for acceptance[1], [5], [6] and negative[3]; knownGaps[5] adds only the new clause's red state and the rule that it is never scored from the pin",
+          "ref": "#24802"
         }
       ]
     },