Repository navigation
Commit a7199ef
docs(qa): 18.0 sweep anchor round 15 — QA round 4's ref-accuracy findings, scored per RUNNER rule 6 (a pin scores only the layer it drives) (#24891)
Refs #24579
Clause-②: no
Round 15 of the 18.0 sweep's wave anchor (claim `6290925424`, PM loop
round 25). It applies the "Checklist-accuracy findings / fixture gaps"
section of QA round 4's Tier 1 record (#24840, framework `307ce4c4`)
under RUNNER rule 6, "A pin scores only the layer it drives"
(`RUNNER.md` rule 6, landed by PR #24811), and round 6's theme-B rule
(PR #24697).
Only checklist text changes. 21 items in 9 area files each take one
`revision` bump and one `history` row, and each row names #24840 and the
finding. There is no product code, test, pin, changeset text or
`RUNNER.md` change. `records-forms.record-discussion-mentions` is
untouched.
- Base `b1676441ba`, which is PR #24869's merge commit. `origin/main`
`98c3d00447` is merged in `c4da0e622b` and touches no `docs/qa/**` path.
- Every item and every pin named below was re-read at the base before it
was written. Rounds 12 to 14 edited several of these items after
#24840's base, and the items they settled are listed under "Items left
alone".
- A scripted span editor made every edit. It never re-serialises a file.
For each area file it re-parses the result and asserts four things: the
new document equals the old one with the declared ops applied; every
other item and top-level key is value-equal; each edited item's changed
paths are inside its declared fields plus `revision` and the one new
history row; and the text outside the edited items is byte-identical. A
whole-branch check against the base agrees: 21 items changed, each with
revision +1 and exactly one appended row.
- The new history rows carry the placeholder ref `#PENDING-24579R15`,
because a row's `ref` names this PR and the PR had no number yet. The
next commit on this branch swaps in this PR's number and changes nothing
else.
## How rule 6 was applied
- A pin scores a clause only when it drives the layer the clause's
oracle sits at: unit, driver, engine, HTTP door, boot log or CLI exit.
The layer is read off the pin's own harness, never off its file name.
- A ref that claims more is narrowed per clause (theme-B). Where no pin
holds a full clause, `automated` is removed and a `fixtures.knownGaps`
line says what the pin does assert and when `automated` may return. That
happened on 6 items.
- Item text that told a runner to score an api or log clause from a
lower pin ("Unit fallback", "score from the pin", "unit-pin verdict")
now cites that pin as evidence of its own layer only. Where the live leg
cannot be staged on stock, the clause is recorded `blocked(fixture)`
with the pin beside it. It is never scored pass.
- Negatives are read the same way. A negative whose FAIL condition is a
door's answer (a 2xx, an envelope, a row the door left) is scored at
that door. A negative whose FAIL condition is a state the pin's own
layer owns and reads directly (a spawn the provider never reaches, a DDL
change the driver's sync makes, a background task that throws) can be
scored at that layer. Three QA round 4 verdicts read differently under
this rule, and they are listed under "Items left alone".
- A missing pin was added only after reading that it asserts the named
clause's whole oracle at that clause's layer. Where it asserts half, the
item says which half instead.
## Changes
| finding (#24840) | item | field | old → new | pin read at `b1676441ba`
(test, layer) |
|:--|:--|:--|:--|:--|
| ref claims more: a2 | `cli.migrate-plan-apply-json` | `automated`
removed; `fixtures.knownGaps[0]` added | "covers acceptance[2] ONLY" →
no ref; the knownGap names the pin as unit evidence for acceptance[2] |
`packages/cli/src/utils/format.exit-code.test.ts`: emitJson / emitText
in-process (unit). acceptance[2] scores `echo $?` after each subcommand
|
| ref claims more: a4 | `cli.verify-verdict-exit-mapping` | `automated`
removed; `fixtures.knownGaps[1]` added | "covers acceptance[4] ONLY" →
no ref | `verify-tenancy-posture.test.ts`: `resolveVerifyMultiTenant`
only (unit; its docblock says it is not an end-to-end `os verify` run) |
| defective pin; missing pin a4/n2 | `cli.dev-automigrate-policy` |
`automated.ref`; `fixtures.knownGaps[2]` added | bare
`sql-driver-schema-drift.test.ts` → `sql-driver-unique-tenancy.test.ts`
for acceptance[4] and negative[2], plus
`sql-driver-schema-drift.test.ts` for negative[0] only; the production
force-off case scores nothing (knownGap) | unique-tenancy "auto-tightens
at boot under autoMigrate: safe when the probe is clean" and "BLOCKS the
tightening on duplicate NULL-organization rows — old index kept, rows
reported" (driver over better-sqlite3); schema-drift "does NOT
auto-apply destructive drift (orphan column kept; warned instead)"
(driver) |
| "409 UNIQUE_VIOLATION over real REST"; sweep-wide a1–a3 |
`records-forms.field-unique-enforcement` | `automated.ref`;
`acceptance[1..3].verify`; `steps[4]` | "SQL family, 409
UNIQUE_VIOLATION over real REST" → per-clause ref (acceptance[4] and
[5], negative[2] and [3]); "pin" / "score from the pin" / "score clause
4 from its unit pin" → driver-level evidence only, and an undrivable
updateMany door is recorded `blocked(fixture)` | both SQL pins drive
`SqlDriver` and map the refusal through a test-local `createAsApi`
helper (driver); `memory-unique-constraint.test.ts` (memory driver) |
| a2/a4 engine-only | `records-forms.encrypted-field-behavior` |
`automated.ref`; `acceptance[3].verify`; `fixtures.knownGaps[1]` | bare
two-pin ref → dogfood scores acceptance[0] (its API oracle), [1] and
negative[0]; `secret-fields.test.ts` scores no api clause. acceptance[3]
"unit-pin verdict" → `blocked(fixture)` on stock, pin beside it |
`field-zoo-roundtrip.dogfood.test.ts` (HTTP door, showcase in-process);
`secret-fields.test.ts` (engine over a stub driver) |
| sweep-wide a1 | `records-forms.cascading-multilevel-and-clear` |
`automated.ref`; `acceptance[1].verify` | objectql pin "acceptance[1],
and acceptance[2]'s unwritten child" → scores no clause; acceptance[1] →
the objectui spec's live-API case | objectui
`e2e/live/cascading-options.spec.ts` "the server rejects an out-of-set
option value, and accepts an in-set one" (HTTP door, live backend), blob
`5f65228948b7` at the console pin `616df6598b42`;
`rule-validator.option-visibility.test.ts` (unit) |
| bare ref | `integration-system.connector-stdio-default-deny` |
`automated.ref` | bare → negative[0] ONLY | `mcp-provider.test.ts`
"DENIES a declarative stdio transport by default, as a plain
(non-retryable) fault" (provider unit over a fake client factory) |
| bare ref | `integration-system.webhook-lifecycle` | `automated`
removed; `fixtures.knownGaps[2]` added | bare → no ref |
`webhook-materialization.dogfood.test.ts`: acceptance[0]'s no-row half
on its own fixture stack (engine); no activation door |
| sweep-wide a0, a3 | `integration-system.external-schema-drift-gate` |
`automated.ref`; `acceptance[0]` and `acceptance[3]` (verify, evidence);
`fixtures.knownGaps[0]` | "Unit fallback" / "Unit pins" / "the unit pins
are the fallback evidence" → plugin-layer evidence beside a
`blocked(fixture)` verdict; the ref names acceptance[4]–[6] and
negative[2] and [4] | the three runtime pins drive
`ExternalValidationPlugin` over stubbed services (plugin unit) |
| ref misses own pin | `integration-system.datasource-admin-lifecycle` |
`automated.ref` | "(+ __tests__/envelope.conformance.test.ts)" → " +
packages/services/service-datasource/src/__tests__/envelope.conformance.test.ts"
| scope unchanged |
| bare ref | `dashboards.dataset-report-authoring` | `automated.ref` |
bare → acceptance[0] ONLY |
`dashboard-designer-roundtrip.dogfood.test.ts`: draft PUT 200, publish
200 (HTTP door); the read-back checks widget count and missing layout,
not acceptance[1] |
| ref misses own pin: a4 | `dashboards.empty-null-bucket-boundaries` |
`automated.ref` | adds `empty-group-bucket-parity.test.ts` for
acceptance[4] | "keys the empty bucket identically on both paths" on
driver-sql and driver-sqlite-wasm (driver, the clause's test oracle) |
| bare ref | `platform-core.protocol-18-handshake-and-stored-replay` |
`automated.ref` | bare five pins → acceptance[0] ONLY, with each pin's
layer named for acceptance[1]–[3] |
`packages-install-protocol-incompatible.test.ts`
(`HttpDispatcher.handlePackages` over a fake kernel); artifact pin
(mocked logger); the rest in-process |
| a3 dogfood is in-process |
`platform-core.metadata-authoring-roundtrip` | `automated.ref`;
`acceptance[3].verify` | dogfood pin bare / "(pinned by …)" → scores no
clause; protocol-layer evidence for acceptance[3] |
`package-first-authoring.dogfood.test.ts`: `protocol.saveMetaItem`
in-process on a booted CRM stack, no `/api/v1/meta` door |
| sweep-wide a0–a4 | `platform-core.settings-audit-secret-fingerprint` |
`automated.ref` | seven cases listed with no clause → acceptance[5] and
negative[2]–[3] ONLY | `settings-audit-secret-digest.test.ts`:
`SettingsService` over stub audit sinks (service unit) |
| a5 is the enumSource gate | `platform-core.lifecycle-retention-sweep`
| `automated.ref`; `acceptance[5].verify` | feature list → per clause;
"the enumSource pin below" → the enumSource variants-freshness ratchet
of `check:platform-checklist`, which counts `LifecycleClassSchema` only
| `lifecycle-service.test.ts` over a fake engine |
| Tier 1 in name only | `automation.flow-node-type-matrix` | `automated`
removed; `fixtures.knownGaps[2]` added | "covers ONE variant of
acceptance[1] only" → no ref | `flow-node.dogfood.test.ts`: one
update_record node |
| ref misses own pin: n0 | `automation.trigger-type-matrix` |
`negative[0]` | "(pinned by …)" → "its 401 half is pinned …; the
no-run-row half is driven by hand"; ref unchanged |
`showcase-anonymous-deny-surfaces.dogfood.test.ts` "anonymous POST
/automation/:name/trigger is denied (401)" (HTTP door), which reads no
run rows |
| ref disclaims the wired lane |
`attachments-storage.field-accept-maxsize-server-enforced` | `automated`
removed; `fixtures.knownGaps[3]` added | handler-direct ref for clauses
0/1/3/4/5/6 → no ref | `file-reference-lifecycle.test.ts` "accept /
maxSize enforcement" (handler-direct over a fake engine), round 12's
update-path case included |
| ref misses own pin: a5/n3 | `identity-auth.admin-lifecycle-operations`
| `automated.ref` | adds the plugin-security case: negative[3] whole,
acceptance[5]'s exemption half | `security-plugin.test.ts` "[#3023] an
engine referential FK clear (__referentialFieldClear) is exempt —
owner_id:null cascade is allowed for a plain member" (plugin unit) |
| sweep-wide a9; pin also asserts n1 |
`approvals.decision-action-matrix` | `automated` removed;
`fixtures.knownGaps[0]` rewritten, `[1]` added; `acceptance[9].verify` |
"acceptance[9] ONLY" and "that pin is how acceptance[9] is scored" →
`blocked(fixture)` on stock, two lower pins beside it; negative[1] joins
no ref | `approval-service.test.ts` #12769 describe and its remind
THROTTLED cases (service over a fake engine);
`rest-approvals-wire-codes.test.ts` (route handler over a mocked
service) |
### The defective pin, for the PM to file
`packages/drivers/driver-sql/src/sql-driver-schema-drift.test.ts`, the
case "is force-disabled under NODE_ENV=production (warns, does not
alter)", stages `relax_not_null`. ADR-0113 made that op `needs_confirm`,
and a safe reconcile applies only `safe` ops. So the column stays NOT
NULL with or without the production branch of `reconcileAndWarnDrift`
(`packages/drivers/driver-sql/src/sql-driver.ts`). The sibling case
"does NOT auto-relax a stray NOT NULL — needs_confirm survives dev
auto-reconcile" asserts the same outcome with no `NODE_ENV` set. The
case also asserts no warning. #24840 names the repair: stage
`replace_unique_index` and assert the warning. This PR does not touch
the test. The ledger now scores nothing from that case and records it in
the item's `knownGaps[2]`.
## Items left alone, with reasons
- `records-forms.list-view-capabilities` (Tier 1 in name only): no
longer holds. Round 14 (PR #24857) removed the stale marker, and the ref
claims acceptance[1] from objectui `saved-view-filter.spec.ts` (blob
`94a201c63a0a` at `616df6598b42`). That spec opens the shipped
`in_progress` view on the live stack, asserts the count bar reads 2, and
asserts that no data request was refused. A rule-array filter is refused
400, so the green is the AST filter being accepted.
- `records-forms.related-list-server-pagination` ("its only pin is
stale"): no longer holds. Round 14 removed the stale marker after
reading the repaired spec, and the ref claims acceptance[2] only.
- `attachments-storage.field-accept-maxsize-server-enforced`: the
finding still holds, so it was applied above. Round 12 added an
acceptance[3] case at the same handler-direct layer, which did not
settle it.
- `automation.trigger-type-matrix` ref: the dogfood case asserts the 401
but not "with NO run row created", so it is not added to the ref.
negative[0] says which half it pins.
- `approvals.decision-action-matrix` negative[1]: not added. The service
pin throws THROTTLED and the route pin maps a mocked rejection to 429,
but neither drives the remind door. "A silent second success" is that
door's answer.
- `identity-auth.admin-lifecycle-operations` acceptance[5]: the added
case is its exemption half. Its refusal half stays partly by hand,
because `admin-platform-admin-standing.dogfood.test.ts` re-reads the
target `sys_user` but never the rows that user owns.
- `records-forms.field-unique-enforcement` acceptance[4]'s clause still
says "the same 409 UNIQUE_VIOLATION over REST". Its oracle is `test`,
and the ref now says that wording rests on the `createAsApi` mapping.
Clause text was not changed in this round.
- `integration-system.datasource-admin-lifecycle`: only respelled, as
#24840 asked. Its ref's scope ("pins route behavior + envelope") is not
narrowed per clause.
- `platform-core.metadata-authoring-roundtrip` keeps `automated.kind`
"api", although its remaining full-clause pin is an objectui live spec.
`kind` is free text that no gate reads.
- QA round 4 verdicts this round reads differently. The record scored
them pass from lower pins: `encrypted-field-behavior` acceptance[3] and
negative[1] (engine), `settings-audit-secret-fingerprint`
negative[0]–[1] (rows on stub sinks), and
`approvals.decision-action-matrix` negative[1] (service). Under the rule
above, each is now evidence beside a hand-driven or `blocked(fixture)`
verdict.
## PM hypotheses, measured
1. "Rounds 12–14 already settled list-view-capabilities,
related-list-server-pagination and field-accept-maxsize's a3." This
holds for the first two. It does not hold for the third: round 12's case
is handler-direct, so the item's ref still scores no clause, and the
finding was applied.
2. "Most bare-ref rows narrow rather than drop." This holds 3 of 4.
connector-stdio, dataset-report and protocol-18 narrow.
webhook-lifecycle drops, because its one pin holds only half of
acceptance[0].
## Acceptance notes
- Located beyond #24840's list, inside items it names, and applied
because they are the same rule-6 class:
- `encrypted-field-behavior` acceptance[3] was a "unit-pin verdict". It
is now `blocked(fixture)` on stock.
- `approvals.decision-action-matrix` acceptance[9].verify said
`READ_BACK_FAILED` rides the generic template-generated 500 and is
deliberately absent from `handleApprovalError`'s prefix map. It has been
a named 500 row there since `5b3ff63cc`
(`packages/rest/src/approvals-routes.ts`), and
`rest-approvals-wire-codes.test.ts` pins it. The verify now says so,
because the rewritten knownGap would otherwise contradict it.
- The same "score from the pin" spelling appears outside this round's
file surface. These items were not edited, and the anchor carries them.
Likely rule-6 contradictions, where an api, log or network clause is
scored from a lower pin:
- `api-backend.packaged-action-disabled-dispatch` knownGaps[0] (the MCP
clause's unit fallback);
- `automation.packaged-flow-subflow-disable-refusal` knownGaps[0]
(engine-pinned arms of api clauses);
- `cli.datasource-introspect-codegen` acceptance[0].verify (a log
clause's half);
- `integration-system.connector-auth-kind-application`
acceptance[0].verify and knownGaps[0] (wire-shape clauses);
- `approvals.email-action-token-door` knownGaps[0] and
`approvals.act-door-and-admin-rescue-slate` requires[1] (legs scored
from pins; their clause oracles are mixed).
- These read as legitimate, because the clause they score is
`test`-oracle: `access-security.activation-write-operator-gate`,
`identity-auth.implicit-account-linking-ownership`,
`records-forms.delete-behavior-matrix` and
`records-forms.field-history-tracking`.
- The objectui specs cited here were read at the console pin
`616df6598b42`. No live run exists at that pin, so their verdicts there
are NOT MEASURED. objectui measured the live suite green at `0b36ffa`
(61 passed), and the blobs are identical at both commits.
- Whether some other dogfood suite asserts one of
`approvals.decision-action-matrix`'s clauses whole, on a booted
showcase, was not measured. The approval dogfood suites in the tree pin
other items' fixture stacks.
## Gates
The derived list is `node scripts/pm/dispatch-gates.mjs --commands
--repo objectstack-ai/objectstack`, computed on the actual change set: 9
paths against merge base `98c3d0044`, at head `c4da0e622b`. It gives 13
commands. Each one ran at that head with its exit code captured before
any pipe, and each exited 0:
`node scripts/check-ci-filter-parity.mjs` · `node
scripts/check-closing-keyword-parity.mjs` · `node
scripts/check-closing-keyword-parity.mjs --self-test` · `node
scripts/check-comment-mask-corpus.mjs` · `pnpm --filter
@objectstack/lint run check:doc-formula-expressions` · `pnpm
check:cross-package-test-inputs` · `pnpm check:doc-authoring` · `pnpm
check:driver-memory-census` · `pnpm check:gitlink-declared` · `pnpm
check:nul-bytes` · `pnpm check:platform-checklist` · `pnpm
check:refd-timer-probe` · `pnpm check:watch-hint-literal`
The first run of `check:doc-formula-expressions` refused with exit 3, a
prerequisite, because `@objectstack/formula` and `@objectstack/lint`
were not built. Both were built under the shared verify lock (`VERDICT
command-exit 0`), and the rerun exited 0. The `--ran` reconciliation
reads "13 derived, 13 run, 0 NOT-MEASURED, 0 UNRUN".
No per-PR CI job runs `pnpm check:platform-checklist`, so this local
reading is its evidence:
| reading | base `b1676441ba` | head `c4da0e622b` |
|:--|:--|:--|
| items | 16 areas, 328 items (323 active, 1 planned) | 16 areas, 328
items (323 active, 1 planned) |
| line citations | 0 survive across 21 swept documents | 0 survive
across 21 swept documents |
| symbol anchors | 965/975 against 430 cited sources, 10 on the named
residual (#16898), 18 file floors held | 965/975 against 430 cited
sources, 10 on the named residual (#16898), 18 file floors held |
PR #24869 merged as `b1676441ba`, this branch's base. `git merge-tree
--write-tree` of this head against its head `45f8e20bca` still exited 0.
There is no ablation, because no tested behaviour changed.
`skip-changeset` was re-measured: `docs/qa/**` appears in none of the 86
package manifests' `files[]`, and the repo root is private.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01FM29ZqLTrnPNXxijioZwMp)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent f3f27be commit a7199ef
9 files changed
Lines changed: 176 additions & 76 deletions
File tree
- docs/qa/platform-checklist/areas
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
417 | 417 | | |
418 | 418 | | |
419 | 419 | | |
420 | | - | |
| 420 | + | |
421 | 421 | | |
422 | 422 | | |
423 | 423 | | |
| |||
428 | 428 | | |
429 | 429 | | |
430 | 430 | | |
431 | | - | |
| 431 | + | |
| 432 | + | |
432 | 433 | | |
433 | 434 | | |
434 | 435 | | |
| |||
504 | 505 | | |
505 | 506 | | |
506 | 507 | | |
507 | | - | |
| 508 | + | |
508 | 509 | | |
509 | 510 | | |
510 | 511 | | |
| |||
514 | 515 | | |
515 | 516 | | |
516 | 517 | | |
517 | | - | |
518 | | - | |
519 | | - | |
520 | | - | |
521 | 518 | | |
522 | 519 | | |
523 | 520 | | |
| |||
539 | 536 | | |
540 | 537 | | |
541 | 538 | | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
542 | 545 | | |
543 | 546 | | |
544 | 547 | | |
| |||
0 commit comments