Repository navigation
Commit 0fe0a59
Fixes #21828
Clause-②: no
## What was wrong
`DatabaseLoader.save` stamped `sys_metadata.checksum` with
`calculateChecksum(data)`, which sorts every map before hashing, and
skipped its write whenever that stamp equalled the stored one. An object
whose only change was the order of its `fields` hashed equal, so
`MetadataManager.register` refreshed the loader's cache with the new
order while the persisted row kept the old one. This is the class of
defect #21790 repaired on the designer's path, here on the `register`
path.
Reproduced at `8832655af2` before the change (scratch probe, not
committed): a `DatabaseLoader.save` of `{ fields: { amount, title } }`
over a row holding `{ fields: { title, amount } }` left version 1, one
history row, and stored order `title,amount`.
## The fix
Landing: `packages/metadata/src/loaders/database-loader.ts`, as the
claim predicted. The producer of the stamp is the loader itself, so
nothing moved to another package.
- **One stamp.** Every loader write (`save` create and update,
`registerRollback`) stamps `contentHash(json, type)`, which is
`hashSpec(JSON.parse(json), type)` from `@objectstack/metadata-core`.
That is the hash `SysMetadataRepository` stamps on the same column, so
the column carries one vocabulary (`sha256:` + 64 hex). It hashes the
JSON the loader stores, so a value JSON cannot carry (a function, an
`undefined` property) is dropped as the stored column drops it, and
`hashSpec` is never handed what it refuses.
- **One comparison, about content.** `save` decides "unchanged" by
re-hashing the STORED body under `hashSpec(body, type)`
(`storedBodyUnchanged`), the comparison `SysMetadataRepository.put`
makes for an ordered-map type. A stored stamp cannot answer it, because
it may come from an older rule (H3).
- **The history write no longer compares stamps.** `createHistoryRecord`
takes the parent row's new stamp from its caller, so the history row
carries the same value. Its own `'update'` comparison (new stamp vs
previous stored stamp) is removed. Its only `'update'` caller is `save`,
which reaches it after the content comparison. A second, stamp-based
comparison can only be wrong: against an order-blind stamp, a reorder
into sorted key order hashes equal and would get no history row
(ablation C below).
- ⛔ No second order-aware checksum, no order key: the map already holds
the order, per triage.
- `calculateChecksum` keeps its behaviour and its export (H4). Its
docblock now says it is not the `sys_metadata.checksum` stamp.
No `packages/spec/src/**` edit, no export added, removed or changed. The
built `dist/index.d.ts` differs only in doc comments:
`DatabaseLoader.createHistoryRecord` is private and emitted without a
signature, and `contentHash` and `storedBodyUnchanged` are
module-private. So no contract review is owed beyond what the `Clause-②:
no` line states.
## H1 — every checksum decision in the loader (at `8832655af2`)
| Site | Computes, compares or exposes | On equality | After this PR |
|---|---|---|---|
| `save` `:1404`, `:1413`–`:1414` | `calculateChecksum(data)` vs the
stored `checksum` | No write; cache set to the new body; success |
`contentHash` stamp; content comparison via `storedBodyUnchanged` |
| `save` `:1433` / `:1466` | Stamps the row on update / create | n/a |
`contentHash` stamp |
| `createHistoryRecord` `:725`, `:728` | `calculateChecksum(metadata)`
vs `previousChecksum`, for `'update'` only | No history row | Takes the
caller's stamp; no comparison |
| `createHistoryRecord` `:782`, `:799`–`:800` | Writes `checksum` /
`previous_checksum` to the history row | n/a | Same columns, same value
as the parent row |
| `registerRollback` `:1356`, `:1366`, `:1372` |
`calculateChecksum(restoredData)`; reads the stored stamp as previous;
stamps the row | No comparison: always writes and appends one `revert`
row | `contentHash` stamp; still no comparison |
| `rowToRecord` `:874` → `load()` `:1002`, `stat()` `:1145` | Exposes
the stored stamp as `etag` | Nothing compares it in the loader (`load`
reads no `ifNoneMatch`) | Unchanged; a row written from now on reports
`sha256:` |
| `getHistoryRecord` `:1217`, `queryHistory` `:1322` | Expose history
`checksum` / `previous_checksum` | `MetadataManager.diff` passes them
through as `checksum1` / `checksum2` and decides `identical` from the
patch | Unchanged |
## H2 — two vocabularies in one column, measured
Scratch probe in `packages/metadata-protocol` (deleted, never
committed): one in-memory engine shared by a `DatabaseLoader` and a
`SysMetadataRepository`, each writer meeting the other's stamp on an
unchanged body.
| Case | At `8832655af2` | After this PR (`9e2b5ea54d`) |
|---|---|---|
| Loader `save` over a repository-stamped `view` | Rewritten: version 1
to 2, stamp `sha256:2d77…` to bare `2d77…`, `update` history row | No
write, version 1, 1 history row |
| Loader `save` over a repository-stamped `object` | Rewritten: version
1 to 2, `update` history row | No write |
| Repository `put` over a loader-stamped `view` | Rewritten: version 1
to 2, bare to `sha256:`, `update` history row | No write: both writers
stamp `sha256:2d77…` |
| Repository `put` over a loader-stamped `object` | No write: the
repository re-hashes the stored body for an ordered-map type | No write:
both stamp `sha256:3c7d…` |
So before this PR each writer's first unchanged write over the other's
row was a phantom version bump with a history row recording no change.
The view's digest was the same hex in both writers, `sha256:` prefix
aside, because a type with no ordered map canonicalizes to the same
sorted JSON.
## H3 — the upgrade without phantom bumps
After the change a row the loader stamped before this release carries a
bare-hex stamp that no `sha256:` hash equals. Comparing stamps would
therefore rewrite every such row on the first `register` after upgrade,
with a version bump and an `update` history row. Ablation B measures
exactly that. So `save` compares content: a legacy-stamped row with an
unchanged body is not rewritten and keeps its stamp until its content
next changes. A reorder of `object.fields` is still a change, including
a reorder into sorted key order against an order-blind stamp.
## H4, H5
- **H4.** `calculateChecksum` (`index.ts:45`) is unchanged and still
exported. Nothing in this repository calls it any more except the new
test, which uses it to produce a legacy stamp.
- **H5.** Readers of a stored loader stamp: the two loader ETags and the
two history readers in H1, `MetadataManager.diff` (pass-through), and
`SysMetadataRepository`, which reads the stamp as its version token:
`rowToItem`, the optimistic lock, and the `storedBodyUnchanged`
short-circuit (H2). The REST metadata ETag is computed from the served
body (`protocol.ts` `getMetaItemCached`), not from the stored stamp.
`protocol.ts` `migrate-stored` hands the stored stamp back as
`storedParentVersion`, where it is compared with the same stored stamp.
Outside the runtime, the offline probe
`packages/objectql/scripts/dry-run-hash-compat.ts` compares the stored
stamp with a type-blind `hashSpec(body)`; see Acceptance notes.
## Pins
New file
`packages/metadata/src/loaders/database-loader-21828-one-content-hash.test.ts`:
real SQLite (`driver-sqlite-wasm`), through `MetadataManager.register`,
read back from the rows.
- §A a field-reorder-only `register` persists the new order (version 2,
`create` + `update` history); the row and its history row carry
`hashSpec(body, 'object')`, one value; a rollback restores the earlier
order in the same vocabulary.
- §B an unordered map's key swap compares equal: no write and no history
row, for an `object` (keys around and inside `fields`) and a `view`.
- §C the upgrade: a row stamped by the old `calculateChecksum` with an
unchanged body is not rewritten (object and view); a reorder into sorted
key order against an order-blind `sha256:` stamp is persisted.
## Reverse verification
Committed first (`9e2b5ea54d`). Each mutation went through `node
scripts/ablation-replace.mjs` in WRAP mode, with a `trap … EXIT INT
TERM` restore in the driver script. Each landing was proved on disk
(anchor count 1 to 0, blob `4da1f0736df3` changed), and each restore was
proved with blob equal to HEAD and an empty `git diff HEAD`. The loader
test imports `./database-loader.js` from source, so no build was
involved.
| Ablation | Mutation | Red | Green |
|---|---|---|---|
| A | `save` restored to the pre-fix key-sorted `calculateChecksum`
stamp and stamp comparison | §A reorder persists, §A one-vocabulary
stamp, §C reorder against an order-blind stamp (3) | §A rollback, both
§B key-swap pins, §C legacy unchanged ×2 (5) |
| B | New stamp, but `newChecksum === previousChecksum` instead of the
content comparison | all three §C upgrade pins (3) | §A ×3, §B ×2 (5) |
| C | The history write's stamp comparison restored | §C reorder against
an order-blind stamp (history 1, not 2) | the other 7 |
Control on the unmutated HEAD, run after each driver pass: 8/8 green.
The first attempt at C was a no-op: its replacement still contained its
anchor, so `ablation-replace` refused (anchor count 1 to 1) and
restored. It was re-anchored on `const historyId = generateId();` and
re-run. The table records the re-run.
## Tests
At `cf0537aa08` (the final commit, `origin/main` `2799155678` merged;
the merge touched only `packages/spec` test files):
- `pnpm --filter @objectstack/metadata typecheck`: exit 0. The new test
is in the program: `tsc --noEmit --listFiles` counts it once.
- `pnpm --filter @objectstack/metadata exec vitest run --maxWorkers=2`:
58 files, 867 tests passed.
- Consumers: no public surface moved (no export, no `.d.ts` signature,
no spec), so per the dispatch contract only the package's own tests are
owed. No other package's test exercises `DatabaseLoader` writes or
stamps: 10 files reference it, with zero
`save`/`register`/`checksum`/`etag` hits.
## Gates
`node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` (no paths) at `cf0537aa08`: 62 derived
commands, the same 62 as at `235e68165e`. All 62 were run at
`cf0537aa08`, every one exit 0, and `--ran` reconciles: "62 derived
famil(ies) accounted for — 62 run, 0 NOT-MEASURED".
- Beyond the dispatch list, the change set added:
`check-adr-0087-registration` ×2, `check-empty-changeset` ×2,
`release-rehearsal-clone --self-test`, `release-pending-publish
--self-test`, `check:engine-double-contract`,
`check:objectql-double-limit`, `check:objectui-changeset`,
`check:pm-changeset-deadline-census`, `check:query-options-erasure`,
`check:type-check-coverage`, `check:type-check-debt`,
`check:where-matcher`. All exit 0.
- Artifact-roster block (55 printed outside the total), at `cf0537aa08`:
52 exit 0. `check-closing-target-claim`, `check-partof-closing-keyword`
and `check-single-claim-paths` answered NOT WIRED (no pull-request
context). They are re-run against this PR in the report on the card.
- The four symbol-anchor sweeps: `check:adr-symbol-anchors`,
`check:scripts-symbol-anchors`, `check:spec-docblock-symbol-anchors`,
`check:adr-anchors`, all exit 0.
- `check:engine-double-contract` asked for no pin row (no new double).
- Lint, narrowed and declared. `pnpm exec eslint --no-inline-config
--format json` on the 3 changed `.ts` files at `235e68165e` (identical
bytes at `cf0537aa08`) linted 3 files, with 0 errors and 0 warnings.
Population: all three match the config's
`packages/**/*.{ts,tsx,mts,cts}` and `**/*.{ts,tsx,mts,cts}` objects.
The changeset is in no lint glob. Invariance: every config object sets
only `ecmaVersion` / `sourceType`, with no `parserOptions.project` and
no typed rules, so this diff cannot move the verdict on any untouched
file. The full `pnpm lint` is CI's.
## Docs
No hand-written page states the loader's checksum rule.
`content/docs/concepts/metadata-lifecycle.mdx` states the `sha256:` rule
for the repository's `put()`, and it is now also true of new loader
writes. No doc edit.
## Acceptance notes
- **Residual on the repository side, for rows stamped before this
release.** `SysMetadataRepository.put` answers "unchanged" for a type
with no ordered map by comparing stamps. So an unchanged-body `put` over
a row the loader stamped before this release (bare hex) still rewrites
it once, with a version bump and an `update` history row (H2, row 3, at
`8832655af2`). New loader stamps match, so this is confined to
pre-release rows and converges after one write each. Reach not measured:
no in-repo host composes a `DatabaseLoader` (only `new MetadataManager({
datasource, driver })` or `setDataEngine` does, and nothing in this
repository calls either). Not filed.
- `packages/objectql/scripts/dry-run-hash-compat.ts` compares a stored
stamp with a type-blind `hashSpec(body)`. So it reports `checksum_drift`
for every `object` row whose `fields` are not in sorted order, whichever
writer stamped it since #21790. Offline probe, not filed.
- `MetadataManager.save(type, …)` passes `type` unfolded, where
`register` folds it with `canonicalMetadataServiceType`. A plural
spelling would hash with no ordered-map row and key a separate row. Read
from source, unexercised, not filed.
---
_Generated by [Claude
Code](https://claude.ai/code/session_017ErfyP2Rx7XWHJA27QjyUi)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
1 parent 18fe681 commit 0fe0a59
4 files changed
Lines changed: 299 additions & 14 deletions
File tree
- .changeset
- packages/metadata/src
- loaders
- utils
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
22 | | - | |
| 22 | + | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
27 | 27 | | |
28 | | - | |
29 | 28 | | |
30 | 29 | | |
31 | 30 | | |
| |||
698 | 697 | | |
699 | 698 | | |
700 | 699 | | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
701 | 710 | | |
702 | 711 | | |
703 | 712 | | |
704 | 713 | | |
705 | 714 | | |
706 | | - | |
| 715 | + | |
| 716 | + | |
707 | 717 | | |
708 | 718 | | |
709 | 719 | | |
| |||
713 | 723 | | |
714 | 724 | | |
715 | 725 | | |
| 726 | + | |
716 | 727 | | |
717 | 728 | | |
718 | 729 | | |
| |||
722 | 733 | | |
723 | 734 | | |
724 | 735 | | |
725 | | - | |
726 | | - | |
727 | | - | |
728 | | - | |
729 | | - | |
730 | | - | |
731 | 736 | | |
732 | 737 | | |
733 | 738 | | |
| |||
1353 | 1358 | | |
1354 | 1359 | | |
1355 | 1360 | | |
1356 | | - | |
| 1361 | + | |
| 1362 | + | |
1357 | 1363 | | |
1358 | 1364 | | |
1359 | 1365 | | |
| |||
1383 | 1389 | | |
1384 | 1390 | | |
1385 | 1391 | | |
| 1392 | + | |
1386 | 1393 | | |
1387 | 1394 | | |
1388 | 1395 | | |
| |||
1401 | 1408 | | |
1402 | 1409 | | |
1403 | 1410 | | |
1404 | | - | |
1405 | 1411 | | |
1406 | 1412 | | |
| 1413 | + | |
| 1414 | + | |
1407 | 1415 | | |
1408 | 1416 | | |
1409 | 1417 | | |
1410 | 1418 | | |
1411 | 1419 | | |
1412 | | - | |
| 1420 | + | |
| 1421 | + | |
1413 | 1422 | | |
1414 | | - | |
| 1423 | + | |
1415 | 1424 | | |
1416 | 1425 | | |
1417 | 1426 | | |
| |||
1444 | 1453 | | |
1445 | 1454 | | |
1446 | 1455 | | |
| 1456 | + | |
1447 | 1457 | | |
1448 | 1458 | | |
1449 | 1459 | | |
| |||
1481 | 1491 | | |
1482 | 1492 | | |
1483 | 1493 | | |
1484 | | - | |
| 1494 | + | |
| 1495 | + | |
1485 | 1496 | | |
1486 | 1497 | | |
1487 | 1498 | | |
| |||
1523 | 1534 | | |
1524 | 1535 | | |
1525 | 1536 | | |
| 1537 | + | |
| 1538 | + | |
| 1539 | + | |
| 1540 | + | |
| 1541 | + | |
| 1542 | + | |
| 1543 | + | |
| 1544 | + | |
| 1545 | + | |
| 1546 | + | |
| 1547 | + | |
| 1548 | + | |
| 1549 | + | |
| 1550 | + | |
| 1551 | + | |
| 1552 | + | |
| 1553 | + | |
| 1554 | + | |
| 1555 | + | |
| 1556 | + | |
| 1557 | + | |
| 1558 | + | |
| 1559 | + | |
| 1560 | + | |
| 1561 | + | |
| 1562 | + | |
| 1563 | + | |
| 1564 | + | |
| 1565 | + | |
| 1566 | + | |
| 1567 | + | |
| 1568 | + | |
| 1569 | + | |
| 1570 | + | |
| 1571 | + | |
| 1572 | + | |
| 1573 | + | |
| 1574 | + | |
| 1575 | + | |
| 1576 | + | |
| 1577 | + | |
| 1578 | + | |
| 1579 | + | |
| 1580 | + | |
| 1581 | + | |
| 1582 | + | |
| 1583 | + | |
| 1584 | + | |
| 1585 | + | |
| 1586 | + | |
| 1587 | + | |
| 1588 | + | |
| 1589 | + | |
| 1590 | + | |
| 1591 | + | |
| 1592 | + | |
1526 | 1593 | | |
1527 | 1594 | | |
1528 | 1595 | | |
| |||
0 commit comments