Repository navigation
Commit 5a871e4
perf(core,metadata-protocol): an authenticated data request pays 6 warm serial rounds instead of 8, and 3 with the landed opt-ins (#24533)
Fixes #10757. This PR carries the card's engine legs (leg 4, leg 5, the
cold double resolution and leg 6) under claim 6264940865; leg A is ruled
out at 6265874149 (#24515 closed not_planned), so every item is landed
or ruled out once this lands.
Clause-②: no (no accept set, served body or authz answer moves: fewer
queries and rounds for the same answer; owes the seat's review of record
before the queue, at `CONTRACT_REVIEW_TIER` by the seat's choice, since
it changes when authz and localization caches are trusted)
## What changed
An authenticated `GET /api/v1/data/OBJECT?$top=1` on the stock showcase
(local PostgreSQL 16) now pays fewer rounds:
- warm admin: **11 queries / 6 rounds**, was 12 / 8;
- warm member: **10 / 6**, was 11 / 7;
- with the landed opt-ins on (grants cache TTL above 0, `$count=false`):
**3 / 3**, was 4 / 4.
The served bodies and the resolved authz envelopes are unchanged.
- **Leg 4, landed**
(`packages/core/src/security/resolve-authz-context.ts`). The Default
Organization read (`sys_organization` by slug) and the activation-ledger
read (`sys_metadata_activation`) are now one wave. Before, they were
awaited one after the other.
- While the organization read is open, the ledger is asked about every
permission-set name that may apply.
- One recorded widening: an organization-owned `admin_full_access` grant
that turns out not to anchor platform standing adds `admin_full_access`
to the ledger's `name IN` list. Nobody reads that verdict, and the
envelope does not move.
- ⛔ No predicate is edited. The organization read is still `{ slug:
'default' }` with limit 1. `grantAppliesInTenant` and
`grantAnchorsPlatformStanding` (the NULL-organization readings that the
C8 lane holds, 6264887348) are called as before and not edited. A pin
asserts the organization read byte-for-byte.
- **Leg 4, the duplicate `sys_user` read: ruled out here.** The two
reads are better-auth's session-to-user read (plugin-auth's adapter) and
`getUserRow` inside the `customSession` callback's
`resolveUserAuthzGrants` call. `getUserRow` is already memoized within
one resolution.
- The callback's caller is in `plugin-auth`. better-auth's user
deliberately does not carry `ai_access` (`auth-manager.ts`, the `user:`
config note). So one read can serve both consumers only through a
`plugin-auth` change (`domain:services`); no card carries it now
(carrier: none), since #24515 closed `not_planned`.
- Adding a core seed option that nothing passes would be a speculative
surface, so none is added. See the report's `open_questions`.
- **Leg 5, landed** (the same file, `localizationCache`). A success
entry is now retired only by:
- a write that can change its answer: a write to `sys_setting` or
`sys_platform_setting`, the two tables `SettingsService.getMany` and the
direct fallback read, observed after it settles by an unkeyed engine
middleware;
- a non-`write` epoch reason (`remote`, `metadata`, `manual`);
- the `localization` settings change seam;
- the TTL.
Before, any write to any object retired it. On an idle boot, the outbox
pollers' claim updates were such writes.
- The failure entry (#10221 / #11877) keeps its TTL-only rule. A pin
proves that a watched write and a `remote` bump do not retire it.
- A success is cached only on an engine with both the write epoch and
`registerMiddleware`. With TTL `0` nothing is registered on the engine.
- **Cold, ruled out with its reading** (`request-grants-memo.ts`, the
`attachedHere` guard in `commit`). The guard is load-bearing:
- A one-shot reading (scratch test, not committed) on the memo suite's
own doubles: a revocation that started before the observer existed and
landed between the first request's two resolutions. With the guard, the
request is authorised without the revoked `auditor` position. With the
guard removed, it is served `auditor`.
- A dist-only experiment with the guard removed measured the first
request at 16 / 8 instead of 23 / 10, so the guard is the whole cause of
the double resolution.
- The only change that keeps the memo's argument whole is to register
the observer before the first request: a boot-time seam, outside this
card's file fence. It would save 7 queries and 2 rounds once per
process. It is not built here (four-axis reasoning in the report).
- **Leg 6, landed in `packages/metadata-protocol` (this lane).** Nothing
changes in `@objectstack/rest`, so nothing is declared to `domain:cli`.
- `getMetaItem`, the served face that `enforceApiAccess` reads on every
`/data` request, now reads the item's active `sys_metadata` rows through
#11967's overlay cache, under a key naming the item. The protocol's own
authoring reads still read the store (the private body, e.g. the
destructive-change check before an object save).
- The overlay cache, for the list read and the by-name read alike, is
now retired by a `sys_metadata` write observed after it settles, by
every non-`write` epoch reason (the metadata bridge's `remote` bump
included), and by the TTL. #11633 §4's leg-D design names this trigger:
"any engine-seam write to `sys_metadata`". Before, any write to any
object retired it, so the outbox pollers and every user's data write
made the next `/data` request read `sys_metadata` again.
- **Docs:** the `OS_LOCALIZATION_CACHE_TTL_MS` and
`OS_METADATA_OVERLAY_CACHE_TTL_MS` rows
(`content/docs/deployment/environment-variables.mdx`) and the
overlay-cache note (`content/docs/concepts/metadata-lifecycle.mdx`) now
state the new retirement rules.
## Reading: local PostgreSQL 16.15, method of 6255731843
- **Setup:** the stock showcase, `pnpm dev -- --fresh -p PORT --database
postgres://…`, a fresh database per boot. Each request is `GET
/api/v1/data/OBJECT?$top=1` with `X-OS-Debug-Timing: json`, 2.5 s apart.
Per persona (admin, and `auditor.demo@example.com`), three objects
(`sys_user_preference`, `showcase_task`, `sys_api_key`), four requests
each.
- **Instrument:** a temporary per-query log in
`SqlDriver.installQueryTiming`, applied to the source, built into
`dist/` and then reverted. The source is blob-equal to `HEAD` and the
dists were rebuilt clean afterwards. The admin's logged count equals
`Server-Timing`'s `N queries` on every request.
- **Rounds:** a query that starts before the latest end of the current
round joins it; otherwise it opens a new round.
- **Background write:** before each object's third request, one untimed
`PATCH` of a `showcase_task` row. That object is unrelated to
localization and to `sys_metadata`.
- **Runs:** `L` = behind a +20 ms RTT proxy (switched on after boot),
`R` = direct.
- **Trees:** `main` = `913fd6df8a`. After = this branch merged with
`origin/main` `f4755a68a`, head `4746b38c1`.
| case | `main` L1 / R1 | after L1 / R1 |
|:--|--:|--:|
| warm admin | 12 / 8 · 12 / 8 | **11 / 6** · **11 / 6** |
| warm member | 11 / 7 · 11 / 7 | **10 / 6** · **10 / 6** |
| first request after boot (admin) | 23 / 12 · 23 / 12 | 23 / 10 · 23 /
10 |
| warm, after a write to an unrelated object | 14 / 9 (admin), 13 / 8
(member): 6 of 6 retired localization | 11 / 6 and 10 / 6: 0 of 6
retired anything |
| opt-ins on (grants TTL 10000, `$count=false`), warm (L2c / R2c) | 4 /
4 · n/a | **3 / 3** · **3 / 3** |
The warm admin request after, per query (final-L1, `sys_user_preference`
#2):
| # | round | query |
|--:|--:|:--|
| 1 | 1 | `sys_session` by token (better-auth) |
| 2 | 2 | `sys_user` by id (better-auth's fallback join, leg A) |
| 3–7 | 3 | wave 1: `sys_user`, `sys_member` ×2,
`sys_user_permission_set`, `sys_user_position` |
| 8–9 | 4 | `sys_organization` by slug **and** `sys_metadata_activation`
(was rounds 4 and 5) |
| 10 | 5 | the data SELECT |
| 11 | 6 | the COUNT |
- **Gone from the warm request:** `sys_metadata` by name (was round 6).
The localization pair is gone too, except at its 30 s TTL (the two
remaining misses in each run land 30.4 s and 30.1 s after the key's
first read).
- **The member's request:** the same, without `sys_organization`.
- **Opt-ins on:** `sys_session`, `sys_user`, the SELECT.
- **Wall time, illustrative only:** a synthetic RTT, not deployment
evidence, and ⛔ no claim about cloud#1521's gate or cloud#1518's
placement. With the proxy, the warm admin DB span is 207 ms on `main`
and 169 ms after. With the opt-ins on it is 111 ms and 79 ms.
**Controls (byte identity, normalized for per-boot random ids and
timestamps):**
- The resolved authz envelope (positions, permissions, `org_user_ids`,
`accessible_org_ids`, posture, tab permissions) is one distinct value
per persona, and it is identical on `main` and after, in every pair
compared (main-R1 against final-R1 and final-L1; main-L2c against
final-L2c and final-R2c).
- Served bodies: 24 / 24 identical for main-R1 against final-R1, and
against the intermediate leg run.
- Across other boot pairs, `showcase_task` picks a different seeded row
under `order by id asc limit 1`, because seeded ids are random per boot.
The titles differ; the keys and totals do not. `sys_user_preference` and
`sys_api_key` are identical in every pair.
## Pins and ablations
Each ablation mutates only this package's source, and the subject is
imported by a relative specifier, so no rebuild leg applies. Each
restore was checked blob-equal to `HEAD`.
- **Leg 4:** `resolve-authz-context.request-rounds.test.ts` (new, 5
cases). The envelope written out, the reads issued, and 2 legs.
- Ablation: the base resolver restored. 4 of 5 red. Three of them red on
the leg count alone (`expected 3 to be 2`), with the envelope and reads
asserted green before it. The fourth is the recorded widening, red on
the ledger read.
- **Leg 5:** `resolve-localization-cache.test.ts`. Its doubles now carry
the engine's middleware seam. New pins:
- writes to `sys_http_delivery`, `sys_notification_delivery`,
`showcase_task` and `sys_session` do not retire the entry, with zero
reads;
- a `sys_platform_setting` write retires it;
- the retirement lands after the write settles;
- `metadata` and `remote` bumps retire it;
- no middleware seam means no cache;
- TTL `0` registers nothing;
- the failure memo survives a watched write.
Ablation: the leg-4 version of the file restored. 9 red, the in-flight
case among them.
- **Leg 6:** `protocol.item-read-overlay-cache.test.ts` (new, 7 cases)
and a new `meta-overlay-cache.test.ts` §10.
- Ablation 1: `servedRead` set to `false`. 4 red; the authoring-read,
no-seam and TTL-0 controls stay green.
- Ablation 2: the epoch-keyed `meta-overlay-cache.ts` and `protocol.ts`
restored. 10 red.
- The prolongation suite's double gained the middleware seam so that its
arms still run against a live cache. Ablation:
`applyRemoteMetadataMutation`'s `bumpWriteEpoch` removed. 3 red.
- **Pin sweep:** `request-grants-memo*` does not move. The
`resolve-authz-context.batch-equivalence` goldens do not move (no
fixture holds an organization-owned `admin_full_access`).
`rest-data-read-one-overlay-read.test.ts` does not move: its engine has
no write-epoch seam, so the cache declines there, as before. The
rest/hono localization pins pass unchanged.
- Moved: the doubles in `resolve-localization-cache.test.ts`,
`meta-overlay-cache.test.ts` and
`protocol.datasource-delete-prolongation.test.ts`, each to carry the
seam a real engine has.
- `scripts/engine-double-contract.pinned.json` learned the new suite's
`findOne` double (`--write`).
## Verification (head `4746b38c1`)
- `pnpm --filter @objectstack/core typecheck`: exit 0. `pnpm --filter
@objectstack/metadata-protocol typecheck`: exit 0. `--listFiles` counts
the new and edited test files in both programs.
- `pnpm --filter @objectstack/core test`: exit 0, 106 files / 2580
tests. `pnpm --filter @objectstack/core test:repo`: exit 0, 6 / 82.
- `metadata-protocol` vitest: exit 0, 248 files passed and 3 skipped,
20723 tests.
- The rest one-overlay-read and business-timezone pins: 46 / 46. The
plugin-hono-server localization pin: 13 / 13.
- `turbo run build` (all 76 packages) after merging `origin/main`: exit
0.
- `node scripts/pm/dispatch-gates.mjs --commands` (no paths) derived 109
families. Every one was run with its exit code captured before any pipe,
and all exited 0 (`check:engine-double-contract` after the ledger
write). `--ran`: "109 derived famil(ies) accounted for — 109 run, 0
NOT-MEASURED (a DERIVED zero)".
- **Fix round 1 (head `084e79e16`), for `Test Core (5/6)`:**
`environment-overlay-five-types.test.ts` stored its overlay row with
`driver.create` after a control read that now fills the by-name overlay
cache, so no engine middleware saw the write. The row now goes through
`engine.insert` under a system context, as every production writer does.
No cache was disabled, no TTL changed and no assertion was weakened.
Before: exit 1, 10 failed. After: 26 / 26. A sweep of the tests that
write `sys_metadata`, `sys_setting` or `sys_platform_setting` straight
into a driver found no other one that reads before it writes. Full
suites at `084e79e16`, each exit 0: objectql 446 / 8392, runtime 422
files, rest 300 files, plugin-security 188 files, service-settings 51 /
881, plugin-hono-server 34 / 422, mcp 40 / 479. objectql and rest
typecheck exit 0. `--ran` over both rounds: "111 derived famil(ies)
accounted for — 111 run, 0 NOT-MEASURED (a DERIVED zero)".
## Acceptance notes
- **Still paid, for the card's remaining lane:** leg A's second round,
ruled out under the current engine contract (6265874149); the duplicate
`sys_user` read (needs a `plugin-auth` seed; carrier: none); the cold
double resolution (a boot-time observer registration); and the COUNT
unless `$count=false`.
- **Grants cache default unchanged:** with its default of `0` (#11633
Fork 4, the maintainer's), leg B still costs 2 rounds per warm request.
This PR does not move it.
- **What the after-reading shows:** with the landed opt-ins on, the warm
request is already at 3 rounds before leg A lands. With leg A it would
be 2.
- **The 1 parent ffba02e commit 5a871e4
15 files changed
Lines changed: 1243 additions & 220 deletions
File tree
- .changeset
- content/docs
- concepts
- deployment
- packages
- core/src/security
- metadata-protocol/src
- objectql/src
- rest/src
- scripts
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
206 | 206 | | |
207 | 207 | | |
208 | 208 | | |
209 | | - | |
210 | | - | |
211 | | - | |
212 | | - | |
213 | | - | |
214 | | - | |
215 | | - | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
216 | 221 | | |
217 | 222 | | |
218 | 223 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
375 | 375 | | |
376 | 376 | | |
377 | 377 | | |
378 | | - | |
379 | | - | |
| 378 | + | |
| 379 | + | |
380 | 380 | | |
381 | 381 | | |
382 | 382 | | |
| |||
0 commit comments