Skip to content

feat(cli,plugin-audit): os migrate audit-mail-bodies masks the archive datasource too - #24934

Merged
objectstack-fleet[bot] merged 12 commits into
mainfrom
claude/issue-24899-archived-mail-bodies
Oct 20, 2026
Merged

objectstack-fleet[bot] merged 12 commits into
mainfrom
claude/issue-24899-archived-mail-bodies

Conversation

@objectstack-fleet

@objectstack-fleet objectstack-fleet Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #24899

Clause-②: yes (widening)

os migrate audit-mail-bodies now masks the mail bodies in the sys_audit_log copies the lifecycle Archiver already moved to the archive datasource. Hot copies masked by the run are archived masked later, so the same run covers those too. The command keeps its dry-run / --apply / --yes / --json shape, and its report is still counts only.

The claim expected Clause-②: no. From the diff it is yes (widening): the --json document gains an archive key, and @objectstack/plugin-audit gains exports. Both packages take a minor changeset.

Measured

Every reading below is from origin/main at 81e6edd46 and is a count, a status or a boolean. No body value was printed.

  1. Which deployments can have an archive datasource, and who reads it.
    • There is no special configuration. The Archiver resolves archive.to by driver name (packages/objectql/src/lifecycle/lifecycle-service.ts:1344, engine.datasource(name) → packages/objectql/src/engine.ts:21671, the drivers map). A driver is registered under the name archive in one of three ways:
      • (a) A stack-config datasources entry. Nothing routes objects to an archive, so the ADR-0062 D2 gate connects it only with autoConnect: true (packages/services/service-datasource/src/datasource-connection-service.ts:301-305, packages/spec/src/data/datasource.zod.ts:665).
      • (b) A runtime datasource row: a sys_metadata row of type datasource, created in Setup → Datasources and rehydrated by os serve at boot (packages/services/service-datasource/src/datasource-admin-plugin.ts:125-139, :734-760).
      • (c) A host plugin that publishes a driver.archive service (packages/objectql/src/plugin.ts:1060-1076).
    • No shipped template or example declares one. There are zero name: 'archive' datasources under examples/ and packages/create-objectstack. Every deployment that declares none keeps every row hot: archive-pending (lifecycle-service.ts:1349-1355).
    • Readers: only the Archiver reads the archive through the platform. engine.datasource( has one caller, lifecycle-service.ts:1344, and the engine never routes sys_audit_log there. Otherwise there is direct database access, and the datasource admin door: GET /datasources/:name/remote-tables plus the object-draft flow, gated by requireDatasourceAdmin (packages/services/service-datasource/src/admin-routes.ts:706-707). A federated object an admin binds over the archived table would serve it over REST.
  2. Does the archive copy pass through the mask? No. It is a verbatim driver-to-driver copy: hot.find → cold.upsert(object, row, ['id']) → hot.bulkDelete (lifecycle-service.ts:1471-1520), with no hook, no writer and no mask. So a hot copy written before fix(platform-objects,plugin-audit,plugin-approvals)!: mail bodies are read only with manage_platform_settings, and the audit mirrors never copy them in clear #24688 and archived before the operator runs the mask lands in the archive in clear. Pinned below in case 4: a copy masked hot is archived masked.
  3. A supported way to run the mask over the archive? None before this PR: the runner took the engine only (packages/plugins/plugin-audit/src/mail-log-body-migration.ts, migrateMailLogBodyCopies). Two facts made one sound:
    • The runner's reads and writes fit the IDataDriver contract (find with where / fields / orderBy / limit, and update(object, id, data, { preserveAudit })).
    • bootSchemaStack already connects declared datasources when asked to (composeDeclaredDatasources, packages/cli/src/utils/schema-migrate.ts:1345-1350).
  4. sys_activity: no archive. Its lifecycle is telemetry, with 14-day retention and rotation (packages/plugins/plugin-audit/src/objects/sys-activity.object.ts:29-34), so its copies are only ever hot.

What changed

  • @objectstack/plugin-audit.
    • migrateArchivedMailLogBodyCopies(driver, logger, options) runs the same pages and planners through a datasource's driver, as the Archiver itself does. preserveAudit keeps each row's updated_at.
    • An archive table nothing was archived into yet reads as no rows, by isMissingTableError. Any other refused read is a failure.
    • New objects option on both runners. The engine runner's behaviour is unchanged.
  • os migrate audit-mail-bodies.
    • After the hot run, it reads each copy table's declared lifecycle.archive.to and runs the archive runner over engine.getDriverByName(to).
    • It boots with composeDeclaredDatasources: true, so a declared archive is connected as os serve connects it.
    • When the target is not connected but an active runtime datasource row of that name exists, the archive reads not-connected and the command exits 1. It does not report a clean archive.
    • The success path ends with exitOneShotCommand, as plan / apply / unmapped-columns do. Without it, the pool the declared connection opened kept the process alive after "Graceful shutdown complete". Measured here: it hung, and TaskStop ended it.
  • Docs: the command's section of content/docs/deployment/cli.mdx, plus its row in the command table.
  • Changeset: .changeset/24899-archived-mail-bodies.md (minor for both packages).

Proof

  • End to end. packages/cli/src/commands/migrate/audit-mail-bodies.archive.integration.test.ts, 7/7.
    • The fixture is a project that declares an archive SQLite datasource (autoConnect: true). A served-shape boot seeds 18.0-writer copies, and the real Archiver (lifecycle.sweep()) moves 3 rows. 2 of them hold a clear body.
    • Dry run: hot to-mask 1; archive examined, scanned 2, to-mask 2; nothing written.
    • --apply: archive rewritten 2; clear copies 0 in either store; every other archived column equal, updated_at included.
    • Re-run: rewritten 0, archived rows unchanged.
    • The copy that was hot during the run, aged and archived by a second sweep: archived masked (archive rows 4, clear 0).
    • No run printed the sentinel.
    • Runtime-row archive: not-connected, exit 1. No archive at all: none, exit 0.
  • Ablation. Removing the archive lookup (const driver = undefined) turns cases 1–4 red (4 failed, 3 passed). Restored to the HEAD blob b6e1355a.
  • Runner unit tests over a real SqliteWasmDriver store provisioned as the Archiver does (syncSchema + upsert): dry run, apply, idempotent re-run, an archive with no table yet, a refused read.

Local verification (head 39fc914f; the diff since 8a87750c is one test-file header comment)

  • @objectstack/plugin-audit: pnpm test 983/983 (60 files); typecheck green.
  • @objectstack/cli: build + typecheck green. The unit project: 4634/4635. The one failure was test/hook-timeout-override-refusal.test.ts CONTROL, a 5000 ms timeout while the gate sweep ran beside it. Re-run alone: 4/4. It spawns a process this diff does not touch.
  • @objectstack/cli integration, run locally because this diff touches that layer: the new archive pin 7/7; data-commands.absent-database + schema-migrate.one-shot-family 151/151. test/json-stdout-purity.e2e.test.ts is left to CI.
  • Changed files: eslint --no-inline-config clean.
  • node scripts/pm/dispatch-gates.mjs --commands: 101 derived, every one run, exit codes captured before any pipe; --ran reconciles 101 / 97 run / 4 NOT MEASURED / 0 unrun.
    • The census was exit 1 (2) on 39fc914f. It is exit 0 on 4fb9afa1.
      • This diff adds one write call site (the archive runner's driver.update), so the count goes 232 → 233.
      • The seat widened the surface in amendment 6295968920 and again in 6296052814.
      • 3340a4ad ran node scripts/tenant-audit-census.mjs --write.
      • 105a8110 and 4fb9afa1 made the 11 hand-written prose figures +1, numbers only.
      • On 4fb9afa1, node scripts/check-tenant-audit-census.mjs --self-test passes 27 cases, and the plain run reports 233 write call sites certified and 23 prose figures held. Both exit 0.
      • The new census row: packages/plugins/plugin-audit/src/mail-log-body-migration.ts · update · object chosen at run time (the CLI limits the runner to sys_audit_log) · tenancy undecidable · context options unreadable · 1. The write goes to the archive datasource through its own driver, which no tenant scope covers. It masks every organization's archived copies by design, and only the operator command reaches it.
    • exit 3, prerequisite not met (4), NOT MEASURED, left to CI: check-plugin-teardown-shape --self-test (shallow clone), spec check:skill-examples, check:dual-build-cjs-loads, check:i18n-coverage (each needs the full monorepo build).
    • every other family: exit 0.
  • node scripts/pm/check-governed-merges.mjs --branch origin/claude/issue-24899-archived-mail-bodies: NOT governed; 791 changed lines.

Surface question (answered)

The seat widened the card's surface to the census regeneration and its 11 prose figures (6295968920, 6296052814 on #24899, plus the line-210 figure by message). The census is regenerated by the generator only, and Lint & Repo Gates' census step is green on 4fb9afa1.

Acceptance notes

  • Behaviour change of the boot: the command now connects the datasources the config declares, as os serve does. A declared datasource that fails to connect therefore fails the command's boot, as it fails os serve's. Before, the command never tried to connect it.

  • Stale comment, fixed here (f72911d8, amendment 6296052814): packages/cli/src/utils/schema-migrate.ts:1176-1179 now names os migrate audit-mail-bodies among the callers of composeDeclaredDatasources. This diff had made the old text false.

  • Not covered by this command: an archive registered only by a host plugin (driver.archive) that the CLI boot does not compose. It reads none unless a runtime datasource row of that name exists. A code-declared archive without autoConnect: true is never connected by os serve either, so the Archiver never archived to it.

  • 17.x: the writer mask (fix(platform-objects,plugin-audit,plugin-approvals)!: mail bodies are read only with manage_platform_settings, and the audit mirrors never copy them in clear #24688) and this command shipped in 18.0.0. A 17.x line, if cut, has neither, so it would owe the whole mail-body note (hot and archive), not only this half.


Generated by Claude Code

Census, surface and stale-comment lines updated by the epic seat on 2026-10-20 (about 16:50Z) to match head 4fb9afa1.

…rce's driver too

Claude-Session: https://claude.ai/code/session_01WQWCrorWxW5zDh4n6jCEF5
Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WQWCrorWxW5zDh4n6jCEF5
Co-authored-by: Claude <noreply@anthropic.com>
…e datasource too

The Archiver copies sys_audit_log rows past the hot window to the archive
datasource verbatim, so copies archived before the writer masked kept the
mail body in clear out of the hot run's reach. The command now runs the
same mask through the declared archive target's driver, and refuses to
report a runtime archive datasource it cannot connect as clean.

Claude-Session: https://claude.ai/code/session_01WQWCrorWxW5zDh4n6jCEF5
Co-authored-by: Claude <noreply@anthropic.com>
@github-actions github-actions Bot added size/l documentation Improvements or additions to documentation tests tooling labels Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 2 package(s): @objectstack/cli, @objectstack/plugin-audit, touching 22 documentable anchor(s). ⚠️ 1 changed file(s) yielded no anchor (packages/plugins/plugin-audit/src/index.ts), so the pages documenting them are NOT COVERED by this run — this is not a clean bill of health for those files.

35 hand-written doc(s) name something this change touched — list omitted above 15 rows. Re-derive on the tree named below: node scripts/docs-audit/affected-docs.mjs --json 71696121a64d8bd057fac969a72a84295890ed37.

⛔ 12 release-owned page(s) also affected — read-only, see AGENTS.md Documentation Guardrails.

What this run could not see
  • 1 changed file(s) yielded no anchor (packages/plugins/plugin-audit/src/index.ts) — pages documenting those are invisible to this run
  • 7 name(s) were too generic to anchor anything (single lowercase words)
  • the SDK route bridge reached 54 of 206 client-bound route-ledger rows — the other 152 have no registrar path: tail to select them, so pages documenting THEIR client methods cannot appear above, on this or any run. Of those 152: 0 are remediable by widening that discovery convention (an in-repo file declares the path; the convention did not scan it); 55 are structural — on a ledger where NOT ONE row is declared in-repo, so no discovery change reaches them at any price; 97 are undecided (no in-repo declaration, on a ledger that has other in-repo registrars — absence and an unreadable spelling are not distinguishable here). The rows themselves: node scripts/docs-audit/affected-docs.mjs --bridge-coverage
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.
  • a key NAME is not a key, so the hand re-read the line above prescribes can land on the wrong schema. The same spelling is authorable on one governed type and a [REMOVED] tombstone on another for each of active, aria, joins, objects, template, tools and version (censused on [finding] tools is a key on BOTH AgentSchema (tombstoned, dead) and SkillSchema (live, cloud-attested), so a name-based search attributes skill examples to the agent key — it produced a false stop-the-line alarm on PR #19059 #19093 over the liveness ledger's governed types, top-level keys); nothing in a search result distinguishes the two, so a grep hit on a LIVE example reads as evidence about the DEAD key. Measured on fix(spec): the agent.tools liveness row says dead — it claimed live on a key the schema tombstoned #19059: content/docs/ai/agents.mdx was reported as contradicting the agent.tools tombstone over its tools: example at :161, which is inside the defineSkill({ block opened at :155 — the page was already correct. Settle ownership by PARSING the value against both schemas, never by the name: that literal PASSES SkillSchema, and as an AgentSchema it FAILS at tools with the tombstone prescription. ⛔ These names are not the whole class — a key retired through a .strict() guidance map leaves no tombstone in the walked shape and none of them here (tool.category, live as AIToolDefinition.category).

Coarse fallback — 35 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 71696121a64d8bd057fac969a72a84295890ed37 → packageMentionDocs.

Which tree this was computed on

This run read content/docs from 32c016a1ec3832a25423d5beb542b35c73da9547 — the merge of head 791db71c37806719de5ba66281186385cc0a0100 into base 71696121a64d8bd057fac969a72a84295890ed37, which is what actions/checkout gives a pull_request run. Not the PR head.

A worktree cut from an older main holds a different content/docs, so re-deriving there can legitimately return a different list — that is a different tree, not a wrong row. To answer on the same tree:

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 32c016a1ec3832a25423d5beb542b35c73da9547 && git checkout 32c016a1ec3832a25423d5beb542b35c73da9547
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin 71696121a64d8bd057fac969a72a84295890ed37 791db71c37806719de5ba66281186385cc0a0100 && git checkout -B drift-repro 71696121a64d8bd057fac969a72a84295890ed37 && git merge --no-ff 791db71c37806719de5ba66281186385cc0a0100

node scripts/docs-audit/affected-docs.mjs --json 71696121a64d8bd057fac969a72a84295890ed37

⚠️ That checkout carried uncommitted changes, so the commit above does not fully identify what was read.

Advisory only, and a precision-first one (#9192): a page is listed because it names a
symbol, wire route or SDK method this diff touched — not because it mentions a changed
package. Each row says which anchor put it there, so a wrong row is reportable rather than
merely annoying. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs 71696121a64d8bd057fac969a72a84295890ed37 → pass the list as
args.docs, on the commit named under Which tree this was computed on.

…'s write site

Generator output only (node scripts/tenant-audit-census.mjs --write).

Claude-Session: https://claude.ai/code/session_01WQWCrorWxW5zDh4n6jCEF5
Co-authored-by: Claude <noreply@anthropic.com>
…urces caller list to this diff

The ten hand-written census figures this diff moved, each +1, and the
caller list that os migrate audit-mail-bodies now joins.

Claude-Session: https://claude.ai/code/session_01WQWCrorWxW5zDh4n6jCEF5
Co-authored-by: Claude <noreply@anthropic.com>
…ces' caller list

The previous commit's message names this change; it landed here.

Claude-Session: https://claude.ai/code/session_01WQWCrorWxW5zDh4n6jCEF5
Co-authored-by: Claude <noreply@anthropic.com>
@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review

Served-tier: TIER_DEFAULT
Head-sha: 4fb9afa11c69c5542591f1cc16f93725c24cf85e
Local-runs: none
Scope: packages/plugins/plugin-audit/src/mail-log-body-migration.ts, packages/plugins/plugin-audit/src/mail-log-body-migration.test.ts, packages/plugins/plugin-audit/src/index.ts, packages/cli/src/commands/migrate/audit-mail-bodies.ts, packages/cli/src/commands/migrate/audit-mail-bodies.archive.integration.test.ts, packages/cli/src/utils/schema-migrate.ts; read to judge them: packages/plugins/plugin-audit/src/mail-log-body-mask.ts, packages/platform-objects/src/audit/mail-log-body.ts, packages/plugins/plugin-audit/src/audit-writers.ts, packages/plugins/plugin-audit/src/objects/sys-audit-log.object.ts, packages/plugins/plugin-audit/src/objects/sys-activity.object.ts, packages/plugins/plugin-audit/package.json, packages/spec/src/contracts/data-driver.ts, packages/spec/src/data/driver.zod.ts, packages/drivers/driver-sql/src/sql-driver.ts, packages/types/src/driver-error-classification.ts, packages/objectql/src/engine.ts, packages/objectql/src/lifecycle/lifecycle-service.ts, packages/objectql/src/plugin.ts, packages/runtime/src/driver-plugin.ts, packages/services/service-datasource/src/datasource-connection-service.ts, packages/services/service-datasource/src/datasource-admin-plugin.ts, packages/cli/src/utils/one-shot-exit.ts, packages/cli/bin/run.js, packages/cli/src/commands/migrate/plan.ts, packages/cli/src/commands/migrate/apply.ts, packages/cli/src/commands/migrate/unmapped-columns.ts, scripts/tenant-audit-census.mjs, scripts/check-tenant-audit-census.mjs, .github/workflows/lint.yml, AGENTS.md

This is a read-only review of the head, read in a detached worktree that was removed afterwards. It covers the full diff from merge-base 81e6edd46 to the head: 10 files, +785/−66, in 10 commits. It also covers card #24899 with its claim 6293910100, amendments 6295968920 and 6296052814, and os-dev-report 6295884727; the PR body, including the two Acceptance notes the seat asked to have judged; and the head's check-runs and its two in-progress check-suites. Nothing was built, tested or re-gated, and nothing was written to GitHub. No mail body value, link, token, address or fixture string appears in this record: body columns are named by field and rows are referred to by count.

① Derived judgments

1. Archive mask correctness

  • Selection and mask value: correct.
    • Both runners share one loop, runMailLogBodyCopies (mail-log-body-migration.ts:258-339).
    • Each page reads only object_name $in MAIL_LOG_OBJECTS (:284). That list is exactly sys_email and sys_identity_email (platform-objects/src/audit/mail-log-body.ts:22).
    • The planner re-checks the object name through maskMailLogBodyCopy (mail-log-body-mask.ts:29-30). So no other object's row can be patched, even by a driver that ignored the filter.
    • The masked keys are MAIL_LOG_BODY_FIELDS, which is body_text and body_html (mail-log-body.ts:25). A set body becomes SECRET_MASK, null stays null, and an absent key is never invented (mail-log-body-mask.ts:32-37).
    • The writer's ledgerView calls the same helper (audit-writers.ts:1774). The writer, the hot runner and the archive runner therefore produce one mask value.
  • Columns walked: complete.
    • The runner walks new_value and old_value (mail-log-body-migration.ts:115). These are the writer's two copy columns (audit-writers.ts:2045-2046).
    • The writer also writes metadata on an audit row, but only attribution goes there (:2077-2081). So no copy column is missed.
    • Both copy columns are textarea (sys-audit-log.object.ts:383, :391), so a driver returns strings and the string-only planner (:94) is not bypassed.
    • The writer's safeStringify is plain JSON.stringify (audit-writers.ts:520-522), so no stored copy is truncated into text the planner cannot parse.
  • Every other archived column stays byte-identical, updated_at included.
    • The archive write is driver.update(object, id, patch plus updated_at, { preserveAudit: true }) (mail-log-body-migration.ts:247-250). The patch holds only the column or columns the planner changed.
    • SqlDriver keeps a supplied updated_at exactly when preserveAudit is true and the value is non-null (sql-driver.ts:8568-8570, applied at :8819). Otherwise it issues the patch by id only (:8805-8843). The contract says the same (driver.zod.ts:80-88).
    • When updated_at is null, the write omits preserveAudit and the driver would stamp now. That branch is not reached on an archived row: the Archiver's upsert stamps a null updated_at (sql-driver.ts:8554-8558).
    • The integration test reads both stores raw (SELECT * through its own better-sqlite3 connection, test:212-219). It compares every column except the two copy columns, updated_at included (test:306-313). On SQLite this is a byte-level proof.
  • Idempotent: yes. A masked copy plans to null. This is pinned by the unit re-run (rewritten 0, rows deep-equal) and by integration case 3.
  • Edge cases: handled.
    • No table yet: isMissingTableError(e, object), with the object named, reads as no rows (:243).
    • Any other refused read: it counts one failure, warns, and stops that table (:291-300). The command then exits 1.
    • Pagination: keyset on id, with $gt and a limit of 500 (:280-329). The unit test pages at size 2 over 4 rows.
    • Partial runs: each write is one single-row update. An interrupted run leaves whole rows done and the rest for an idempotent re-run. No row is half-written.
  • Target: only the archive driver, only the two copy tables.
    • engine.getDriverByName(to) (audit-mail-bodies.ts:283) reads the same drivers map that engine.datasource(name) reads (engine.ts:21491-21493 and :21671-21677). That is what the Archiver resolves its target with (lifecycle-service.ts:1344).
    • to comes from the registered object's lifecycle.archive.to (audit-mail-bodies.ts:274-279). The sweep reads the same field (obj.lifecycle, lifecycle-service.ts:738); governance overrides only windows, never to (:893-900).
    • The table list is fixed, and objects can only narrow it (mail-log-body-migration.ts:266, :276). No caller can aim the runner at another object. It reaches the hot store only if a deployment registers its hot driver under the archive's name.
    • The CLI boot never sweeps (armLifecycleSweep: false, schema-migrate.ts:1235-1238). So connecting the archive cannot make the CLI itself move rows.

2. The CLI

  • Shape: kept.
    • The flags are unchanged from the base: database-url, apply, yes, force and json (audit-mail-bodies.ts:132-147).
    • The occupancy refusal, the --yes prompt and the JSON confirmation_required error are unchanged (:167-210).
    • Neither the base nor the head has a backup-attestation flag, so there was none to keep or lose.
  • Dry run: writes nothing to either store.
    • The runner skips every write unless apply is set (mail-log-body-migration.ts:310).
    • The boot keeps deferSchemaDdl and readOnlyProbe (audit-mail-bodies.ts:231). The declared-datasource publisher inherits readOnlyProbe (schema-migrate.ts:649-657, :1351), so an absent SQLite archive opens empty in memory and is not created.
    • The runtime-row probe is a read (audit-mail-bodies.ts:48-62).
  • How targets are found.
    • The command reads every copy table's declared lifecycle.archive.to (:274-279). Only sys_audit_log declares one; sys_activity is telemetry (sys-activity.object.ts:29-34).
    • When no driver is registered under to, it probes sys_metadata for { type: 'datasource', name, state: 'active' } (:55). That is the filter os serve's rehydration reads (datasource-admin-plugin.ts:130).
    • A row found reads not-connected and exits 1. A refused probe reads unknown and exits 1. Neither reads none and exits 0.
    • But none is also the reading for three cases where an archive exists: a declared archive this boot failed to connect, a declared archive the CLI's connect policy refused, and an archive a host registers outside the stack config. See B1 and B2.
  • --json: a widening. The old keys database, apply, report and duration are kept, and one key, archive, is added (:301). report is still the hot runner's report, unchanged.
  • Nothing printed carries a body.
    • The new lines print only datasource names, object names and counts (:314-340).
    • A runner warn line carries an error message. For a write, its bindings are the already-masked patch (mail-log-body-migration.ts:318-323). For a read, they are object names and an id.
    • Integration case 5 asserts that none of the five runs printed the sentinel.
  • exitOneShotCommand: clean exit.
    • run() awaits mask() and then exitOneShotCommand(process.exitCode or 0) (audit-mail-bodies.ts:156-159). This is the pattern of plan.ts:132 and apply.ts:154.
    • stack.shutdown() runs in finally (:363-365).
    • The failure paths exit through oclif's handle(), which exits after its report (bin/run.js:186-190, :258).

3. The future-archived rows claim: true.

  • The Archiver copies verbatim, with no hook in between: cold.upsert(object, row, ['id']) (lifecycle-service.ts:1501-1503). So a hot copy masked before it moves arrives masked.
  • One invocation runs the hot half and then the archive half. A row the sweep moves before the hot half reaches it is already in the archive when the archive half starts. The operator therefore needs no ordering against the sweep.
  • Case 4 is a real proof of the mechanism (test:327-332). The young copy masked hot by --apply (case 2: hot clear 0) is aged and moved by a second real sweep. It lands among 4 archive rows, with 0 clear.
  • The docs say this plainly: "A hot copy masked first is archived masked" (cli.mdx:1515-1517).
  • They do not mention the sweep schedule. A live server sweeps hourly, on an in-process timer, not a cron job (DEFAULT_LIFECYCLE_SWEEP_MS, lifecycle-service.ts:81, :512-519). See B6 for the narrow race this leaves.

4. Tests

  • Real Archiver, real driver. The integration test runs lifecycle.sweep() (test:142) over a real SqlDriver on better-sqlite3 in both stores. The CLI is spawned from source (test:59, :191-209).
  • Counts and booleans: mostly. Cases 2 and 3 also compare non-body columns and whole masked rows with toEqual (test:311, :314, :324). On failure those would print row contents. The fixtures are sentinels, so this is harmless (B8).
  • Ablation reproduces by reading. With the lookup removed, the fixture falls to the runtime-row probe and reads none. Cases 1 to 4 then fail and cases 0, 5 and 6 pass. That is 4 red and 3 green, as the dev reported.
  • Positive control: present. Case 1 asserts hot to-mask 1, and case 2 asserts hot clear 0 (test:290, :305).
  • Gaps.
    • Case 1's "writes nothing" check reads archivedBefore, a snapshot taken before the dry run (test:244-246, :297). As a dry-run pin it is vacuous (B5).
    • No case declares an archive that fails to connect (B1).
  • Unit tests are meaningful (mail-log-body-migration.test.ts:271-357). They use a real SqliteWasmDriver provisioned as the Archiver does. They cover the dry run; apply with page size 2, other-column equality and a deep-equal re-run; the missing table; and the refused read.

5. Docs and changeset

  • cli.mdx, partly matching.
    • The examples, flags, and the examined, not-connected and unknown rows match the code, exit codes included (cli.mdx:1535, :1537-1538 against audit-mail-bodies.ts:293-297).
    • The none row (cli.mdx:1536) is false in the B1 and B2 cases. So is "so it reaches an archive declared there" (:1519-1520).
    • The command-table row says "Covers the archive datasource" (:1207). That holds only within B1 and B2's limits.
  • Changeset, sentence by sentence.
    • True:
      • The 18.0.0 command was hot-only.
      • The lifecycle values match sys-audit-log.object.ts:36-40.
      • The Archiver copies verbatim, then deletes hot.
      • Clear bodies stay in the archive for as long as it keeps them.
      • The engine never routes sys_audit_log to the archive.
      • The command runs the hot half first, then the archive run in the same mode through the archive's driver.
      • sys_activity declares no archive.
      • Every other column is kept, updated_at included.
      • The run is idempotent and reports counts only.
      • A hot copy masked first is archived masked.
      • An archive is declared with autoConnect: true (gate (c), datasource-connection-service.ts:302).
      • A runtime-row archive reads not-connected and exits 1.
      • The --json archive entry shape and its four statuses match, and report is unchanged.
      • The export list matches index.ts:76-90.
    • Not true as written: "The command now connects the datasources the stack config declares, as os serve does."
      • It connects managed declarations only (MANAGED_DATASOURCES_ONLY, schema-migrate.ts:615-624).
      • An archive declared with autoConnect: true that fails to connect is left unconnected. The command then reads none (B1).
      • A fail-fast declaration that cannot connect now fails the run, and the changeset does not say so (B3).
  • No narrowing. No export of @objectstack/plugin-audit is removed or renamed. The type-export line is split, and both old types are kept. The options parameter of migrateMailLogBodyCopies is a superset of the old literal type.

6. Census

  • Generated region and counts file: consistent with a regeneration.
    • The gate runs check-tenant-audit-census.mjs --self-test and the check itself (lint.yml:2004-2008, inside the job named "Lint · repo & tooling gates"). That job is green on the head. Check A compares the committed region with a fresh run.
    • The new row reads mail-log-body-migration.ts, update, object, undecidable, options unreadable. That is what the generator gives a non-literal options argument (tenant-audit-census.mjs:1405-1421) together with a run-time object name.
    • Every figure that moved is +1 on a count the one new site moves.
    • The 39fc914f7 measurement stamp is the regeneration commit's parent. It sits in the generated region and is required to be present, not compared (check-tenant-audit-census.mjs:512, :524-554).
  • Prose: 11 number-only edits, each +1.
    • Ten match enforced patterns (check-tenant-audit-census.mjs:289-400): mdx line 125 (two), lines 153-154 (two), line 190, line 192 (two), line 193, line 210's population, and line 214.
    • The eleventh is line 210's "93" to "94", which no pattern checks. It is true against the census's own 94.
    • "(56%)" is still right: 131 of 233.
  • Posture: accurate.
    • The method is update.
    • The object is an arrow-function parameter, so it is chosen at run time.
    • Tenancy reads undecidable, and in fact sys_audit_log declares tenant: false (sys-audit-log.object.ts:31).
    • The options argument is a ternary, so it is unreadable.
  • No cross-tenant door.
    • The runner acquires nothing. It acts on a driver its caller already holds, and that driver can already write anything.
    • It can only write SECRET_MASK over two body keys, inside the fixed copy columns of mail-log copies.
    • Its only caller is the CLI command (audit-mail-bodies.ts:285). No route, hook or plugin reaches it.
    • The ledger has no tenant column for a fence to key on. Masking every organization's copies is the purpose, as amendment 2 point 3 says.

7. schema-migrate.ts comment: comment-only, and true. The edit at :1172-1181 touches comment lines only. The four callers it names are exactly the four non-test callers: plan.ts:193, apply.ts:225, unmapped-columns.ts:303 and audit-mail-bodies.ts:230.

② Semver level

minor for both packages is right, and so is Clause-②: yes (widening).

  • plugin-audit adds migrateArchivedMailLogBodyCopies, three types, and an optional objects option.
  • The CLI's --json document gains archive and keeps every old key.
  • Nothing authorable is removed or renamed. AGENTS.md:1087-1092 defines a narrowing as the removal or rename of a spec key, an export or a config field.

Seat point (1): the boot behaviour change.

  • What changed: the command now connects declared datasources. Before, it never tried, because no connection publisher was composed (schema-migrate.ts:600-605). A bound, mapped or boot-critical declaration that cannot connect now fails the command's boot with exit 1, where the run used to pass.
  • Precedent: that is the verdict os serve, os migrate plan and os migrate apply already give, and it keeps the OS_ALLOW_DRIVER_CONNECT_FAILURE escape (datasource-connection-service.ts:953, :1019).
  • Judgment: I do not read it as a Clause-② narrowing, since no authorable surface narrows. minor stands.
  • But: the changeset does not state this consequence and owes one line (B3).
  • The PR body's sentence on it ("A declared datasource that fails to connect therefore fails the command's boot") is true only for fail-fast declarations. It is false for the autoConnect-only archive (B1).
  • The new exit 1 on not-connected and unknown is the fix itself, and the changeset states it.

③ Boundary flags

B1 (BLOCKING): an under-mask reported clean, and false docs sentences. A declared archive that this boot does not connect reads none and exits 0.

  • Why the connect fails silently:
    • An archive declaration is addressed only by gate (c), autoConnect: true. "Gate (c) is not a fail-fast trigger" (datasource-connection-service.ts:290, :302).
    • So handleFailure finds no cause, logs "degrading (datasource left unconnected)" and returns without throwing (:968-1015). connectDeclared throws only on a fatal verdict (:376-397).
  • What the CLI then reports:
    • With no driver, it falls to the runtime-row probe (audit-mail-bodies.ts:283-291). A config-declared archive has no such row, so the reading is none.
    • It prints "none on this deployment — nothing archived, nothing to mask" (:323) and exits 0. In --json the entry reads status: 'none'.
  • A second trigger:
    • The CLI's connect policy refuses a non-managed schemaMode (skipped-policy, schema-migrate.ts:615-624, datasource-connection-service.ts:563-569).
    • os serve connects such a datasource under gate (a) (:301), so the Archiver fills it. The CLI then reads none the same way.
  • Why this matters for a p1 fix: an operator who runs the command where the archive is unreachable gets a clean exit 0. Missing credentials or a different network segment are enough. Meanwhile the archive keeps clear reset, verification, magic-link and invitation bodies for up to 7 years.
  • False sentences:
    • cli.mdx:1536 says none means "No such datasource … so nothing was archived".
    • cli.mdx:1519-1520 says "so it reaches an archive declared there".
    • The changeset says it "connects the datasources the stack config declares, as os serve does".
    • The class header promises "Absence is loud" (audit-mail-bodies.ts:114-119).
  • Smallest honest fix:
    1. In the no-driver branch (:290-291), read the connection service's retained verdicts before the runtime-row probe, by listConnectionStates(), the way findDeclaredDatasourceDrivers does (schema-migrate.ts:690-697). If a declaration named to is addressed by gate D2 but not available, report not-connected, with its retained reason, and exit 1.
    2. Keep none only for an unaddressed declaration or no declaration at all. In both cases os serve does not connect it either, so the Archiver held every row hot.
    3. Add one integration case: a declared autoConnect: true archive that cannot connect must read not-connected and exit 1.

B2 (BLOCKING): a false docs sentence, and the operator is not told (seat point 2). A host-registered archive reads none and exits 0.

  • The case: a host plugin publishes a driver.archive service, for example through DriverPlugin (runtime/src/driver-plugin.ts:39-41, adopted at objectql/src/plugin.ts:1060-1076). The CLI boot does not compose that host, so the command reads none and exits 0 while the archive holds clear bodies.
  • Where it is disclosed: only in the PR body's Acceptance notes, which do not ship. cli.mdx, the changeset, the command's description (:120-124) and its none line (:323) all say nothing of it. cli.mdx:1536's "so nothing was archived" is false for this case.
  • Blocking for a p1 security fix: yes. The shipped docs and output state a conclusion the code cannot know.
  • Smallest honest fix: text only, since this boot cannot detect a host it does not compose. Reword the none output line, the cli.mdx none row and the changeset to say what was checked: no datasource of that name is declared and connected on this boot, and no active runtime row names one. Drop "so nothing was archived". State that an archive a host registers in code outside the stack config is not reached, and must be declared in the stack config's datasources, with autoConnect: true, to be masked. This rewording also serves B1's none.

B3 (non-blocking, owed in the same pass): the changeset omits the boot consequence (seat point 1). Add one line: a bound, mapped or boot-critical declared datasource that cannot connect now fails the command's boot with exit 1, as on os serve. Name the OS_ALLOW_DRIVER_CONNECT_FAILURE escape. This is not a Clause-② narrowing (see ②).

B4 (non-blocking, pre-existing): a contradicted sentence next to the new text. cli.mdx:1503-1506 says the copies "age out with each table's retention (... 90 for the audit log)". The object says rows are retained hot when there is no archive (sys-audit-log.object.ts:32-35), and moved for 7 years when there is one. The new none row now contradicts that sentence. Fix it in the same pass.

B5 (non-blocking): case 1's dry-run pin is vacuous. It asserts on the pre-run snapshot (test:244-246, :297). Snapshot both stores after runs.dry and compare.

B6 (non-blocking): a concurrent-sweep race.

  • A live server sweeps hourly (lifecycle-service.ts:81). An Archiver batch could read a row before the hot half masks it, then upsert it after the archive half's cursor has passed its id. That row lands in the archive in clear.
  • The window is narrow and a re-run converges. The SQLite occupancy check (audit-mail-bodies.ts:167-187) covers only SQLite.
  • Suggest one docs sentence: run with the server stopped, or re-run once and read 0 to mask in both halves.

B7 (open question): the archive target ignores --database-url. The archive comes from the stack config, not from that flag. Yet the prompt says "this database and its archive" (:204), and the output names the archive only by datasource name. os migrate plan and apply share this. Should the report name the archive's location, as it does stack.dbLabel for the primary?

B8 (non-blocking): the integration test is not strictly counts and booleans only. A few toEqual row comparisons (test:311, :314, :324) would print row contents on failure. The fixtures are sentinels, so nothing secret is at stake.

CI on the head (read at 16:53Z). No check-run is red, so no owner is to blame.

Implemented-by: claude/issue-24899-archived-mail-bodies
Reviewed-by: session_01Y5583bs2RDd53MH6kM1ca3

VERDICT: FAIL

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Contract review (delta re-review after FAIL at 4fb9afa)

Served-tier: TIER_DEFAULT
Head-sha: 791db71c37806719de5ba66281186385cc0a0100
Local-runs: none
Scope: packages/cli/src/commands/migrate/audit-mail-bodies.ts, packages/cli/src/commands/migrate/audit-mail-bodies.archive.integration.test.ts, content/docs/deployment/cli.mdx, .changeset/24899-archived-mail-bodies.md; read to judge them: packages/services/service-datasource/src/datasource-connection-service.ts (listConnectionStates, errMsg, the failed-degraded and skipped-policy verdicts)

This is a read-only, in-seat review by the epic seat of the delta 4fb9afa1..791db71c: 4 files, +147/−39. It is judged against record 6296526345's B1 to B8. The dev's re-run of both census checks reports exit 0, and no write site moved. Nothing was built or run here.

① Derived judgments

  • B1 (BLOCKING before): CLOSED.
    • declaredConnectFailure reads datasource-connection.listConnectionStates() for the archive's name. It returns a verdict whenever that state's availability is not available. That covers failed-degraded (the gate (c) autoConnect failure that only warns), skipped-policy (the CLI's refusal of a non-managed schemaMode), and the other skipped states.
    • It runs in the no-driver branch before the runtime-row probe, and such an entry reads not-connected with connection: { status, reason }. archiveFailures counts not-connected, so the run exits 1.
    • none is now left only for a name with no retained state (nothing declared, or a declaration the D2 gate never attempted, which os serve would not connect either) and no active runtime row.
    • The new integration case 7 declares an autoConnect: true Postgres archive on a port nothing listens on. It asserts exit 1, status: 'not-connected', connection.status: 'failed-degraded' and no report. The dev's ablation of connection turns exactly that case red.
  • B2 (BLOCKING before): CLOSED.
    • The none output line, the cli.mdx none row and the changeset now say only what was checked, and drop "so nothing was archived".
    • Each of them states that an archive a host registers in code outside the stack config is not reached and reads none, and that it must be declared in the config's datasources with autoConnect: true to be masked.
    • The class header and the ArchiveOutcome docblock say the same.
  • B3: CLOSED. The changeset adds the line on the boot's new failure mode for a bound, mapped or boot-critical declaration, and names OS_ALLOW_DRIVER_CONNECT_FAILURE.
  • B4: CLOSED. cli.mdx now says audit-log copies do not age out: they are retained hot with no archive, and moved after 90 days and kept seven years with one.
  • B5: CLOSED. Case 1 snapshots both stores after the dry run and compares each with its pre-run snapshot (sameRows).
  • B6: CLOSED. cli.mdx and the changeset both say to run with the server stopped, or re-run once and read 0 in both halves.
  • B7: not done, by the seat's ruling (a nice-to-have shared with os migrate plan and apply).
  • B8: CLOSED.
    • The whole-row comparisons are now booleans (isDeepStrictEqual, sameRows).
    • The per-column check collects only id.column names.
    • The sentinel sweep now covers all six runs.
  • Other sentences, checked against the code:
    • cli.mdx's "connects the managed datasources the stack config declares through the same connect service os serve uses" is true. It is MANAGED_DATASOURCES_ONLY through the shared service.
    • The not-connected row's two causes match the two branches.
    • The --json line { datasource, objects, status, report?, connection? } matches ArchiveOutcome.

② Semver level

minor for both packages and Clause-②: yes (widening) still hold.

  • The delta adds an optional connection key to a --json entry, and adds not-connected cases to a status that already existed.
  • Nothing accepted before is refused, apart from the boot consequence the changeset now states. Record 6296526345 judged that consequence not a Clause-② narrowing.

③ Boundary flags

  • R1, not blocking. connection.reason is the connect error's raw message (errMsg, datasource-connection-service.ts:1056-1058), and the command now prints it.
    • os serve already logs that same text in its degrading warn line, so this adds no new exposure.
    • It does mean that a driver error which echoes a connection string would reach this output, as it already reaches the serve log.
    • No change is owed here. Redacting connect reasons belongs to the connection service, not to this command.
  • R2, not blocking, outside this card's surface (the dev's finding).
    • bin/run-dev.js pins NODE_ENV=development. Under it, sqlite-driver-fallback.ts steps an unopenable SQLite file down to an in-memory engine.
    • So a development-mode run against an unreachable SQLite archive reads connected, then examined, scanned 0, exit 0.
    • The production entry bin/run.js fails the read and exits 1.
    • The seat files this as its own card.
  • CI on the head (read at about 17:30Z): every completed check-run is green. Test Core shards 1 to 6 were still running. The seat lands the PR only when they are green.

Implemented-by: claude/issue-24899-archived-mail-bodies
Reviewed-by: session_01Y5583bs2RDd53MH6kM1ca3

VERDICT: PASS

@objectstack-fleet

Copy link
Copy Markdown
Contributor Author

Seat ACCEPT · epic PM session_01Y5583bs2RDd53MH6kM1ca3 (os-steve) · 2026-10-20T17:56Z. PR #24934 at 791db71c37806719de5ba66281186385cc0a0100 (#24899, priority:p1 security; os-dev-report 6295884727). ACCEPT.

Seat check

  • Shape.
    • 10 paths, +851 lines on the final head: plugin-audit's archive runner and its tests; the os migrate audit-mail-bodies archive half and its integration pin; the command's cli.mdx section; one changeset; and the census regeneration.
    • check-governed-merges: NOT governed, under 3,000 lines. This is an ordinary queue landing.
    • The surface was widened in two recorded amendments, 6295968920 and 6296052814.
  • Review.
    • TIER_DEFAULT (the declaration face, Clause-②: yes (widening)).
    • FAIL 6296526345 at 4fb9afa1: B1, a declared archive this boot did not connect read none, exit 0; B2, a host-registered archive was reported clean in shipped text.
    • PASS 6296994906 at 791db71c, with B1 to B6 and B8 closed. B7 is not done, by the seat's ruling.
  • Measured (by the dev, on the real Archiver and real drivers, counts only):
    • The Archiver copies sys_audit_log verbatim, so pre-mask copies sit in clear in the archive.
    • --apply masks them, keeps every other column byte-identical, and a re-run rewrites 0.
    • A hot copy masked first is archived masked.
    • A declared archive that cannot be reached reads not-connected, exit 1. A runtime-row archive reads not-connected, exit 1.
    • The sentinel body was printed by none of the six runs.
  • Gate check-runs on the head: the required checks (TypeScript Type Check, Test Core, Build Core, Lint & Repo Gates, Governed Surface Queue Guard) all succeeded. All 42 latest check-runs are success or skipped.
  • Accepted deviation: B7, naming the archive's location in the output, is left out, as shared nice-to-have scope with os migrate plan and apply.
  • Out of scope, and where it goes:
  • Landing: the seat flips the PR ready and enables auto-merge. Fixes #24899 closes the card.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/l tests tooling

Projects

None yet

2 participants