feat(cli,plugin-audit): os migrate audit-mail-bodies masks the archive datasource too

.changeset/24899-archived-mail-bodies.md

@@ -0,0 +1,19 @@
+---
+'@objectstack/cli': minor
+'@objectstack/plugin-audit': minor
+---
+
+feat(cli,plugin-audit): `os migrate audit-mail-bodies` also masks the mail bodies in the audit copies already moved to the `archive` datasource
+
+Clause-②: yes (widening)
+
+**Re-run `os migrate audit-mail-bodies --apply` on a deployment with an archive datasource.** The 18.0.0 command masked the hot audit tables only.
+
+- **What was left in clear.** `sys_audit_log` declares `lifecycle.archive: { after: '90d', to: 'archive', keep: '7y' }`. Past the hot window, the lifecycle Archiver copies its rows to the datasource named `archive` as they are, then deletes them hot. A `sys_email` / `sys_identity_email` copy archived before the audit writer masked the mail body still holds `body_text` / `body_html` in clear there, password-reset, verification, magic-link and invitation links included, for as long as the archive keeps it. The engine never routes `sys_audit_log` to the archive, so the hot run could not reach those rows.
+- **What changed.** After the hot run, the command reads the archive target each copy table declares (`sys_audit_log` only; `sys_activity` declares none) and runs the same mask, in the same dry-run or `--apply` mode, through that datasource's driver. Each archived row keeps every other column, `updated_at` included. The run is idempotent and its report is counts only. A hot copy masked first is archived masked, so one run covers both the copies already archived and the ones still to be.
+- **Which archives it reaches.** The command now connects the managed datasources the stack config declares, through the same connect service `os serve` uses. An archive datasource is declared with `autoConnect: true`, since nothing routes objects to it. It is NOT examined, and the run reports `not-connected` and exits `1` rather than reporting a clean archive, in two cases. In the first, the archive is declared but this boot did not connect it; the entry carries the connect status and reason. In the second, the archive is a runtime datasource created in Setup → Datasources, which `os serve` connects and this command does not; declare it in the config's `datasources` and re-run.
+- **What `none` means.** No datasource of that name is declared and connected on this boot, none was attempted, and no active runtime datasource row names one. That is all it checks. An archive a host registers in code outside the stack config is not reached and reads `none`. To mask it, declare it in the stack config's `datasources` with `autoConnect: true`.
+- **The command's boot can now fail on a declared datasource.** A bound, mapped or boot-critical declared datasource that cannot connect now fails the command's boot with exit `1`, as it fails `os serve`'s. `OS_ALLOW_DRIVER_CONNECT_FAILURE` is the same escape.
+- **Run it with the server stopped**, or re-run it once and read 0 to mask in both halves. A live server's Archiver can move a row between the hot run and the archive run.
+- **New in the `--json` document:** an `archive` array, one entry per declared target: `{ datasource, objects, status, report?, connection? }`, with `status` one of `examined`, `none`, `not-connected` or `unknown`. The `report` key and its shape are unchanged.
+- **New exports from `@objectstack/plugin-audit`:** `migrateArchivedMailLogBodyCopies(driver, logger, options)`, the runner over a datasource's driver, and an `objects` option on both runners, which limits a run to the named copy tables. Types `MailLogBodyArchiveDriver`, `MailLogBodyCopyObject` and `MailLogBodyMigrationOptions`.

content/docs/deployment/cli.mdx

@@ -1204,7 +1204,7 @@ where the data lives.
 | `os migrate comment-reactions` | Carry the emoji reactions a 17.x deployment stored in the retired `sys_comment.reactions` column into `sys_comment_reaction` records, list every stored value that does not read, and record the verified run that lets `os migrate apply --allow-destructive` drop the column |
 | `os migrate organization-ownership` | The v18 ownership ceremony (ADR-0131 D10): write a plan of what each table's rows will become, then `--apply` that reviewed plan. It upgrades a database 17.x or an 18.0 pre-release (`18.0.0-next.*`) created; a v18 server refuses to start on one until it has run. See [below](#os-migrate-organization-ownership) |
 | `os migrate activity-changed-fields` | Stamp the two read-rule columns (`changed_fields`, `milestone_key`) on the `sys_activity` rows written before they existed. Until it runs, those rows are withheld from every reader but the system, so run it at once after upgrading. No flag; idempotent |
-| `os migrate audit-mail-bodies` | Mask the mail bodies (`body_text`, `body_html`) that `sys_audit_log` and `sys_activity` copies of `sys_email` / `sys_identity_email` rows still hold in clear, written before the audit writer masked them. Reports counts only. No flag; idempotent. See [below](#os-migrate-audit-mail-bodies) |
+| `os migrate audit-mail-bodies` | Mask the mail bodies (`body_text`, `body_html`) that `sys_audit_log` and `sys_activity` copies of `sys_email` / `sys_identity_email` rows still hold in clear, written before the audit writer masked them. Covers the `archive` datasource `sys_audit_log` archives to as well. Reports counts only. No flag; idempotent. See [below](#os-migrate-audit-mail-bodies) |
 | `os migrate comment-parent-pair` | Stamp the parent pair (`object_name`, `record_id`) on the `sys_comment` rows written before it existed. Until it runs, a broad comment read past 2,000 candidate rows omits comments the reader may see, and its total agrees with the short list, so run it at once after upgrading. No flag; idempotent |
 | `os migrate meta --stored` | Replay the metadata conversion chain over this deployment's `sys_metadata` rows and rewrite the ones still carrying a pre-protocol shape. Hygiene, not a gate — nothing depends on it having run |
 | `os migrate security-catalog-overlays` | List the environment-wide permission-set and position rows (the legacy plurals `permissions` / `positions` included) stored over a name a configured package holds — the rows a v18 cold boot refuses — each with the package that holds its name; `--apply` deletes them, one history tombstone per row. Run it before the first v18 boot, with the flags and environment the deployment boots with: the platform security plugin's sets are held only where `os serve` composes it (an auth secret set, and the `auth` tier on), and the step takes `serve`'s `--preset` and `--dev` with `serve`'s meaning. Adopts nothing; exits 1 while it lists a row |
@@ -1501,9 +1501,55 @@ value.
 
 **Idempotent**: a masked body is the mask, so a second run masks nothing, and
 re-running is how an interrupted run is finished. A failed read or write exits
-`1`. The copies age out with each table's retention (14 days for activity, 90
-for the audit log, unless the deployment lengthens it); run it once after
-upgrading rather than wait for that.
+`1`. Activity copies age out with its 14-day retention. Audit-log copies do not
+age out: with no archive datasource they are retained hot, and with one they
+are moved there after 90 days and kept for seven years. Run it once after
+upgrading.
+
+**Run it with the server stopped**, or re-run it once and read 0 to mask in
+both halves. A live server's Archiver sweeps on its own schedule, and a row it
+moves between the hot run and the archive run can land in the archive
+unmasked.
+
+**The archive datasource too.** `sys_audit_log` declares
+`lifecycle.archive: { after: '90d', to: 'archive', keep: '7y' }`: past the hot
+window, the lifecycle Archiver copies its rows to the datasource named
+`archive` as they are, and deletes them hot. A copy archived before the writer
+masked still holds the body there, for up to seven years. After the hot run,
+the command reads the archive target `sys_audit_log` declares and runs the same
+mask, in the same dry-run or `--apply` mode, over that datasource. Its rows keep
+every other column, `updated_at` included. A hot copy masked first is archived
+masked, so one run covers both the copies already archived and the ones still
+to be. `sys_activity` declares no archive: its copies are only ever hot.
+
+The command connects the managed datasources the stack config declares through
+the same connect service `os serve` uses. A managed datasource nothing routes
+to is connected only with `autoConnect: true`, which is how an archive
+datasource is declared:
+
+```ts
+datasources: [
+  { name: 'archive', driver: 'sqlite', config: { filename: './data/archive.db' }, autoConnect: true },
+],
+```
+
+The `--json` document carries one `archive` entry per declared target, with a
+`status`:
+
+| `status` | Meaning | Exit |
+| :--- | :--- | :--- |
+| `examined` | Connected and masked (or counted); `report` holds the counts | `0`, or `1` on a failed read or write |
+| `none` | No datasource of that name is declared and connected on this boot, none was attempted, and no active runtime datasource row names one. That is all it checked | `0` |
+| `not-connected` | The archive was **not** examined. Either the stack config declares it and this boot did not connect it (`connection` carries the connect status and reason), or the database holds an active runtime datasource of that name (one created in Setup → Datasources), which `os serve` connects and this command does not | `1` |
+| `unknown` | Not connected, and the datasource rows could not be read | `1` |
+
+On `not-connected` for a runtime datasource, declare the archive datasource in
+the stack config's `datasources` (with `autoConnect: true`), then re-run. For a
+declared one, make it reachable from where the command runs, then re-run.
+
+An archive that a host registers in code outside the stack config (a plugin
+that publishes the driver itself) is not reached and reads `none`. To mask it,
+declare it in the stack config's `datasources` with `autoConnect: true`.
 
 #### `os migrate organization-ownership`
 

content/docs/permissions/tenant-audit-census.mdx

@@ -122,7 +122,7 @@ are reported as `undecidable` rather than assumed either way.
 
 The same holds twice over for the context. An options argument spelled as a
 literal can be read; one spelled `options`, `{ ...opts }`, or handed through a
-forwarding shim cannot, and **50 of the 232 sites are spelled that way**. A
+forwarding shim cannot, and **51 of the 233 sites are spelled that way**. A
 context resolved from an inline literal or a local `const` can be tested for
 `isSystem`; one arriving from a helper call cannot.
 
@@ -150,8 +150,8 @@ now **0**: nothing on this surface threads a context that provably lacks the fla
 
 **"No tenant context" counted sites it had not read.** An options argument the
 walker could not parse was folded into the same bucket as one it had read and
-found empty. That published **58 sites "carrying no tenant context at all"**
-when 8 said so and 50 were simply unread — an over-claim in the *alarming*
+found empty. That published **59 sites "carrying no tenant context at all"**
+when 8 said so and 51 were simply unread — an over-claim in the *alarming*
 direction, on the very figure this page tells other cards to cite. `carries` is
 now three-valued, and an unreadable argument can never contribute to the
 provable count.
@@ -187,10 +187,10 @@ reproduce them. Where it disagrees, it disagrees on the page:
 
 | carried figure | where it survives | this census |
 | :--- | :--- | ---: |
-| 175 write call sites | quoted in the merged changeset | **232** |
+| 175 write call sites | quoted in the merged changeset | **233** |
 | 24 carrying no tenant context | quoted in the merged changeset | **2** provable and tenancy-enabled; **29** more whose options argument is unreadable |
-| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **156 of 232** decidable, **76** undecidable |
-| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 131 decidably elevated, 0 decidably not, 93 undecidable |
+| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **156 of 233** decidable, **77** undecidable |
+| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 131 decidably elevated, 0 decidably not, 94 undecidable |
 | 141 and 132, two independent re-derivations | the card that filed this work | — |
 
 **The differences are not reconciled, and deliberately so.** The old census's
@@ -207,11 +207,11 @@ would report a smaller number and would not say so.
 
 The fourth row is the one worth flagging to anyone citing it. **The 135 / 77%
 figure has no surviving corroboration anywhere in the tree.** This census reads
-131 of 232 (56%) as decidably elevated, with 93 more whose elevation is a
+131 of 233 (56%) as decidably elevated, with 94 more whose elevation is a
 run-time fact — so the claim is neither confirmed nor refuted, and the honest
 answer is that a static reading cannot settle it.
 
-⇒ **Cite `2 / 232`, and say what it is**: the sites whose options argument was
+⇒ **Cite `2 / 233`, and say what it is**: the sites whose options argument was
 READ and holds no tenant context, against a decidably tenancy-enabled object.
 That is the control's provable yield surface. ⛔ Do not cite it as "the sites
 without tenant context" — **29 further sites** have an options argument this
@@ -223,31 +223,31 @@ cannot read, and they are neither in nor out.
 
 | what | count |
 | :--- | ---: |
-| write call sites on the application surface | **232** |
+| write call sites on the application surface | **233** |
 | …whose object name is statically decidable | 156 |
-| …whose object name is chosen at run time | 76 |
+| …whose object name is chosen at run time | 77 |
 | …against an object with tenancy ENABLED | 155 |
 | …against an object that declares tenancy off | 1 |
 | threading a tenant context | 174 |
 | PROVABLY carrying none (options read, no context key) | **8** |
 | …of those, against a decidably tenancy-enabled object | **2** |
-| options argument UNREADABLE — may or may not carry one | 50 |
+| options argument UNREADABLE — may or may not carry one | 51 |
 | …of those, against a decidably tenancy-enabled object | 29 |
 | threading a decidably ELEVATED (`isSystem`) context | 131 |
 | threading a context that is decidably NOT elevated | 0 |
-| threading a context whose elevation is a run-time fact | 93 |
+| threading a context whose elevation is a run-time fact | 94 |
 
 | how the instrument reached the site | count |
 | :--- | ---: |
-| receiver carried a readable engine type | 191 |
+| receiver carried a readable engine type | 192 |
 | receiver erased, placed by the object NAME | 31 |
 | receiver erased, placed by an `object: string` PARAMETER | 5 |
 | receiver erased, placed by an `UNTYPED_RECEIVERS` row | 5 |
 
 | object name spelled inline | 107 |
 | object name spelled through a `const` | 49 |
 | object name is an `object: string` parameter | 7 |
-| object name is some other run-time expression | 69 |
+| object name is some other run-time expression | 70 |
 
 ### Subtractions the census could NOT defend — enforced
 
@@ -297,13 +297,13 @@ holds still. They are required to be HERE and to say WHEN they were true;
 their values are not compared. The reasoning, and the measurement behind it,
 are in `scripts/check-tenant-audit-census.mjs`.
 
-Measured on 2026-10-20 at `300e9d436`.
+Measured on 2026-10-20 at `39fc914f7`.
 
 | corpus scale (not enforced) | count |
 | :--- | ---: |
-| tracked non-test sources scanned | 675 |
-| engine-shaped types recognised | 80 |
+| tracked non-test sources scanned | 681 |
+| engine-shaped types recognised | 82 |
 | declared objects in the registry | 118 |
-| same-named calls subtracted as non-engine | 166 |
+| same-named calls subtracted as non-engine | 169 |
 
 {/* END GENERATED: tenant-audit-census */}

docs/audits/2026-08-tenant-audit-write-call-sites.counts.md

@@ -33,19 +33,19 @@ silent, and `node scripts/tenant-audit-census.mjs --write` is the resolution.
 
 | Measure | Value |
 |---|---:|
-| Write call sites | 232 |
+| Write call sites | 233 |
 | Object name statically decidable | 156 |
-| Object name chosen at run time | 76 |
+| Object name chosen at run time | 77 |
 | Against a tenancy-enabled object | 155 |
 | Against an object declaring tenancy off | 1 |
 | Threading a tenant context | 174 |
 | Provably carrying none | 8 |
 | …and decidably tenancy-enabled | 2 |
-| Options argument unreadable | 50 |
+| Options argument unreadable | 51 |
 | …and decidably tenancy-enabled | 29 |
 | Threading a decidably elevated context | 131 |
 | Threading a decidably non-elevated context | 0 |
-| Threading a context of undecidable elevation | 93 |
+| Threading a context of undecidable elevation | 94 |
 
 ## Subtractions the census could NOT defend — enforced
 
@@ -90,14 +90,14 @@ holds still. They are required to be HERE and to say WHEN they were true;
 their values are not compared. The reasoning, and the measurement behind it,
 are in `scripts/check-tenant-audit-census.mjs`.
 
-Measured on 2026-10-20 at `300e9d436`.
+Measured on 2026-10-20 at `39fc914f7`.
 
 | corpus scale (not enforced) | count |
 | :--- | ---: |
-| tracked non-test sources scanned | 675 |
-| engine-shaped types recognised | 80 |
+| tracked non-test sources scanned | 681 |
+| engine-shaped types recognised | 82 |
 | declared objects in the registry | 118 |
-| same-named calls subtracted as non-engine | 166 |
+| same-named calls subtracted as non-engine | 169 |
 
 ## Every site
 
@@ -121,6 +121,7 @@ Measured on 2026-10-20 at `300e9d436`.
 | `packages/plugins/plugin-audit/src/auth-event-audit.ts` | `insert` | `sys_audit_log` | enabled | options unreadable | 1 |
 | `packages/plugins/plugin-audit/src/comment-parent-pair-backfill.ts` | `update` | `sys_comment` | enabled | elevated | 1 |
 | `packages/plugins/plugin-audit/src/mail-log-body-migration.ts` | `update` | `object` | undecidable | elevated | 1 |
+| `packages/plugins/plugin-audit/src/mail-log-body-migration.ts` | `update` | `object` | undecidable | options unreadable | 1 |
 | `packages/plugins/plugin-audit/src/read-audit.ts` | `insert` | `sys_audit_log` | enabled | options unreadable | 1 |
 | `packages/plugins/plugin-audit/src/stored-metadata-body-migration.ts` | `update` | `object` | undecidable | elevated | 1 |
 | `packages/plugins/plugin-auth/src/admin-import-users.ts` | `insert` | `sys_audit_log` | enabled | options unreadable | 1 |