Skip to content

Commit 8258181

Browse files
feat(service-mailbox): the inbound-mail ingest seam — a raw RFC 822 message becomes one sys_mail_message row, associated at ingest, deduplicated, automatic mail suppressed, re-matched on request (stage 1 of #22871) (#23250)
Part of #22871 Clause-②: yes (the package gains an exported ingest entry point it does not export today; a widening of the public surface) Stage 1 of 2 (triage `6137138392`): the ingest seam in `@objectstack/service-mailbox`. It reads one raw RFC 822 message and its target mailbox and writes `sys_mail_message` and `sys_mail_association` rows. There is no IMAP in this PR and nothing under `packages/connectors/**`. `connector-imap` is stage 2, a separate PR after this one lands, and #22871 remains open for it. Inline-forward parsing is #23215 (`Blocked-by: #22871`), which also remains open: no code path here reads a body's "Forwarded message" block. ## The entry points (the `Clause-②` surface) `MailboxServicePlugin` now registers **`MailIngestService` as the kernel service `mailbox`** (`MAILBOX_SERVICE`) in `init()`. That follows the sibling convention: `service-knowledge`, `service-settings`, `service-storage` and `service-messaging` each register their engine as a kernel service in `init()`. The engine is resolved on each call (`ctx.getService('objectql')`), never cached at construction. New exports from the package index: - values: `MailIngestService`, `MAILBOX_SERVICE`, `MailboxServiceError`, `MAIL_MATCH_ADDRESS_SOURCES`, `MAIL_SUPPRESSION_REASONS`; - types: `MailIngestInput`, `MailIngestResult`, `MailRematchInput`, `MailRematchResult`, `MailboxCallerContext`, `MailIngestEngine`, `MailIngestLogger`, `MailAssociationMatch`, `MailMatchAddress`, `MailMatchAddressSource`, `MailSuppressionReason`, `MailboxServiceErrorCode`, `RawMailMessage`. The pure steps (`parse-message`, `match-addresses`, `match-records`, `suppression`, `dedupe-key`) stay internal to `src/ingest/`. ### `ingest({ mailboxId, raw, receivedAt? }, context)` answers `stored`, `duplicate` or `suppressed` 1. **Who may call.** The system context only. A message enters a mailbox from its mail source, never from a user. Any other caller gets `PERMISSION_DENIED` (403) and nothing is written. 2. **Parse** (`postal-mime`, see below). Participants are the addresses of From, Sender, Reply-To, To and Cc, under source `outer`. The same five headers of each top-level `message/rfc822` part (RFC 2046 §5.2.1, forward as attachment) are read under source `attachment`. A forward nested inside the forwarded part is not read. The mailbox's own address is removed. Matching is ASCII case-insensitive: only ASCII letters are folded. 3. **Suppression** comes before anything is written. These signals are suppressed: - RFC 3834 `Auto-Submitted` with any value but `no`; - `Precedence: bulk`, `junk` or `list`; - `X-Autoreply` or `X-Autorespond` with a value other than `no`, `false` or `0`; - a top-level `multipart/report; report-type=delivery-status` (RFC 6522 and RFC 3464): a bounce. **A suppressed message writes no row.** That is what the object's fields express: `association_state` declares only `associated` and `unassociated` (measured in `sys-mail-message.object.ts`), so there is no suppressed state to write. 4. **Dedupe.** `dedupe_key` is `mid:` plus the SHA-256 of the Message-ID, brackets stripped. For a message without a Message-ID it is `hdr:` plus the SHA-256 of the header block as received. A pre-read of `(mailbox_id, dedupe_key)` answers `duplicate`. When that read misses a concurrent winner, the unique index refuses the second insert (`DUPLICATE_RECORD`), and the seam re-reads and answers `duplicate`. 5. **Match.** This is the seam of Q1 A. Every registered object with an `email` field declaring `mailAssociation: true` is read once per chunk of addresses, under the system context and filtered to the mailbox's `organization_id`. The read uses `$icontains`, which every driver executes with its own LIKE escaping. Exact equality is then decided on the value read back, so `bob@acme.test` never matches `jimbob@acme.test`. A flagged object without an organization column cannot hold one organization's records. It is skipped, with one warning. No resolver contract exists, and none is registered anywhere. 6. **Write**, in this order, every row a system write that carries the mailbox's organization (ADR-0131 D9): 1. the message, `unassociated`, with `match_addresses` as a JSON list of `{ address, source }`; 2. one `sys_mail_association` per matched record (`matched_by: 'ingest'`); 3. the flip to `associated`. A failure between steps leaves the message in the pool with whatever links landed, and a re-match completes it. Threading fills `message_id`, `in_reply_to` and `reference_ids` (RFC 5322 §3.6.4 ids, brackets stripped). Display columns are cut to their declared `maxLength` on whole addresses, and U+0000 never reaches a text column. ### `rematch({ mailboxId, messageIds? }, context)` - **Replay.** It walks the mailbox's messages with the shared `keysetWalk` (`@objectstack/types`) and replays each one's stored `match_addresses` (D2) against the records that exist now. - **Additive** (Q5 A). It adds the missing links (`matched_by: 'rematch'`), and a pool message moves to `associated` once it has one. It never deletes a link. - **Unreadable sets.** A stored set that is not the stored shape is reported in `unreadable` and never guessed at. - **Who may call** (Q4 A for this entry point). The system context; a caller in the mailbox's organization who holds `manage_mailboxes`; or the mailbox's owner (`sys_mailbox.owner_user_id`). - Anyone else in that organization gets `PERMISSION_DENIED` (403). - A mailbox in another organization answers `RECORD_NOT_FOUND` (404): another organization's mailbox does not exist for the caller. - The shape matches the sibling capability-held actions: `plugin-security`'s `assertDeploymentAuthority` (system passes; else `systemPermissions.includes(CAP)`; else a 403 `PERMISSION_DENIED`) and `service-datasource`'s admin routes. - The read gates (`mail-read-visibility.ts`, from #22870) already enforce the exemption and the owner limb for reads, and are unchanged. ## Zone 2 measurements 1. **What #22870 landed** is on `main` at `150da61aa`, read before the build, and this PR changes none of it: - the three objects, with `match_addresses`, `association_state`, `dedupe_key` and the unique `(mailbox_id, dedupe_key)` index; - both read gates; - `MAILBOX_ADMIN_CAPABILITY`; - the `mailbox` token (`CAPABILITY_PROVIDERS.mailbox`); - `FieldSchema.mailAssociation` and its refine. No object shape changed. 2. **Matching follows the ruling, not the card's older acceptance text.** The tests use fixture objects with `mailAssociation: true` email fields. No resolver is registered. Raw RFC 822 strings are fed straight to the seam. 3. **Q4 A.** See the re-match bullet above. Ingest is system-only rather than capability-gated, because a holder who could ingest could write a message in a correspondent's name onto every record that correspondent matches. 4. **MIME parsing.** One new runtime dependency, `postal-mime@^4.0.5`. Measured on the npm registry, 2026-10-13: | option | licence | dependencies | module builds | runtimes | size | |:--|:--|:--|:--|:--|:--| | `postal-mime` 4.0.5 (2026-10-07) | MIT-0 | 0 | ESM and CJS (`exports` import/require) | Node ≥ 20, browsers, Web Workers, Cloudflare Workers | 1,044,543 bytes unpacked (src, both builds, maps, d.ts) | | `mailparser` 3.9.38 | MIT | 10 direct (`he`, `tlds`, `libmime`, `iconv-lite`, `linkify-it`, `nodemailer`, `punycode.js`, `html-to-text`, `encoding-japanese`, `@zone-eu/mailsplit`) | CJS only (`main`, no `exports`) | Node streams only | 88,666 bytes, plus its tree | | hand-written subset | — | 0 | — | — | RFC 5322 unfolding, RFC 2047 B/Q words with charset decoding, the address-list grammar (quoted strings, comments, groups), the multipart walk, QP and base64 | - **Why `postal-mime`.** It has no transitive dependencies, a permissive licence, both module builds, and the edge runtimes. A hand-written parser would re-walk the edge cases `postal-mime` 4.0.1 to 4.0.5 fixed in eleven days, 18 parser corrections by its changelog. - **Why the 4.x line.** 3.0.1 (2026-09-25) fixed eight super-linear parse paths, and 4.0.5 adds the `maxPartCount` bound. 2.x has neither. - **The CJS build works.** A probe `require`d the built `dist/index.cjs` and ingested a message through it: the subject's encoded word decoded, the display name came through, and both addresses landed in the participant set. That covers esbuild's `__toESM` interop for the default export and for `addressParser`. - **Lockfile.** It changed only through `pnpm add`. The other additions are the workspace links `@objectstack/types` (runtime, already installed through `@objectstack/core`) and `@objectstack/objectql` and `@objectstack/driver-sql` (dev only, for the real-engine tests). `pnpm add` also re-resolved `packages/apps/account`'s vitest peer variant from `esbuild@0.28.1` to `0.28.2`, which dropped three stale snapshot blocks. That is pnpm's own normalization; the file was not edited by hand. 5. **Attachments are not stored. This is a deviation from the card's scope line.** - The surface exists: `sys_attachment` lives in `service-storage`, and `sys_mail_message` has `enable.files: true`. - Nothing on the server writes one: `git grep "insert('sys_attachment'"` gives 0 non-test hits. Attachment rows are created through the storage routes' upload flow. - Storing them needs a storage-side writer, which is outside this stage. No object was invented. - Consequence: a message forwarded as an attachment is stored with the forwarder's own text, and the forwarded message's body is not kept. - Carrier: a follow-up for the seat to file (dedupe words below), or stage 2. 6. **Dedupe, threading and suppression** are as above. The suppressed outcome is "no row", by measurement. 7. **How the engine is reached.** The kernel service `mailbox` plus the export. The plugin's module doc and the README now say what is true: the seam exists, and no mail source feeds it yet. ## Census: in-tree objects that will start associating once a mailbox exists **None.** `git grep mailAssociation -- examples packages` at `150da61aa`, `packages/spec` excluded, finds only classification tables, lint tests and this package's prose. No object declares the key. Control: `Field.email` appears in 6 files under `examples/`. The sibling `hotcrm` checkout at `0f84c9c` has 0 hits, with 2 `Field.email` files as the control. The tests carry their own fixture objects. ## Tests (at `0324d20f2`) `pnpm --filter @objectstack/service-mailbox test`: 6 files, 64 tests, green. `typecheck` is green and its program includes the test files (`tsc --listFiles` counts them). - `src/ingest/mail-ingest-service.test.ts` runs against a **real** `ObjectQL` over `driver-sql` and better-sqlite3 `:memory:`, with the real three objects and their unique indexes, and three fixture app objects: two flagged on differently named fields, and one email field left unflagged. It covers: - one row per forwarded message, associated through its `attachment` part; - dedupe, three ways: a second ingest, a message with no Message-ID, and a pre-read that misses the stored row; - the pool and the re-match; - additive re-match; - an unreadable stored set; - auto-reply and bounce suppression; - organization scope, an unflagged field, exact equality, and the mailbox's own address; - threading; - who may call (`code` plus `status`). - `src/mailbox-service-kernel.test.ts` boots a real `ObjectKernel` with `ObjectQLPlugin` and `MailboxServicePlugin`, and ingests through `kernel.getService('mailbox')`. - `src/ingest/parse-message.test.ts` and `src/ingest/suppression.test.ts` cover the pure steps: participants, groups and encoded words; a forward as attachment, a nested forward, and the inline control; the participant set's stored shape; each suppression signal with its negative control; the dedupe-key derivations. - `src/mailbox-service-plugin.test.ts` pins that the plugin registers exactly the `mailbox` service. ### Negative controls: each behaviour removed, by ablation Every mutation went through `scripts/ablation-replace.mjs`. The anchor had to hit exactly once, the blob change was proven on disk, a trap restored the file, and the restore was proven (`blob == HEAD`, `git diff HEAD` empty) every time. The three ingest test files ran against the mutated **source**: the tests import the seam by relative path, so no `dist/` is involved. These are one-off proofs, and no permanent test files were added for them. | # | behaviour removed | result | |:--|:--|:--| | A1 | dedupe (the key made random per call) | **red**, 5 tests: the second-ingest, header-block and missed-pre-read dedupe cases, and both key derivations. The first attempt was refused by the tool, because the replacement contained the anchor, so it never landed; it was re-anchored and run | | A2 | the dedupe pre-read | **green** (expected): the unique key plus the `DUPLICATE_RECORD` catch still hold one row | | A2b | the `DUPLICATE_RECORD` catch | **red**: the missed-pre-read case | | A3 | the pool (messages stored `associated`) | **red**: the pool and re-match case | | A4 | the re-match link write | **red**: both re-match cases | | A5 | suppression's early return | **red**: the auto-reply and bounce cases | | A6 | the bounce signal | **red**: the bounce case, unit and engine | | A7 | the `attachment` source | **red**: the forwarded-association case | | A8 | the organization filter | **red**: the organization-scope case | | A9 | the in-chunk equality filter alone | **green**. Exactness is held a second time by the per-address lookup in `linksFor`, so removing one of the two layers leaves the test green | | A9b | substring matching (both layers) | **red**: the organization-scope case, through `eviltwin@` | | A10 | the mailbox's own-address exclusion | **red**, 3 tests | | A11 | ingest's system-only gate | **red**: the ingest refusal | | A12 | re-match's organization scope | **red**: the other-organization 404 | | A13 | re-match's capability and owner requirement | **red**: the member refusal | A1 to A6 ran at `451638f33`. A2b, A7 to A13 and A9b ran at `a869f533e`. The test file changed between the two only by the missed-pre-read case. ## Gates - **Derived families.** `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` was run with no paths at the final head `0324d20f2` and printed 108 commands. Each was run with its exit code captured before any pipe: **108 × exit 0**. The `--ran` reconcile reads: "108 derived famil(ies) accounted for — 108 run, 0 NOT-MEASURED (a DERIVED zero — all 108 recorded an exit code and none of them is 3)". - The PM's lead list (79 commands at `150da61aa`) is a subset of this list. The extra families come from the census pages this PR edits (the doc families) and from the package's new reads: `check:engine-double-contract`, `check:where-matcher`, `check:objectql-double-limit`, `check:query-options-erasure`, `check:i18n`, `check:type-check-*`, `check:dispatcher-error-vocabulary` and `check:error-status-conformance`. - An earlier run at `ae14c39c5` found three gates red, now fixed: - `check:system-context-census` and `check:tenant-audit-census`, fixed in `6ad94f940`; - `check:query-options-erasure`, red on three test sites that erased their engine options to `any`, fixed in `0324d20f2`. - That run's `check:dual-build-cjs-loads` and `check:i18n` answered PREREQUISITE NOT MET (exit 3) on an unbuilt workspace. At the final head both ran and passed. - **Build.** - `pnpm turbo run build --filter=@objectstack/service-mailbox... --concurrency=2`: 16/16 tasks green at `ae14c39c5`. - `pnpm --filter @objectstack/service-mailbox build`: ESM, CJS and DTS built, `check-dts-emitted` 2/2. The CJS probe is under Zone 2 item 4. - **Lint, narrowed and proven.** `eslint --no-inline-config --format json` over this PR's 16 changed `.ts` files at `0324d20f2` reports 16 files, 0 errors, 0 warnings and 0 fatal. - The population comes from `eslint.config.mjs`'s `files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']`. The 16 files are the diff's whole lintable set; the rest of the diff is `.md`, `.mdx`, `.json` and `.yaml`. - The file count is read from the JSON report. - The config enables no type-aware linting (no `parserOptions.project`, as its own note records), so this diff cannot move a verdict on a file it does not touch. - The repo-wide `pnpm lint` is CI's. - **Control bytes.** `grep -naP` for control bytes over every changed file finds none. The one NUL in the code is the escape spelling inside a regex literal. ## Files outside the claim's surface, and why - `content/docs/permissions/system-context.mdx`: row 47b anchors `mail-ingest-service.ts#ingest` and `#authorizeRematch`. `check:system-context-census` refuses a file with elevation reads that no row anchors. #22870 added row 47 the same way. - `content/docs/permissions/tenant-audit-census.mdx` and `docs/audits/2026-08-tenant-audit-write-call-sites.counts.md`: the three new engine writes are regenerated with `node scripts/tenant-audit-census.mjs --write`, and the page's six hand-written counts move from 225 to 228. `check:tenant-audit-census` requires both. Before that, the `engine` locals were typed as `MailIngestEngine` so the census could place them. - `packages/services/service-mailbox/vitest.config.ts`: source aliases for `@objectstack/types`, `@objectstack/objectql` and `@objectstack/driver-sql`. `check:test-source-alias` requires them, and its ledger is shrink-only. ## Acceptance notes - **`mailAssociation` liveness stays `planned` + `authorWarn`.** The reader now exists (`MailRecordMatcher.match` in `src/ingest/match-records.ts`), but nothing feeds it mail until stage 2. The row's own note says to flip it "when the ingest lands", and the author-visible effect arrives with the connector, so the flip and the `authorHint` text ("no mailbox service reads this yet") are stage 2's. Carrier: stage 2 of #22871. - **Not suppressed:** - a `multipart/report` of another report type (an MDN read receipt), pinned as a control; - a `message/global` (RFC 6532) forwarded part. - **Matching reads with `$icontains`**, which no index serves. That costs one read per flagged field per 50 addresses, each bounded at 1,000 rows with a warning. - **Dedupe words** for the attachment follow-up: inbound mail attachments, sys_attachment writer, mail message attachment storage, forwarded message body. --- _Generated by [Claude Code](https://claude.ai/code/session_01BRQ9Mp1yH7R1az6MaUM18H)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 4f5679c commit 8258181

23 files changed

Lines changed: 2233 additions & 88 deletions
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
'@objectstack/service-mailbox': minor
3+
---
4+
5+
Inbound mail has an ingest seam: `MailIngestService`, the kernel service `mailbox`, turns one raw RFC 822 message into one `sys_mail_message` row, associates it at ingest with the records whose `mailAssociation` email fields hold a participant's address, and re-matches a mailbox on request
6+
7+
Clause-②: yes (the package gains an exported ingest entry point it does not export today; a widening of the public surface)
8+
9+
- **`ingest({ mailboxId, raw, receivedAt? }, context)`** reads one message and answers `stored`, `duplicate` or `suppressed`. Only the system context may call it: a message enters a mailbox from its mail source, never from a user.
10+
- **Participants.** The addresses of From, Sender, Reply-To, To and Cc, plus the same five headers of a message forwarded as an attachment (a `message/rfc822` part of the received message). The mailbox's own address is left out. The set is stored on the row in `match_addresses`, a JSON list of `{ address, source }` where `source` is `outer` or `attachment`. A forward quoted inline in the body is not read yet, and `inline` is declared so a later reader of it can store its addresses.
11+
- **Association.** The message is linked, one `sys_mail_association` row each (`matched_by: 'ingest'`), to every record of every object with an `email` field declaring `mailAssociation: true` whose value equals a participant address (ASCII case-insensitive), inside the mailbox's organization only. A flagged object with no organization column is skipped and logged once. No app code and no resolver service is involved. A message nothing matches is stored `association_state: 'unassociated'`: the restricted pool.
12+
- **Dedupe.** One row per message per mailbox. The key is `mid:` plus the SHA-256 of the Message-ID, or `hdr:` plus the SHA-256 of the header block as received for a message without one. A second fetch answers `duplicate` and writes nothing, and the unique key settles a concurrent pair.
13+
- **Threading.** `message_id`, `in_reply_to` and `reference_ids` hold the RFC 5322 ids with their angle brackets stripped.
14+
- **Suppression.** No row is written for an auto-reply (`Auto-Submitted` other than `no`, `Precedence: bulk`, `junk` or `list`, `X-Autoreply` or `X-Autorespond` other than `no`) or a bounce (`multipart/report; report-type=delivery-status`).
15+
- **`rematch({ mailboxId, messageIds? }, context)`** replays each message's stored participant set against the records that exist now. It adds the missing links (`matched_by: 'rematch'`) and moves a pool message to `associated` once it has one. It never removes a link. A stored set that is not the stored shape is reported in `unreadable` and not matched. The system context may run it, and so may the mailbox's owner and a `manage_mailboxes` holder in the mailbox's organization. Anyone else is refused with `PERMISSION_DENIED` (403), and another organization's mailbox answers `RECORD_NOT_FOUND` (404).
16+
- **New exports:** `MailIngestService`, `MAILBOX_SERVICE`, `MailboxServiceError` (`code` and `status`), `MAIL_MATCH_ADDRESS_SOURCES`, `MAIL_SUPPRESSION_REASONS`, and the types `MailIngestInput`, `MailIngestResult`, `MailRematchInput`, `MailRematchResult`, `MailboxCallerContext`, `MailIngestEngine`, `MailIngestLogger`, `MailAssociationMatch`, `MailMatchAddress`, `MailMatchAddressSource`, `MailSuppressionReason`, `MailboxServiceErrorCode` and `RawMailMessage`. `MailboxServicePlugin` registers the service in `init()`.
17+
- **New dependencies:** `postal-mime` (MIT-0, no dependencies of its own, ES module and CommonJS builds) reads the MIME structure, the encoded words and the address lists. `@objectstack/types`, already installed through `@objectstack/core`, supplies the keyset walk the re-match pages with.
18+
- **Not yet:** no mail source feeds the seam (the IMAP connector ships separately), and attachments are not stored, so a message forwarded as an attachment keeps the forwarder's own text and not the forwarded message's body.

‎content/docs/permissions/system-context.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,7 @@ One of the largest consumers of the flag;
170170
| 45 | Comment access hooks return early (insert + update + delete, and the read AST), and so do the activity and audit-log read gates (their read AST), the activity field redaction and the audit-log field redaction (the rows a read serves), and the query guard over both objects' value-bearing columns (the read AST) | plugin-audit | Get: the whole activity row and the whole `sys_audit_log` row, its before/after snapshots included, on `find` / `findOne` — the audit writer, each read gate's own pre-scan and the activity redaction's — and a filter, sort or grouping by those columns. Lose: comment visibility scoping, the narrowing of `sys_activity` and of `sys_audit_log` to rows whose parent record the caller can read, the redaction of a parent field's value from an activity row's text and recorded change and from a ledger row's before/after snapshots, the withholding of an activity row recording an update whose every changed field the reader is withheld (on `count` and `aggregate` too, so a total agrees with the rows), and the refusal of a query over those columns for a reader withheld a field of the objects it can reach | `packages/plugins/plugin-audit/src/comment-access-hooks.ts#installCommentAccessHooks`, `#installCommentReadVisibility`, `packages/plugins/plugin-audit/src/activity-read-visibility.ts#installActivityReadVisibility`, `packages/plugins/plugin-audit/src/audit-log-read-visibility.ts#installAuditLogReadVisibility`, `packages/plugins/plugin-audit/src/activity-field-redaction.ts#installActivityFieldRedaction`, `packages/plugins/plugin-audit/src/audit-log-field-redaction.ts#installAuditLogFieldRedaction`, `packages/plugins/plugin-audit/src/parent-field-query-guard.ts#installParentFieldQueryGuard` |
171171
| 46 | Knowledge search returns hits unfiltered | service-knowledge | Lose: the permission filter over search results | `packages/services/service-knowledge/src/knowledge-service.ts#applyPermissionFilter` |
172172
| 47 | The inbound-mail read gates return early (the read AST) | service-mailbox | Get: every `sys_mail_message` and `sys_mail_association` row on `find` / `findOne` — the ingest writer and each gate's own pre-scan, which read the association and mailbox rows a decision needs. Lose: the narrowing of a message to callers who can read a record it is associated with or who own its mailbox (so the unassociated pool is restricted), and of an association to callers who can read the record it names (on `count` and `aggregate` too, so a total agrees with the rows) | `packages/services/service-mailbox/src/mail-read-visibility.ts#gateMiddleware` |
173+
| 47b | The inbound-mail ingest seam: `ingest` admits the system context ALONE, and `rematch` admits it without the organization, owner or `manage_mailboxes` check | service-mailbox | Get: a mail source hands a mailbox its messages, and server code re-matches any mailbox. Lose: nothing a user held — every other caller of `ingest` is refused (`PERMISSION_DENIED`), because an ingested message is shown on every record its correspondents' addresses match and a user who could hand the seam a message could write one in a correspondent's name; every other caller of `rematch` must be in the mailbox's organization and own the mailbox or hold `manage_mailboxes` | `packages/services/service-mailbox/src/ingest/mail-ingest-service.ts#ingest`, `#authorizeRematch` |
173174

174175
### 5. Actions, metadata plane, provenance, the organization wall
175176

‎content/docs/permissions/tenant-audit-census.mdx‎

Lines changed: 17 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -122,7 +122,7 @@ are reported as `undecidable` rather than assumed either way.
122122

123123
The same holds twice over for the context. An options argument spelled as a
124124
literal can be read; one spelled `options`, `{ ...opts }`, or handed through a
125-
forwarding shim cannot, and **53 of the 225 sites are spelled that way**. A
125+
forwarding shim cannot, and **53 of the 228 sites are spelled that way**. A
126126
context resolved from an inline literal or a local `const` can be tested for
127127
`isSystem`; one arriving from a helper call cannot.
128128

@@ -187,10 +187,10 @@ reproduce them. Where it disagrees, it disagrees on the page:
187187

188188
| carried figure | where it survives | this census |
189189
| :--- | :--- | ---: |
190-
| 175 write call sites | quoted in the merged changeset | **225** |
190+
| 175 write call sites | quoted in the merged changeset | **228** |
191191
| 24 carrying no tenant context | quoted in the merged changeset | **2** provable and tenancy-enabled; **32** more whose options argument is unreadable |
192-
| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **155 of 225** decidable, **70** undecidable |
193-
| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 120 decidably elevated, 0 decidably not, 97 undecidable |
192+
| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **155 of 228** decidable, **73** undecidable |
193+
| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 123 decidably elevated, 0 decidably not, 97 undecidable |
194194
| 141 and 132, two independent re-derivations | the card that filed this work | — |
195195

196196
**The differences are not reconciled, and deliberately so.** The old census's
@@ -207,11 +207,11 @@ would report a smaller number and would not say so.
207207

208208
The fourth row is the one worth flagging to anyone citing it. **The 135 / 77%
209209
figure has no surviving corroboration anywhere in the tree.** This census reads
210-
120 of 225 (53%) as decidably elevated, with 97 more whose elevation is a
210+
123 of 228 (54%) as decidably elevated, with 97 more whose elevation is a
211211
run-time fact — so the claim is neither confirmed nor refuted, and the honest
212212
answer is that a static reading cannot settle it.
213213

214-
⇒ **Cite `2 / 225`, and say what it is**: the sites whose options argument was
214+
⇒ **Cite `2 / 228`, and say what it is**: the sites whose options argument was
215215
READ and holds no tenant context, against a decidably tenancy-enabled object.
216216
That is the control's provable yield surface. ⛔ Do not cite it as "the sites
217217
without tenant context" — **32 further sites** have an options argument this
@@ -223,31 +223,31 @@ cannot read, and they are neither in nor out.
223223

224224
| what | count |
225225
| :--- | ---: |
226-
| write call sites on the application surface | **225** |
226+
| write call sites on the application surface | **228** |
227227
| …whose object name is statically decidable | 155 |
228-
| …whose object name is chosen at run time | 70 |
228+
| …whose object name is chosen at run time | 73 |
229229
| …against an object with tenancy ENABLED | 154 |
230230
| …against an object that declares tenancy off | 1 |
231-
| threading a tenant context | 164 |
231+
| threading a tenant context | 167 |
232232
| PROVABLY carrying none (options read, no context key) | **8** |
233233
| …of those, against a decidably tenancy-enabled object | **2** |
234234
| options argument UNREADABLE — may or may not carry one | 53 |
235235
| …of those, against a decidably tenancy-enabled object | 32 |
236-
| threading a decidably ELEVATED (`isSystem`) context | 120 |
236+
| threading a decidably ELEVATED (`isSystem`) context | 123 |
237237
| threading a context that is decidably NOT elevated | 0 |
238238
| threading a context whose elevation is a run-time fact | 97 |
239239

240240
| how the instrument reached the site | count |
241241
| :--- | ---: |
242-
| receiver carried a readable engine type | 183 |
242+
| receiver carried a readable engine type | 186 |
243243
| receiver erased, placed by the object NAME | 30 |
244244
| receiver erased, placed by an `object: string` PARAMETER | 7 |
245245
| receiver erased, placed by an `UNTYPED_RECEIVERS` row | 5 |
246246

247247
| object name spelled inline | 103 |
248248
| object name spelled through a `const` | 52 |
249249
| object name is an `object: string` parameter | 9 |
250-
| object name is some other run-time expression | 61 |
250+
| object name is some other run-time expression | 64 |
251251

252252
### Subtractions the census could NOT defend — enforced
253253

@@ -297,13 +297,13 @@ holds still. They are required to be HERE and to say WHEN they were true;
297297
their values are not compared. The reasoning, and the measurement behind it,
298298
are in `scripts/check-tenant-audit-census.mjs`.
299299

300-
Measured on 2026-10-12 at `c60e08810`.
300+
Measured on 2026-10-13 at `ae14c39c5`.
301301

302302
| corpus scale (not enforced) | count |
303303
| :--- | ---: |
304-
| tracked non-test sources scanned | 629 |
305-
| engine-shaped types recognised | 71 |
306-
| declared objects in the registry | 117 |
307-
| same-named calls subtracted as non-engine | 161 |
304+
| tracked non-test sources scanned | 654 |
305+
| engine-shaped types recognised | 73 |
306+
| declared objects in the registry | 120 |
307+
| same-named calls subtracted as non-engine | 160 |
308308

309309
{/* END GENERATED: tenant-audit-census */}

‎docs/audits/2026-08-tenant-audit-write-call-sites.counts.md‎

Lines changed: 12 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -33,17 +33,17 @@ silent, and `node scripts/tenant-audit-census.mjs --write` is the resolution.
3333

3434
| Measure | Value |
3535
|---|---:|
36-
| Write call sites | 225 |
36+
| Write call sites | 228 |
3737
| Object name statically decidable | 155 |
38-
| Object name chosen at run time | 70 |
38+
| Object name chosen at run time | 73 |
3939
| Against a tenancy-enabled object | 154 |
4040
| Against an object declaring tenancy off | 1 |
41-
| Threading a tenant context | 164 |
41+
| Threading a tenant context | 167 |
4242
| Provably carrying none | 8 |
4343
| …and decidably tenancy-enabled | 2 |
4444
| Options argument unreadable | 53 |
4545
| …and decidably tenancy-enabled | 32 |
46-
| Threading a decidably elevated context | 120 |
46+
| Threading a decidably elevated context | 123 |
4747
| Threading a decidably non-elevated context | 0 |
4848
| Threading a context of undecidable elevation | 97 |
4949

@@ -90,14 +90,14 @@ holds still. They are required to be HERE and to say WHEN they were true;
9090
their values are not compared. The reasoning, and the measurement behind it,
9191
are in `scripts/check-tenant-audit-census.mjs`.
9292

93-
Measured on 2026-10-12 at `c60e08810`.
93+
Measured on 2026-10-13 at `ae14c39c5`.
9494

9595
| corpus scale (not enforced) | count |
9696
| :--- | ---: |
97-
| tracked non-test sources scanned | 629 |
98-
| engine-shaped types recognised | 71 |
99-
| declared objects in the registry | 117 |
100-
| same-named calls subtracted as non-engine | 161 |
97+
| tracked non-test sources scanned | 654 |
98+
| engine-shaped types recognised | 73 |
99+
| declared objects in the registry | 120 |
100+
| same-named calls subtracted as non-engine | 160 |
101101

102102
## Every site
103103

@@ -208,6 +208,9 @@ Measured on 2026-10-12 at `c60e08810`.
208208
| `packages/services/service-job/src/db-job-adapter.ts` | `update` | `sys_job` | enabled | elevated | 3 |
209209
| `packages/services/service-job/src/db-job-adapter.ts` | `insert` | `sys_job_run` | enabled | elevated | 1 |
210210
| `packages/services/service-job/src/db-job-adapter.ts` | `update` | `sys_job_run` | enabled | elevated | 1 |
211+
| `packages/services/service-mailbox/src/ingest/mail-ingest-service.ts` | `insert` | `MAIL_ASSOCIATION_OBJECT` | undecidable | elevated | 1 |
212+
| `packages/services/service-mailbox/src/ingest/mail-ingest-service.ts` | `insert` | `MAIL_MESSAGE_OBJECT` | undecidable | elevated | 1 |
213+
| `packages/services/service-mailbox/src/ingest/mail-ingest-service.ts` | `update` | `MAIL_MESSAGE_OBJECT` | undecidable | elevated | 1 |
211214
| `packages/services/service-messaging/src/inbox-channel.ts` | `insert` | `objectName` | undecidable | context, elevation undecidable | 1 |
212215
| `packages/services/service-messaging/src/inbox-channel.ts` | `insert` | `receiptObject` | undecidable | context, elevation undecidable | 1 |
213216
| `packages/services/service-messaging/src/messaging-service.ts` | `insert` | `RECEIPT_OBJECT` | undecidable | context, elevation undecidable | 1 |

‎packages/services/service-mailbox/README.md‎

Lines changed: 26 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
Inbound mail for ObjectStack: the mirrored message, its links to an app's records, and the read gates that keep a mailbox's unmatched mail restricted.
44

5-
> ⚠️ **Nothing ingests mail yet.** This release ships the objects and their read gates only. The IMAP connector and the ingest engine (deduplication, association stamping from the email fields an app flags with `mailAssociation`, and re-match) are not yet shipped, so no row is written by this package today.
5+
> ⚠️ **No mail source is connected yet.** This package ships the objects, their read gates and the ingest seam that writes them. The IMAP connector that fetches a mailbox's mail and hands each message to the seam is not yet shipped, so on a deployment today no message arrives unless server code calls the seam itself.
66
77
## What this package provides
88

@@ -14,6 +14,17 @@ Inbound mail for ObjectStack: the mirrored message, its links to an app's record
1414
- a **message** is served to a caller who can read a record it is associated with, to the owner of its mailbox, or to a holder of `manage_mailboxes`. An unassociated message has no record to read, so an ordinary member is refused it as not found (`404 RECORD_NOT_FOUND`): the pool is restricted, never organization-wide;
1515
- an **association** is served to a caller who can read the record it names, or to a holder of `manage_mailboxes`.
1616
- **`manage_mailboxes`** — an organization capability declared in `@objectstack/spec`. Platform administrators and organization administrators hold it by default; every other position only by explicit grant. Object grants and field-level security still apply to a holder.
17+
- **The ingest seam** — `MailIngestService`, registered as the kernel service `mailbox` (`MAILBOX_SERVICE`):
18+
- **`ingest({ mailboxId, raw, receivedAt? }, context)`** reads one raw RFC 822 message and writes at most one `sys_mail_message` row, answering `stored`, `duplicate` or `suppressed`:
19+
- **participants** are the addresses of From, Sender, Reply-To, To and Cc, plus the same headers of a message forwarded as an attachment (a `message/rfc822` part). The mailbox's own address is left out. They are stored on the row in `match_addresses`, each with its source (`outer` or `attachment`). A forward quoted inline in the body is not read yet;
20+
- **association** — every record of every object whose email field declares `mailAssociation: true` and holds one of those addresses (ASCII case-insensitive) is linked to the message, inside the mailbox's organization only, with one `sys_mail_association` row each (`matched_by: 'ingest'`). A message no record matches waits in the pool (`association_state: 'unassociated'`);
21+
- **dedupe** — one row per message per mailbox, keyed on the Message-ID (or, for a message without one, on its header block as received), so a second fetch answers `duplicate` and writes nothing;
22+
- **threading** — `message_id`, `in_reply_to` and `reference_ids` hold the ids of RFC 5322 §3.6.4, angle brackets stripped;
23+
- **suppression** — an auto-reply (`Auto-Submitted` other than `no`, `Precedence: bulk|junk|list`, `X-Autoreply`, `X-Autorespond`) or a bounce (`multipart/report; report-type=delivery-status`) is not stored at all.
24+
25+
Only the system context may ingest: a message enters a mailbox from its mail source, never from a user.
26+
- **`rematch({ mailboxId, messageIds? }, context)`** replays each message's stored participant set against the records that exist now and adds the links that are missing (`matched_by: 'rematch'`), moving a pool message to `associated` once it has one. It never removes a link. The system context may run it, and so may the mailbox's owner and a `manage_mailboxes` holder of the mailbox's organization; anyone else is refused (`PERMISSION_DENIED`), and another organization's mailbox is not found (`RECORD_NOT_FOUND`).
27+
- Attachments are not stored yet, so a message forwarded as an attachment is stored with the forwarder's own text, and the forwarded message's body is not kept.
1728

1829
## Usage
1930

@@ -26,4 +37,18 @@ export default defineStack({
2637
});
2738
```
2839

40+
Flag the email fields that identify a correspondent:
41+
42+
```typescript
43+
import { ObjectSchema, Field } from '@objectstack/spec/data';
44+
45+
export const Contact = ObjectSchema.create({
46+
name: 'crm_contact',
47+
label: 'Contact',
48+
fields: {
49+
email: Field.email({ label: 'Email', mailAssociation: true }),
50+
},
51+
});
52+
```
53+
2954
Grant object read on `sys_mail_message` and `sys_mail_association` to the positions that should see mail on the records they can read; grant `manage_mailboxes` to the people who administer mailboxes and their unassociated pool.

0 commit comments

Comments
 (0)