diff --git a/.changeset/22871-mail-ingest-seam.md b/.changeset/22871-mail-ingest-seam.md new file mode 100644 index 00000000000..c150d818835 --- /dev/null +++ b/.changeset/22871-mail-ingest-seam.md @@ -0,0 +1,18 @@ +--- +'@objectstack/service-mailbox': minor +--- + +Inbound mail has an ingest seam: `MailIngestService`, the kernel service `mailbox`, turns one raw RFC 822 message into one `sys_mail_message` row, associates it at ingest with the records whose `mailAssociation` email fields hold a participant's address, and re-matches a mailbox on request + +Clause-②: yes (the package gains an exported ingest entry point it does not export today; a widening of the public surface) + +- **`ingest({ mailboxId, raw, receivedAt? }, context)`** reads one message and answers `stored`, `duplicate` or `suppressed`. Only the system context may call it: a message enters a mailbox from its mail source, never from a user. + - **Participants.** The addresses of From, Sender, Reply-To, To and Cc, plus the same five headers of a message forwarded as an attachment (a `message/rfc822` part of the received message). The mailbox's own address is left out. The set is stored on the row in `match_addresses`, a JSON list of `{ address, source }` where `source` is `outer` or `attachment`. A forward quoted inline in the body is not read yet, and `inline` is declared so a later reader of it can store its addresses. + - **Association.** The message is linked, one `sys_mail_association` row each (`matched_by: 'ingest'`), to every record of every object with an `email` field declaring `mailAssociation: true` whose value equals a participant address (ASCII case-insensitive), inside the mailbox's organization only. A flagged object with no organization column is skipped and logged once. No app code and no resolver service is involved. A message nothing matches is stored `association_state: 'unassociated'`: the restricted pool. + - **Dedupe.** One row per message per mailbox. The key is `mid:` plus the SHA-256 of the Message-ID, or `hdr:` plus the SHA-256 of the header block as received for a message without one. A second fetch answers `duplicate` and writes nothing, and the unique key settles a concurrent pair. + - **Threading.** `message_id`, `in_reply_to` and `reference_ids` hold the RFC 5322 ids with their angle brackets stripped. + - **Suppression.** No row is written for an auto-reply (`Auto-Submitted` other than `no`, `Precedence: bulk`, `junk` or `list`, `X-Autoreply` or `X-Autorespond` other than `no`) or a bounce (`multipart/report; report-type=delivery-status`). +- **`rematch({ mailboxId, messageIds? }, context)`** replays each message's stored participant set against the records that exist now. It adds the missing links (`matched_by: 'rematch'`) and moves a pool message to `associated` once it has one. It never removes a link. A stored set that is not the stored shape is reported in `unreadable` and not matched. The system context may run it, and so may the mailbox's owner and a `manage_mailboxes` holder in the mailbox's organization. Anyone else is refused with `PERMISSION_DENIED` (403), and another organization's mailbox answers `RECORD_NOT_FOUND` (404). +- **New exports:** `MailIngestService`, `MAILBOX_SERVICE`, `MailboxServiceError` (`code` and `status`), `MAIL_MATCH_ADDRESS_SOURCES`, `MAIL_SUPPRESSION_REASONS`, and the types `MailIngestInput`, `MailIngestResult`, `MailRematchInput`, `MailRematchResult`, `MailboxCallerContext`, `MailIngestEngine`, `MailIngestLogger`, `MailAssociationMatch`, `MailMatchAddress`, `MailMatchAddressSource`, `MailSuppressionReason`, `MailboxServiceErrorCode` and `RawMailMessage`. `MailboxServicePlugin` registers the service in `init()`. +- **New dependencies:** `postal-mime` (MIT-0, no dependencies of its own, ES module and CommonJS builds) reads the MIME structure, the encoded words and the address lists. `@objectstack/types`, already installed through `@objectstack/core`, supplies the keyset walk the re-match pages with. +- **Not yet:** no mail source feeds the seam (the IMAP connector ships separately), and attachments are not stored, so a message forwarded as an attachment keeps the forwarder's own text and not the forwarded message's body. diff --git a/content/docs/permissions/system-context.mdx b/content/docs/permissions/system-context.mdx index 1ef41c9baa7..c1084bb229f 100644 --- a/content/docs/permissions/system-context.mdx +++ b/content/docs/permissions/system-context.mdx @@ -170,6 +170,7 @@ One of the largest consumers of the flag; | 45 | Comment access hooks return early (insert + update + delete, and the read AST), and so do the activity and audit-log read gates (their read AST), the activity field redaction and the audit-log field redaction (the rows a read serves), and the query guard over both objects' value-bearing columns (the read AST) | plugin-audit | Get: the whole activity row and the whole `sys_audit_log` row, its before/after snapshots included, on `find` / `findOne` — the audit writer, each read gate's own pre-scan and the activity redaction's — and a filter, sort or grouping by those columns. Lose: comment visibility scoping, the narrowing of `sys_activity` and of `sys_audit_log` to rows whose parent record the caller can read, the redaction of a parent field's value from an activity row's text and recorded change and from a ledger row's before/after snapshots, the withholding of an activity row recording an update whose every changed field the reader is withheld (on `count` and `aggregate` too, so a total agrees with the rows), and the refusal of a query over those columns for a reader withheld a field of the objects it can reach | `packages/plugins/plugin-audit/src/comment-access-hooks.ts#installCommentAccessHooks`, `#installCommentReadVisibility`, `packages/plugins/plugin-audit/src/activity-read-visibility.ts#installActivityReadVisibility`, `packages/plugins/plugin-audit/src/audit-log-read-visibility.ts#installAuditLogReadVisibility`, `packages/plugins/plugin-audit/src/activity-field-redaction.ts#installActivityFieldRedaction`, `packages/plugins/plugin-audit/src/audit-log-field-redaction.ts#installAuditLogFieldRedaction`, `packages/plugins/plugin-audit/src/parent-field-query-guard.ts#installParentFieldQueryGuard` | | 46 | Knowledge search returns hits unfiltered | service-knowledge | Lose: the permission filter over search results | `packages/services/service-knowledge/src/knowledge-service.ts#applyPermissionFilter` | | 47 | The inbound-mail read gates return early (the read AST) | service-mailbox | Get: every `sys_mail_message` and `sys_mail_association` row on `find` / `findOne` — the ingest writer and each gate's own pre-scan, which read the association and mailbox rows a decision needs. Lose: the narrowing of a message to callers who can read a record it is associated with or who own its mailbox (so the unassociated pool is restricted), and of an association to callers who can read the record it names (on `count` and `aggregate` too, so a total agrees with the rows) | `packages/services/service-mailbox/src/mail-read-visibility.ts#gateMiddleware` | +| 47b | The inbound-mail ingest seam: `ingest` admits the system context ALONE, and `rematch` admits it without the organization, owner or `manage_mailboxes` check | service-mailbox | Get: a mail source hands a mailbox its messages, and server code re-matches any mailbox. Lose: nothing a user held — every other caller of `ingest` is refused (`PERMISSION_DENIED`), because an ingested message is shown on every record its correspondents' addresses match and a user who could hand the seam a message could write one in a correspondent's name; every other caller of `rematch` must be in the mailbox's organization and own the mailbox or hold `manage_mailboxes` | `packages/services/service-mailbox/src/ingest/mail-ingest-service.ts#ingest`, `#authorizeRematch` | ### 5. Actions, metadata plane, provenance, the organization wall diff --git a/content/docs/permissions/tenant-audit-census.mdx b/content/docs/permissions/tenant-audit-census.mdx index 96542330024..7546a0b13d9 100644 --- a/content/docs/permissions/tenant-audit-census.mdx +++ b/content/docs/permissions/tenant-audit-census.mdx @@ -122,7 +122,7 @@ are reported as `undecidable` rather than assumed either way. The same holds twice over for the context. An options argument spelled as a literal can be read; one spelled `options`, `{ ...opts }`, or handed through a -forwarding shim cannot, and **53 of the 225 sites are spelled that way**. A +forwarding shim cannot, and **53 of the 228 sites are spelled that way**. A context resolved from an inline literal or a local `const` can be tested for `isSystem`; one arriving from a helper call cannot. @@ -187,10 +187,10 @@ reproduce them. Where it disagrees, it disagrees on the page: | carried figure | where it survives | this census | | :--- | :--- | ---: | -| 175 write call sites | quoted in the merged changeset | **225** | +| 175 write call sites | quoted in the merged changeset | **228** | | 24 carrying no tenant context | quoted in the merged changeset | **2** provable and tenancy-enabled; **32** more whose options argument is unreadable | -| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **155 of 225** decidable, **70** undecidable | -| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 120 decidably elevated, 0 decidably not, 97 undecidable | +| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **155 of 228** decidable, **73** undecidable | +| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 123 decidably elevated, 0 decidably not, 97 undecidable | | 141 and 132, two independent re-derivations | the card that filed this work | — | **The differences are not reconciled, and deliberately so.** The old census's @@ -207,11 +207,11 @@ would report a smaller number and would not say so. The fourth row is the one worth flagging to anyone citing it. **The 135 / 77% figure has no surviving corroboration anywhere in the tree.** This census reads -120 of 225 (53%) as decidably elevated, with 97 more whose elevation is a +123 of 228 (54%) as decidably elevated, with 97 more whose elevation is a run-time fact — so the claim is neither confirmed nor refuted, and the honest answer is that a static reading cannot settle it. -⇒ **Cite `2 / 225`, and say what it is**: the sites whose options argument was +⇒ **Cite `2 / 228`, and say what it is**: the sites whose options argument was READ and holds no tenant context, against a decidably tenancy-enabled object. That is the control's provable yield surface. ⛔ Do not cite it as "the sites without tenant context" — **32 further sites** have an options argument this @@ -223,23 +223,23 @@ cannot read, and they are neither in nor out. | what | count | | :--- | ---: | -| write call sites on the application surface | **225** | +| write call sites on the application surface | **228** | | …whose object name is statically decidable | 155 | -| …whose object name is chosen at run time | 70 | +| …whose object name is chosen at run time | 73 | | …against an object with tenancy ENABLED | 154 | | …against an object that declares tenancy off | 1 | -| threading a tenant context | 164 | +| threading a tenant context | 167 | | PROVABLY carrying none (options read, no context key) | **8** | | …of those, against a decidably tenancy-enabled object | **2** | | options argument UNREADABLE — may or may not carry one | 53 | | …of those, against a decidably tenancy-enabled object | 32 | -| threading a decidably ELEVATED (`isSystem`) context | 120 | +| threading a decidably ELEVATED (`isSystem`) context | 123 | | threading a context that is decidably NOT elevated | 0 | | threading a context whose elevation is a run-time fact | 97 | | how the instrument reached the site | count | | :--- | ---: | -| receiver carried a readable engine type | 183 | +| receiver carried a readable engine type | 186 | | receiver erased, placed by the object NAME | 30 | | receiver erased, placed by an `object: string` PARAMETER | 7 | | receiver erased, placed by an `UNTYPED_RECEIVERS` row | 5 | @@ -247,7 +247,7 @@ cannot read, and they are neither in nor out. | object name spelled inline | 103 | | object name spelled through a `const` | 52 | | object name is an `object: string` parameter | 9 | -| object name is some other run-time expression | 61 | +| object name is some other run-time expression | 64 | ### Subtractions the census could NOT defend — enforced @@ -297,13 +297,13 @@ holds still. They are required to be HERE and to say WHEN they were true; their values are not compared. The reasoning, and the measurement behind it, are in `scripts/check-tenant-audit-census.mjs`. -Measured on 2026-10-12 at `c60e08810`. +Measured on 2026-10-13 at `ae14c39c5`. | corpus scale (not enforced) | count | | :--- | ---: | -| tracked non-test sources scanned | 629 | -| engine-shaped types recognised | 71 | -| declared objects in the registry | 117 | -| same-named calls subtracted as non-engine | 161 | +| tracked non-test sources scanned | 654 | +| engine-shaped types recognised | 73 | +| declared objects in the registry | 120 | +| same-named calls subtracted as non-engine | 160 | {/* END GENERATED: tenant-audit-census */} diff --git a/docs/audits/2026-08-tenant-audit-write-call-sites.counts.md b/docs/audits/2026-08-tenant-audit-write-call-sites.counts.md index 6a61d0062ff..d7a1fe0e1a3 100644 --- a/docs/audits/2026-08-tenant-audit-write-call-sites.counts.md +++ b/docs/audits/2026-08-tenant-audit-write-call-sites.counts.md @@ -33,17 +33,17 @@ silent, and `node scripts/tenant-audit-census.mjs --write` is the resolution. | Measure | Value | |---|---:| -| Write call sites | 225 | +| Write call sites | 228 | | Object name statically decidable | 155 | -| Object name chosen at run time | 70 | +| Object name chosen at run time | 73 | | Against a tenancy-enabled object | 154 | | Against an object declaring tenancy off | 1 | -| Threading a tenant context | 164 | +| Threading a tenant context | 167 | | Provably carrying none | 8 | | …and decidably tenancy-enabled | 2 | | Options argument unreadable | 53 | | …and decidably tenancy-enabled | 32 | -| Threading a decidably elevated context | 120 | +| Threading a decidably elevated context | 123 | | Threading a decidably non-elevated context | 0 | | Threading a context of undecidable elevation | 97 | @@ -90,14 +90,14 @@ holds still. They are required to be HERE and to say WHEN they were true; their values are not compared. The reasoning, and the measurement behind it, are in `scripts/check-tenant-audit-census.mjs`. -Measured on 2026-10-12 at `c60e08810`. +Measured on 2026-10-13 at `ae14c39c5`. | corpus scale (not enforced) | count | | :--- | ---: | -| tracked non-test sources scanned | 629 | -| engine-shaped types recognised | 71 | -| declared objects in the registry | 117 | -| same-named calls subtracted as non-engine | 161 | +| tracked non-test sources scanned | 654 | +| engine-shaped types recognised | 73 | +| declared objects in the registry | 120 | +| same-named calls subtracted as non-engine | 160 | ## Every site @@ -208,6 +208,9 @@ Measured on 2026-10-12 at `c60e08810`. | `packages/services/service-job/src/db-job-adapter.ts` | `update` | `sys_job` | enabled | elevated | 3 | | `packages/services/service-job/src/db-job-adapter.ts` | `insert` | `sys_job_run` | enabled | elevated | 1 | | `packages/services/service-job/src/db-job-adapter.ts` | `update` | `sys_job_run` | enabled | elevated | 1 | +| `packages/services/service-mailbox/src/ingest/mail-ingest-service.ts` | `insert` | `MAIL_ASSOCIATION_OBJECT` | undecidable | elevated | 1 | +| `packages/services/service-mailbox/src/ingest/mail-ingest-service.ts` | `insert` | `MAIL_MESSAGE_OBJECT` | undecidable | elevated | 1 | +| `packages/services/service-mailbox/src/ingest/mail-ingest-service.ts` | `update` | `MAIL_MESSAGE_OBJECT` | undecidable | elevated | 1 | | `packages/services/service-messaging/src/inbox-channel.ts` | `insert` | `objectName` | undecidable | context, elevation undecidable | 1 | | `packages/services/service-messaging/src/inbox-channel.ts` | `insert` | `receiptObject` | undecidable | context, elevation undecidable | 1 | | `packages/services/service-messaging/src/messaging-service.ts` | `insert` | `RECEIPT_OBJECT` | undecidable | context, elevation undecidable | 1 | diff --git a/packages/services/service-mailbox/README.md b/packages/services/service-mailbox/README.md index 510621b67b6..a65cc113c8a 100644 --- a/packages/services/service-mailbox/README.md +++ b/packages/services/service-mailbox/README.md @@ -2,7 +2,7 @@ Inbound mail for ObjectStack: the mirrored message, its links to an app's records, and the read gates that keep a mailbox's unmatched mail restricted. -> ⚠️ **Nothing ingests mail yet.** This release ships the objects and their read gates only. The IMAP connector and the ingest engine (deduplication, association stamping from the email fields an app flags with `mailAssociation`, and re-match) are not yet shipped, so no row is written by this package today. +> ⚠️ **No mail source is connected yet.** This package ships the objects, their read gates and the ingest seam that writes them. The IMAP connector that fetches a mailbox's mail and hands each message to the seam is not yet shipped, so on a deployment today no message arrives unless server code calls the seam itself. ## What this package provides @@ -14,6 +14,17 @@ Inbound mail for ObjectStack: the mirrored message, its links to an app's record - a **message** is served to a caller who can read a record it is associated with, to the owner of its mailbox, or to a holder of `manage_mailboxes`. An unassociated message has no record to read, so an ordinary member is refused it as not found (`404 RECORD_NOT_FOUND`): the pool is restricted, never organization-wide; - an **association** is served to a caller who can read the record it names, or to a holder of `manage_mailboxes`. - **`manage_mailboxes`** — an organization capability declared in `@objectstack/spec`. Platform administrators and organization administrators hold it by default; every other position only by explicit grant. Object grants and field-level security still apply to a holder. +- **The ingest seam** — `MailIngestService`, registered as the kernel service `mailbox` (`MAILBOX_SERVICE`): + - **`ingest({ mailboxId, raw, receivedAt? }, context)`** reads one raw RFC 822 message and writes at most one `sys_mail_message` row, answering `stored`, `duplicate` or `suppressed`: + - **participants** are the addresses of From, Sender, Reply-To, To and Cc, plus the same headers of a message forwarded as an attachment (a `message/rfc822` part). The mailbox's own address is left out. They are stored on the row in `match_addresses`, each with its source (`outer` or `attachment`). A forward quoted inline in the body is not read yet; + - **association** — every record of every object whose email field declares `mailAssociation: true` and holds one of those addresses (ASCII case-insensitive) is linked to the message, inside the mailbox's organization only, with one `sys_mail_association` row each (`matched_by: 'ingest'`). A message no record matches waits in the pool (`association_state: 'unassociated'`); + - **dedupe** — one row per message per mailbox, keyed on the Message-ID (or, for a message without one, on its header block as received), so a second fetch answers `duplicate` and writes nothing; + - **threading** — `message_id`, `in_reply_to` and `reference_ids` hold the ids of RFC 5322 §3.6.4, angle brackets stripped; + - **suppression** — an auto-reply (`Auto-Submitted` other than `no`, `Precedence: bulk|junk|list`, `X-Autoreply`, `X-Autorespond`) or a bounce (`multipart/report; report-type=delivery-status`) is not stored at all. + + Only the system context may ingest: a message enters a mailbox from its mail source, never from a user. + - **`rematch({ mailboxId, messageIds? }, context)`** replays each message's stored participant set against the records that exist now and adds the links that are missing (`matched_by: 'rematch'`), moving a pool message to `associated` once it has one. It never removes a link. The system context may run it, and so may the mailbox's owner and a `manage_mailboxes` holder of the mailbox's organization; anyone else is refused (`PERMISSION_DENIED`), and another organization's mailbox is not found (`RECORD_NOT_FOUND`). + - Attachments are not stored yet, so a message forwarded as an attachment is stored with the forwarder's own text, and the forwarded message's body is not kept. ## Usage @@ -26,4 +37,18 @@ export default defineStack({ }); ``` +Flag the email fields that identify a correspondent: + +```typescript +import { ObjectSchema, Field } from '@objectstack/spec/data'; + +export const Contact = ObjectSchema.create({ + name: 'crm_contact', + label: 'Contact', + fields: { + email: Field.email({ label: 'Email', mailAssociation: true }), + }, +}); +``` + Grant object read on `sys_mail_message` and `sys_mail_association` to the positions that should see mail on the records they can read; grant `manage_mailboxes` to the people who administer mailboxes and their unassociated pool. diff --git a/packages/services/service-mailbox/package.json b/packages/services/service-mailbox/package.json index f8c4c3395a7..201bad5d308 100644 --- a/packages/services/service-mailbox/package.json +++ b/packages/services/service-mailbox/package.json @@ -26,10 +26,14 @@ "dependencies": { "@objectstack/core": "workspace:*", "@objectstack/platform-objects": "workspace:*", - "@objectstack/spec": "workspace:*" + "@objectstack/spec": "workspace:*", + "@objectstack/types": "workspace:*", + "postal-mime": "^4.0.5" }, "devDependencies": { + "@objectstack/driver-sql": "workspace:*", "@objectstack/lint": "workspace:*", + "@objectstack/objectql": "workspace:*", "@types/node": "^26.6.3", "typescript": "^6.0.3", "vitest": "^4.1.11" diff --git a/packages/services/service-mailbox/src/index.ts b/packages/services/service-mailbox/src/index.ts index d7bc61044d9..91b504c8ab2 100644 --- a/packages/services/service-mailbox/src/index.ts +++ b/packages/services/service-mailbox/src/index.ts @@ -15,3 +15,23 @@ export { type MailGateLogger, type MailReadMiddlewareCtx, } from './mail-read-visibility.js'; +export { + MAILBOX_SERVICE, + MailIngestService, + MailboxServiceError, + MAIL_MATCH_ADDRESS_SOURCES, + MAIL_SUPPRESSION_REASONS, + type MailAssociationMatch, + type MailIngestEngine, + type MailIngestLogger, + type MailIngestInput, + type MailIngestResult, + type MailMatchAddress, + type MailMatchAddressSource, + type MailRematchInput, + type MailRematchResult, + type MailSuppressionReason, + type MailboxCallerContext, + type MailboxServiceErrorCode, + type RawMailMessage, +} from './ingest/index.js'; diff --git a/packages/services/service-mailbox/src/ingest/dedupe-key.ts b/packages/services/service-mailbox/src/ingest/dedupe-key.ts new file mode 100644 index 00000000000..ff106ac6bba --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/dedupe-key.ts @@ -0,0 +1,36 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The dedupe key of one inbound message: the value of `sys_mail_message`'s + * `dedupe_key`, unique per mailbox (`(mailbox_id, dedupe_key)`), so a message + * fetched twice is one row. + * + * - `mid:` — of the Message-ID (angle brackets stripped, trimmed; + * RFC 5322 §3.6.4 makes it the message's identity), for a message that has + * one; + * - `hdr:` — of the header block exactly as received, for a + * message that has none. A second fetch of the same stored message hands + * over the same header block; two different messages without a Message-ID + * do not share one, because their `Received` / `Date` lines differ. + * + * The prefix says which derivation produced the key, so a reader of the row + * can tell a Message-ID identity from the fallback. Hashing keeps the key + * within the column's 128 characters whatever the input's length. + * + * SHA-256 is read from the Web Crypto global (`crypto.subtle`), which Node 20+, + * browsers and Workers all provide. + */ + +import type { ParsedInboundMessage } from './parse-message.js'; + +async function sha256Hex(text: string): Promise { + const digest = await globalThis.crypto.subtle.digest('SHA-256', new TextEncoder().encode(text)); + return [...new Uint8Array(digest)].map((b) => b.toString(16).padStart(2, '0')).join(''); +} + +/** The dedupe key of `message` (see the module header for the two derivations). */ +export async function dedupeKeyOf(message: Pick): Promise { + const id = message.messageId?.trim(); + if (id) return `mid:${await sha256Hex(id)}`; + return `hdr:${await sha256Hex(message.rawHeaderBlock)}`; +} diff --git a/packages/services/service-mailbox/src/ingest/index.ts b/packages/services/service-mailbox/src/ingest/index.ts new file mode 100644 index 00000000000..8a93ae06c2b --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/index.ts @@ -0,0 +1,24 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The inbound-mail ingest seam: raw RFC 822 message → `sys_mail_message` row + * and its `sys_mail_association` links, plus the explicit re-match. + */ + +export { + MAILBOX_SERVICE, + MailIngestService, + MailboxServiceError, + type MailIngestEngine, + type MailIngestLogger, + type MailIngestInput, + type MailIngestResult, + type MailRematchInput, + type MailRematchResult, + type MailboxCallerContext, + type MailboxServiceErrorCode, +} from './mail-ingest-service.js'; +export { MAIL_MATCH_ADDRESS_SOURCES, type MailMatchAddress, type MailMatchAddressSource } from './match-addresses.js'; +export type { MailAssociationMatch } from './match-records.js'; +export type { RawMailMessage } from './parse-message.js'; +export { MAIL_SUPPRESSION_REASONS, type MailSuppressionReason } from './suppression.js'; diff --git a/packages/services/service-mailbox/src/ingest/mail-ingest-service.test.ts b/packages/services/service-mailbox/src/ingest/mail-ingest-service.test.ts new file mode 100644 index 00000000000..11c0c9e3fec --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/mail-ingest-service.test.ts @@ -0,0 +1,459 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The ingest seam against the REAL engine: a real {@link ObjectQL} over + * `@objectstack/driver-sql` + better-sqlite3 `:memory:`, the real three mail + * objects with their real unique indexes, and fixture app objects whose email + * fields declare `mailAssociation`. A fake engine here would be written by the + * same hand as the assertions and would decide the very things measured — + * whether the dedupe key's unique index holds, whether `$icontains` folds case + * on both sides, whether a system write carrying an organization lands. + * + * No resolver is registered anywhere: the app's declaration is the whole seam. + * Every message is fed as a raw RFC 822 string; no IMAP is involved. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { ObjectQL } from '@objectstack/objectql'; +import { SqlDriver } from '@objectstack/driver-sql'; +import { ObjectSchema, Field } from '@objectstack/spec/data'; +// Engine options are DECLARED, never erased to `any` (`pnpm check:query-options-erasure`). +import type { DataEngineInsertOptions, EngineQueryOptions, EngineUpdateOptions } from '@objectstack/spec/data'; +import { SysMailAssociation, SysMailbox, SysMailMessage } from '../objects/index.js'; +import { MAILBOX_ADMIN_CAPABILITY } from '../mail-read-visibility.js'; +import { MailIngestService, MailboxServiceError, type MailIngestEngine } from './mail-ingest-service.js'; +import { parseMatchAddresses } from './match-addresses.js'; +import { bounce, forwardAsAttachment, plainMessage, rfc822 } from './messages.fixture.js'; + +const SYSTEM = { isSystem: true } as const; +const ORG_A = 'org_mail_a'; +const ORG_B = 'org_mail_b'; + +/** An app object whose correspondent field is flagged. */ +const Contact = ObjectSchema.create({ + name: 'mailtest_contact', + label: 'Contact', + fields: { + name: Field.text({ label: 'Name' }), + email: Field.email({ label: 'Email', mailAssociation: true }), + }, +}); +/** A second flagged object, flagged on a differently named field. */ +const Lead = ObjectSchema.create({ + name: 'mailtest_lead', + label: 'Lead', + fields: { + name: Field.text({ label: 'Name' }), + work_email: Field.email({ label: 'Work Email', mailAssociation: true }), + }, +}); +/** An email field that does NOT declare the flag: never matched. */ +const Vendor = ObjectSchema.create({ + name: 'mailtest_vendor', + label: 'Vendor', + fields: { + name: Field.text({ label: 'Name' }), + email: Field.email({ label: 'Email' }), + }, +}); + +type Row = Record; + +describe('the ingest seam on a real engine', () => { + let engine: ObjectQL; + let service: MailIngestService; + const warnings: string[] = []; + let mailboxSeq = 0; + + const find = (object: string, where: Row): Promise => + engine.find(object, { where, limit: 500, context: SYSTEM } satisfies EngineQueryOptions) as Promise; + const insert = async (object: string, data: Row): Promise => + String(((await engine.insert(object, data, { context: SYSTEM } satisfies DataEngineInsertOptions)) as Row).id); + + /** A fresh mailbox per test, so every count below is that test's own. */ + const newMailbox = async (organization = ORG_A, owner = 'usr_owner') => + insert('sys_mailbox', { + organization_id: organization, + name: `Inbox ${++mailboxSeq}`, + address: 'Inbox@Corp.test', + owner_user_id: owner, + }); + const messagesIn = (mailboxId: string) => find('sys_mail_message', { mailbox_id: mailboxId }); + const linksOf = (messageId: string) => find('sys_mail_association', { mail_message_id: messageId }); + + beforeAll(async () => { + engine = new ObjectQL(); + engine.registerDriver( + new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }), + true, + ); + await engine.init(); + for (const def of [SysMailbox, SysMailMessage, SysMailAssociation, Contact, Lead, Vendor]) { + engine.registry.registerObject(def as any, 'mailbox-test', 'mailbox-test'); + } + await engine.syncSchemas(); + service = new MailIngestService({ + engine: () => engine as unknown as MailIngestEngine, + logger: { warn: (m: string) => warnings.push(m) }, + }); + }); + + afterAll(async () => { + try { + await engine?.destroy(); + } catch { + /* noop */ + } + }); + + it('arms: every flagged object carries the organization column the matcher scopes by', () => { + for (const name of ['mailtest_contact', 'mailtest_lead', 'mailtest_vendor']) { + expect(engine.registry.getObject(name)?.fields?.organization_id, name).toBeTruthy(); + } + }); + + describe('a forwarded message lands as ONE row, associated through its forwarded part', () => { + it('stamps the association the attachment part names, with its source on the stored participant set', async () => { + const mailbox = await newMailbox(); + const contact = await insert('mailtest_contact', { organization_id: ORG_A, name: 'C', email: 'customer@ACME.test' }); + const original = rfc822( + [ + ['From', 'Customer '], + ['To', 'rep@corp.test'], + ['Message-ID', ''], + ], + 'What does it cost?', + ); + + const result = await service.ingest({ mailboxId: mailbox, raw: forwardAsAttachment({ original }) }, SYSTEM); + + expect(result.outcome).toBe('stored'); + const rows = await messagesIn(mailbox); + expect(rows).toHaveLength(1); + const row = rows[0]!; + expect(row.association_state).toBe('associated'); + expect(row.organization_id).toBe(ORG_A); + expect(row.from_address).toBe('rep@corp.test'); + expect(row.subject).toBe('Fwd: A question'); + expect(row.message_id).toBe('fwd-1@corp.test'); + expect(row.dedupe_key).toMatch(/^mid:[0-9a-f]{64}$/); + // The participant set as stored: the forwarder under `outer`, the + // forwarded message's people under `attachment`, the mailbox itself in + // neither. + expect(parseMatchAddresses(row.match_addresses)).toEqual([ + { address: 'rep@corp.test', source: 'outer' }, + { address: 'customer@acme.test', source: 'attachment' }, + { address: 'rep@corp.test', source: 'attachment' }, + ]); + const links = await linksOf(row.id); + expect(links.map((l) => [l.object_name, l.record_id, l.matched_field, l.matched_address, l.matched_by])).toEqual([ + ['mailtest_contact', contact, 'email', 'customer@acme.test', 'ingest'], + ]); + expect(links[0]!.organization_id).toBe(ORG_A); + }); + + it('control: the same forward with no forwarded part names only the forwarder, and pools', async () => { + const mailbox = await newMailbox(); + await insert('mailtest_contact', { organization_id: ORG_A, name: 'C', email: 'customer2@acme.test' }); + const result = await service.ingest( + { mailboxId: mailbox, raw: plainMessage({ From: 'Rep ', 'Message-ID': '' }) }, + SYSTEM, + ); + expect(result).toMatchObject({ outcome: 'stored', associationState: 'unassociated', associations: [] }); + }); + }); + + describe('dedupe', () => { + it('a second ingest of the same message writes nothing: the row count stays 1', async () => { + const mailbox = await newMailbox(); + await insert('mailtest_contact', { organization_id: ORG_A, name: 'C', email: 'dedupe@acme.test' }); + const raw = plainMessage({ From: 'dedupe@acme.test', 'Message-ID': '' }); + + const first = await service.ingest({ mailboxId: mailbox, raw }, SYSTEM); + const second = await service.ingest({ mailboxId: mailbox, raw }, SYSTEM); + + expect(first.outcome).toBe('stored'); + expect(second).toEqual({ outcome: 'duplicate', messageId: (first as { messageId: string }).messageId }); + const rows = await messagesIn(mailbox); + expect(rows).toHaveLength(1); + expect(await linksOf(rows[0]!.id)).toHaveLength(1); + }); + + it('a dedupe read that misses the stored row (a concurrent ingest’s window) is settled by the unique key', async () => { + const mailbox = await newMailbox(); + const raw = plainMessage({ From: 'race@acme.test', 'Message-ID': '' }); + const first = await service.ingest({ mailboxId: mailbox, raw }, SYSTEM); + expect(first.outcome).toBe('stored'); + + // The second ingest's dedupe read runs before the first one's insert is + // visible — modelled by that one read answering nothing. Every other + // call reaches the real engine unchanged. + let missed = false; + const racingEngine = new Proxy(engine, { + get(target, prop) { + if (prop === 'find') { + return async (object: string, options: Row) => { + if (!missed && object === 'sys_mail_message' && options?.where && 'dedupe_key' in options.where) { + missed = true; + return []; + } + return target.find(object, options as EngineQueryOptions); + }; + } + const value = Reflect.get(target, prop); + return typeof value === 'function' ? value.bind(target) : value; + }, + }); + const racing = new MailIngestService({ + engine: () => racingEngine as unknown as MailIngestEngine, + logger: { warn: (m: string) => warnings.push(m) }, + }); + + const second = await racing.ingest({ mailboxId: mailbox, raw }, SYSTEM); + expect(missed).toBe(true); + expect(second).toEqual({ outcome: 'duplicate', messageId: (first as { messageId: string }).messageId }); + expect(await messagesIn(mailbox)).toHaveLength(1); + }); + + it('a message with no Message-ID dedupes on its header block', async () => { + const mailbox = await newMailbox(); + const raw = plainMessage({ 'Message-ID': null, From: 'no-id@acme.test' }); + expect((await service.ingest({ mailboxId: mailbox, raw }, SYSTEM)).outcome).toBe('stored'); + expect((await service.ingest({ mailboxId: mailbox, raw }, SYSTEM)).outcome).toBe('duplicate'); + const rows = await messagesIn(mailbox); + expect(rows).toHaveLength(1); + expect(rows[0]!.dedupe_key).toMatch(/^hdr:/); + }); + + it('control: a different message is its own row, and the same message in another mailbox is that mailbox’s', async () => { + const mailbox = await newMailbox(); + const other = await newMailbox(); + const one = plainMessage({ 'Message-ID': '' }); + await service.ingest({ mailboxId: mailbox, raw: one }, SYSTEM); + await service.ingest({ mailboxId: mailbox, raw: plainMessage({ 'Message-ID': '' }) }, SYSTEM); + await service.ingest({ mailboxId: other, raw: one }, SYSTEM); + expect(await messagesIn(mailbox)).toHaveLength(2); + expect(await messagesIn(other)).toHaveLength(1); + }); + }); + + describe('the pool and the re-match', () => { + it('an unmatched message lands in the pool, and the re-match associates it once a matching record exists', async () => { + const mailbox = await newMailbox(); + const raw = plainMessage({ From: 'Prospect ', 'Message-ID': '' }); + + const stored = await service.ingest({ mailboxId: mailbox, raw }, SYSTEM); + expect(stored).toMatchObject({ outcome: 'stored', associationState: 'unassociated', associations: [] }); + const [pooled] = await messagesIn(mailbox); + expect(pooled!.association_state).toBe('unassociated'); + expect(await linksOf(pooled!.id)).toHaveLength(0); + + // Nothing to find yet: the re-match changes nothing. + expect(await service.rematch({ mailboxId: mailbox }, SYSTEM)).toEqual({ + scanned: 1, linksAdded: 0, associated: 0, unreadable: [], + }); + + const lead = await insert('mailtest_lead', { organization_id: ORG_A, name: 'P', work_email: 'prospect@NEW.test' }); + expect(await service.rematch({ mailboxId: mailbox }, SYSTEM)).toEqual({ + scanned: 1, linksAdded: 1, associated: 1, unreadable: [], + }); + const [after] = await messagesIn(mailbox); + expect(after!.association_state).toBe('associated'); + const links = await linksOf(after!.id); + expect(links.map((l) => [l.object_name, l.record_id, l.matched_field, l.matched_by])).toEqual([ + ['mailtest_lead', lead, 'work_email', 'rematch'], + ]); + + // Idempotent: a second re-match adds nothing. + expect(await service.rematch({ mailboxId: mailbox }, SYSTEM)).toMatchObject({ linksAdded: 0, associated: 0 }); + }); + + it('re-match only adds: a link whose record no longer matches is kept, and a new match is added beside it', async () => { + const mailbox = await newMailbox(); + const first = await insert('mailtest_contact', { organization_id: ORG_A, name: 'A', email: 'shared@acme.test' }); + await service.ingest( + { mailboxId: mailbox, raw: plainMessage({ From: 'shared@acme.test', 'Message-ID': '' }) }, + SYSTEM, + ); + await engine.update('mailtest_contact', { email: 'moved@acme.test' }, { + where: { id: first }, + context: SYSTEM, + } satisfies EngineUpdateOptions); + const second = await insert('mailtest_contact', { organization_id: ORG_A, name: 'B', email: 'shared@acme.test' }); + + expect(await service.rematch({ mailboxId: mailbox }, SYSTEM)).toMatchObject({ linksAdded: 1, associated: 0 }); + const [row] = await messagesIn(mailbox); + const links = (await linksOf(row!.id)).map((l) => [l.record_id, l.matched_by]); + expect(links).toEqual(expect.arrayContaining([[first, 'ingest'], [second, 'rematch']])); + expect(links).toHaveLength(2); + }); + + it('a stored participant set that is not the stored shape is reported, never guessed at', async () => { + const mailbox = await newMailbox(); + const id = await insert('sys_mail_message', { + organization_id: ORG_A, + mailbox_id: mailbox, + received_at: new Date().toISOString(), + association_state: 'unassociated', + dedupe_key: 'mid:broken', + match_addresses: 'customer@acme.test', + }); + expect(await service.rematch({ mailboxId: mailbox }, SYSTEM)).toEqual({ + scanned: 1, linksAdded: 0, associated: 0, unreadable: [id], + }); + }); + }); + + describe('suppression', () => { + it('an auto-reply writes no row', async () => { + const mailbox = await newMailbox(); + await insert('mailtest_contact', { organization_id: ORG_A, name: 'C', email: 'away@acme.test' }); + const result = await service.ingest( + { + mailboxId: mailbox, + raw: plainMessage({ From: 'away@acme.test', 'Auto-Submitted': 'auto-replied', 'Message-ID': '' }), + }, + SYSTEM, + ); + expect(result).toEqual({ outcome: 'suppressed', reason: 'auto_submitted' }); + expect(await messagesIn(mailbox)).toHaveLength(0); + }); + + it('control: the same message without the auto-reply marker is stored and associated', async () => { + const mailbox = await newMailbox(); + await insert('mailtest_contact', { organization_id: ORG_A, name: 'C', email: 'present@acme.test' }); + const result = await service.ingest( + { mailboxId: mailbox, raw: plainMessage({ From: 'present@acme.test', 'Message-ID': '' }) }, + SYSTEM, + ); + expect(result).toMatchObject({ outcome: 'stored', associationState: 'associated' }); + expect(await messagesIn(mailbox)).toHaveLength(1); + }); + + it('a bounce writes no row', async () => { + const mailbox = await newMailbox(); + expect(await service.ingest({ mailboxId: mailbox, raw: bounce() }, SYSTEM)).toEqual({ + outcome: 'suppressed', + reason: 'delivery_report', + }); + expect(await messagesIn(mailbox)).toHaveLength(0); + }); + }); + + describe('matching reads the declaration, inside the mailbox’s organization', () => { + it('matches a flagged field only, in the mailbox’s own organization only', async () => { + const mailbox = await newMailbox(ORG_A); + const own = await insert('mailtest_contact', { organization_id: ORG_A, name: 'Own', email: 'twin@acme.test' }); + await insert('mailtest_contact', { organization_id: ORG_B, name: 'Other org', email: 'twin@acme.test' }); + await insert('mailtest_vendor', { organization_id: ORG_A, name: 'Unflagged', email: 'twin@acme.test' }); + // A substring of a stored value is not that value. + await insert('mailtest_contact', { organization_id: ORG_A, name: 'Longer', email: 'eviltwin@acme.test' }); + + const result = await service.ingest( + { mailboxId: mailbox, raw: plainMessage({ From: 'Twin ', 'Message-ID': '' }) }, + SYSTEM, + ); + expect(result).toMatchObject({ + outcome: 'stored', + associations: [{ object: 'mailtest_contact', recordId: own, matchedField: 'email', matchedAddress: 'twin@acme.test' }], + }); + }); + + it('never matches the mailbox’s own address, whatever its case', async () => { + const mailbox = await newMailbox(ORG_A); + await insert('mailtest_contact', { organization_id: ORG_A, name: 'Inbox as contact', email: 'inbox@corp.test' }); + const result = await service.ingest( + { mailboxId: mailbox, raw: plainMessage({ From: 'stranger@far.test', 'Message-ID': '' }) }, + SYSTEM, + ); + expect(result).toMatchObject({ outcome: 'stored', associationState: 'unassociated' }); + }); + + it('stores threading ids and display columns', async () => { + const mailbox = await newMailbox(ORG_A); + await service.ingest( + { + mailboxId: mailbox, + raw: plainMessage({ + 'Message-ID': '', + 'In-Reply-To': '', + References: ' ', + Cc: 'a@acme.test, B ', + }), + receivedAt: '2026-10-12T11:00:00.000Z', + }, + SYSTEM, + ); + const [row] = await messagesIn(mailbox); + expect(row).toMatchObject({ + in_reply_to: 'reply-1@acme.test', + reference_ids: 'root@acme.test reply-1@acme.test', + to_addresses: 'inbox@corp.test', + cc_addresses: 'a@acme.test, b@acme.test', + from_name: 'Customer', + }); + expect(new Date(row!.received_at).toISOString()).toBe('2026-10-12T11:00:00.000Z'); + expect(new Date(row!.sent_at).toISOString()).toBe('2026-10-12T09:00:00.000Z'); + }); + }); + + describe('who may call', () => { + const refusal = async (run: () => Promise) => { + try { + await run(); + } catch (err) { + expect(err).toBeInstanceOf(MailboxServiceError); + const e = err as MailboxServiceError; + return { code: e.code, status: e.status }; + } + throw new Error('expected a refusal'); + }; + + it('ingest is refused to anything but the system context, and writes nothing', async () => { + const mailbox = await newMailbox(ORG_A, 'usr_owner'); + const asOwnerHolder = { userId: 'usr_owner', tenantId: ORG_A, systemPermissions: [MAILBOX_ADMIN_CAPABILITY] }; + expect(await refusal(() => service.ingest({ mailboxId: mailbox, raw: plainMessage() }, asOwnerHolder))).toEqual({ + code: 'PERMISSION_DENIED', + status: 403, + }); + expect(await messagesIn(mailbox)).toHaveLength(0); + }); + + it('re-match is served to the mailbox’s owner and to a manage_mailboxes holder of its organization', async () => { + const mailbox = await newMailbox(ORG_A, 'usr_owner'); + expect(await service.rematch({ mailboxId: mailbox }, { userId: 'usr_owner', tenantId: ORG_A })).toMatchObject({ scanned: 0 }); + expect( + await service.rematch( + { mailboxId: mailbox }, + { userId: 'usr_admin', tenantId: ORG_A, systemPermissions: [MAILBOX_ADMIN_CAPABILITY] }, + ), + ).toMatchObject({ scanned: 0 }); + }); + + it('re-match is refused to a member who neither owns the mailbox nor holds the capability', async () => { + const mailbox = await newMailbox(ORG_A, 'usr_owner'); + expect( + await refusal(() => + service.rematch({ mailboxId: mailbox }, { userId: 'usr_member', tenantId: ORG_A, systemPermissions: ['setup.access'] }), + ), + ).toEqual({ code: 'PERMISSION_DENIED', status: 403 }); + }); + + it('another organization’s mailbox is not found, even by a capability holder; nor is a mailbox that does not exist', async () => { + const mailbox = await newMailbox(ORG_A, 'usr_owner'); + const holderElsewhere = { userId: 'usr_owner', tenantId: ORG_B, systemPermissions: [MAILBOX_ADMIN_CAPABILITY] }; + expect(await refusal(() => service.rematch({ mailboxId: mailbox }, holderElsewhere))).toEqual({ + code: 'RECORD_NOT_FOUND', + status: 404, + }); + expect(await refusal(() => service.rematch({ mailboxId: 'no_such_mailbox' }, SYSTEM))).toEqual({ + code: 'RECORD_NOT_FOUND', + status: 404, + }); + expect(await refusal(() => service.ingest({ mailboxId: 'no_such_mailbox', raw: plainMessage() }, SYSTEM))).toEqual({ + code: 'RECORD_NOT_FOUND', + status: 404, + }); + }); + }); +}); diff --git a/packages/services/service-mailbox/src/ingest/mail-ingest-service.ts b/packages/services/service-mailbox/src/ingest/mail-ingest-service.ts new file mode 100644 index 00000000000..1c5718a717f --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/mail-ingest-service.ts @@ -0,0 +1,558 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The ingest seam: one raw RFC 822 message into one mailbox becomes one + * `sys_mail_message` row and its `sys_mail_association` links — and the + * explicit re-match that adds the links a later record makes possible. + * + * Decisions this module carries out (design record on #8998, ruling 6116489638 + * on #22870): + * + * - **Mirror, not adopt** (decision 1). The row holds what the message says: + * participants, threading ids, subject, the text and HTML bodies as sent. + * - **Associated at ingest, re-matched explicitly** (decision 2). The matcher + * ({@link MailRecordMatcher}) runs once, at ingest, over the participant set, + * and the set is stored on the row (`match_addresses`, with each address's + * source), so {@link MailIngestService.rematch} replays exactly that input. + * Re-match only ADDS links (Q5 A): a link is a recorded fact, never + * silently withdrawn. + * - **Unmatched mail waits in the pool** (decision 3): a message no record + * matches is stored `association_state: 'unassociated'`, never dropped. The + * read gates keep the pool to the mailbox's owner and `manage_mailboxes` + * holders (`mail-read-visibility.ts`). + * - **Forward as attachment is read** (Q3 C, phase 1): the headers of a + * `message/rfc822` part are participants too, under source `attachment`. + * + * ## Writes, in order + * + * 1. the message row, `unassociated`; + * 2. one link per matched record (`matched_by: 'ingest'`); + * 3. `association_state: 'associated'`, once its links exist. + * + * A failure between them leaves the message in the pool with the links that + * landed — visible to the mailbox's owner and administrators — and a re-match + * completes it. The failure itself reaches the caller. + * + * Every write is a system write that carries the mailbox's organization + * (ADR-0131 D9), and every match is read inside it. + * + * ## Who may call + * + * - `ingest`: system context only. A message enters a mailbox from its mail + * source; a user who could hand the seam a message could put words in a + * correspondent's mouth on every record that correspondent matches. + * - `rematch`: system context, or a caller in the mailbox's organization who + * holds `manage_mailboxes` or owns the mailbox — the same people the pool's + * read gate serves (Q4 A). A mailbox in another organization is not found. + */ + +import { keysetWalk } from '@objectstack/types'; +import { + MAILBOX_ADMIN_CAPABILITY, + MAILBOX_OBJECT, + MAIL_ASSOCIATION_OBJECT, + MAIL_MESSAGE_OBJECT, +} from '../mail-read-visibility.js'; +import { dedupeKeyOf } from './dedupe-key.js'; +import { + collectMatchAddresses, + parseMatchAddresses, + serializeMatchAddresses, + type MailMatchAddress, +} from './match-addresses.js'; +import { + MailRecordMatcher, + linksFor, + type MailAssociationMatch, + type MailMatchEngine, +} from './match-records.js'; +import { parseInboundMessage, participantAddresses, type RawMailMessage } from './parse-message.js'; +import { suppressionReasonOf, type MailSuppressionReason } from './suppression.js'; + +/** The kernel service name the mailbox plugin registers {@link MailIngestService} under. */ +export const MAILBOX_SERVICE = 'mailbox'; + +/** The engine surface the seam reads and writes through. */ +export interface MailIngestEngine extends MailMatchEngine { + insert(object: string, data: Record, options: Record): Promise; + update(object: string, data: Record, options: Record): Promise; +} + +/** The logger surface the seam reports through. */ +export interface MailIngestLogger { + warn(message: string): void; +} + +/** + * The caller of an entry point, as the request's execution context carries + * it: `isSystem`, the user, the active organization (`tenantId`) and the + * resolved capability set (`systemPermissions`). + */ +export interface MailboxCallerContext { + isSystem?: boolean; + userId?: unknown; + tenantId?: unknown; + systemPermissions?: unknown; +} + +/** One message handed to the seam. */ +export interface MailIngestInput { + /** The `sys_mailbox` the message arrived in. */ + mailboxId: string; + /** The message as received (RFC 822 / RFC 5322). */ + raw: RawMailMessage; + /** + * When the mailbox received it (the IMAP INTERNALDATE, RFC 3501), as an ISO + * timestamp or a `Date`. Absent: the time of the ingest. + */ + receivedAt?: string | Date; +} + +/** What one ingest did. */ +export type MailIngestResult = + | { + outcome: 'stored'; + /** The new `sys_mail_message` row. */ + messageId: string; + associationState: 'associated' | 'unassociated'; + /** One per `sys_mail_association` row written; empty for a pool message. */ + associations: MailAssociationMatch[]; + } + | { + /** The mailbox already holds this message (same dedupe key); nothing was written. */ + outcome: 'duplicate'; + messageId: string; + } + | { + /** An auto-reply or a bounce: nothing was written. */ + outcome: 'suppressed'; + reason: MailSuppressionReason; + }; + +/** What to re-match. */ +export interface MailRematchInput { + mailboxId: string; + /** Only these messages of the mailbox. Absent: every message of the mailbox. */ + messageIds?: string[]; +} + +/** What one re-match did. */ +export interface MailRematchResult { + /** Messages read. */ + scanned: number; + /** Links added (`matched_by: 'rematch'`). */ + linksAdded: number; + /** Messages that left the pool (`unassociated` → `associated`). */ + associated: number; + /** Messages whose stored `match_addresses` is not the stored shape: not matched, never guessed. */ + unreadable: string[]; +} + +/** The refusal codes the entry points answer with (ADR-0112 standard members). */ +export type MailboxServiceErrorCode = 'PERMISSION_DENIED' | 'RECORD_NOT_FOUND' | 'VALIDATION_ERROR' | 'INTERNAL_ERROR'; + +const STATUS_OF: Record = { + PERMISSION_DENIED: 403, + RECORD_NOT_FOUND: 404, + VALIDATION_ERROR: 400, + INTERNAL_ERROR: 500, +}; + +/** A refusal of an entry point, carrying its ADR-0112 `code` and HTTP `status`. */ +export class MailboxServiceError extends Error { + readonly status: number; + readonly statusCode: number; + constructor(readonly code: MailboxServiceErrorCode, message: string) { + super(message); + this.name = 'MailboxServiceError'; + this.status = STATUS_OF[code]; + this.statusCode = this.status; + } +} + +/** Messages read per re-match page. */ +export const REMATCH_PAGE_SIZE = 200; +/** Existing links read per re-match page. */ +export const REMATCH_LINK_SCAN_LIMIT = 5_000; + +const SYSTEM_CTX = { isSystem: true } as const; +const DUPLICATE_RECORD = 'DUPLICATE_RECORD'; + +/** Display-column bounds (`sys_mail_message`'s declared `maxLength`s). */ +const BOUND = { + message_id: 998, + subject: 998, + from_address: 320, + from_name: 255, + sender_address: 320, + reply_to: 4000, + to_addresses: 4000, + cc_addresses: 4000, +} as const; + +interface MailboxRow { + id: string; + address: string | null; + ownerUserId: string | null; + organizationId: string | null; +} + +const idOf = (value: unknown): string | null => + (typeof value === 'string' && value !== '') || typeof value === 'number' ? String(value) : null; + +/** U+0000 never reaches a text column (a SQL text type refuses it). */ +const storable = (value: string): string => value.replace(/\u0000/g, ''); + +/** `value` cut to `max` UTF-16 units, never inside a surrogate pair. */ +function fit(value: string | null | undefined, max: number): string | null { + if (value === null || value === undefined) return null; + const clean = storable(value); + if (clean.length <= max) return clean; + let end = max; + const last = clean.charCodeAt(end - 1); + if (last >= 0xd800 && last <= 0xdbff) end--; + return clean.slice(0, end); +} + +/** Addresses joined with `, `, whole addresses only, within `max`. */ +function joinAddresses(addresses: ReadonlyArray, max: number): string | null { + let out = ''; + for (const raw of addresses) { + const address = storable(raw); + const next = out ? `${out}, ${address}` : address; + if (next.length > max) break; + out = next; + } + return out === '' ? null : out; +} + +function isoOrNull(value: string | Date | undefined): string | null { + if (value === undefined) return null; + const at = value instanceof Date ? value : new Date(value); + return Number.isNaN(at.getTime()) ? null : at.toISOString(); +} + +function isDuplicateRecord(err: unknown): boolean { + return typeof err === 'object' && err !== null && (err as { code?: unknown }).code === DUPLICATE_RECORD; +} + +function distinctAddresses(addresses: ReadonlyArray): string[] { + return [...new Set(addresses.map((a) => a.address))]; +} + +/** + * The ingest seam and the re-match action. The engine is resolved on each + * call, never cached at construction, so a plugin may build this before the + * engine's own registration completes. + */ +export class MailIngestService { + private readonly matcher: MailRecordMatcher; + + constructor( + private readonly options: { + /** The data engine (`objectql`), resolved per call. */ + engine: () => MailIngestEngine; + logger: MailIngestLogger; + /** The ingest clock, for a message with no `receivedAt`. */ + now?: () => Date; + }, + ) { + this.matcher = new MailRecordMatcher(options.engine, options.logger); + } + + /** + * Ingest one raw message into one mailbox: suppress it, find it already + * stored, or store it with its associations. System context only. + */ + async ingest(input: MailIngestInput, context: MailboxCallerContext): Promise { + if (context?.isSystem !== true) { + throw new MailboxServiceError( + 'PERMISSION_DENIED', + 'Ingesting mail requires the system context: a message enters a mailbox from its mail source, never ' + + "from a user, because an ingested message is shown on every record its correspondents' addresses match.", + ); + } + if (typeof input?.mailboxId !== 'string' || input.mailboxId === '') { + throw new MailboxServiceError('VALIDATION_ERROR', 'Ingesting mail requires the id of the mailbox it arrived in.'); + } + const receivedAt = input.receivedAt === undefined ? (this.options.now?.() ?? new Date()).toISOString() : isoOrNull(input.receivedAt); + if (receivedAt === null) { + throw new MailboxServiceError('VALIDATION_ERROR', '`receivedAt` must be a valid timestamp or Date when it is given.'); + } + + const engine: MailIngestEngine = this.options.engine(); + const mailbox = await this.mailbox(engine, input.mailboxId); + const organizationId = this.organizationOf(mailbox); + + let parsed; + try { + parsed = await parseInboundMessage(input.raw); + } catch (err) { + throw new MailboxServiceError( + 'VALIDATION_ERROR', + `The message for mailbox '${mailbox.id}' could not be read as an RFC 822 message: ${(err as Error)?.message ?? err}`, + ); + } + if (parsed.unreadForwardedParts > 0) { + this.options.logger.warn( + `[mailbox] a message for mailbox '${mailbox.id}' carries ${parsed.unreadForwardedParts} forwarded part(s) that ` + + 'could not be read; their addresses take no part in matching, and the message is stored without them.', + ); + } + + const reason = suppressionReasonOf(parsed); + if (reason !== null) return { outcome: 'suppressed', reason }; + + const dedupeKey = await dedupeKeyOf(parsed); + const existing = await this.storedId(engine, mailbox.id, dedupeKey); + if (existing !== null) return { outcome: 'duplicate', messageId: existing }; + + const matchAddresses = collectMatchAddresses( + [ + { source: 'outer', addresses: participantAddresses(parsed.outer) }, + ...parsed.forwarded.map((part) => ({ source: 'attachment' as const, addresses: participantAddresses(part) })), + ], + mailbox.address ? [mailbox.address] : [], + ); + const addresses = distinctAddresses(matchAddresses); + const links = linksFor(addresses, await this.matcher.match(organizationId, addresses)); + + const outer = parsed.outer; + const row: Record = { + organization_id: organizationId, + mailbox_id: mailbox.id, + message_id: fit(parsed.messageId, BOUND.message_id), + in_reply_to: parsed.inReplyTo.length ? storable(parsed.inReplyTo.join(' ')) : null, + reference_ids: parsed.references.length ? storable(parsed.references.join(' ')) : null, + subject: fit(parsed.subject, BOUND.subject), + from_address: fit(outer.from[0], BOUND.from_address), + from_name: fit(parsed.fromName, BOUND.from_name), + sender_address: fit(outer.sender[0], BOUND.sender_address), + reply_to: joinAddresses(outer.replyTo, BOUND.reply_to), + to_addresses: joinAddresses(outer.to, BOUND.to_addresses), + cc_addresses: joinAddresses(outer.cc, BOUND.cc_addresses), + sent_at: parsed.sentAt, + received_at: receivedAt, + body_text: parsed.text === null ? null : storable(parsed.text), + body_html: parsed.html === null ? null : storable(parsed.html), + match_addresses: serializeMatchAddresses(matchAddresses), + association_state: 'unassociated', + dedupe_key: dedupeKey, + }; + + let messageId: string | null; + try { + messageId = idOf(((await engine.insert(MAIL_MESSAGE_OBJECT, row, { context: { ...SYSTEM_CTX } })) as { id?: unknown })?.id); + } catch (err) { + // A concurrent ingest of the same message won the unique key: the + // message is stored once, which is the outcome this call wanted. + if (isDuplicateRecord(err)) { + const winner = await this.storedId(engine, mailbox.id, dedupeKey); + if (winner !== null) return { outcome: 'duplicate', messageId: winner }; + } + throw err; + } + if (messageId === null) { + throw new MailboxServiceError('INTERNAL_ERROR', `The engine stored a message for mailbox '${mailbox.id}' but returned no id.`); + } + if (links.length === 0) return { outcome: 'stored', messageId, associationState: 'unassociated', associations: [] }; + + for (const link of links) await this.link(engine, organizationId, messageId, link, 'ingest'); + await this.markAssociated(engine, messageId); + return { outcome: 'stored', messageId, associationState: 'associated', associations: links }; + } + + /** + * Re-match a mailbox's messages: replay each one's stored `match_addresses` + * against the records that exist now and add the links that are missing. + * Additive only. System context, or a `manage_mailboxes` holder or the + * mailbox's owner in its organization. + */ + async rematch(input: MailRematchInput, context: MailboxCallerContext): Promise { + if (typeof input?.mailboxId !== 'string' || input.mailboxId === '') { + throw new MailboxServiceError('VALIDATION_ERROR', 'A re-match requires the id of the mailbox to re-match.'); + } + const only = input.messageIds; + if (only !== undefined && (!Array.isArray(only) || only.some((id) => typeof id !== 'string' || id === ''))) { + throw new MailboxServiceError('VALIDATION_ERROR', '`messageIds` must be a list of message ids when it is given.'); + } + + const engine: MailIngestEngine = this.options.engine(); + const mailbox = await this.mailbox(engine, input.mailboxId); + this.authorizeRematch(context, mailbox); + const organizationId = this.organizationOf(mailbox); + + const result: MailRematchResult = { scanned: 0, linksAdded: 0, associated: 0, unreadable: [] }; + if (only !== undefined && only.length === 0) return result; + + // Seek by `id`: this walk writes the rows it reads (their state), so an + // offset would count into a set its own writes are changing. + const walk = keysetWalk>( + (query) => + engine.find(MAIL_MESSAGE_OBJECT, { + ...query, + fields: ['id', 'match_addresses', 'association_state'], + context: { ...SYSTEM_CTX }, + }), + { + where: only === undefined ? { mailbox_id: mailbox.id } : { mailbox_id: mailbox.id, id: { $in: [...new Set(only)] } }, + pageSize: REMATCH_PAGE_SIZE, + }, + ); + for await (const page of walk.pages()) await this.rematchPage(engine, organizationId, page, result); + return result; + } + + private async rematchPage( + engine: MailIngestEngine, + organizationId: string, + page: ReadonlyArray>, + result: MailRematchResult, + ): Promise { + const messages: Array<{ id: string; addresses: string[]; state: unknown }> = []; + for (const row of page) { + const id = idOf(row.id); + if (id === null) continue; + result.scanned++; + const stored = parseMatchAddresses(row.match_addresses); + if (stored === null) { + result.unreadable.push(id); + continue; + } + messages.push({ id, addresses: distinctAddresses(stored), state: row.association_state }); + } + if (messages.length === 0) return; + + const matches = await this.matcher.match(organizationId, messages.flatMap((m) => m.addresses)); + const existing = new Map>(); + const present = await engine.find(MAIL_ASSOCIATION_OBJECT, { + where: { mail_message_id: { $in: messages.map((m) => m.id) } }, + fields: ['mail_message_id', 'object_name', 'record_id'], + limit: REMATCH_LINK_SCAN_LIMIT, + context: { ...SYSTEM_CTX }, + }); + if (present.length >= REMATCH_LINK_SCAN_LIMIT) { + // Still additive and still once per record (the unique key refuses a + // second link), but a message whose links were not read may stay + // `unassociated` until a narrower re-match (`messageIds`) reads them. + this.options.logger.warn( + `[mailbox] re-match read the ${REMATCH_LINK_SCAN_LIMIT}-row bound of existing links for one page of messages; ` + + 're-match fewer messages at a time (messageIds) to read them all.', + ); + } + for (const link of present) { + const messageId = idOf(link.mail_message_id); + if (messageId === null) continue; + let keys = existing.get(messageId); + if (!keys) existing.set(messageId, (keys = new Set())); + keys.add(`${String(link.object_name)} ${String(link.record_id)}`); + } + + for (const message of messages) { + const have = existing.get(message.id) ?? new Set(); + let added = 0; + for (const link of linksFor(message.addresses, matches)) { + if (have.has(`${link.object} ${link.recordId}`)) continue; + if (await this.link(engine, organizationId, message.id, link, 'rematch')) added++; + have.add(`${link.object} ${link.recordId}`); + } + result.linksAdded += added; + if (message.state !== 'associated' && have.size > 0) { + await this.markAssociated(engine, message.id); + result.associated++; + } + } + } + + /** Write one link; `false` when the unique key says it already exists. */ + private async link( + engine: MailIngestEngine, + organizationId: string, + messageId: string, + link: MailAssociationMatch, + matchedBy: 'ingest' | 'rematch', + ): Promise { + try { + await engine.insert( + MAIL_ASSOCIATION_OBJECT, + { + organization_id: organizationId, + mail_message_id: messageId, + object_name: link.object, + record_id: link.recordId, + matched_address: link.matchedAddress, + matched_field: link.matchedField, + matched_by: matchedBy, + }, + { context: { ...SYSTEM_CTX } }, + ); + return true; + } catch (err) { + if (isDuplicateRecord(err)) return false; + throw err; + } + } + + private async markAssociated(engine: MailIngestEngine, messageId: string): Promise { + await engine.update( + MAIL_MESSAGE_OBJECT, + { association_state: 'associated' }, + { where: { id: messageId }, context: { ...SYSTEM_CTX } }, + ); + } + + private async storedId(engine: MailIngestEngine, mailboxId: string, dedupeKey: string): Promise { + const [row] = await engine.find(MAIL_MESSAGE_OBJECT, { + where: { mailbox_id: mailboxId, dedupe_key: dedupeKey }, + fields: ['id'], + limit: 1, + context: { ...SYSTEM_CTX }, + }); + return idOf(row?.id); + } + + private async mailbox(engine: MailIngestEngine, mailboxId: string): Promise { + const [row] = await engine.find(MAILBOX_OBJECT, { + where: { id: mailboxId }, + fields: ['id', 'address', 'owner_user_id', 'organization_id'], + limit: 1, + context: { ...SYSTEM_CTX }, + }); + const id = idOf(row?.id); + if (!row || id === null) throw new MailboxServiceError('RECORD_NOT_FOUND', `Mailbox '${mailboxId}' was not found.`); + return { + id, + address: typeof row.address === 'string' ? row.address : null, + ownerUserId: idOf(row.owner_user_id), + organizationId: idOf(row.organization_id), + }; + } + + private organizationOf(mailbox: MailboxRow): string { + if (mailbox.organizationId === null) { + throw new MailboxServiceError( + 'INTERNAL_ERROR', + `Mailbox '${mailbox.id}' carries no organization, so its mail cannot be matched or stored inside one. ` + + 'Every mailbox row is written with the organization that owns it.', + ); + } + return mailbox.organizationId; + } + + private authorizeRematch(context: MailboxCallerContext, mailbox: MailboxRow): void { + if (context?.isSystem === true) return; + const organization = idOf(context?.tenantId); + if (organization === null || organization !== mailbox.organizationId) { + // Another organization's mailbox does not exist for this caller. + throw new MailboxServiceError('RECORD_NOT_FOUND', `Mailbox '${mailbox.id}' was not found.`); + } + const held = context?.systemPermissions; + if (Array.isArray(held) && held.includes(MAILBOX_ADMIN_CAPABILITY)) return; + const userId = idOf(context?.userId); + if (userId !== null && userId === mailbox.ownerUserId) return; + throw new MailboxServiceError( + 'PERMISSION_DENIED', + `Re-matching mailbox '${mailbox.id}' requires the \`${MAILBOX_ADMIN_CAPABILITY}\` capability or ownership of the mailbox.`, + ); + } +} diff --git a/packages/services/service-mailbox/src/ingest/match-addresses.ts b/packages/services/service-mailbox/src/ingest/match-addresses.ts new file mode 100644 index 00000000000..0e6ad029e9d --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/match-addresses.ts @@ -0,0 +1,127 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The participant set a message is matched with — its addresses and where in + * the message each one was read (ruling 6116489638 on #22870, Q3 C phase 1). + * + * A message row stores the set it was matched with in `match_addresses`, so a + * re-match replays exactly that input (decision 2: an association is a + * deterministic, auditable fact) and a reader can tell which part of the + * message supplied an address. + * + * ## The stored shape + * + * A JSON array of `{ "address": string, "source": MailMatchAddressSource }` + * objects, in the order the message carried them, outer headers first. One + * entry per (address, source) pair: an address the outer headers and a + * forwarded part both name appears once under each source. The address is a + * normalized addr-spec ({@link normalizeMailAddress}). + * + * A stored value that is not this shape is not replayed: {@link + * parseMatchAddresses} answers `null`, and the re-match that read it reports + * the message as unreadable instead of matching it against a guess. + */ + +/** + * Where in a message an address was read. + * + * - `outer` — the received message's own From, Sender, Reply-To, To and Cc; + * - `attachment` — the same headers of a `message/rfc822` part of the + * received message (RFC 2046 §5.2.1: a message forwarded as an attachment); + * - `inline` — a forwarded message quoted in the body text. Declared so a + * stored row that carries it replays; this package writes it nowhere yet. + */ +export const MAIL_MATCH_ADDRESS_SOURCES = ['outer', 'attachment', 'inline'] as const; + +/** One of {@link MAIL_MATCH_ADDRESS_SOURCES}. */ +export type MailMatchAddressSource = (typeof MAIL_MATCH_ADDRESS_SOURCES)[number]; + +/** One participant address and the part of the message it was read from. */ +export interface MailMatchAddress { + /** A normalized addr-spec: trimmed, ASCII letters lower-cased. */ + address: string; + source: MailMatchAddressSource; +} + +/** The longest address the association row stores (`matched_address`). */ +export const MAIL_ADDRESS_MAX_LENGTH = 320; + +const SOURCES: ReadonlySet = new Set(MAIL_MATCH_ADDRESS_SOURCES); + +/** + * An addr-spec as the matcher compares it, or `null` when the value is not one. + * + * Matching is ASCII case-insensitive (the `mailAssociation` contract), so only + * ASCII letters are folded; every other character is kept as written. A value + * with no `@`, an empty local part or domain, whitespace inside it, or more than + * {@link MAIL_ADDRESS_MAX_LENGTH} characters is not an address anything can + * match. + */ +export function normalizeMailAddress(value: unknown): string | null { + if (typeof value !== 'string') return null; + const trimmed = value.trim(); + if (trimmed.length === 0 || trimmed.length > MAIL_ADDRESS_MAX_LENGTH) return null; + if (/\s/.test(trimmed)) return null; + const at = trimmed.lastIndexOf('@'); + if (at <= 0 || at === trimmed.length - 1) return null; + return trimmed.replace(/[A-Z]/g, (c) => c.toLowerCase()); +} + +/** + * The participant set for one message: `outer` addresses first, then + * `attachment` ones, each normalized, the mailbox's own addresses removed, and + * each (address, source) pair kept once. + */ +export function collectMatchAddresses( + parts: ReadonlyArray<{ source: MailMatchAddressSource; addresses: ReadonlyArray }>, + mailboxAddresses: ReadonlyArray, +): MailMatchAddress[] { + const own = new Set(mailboxAddresses.map(normalizeMailAddress).filter((a): a is string => a !== null)); + const seen = new Set(); + const out: MailMatchAddress[] = []; + for (const part of parts) { + for (const raw of part.addresses) { + const address = normalizeMailAddress(raw); + if (address === null || own.has(address)) continue; + const key = `${part.source} ${address}`; + if (seen.has(key)) continue; + seen.add(key); + out.push({ address, source: part.source }); + } + } + return out; +} + +/** The stored spelling of a participant set (`match_addresses`). */ +export function serializeMatchAddresses(addresses: ReadonlyArray): string { + return JSON.stringify(addresses.map(({ address, source }) => ({ address, source }))); +} + +/** + * A stored participant set, or `null` when the value is not the stored shape: + * not a JSON array, an entry that is not exactly `{ address, source }`, a source + * outside {@link MAIL_MATCH_ADDRESS_SOURCES}, or an address that is not already + * normalized. An empty stored value (`null`, `''`) is the empty set. + */ +export function parseMatchAddresses(stored: unknown): MailMatchAddress[] | null { + if (stored === null || stored === undefined || stored === '') return []; + if (typeof stored !== 'string') return null; + let value: unknown; + try { + value = JSON.parse(stored); + } catch { + return null; + } + if (!Array.isArray(value)) return null; + const out: MailMatchAddress[] = []; + for (const entry of value) { + if (!entry || typeof entry !== 'object' || Array.isArray(entry)) return null; + const keys = Object.keys(entry); + if (keys.length !== 2 || !keys.includes('address') || !keys.includes('source')) return null; + const { address, source } = entry as Record; + if (typeof source !== 'string' || !SOURCES.has(source)) return null; + if (typeof address !== 'string' || normalizeMailAddress(address) !== address) return null; + out.push({ address, source: source as MailMatchAddressSource }); + } + return out; +} diff --git a/packages/services/service-mailbox/src/ingest/match-records.ts b/packages/services/service-mailbox/src/ingest/match-records.ts new file mode 100644 index 00000000000..3242fc3fd36 --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/match-records.ts @@ -0,0 +1,188 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * The association seam: given participant addresses, which records does a + * message belong to? (ruling 6116489638 on #22870, Q1 A.) + * + * The app answers it by DECLARATION, never by code: every `email` field + * flagged `mailAssociation: true` (`FieldSchema`, `@objectstack/spec/data`) + * names a field whose value identifies a correspondent. A record matches when + * its value in such a field equals one of the participant addresses — ASCII + * case-insensitive, inside the mailbox's organization only. Each match is one + * `{ object, recordId, matchedField, matchedAddress }`, which becomes one + * `sys_mail_association` row. No match means the pool. + * + * The platform learns which fields carry a correspondent's address and never + * what a contact is: every object that flags a field is read the same way. + * + * ## The read + * + * One system-context read per (flagged field, chunk of addresses), scoped by + * the object's `organization_id` column. The comparison is `$icontains` — the + * case-insensitive operator every driver executes with its own LIKE escaping — + * and the exact equality is then decided here on the value read back, so a + * substring hit (`bob@acme.com` inside `jimbob@acme.com`) never matches. A + * flagged object with no `organization_id` column (tenancy disabled, or + * declared platform-global) holds no organization's records, so nothing in + * one mailbox's organization can match it; it is skipped, and said so once. + */ + +import { MAIL_ASSOCIATION_OBJECT, MAIL_MESSAGE_OBJECT, MAILBOX_OBJECT } from '../mail-read-visibility.js'; +import { normalizeMailAddress } from './match-addresses.js'; + +/** One match: the record a participant address identifies, and how. */ +export interface MailAssociationMatch { + /** Machine name of the record's object. */ + object: string; + recordId: string; + /** The flagged email field whose value equalled the address. */ + matchedField: string; + /** The participant address, normalized. */ + matchedAddress: string; +} + +/** The engine surface the matcher reads. */ +export interface MailMatchEngine { + find(object: string, options: Record): Promise>>; + registry?: { getAllObjects?(): ReadonlyArray }; +} + +/** The logger surface the matcher reports through. */ +export interface MailMatchLogger { + warn(message: string): void; +} + +/** One flagged field: an `email` field declaring `mailAssociation: true`. */ +export interface MailAssociationField { + object: string; + field: string; +} + +/** Addresses per read: one `$or` branch each. */ +export const MATCH_ADDRESS_CHUNK = 50; +/** Rows per read. Beyond it the read may be incomplete, and is said so. */ +export const MATCH_SCAN_LIMIT = 1_000; + +const SYSTEM_CTX = { isSystem: true } as const; +const ORGANIZATION_COLUMN = 'organization_id'; +const MAIL_OBJECTS: ReadonlySet = new Set([MAIL_MESSAGE_OBJECT, MAIL_ASSOCIATION_OBJECT, MAILBOX_OBJECT]); + +type AnyRecord = Record; +const isRecord = (v: unknown): v is AnyRecord => typeof v === 'object' && v !== null && !Array.isArray(v); + +/** + * The flagged fields of every registered object, in registry order, and the + * flagged objects that carry no organization column (which cannot be matched). + */ +export function flaggedFieldsOf(engine: MailMatchEngine): { + fields: MailAssociationField[]; + unscoped: string[]; +} { + const fields: MailAssociationField[] = []; + const unscoped: string[] = []; + for (const schema of engine.registry?.getAllObjects?.() ?? []) { + if (!isRecord(schema) || typeof schema.name !== 'string' || MAIL_OBJECTS.has(schema.name)) continue; + const defs = isRecord(schema.fields) ? schema.fields : {}; + const flagged = Object.entries(defs) + .filter(([, def]) => isRecord(def) && def.type === 'email' && def.mailAssociation === true) + .map(([field]) => field); + if (flagged.length === 0) continue; + if (!(ORGANIZATION_COLUMN in defs)) { + unscoped.push(schema.name); + continue; + } + for (const field of flagged) fields.push({ object: schema.name, field }); + } + return { fields, unscoped }; +} + +/** + * The record matcher. Holds the "said once" memory for unscoped objects, so a + * long-lived instance does not repeat itself per message. + */ +export class MailRecordMatcher { + private readonly reportedUnscoped = new Set(); + + constructor( + private readonly engine: () => MailMatchEngine, + private readonly logger: MailMatchLogger, + ) {} + + /** + * Every match for `addresses` inside `organizationId`, keyed by address. + * An address with no match has no key. + */ + async match(organizationId: string, addresses: ReadonlyArray): Promise> { + const out = new Map(); + const wanted = [...new Set(addresses.map(normalizeMailAddress).filter((a): a is string => a !== null))]; + if (wanted.length === 0) return out; + + const engine = this.engine(); + const { fields, unscoped } = flaggedFieldsOf(engine); + for (const object of unscoped) { + if (this.reportedUnscoped.has(object)) continue; + this.reportedUnscoped.add(object); + this.logger.warn( + `[mailbox] '${object}' flags an email field with mailAssociation but has no organization column, so no ` + + "mailbox's mail can be matched to its records; it is skipped. Matching is scoped to a mailbox's " + + 'organization, and only an organization-scoped object holds records of one.', + ); + } + + for (const { object, field } of fields) { + for (let i = 0; i < wanted.length; i += MATCH_ADDRESS_CHUNK) { + const chunk = wanted.slice(i, i + MATCH_ADDRESS_CHUNK); + const rows = await engine.find(object, { + where: { + $and: [ + { [ORGANIZATION_COLUMN]: organizationId }, + { $or: chunk.map((address) => ({ [field]: { $icontains: address } })) }, + ], + }, + fields: ['id', field], + limit: MATCH_SCAN_LIMIT, + context: { ...SYSTEM_CTX }, + }); + if (rows.length >= MATCH_SCAN_LIMIT) { + this.logger.warn( + `[mailbox] matching '${object}.${field}' read the ${MATCH_SCAN_LIMIT}-row bound for one set of ` + + 'participant addresses, so some matching records may not be associated; a re-match adds them later.', + ); + } + const inChunk = new Set(chunk); + for (const row of rows) { + const address = normalizeMailAddress(row[field]); + const id = row.id; + if (address === null || !inChunk.has(address)) continue; + if (typeof id !== 'string' && typeof id !== 'number') continue; + let list = out.get(address); + if (!list) out.set(address, (list = [])); + list.push({ object, recordId: String(id), matchedField: field, matchedAddress: address }); + } + } + } + return out; + } +} + +/** + * The links for one message: the matches of each of its addresses, in address + * order, one per record (a record two addresses or two fields match is linked + * once, by the first). + */ +export function linksFor( + addresses: ReadonlyArray, + matches: ReadonlyMap>, +): MailAssociationMatch[] { + const seen = new Set(); + const out: MailAssociationMatch[] = []; + for (const address of addresses) { + for (const match of matches.get(address) ?? []) { + const key = `${match.object} ${match.recordId}`; + if (seen.has(key)) continue; + seen.add(key); + out.push(match); + } + } + return out; +} diff --git a/packages/services/service-mailbox/src/ingest/messages.fixture.ts b/packages/services/service-mailbox/src/ingest/messages.fixture.ts new file mode 100644 index 00000000000..d170c548474 --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/messages.fixture.ts @@ -0,0 +1,100 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Raw RFC 822 fixtures for the ingest tests — synthetic addresses only, CRLF + * line breaks as a mail store hands them over. + */ + +export type HeaderList = ReadonlyArray; + +/** One message: its header lines, an empty line, its body. */ +export function rfc822(headers: HeaderList, body: string): string { + return [...headers.map(([name, value]) => `${name}: ${value}`), '', body].join('\r\n'); +} + +/** A plain-text message with the usual headers, overridable one by one. */ +export function plainMessage(over: Record = {}, body = 'Hello.'): string { + const base: Record = { + From: 'Customer ', + To: 'Sales Inbox ', + Subject: 'A question', + 'Message-ID': '', + Date: 'Mon, 12 Oct 2026 09:00:00 +0000', + 'MIME-Version': '1.0', + 'Content-Type': 'text/plain; charset=utf-8', + }; + const merged: Array<[string, string]> = []; + for (const [name, value] of Object.entries({ ...base, ...over })) if (value !== null) merged.push([name, value]); + return rfc822(merged, body); +} + +/** + * A message forwarded AS AN ATTACHMENT (RFC 2046 §5.2.1): the forwarder's note + * plus the original message as a `message/rfc822` part. + */ +let boundarySeq = 0; + +export function forwardAsAttachment(options: { + forwarder?: string; + to?: string; + messageId?: string; + original: string; +}): string { + // A boundary of its own per message, as every client mints one: a forward + // nested inside another must not share its delimiter. + const boundary = `fwd-${++boundarySeq}-boundary`; + return rfc822( + [ + ['From', options.forwarder ?? 'Rep '], + ['To', options.to ?? 'inbox@corp.test'], + ['Subject', 'Fwd: A question'], + ['Message-ID', options.messageId ?? ''], + ['Date', 'Mon, 12 Oct 2026 10:00:00 +0000'], + ['MIME-Version', '1.0'], + ['Content-Type', `multipart/mixed; boundary="${boundary}"`], + ], + [ + `--${boundary}`, + 'Content-Type: text/plain; charset=utf-8', + '', + "See the customer's question attached.", + `--${boundary}`, + 'Content-Type: message/rfc822', + 'Content-Disposition: attachment; filename="question.eml"', + '', + options.original, + `--${boundary}--`, + '', + ].join('\r\n'), + ); +} + +/** A delivery-status report (RFC 3464 inside RFC 6522 `multipart/report`): a bounce. */ +export function bounce(): string { + return rfc822( + [ + ['From', 'Mail Delivery System '], + ['To', 'inbox@corp.test'], + ['Subject', 'Undelivered Mail Returned to Sender'], + ['Message-ID', ''], + ['MIME-Version', '1.0'], + ['Content-Type', 'multipart/report; report-type=delivery-status; boundary="dsn"'], + ], + [ + '--dsn', + 'Content-Type: text/plain; charset=us-ascii', + '', + 'Your message to customer@acme.test could not be delivered.', + '--dsn', + 'Content-Type: message/delivery-status', + '', + 'Reporting-MTA: dns; relay.test', + '', + 'Final-Recipient: rfc822; customer@acme.test', + 'Action: failed', + 'Status: 5.1.1', + '--dsn--', + '', + ].join('\r\n'), + ); +} diff --git a/packages/services/service-mailbox/src/ingest/parse-message.test.ts b/packages/services/service-mailbox/src/ingest/parse-message.test.ts new file mode 100644 index 00000000000..e27a8fb82ac --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/parse-message.test.ts @@ -0,0 +1,161 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, expect, it } from 'vitest'; +import { messageIdsOf, parseInboundMessage, participantAddresses, rawHeaderBlockOf } from './parse-message.js'; +import { + collectMatchAddresses, + normalizeMailAddress, + parseMatchAddresses, + serializeMatchAddresses, +} from './match-addresses.js'; +import { forwardAsAttachment, plainMessage, rfc822 } from './messages.fixture.js'; + +const ORIGINAL = rfc822( + [ + ['From', 'Customer '], + ['To', 'Rep '], + ['Cc', 'Buyer '], + ['Subject', 'A question'], + ['Message-ID', ''], + ], + 'What does it cost?', +); + +describe('parseInboundMessage — the participants of the received message (source `outer`)', () => { + it('reads every address of From, Sender, Reply-To, To and Cc, display names, groups and folding included', async () => { + const parsed = await parseInboundMessage( + rfc822( + [ + ['From', '"Doe, Jane" , john@acme.test'], + ['Sender', 'assistant@acme.test'], + ['Reply-To', 'replies@acme.test'], + ['To', 'Sales Inbox ,\r\n =?utf-8?B?5byg5LiJ?= '], + ['Cc', 'Team: one@acme.test, two@acme.test;'], + ['Message-ID', ''], + ], + 'Body', + ), + ); + expect(parsed.outer).toEqual({ + from: ['jane@acme.test', 'john@acme.test'], + sender: ['assistant@acme.test'], + replyTo: ['replies@acme.test'], + to: ['inbox@corp.test', 'zhang@acme.test'], + cc: ['one@acme.test', 'two@acme.test'], + }); + expect(participantAddresses(parsed.outer)).toEqual([ + 'jane@acme.test', 'john@acme.test', 'assistant@acme.test', 'replies@acme.test', + 'inbox@corp.test', 'zhang@acme.test', 'one@acme.test', 'two@acme.test', + ]); + expect(parsed.fromName).toBe('Doe, Jane'); + }); + + it('reads the threading ids with their angle brackets stripped', async () => { + const parsed = await parseInboundMessage( + plainMessage({ 'In-Reply-To': '', References: '\r\n ' }), + ); + expect(parsed.messageId).toBe('plain-1@acme.test'); + expect(parsed.inReplyTo).toEqual(['a@acme.test']); + expect(parsed.references).toEqual(['root@acme.test', 'a@acme.test']); + expect(messageIdsOf('bare-token other-token')).toEqual(['bare-token', 'other-token']); + expect(messageIdsOf(null)).toEqual([]); + }); + + it('reads the subject with its encoded words decoded, the text body and the Date as an ISO timestamp', async () => { + const parsed = await parseInboundMessage(plainMessage({ Subject: '=?utf-8?Q?Caf=C3=A9_order?=' }, 'Line one')); + expect(parsed.subject).toBe('Café order'); + expect(parsed.text?.trim()).toBe('Line one'); + expect(parsed.sentAt).toBe('2026-10-12T09:00:00.000Z'); + }); + + it('keeps the header block exactly as received, up to the first empty line', () => { + const raw = 'From: a@acme.test\r\nSubject: x\r\n\r\nbody\r\n\r\nmore'; + expect(rawHeaderBlockOf(raw)).toBe('From: a@acme.test\r\nSubject: x'); + expect(rawHeaderBlockOf(new TextEncoder().encode(raw))).toBe('From: a@acme.test\r\nSubject: x'); + }); +}); + +describe('parseInboundMessage — a message forwarded as an attachment (source `attachment`)', () => { + it("reads the headers of a top-level message/rfc822 part as the forwarded message's participants", async () => { + const parsed = await parseInboundMessage(forwardAsAttachment({ original: ORIGINAL })); + expect(parsed.outer.from).toEqual(['rep@corp.test']); + expect(parsed.forwarded).toHaveLength(1); + expect(participantAddresses(parsed.forwarded[0]!)).toEqual(['Customer@Acme.test', 'rep@corp.test', 'buyer@acme.test']); + expect(parsed.unreadForwardedParts).toBe(0); + // The forwarded message is not folded into the received message's own text. + expect(parsed.text).not.toContain('What does it cost?'); + }); + + it('does not read a forward nested inside the forwarded part', async () => { + const nested = forwardAsAttachment({ + forwarder: 'Customer ', + to: 'rep@corp.test', + messageId: '', + original: rfc822([['From', 'deep@elsewhere.test'], ['Message-ID', '']], 'deep'), + }); + const parsed = await parseInboundMessage(forwardAsAttachment({ original: nested })); + expect(parsed.forwarded).toHaveLength(1); + expect(participantAddresses(parsed.forwarded[0]!)).toEqual(['customer@acme.test', 'rep@corp.test']); + }); + + it('control: a forward quoted inline in the body names no forwarded participants (that is the `inline` source)', async () => { + const parsed = await parseInboundMessage( + plainMessage( + { From: 'Rep ' }, + '---------- Forwarded message ---------\r\nFrom: Customer \r\nTo: rep@corp.test\r\n\r\nHi', + ), + ); + expect(parsed.forwarded).toEqual([]); + expect(participantAddresses(parsed.outer)).toEqual(['rep@corp.test', 'inbox@corp.test']); + }); +}); + +describe('the participant set (`match_addresses`)', () => { + it('normalizes ASCII letters only, and refuses what is not an addr-spec', () => { + expect(normalizeMailAddress(' Jane.DOE@Acme.TEST ')).toBe('jane.doe@acme.test'); + expect(normalizeMailAddress('Ünïcode@Acme.test')).toBe('Ünïcode@acme.test'); + for (const bad of ['', 'no-at-sign', '@acme.test', 'jane@', 'a b@acme.test', 42, null, `${'x'.repeat(320)}@a.t`]) { + expect(normalizeMailAddress(bad), String(bad)).toBeNull(); + } + }); + + it('lists outer addresses first, drops the mailbox own address, and keeps each (address, source) pair once', () => { + const set = collectMatchAddresses( + [ + { source: 'outer', addresses: ['Rep@Corp.test', 'INBOX@corp.test', 'rep@corp.test', 'nonsense'] }, + { source: 'attachment', addresses: ['customer@acme.test', 'rep@corp.test', 'Inbox@Corp.Test'] }, + ], + ['inbox@corp.test'], + ); + expect(set).toEqual([ + { address: 'rep@corp.test', source: 'outer' }, + { address: 'customer@acme.test', source: 'attachment' }, + { address: 'rep@corp.test', source: 'attachment' }, + ]); + }); + + it('round-trips through its stored spelling, and accepts the declared `inline` source a later parser writes', () => { + const set = [ + { address: 'a@acme.test', source: 'outer' as const }, + { address: 'b@acme.test', source: 'inline' as const }, + ]; + expect(parseMatchAddresses(serializeMatchAddresses(set))).toEqual(set); + expect(parseMatchAddresses(null)).toEqual([]); + expect(parseMatchAddresses('')).toEqual([]); + }); + + it('refuses a stored value that is not the stored shape rather than guessing at it', () => { + for (const bad of [ + 'a@acme.test', + '{"address":"a@acme.test","source":"outer"}', + '[{"address":"a@acme.test"}]', + '[{"address":"a@acme.test","source":"body"}]', + '[{"address":"A@acme.test","source":"outer"}]', + '[{"address":"a@acme.test","source":"outer","extra":1}]', + '[null]', + 42, + ]) { + expect(parseMatchAddresses(bad), String(bad)).toBeNull(); + } + }); +}); diff --git a/packages/services/service-mailbox/src/ingest/parse-message.ts b/packages/services/service-mailbox/src/ingest/parse-message.ts new file mode 100644 index 00000000000..7b6aa76f001 --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/parse-message.ts @@ -0,0 +1,212 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Raw RFC 822 message → the facts the ingest seam stores and matches on. + * + * MIME is read by `postal-mime` (MIT-0, no dependencies, one TypeScript source + * compiled to both an ES module and a CommonJS build, and written for Node, + * browsers and Workers alike): RFC 5322 header unfolding, RFC 2047 encoded + * words, address lists and groups, the RFC 2046 multipart walk, and the + * transfer and charset decoding of the text and HTML bodies. This module adds + * only what the seam itself decides: + * + * - the participant addresses, per part: the received message's own From, + * Sender, Reply-To, To and Cc (`outer`), and the same five headers of each + * `message/rfc822` part of the received message (`attachment` — a message + * forwarded as an attachment, RFC 2046 §5.2.1). Such a part is read for its + * headers only, and a part nested inside it is not read at all: the seam + * reads the forward the user made, not the forwards inside it. A forwarded + * message quoted inline in the body is not read (that is the `inline` + * source, and its parser is separate work); + * - the threading ids (Message-ID, In-Reply-To, References, RFC 5322 §3.6.4), + * angle brackets stripped; + * - the header values automatic-mail suppression reads, and the top-level + * media type (`multipart/report`, RFC 6522, carries delivery reports); + * - the header block as received, for the dedupe fallback. + */ + +import PostalMime, { addressParser } from 'postal-mime'; +import type { Address, Email, Header } from 'postal-mime'; + +/** What the ingest seam accepts as one raw message. */ +export type RawMailMessage = string | Uint8Array | ArrayBuffer; + +/** The addresses one part of a message names, per participant header. */ +export interface MailParticipantHeaders { + from: string[]; + sender: string[]; + replyTo: string[]; + to: string[]; + cc: string[]; +} + +/** One parsed inbound message. */ +export interface ParsedInboundMessage { + /** Message-ID, angle brackets stripped, or `null` when the message has none. */ + messageId: string | null; + inReplyTo: string[]; + references: string[]; + subject: string | null; + /** The display name of the first From mailbox. */ + fromName: string | null; + /** The `Date` header as an ISO timestamp, or `null` when absent or unreadable. */ + sentAt: string | null; + text: string | null; + html: string | null; + /** The received message's own participants (the `outer` source). */ + outer: MailParticipantHeaders; + /** One entry per `message/rfc822` part of the received message (the `attachment` source). */ + forwarded: MailParticipantHeaders[]; + /** + * How many `message/rfc822` parts could not be read (the parser refused + * them) or were past {@link MAX_FORWARDED_PARTS}. Their addresses are not in + * {@link forwarded}; the message itself is still stored. + */ + unreadForwardedParts: number; + /** Lower-cased header name → the first value, unfolded, encoded words left as sent. */ + header(name: string): string | null; + /** The top-level media type, lower-cased, without parameters (`text/plain` when absent). */ + mediaType: string; + /** The value of the top-level Content-Type's `report-type` parameter, lower-cased. */ + reportType: string | null; + /** The header block exactly as received: everything before the first empty line. */ + rawHeaderBlock: string; +} + +/** + * Parser bounds. `postal-mime`'s own limits stay at its defaults (nesting + * depth, header bytes, part count); this one bounds how many forwarded parts + * a single message may hand the matcher. + */ +export const MAX_FORWARDED_PARTS = 20; + +const PARSE_OPTIONS = { + // Every `message/rfc822` part is an attachment, never folded into the outer + // body: its headers are read separately below, and its body is not the + // received message's own text. + forceRfc822Attachments: true, + attachmentEncoding: 'arraybuffer', +} as const; + +/** The angle-bracketed msg-ids of a header value; bare tokens when it has none. */ +export function messageIdsOf(value: string | null | undefined): string[] { + if (!value) return []; + const bracketed = [...value.matchAll(/<([^<>]*)>/g)].map((m) => m[1]!.trim()).filter((id) => id.length > 0); + if (bracketed.length > 0) return bracketed; + return value.split(/\s+/).map((t) => t.trim()).filter((t) => t.length > 0); +} + +function firstHeader(headers: ReadonlyArray
, name: string): string | null { + const hit = headers.find((h) => h.key === name); + return hit ? hit.value : null; +} + +function addressesOf(value: string | null): string[] { + if (!value) return []; + const out: string[] = []; + const visit = (list: ReadonlyArray
) => { + for (const entry of list) { + if (entry.group) visit(entry.group); + else if (entry.address) out.push(entry.address); + } + }; + visit(addressParser(value)); + return out; +} + +function participantsOf(headers: ReadonlyArray
): MailParticipantHeaders { + return { + from: addressesOf(firstHeader(headers, 'from')), + sender: addressesOf(firstHeader(headers, 'sender')), + replyTo: addressesOf(firstHeader(headers, 'reply-to')), + to: addressesOf(firstHeader(headers, 'to')), + cc: addressesOf(firstHeader(headers, 'cc')), + }; +} + +function firstMailboxName(value: string | null): string | null { + if (!value) return null; + for (const entry of addressParser(value, { flatten: true })) { + if (entry.address) return entry.name ? entry.name : null; + } + return null; +} + +function isoDate(value: string | undefined): string | null { + if (!value) return null; + const at = new Date(value); + return Number.isNaN(at.getTime()) ? null : at.toISOString(); +} + +function asText(raw: RawMailMessage): string { + if (typeof raw === 'string') return raw; + // The WHATWG `latin1` label decodes one character per byte, so the header + // block below stands for the bytes as received whatever their encoding. + return new TextDecoder('latin1').decode(raw instanceof Uint8Array ? raw : new Uint8Array(raw)); +} + +/** Everything before the first empty line, line breaks as received. */ +export function rawHeaderBlockOf(raw: RawMailMessage): string { + const text = asText(raw); + const match = /\r?\n\r?\n/.exec(text); + return match ? text.slice(0, match.index) : text; +} + +function mediaTypeOf(contentType: string | null): { mediaType: string; reportType: string | null } { + if (!contentType) return { mediaType: 'text/plain', reportType: null }; + const mediaType = contentType.split(';')[0]!.trim().toLowerCase() || 'text/plain'; + const report = /(?:^|;)\s*report-type\s*=\s*"?([^";\s]+)"?/i.exec(contentType); + return { mediaType, reportType: report ? report[1]!.toLowerCase() : null }; +} + +/** Parse one raw message. Throws when `postal-mime` refuses it (a bound exceeded). */ +export async function parseInboundMessage(raw: RawMailMessage): Promise { + const email: Email = await PostalMime.parse(raw, PARSE_OPTIONS); + const forwarded: MailParticipantHeaders[] = []; + let unreadForwardedParts = 0; + for (const attachment of email.attachments) { + if (attachment.mimeType.toLowerCase() !== 'message/rfc822') continue; + if (forwarded.length >= MAX_FORWARDED_PARTS) { + unreadForwardedParts++; + continue; + } + const content = attachment.content; + try { + const inner: Email = await PostalMime.parse( + typeof content === 'string' ? content : content instanceof Uint8Array ? content : new Uint8Array(content), + PARSE_OPTIONS, + ); + forwarded.push(participantsOf(inner.headers)); + } catch { + // A forwarded part the parser refuses names no addresses; the received + // message is still mirrored (never dropped), and the count says so. + unreadForwardedParts++; + } + } + + const headers = email.headers; + const { mediaType, reportType } = mediaTypeOf(firstHeader(headers, 'content-type')); + const [messageId] = messageIdsOf(email.messageId ?? null); + return { + messageId: messageId ?? null, + inReplyTo: messageIdsOf(firstHeader(headers, 'in-reply-to')), + references: messageIdsOf(firstHeader(headers, 'references')), + subject: email.subject ?? null, + fromName: firstMailboxName(firstHeader(headers, 'from')), + sentAt: isoDate(email.date), + text: email.text ?? null, + html: email.html ?? null, + outer: participantsOf(headers), + forwarded, + unreadForwardedParts, + header: (name: string) => firstHeader(headers, name.toLowerCase()), + mediaType, + reportType, + rawHeaderBlock: rawHeaderBlockOf(raw), + }; +} + +/** Every address one part names, in header order: From, Sender, Reply-To, To, Cc. */ +export function participantAddresses(part: MailParticipantHeaders): string[] { + return [...part.from, ...part.sender, ...part.replyTo, ...part.to, ...part.cc]; +} diff --git a/packages/services/service-mailbox/src/ingest/suppression.test.ts b/packages/services/service-mailbox/src/ingest/suppression.test.ts new file mode 100644 index 00000000000..340fbaa686b --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/suppression.test.ts @@ -0,0 +1,72 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +import { describe, expect, it } from 'vitest'; +import { parseInboundMessage } from './parse-message.js'; +import { suppressionReasonOf } from './suppression.js'; +import { dedupeKeyOf } from './dedupe-key.js'; +import { bounce, plainMessage, rfc822 } from './messages.fixture.js'; + +const reasonFor = async (raw: string) => suppressionReasonOf(await parseInboundMessage(raw)); + +describe('automatic-mail suppression', () => { + it('control: a message a correspondent wrote is not suppressed', async () => { + expect(await reasonFor(plainMessage())).toBeNull(); + }); + + it.each([ + ['Auto-Submitted', 'auto-replied', 'auto_submitted'], + ['Auto-Submitted', 'auto-generated (vacation)', 'auto_submitted'], + ['Auto-Submitted', 'Auto-Notified; owner-email="a@acme.test"', 'auto_submitted'], + ['Precedence', 'bulk', 'precedence'], + ['Precedence', 'Junk', 'precedence'], + ['Precedence', 'list', 'precedence'], + ['X-Autoreply', 'yes', 'autoreply_header'], + ['X-Autorespond', 'Out of office', 'autoreply_header'], + ])('%s: %s is suppressed as %s', async (name, value, reason) => { + expect(await reasonFor(plainMessage({ [name]: value }))).toBe(reason); + }); + + it.each([ + ['Auto-Submitted', 'no'], + ['Precedence', 'first-class'], + ['X-Autoreply', 'no'], + ['X-Autorespond', 'false'], + ])('control: %s: %s is not suppressed', async (name, value) => { + expect(await reasonFor(plainMessage({ [name]: value }))).toBeNull(); + }); + + it('a delivery-status report (a bounce) is suppressed', async () => { + expect(await reasonFor(bounce())).toBe('delivery_report'); + }); + + it('control: a multipart/report of another report type is not read as a bounce', async () => { + const receipt = rfc822( + [ + ['From', 'customer@acme.test'], + ['Message-ID', ''], + ['MIME-Version', '1.0'], + ['Content-Type', 'multipart/report; report-type=disposition-notification; boundary="r"'], + ], + '--r\r\nContent-Type: text/plain\r\n\r\nRead.\r\n--r--\r\n', + ); + expect(await reasonFor(receipt)).toBeNull(); + }); +}); + +describe('the dedupe key', () => { + it('is the Message-ID identity, prefixed `mid:`, whatever the brackets', async () => { + const a = await dedupeKeyOf({ messageId: 'x@acme.test', rawHeaderBlock: 'one' }); + const b = await dedupeKeyOf({ messageId: ' x@acme.test ', rawHeaderBlock: 'two' }); + expect(a).toBe(b); + expect(a).toMatch(/^mid:[0-9a-f]{64}$/); + expect(await dedupeKeyOf({ messageId: 'y@acme.test', rawHeaderBlock: 'one' })).not.toBe(a); + }); + + it('falls back to the header block as received, prefixed `hdr:`, for a message with no Message-ID', async () => { + const a = await dedupeKeyOf({ messageId: null, rawHeaderBlock: 'From: a@acme.test\r\nDate: x' }); + expect(a).toMatch(/^hdr:[0-9a-f]{64}$/); + expect(await dedupeKeyOf({ messageId: '', rawHeaderBlock: 'From: a@acme.test\r\nDate: x' })).toBe(a); + expect(await dedupeKeyOf({ messageId: null, rawHeaderBlock: 'From: a@acme.test\r\nDate: y' })).not.toBe(a); + expect(a.length).toBeLessThanOrEqual(128); + }); +}); diff --git a/packages/services/service-mailbox/src/ingest/suppression.ts b/packages/services/service-mailbox/src/ingest/suppression.ts new file mode 100644 index 00000000000..b80150d38ef --- /dev/null +++ b/packages/services/service-mailbox/src/ingest/suppression.ts @@ -0,0 +1,61 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Automatic-mail suppression: which inbound messages are not mirrored at all. + * + * An auto-reply or a bounce is mail a machine wrote about mail, not a message a + * correspondent sent, so it would put noise on every record its addresses + * match. Such a message is SUPPRESSED: no `sys_mail_message` row is written, + * and the ingest result names the reason. `sys_mail_message` declares no + * suppressed state (`association_state` is `associated` / `unassociated`), so + * a suppressed message is not a row in the pool either — "never dropped" + * (decision 3) is about messages no record matched, not about these. + * + * The signals, each a declared header or media type, never a guess from the + * subject line: + * + * - `auto_submitted` — RFC 3834 §5: `Auto-Submitted` with any value but `no` + * (`auto-generated`, `auto-replied`, `auto-notified`); + * - `precedence` — `Precedence: bulk`, `junk` or `list`, the de-facto marker + * RFC 3834 §3.1.8 tells responders to honour; + * - `autoreply_header` — `X-Autoreply` or `X-Autorespond` with a value other + * than `no` / `false` / `0`, the vendor headers common responders set; + * - `delivery_report` — a top-level `multipart/report` with + * `report-type=delivery-status` (RFC 6522, RFC 3464): a bounce. + */ + +import type { ParsedInboundMessage } from './parse-message.js'; + +/** Why a message was suppressed. */ +export const MAIL_SUPPRESSION_REASONS = ['auto_submitted', 'precedence', 'autoreply_header', 'delivery_report'] as const; + +/** One of {@link MAIL_SUPPRESSION_REASONS}. */ +export type MailSuppressionReason = (typeof MAIL_SUPPRESSION_REASONS)[number]; + +const PRECEDENCE_SUPPRESSED: ReadonlySet = new Set(['bulk', 'junk', 'list']); +const NEGATIVE_FLAG: ReadonlySet = new Set(['', 'no', 'false', '0']); + +/** The first token of a header value, lower-cased, parameters and comments dropped. */ +function tokenOf(value: string | null): string | null { + if (value === null) return null; + return value.replace(/\([^)]*\)/g, ' ').split(';')[0]!.trim().toLowerCase(); +} + +/** The reason this message is suppressed, or `null` when it is mirrored. */ +export function suppressionReasonOf( + message: Pick, +): MailSuppressionReason | null { + const autoSubmitted = tokenOf(message.header('auto-submitted')); + if (autoSubmitted !== null && autoSubmitted !== '' && autoSubmitted !== 'no') return 'auto_submitted'; + + const precedence = tokenOf(message.header('precedence')); + if (precedence !== null && PRECEDENCE_SUPPRESSED.has(precedence)) return 'precedence'; + + for (const name of ['x-autoreply', 'x-autorespond']) { + const flag = tokenOf(message.header(name)); + if (flag !== null && !NEGATIVE_FLAG.has(flag)) return 'autoreply_header'; + } + + if (message.mediaType === 'multipart/report' && message.reportType === 'delivery-status') return 'delivery_report'; + return null; +} diff --git a/packages/services/service-mailbox/src/mailbox-service-kernel.test.ts b/packages/services/service-mailbox/src/mailbox-service-kernel.test.ts new file mode 100644 index 00000000000..46babbb9304 --- /dev/null +++ b/packages/services/service-mailbox/src/mailbox-service-kernel.test.ts @@ -0,0 +1,76 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * How the ingest seam is reached: through the kernel, as the `mailbox` + * service the plugin registers, against the engine `ObjectQLPlugin` provides. + * A real `ObjectKernel` boot — the seam is not handed an engine here, it + * resolves the kernel's own on the call — over `@objectstack/driver-sql` + + * better-sqlite3 `:memory:`. + */ + +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import { ObjectKernel } from '@objectstack/core'; +import { ObjectQLPlugin, type ObjectQL } from '@objectstack/objectql'; +import { SqlDriver } from '@objectstack/driver-sql'; +import { ObjectSchema, Field } from '@objectstack/spec/data'; +import type { DataEngineInsertOptions, EngineQueryOptions } from '@objectstack/spec/data'; +import { MAILBOX_SERVICE, MailIngestService, MailboxServicePlugin } from './index.js'; +import { plainMessage } from './ingest/messages.fixture.js'; + +const SYSTEM = { isSystem: true } as const; +const ORG = 'org_kernel_mail'; + +const Contact = ObjectSchema.create({ + name: 'kerneltest_contact', + label: 'Contact', + fields: { + name: Field.text({ label: 'Name' }), + email: Field.email({ label: 'Email', mailAssociation: true }), + }, +}); + +describe('the `mailbox` kernel service on a real kernel boot', () => { + let kernel: ObjectKernel; + let ql: ObjectQL; + + beforeAll(async () => { + kernel = new ObjectKernel({ logger: { level: 'fatal' } }); + await kernel.use(new ObjectQLPlugin()); + await kernel.use(new MailboxServicePlugin()); + await kernel.bootstrap(); + ql = kernel.getService('objectql'); + const driver = new SqlDriver({ client: 'better-sqlite3', connection: { filename: ':memory:' }, useNullAsDefault: true }); + await driver.connect(); + ql.registerDriver(driver, true); + ql.registry.registerObject(Contact as any, 'mailbox-kernel-test', 'mailbox-kernel-test'); + await ql.syncSchemas(); + }, 60_000); + + afterAll(async () => { + try { + await kernel?.shutdown(); + } catch { + /* best-effort teardown */ + } + }); + + it('is the ingest seam, and it stores and associates a message through the kernel’s own engine', async () => { + const service = kernel.getService(MAILBOX_SERVICE); + expect(service).toBeInstanceOf(MailIngestService); + + const insert = async (object: string, data: Record) => + String(((await ql.insert(object, data, { context: SYSTEM } satisfies DataEngineInsertOptions)) as { id: unknown }).id); + const mailbox = await insert('sys_mailbox', { organization_id: ORG, name: 'Inbox', address: 'inbox@corp.test' }); + const contact = await insert('kerneltest_contact', { organization_id: ORG, name: 'C', email: 'customer@acme.test' }); + + const result = await service.ingest({ mailboxId: mailbox, raw: plainMessage({ 'Message-ID': '' }) }, SYSTEM); + + expect(result).toMatchObject({ + outcome: 'stored', + associationState: 'associated', + associations: [{ object: 'kerneltest_contact', recordId: contact, matchedField: 'email', matchedAddress: 'customer@acme.test' }], + }); + const rows = (await ql.find('sys_mail_message', { where: { mailbox_id: mailbox }, context: SYSTEM } satisfies EngineQueryOptions)) as unknown[]; + expect(rows).toHaveLength(1); + }); +}); diff --git a/packages/services/service-mailbox/src/mailbox-service-plugin.test.ts b/packages/services/service-mailbox/src/mailbox-service-plugin.test.ts index e965e345eb8..5531b9efa20 100644 --- a/packages/services/service-mailbox/src/mailbox-service-plugin.test.ts +++ b/packages/services/service-mailbox/src/mailbox-service-plugin.test.ts @@ -6,7 +6,9 @@ import { PLATFORM_CAPABILITIES } from '@objectstack/spec/security'; import { CAPABILITY_PROVIDERS } from '@objectstack/core'; import { MailboxServicePlugin, + MailIngestService, MAILBOX_ADMIN_CAPABILITY, + MAILBOX_SERVICE, MAILBOX_SERVICE_MANIFEST_ID, SysMailAssociation, SysMailbox, @@ -16,16 +18,21 @@ import { /** A plugin context that records the manifest it is handed and the hooks it arms. */ function fakeContext() { const registered: Array> = []; + const services = new Map(); const hooks: Record Promise>> = {}; const ctx: any = { getService: (name: string) => { if (name === 'manifest') return { register: (m: Record) => registered.push(m) }; throw new Error(`no service ${name}`); }, + registerService: (name: string, service: unknown) => { + if (services.has(name)) throw new Error(`service ${name} registered twice`); + services.set(name, service); + }, hook: (event: string, fn: () => Promise) => (hooks[event] ??= []).push(fn), logger: { info: () => {}, warn: () => {}, error: () => {} }, }; - return { ctx, registered, hooks }; + return { ctx, registered, services, hooks }; } describe('MailboxServicePlugin', () => { @@ -41,6 +48,16 @@ describe('MailboxServicePlugin', () => { ]); }); + it('registers the ingest seam as the `mailbox` kernel service, and only that service', async () => { + const { ctx, services } = fakeContext(); + await new MailboxServicePlugin().init(ctx); + expect([...services.keys()]).toEqual([MAILBOX_SERVICE]); + const service = services.get(MAILBOX_SERVICE) as MailIngestService; + expect(service).toBeInstanceOf(MailIngestService); + expect(typeof service.ingest).toBe('function'); + expect(typeof service.rematch).toBe('function'); + }); + it('is the provider the `mailbox` capability token names, under its registered identity', () => { const row = CAPABILITY_PROVIDERS.mailbox; expect(row?.pkg).toBe('@objectstack/service-mailbox'); diff --git a/packages/services/service-mailbox/src/mailbox-service-plugin.ts b/packages/services/service-mailbox/src/mailbox-service-plugin.ts index 68241d361fd..aed863979c6 100644 --- a/packages/services/service-mailbox/src/mailbox-service-plugin.ts +++ b/packages/services/service-mailbox/src/mailbox-service-plugin.ts @@ -4,6 +4,7 @@ import type { Plugin, PluginContext } from '@objectstack/core'; import type { II18nService } from '@objectstack/spec/contracts'; import { SysMailAssociation, SysMailbox, SysMailMessage } from './objects/index.js'; import { installMailReadVisibility, type MailGateEngine } from './mail-read-visibility.js'; +import { MAILBOX_SERVICE, MailIngestService, type MailIngestEngine } from './ingest/index.js'; /** The manifest id this plugin registers its objects under. */ export const MAILBOX_SERVICE_MANIFEST_ID = 'com.objectstack.service.mailbox'; @@ -22,9 +23,15 @@ export const MAILBOX_SERVICE_MANIFEST_ID = 'com.objectstack.service.mailbox'; * owner, and to holders of `manage_mailboxes` — so the unassociated pool is * restricted by default, never organization-wide. * - * The ingest engine that writes these rows (the IMAP connector, dedupe, - * association stamping and re-match) is the next half of the inbound-mail - * programme and lands in this package. + * Registers the ingest seam as the kernel service `mailbox` + * ({@link MAILBOX_SERVICE}, a {@link MailIngestService}): `ingest` turns one + * raw RFC 822 message into one `sys_mail_message` row, deduplicated per + * mailbox, with automatic mail (auto-replies, bounces) suppressed, and stamps + * its associations from the email fields an app flags with `mailAssociation`; + * `rematch` replays a mailbox's stored participant sets against the records + * that exist now and adds the missing links. Nothing feeds `ingest` yet: the + * IMAP connector that fetches a mailbox's mail is the next half of the + * inbound-mail programme. */ export class MailboxServicePlugin implements Plugin { name = MAILBOX_SERVICE_MANIFEST_ID; @@ -43,6 +50,16 @@ export class MailboxServicePlugin implements Plugin { objects: [SysMailbox, SysMailMessage, SysMailAssociation], }); + // The engine is resolved on each call, not here: the seam is reachable + // as soon as it is registered, and reads the engine only when used. + ctx.registerService( + MAILBOX_SERVICE, + new MailIngestService({ + engine: () => ctx.getService('objectql'), + logger: ctx.logger, + }), + ); + // ADR-0029 D8 — contribute the objects' translations on kernel:ready. ctx.hook('kernel:ready', async () => { try { diff --git a/packages/services/service-mailbox/vitest.config.ts b/packages/services/service-mailbox/vitest.config.ts index 431ebf16e79..c82820a8b8b 100644 --- a/packages/services/service-mailbox/vitest.config.ts +++ b/packages/services/service-mailbox/vitest.config.ts @@ -22,6 +22,9 @@ export default defineConfig({ alias: [ { find: /^@objectstack\/lint$/, replacement: path.resolve(__dirname, '../../lint/src/index.ts') }, { find: /^@objectstack\/core$/, replacement: path.resolve(__dirname, '../../core/src/index.ts') }, + { find: /^@objectstack\/types$/, replacement: path.resolve(__dirname, '../../types/src/index.ts') }, + { find: /^@objectstack\/objectql$/, replacement: path.resolve(__dirname, '../../objectql/src/index.ts') }, + { find: /^@objectstack\/driver-sql$/, replacement: path.resolve(__dirname, '../../drivers/driver-sql/src/index.ts') }, { find: /^@objectstack\/platform-objects\/apps$/, replacement: path.resolve(__dirname, '../../platform-objects/src/apps/index.ts') }, { find: /^@objectstack\/spec$/, replacement: path.resolve(__dirname, '../../spec/src/index.ts') }, { find: /^@objectstack\/spec\/data$/, replacement: path.resolve(__dirname, '../../spec/src/data/index.ts') }, diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 65f01aff0a3..7690b78fe57 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -402,7 +402,7 @@ importers: version: 6.0.3 vitest: specifier: ^4.1.11 - version: 4.1.11(@opentelemetry/api@1.9.1)(@types/node@26.6.3)(@vitest/coverage-v8@4.1.11)(happy-dom@20.10.2)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) + version: 4.1.11(@opentelemetry/api@1.9.1)(@types/node@26.6.3)(@vitest/coverage-v8@4.1.11)(happy-dom@20.10.2)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) packages/apps/setup: dependencies: @@ -2728,10 +2728,22 @@ importers: '@objectstack/spec': specifier: workspace:* version: link:../../spec + '@objectstack/types': + specifier: workspace:* + version: link:../../types + postal-mime: + specifier: ^4.0.5 + version: 4.0.5 devDependencies: + '@objectstack/driver-sql': + specifier: workspace:* + version: link:../../drivers/driver-sql '@objectstack/lint': specifier: workspace:* version: link:../../lint + '@objectstack/objectql': + specifier: workspace:* + version: link:../../objectql '@types/node': specifier: ^26.6.3 version: 26.6.3 @@ -8395,6 +8407,10 @@ packages: points-on-path@0.2.1: resolution: {integrity: sha512-25ClnWWuw7JbWZcgqY/gJ4FQWadKxGWk+3kR/7kD0tCaDtPPMj7oHu2ToLaVhfpnHrZzYby2w6tUA0eOIuUg8g==} + postal-mime@4.0.5: + resolution: {integrity: sha512-fWu1Cmo2ef6LCwgHU1fVrqo55eyFP31F7hH0j/wMWkQYYdh+H8zz3nBf3EYRQelkmTfDrbDfTvPUQEBT8DS/jw==} + engines: {node: '>=20'} + postcss-load-config@6.0.1: resolution: {integrity: sha512-oPtTM4oerL+UXmx+93ytZVN82RrlY/wPUV8IeDxFrzIjXOLF1pN+EmKPLbubvKHT2HC20xXsCAH2Z+CKV6Oz/g==} engines: {node: '>= 18'} @@ -11885,15 +11901,6 @@ snapshots: chai: 6.2.2 tinyrainbow: 3.1.0 - '@vitest/mocker@4.1.11(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1))': - dependencies: - '@vitest/spy': 4.1.11 - estree-walker: 3.0.3 - magic-string: 0.30.21 - optionalDependencies: - msw: 2.14.6(@types/node@26.6.3)(typescript@6.0.3) - vite: 8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1) - '@vitest/mocker@4.1.11(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1))': dependencies: '@vitest/spy': 4.1.11 @@ -14970,6 +14977,8 @@ snapshots: path-data-parser: 0.1.0 points-on-curve: 0.2.0 + postal-mime@4.0.5: {} + postcss-load-config@6.0.1(jiti@2.7.0)(postcss@8.5.28)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1): dependencies: lilconfig: 3.1.3 @@ -16001,21 +16010,6 @@ snapshots: '@types/unist': 3.0.3 vfile-message: 4.0.3 - vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1): - dependencies: - lightningcss: 1.33.0 - picomatch: 4.0.7 - postcss: 8.5.28 - rolldown: 1.0.3 - tinyglobby: 0.2.17 - optionalDependencies: - '@types/node': 26.6.3 - esbuild: 0.28.1 - fsevents: 2.3.3 - jiti: 2.7.0 - tsx: 4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6) - yaml: 2.9.1 - vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1): dependencies: lightningcss: 1.33.0 @@ -16031,37 +16025,6 @@ snapshots: tsx: 4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6) yaml: 2.9.1 - vitest@4.1.11(@opentelemetry/api@1.9.1)(@types/node@26.6.3)(@vitest/coverage-v8@4.1.11)(happy-dom@20.10.2)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)): - dependencies: - '@vitest/expect': 4.1.11 - '@vitest/mocker': 4.1.11(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)) - '@vitest/pretty-format': 4.1.11 - '@vitest/runner': 4.1.11 - '@vitest/snapshot': 4.1.11 - '@vitest/spy': 4.1.11 - '@vitest/utils': 4.1.11 - es-module-lexer: 2.3.1 - expect-type: 1.4.0 - magic-string: 0.30.21 - obug: 2.1.4 - pathe: 2.0.3 - picomatch: 4.0.5 - std-env: 4.2.0 - tinybench: 2.9.0 - tinyexec: 1.3.0 - tinyglobby: 0.2.17 - tinyrainbow: 3.1.0 - vite: 8.0.16(@types/node@26.6.3)(esbuild@0.28.1)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1) - why-is-node-running: 2.3.0 - optionalDependencies: - '@opentelemetry/api': 1.9.1 - '@types/node': 26.6.3 - '@vitest/coverage-v8': 4.1.11(vitest@4.1.11) - happy-dom: 20.10.2 - jsdom: 30.1.1(@noble/hashes@2.4.0) - transitivePeerDependencies: - - msw - vitest@4.1.11(@opentelemetry/api@1.9.1)(@types/node@26.6.3)(@vitest/coverage-v8@4.1.11)(happy-dom@20.10.2)(jsdom@30.1.1(@noble/hashes@2.4.0))(msw@2.14.6(@types/node@26.6.3)(typescript@6.0.3))(vite@8.0.16(@types/node@26.6.3)(esbuild@0.28.2)(jiti@2.7.0)(tsx@4.23.15(patch_hash=4d1d35da1809be2a945519cb26890e48107810b9b66c6913a6a89abb5ad6a1e6))(yaml@2.9.1)): dependencies: '@vitest/expect': 4.1.11