Skip to content

Commit b60eb78

Browse files
feat(service-mailbox): read inline forwards from Gmail, Outlook and Apple Mail, for an organization user's forward only (#23459)
Fixes #23215 Clause-②: no The inline-forward half of the ingest seam that ruling `6116489638` (Q3 C, staged) left for this card. `@objectstack/service-mailbox`'s ingest now reads the "Forwarded message" block that Gmail, Outlook and Apple Mail write into a forward's body, and only for a forward sent by a user of the mailbox's organization. The parsed participants join `match_addresses` under the already-declared source `inline`, after the `outer` and `attachment` entries. No spec, connector-imap or attachment-storage file is touched, and the package entry gains no export (the built `dist/index.d.ts` names none of the new symbols), so the claim's `Clause-②: no` stands. ## What changed - **`src/ingest/inline-forward.ts` (new, pure).** `parseInlineForward(text)` answers the block's format and participants, or `null`. It reads three formats, each in its English text/plain shape: - Gmail: the `---------- Forwarded message ---------` line, then `From:`, `Date:`, `Subject:`, `To:`, `Cc:`. - Outlook on the web and Microsoft 365: a line of 32 underscores, then `From:`, `Sent:`, `To:`, `Cc:` (a semicolon list), `Subject:`. - Apple Mail: `Begin forwarded message:`, an empty line, then `From:`, `Subject:`, `Date:`, `To:`, `Cc:`, `Reply-To:`. Each line may carry the `>` quote marker the client's plain-text rendering adds. - **What is recognised.** The first separator line in the body decides. Its block counts only when it is complete: - the header lines follow the separator directly (Apple Mail allows one empty line first); - every header line is a label of that format, each label appears once, and `From:` comes first; - the format's date line is present; - `From:` names an address. Anything else answers `null`, and nothing is guessed. That covers another client's or locale's labels, a wrapped header line, prose after the separator, a block quoted by a reply, and an incomplete first block followed by a complete one. - **`MailIngestService.ingest`.** The set is built as outer, then attachment, then inline. The inline part is read only behind the organization-user door (next section). `rematch` is unchanged. - **Docs.** The docblocks are updated: `match-addresses.ts` no longer says `inline` is written nowhere, `parse-message.ts` no longer calls the inline source separate work, and `mail-ingest-service.ts` lists the new decision. The package README's sentence "A forward quoted inline in the body is not read yet" is rewritten. - **Changeset.** `.changeset/23215-inline-forward-parse.md` grades `@objectstack/service-mailbox` as `minor`, a behaviour widening. It carries the door sentence and the miss sentence. ## The organization-user reading (H2), as implemented **Measured reach.** `ingest` holds the ObjectQL engine (system context), the mailbox row (`id`, `address`, `owner_user_id`, `organization_id`) and the registry. **The door is open only when all three hold:** 1. The outer `From` names exactly one address. Two or more keep it closed. 2. A `sys_user` row's `email` equals that address, ASCII case-insensitive. The row is found with `$icontains`, and equality is decided on the value read back, the same way the record matcher decides. The read is bounded at `SENDER_USER_SCAN_LIMIT` (100), and the bound is reported when it is hit. 3. That user has a `sys_member` row with `organization_id` equal to the mailbox's organization. A membership in another organization does not open the door. **How the reads run:** - Both are system reads. - They run only for a message whose body carries a recognised block. The parser is pure, so it runs first, and a message with no forward pays nothing. - The door adds no dependency and no published contract: `sys_user` and `sys_member` are the identity objects `@objectstack/plugin-auth` registers. - Where those objects are not registered, the door stays closed, and the first message that carries a block says so at `warn`. **Not implemented:** - The dispatch's other candidate, "From is the mailbox's own or declared address". A mailbox address is not a user, and the ruling's door is "an organisation user". - Any sender allow-list. ## Formats (H3) - **Sources.** The shapes are the client output in the MIT-licensed `email-forward-parser` (Crisp IM SAS): `test/fixtures/gmail_en_body.txt`, `outlook_live_body.txt`, `apple_mail_en_body.txt`, and the per-client separator patterns its parser documents. `messages.fixture.ts` reproduces each shape with synthetic addresses and cites its source. - **Body read.** `ParsedInboundMessage.text` is the text/plain part. Measured: postal-mime gives `text` as null for a message whose only body is HTML (pinned). So an HTML-only forward is not read and pools. All three clients send a text/plain alternative. - **Localized labels.** - Gmail localizes only the From label (`Von:`, `De :`). - Outlook localizes the labels. - Apple Mail localizes the labels and the separator. This PR reads the English labels only. A localized block is not recognised and pools (pinned with Gmail's `Von:`). ## Re-match (H4) `rematch` reads `id`, `match_addresses` and `association_state` only, and replays the stored set through `parseMatchAddresses`. It never reads `body_text`. Pinned both ways: - an org user's inline forward that pooled is associated by a later re-match once a record matches its `inline` address; - a stored row whose body quotes a forward its stored set does not carry is not associated by re-match. ## Tests (all at `a870b3c82`, the head of this PR) **Package checks.** Each ran through `scripts/pm/os-verify-lock.sh` and printed `VERDICT command-exit 0`. - `pnpm --workspace-concurrency=2 --filter '@objectstack/service-mailbox...' build` (the package and its upstream closure): exit 0. - `pnpm --filter @objectstack/service-mailbox typecheck`: exit 0. The tsconfig includes `src`, test files too; `check:type-check-coverage` exit 0. - `pnpm --filter @objectstack/service-mailbox exec vitest run --maxWorkers=2 --reporter=verbose`: **9 files, 132 tests passed**. That is 101 at the base plus 31 new: `inline-forward.test.ts` 24, and 7 in `mail-ingest-service.test.ts`. The kernel boot test `mailbox-service-kernel.test.ts` (1) and `mailbox-sync-kernel.test.ts` (4) are green. **The pins, on a real ObjectQL engine over better-sqlite3, with the real `sys_user` and `sys_member` registered:** - **One per client format** (Gmail, Outlook, Apple Mail). The forward associates through the original sender's `mailAssociation` field, and the stored set equals outer `rep@corp.test`, then inline `customer-FORMAT@acme.test` and `rep@corp.test`. The control is the same message without the block, which pools with an outer-only set. - **An unrecognised block pools** (Gmail's German `Von:`), with no `inline` entry. - **The door stays closed** for no user, for a user who is a member of another organization only, and for two From addresses. The control is the same block forwarded by the organization's user, which associates. - **No identity objects registered.** The forward is not read, and the warning is said once over two messages. - **Re-match** replays `inline` and never re-reads the body. **Ablation.** One, on the parser's call, run at `0c49e3c8e` with the implementation committed. `node scripts/ablation-replace.mjs` replaced `const forward = parseInlineForward(parsed.text);` with a constant `null`. The tool printed: anchor x1 to x0, marker x0 to x1, blob `ecbf9870d513` to `018d7b59ea60`, and an on-disk count of 1 marker and 0 anchors before the run. The direction was RED: `mail-ingest-service.test.ts` reported 6 failed and 21 passed, namely the three per-format pins, the door's control leg, the said-once pin and the re-match replay. The restore was proven: blob `ecbf9870d513` equals HEAD, `git diff HEAD` is empty, `git status --porcelain` is empty, and the marker count is 0. The subject is imported by relative path inside the package, so the run read `src` and no `dist/` was involved. ## Gates `node scripts/pm/dispatch-gates.mjs --commands` (no paths) derived 67 commands at `a870b3c82`, the same list as at `0c49e3c8e`. Each command ran locally with its exit code captured before any pipe. **Local result.** At `a870b3c82`, 67 of 67 exited 0. `node scripts/pm/dispatch-gates.mjs --ran` printed: `67 derived famil(ies) accounted for — 67 run, 0 NOT-MEASURED`. **The earlier run.** The first run, at `0c49e3c8e`, recorded `check:dual-build-cjs-loads` and `check:i18n` as exit 3 (PREREQUISITE NOT MET, workspace packages without `dist/`). The rerun measured both green: - `check:dual-build-cjs-loads`: 110 entry points across 68 packages load. - `check:i18n`: `check-i18n-bundles: OK (10 package(s) — all bundles in sync`. **Freshness.** The reconciliation says this tree is 6 commits behind `origin/main` `7b3d5585c`, in four `scripts/pm/fleet-write/*` files. None of those commits touches `service-mailbox`. In its dependency closure, the only change is a refusal message in `packages/spec/src/data/controlled-by-parent-master.ts`. So this PR's CI, which runs on the merge ref, is the reading for that range. Every family CI runs is read from this PR's own check-runs, not re-run locally. ## Consumer readings `git grep` at `a870b3c82` found 0 hits in `examples/**` and 0 in `packages/qa/**` for `match_addresses`, `MAIL_MATCH_ADDRESS_SOURCES`, `collectMatchAddresses`, `participantAddresses` and `MailIngestService`: - **`service-mailbox` string.** One hit in `packages/qa`: a comment in `mail-message-pool-read-gate.dogfood.test.ts`. That test writes its rows directly and never calls `ingest`. - **`'inline'`.** The 11 qa hits are unrelated: seed `mode` values and the expression-conformance roster. - **The showcase.** It composes `requires: ['mailbox']` (`examples/app-showcase/objectstack.config.ts:108`) with no IMAP source, so its mail arrives only through the service. ## Acceptance notes - **File surface.** Beyond `src/ingest/**` and the changeset, this touches two files: - `packages/services/service-mailbox/README.md`: its sentence that inline forwards are not read became false with this change. - `vitest.config.ts`: source aliases for `@objectstack/platform-objects/identity`, `@objectstack/spec/shared` and `@objectstack/spec/identity`, so the real-engine test registers the real identity objects from source. - **Pending stage-1 changeset.** `.changeset/22871-mail-ingest-seam.md` (pending, another card's) says "A forward quoted inline in the body is not read yet". It is not edited here. Both changesets publish in the same release, and this one states the change. - **Not read, and the message pools.** Localized blocks; Outlook desktop's plain-text `-----Original Message-----` and its no-separator HTML alternative (no cited client sample); forwards rendered only in HTML. - **Not consulted by the door.** `sys_user.banned` is not read. The door is membership only, which is the ruling's "organisation user". --- _Generated by [Claude Code](https://claude.ai/code/session_01YRp5z4kK6vgYkrdH97ztsy)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 8d5b7bb commit b60eb78

11 files changed

Lines changed: 775 additions & 13 deletions
Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
---
2+
'@objectstack/service-mailbox': minor
3+
---
4+
5+
Inbound mail reads a forward quoted inline in the body: a message an organization user forwards inline from Gmail, Outlook or Apple Mail now associates with the records its original participants match
6+
7+
Clause-②: no
8+
9+
- **What is read.** The "Forwarded message" block each of the three clients writes into the text/plain body, with its English labels: Gmail's `---------- Forwarded message ---------` block (From, Date, Subject, To, Cc), the block Outlook on the web and Microsoft 365 write under a line of 32 underscores (From, Sent, To, Cc, Subject), and Apple Mail's `Begin forwarded message:` block (From, Subject, Date, To, Cc, Reply-To). Its From, Reply-To, To and Cc addresses join the participant set in `match_addresses` under the source `inline`, after the outer and attachment entries. `rematch` replays them with the rest of the stored set and never reads the body again.
10+
- **Only an organization user's forward is read.** The block is read only when the message's From names one address and that address is the email of a user (`sys_user`, ASCII case-insensitive) who is a member (`sys_member`) of the mailbox's organization. A forward from anyone else, a member of another organization included, is matched on its own headers only. Where the identity objects are not registered (no auth plugin is composed), no inline forward is read, and the first message that carries one says so in the log.
11+
- **An unrecognised block pools.** A block the parser does not recognise is not guessed at: labels of another client or locale (Gmail in German writes `Von:`), a header line wrapped onto two lines, a block with no date line, a forward quoted by a reply, or a message with an HTML body only. The message is matched on its other participants, so a forward whose only match was in the block waits in the pool. Only the first block in the body is read; a forward quoted inside that forward is not.

‎packages/services/service-mailbox/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Inbound mail for ObjectStack: the mirrored message, its links to an app's record
1818
- **`manage_mailboxes`** — an organization capability declared in `@objectstack/spec`. Platform administrators and organization administrators hold it by default; every other position only by explicit grant. Object grants and field-level security still apply to a holder.
1919
- **The ingest seam** — `MailIngestService`, registered as the kernel service `mailbox` (`MAILBOX_SERVICE`):
2020
- **`ingest({ mailboxId, raw, receivedAt? }, context)`** reads one raw RFC 822 message and writes at most one `sys_mail_message` row, answering `stored`, `duplicate` or `suppressed`:
21-
- **participants** are the addresses of From, Sender, Reply-To, To and Cc, plus the same headers of a message forwarded as an attachment (a `message/rfc822` part, or its internationalized twin `message/global`, RFC 6532). At most `MAX_FORWARDED_PARTS` (20) forwarded parts of one message are read; the rest, and any part the MIME parser refuses, take no part in matching, and the message is stored without them (a warning names how many). The mailbox's own address is left out. They are stored on the row in `match_addresses`, each with its source (`outer` or `attachment`). A forward quoted inline in the body is not read yet;
21+
- **participants** are the addresses of From, Sender, Reply-To, To and Cc, plus the same headers of a message forwarded as an attachment (a `message/rfc822` part, or its internationalized twin `message/global`, RFC 6532). At most `MAX_FORWARDED_PARTS` (20) forwarded parts of one message are read; the rest, and any part the MIME parser refuses, take no part in matching, and the message is stored without them (a warning names how many). The mailbox's own address is left out. A forward quoted inline in the body is read too, for an organization user's forward only: the "Forwarded message" block Gmail, Outlook (on the web and Microsoft 365) or Apple Mail writes into the text/plain body, with its English labels, adds its From, Reply-To, To and Cc — when the message's From is a user (`sys_user`) who is a member (`sys_member`) of the mailbox's organization. A block the parser does not recognise is not guessed at, and a forward from anyone else is not read: the message is matched on its other participants. The participants are stored on the row in `match_addresses`, each with its source (`outer`, `attachment` or `inline`);
2222
- **association** — every record of every object whose email field declares `mailAssociation: true` and holds one of those addresses (ASCII case-insensitive) is linked to the message, inside the mailbox's organization only, with one `sys_mail_association` row each (`matched_by: 'ingest'`). A message no record matches waits in the pool (`association_state: 'unassociated'`);
2323
- **dedupe** — one row per message per mailbox, keyed on the Message-ID (or, for a message without one, on its header block as received), so a second fetch answers `duplicate` and writes nothing;
2424
- **threading** — `message_id`, `in_reply_to` and `reference_ids` hold the ids of RFC 5322 §3.6.4, angle brackets stripped;
Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* The inline-forward parser on its own: the three client formats it reads,
5+
* and the blocks it refuses to guess at. The blocks are the fixtures'
6+
* reproductions of each client's text/plain output (`messages.fixture.ts`
7+
* names the source of each shape), and the end-to-end cases read them back
8+
* through the MIME parser, as the ingest seam does.
9+
*/
10+
11+
import { describe, expect, it } from 'vitest';
12+
import { INLINE_FORWARD_FORMATS, parseInlineForward } from './inline-forward.js';
13+
import { parseInboundMessage, participantAddresses } from './parse-message.js';
14+
import {
15+
appleMailForwardBlock,
16+
forwardInline,
17+
gmailForwardBlock,
18+
outlookForwardBlock,
19+
rfc822,
20+
} from './messages.fixture.js';
21+
22+
const CUSTOMER_SET = ['customer@acme.test', 'rep@corp.test', 'buyer@acme.test', 'partner@globex.test'];
23+
24+
describe('parseInlineForward — the three client formats', () => {
25+
it('reads exactly the formats it declares', () => {
26+
expect([...INLINE_FORWARD_FORMATS]).toEqual(['gmail', 'outlook', 'apple_mail']);
27+
});
28+
29+
it('reads a Gmail block: From, To and Cc, in header order', () => {
30+
expect(parseInlineForward(gmailForwardBlock())).toEqual({
31+
format: 'gmail',
32+
participants: {
33+
from: ['customer@acme.test'],
34+
sender: [],
35+
replyTo: [],
36+
to: ['rep@corp.test'],
37+
cc: ['buyer@acme.test', 'partner@globex.test'],
38+
},
39+
});
40+
});
41+
42+
it('reads an Outlook block, its `addr <addr>` To and its `;` Cc list included', () => {
43+
expect(parseInlineForward(outlookForwardBlock())).toEqual({
44+
format: 'outlook',
45+
participants: {
46+
from: ['customer@acme.test'],
47+
sender: [],
48+
replyTo: [],
49+
to: ['rep@corp.test'],
50+
cc: ['buyer@acme.test', 'partner@globex.test'],
51+
},
52+
});
53+
});
54+
55+
it('reads an Apple Mail block through its quote markers, Reply-To included', () => {
56+
expect(parseInlineForward(appleMailForwardBlock({ replyTo: 'Replies <replies@acme.test>' }))).toEqual({
57+
format: 'apple_mail',
58+
participants: {
59+
from: ['customer@acme.test'],
60+
sender: [],
61+
replyTo: ['replies@acme.test'],
62+
to: ['rep@corp.test'],
63+
cc: ['buyer@acme.test', 'partner@globex.test'],
64+
},
65+
});
66+
});
67+
68+
it('reads each format out of a received message body, through the MIME parser', async () => {
69+
for (const [format, block] of [
70+
['gmail', gmailForwardBlock()],
71+
['outlook', outlookForwardBlock()],
72+
['apple_mail', appleMailForwardBlock()],
73+
] as const) {
74+
const parsed = await parseInboundMessage(forwardInline({ messageId: `<e2e-${format}@corp.test>`, block }));
75+
const forward = parseInlineForward(parsed.text);
76+
expect(forward?.format, format).toBe(format);
77+
expect(participantAddresses(forward!.participants), format).toEqual(CUSTOMER_SET);
78+
}
79+
});
80+
81+
it('control: the same message without the block carries no forward', async () => {
82+
const parsed = await parseInboundMessage(forwardInline({ messageId: '<e2e-none@corp.test>', block: null }));
83+
expect(parsed.text).toContain('below');
84+
expect(parseInlineForward(parsed.text)).toBeNull();
85+
});
86+
87+
it('reads the forward the user made, not a forward quoted inside it', () => {
88+
const nested = gmailForwardBlock({ body: `Forwarding this on.\r\n\r\n${outlookForwardBlock({ from: 'Deep <deep@elsewhere.test>' })}` });
89+
expect(participantAddresses(parseInlineForward(nested)!.participants)).toEqual(CUSTOMER_SET);
90+
});
91+
});
92+
93+
describe('parseInlineForward — a block it does not recognise is not guessed at', () => {
94+
const unrecognised: Array<[string, string]> = [
95+
['no separator at all', 'From: Customer <customer@acme.test>\r\nDate: today\r\n\r\nHi'],
96+
['a separator followed by prose', '---------- Forwarded message ---------\r\nHere it is.'],
97+
['a localized From label (Gmail in German writes `Von:`)', gmailForwardBlock().replace('From: ', 'Von: ')],
98+
['a label of another format (`Sent:` in a Gmail block)', gmailForwardBlock().replace('Date: ', 'Sent: ')],
99+
['a label no format writes', gmailForwardBlock().replace('Cc: ', 'Bcc: ')],
100+
['a label twice', gmailForwardBlock({ cc: null }).replace('Subject: A question', 'To: second@acme.test')],
101+
['From not first', gmailForwardBlock().replace(/(From: [^\r]*)\r\n(Date: [^\r]*)/, '$2\r\n$1')],
102+
['no date line', gmailForwardBlock().replace(/Date: [^\r]*\r\n/, '')],
103+
['a From that names no address', gmailForwardBlock({ from: 'Customer' })],
104+
['a wrapped header line', gmailForwardBlock({ cc: 'Buyer <buyer@acme.test>,\r\n Partner <partner@globex.test>' })],
105+
['a Gmail block quoted by a reply', gmailForwardBlock().split('\r\n').map((l) => `> ${l}`).join('\r\n')],
106+
['the separator of a client it does not read (Thunderbird)', gmailForwardBlock().replace('---------- Forwarded message ---------', '-------- Forwarded Message --------')],
107+
['the separator of a client it does not read (Outlook desktop, plain text)', outlookForwardBlock().replace('________________________________', '-----Original Message-----')],
108+
['an Apple Mail separator followed by prose', '> Begin forwarded message:\r\n>\r\n> Have a look.'],
109+
];
110+
111+
for (const [name, text] of unrecognised) {
112+
it(`answers null for ${name}`, () => {
113+
expect(parseInlineForward(text)).toBeNull();
114+
});
115+
}
116+
117+
it('the first separator decides: an incomplete block is not skipped to reach a complete one below it', () => {
118+
const text = `${gmailForwardBlock().replace('From: ', 'Von: ')}\r\n\r\n${outlookForwardBlock()}`;
119+
expect(parseInlineForward(text)).toBeNull();
120+
// Control: the complete block alone is read.
121+
expect(parseInlineForward(outlookForwardBlock())?.format).toBe('outlook');
122+
});
123+
124+
it('answers null for an empty or absent body', () => {
125+
expect(parseInlineForward('')).toBeNull();
126+
expect(parseInlineForward(null)).toBeNull();
127+
expect(parseInlineForward(undefined)).toBeNull();
128+
});
129+
130+
it('reads the text/plain body only: a message with an HTML body only carries no text to read', async () => {
131+
const html = `<div>See below.<br><br>${gmailForwardBlock().split('\r\n').join('<br>')}</div>`;
132+
const parsed = await parseInboundMessage(
133+
rfc822(
134+
[
135+
['From', 'Rep <rep@corp.test>'],
136+
['To', 'inbox@corp.test'],
137+
['Message-ID', '<html-only@corp.test>'],
138+
['MIME-Version', '1.0'],
139+
['Content-Type', 'text/html; charset=utf-8'],
140+
],
141+
html,
142+
),
143+
);
144+
expect(parsed.html).toContain('Forwarded message');
145+
expect(parsed.text).toBeNull();
146+
expect(parseInlineForward(parsed.text)).toBeNull();
147+
});
148+
});
Lines changed: 191 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,191 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* A forward quoted INLINE in the body text: the `inline` source of the
5+
* participant set (ruling 6116489638 on #22870, Q3 C, the staged half).
6+
*
7+
* Exactly three client formats are read, as each client writes the forward
8+
* into the text/plain body with English labels:
9+
*
10+
* - **Gmail** — the line `---------- Forwarded message ---------`, then
11+
* `From:`, `Date:`, `Subject:`, `To:` and `Cc:` lines;
12+
* - **Outlook** (Outlook on the web, Outlook.com, Microsoft 365) — a line of
13+
* 32 underscores, then `From:`, `Sent:`, `To:`, `Cc:` and `Subject:` lines;
14+
* - **Apple Mail** — the line `Begin forwarded message:`, an empty line, then
15+
* `From:`, `Subject:`, `Date:`, `To:`, `Cc:` and `Reply-To:` lines, each
16+
* line of the block carrying the `>` quote marker the client's plain-text
17+
* rendering adds (or not).
18+
*
19+
* The shapes are those of the client output collected in the MIT-licensed
20+
* `email-forward-parser` (Crisp IM SAS) — its fixtures `gmail_en_body.txt`,
21+
* `outlook_live_body.txt` and `apple_mail_en_body.txt`, and the separator
22+
* patterns its parser documents per client. The fixtures in
23+
* `messages.fixture.ts` reproduce them.
24+
*
25+
* ## What is recognised, and nothing else
26+
*
27+
* The FIRST separator line of the three formats in the body decides: the
28+
* forward the user made sits above any forward quoted inside it, which is not
29+
* read (as a `message/rfc822` part nested in a forwarded part is not). Its
30+
* block is recognised only when it is complete:
31+
*
32+
* - the header lines follow the separator directly (Apple Mail: after at most
33+
* one empty line) and run to the first empty line or the end of the text;
34+
* - every one of them is `Label: value` with a label of THAT format, each
35+
* label at most once, and `From:` comes first;
36+
* - the format's date line (`Date:`, or Outlook's `Sent:`) is present;
37+
* - `From:` names at least one address.
38+
*
39+
* Anything else — a separator followed by prose, a label of another format or
40+
* locale (Gmail's `Von:`, Outlook's `Gesendet:`), a wrapped header line, a
41+
* block with no date line, a `From:` naming no address — is not recognised,
42+
* and {@link parseInlineForward} answers `null`: nothing is guessed, and the
43+
* message is matched on its other participants only, which may pool it.
44+
*
45+
* The body read is the text/plain one (`ParsedInboundMessage.text`). A message
46+
* with an HTML body only has none, so its forward is not read either.
47+
*
48+
* Pure: text in, participants out. Whether a recognised forward's participants
49+
* are USED is the ingest seam's decision (`mail-ingest-service.ts`): only for a
50+
* forward sent by a user of the mailbox's organization.
51+
*/
52+
53+
import { addressParser } from 'postal-mime';
54+
import type { Address } from 'postal-mime';
55+
import { normalizeMailAddress } from './match-addresses.js';
56+
import type { MailParticipantHeaders } from './parse-message.js';
57+
58+
/** The client formats {@link parseInlineForward} reads. */
59+
export const INLINE_FORWARD_FORMATS = ['gmail', 'outlook', 'apple_mail'] as const;
60+
61+
/** One of {@link INLINE_FORWARD_FORMATS}. */
62+
export type InlineForwardFormat = (typeof INLINE_FORWARD_FORMATS)[number];
63+
64+
/** A recognised inline forward: its format and the original message's participants. */
65+
export interface InlineForward {
66+
format: InlineForwardFormat;
67+
/** The addresses the block names, per participant header (`sender` is always empty). */
68+
participants: MailParticipantHeaders;
69+
}
70+
71+
type ParticipantHeader = keyof MailParticipantHeaders;
72+
73+
interface ClientFormat {
74+
format: InlineForwardFormat;
75+
/** Matched against one line, trailing whitespace removed (and the quote marker, where the client quotes). */
76+
separator: RegExp;
77+
/** Every label of the block → the participant header it fills, or `null` for a label read for its presence only. */
78+
labels: ReadonlyMap<string, ParticipantHeader | null>;
79+
/** The label the format always writes the original's date under. */
80+
dateLabel: string;
81+
/** Each line of the block may carry one `>` quote marker. */
82+
quoted: boolean;
83+
/** One empty line may sit between the separator and the header lines. */
84+
emptyLineBeforeHeaders: boolean;
85+
}
86+
87+
const FORMATS: ReadonlyArray<ClientFormat> = [
88+
{
89+
format: 'gmail',
90+
// Ten dashes and nine today; the pattern documented for the client allows 8–10 either side.
91+
separator: /^-{8,10} Forwarded message -{8,10}$/,
92+
labels: new Map<string, ParticipantHeader | null>([
93+
['From', 'from'],
94+
['Date', null],
95+
['Subject', null],
96+
['To', 'to'],
97+
['Cc', 'cc'],
98+
]),
99+
dateLabel: 'Date',
100+
quoted: false,
101+
emptyLineBeforeHeaders: false,
102+
},
103+
{
104+
format: 'outlook',
105+
separator: /^_{32}$/,
106+
labels: new Map<string, ParticipantHeader | null>([
107+
['From', 'from'],
108+
['Sent', null],
109+
['To', 'to'],
110+
['Cc', 'cc'],
111+
['Subject', null],
112+
]),
113+
dateLabel: 'Sent',
114+
quoted: false,
115+
emptyLineBeforeHeaders: false,
116+
},
117+
{
118+
format: 'apple_mail',
119+
separator: /^Begin forwarded message:$/,
120+
labels: new Map<string, ParticipantHeader | null>([
121+
['From', 'from'],
122+
['Subject', null],
123+
['Date', null],
124+
['To', 'to'],
125+
['Cc', 'cc'],
126+
['Reply-To', 'replyTo'],
127+
]),
128+
dateLabel: 'Date',
129+
quoted: true,
130+
emptyLineBeforeHeaders: true,
131+
},
132+
];
133+
134+
const HEADER_LINE = /^([^\s:]+):(?: (.*))?$/;
135+
136+
/** One line as a format reads it: trailing whitespace dropped, and one quote marker where the client quotes. */
137+
function lineAs(format: ClientFormat, raw: string): string {
138+
const line = raw.trimEnd();
139+
return format.quoted ? line.replace(/^> ?/, '') : line;
140+
}
141+
142+
/** The addresses of one header value, in order; a name with no address names none. */
143+
function addressesOf(value: string): string[] {
144+
const out: string[] = [];
145+
const visit = (list: ReadonlyArray<Address>) => {
146+
for (const entry of list) {
147+
if (entry.group) visit(entry.group);
148+
else if (entry.address && normalizeMailAddress(entry.address) !== null) out.push(entry.address);
149+
}
150+
};
151+
visit(addressParser(value));
152+
return out;
153+
}
154+
155+
/** The block after a separator at `lines[start - 1]`, or `null` when it is not complete. */
156+
function readBlock(format: ClientFormat, lines: ReadonlyArray<string>, start: number): InlineForward | null {
157+
let at = start;
158+
if (format.emptyLineBeforeHeaders && at < lines.length && lineAs(format, lines[at]!) === '') at++;
159+
160+
const participants: MailParticipantHeaders = { from: [], sender: [], replyTo: [], to: [], cc: [] };
161+
const seen = new Set<string>();
162+
for (; at < lines.length; at++) {
163+
const line = lineAs(format, lines[at]!);
164+
if (line === '') break;
165+
const header = HEADER_LINE.exec(line);
166+
if (!header) return null;
167+
const label = header[1]!;
168+
if (!format.labels.has(label) || seen.has(label)) return null;
169+
if (seen.size === 0 && label !== 'From') return null;
170+
seen.add(label);
171+
const target = format.labels.get(label);
172+
if (target) participants[target] = addressesOf(header[2] ?? '');
173+
}
174+
if (!seen.has(format.dateLabel) || participants.from.length === 0) return null;
175+
return { format: format.format, participants };
176+
}
177+
178+
/**
179+
* The inline forward the body text carries, or `null` when it carries none
180+
* this module recognises (see the module docblock for what is recognised).
181+
*/
182+
export function parseInlineForward(text: string | null | undefined): InlineForward | null {
183+
if (!text) return null;
184+
const lines = text.split(/\r?\n/);
185+
for (let i = 0; i < lines.length; i++) {
186+
for (const format of FORMATS) {
187+
if (format.separator.test(lineAs(format, lines[i]!))) return readBlock(format, lines, i + 1);
188+
}
189+
}
190+
return null;
191+
}

0 commit comments

Comments
 (0)