Skip to content

Commit 3a2884e

Browse files
committed
wip(service-mailbox): real-verb kernel pins, parse pins, README and changeset for mail attachments
Claude-Session: https://claude.ai/code/session_01YRp5z4kK6vgYkrdH97ztsy Co-authored-by: Claude <noreply@anthropic.com>
1 parent 0e98b7f commit 3a2884e

4 files changed

Lines changed: 352 additions & 4 deletions

File tree

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
'@objectstack/service-mailbox': minor
3+
---
4+
5+
Inbound mail stores its attachments: each attachment of an ingested message, and each message it forwards as an attachment as an `.eml` file, is stored through the `file-attachments` service as a file of the message's `sys_mail_message` row
6+
7+
Clause-②: no
8+
9+
- **What is stored, and where.** Every part of a stored message besides its text and HTML bodies (inline parts included, and each forwarded `message/rfc822` or `message/global` part as the `.eml` file it travels as) becomes one `sys_file` and one `sys_attachment` whose parent is the message's own `sys_mail_message` row, written by `IFileAttachmentService.attachFile` in the mailbox's organization after the row, and only for a message the call stored, so a re-delivered copy adds no attachment.
10+
- **Without the storage capability.** A kernel that composes no `file-attachments` service (registered by `@objectstack/service-storage`) stores each message without its attachments, and says so once in the log.
11+
- **Refused attachments.** A part the service refuses, such as one over the `max_upload_mb` upload size limit, is logged at `error` with its file name and the refusal, and the message and its other attachments are still stored.
12+
- **For a host that constructs `MailIngestService` itself.** It takes the service as the optional `attachments` resolver (`MailboxServicePlugin` supplies it), and its `MailIngestLogger` now needs `error`, as `MailSyncLogger` already does.

‎packages/services/service-mailbox/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ Inbound mail for ObjectStack: the mirrored message, its links to an app's record
44

55
> ⚠️ **Mail arrives only where an IMAP account source is composed.** The capability boot (`requires: ['mailbox']`) constructs the plugin with no options, so it mounts the objects, their read gates and the ingest seam, and syncs no mailbox — it says so once at boot. Compose the plugin yourself with `imap.accountFor` (below) to poll mailboxes.
66
>
7-
> ⚠️ **Attachments are not stored.** Nothing on the server writes `sys_attachment` rows today, so a message's attachments — and the body of a message forwarded as an attachment — are not kept; only its participants are read.
7+
> ⚠️ **Attachments are stored only where the storage capability is composed.** A message's attachments are written through the `file-attachments` service that `@objectstack/service-storage` registers (the `storage` capability, on the always-on slate). A kernel that does not compose it stores each message without its attachments, and says so once.
88
99
## What this package provides
1010

@@ -18,15 +18,15 @@ Inbound mail for ObjectStack: the mirrored message, its links to an app's record
1818
- **`manage_mailboxes`** — an organization capability declared in `@objectstack/spec`. Platform administrators and organization administrators hold it by default; every other position only by explicit grant. Object grants and field-level security still apply to a holder.
1919
- **The ingest seam** — `MailIngestService`, registered as the kernel service `mailbox` (`MAILBOX_SERVICE`):
2020
- **`ingest({ mailboxId, raw, receivedAt? }, context)`** reads one raw RFC 822 message and writes at most one `sys_mail_message` row, answering `stored`, `duplicate` or `suppressed`:
21-
- **participants** are the addresses of From, Sender, Reply-To, To and Cc, plus the same headers of a message forwarded as an attachment (a `message/rfc822` part, or its internationalized twin `message/global`, RFC 6532). At most `MAX_FORWARDED_PARTS` (20) forwarded parts of one message are read; the rest, and any part the MIME parser refuses, take no part in matching, and the message is stored without them (a warning names how many). The mailbox's own address is left out. A forward quoted inline in the body is read too, for an organization user's forward only: the "Forwarded message" block Gmail, Outlook (on the web and Microsoft 365) or Apple Mail writes into the text/plain body, with its English labels, adds its From, Reply-To, To and Cc — when the message's From is a user (`sys_user`) who is a member (`sys_member`) of the mailbox's organization. A block the parser does not recognise is not guessed at, and a forward from anyone else is not read: the message is matched on its other participants. The participants are stored on the row in `match_addresses`, each with its source (`outer`, `attachment` or `inline`);
21+
- **participants** are the addresses of From, Sender, Reply-To, To and Cc, plus the same headers of a message forwarded as an attachment (a `message/rfc822` part, or its internationalized twin `message/global`, RFC 6532). At most `MAX_FORWARDED_PARTS` (20) forwarded parts of one message are read; the rest, and any part the MIME parser refuses, take no part in matching (a warning names how many), and are still stored as attachments. The mailbox's own address is left out. A forward quoted inline in the body is read too, for an organization user's forward only: the "Forwarded message" block Gmail, Outlook (on the web and Microsoft 365) or Apple Mail writes into the text/plain body, with its English labels, adds its From, Reply-To, To and Cc — when the message's From is a user (`sys_user`) who is a member (`sys_member`) of the mailbox's organization. A block the parser does not recognise is not guessed at, and a forward from anyone else is not read: the message is matched on its other participants. The participants are stored on the row in `match_addresses`, each with its source (`outer`, `attachment` or `inline`);
2222
- **association** — every record of every object whose email field declares `mailAssociation: true` and holds one of those addresses (ASCII case-insensitive) is linked to the message, inside the mailbox's organization only, with one `sys_mail_association` row each (`matched_by: 'ingest'`). A message no record matches waits in the pool (`association_state: 'unassociated'`);
2323
- **dedupe** — one row per message per mailbox, keyed on the Message-ID (or, for a message without one, on its header block as received), so a second fetch answers `duplicate` and writes nothing;
2424
- **threading** — `message_id`, `in_reply_to` and `reference_ids` hold the ids of RFC 5322 §3.6.4, angle brackets stripped;
2525
- **suppression** — an auto-reply (`Auto-Submitted` other than `no`, `Precedence: bulk|junk|list`, `X-Autoreply`, `X-Autorespond`) or a bounce (`multipart/report; report-type=delivery-status`) is not stored at all.
2626

2727
Only the system context may ingest: a message enters a mailbox from its mail source, never from a user.
2828
- **`rematch({ mailboxId, messageIds? }, context)`** replays each message's stored participant set against the records that exist now and adds the links that are missing (`matched_by: 'rematch'`), moving a pool message to `associated` once it has one. It never removes a link. The system context may run it, and so may the mailbox's owner and a `manage_mailboxes` holder of the mailbox's organization; anyone else is refused (`PERMISSION_DENIED`), and another organization's mailbox is not found (`RECORD_NOT_FOUND`).
29-
- Attachments are not stored yet, so a message forwarded as an attachment is stored with the forwarder's own text, and the forwarded message's body is not kept.
29+
- **attachments** — each part of a stored message besides its text and HTML bodies is stored as a file of its `sys_mail_message` row: one `sys_file` and one `sys_attachment` (parent `sys_mail_message`, the row's id), through `IFileAttachmentService.attachFile`, the kernel service `file-attachments` (`FILE_ATTACHMENT_SERVICE`). Both rows land in the mailbox's organization, and the deployment's upload size limit (`max_upload_mb`) applies. That covers every file the message carries, a part marked `Content-Disposition: inline` included (Apple Mail marks attached files that way, and an image the HTML body shows by `cid:` reference is part of the message), and every message forwarded as an attachment (`message/rfc822` or `message/global`), stored as the `.eml` file it travels as: named by the part's own file name, else by its Subject, else `forwarded.eml`. A part with no file name is `attachment-<n>`, its position among the message's parts. Parts are stored only for a message this call stored, after its row: a duplicate or a suppressed message stores none, so a re-delivery adds no attachment, and a re-match never reads a part. A part the service refuses (over the upload size limit, for instance) is logged at `error` with its name and the refusal, and the message and its other parts are still stored. No column of the row names a part, and no byte is kept in a column;
3030
- **The IMAP sync** — `MailboxSyncService`, composed with `MailboxServicePlugin({ imap })` and run as the interval job `mailbox:imap-sync` on the composed job service (`requires: ['job']`):
3131
- for every mailbox your `accountFor` answers, it opens a **read-only** session on its IMAP account through `@objectstack/connector-imap` — the folder is opened with `EXAMINE` and bodies are fetched with `BODY.PEEK[]`, so the mailbox is never altered — and hands each message above the stored cursor, in UID order, to `ingest` under the system context. Dedupe, threading, association, the pool and suppression are the seam's, unchanged;
3232
- **the cursor** is kept on the mailbox row — `imap_folder`, `imap_uid_validity`, `imap_last_uid`, `imap_synced_at` — and written by the sync only. A cursor taken under another UIDVALIDITY, or in another folder, is discarded and the folder is read from its start; every message already stored then answers `duplicate`;

‎packages/services/service-mailbox/src/ingest/parse-message.test.ts‎

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ import {
1414
parseMatchAddresses,
1515
serializeMatchAddresses,
1616
} from './match-addresses.js';
17-
import { forwardAsAttachment, plainMessage, rfc822 } from './messages.fixture.js';
17+
import { binaryBytes, forwardAsAttachment, messageWithParts, plainMessage, rfc822 } from './messages.fixture.js';
1818

1919
const ORIGINAL = rfc822(
2020
[
@@ -137,6 +137,9 @@ describe('parseInboundMessage — a message forwarded as an attachment (source `
137137
const parsed = await parseInboundMessage(raw);
138138
expect(parsed.forwarded).toHaveLength(MAX_FORWARDED_PARTS);
139139
expect(parsed.unreadForwardedParts).toBe(2);
140+
// Their addresses are not read, but their bytes are kept as attachments.
141+
expect(parsed.attachments).toHaveLength(MAX_FORWARDED_PARTS + 2);
142+
expect(parsed.attachments.slice(-2).map((a) => a.forwarded)).toEqual([{ subject: null }, { subject: null }]);
140143
});
141144

142145
it('control: a forward quoted inline in the body names no forwarded participants (that is the `inline` source)', async () => {
@@ -151,6 +154,50 @@ describe('parseInboundMessage — a message forwarded as an attachment (source `
151154
});
152155
});
153156

157+
describe('parseInboundMessage — the parts stored as attachments of the row', () => {
158+
it('lists every part besides the bodies, in message order, with its bytes as carried', async () => {
159+
const png = binaryBytes();
160+
const pdf = binaryBytes(7);
161+
const parsed = await parseInboundMessage(
162+
messageWithParts({
163+
messageId: '<parts-1@corp.test>',
164+
parts: [
165+
{ contentType: 'image/png', disposition: 'inline; filename="logo.png"', contentId: '<logo@corp.test>', bytes: png },
166+
{ contentType: 'application/pdf; name="apple.pdf"', disposition: 'inline; filename="apple.pdf"', bytes: pdf },
167+
{ contentType: 'application/octet-stream', bytes: pdf },
168+
{ contentType: 'message/rfc822', disposition: 'attachment; filename="question.eml"', message: ORIGINAL },
169+
{ contentType: 'message/global', message: ORIGINAL },
170+
],
171+
}),
172+
);
173+
expect(parsed.attachments.map((a) => [a.filename, a.mimeType, a.forwarded])).toEqual([
174+
['logo.png', 'image/png', null],
175+
['apple.pdf', 'application/pdf', null],
176+
[null, 'application/octet-stream', null],
177+
['question.eml', 'message/rfc822', { subject: 'A question' }],
178+
[null, 'message/global', { subject: 'A question' }],
179+
]);
180+
expect(parsed.attachments[0]!.content).toEqual(png);
181+
expect(parsed.attachments[1]!.content).toEqual(pdf);
182+
// A forwarded message is kept exactly as it travels in its part.
183+
expect(new TextDecoder().decode(parsed.attachments[3]!.content)).toBe(ORIGINAL);
184+
expect(parsed.text?.trim()).toBe('See attached.');
185+
});
186+
187+
it('control: a message with no part besides its body lists none, and a second text part joins the body', async () => {
188+
expect((await parseInboundMessage(plainMessage())).attachments).toEqual([]);
189+
const boundary = 'two-texts';
190+
const parsed = await parseInboundMessage(
191+
rfc822(
192+
[['From', 'rep@corp.test'], ['MIME-Version', '1.0'], ['Content-Type', `multipart/mixed; boundary="${boundary}"`]],
193+
[`--${boundary}`, 'Content-Type: text/plain', '', 'one', `--${boundary}`, 'Content-Type: text/plain', '', 'two', `--${boundary}--`, ''].join('\r\n'),
194+
),
195+
);
196+
expect(parsed.attachments).toEqual([]);
197+
expect(parsed.text).toContain('two');
198+
});
199+
});
200+
154201
describe('the participant set (`match_addresses`)', () => {
155202
it('normalizes ASCII letters only, and refuses what is not an addr-spec', () => {
156203
expect(normalizeMailAddress(' Jane.DOE@Acme.TEST ')).toBe('jane.doe@acme.test');

0 commit comments

Comments
 (0)