Skip to content

Commit 620a668

Browse files
committed
Merge remote-tracking branch 'origin/main' into claude/issue-23320-slot-judges-at-parse
Claude-Session: https://claude.ai/code/session_01YRp5z4kK6vgYkrdH97ztsy Co-authored-by: Claude <noreply@anthropic.com>
2 parents 92aefd3 + dc39326 commit 620a668

122 files changed

Lines changed: 11741 additions & 433 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
---
2+
'@objectstack/connector-imap': minor
3+
'@objectstack/service-mailbox': minor
4+
---
5+
6+
Inbound mail has a mail source: the new `@objectstack/connector-imap` reads a mailbox over IMAP, and `@objectstack/service-mailbox` polls each mailbox's folder into its ingest seam, keeping a UID cursor on `sys_mailbox`
7+
8+
Clause-②: yes (a new published package, `@objectstack/connector-imap`, and its exports; new exports and plugin options in `@objectstack/service-mailbox`, and four new fields on its published `sys_mailbox` object)
9+
10+
- **`@objectstack/connector-imap` (new) — protocol and authentication only.**
11+
- `openImapSession({ account, credentials, timeoutMs?, maxLiteralBytes?, maxCommandBytes? })` opens a read-only IMAP4rev1 session: `examine(folder)` (`EXAMINE`, never `SELECT`; answers `uidValidity`, `uidNext`, `exists`), `listMessages(fromUid)`, `fetchMessages(uids)` (`BODY.PEEK[]`, so no message is marked `\Seen`), `logout()` and `close()`. A folder opened without a UIDVALIDITY is refused.
12+
- The account and credential abstraction is shaped for the per-user mailbox: a `MailAccount` (`{ id, username, imap: { host, port?, security? } }`) and a `MailCredentialProvider` that resolves the account's credential each time a session opens. `staticCredentials(credential)` answers one fixed credential.
13+
- A `password` credential is sent with `AUTHENTICATE PLAIN` when the server offers it, otherwise `LOGIN`; an `oauth2` access token with `AUTHENTICATE XOAUTH2`. A server that refuses `LOGIN` and offers no `PLAIN` is refused before anything is sent.
14+
- `security: 'tls'` (the default) is implicit TLS with the certificate verified. A plaintext connection (`security: 'none'`) is refused to any host but loopback, before a socket opens.
15+
- Failures are `ImapError` with a `kind`: `configuration`, `connection`, `timeout`, `authentication`, `refused`, `protocol` or `oversized`.
16+
- An IMAP server is treated as an untrusted peer. A line above 4 MiB, a literal above `maxLiteralBytes` (64 MiB by default), a response of more than `IMAP_MAX_RESPONSE_LITERALS` (32) literals, and more than `maxCommandBytes` of responses to one command (128 MiB by default) are refused as `oversized` before the excess is kept, and the connection is closed.
17+
- `@objectstack/connector-imap/testing` exports `ImapServerDouble`, an in-process IMAP server on loopback for tests.
18+
- No runtime dependencies.
19+
- **`@objectstack/service-mailbox` — the IMAP sync.**
20+
- `new MailboxServicePlugin({ imap: { accountFor, pollIntervalMs?, batchSize?, maxMessagesPerRun?, maxMessageBytes?, maxBatchBytes? } })` schedules the interval job `mailbox:imap-sync` on the composed job service. For each mailbox `accountFor` answers (asked per mailbox per run; `null` for a mailbox the host does not sync), it fetches the messages above the stored cursor, in UID order, and hands each one to `ingest` under the system context. Dedupe, association, the pool and auto-reply suppression are unchanged.
21+
- Without `imap` — the capability boot constructs the plugin with no options — no mailbox is synced, and the boot says so once. With `imap` but no job service, the boot warns, and `plugin.syncService.syncAll({ isSystem: true })` runs a pass on the host's own schedule. A run in which a mailbox could not be synced reports `degraded` to the job service.
22+
- `sys_mailbox` gains the cursor, written by the sync only: `imap_folder`, `imap_uid_validity`, `imap_last_uid` and `imap_synced_at`. A cursor taken under another UIDVALIDITY, or in another folder, is discarded and the folder is read from its start; the messages already stored answer `duplicate`. No credential is stored on the object: the account and its credential come from `accountFor`.
23+
- The cursor moves after each batch past every message stored, already stored, suppressed or skipped. A store failure stops the run below the failed message, so the next run retries it. A message above `maxMessageBytes` (32 MiB by default) or one the seam refuses as unreadable is skipped and logged at `error` with its UID.
24+
- A batch is bounded by count (`batchSize`, 25) and by listed bytes (`maxBatchBytes`, 64 MiB). The session's literal bound is `maxMessageBytes` plus a 64 KiB framing envelope, so a server that omits or understates a message's size cannot hand over more. That message is skipped as `too_large`, after the batch is read again one message at a time on a fresh session, and the rest of the mailbox keeps arriving.
25+
- A forwarded `message/global` part (RFC 6532, the internationalized `message/rfc822`) is now read as an `attachment` source, the same as `message/rfc822`.
26+
- New exports: `MailboxSyncService`, `MAILBOX_SYNC_JOB`, `DEFAULT_MAILBOX_POLL_INTERVAL_MS`, `DEFAULT_SYNC_FOLDER`, `DEFAULT_SYNC_BATCH_SIZE`, `DEFAULT_SYNC_MAX_MESSAGES_PER_RUN`, `DEFAULT_SYNC_MAX_MESSAGE_BYTES`, `DEFAULT_SYNC_MAX_BATCH_BYTES`, `SYNC_LITERAL_ENVELOPE_BYTES`, and the types `MailboxServicePluginOptions`, `MailboxImapSyncOptions`, `MailboxAccountResolver`, `MailboxImapAccount`, `MailboxSyncTarget`, `MailboxSyncResult`, `MailboxSyncAllResult`, `MailboxSyncSkipReason`, `MailboxSyncServiceOptions`, `MailSyncEngine` and `MailSyncLogger`.
27+
- This supersedes the ingest seam entry's "no mail source feeds the seam": the IMAP sync does. Attachments are still not stored.
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
'@objectstack/types': minor
3+
'@objectstack/service-storage': patch
4+
'@objectstack/service-settings': patch
5+
'@objectstack/service-i18n': patch
6+
'@objectstack/plugin-sharing': patch
7+
---
8+
9+
fix: a route's catch-all `500` answers its own generic text, never the caught error's message, and `@objectstack/types` exports the writer that does it, `sendServerFault`
10+
11+
Clause-②: yes (widening) — `@objectstack/types` gains one export, `sendServerFault`; nothing is removed, renamed or narrowed, and no response field changes shape.
12+
13+
A declared 5xx does not carry the text of the error it caught: that text can name server internals, such as a path, a host or a table. The REST layer already withheld it. The storage, settings and i18n route families and the share-link routes answer in the nested `{ success: false, error: { code, message } }` envelope and did not: their catch-all `500` sent the caught error's message to the caller.
14+
15+
- **What a caller now reads.** The same `status` and `code` as before, with the route's own generic text as `error.message`: `Internal error` on the storage and i18n routes, the route's message on the settings routes (`Failed to list manifests`, `Failed to read namespace`, `Failed to write namespace`, `Action failed`), and the route's `Failed to …` message on the share-link routes. On the storage routes this includes the metadata store's outage sentence, which an operator now reads in the server log.
16+
- **What the operator reads.** The original error, stack included, in one `[5xx]` server-fault line. The storage byte routes, which already withheld the text, logged two lines per fault and now log one. A line names the mounted route pattern when the route passes one, never the request URL.
17+
- **Unchanged.** Every `4xx` answer. The settings `SETTINGS_CRYPTO_UNAVAILABLE` refusal, which keeps its prescription. The authorization-store outage `503`, which keeps its declared code and message. The storage `501 NOT_IMPLEMENTED` answers. On the share-link routes, any error that declares its own `status` keeps the envelope it declares.
18+
- **`sendServerFault(res, status, code, message, thrown, request?)`** (new, `@objectstack/types`): writes `message` in the declared envelope, and hands `thrown` to the server-fault log with its stack, as the fault's one line. Use it for an arm that answers a caught fault. Keep `sendError` for a `4xx` and for a `5xx` whose text you wrote for the reader. `request` is the mounted route pattern, never the request URL.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/plugin-security': minor
3+
---
4+
5+
feat(security)!: an anonymous caller never owns a record and never sets its owner — the owner-anchor stamp's guest branch (ADR-0138 E3, D1)
6+
7+
Clause-②: no (narrowing) — a guest-supplied owner is refused at the endpoints and the owner column never holds the guest or the system principal. That tightens what a guest write may carry and admits nothing refused today.
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) No authorable key, accept set of a metadata schema or stored shape moves: no spec schema, metadata type, stored row or published response field is added, removed, renamed or re-shaped. What narrows is a value an anonymous caller may send in a write, judged by the security middleware at run time; no metadata names it, so there is nothing for `objectstack migrate meta` to rewrite, and the record's own row for this card states the disposition "none: no new key, and nothing moves". The other categories are closed on facts: the package publishes (not unpublished); no ADR-0087 id covers this and this diff adds none (not registered or already-registered); no exported TypeScript declaration changes (not runtime-interface-only or type-surface-only); no export is removed (not runtime-export-retired). -->
10+
11+
**BREAKING** (an accept-set narrowing), shipped as `minor` under the launch-window convention for breaking changes.
12+
13+
ADR-0138 D1 rules that the guest is a principal, never an owner. The owner-anchor stamp in `SecurityPlugin` (the step that stamps an absent `owner_id` with the acting user and refuses a forged one) now carries the guest branch the record names as D1's enforcement point. It applies to the guest envelope, an anonymous request served by an `authRequired: false` endpoint:
14+
15+
- **Nothing is stamped for a guest.** An absent owner stays unset: never the guest, never the system principal. An empty `owner_id` (an empty string) is left unset too, where it used to be stored as an empty string.
16+
- **A guest-supplied owner is refused**, on create and on update, with `PERMISSION_DENIED` (`403`), whatever the guest is granted. The transfer grant is never consulted for a guest. Before this change a guest whose bound permission set carried the transfer bit could create a row owned by any user, the system principal included. The anchor tier already refuses that bit on a guest binding, so no binding a deployment can make today reached it, and this branch does not depend on the tier. Without the transfer bit the refusal already happened before this change; its message now says that no grant admits a guest's owner, instead of naming the transfer grant as the remedy.
17+
- **An owner the scenario sets stands.** A `beforeInsert` hook runs after the stamp, inside the engine's operation, and a record-change flow writes as itself, so neither is mistaken for a value the guest sent.
18+
19+
Unchanged: the public form doors, which strip a submitted owner and leave it unset (`PUBLIC_FORM_SERVER_MANAGED_FIELDS`); every signed-in caller, whose absent owner is still stamped with its own id and whose foreign owner still needs the transfer grant; and system writes.
20+
21+
What a guest client does: leave `owner_id` out of an anonymous write. The object's own hooks or record-change flows decide who owns the row, and the row stays unowned when none does.
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
'@objectstack/cli': minor
3+
'@objectstack/spec': minor
4+
---
5+
6+
feat(cli,spec): `os migrate organization-ownership --fate-supplement FILE`. A host declares the ADR-0131 D10 fates of the objects it provides, and the ceremony plans and applies them as it does the platform's own rows
7+
8+
Clause-②: yes (widening)
9+
10+
The v18 organization-ownership ceremony refuses any platform-prefixed table that carries `organization_id` when its inventory gives that table no fate. That is right for the platform's own objects, but a host's objects have no row in the inventory, so every database a host runs was refused. The host now declares those fates itself, in one JSON file, and passes the file to the command. **Adoption: a host passes its supplement through the flag; with no flag, the ceremony behaves as before.**
11+
12+
- **The file is declared in the spec.** `OrganizationOwnershipSupplementSchema` (`@objectstack/spec/system`) is a JSON array of `OrganizationOwnershipSupplementEntrySchema` rows. Each row gives `object`, `fate` (`column-drop`, `mirror-deletion`, `attribution` or `report`), `citation`, and only the fields its fate reads: `anchors` (required for `attribution`), `mirror`, `departures`, `categories`, `notNullExempt` and `note`. Every shape refuses an unknown key, and a key written on the wrong fate is refused rather than ignored.
13+
- **Both modes read it.** The plan plans each row on the same terms as a platform row, and `--apply` performs it the same way. A row whose table is not on the database being planned is listed `absent`, so one supplement serves both a control-plane database and a tenant database.
14+
- **It is refused, never half-read.** The command refuses a missing file, a file that is not JSON, a row the schema refuses, two rows for one object, or a row naming an object the platform inventory already gives a fate. The refusal comes before the database is opened, inside the `--json` envelope (`plan_refused` / `apply_refused`), and names the file, the row index and the object. A row naming one of the cloud-provided objects that the inventory lists with no fate (`cloudCarried`) replaces that placeholder row.
15+
- **Nothing is read without the flag:** no environment variable and no conventional path. A host table with no declared fate is still refused by name, and the refusal now names `--fate-supplement FILE` as the remedy.
16+
- The supplement's rows are part of the plan's `inventoryDigest`, so the apply's checkpoint is keyed on the supplement as well.
17+
18+
```bash
19+
os migrate organization-ownership --fate-supplement host-fates.json --out plan.json
20+
os migrate organization-ownership --fate-supplement host-fates.json --apply --backup-taken
21+
```
Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
---
2+
'@objectstack/client': minor
3+
---
4+
5+
feat(client): `meta.deleteItem` can name the package whose row it removes (`packageId`, sent as `?package=`), so a caller can follow the `409 RESOURCE_CONFLICT` remedy the reset door prescribes for an item two packages hold (#23293)
6+
7+
Clause-②: yes (widening)
8+
9+
**What is new.** `DeleteMetaItemOptions` gains `packageId?: string`. Both `deleteItem` declarations take it, the unscoped `ObjectStackClient.meta` and the environment-scoped `client.environment(id).meta`, and send it as `?package=<id>` through the one query builder they share. The name, the query key and the omission rule are the ones `saveItem`'s `packageId` already uses: only a non-empty id reaches the wire, and `undefined` and `''` send nothing.
10+
11+
**Why.** A metadata item can hold one row per package (ADR-0048). The REST reset door `DELETE /api/v1/meta/:type/:name` reads `?package=` and refuses a delete that names no package of an item two or more packages hold, with `409 RESOURCE_CONFLICT`. The refusal names the packages and the remedy, which is to name one with `?package=`. This client had no option for it, so an SDK caller (an agent, the CLI, an app) could not do what the refusal said.
12+
13+
FROM → TO:
14+
15+
```ts
16+
// FROM: refused 409 RESOURCE_CONFLICT when two packages hold the item, with no way to name one
17+
await client.meta.deleteItem('view', 'shared_grid');
18+
19+
// TO: removes com.acme.crm's row of the item and no other package's row
20+
await client.meta.deleteItem('view', 'shared_grid', { packageId: 'com.acme.crm' });
21+
```
22+
23+
**Unchanged.** A delete without `packageId` sends the same request as before: no query string unless `state: 'draft'` is set, and no header unless `ifMatch` is set. `packageId` combines with `ifMatch` and `state` (`?state=draft&package=<id>`). `?dropStorage` is still not an option.
Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
---
2+
'@objectstack/metadata-protocol': patch
3+
---
4+
5+
fix(metadata-protocol): an environment kernel that hydrates its own `sys_metadata` writes a published body through to its engine registry, so a narrowed permission set stops granting its old body without a restart
6+
7+
Clause-②: no
8+
9+
- **What was wrong.** The protocol's registry write-through skipped every type except `object` whenever the protocol had an `environmentId`. A kernel that hydrates its own store at boot (the standalone stack's `env_local`, or a hosted kernel with `hydrateMetadataFromDb`) put every stored body into its registry at boot and then never updated it. A save was accepted and served by `GET /api/v1/meta/:type/:name`, but the engine registry kept the boot-time body until a restart, and held nothing for an item created after boot. The security catalog reads that registry first, so a permission set narrowed through `PUT /api/v1/meta/permission/:name` kept resolving its earlier, wider body, and a permission set published after boot was not in the catalog at all: a user granted it was refused, and an `isDefault` set recorded no `everyone` suggestion.
10+
- **What changed.** The write-through now asks whose registry it is, not whether the kernel has an environment. It registers the published body wherever the registry takes this kernel's stored rows: on an unscoped kernel, as before, and on an environment kernel once the boot has hydrated that kernel's own store into it (`loadMetaFromDb`). The delete's heal re-registers a MetadataService baseline on the same kernels, so a save followed by a reset does not leave the deleted body in the registry.
11+
- **The owner.** The registry entry of a stored row now carries the row's package binding as `_packageId` on the boot and on the write-through, as a list read already stamped it, so a runtime package's `isDefault` set published through the metadata door is owned in the security catalog at once and records its `everyone` suggestion, on a host kernel and on a hydrating environment kernel alike; a row bound to no package stays unowned, and an artifact's own envelope still wins.
12+
- **Unchanged.** An environment kernel whose store is never hydrated into its registry still writes nothing into it, so a registry shared with unscoped callers never receives a project-scoped row. The `object` branch and the organization-scope refusal are unchanged. What the registry holds after a save is what it holds after a restart.

0 commit comments

Comments
 (0)