diff --git a/.changeset/24529-declared-webhooks-from-registry.md b/.changeset/24529-declared-webhooks-from-registry.md new file mode 100644 index 00000000000..4fad5d6836d --- /dev/null +++ b/.changeset/24529-declared-webhooks-from-registry.md @@ -0,0 +1,30 @@ +--- +'@objectstack/plugin-webhooks': minor +'@objectstack/runtime': minor +'@objectstack/spec': patch +--- + +feat(plugin-webhooks,runtime)!: declared webhooks are read from the registry — the boot seeder retires, no `sys_webhook` row is written for one, and it is switched through the activation ledger + +Clause-②: yes (narrowing) + + + +**BREAKING** (a narrowing of where a declared webhook can be edited), shipped as `minor` under the launch-window convention for breaking changes. + +ADR-0131 D2 says no seeder writes declared items into a table, and D6 charts behavioural types under regime C. A webhook declared with `defineStack({ webhooks })` now follows both, as declared sharing rules already do: the dispatcher reads the declaration from the metadata registry on every subscription-cache refresh, and `bootstrapDeclaredWebhooks` no longer runs. + +What changes: + +- **No `sys_webhook` row for a declared webhook.** Before: every boot wrote or refreshed one `managed_by: 'package'` row per declared webhook, shown and editable in Setup → Integrations → Webhooks. Now: a fresh install writes none, and Setup lists only the webhooks an organization authors. The declared webhook still delivers, with the same URL, method, triggers, `timeoutMs`, signing secret and headers; its secret and headers are code, held in memory, and never stored. Its deliveries carry `ref_id = declared:`. +- **A declared webhook is read-only, and switched through the activation ledger.** Before: an administrator edited the seeded row in Setup (`customized: true`). Now: switch it on or off for the deployment with `POST /api/v1/security/_activation/webhook/` and `{ "enabled": true | false }` — one `sys_metadata_activation` row of type `webhook`, which overrides the declared `isActive` in either direction, so a webhook shipped inactive is switched on the same way. The door is the declared-sharing-rule door with `webhook` admitted: `manage_metadata`, then the platform operator under a wall; `404` for a name nothing declares; `503` when no declared-webhook reader is bound or it cannot be read. The dispatcher refreshes in the request that wrote the row. Remedy for a change beyond on/off: author a copy under a new name as a `sys_webhook` row, and switch the declared one off. +- **`label` and `description` on a declared webhook are no longer shown anywhere.** The seeded row was their only reader. They are display-only, so no delivery changes; the liveness ledger marks them dead, owed enforce-or-remove. + +On a database an 18.0 install created (expand first — nothing is moved or deleted): + +- The rows the 18.0 seeder wrote stay where they are. A pristine one (`managed_by: 'package'`, not `customized`) is ignored in favour of its declaration, so a declared webhook is never delivered twice; it follows the declaration's URL and switch, as the seeder's rewrite on every boot made it do. +- A seeded row an administrator edited (`customized: true`), or a row an administrator authored under a declared name, answers as itself, and the declaration is not dispatched beside it — an administrator's `active: false` on a noisy webhook keeps holding. This is logged at `warn` once per name per process, with the remedy. +- A seeded row whose declaration was removed keeps delivering, as it did on 18.0. +- Redelivering an old delivery of a pristine seeded row is judged by the declaration that now answers for it, so a webhook switched off through the ledger is not re-sent. + +Deleting the seeded rows, and re-pointing their `sys_http_delivery` history at the declaration, is a step of the ownership migration in a later 18.x release. diff --git a/content/docs/automation/webhooks.mdx b/content/docs/automation/webhooks.mdx index b5397669cb9..2b0499a3c0d 100644 --- a/content/docs/automation/webhooks.mdx +++ b/content/docs/automation/webhooks.mdx @@ -80,9 +80,38 @@ Two persisted objects underpin the runtime, owned by different packages. Both are defined as `ObjectSchema.create()` schemas so they participate in CRUD, permissions, audit, and Studio UI without bespoke code. +### 3.0 Declared webhooks (code) and `sys_webhook` rows + +A webhook reaches the dispatcher one of two ways: + +- **Declared in code** — `defineStack({ webhooks })` / `defineWebhook()`. The + declaration is read from the metadata registry on every subscription-cache + refresh, and **no `sys_webhook` row is written for it** (ADR-0131 D2). Its + signing secret and headers are code too, held in memory with it and never + stored. A declared webhook is read-only: switch it on or off for the + deployment with + `POST /api/v1/security/_activation/webhook/:name` and body + `{ "enabled": true | false }`, which writes one `sys_metadata_activation` row + of type `webhook` and overrides the declared `isActive` in either direction + (ADR-0131 D6, regime C — the door, its authority tiers and its refusals are + the ones the declared-sharing-rule switch uses). To change one, author a copy + under a new name as a `sys_webhook` row and switch the declared one off. + Deliveries of a declared webhook carry `ref_id = declared:`. +- **Authored by an organization** — a `sys_webhook` row (§3.1), switched by its + own `active` field. + +On a database an 18.0 install created, the rows that release's boot seeder +wrote for declared webhooks (`managed_by: 'package'`) are still there. Until the +ownership migration removes them, a pristine one is ignored in favour of its +declaration, so the webhook is never delivered twice; one an administrator +edited (`customized: true`) answers as itself, and the declaration of the same +name is not dispatched beside it — an administrator's `active: false` keeps +holding. + ### 3.1 `sys_webhook` -The subscription record. One row per "I want webhook X to fire for object Y". +The subscription record an organization authors. One row per "I want webhook +X to fire for object Y". The authored envelope — the per-attempt `timeoutMs` and the rest of the `Webhook` config — is carried in `definition_json`, a serialised `Webhook` JSON (canonical schema: `WebhookSchema`, exported from @@ -149,8 +178,9 @@ readable through an ordinary `GET /api/v1/data/sys_webhook`. They now go to `signing_secret` and `headers_secret`; existing rows are migrated on boot by one idempotent sweep that moves both passengers and strips them from the blob in a single update. Authoring does not change — `defineWebhook({ secret, headers })` -is written exactly as before, and the boot materializer routes each value to its -own column. +is written exactly as before; a declared webhook's values are now read from the +declaration itself and stored nowhere (§3.0), and an authored row's go to their +own columns. A row the sweep has not converted — for example a pre-v17 row on a runtime with no `CryptoProvider` wired, where the encrypted write is refused and the row is @@ -366,9 +396,9 @@ UNIQUE constraint. For each incoming event the subscriber: -1. Loads `sys_webhook` rows where `active = true` AND `object_name` - matches the event's object AND the event's action is in the row's - `triggers`. +1. Loads `sys_webhook` rows where `active = true`, and the declared + webhooks switched on (§3.0), whose `object_name` matches the event's + object AND whose `triggers` include the event's action. 2. For each match, builds the payload (§5). 3. Enqueues a `sys_http_delivery` row (`source = 'webhook'`). diff --git a/content/docs/permissions/tenant-audit-census.mdx b/content/docs/permissions/tenant-audit-census.mdx index f4daf3476a2..2431dab14da 100644 --- a/content/docs/permissions/tenant-audit-census.mdx +++ b/content/docs/permissions/tenant-audit-census.mdx @@ -122,7 +122,7 @@ are reported as `undecidable` rather than assumed either way. The same holds twice over for the context. An options argument spelled as a literal can be read; one spelled `options`, `{ ...opts }`, or handed through a -forwarding shim cannot, and **52 of the 232 sites are spelled that way**. A +forwarding shim cannot, and **50 of the 230 sites are spelled that way**. A context resolved from an inline literal or a local `const` can be tested for `isSystem`; one arriving from a helper call cannot. @@ -150,8 +150,8 @@ now **0**: nothing on this surface threads a context that provably lacks the fla **"No tenant context" counted sites it had not read.** An options argument the walker could not parse was folded into the same bucket as one it had read and -found empty. That published **60 sites "carrying no tenant context at all"** -when 8 said so and 52 were simply unread — an over-claim in the *alarming* +found empty. That published **58 sites "carrying no tenant context at all"** +when 8 said so and 50 were simply unread — an over-claim in the *alarming* direction, on the very figure this page tells other cards to cite. `carries` is now three-valued, and an unreadable argument can never contribute to the provable count. @@ -187,10 +187,10 @@ reproduce them. Where it disagrees, it disagrees on the page: | carried figure | where it survives | this census | | :--- | :--- | ---: | -| 175 write call sites | quoted in the merged changeset | **232** | -| 24 carrying no tenant context | quoted in the merged changeset | **2** provable and tenancy-enabled; **31** more whose options argument is unreadable | -| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **158 of 232** decidable, **74** undecidable | -| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 129 decidably elevated, 0 decidably not, 95 undecidable | +| 175 write call sites | quoted in the merged changeset | **230** | +| 24 carrying no tenant context | quoted in the merged changeset | **2** provable and tenancy-enabled; **29** more whose options argument is unreadable | +| 127 of 175 statically decidable, 48 runtime-parameter-name sites | restated on the `isSystem`-scoping card | **156 of 230** decidable, **74** undecidable | +| 135 (77%) silenced by the `isSystem` guard before the posture gate | the lost issue body — **no surviving corroboration** | **not reproduced**: 129 decidably elevated, 0 decidably not, 93 undecidable | | 141 and 132, two independent re-derivations | the card that filed this work | — | **The differences are not reconciled, and deliberately so.** The old census's @@ -201,20 +201,20 @@ at any commit. Two structural facts do plausibly widen this reading against any hand or regex one, and both are counted in the generated tables below: the 41 sites reached -through an erased (`any`) receiver, and the 51 that name their object through a +through an erased (`any`) receiver, and the 49 that name their object through a `const` rather than inline. An instrument that read either the way a person does would report a smaller number and would not say so. The fourth row is the one worth flagging to anyone citing it. **The 135 / 77% figure has no surviving corroboration anywhere in the tree.** This census reads -129 of 232 (56%) as decidably elevated, with 95 more whose elevation is a +129 of 230 (56%) as decidably elevated, with 93 more whose elevation is a run-time fact — so the claim is neither confirmed nor refuted, and the honest answer is that a static reading cannot settle it. -⇒ **Cite `2 / 232`, and say what it is**: the sites whose options argument was +⇒ **Cite `2 / 230`, and say what it is**: the sites whose options argument was READ and holds no tenant context, against a decidably tenancy-enabled object. That is the control's provable yield surface. ⛔ Do not cite it as "the sites -without tenant context" — **31 further sites** have an options argument this +without tenant context" — **29 further sites** have an options argument this cannot read, and they are neither in nor out. {/* BEGIN GENERATED: tenant-audit-census (scripts/tenant-audit-census.mjs) — DO NOT EDIT */} @@ -223,29 +223,29 @@ cannot read, and they are neither in nor out. | what | count | | :--- | ---: | -| write call sites on the application surface | **232** | -| …whose object name is statically decidable | 158 | +| write call sites on the application surface | **230** | +| …whose object name is statically decidable | 156 | | …whose object name is chosen at run time | 74 | -| …against an object with tenancy ENABLED | 157 | +| …against an object with tenancy ENABLED | 155 | | …against an object that declares tenancy off | 1 | | threading a tenant context | 172 | | PROVABLY carrying none (options read, no context key) | **8** | | …of those, against a decidably tenancy-enabled object | **2** | -| options argument UNREADABLE — may or may not carry one | 52 | -| …of those, against a decidably tenancy-enabled object | 31 | +| options argument UNREADABLE — may or may not carry one | 50 | +| …of those, against a decidably tenancy-enabled object | 29 | | threading a decidably ELEVATED (`isSystem`) context | 129 | | threading a context that is decidably NOT elevated | 0 | -| threading a context whose elevation is a run-time fact | 95 | +| threading a context whose elevation is a run-time fact | 93 | | how the instrument reached the site | count | | :--- | ---: | -| receiver carried a readable engine type | 191 | +| receiver carried a readable engine type | 189 | | receiver erased, placed by the object NAME | 31 | | receiver erased, placed by an `object: string` PARAMETER | 5 | | receiver erased, placed by an `UNTYPED_RECEIVERS` row | 5 | | object name spelled inline | 107 | -| object name spelled through a `const` | 51 | +| object name spelled through a `const` | 49 | | object name is an `object: string` parameter | 7 | | object name is some other run-time expression | 67 | @@ -297,7 +297,7 @@ holds still. They are required to be HERE and to say WHEN they were true; their values are not compared. The reasoning, and the measurement behind it, are in `scripts/check-tenant-audit-census.mjs`. -Measured on 2026-10-19 at `030483e7f`. +Measured on 2026-10-19 at `fa6a837e7`. | corpus scale (not enforced) | count | | :--- | ---: | diff --git a/docs/audits/2026-08-tenant-audit-write-call-sites.counts.md b/docs/audits/2026-08-tenant-audit-write-call-sites.counts.md index d26e4b18034..d26ced0960a 100644 --- a/docs/audits/2026-08-tenant-audit-write-call-sites.counts.md +++ b/docs/audits/2026-08-tenant-audit-write-call-sites.counts.md @@ -33,19 +33,19 @@ silent, and `node scripts/tenant-audit-census.mjs --write` is the resolution. | Measure | Value | |---|---:| -| Write call sites | 232 | -| Object name statically decidable | 158 | +| Write call sites | 230 | +| Object name statically decidable | 156 | | Object name chosen at run time | 74 | -| Against a tenancy-enabled object | 157 | +| Against a tenancy-enabled object | 155 | | Against an object declaring tenancy off | 1 | | Threading a tenant context | 172 | | Provably carrying none | 8 | | …and decidably tenancy-enabled | 2 | -| Options argument unreadable | 52 | -| …and decidably tenancy-enabled | 31 | +| Options argument unreadable | 50 | +| …and decidably tenancy-enabled | 29 | | Threading a decidably elevated context | 129 | | Threading a decidably non-elevated context | 0 | -| Threading a context of undecidable elevation | 95 | +| Threading a context of undecidable elevation | 93 | ## Subtractions the census could NOT defend — enforced @@ -90,7 +90,7 @@ holds still. They are required to be HERE and to say WHEN they were true; their values are not compared. The reasoning, and the measurement behind it, are in `scripts/check-tenant-audit-census.mjs`. -Measured on 2026-10-19 at `030483e7f`. +Measured on 2026-10-19 at `fa6a837e7`. | corpus scale (not enforced) | count | | :--- | ---: | @@ -183,8 +183,6 @@ Measured on 2026-10-19 at `030483e7f`. | `packages/plugins/plugin-sharing/src/sharing-service.ts` | `delete` | `sys_record_share` | enabled | elevated | 2 | | `packages/plugins/plugin-sharing/src/sharing-service.ts` | `insert` | `sys_record_share` | enabled | elevated | 1 | | `packages/plugins/plugin-sharing/src/sharing-service.ts` | `update` | `sys_record_share` | enabled | elevated | 1 | -| `packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts` | `insert` | `sys_webhook` | enabled | options unreadable | 1 | -| `packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts` | `update` | `sys_webhook` | enabled | options unreadable | 1 | | `packages/plugins/plugin-webhooks/src/migrate-webhook-secrets.ts` | `update` | `subscriptionsObject` | undecidable | options unreadable | 1 | | `packages/services/service-automation/src/builtin/crud-nodes.ts` | `delete` | `objectName` | undecidable | context, elevation undecidable | 1 | | `packages/services/service-automation/src/builtin/crud-nodes.ts` | `insert` | `objectName` | undecidable | context, elevation undecidable | 1 | diff --git a/docs/qa/platform-checklist/areas/integration-system.json b/docs/qa/platform-checklist/areas/integration-system.json index a05b3c99f7e..62b6f4b4803 100644 --- a/docs/qa/platform-checklist/areas/integration-system.json +++ b/docs/qa/platform-checklist/areas/integration-system.json @@ -519,17 +519,17 @@ }, { "id": "integration-system.webhook-lifecycle", - "title": "Outbound webhooks materialize (spec object→object_name, isActive→active), fire per trigger variant through the sys_http_delivery outbox with HMAC + timeout honored, redeliver terminal rows tenant-scoped via POST /api/v1/webhooks/redeliver, reject retired trigger kinds, and never clobber admin edits", + "title": "Outbound webhooks are read from the registry (spec object→object_name, no sys_webhook row) and switched through the activation ledger, fire per trigger variant through the sys_http_delivery outbox with HMAC + timeout honored, redeliver terminal rows tenant-scoped via POST /api/v1/webhooks/redeliver, and reject retired trigger kinds", "since": "v15", "status": "active", - "revision": 7, + "revision": 8, "priority": "P1", "surface": "mixed", "fixtures": { "app": "showcase", "requires": [ - "a reachable webhook receiver (local echo server on the run's own port range) and a SCRATCH webhook authored INSIDE the showcase organization that points at it — Setup → Integrations → Webhooks → New as the signed-in admin, or POST /api/v1/data/sys_webhook under that session, so the row carries the organization. ⚠ The shipped showcase_task_changed cannot carry the per-record or bulk delivery legs: it materializes ORG-LESS (a declared webhook belongs to no organization), and the auto-enqueuer hands an organization-walled event only to a subscription of that organization (packages/plugins/plugin-webhooks/src/auto-enqueuer.ts, AutoEnqueuer.admitsOrganization). On the org-walled showcase it therefore never fans out, while Setup reads it active:true (QA run #23166)", - "the shipped fixture: showcase_task_changed (object showcase_task, triggers create/update/delete, isActive:false ON PURPOSE — flipping it active in Setup is part of the test, examples/app-showcase/src/automation/webhooks/index.ts)", + "a reachable webhook receiver (local echo server on the run's own port range) and a SCRATCH webhook authored INSIDE the showcase organization that points at it — Setup → Integrations → Webhooks → New as the signed-in admin, or POST /api/v1/data/sys_webhook under that session, so the row carries the organization. ⚠ The shipped showcase_task_changed cannot carry the per-record or bulk delivery legs: it is a declared webhook, read-only, so its url stays the placeholder hooks.example and cannot be pointed at the receiver. Once switched on it DOES deliver under `single` — a declared webhook is deployment-wide there and each delivery is stamped with the event's organization (packages/plugins/plugin-webhooks/src/auto-enqueuer.ts, AutoEnqueuer.admitsOrganization, CachedSubscription.deploymentWide) — and under a walled posture it would be refused as belonging to no organization", + "the shipped fixture: showcase_task_changed (object showcase_task, triggers create/update/delete, isActive:false ON PURPOSE — switching it on through the activation door is part of the test, examples/app-showcase/src/automation/webhooks/index.ts)", "a predicate multi-write path for the bulk variants (update/delete with multi:true on showcase_task) — NOT reachable over REST, see the knownGap below: author a flow with an `update_record` / `delete_record` node carrying multi:true and fire it through the api trigger" ], "knownGaps": [ @@ -547,21 +547,21 @@ "retired: api (manual fire) — no fire path exists (#3196); parse-rejected" ], "steps": [ - "boot the showcase; verify the materializer bridge: read sys_webhook over /api/v1/data and locate showcase_task_changed with object_name:'showcase_task', active:false, managed_by:'package', and the envelope in definition_json", - "in Setup → Integrations → Webhooks flip the shipped showcase_task_changed row active and point its url at the local receiver (an admin edit — it must stamp customized). This edit is the subject of the seed-not-clobber clause (acceptance[6]) ONLY: the row stays org-less, so it delivers nothing on the org-walled showcase (fixtures.requires[0]). Author the in-organization scratch webhook beside it for the delivery legs", + "boot the showcase on a fresh database; verify the declared webhook is NOT materialized (ADR-0131 D2 retired the boot seeder): read sys_webhook over /api/v1/data and find NO showcase_task_changed row — the dispatcher reads the declaration from the registry", + "switch the shipped declared showcase_task_changed on as the admin: POST /api/v1/security/_activation/webhook/showcase_task_changed with { enabled: true } — 200 { type: 'webhook', name, enabled: true }, and one sys_metadata_activation row of metadata_type 'webhook'; an undeclared name answers 404. A declared webhook is read-only: pointing it at the receiver means a copy under a new name (the scratch webhook below). This switch is the subject of acceptance[6] ONLY: switched on, the declaration delivers under `single` (one sys_http_delivery row with ref_id declared:showcase_task_changed, stamped with the Default Organization, per task change), but only to its placeholder url (fixtures.requires[0]). Author the in-organization scratch webhook beside it for the delivery legs", "with the in-organization scratch webhook (fixtures.requires[0]) active and pointed at the receiver, create, update, then delete a showcase_task; capture the three deliveries at the receiver (headers incl. the HMAC signature when a secret is set, body incl. recordId). The create needs showcase_task's required `project` lookup — pass an existing showcase_project id, or the create answers 400 VALIDATION_FAILED 'Project is required' and no create event fires", "run a predicate multi-update and multi-delete (multi:true) matching several rows — through a FLOW node, not REST: author update_record / delete_record nodes with multi:true and fire them via the api trigger (the REST batch routes strip options.multi, #3897 — see the knownGap); capture the bulk deliveries and their { object, matched } shape", "read sys_http_delivery over /api/v1/data: one row per delivery with status/attempts/lastStatusCode", "kill the receiver and mutate again; re-read the delivery row through its retry/failure states", "restore the receiver, let the failing row reach a terminal status (failed/dead), then POST /api/v1/webhooks/redeliver with { deliveryId: } as an authenticated user: capture the 200 { id, status } and the receiver's re-hit — compare the replayed X-Objectstack-Signature byte-for-byte against the original delivery's; re-read the row; then probe the refusals: anonymous POST (401), non-JSON body (400 INVALID_REQUEST), body without deliveryId (400 MISSING_REQUIRED_FIELD), a garbage/unknown deliveryId (404), a row still pending/in_flight (409 DELIVERY_NOT_ELIGIBLE)", - "redeploy/reboot and confirm the admin-edited row survived re-seed (customized rows are never clobbered)" + "redeploy/reboot and confirm the activation switch survived: the sys_metadata_activation row is still there and the declaration still reads switched on (the switch is deployment state; the declaration is code)" ], "acceptance": [ { - "clause": "authoring is LIVE, not a no-op: the declared webhook materializes into the sys_webhook row the dispatcher reads, with the two documented remaps (object→object_name, isActive→active) and the envelope in definition_json (#3461/#3489)", + "clause": "authoring is LIVE, not a no-op: the declared webhook is read from the registry by the dispatcher (ADR-0131 D2) — NO sys_webhook row is written for it — and the activation door finds it by name", "oracle": "api", - "verify": "the sys_webhook row fields against bootstrap-declared-webhooks mapWebhookToRow", - "evidence": "the row read" + "verify": "zero sys_webhook rows named showcase_task_changed on a fresh boot; the activation door answers 200 for it and 404 for an undeclared name (declared-webhooks.ts DeclaredWebhookSource, the projection in auto-enqueuer.ts projectDeclared)", + "evidence": "the row read + the two door responses" }, { "clause": "each per-record trigger variant delivers exactly its event: create/update/delete each produce one receiver hit whose payload names the event and carries the recordId — verified per-variant, none inferred", @@ -594,9 +594,9 @@ "evidence": "the two build errors" }, { - "clause": "seed-not-clobber: the admin-edited row (customized:true) survives redeploys — the deactivation/receiver-URL edit is still there after reboot", + "clause": "the activation switch survives redeploys: the sys_metadata_activation row (metadata_type webhook) the door wrote is still in force after reboot — a declared webhook's on/off lives in the ledger, never in a row (ADR-0131 D6 regime C)", "oracle": "api", - "verify": "post-reboot sys_webhook row keeps the admin's values (bootstrap-declared-webhooks.ts)", + "verify": "post-reboot sys_metadata_activation row, and the declaration still reads switched on (declared-webhooks.ts readWebhookActivation)", "evidence": "before/after row reads" } ], @@ -615,8 +615,8 @@ }, "source": [ "packages/spec/src/automation/webhook.zod.ts#WebhookTriggerType (WebhookTriggerType enum + why undelete/api are absent; materialization contract; strict shape #4001)", - "packages/spec/liveness/webhook.json (all 11 props live via the #3489 bridge; per-prop line refs)", - "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts + auto-enqueuer.ts (remaps; trigger→event mapping incl. the #4639 bulk pair; #3196 unknown-trigger warn; seed-not-clobber)", + "packages/spec/liveness/webhook.json (9 props live on the registry read; label/description dead since the boot seeder retired)", + "packages/plugins/plugin-webhooks/src/declared-webhooks.ts + auto-enqueuer.ts (the registry read and its remaps; trigger→event mapping incl. the #4639 bulk pair; #3196 unknown-trigger warn; the activation-ledger switch; an 18.0 database's seeded rows read beside the declaration)", "packages/services/service-messaging/src/http-outbox.ts#HttpRedeliverError (delivery statuses, attempts; HttpRedeliverError + assertHttpRedeliverable/assertRedeliverAllowed — terminal-only, the attempts===0 parked-row refusal, the fail-closed guard) + packages/services/service-messaging/src/http-signature-at-rest.integration.test.ts (redelivery replays the stored signature byte-for-byte)", "packages/plugins/plugin-webhooks/src/webhook-outbox-plugin.ts#session (POST /api/v1/webhooks/redeliver — session auth with NO capability gate by design ('every authenticated user counts'), #10740 activeOrganizationId threading, the 401/400/404/409/500 arms) + (the #8069 redeliver-guard install; its ABSENCE is an error-level log, not a silent pass)", "packages/rest/src/rest-server.ts#deleteMany (#3897 — the batch routes parse against the spec contract and Zod STRIPS unknown keys, so options.multi/options.where cannot ride in; deleteMany deletes per id. This is why the bulk clause must be driven from a flow node, not REST)", @@ -665,6 +665,12 @@ "date": "2026-10-16", "change": "steps[2] names the create body the delivery leg needs (checklist-accuracy finding of QA run #23922, wave #23878): a showcase_task create over REST requires its `project` lookup and answers 400 VALIDATION_FAILED 'Project is required' without one, so the create variant never fires. The clauses are unchanged", "ref": "#24036" + }, + { + "revision": 8, + "date": "2026-10-19", + "change": "ADR-0131 D2/D6: the declared-webhook boot seeder retired, so a declared webhook is read from the registry and writes no sys_webhook row, and is switched through POST /api/v1/security/_activation/webhook/:name (regime C). steps[0], steps[1] and the reboot step, acceptance[0] and acceptance[6], fixtures.requires[0]/[1] and the source list now say so; the delivery, bulk, redeliver and retired-trigger clauses are unchanged", + "ref": "#24529" } ], "enumSource": { diff --git a/examples/app-showcase/src/automation/webhooks/index.ts b/examples/app-showcase/src/automation/webhooks/index.ts index 222686327a6..e04a91ea3c1 100644 --- a/examples/app-showcase/src/automation/webhooks/index.ts +++ b/examples/app-showcase/src/automation/webhooks/index.ts @@ -3,26 +3,28 @@ import { defineWebhook } from '@objectstack/spec/automation'; /** - * Outbound webhook — fans out task changes to an external endpoint. - * Authored via `defineStack({ webhooks })`; on boot the webhooks - * plugin materializes this into a `sys_webhook` row (#3461) that the - * auto-enqueuer dispatches off. + * Outbound webhook — fans out task lifecycle events (create/update/delete) + * to an external endpoint. No `label` / `description`: a declared webhook has + * no Setup row to show them on, and the liveness ledger marks both dead. + * Authored via `defineStack({ webhooks })`; the webhooks plugin's dispatcher + * reads this declaration from the registry (ADR-0131 D2) — no `sys_webhook` + * row is written for it. * * Shipped INACTIVE on purpose: `hooks.example` is a placeholder endpoint, so an * active subscription would emit a failed HTTP delivery on every task change. - * The demo flow is to open Setup → Integrations → Webhooks and flip this row to - * Active (pointing it at a real endpoint) — that admin edit is remembered - * (`customized: true`) and survives redeploys. + * A declared webhook is read-only (ADR-0131 D6, regime C). The demo flow is to + * author a copy under a new name in Setup → Integrations → Webhooks, pointed at + * a real endpoint; this declaration itself is switched on or off with + * `POST /api/v1/security/_activation/webhook/showcase_task_changed` and + * `{ "enabled": true }`, which survives redeploys. */ export const TaskChangedWebhook = defineWebhook({ name: 'showcase_task_changed', - label: 'Task Changed → External', object: 'showcase_task', triggers: ['create', 'update', 'delete'], url: 'https://hooks.example/showcase/task', method: 'POST', isActive: false, - description: 'Sends task lifecycle events to an external system. Activate in Setup and point at a real endpoint.', }); export const allWebhooks = [TaskChangedWebhook]; diff --git a/packages/lint/src/lint-liveness-properties.test.ts b/packages/lint/src/lint-liveness-properties.test.ts index 7b42f7e8594..1c22df12954 100644 --- a/packages/lint/src/lint-liveness-properties.test.ts +++ b/packages/lint/src/lint-liveness-properties.test.ts @@ -247,26 +247,51 @@ describe('lintLivenessProperties', () => { expect(findings).toEqual([]); }); - // ── webhook (#3461 bridge landed → #3490; #3494 prune) ──────────────────── - // Two things closed the old "entire surface is dead" state: #3494 PRUNED the - // aspirational dead props (body/payloadFields/includeSession/retryPolicy/tags/ - // authentication) from the schema, and the #3489 materializer bridge makes - // every REMAINING prop live (object/isActive/url/triggers/method/name/headers/ - // secret/timeoutMs/label/description). So a webhook has no misleading dead - // surface left — authoring one is silent. Runs against the REAL webhook.json. - - it('does not warn on an authored webhook — all remaining props are live (#3490)', () => { + // ── webhook (#3461 bridge landed → #3490; #3494 prune; ADR-0131 D2) ───── + // #3494 PRUNED the aspirational dead props from the schema and the #3489 + // materializer bridge made every remaining prop live. ADR-0131 D2 then + // retired that bridge: a declared webhook is read from the registry and has + // no Setup row, so the two display-only props, `label` and `description`, + // lost their only reader and the ledger marks them dead. Every delivery- + // bearing prop stays silent. Runs against the REAL webhook.json, so a ledger + // flip that is not carried to authors fails here. + + it('warns on a declared webhook\'s label and description only — every delivery-bearing prop is live', () => { const findings = lintLivenessProperties({ webhooks: [{ name: 'showcase_task_changed', + label: 'Task Changed', object: 'showcase_task', triggers: ['create', 'update', 'delete'], url: 'https://hooks.example/showcase/task', method: 'POST', + headers: { 'X-Team': 'crm' }, + secret: 'whsec_x', + timeoutMs: 5000, isActive: true, description: 'Sends task lifecycle events to an external system.', }], }); + expect(findings.map((f) => [f.rule, f.message.match(/`(\w+)`/)?.[1]])).toEqual([ + ['liveness-dead-property', 'label'], + ['liveness-dead-property', 'description'], + ]); + for (const f of findings) { + expect(f.where).toBe("webhook 'showcase_task_changed'"); + expect(f.hint).toContain('no Setup row'); + } + }); + + it('stays silent on a declared webhook that authors only what is delivered on', () => { + const findings = lintLivenessProperties({ + webhooks: [{ + name: 'showcase_task_changed', + object: 'showcase_task', + triggers: ['create'], + url: 'https://hooks.example/showcase/task', + isActive: false, + }], + }); expect(findings).toEqual([]); }); diff --git a/packages/plugins/plugin-webhooks/src/auto-enqueuer.ts b/packages/plugins/plugin-webhooks/src/auto-enqueuer.ts index a7afb3b09e3..bb4bbf1025d 100644 --- a/packages/plugins/plugin-webhooks/src/auto-enqueuer.ts +++ b/packages/plugins/plugin-webhooks/src/auto-enqueuer.ts @@ -3,7 +3,7 @@ import type { IDataEngine, IRealtimeService, RealtimeEventPayload } from '@objectstack/spec/contracts'; import type { WebhookTriggerType } from '@objectstack/spec/automation'; import type { EnqueueHttpInput } from '@objectstack/service-messaging'; -import { omitInternalFieldsFromWriteResponse } from '@objectstack/core'; +import { METADATA_ACTIVATION_TABLE, omitInternalFieldsFromWriteResponse } from '@objectstack/core'; import { WEBHOOK_SECRET_FIELD, WEBHOOK_SECRET_REFUSAL_CODE, @@ -11,16 +11,24 @@ import { onCryptoProviderChange, readLegacySecret, resolveWebhookSecret, + splitWebhookSecret, } from './webhook-secret.js'; import { WEBHOOK_HEADERS_FIELD, readLegacyHeaders, resolveWebhookHeaders, + splitWebhookHeaders, } from './webhook-headers.js'; import { WebhookLegacyCleartextError, type LegacyDefinitionCredentialKey, } from './webhook-legacy-cleartext.js'; +import { + classifyDeclaredNameHolder, + declaredWebhookRefId, + type DeclaredWebhook, + type DeclaredWebhookReader, +} from './declared-webhooks.js'; /** * [#21913] The execution context the subscription cache refresh @@ -146,6 +154,20 @@ interface CachedSubscription { * its own organization's. */ organizationId?: string; + /** + * [ADR-0131 D2/D9] Set on a DECLARED webhook when the deployment's posture + * builds no organization wall (`single`). A declared webhook is the + * application's, not one organization's, so it has no `organizationId` of + * its own — and under `single` every record event is stamped with the + * Default Organization (D9), so the org-less refusal below would make it + * receive nothing at all. Deployment-wide, it receives every organization's + * events and each delivery row is stamped with the EVENT's organization: + * exactly what the 18.0 seeded copy did, whose row the boot stamped with + * that same Default Organization. Never set under a wall, where the + * ruling's org-less refusal ({@link AutoEnqueuer.admitsOrganization}) + * stands. + */ + deploymentWide?: boolean; /** * [#8069] Set when a credential this subscription needs could not be * recovered. The subscription stays CACHED — that is the change — but every @@ -175,6 +197,23 @@ export interface AutoEnqueuerOptions { */ refreshIntervalMs?: number; + /** + * [ADR-0131 D2/D6] The declared webhooks (`defineStack({ webhooks })`), + * read from the registry with their activation-ledger switch resolved. + * Each one switched on is a subscription of its own, beside the + * `sys_webhook` rows — see `declared-webhooks.ts` for how a row holding a + * declared name is read. Absent: rows only. + */ + declaredWebhooks?: DeclaredWebhookReader; + + /** + * Whether the deployment's tenancy posture builds no organization wall, + * read on every refresh. True makes each declared webhook deployment-wide + * (see `CachedSubscription.deploymentWide`); absent or false keeps it + * org-less, refused for an organization-walled event. + */ + declaredWebhooksDeploymentWide?: () => boolean; + logger?: OptionalLogger; } @@ -277,6 +316,17 @@ export class AutoEnqueuer { * is deleted and re-created reports again. */ private readonly organizationRefusalReported = new Set(); + /** [ADR-0131 D2/D6] The declared-webhook reader, when the host binds one. */ + private readonly declared?: DeclaredWebhookReader; + private readonly declaredDeploymentWide?: () => boolean; + /** Self-heal subscription on the activation ledger — bound only with {@link declared}. */ + private subIdLedger: string | undefined; + /** + * Declared names already reported as withheld behind a `sys_webhook` row + * that holds the name (see {@link resolveDeclared}) — said once per name + * per process, the #8022 say-once rule. + */ + private readonly declarationWithheldReported = new Set(); constructor( private readonly engine: IDataEngine, @@ -287,6 +337,8 @@ export class AutoEnqueuer { this.subscriptionsObject = opts.subscriptionsObject ?? 'sys_webhook'; this.refreshIntervalMs = opts.refreshIntervalMs ?? 60_000; this.logger = opts.logger; + this.declared = opts.declaredWebhooks; + this.declaredDeploymentWide = opts.declaredWebhooksDeploymentWide; } /** @@ -322,6 +374,18 @@ export class AutoEnqueuer { { object: this.subscriptionsObject }, ); + // [ADR-0131 D6 regime C] A declared webhook is switched by an + // activation-ledger row, not by a `sys_webhook` write — so a switch + // refreshes the cache the way a row edit does, in the request that + // wrote it rather than at the next periodic refresh. + if (this.declared) { + this.subIdLedger = await this.realtime.subscribe( + 'webhook-auto-enqueuer-activation', + (event) => this.handleLedgerEvent(event), + { object: METADATA_ACTIVATION_TABLE }, + ); + } + if (this.refreshIntervalMs > 0) { this.refreshTimer = setInterval(() => { this.refresh().catch((err) => @@ -338,10 +402,12 @@ export class AutoEnqueuer { this.running = false; if (this.subId) await this.realtime.unsubscribe(this.subId); if (this.subIdSelfHeal) await this.realtime.unsubscribe(this.subIdSelfHeal); + if (this.subIdLedger) await this.realtime.unsubscribe(this.subIdLedger); if (this.refreshTimer) clearInterval(this.refreshTimer); this.unbindCryptoListener?.(); this.subId = undefined; this.subIdSelfHeal = undefined; + this.subIdLedger = undefined; this.refreshTimer = undefined; this.unbindCryptoListener = undefined; } @@ -401,8 +467,22 @@ export class AutoEnqueuer { return; } + // [ADR-0131 D2/D6] The declared webhooks, and the 18.0 mirror rows they + // replace. A failed read keeps the previous cache, exactly as a failed + // row read above does: neither a double fire nor a silent loss. + const declared = await this.resolveDeclared(); + if (!declared) return; + const next = new Map(); + const add = (sub: CachedSubscription) => { + // Empty objectName == "any object" → indexed under '*'. + const key = sub.objectName ?? '*'; + const arr = next.get(key) ?? []; + arr.push(sub); + next.set(key, arr); + }; for (const row of rows) { + if (declared.mirrorIds.has(String(row?.id))) continue; const sub = this.parseRow(row); if (!sub) continue; // [#7799, #7986] Neither credential is in the row we just read — @@ -416,12 +496,9 @@ export class AutoEnqueuer { // as undeliverable instead of silently discarded (#8069). See // `attachCredentials`. await this.attachCredentials(sub, row); - // Empty objectName == "any object" → indexed under '*'. - const key = sub.objectName ?? '*'; - const arr = next.get(key) ?? []; - arr.push(sub); - next.set(key, arr); + add(sub); } + for (const sub of declared.subscriptions) add(sub); this.subscriptions.clear(); for (const [k, v] of next) this.subscriptions.set(k, v); @@ -432,6 +509,7 @@ export class AutoEnqueuer { // broken would have its first report suppressed as a repeat. if (this.droppedForSecret.size > 0 || this.organizationRefusalReported.size > 0) { const live = new Set(rows.map((r) => String(r?.id))); + for (const sub of declared.subscriptions) live.add(sub.id); for (const id of this.droppedForSecret) { if (!live.has(id)) this.droppedForSecret.delete(id); } @@ -444,9 +522,133 @@ export class AutoEnqueuer { this.logger?.debug?.('[webhook-auto-enqueuer] cache refreshed', { objects: this.subscriptions.size, rows: rows.length, + declared: declared.subscriptions.length, }); } + /** + * [ADR-0131 D2/D6] The declared webhooks switched on, as subscriptions, + * and the ids of the 18.0 mirror rows they replace. `undefined` when a read + * failed, so the caller keeps the cache it has. + * + * A `sys_webhook` row holding a declared name — any row, active or not — is + * read the way the 18.0 seeder would have treated it on this boot + * (`classifyDeclaredNameHolder`): a pristine mirror is skipped and the + * declaration answers; a customized or administrator-authored row answers + * as itself and the declaration is withheld, said once per name. That is + * what keeps a declared webhook from firing twice on an upgraded database, + * and an administrator's `active: false` on a seeded row in force. + */ + private async resolveDeclared(): Promise< + { subscriptions: CachedSubscription[]; mirrorIds: Set } | undefined + > { + const empty = { subscriptions: [], mirrorIds: new Set() }; + if (!this.declared) return empty; + let declared: DeclaredWebhook[]; + try { + declared = await this.declared.list(); + } catch (err) { + this.logger?.warn?.('[webhook-auto-enqueuer] failed to read the declared webhooks', err); + return undefined; + } + if (declared.length === 0) return empty; + + let holders: any[]; + try { + holders = await this.engine.find( + this.subscriptionsObject, + { + where: { name: { $in: declared.map((d) => d.webhook.name) } }, + fields: ['id', 'name', 'managed_by', 'customized'], + }, + { context: SYSTEM_CTX }, + ); + } catch (err) { + this.logger?.warn?.( + `[webhook-auto-enqueuer] failed to read the ${this.subscriptionsObject} rows holding declared names`, + err, + ); + return undefined; + } + + const mirrorIds = new Set(); + const heldByRow = new Set(); + for (const row of Array.isArray(holders) ? holders : []) { + if (classifyDeclaredNameHolder(row) === 'mirror') mirrorIds.add(String(row.id)); + else heldByRow.add(String(row.name)); + } + + let deploymentWide = false; + try { + deploymentWide = this.declaredDeploymentWide?.() === true; + } catch { + /* an unreadable posture is not an open one: org-less, as under a wall */ + } + const subscriptions: CachedSubscription[] = []; + for (const d of declared) { + const name = d.webhook.name; + if (heldByRow.has(name)) { + this.reportDeclarationWithheld(name); + continue; + } + // `active` is the declared `isActive`, or the activation ledger's switch over it. + if (!d.active) continue; + const sub = this.projectDeclared(d); + if (!sub) continue; + if (deploymentWide) sub.deploymentWide = true; + subscriptions.push(sub); + } + return { subscriptions, mirrorIds }; + } + + /** + * One declared webhook as a cached subscription. The shape goes through + * {@link parseRow} — the same trigger drift-guard and the same + * `webhook/without-triggers` refusal as a row — and both credentials come + * from the declaration itself: they are code, held in memory like the + * plaintext a row's refresh decrypts, and never written anywhere. + */ + private projectDeclared(d: DeclaredWebhook): CachedSubscription | null { + const wh = d.webhook; + const sub = this.parseRow({ + id: declaredWebhookRefId(wh.name), + name: wh.name, + object_name: wh.object ?? null, + triggers: wh.triggers ?? [], + url: wh.url, + method: wh.method, + definition_json: JSON.stringify({ timeoutMs: wh.timeoutMs }), + }); + if (!sub) return null; + const { secret } = splitWebhookSecret(wh as Record); + const { headers } = splitWebhookHeaders(wh as Record); + if (secret) sub.secret = secret; + if (headers) sub.headers = headers; + return sub; + } + + /** + * A declared webhook withheld because a `sys_webhook` row holds its name: + * a seeded row an administrator edited before this release, or a row an + * administrator authored under the name. The row answers; the declaration + * does not also fire beside it. Said once per name per process at `warn` — + * the row is in force and dispatching (or switched off on purpose), so + * nothing claimed is being lost. + */ + private reportDeclarationWithheld(name: string): void { + if (this.declarationWithheldReported.has(name)) return; + this.declarationWithheldReported.add(name); + this.logger?.warn?.( + `[webhook-auto-enqueuer] declared webhook '${name}' is not dispatched: a ${this.subscriptionsObject} row ` + + 'of the same name answers for it (a seeded copy edited before declared webhooks were read from the ' + + 'registry, or a row authored under the name). The row stays in force as it is. To return to the ' + + 'declaration, delete the row; to change the declared webhook, author a copy under a new name and ' + + `switch the declared one off (POST /api/v1/security/_activation/webhook/${name} with ` + + '{ "enabled": false }).', + { webhook: name }, + ); + } + /** * [#7799, #7986] Resolve BOTH encrypted credentials for one cached * subscription. Returns `false` when the subscription must be dropped from @@ -997,8 +1199,9 @@ export class AutoEnqueuer { // organization — the one honest tenant in scope on this // fire-and-forget path (no request context exists here). // Absent for an org-less subscription; the row then lands - // NULL, the global-row shape. - organizationId: sub.organizationId, + // NULL, the global-row shape. A deployment-wide declared + // webhook takes the event's organization instead. + organizationId: sub.deploymentWide ? organization.organizationId : sub.organizationId, // [#3946] Envelope keys are written LAST so the event payload // cannot rewrite them. Behaviour-neutral for the engine's own // publishers — since #4626 a `data.record.*` payload is a @@ -1145,7 +1348,7 @@ export class AutoEnqueuer { timeoutMs: sub.timeoutMs, // [#13546] See the per-record path — the subscription's own // organization, absent for an org-less subscription. - organizationId: sub.organizationId, + organizationId: sub.deploymentWide ? organization.organizationId : sub.organizationId, // [#3946] Envelope keys last so the payload cannot rewrite them. payload: { ...payload, @@ -1206,11 +1409,17 @@ export class AutoEnqueuer { * webhook — accepted, and said once so the subscription is not dead while * looking armed. * - * **`none` × `absent` — deliver.** A `single`-posture deployment stamps - * nothing on either side (`postureStampsOrganization` is false there), so - * this cell is every event on every non-walled install; on a walled one - * it is an environment-wide subscription taking an environment-wide - * event. No organization is named anywhere, so there is no wall to cross. + * **`none` × `absent` — deliver.** An environment-wide subscription taking + * an environment-wide event: no organization is named anywhere, so there is + * no wall to cross. ⚠ It is NOT every event on a `single` install: since + * ADR-0131 D9 every record there is stamped with the Default Organization, + * so a `single` event names that organization like a walled one does. + * + * **A deployment-wide declared webhook — deliver, whatever the event names.** + * Checked first ({@link CachedSubscription.deploymentWide}): a declared + * webhook on a posture with no wall belongs to the deployment, so every + * organization's event is its own. Without it the `none` × `org A` row + * above would refuse every event on a `single` install. * * Both refusals are said ONCE per subscription (the #8022 say-once rule, * ledger {@link organizationRefusalReported}): the first refused event is @@ -1224,6 +1433,9 @@ export class AutoEnqueuer { eventOrganizationId: string | undefined, event: RealtimeEventPayload, ): boolean { + // [ADR-0131 D2/D9] A declared webhook on a deployment with no wall + // belongs to the deployment, so every organization's event is its own. + if (sub.deploymentWide) return true; // org A × org A, and none × absent. if (sub.organizationId === eventOrganizationId) return true; // org A × org B — another organization's subscription is simply not a @@ -1282,6 +1494,14 @@ export class AutoEnqueuer { ); } + private handleLedgerEvent(event: RealtimeEventPayload): void { + if (event.object !== METADATA_ACTIVATION_TABLE) return; + if (!event.type?.startsWith('data.record.') && !event.type?.startsWith('data.records.')) return; + this.refresh().catch((err) => + this.logger?.warn?.('[webhook-auto-enqueuer] activation refresh failed', err), + ); + } + /** Test / admin accessor. */ snapshot(): ReadonlyMap> { return this.subscriptions; diff --git a/packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.test.ts b/packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.test.ts deleted file mode 100644 index 3061a59a917..00000000000 --- a/packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.test.ts +++ /dev/null @@ -1,310 +0,0 @@ -// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. - -/** - * bootstrapDeclaredWebhooks — the ingestion bridge that closes #3461. - * - * Verifies that STACK-declared `webhook` metadata (spec shape: `object` / - * `isActive`) is materialized into `sys_webhook` data rows (`object_name` / - * `active` / `definition_json`), idempotently and without clobbering admin - * edits — and that the dispatcher then sees those rows. - * - * This header used to say "stack/connector-declared", mirroring the same false - * clause the implementation's docblock carried: a connector's nested - * `webhooks[]` never becomes a `webhook` metadata item and so never reaches - * this bridge at all. Nothing in THIS file ever exercised a connector — every - * fixture below seeds `declared.webhook` directly — so the word was a claim the - * suite did not make. The claim it does not make is measured against a real - * engine next door, in `bootstrap-declared-webhooks.connector-nested.test.ts`. - */ - -import { describe, expect, it, vi } from 'vitest'; -import { AutoEnqueuer, type HttpEnqueueFn } from './auto-enqueuer.js'; -import { bootstrapDeclaredWebhooks } from './bootstrap-declared-webhooks.js'; -import { bindWebhookProvenanceStamp } from './webhook-provenance.js'; -import { assertEngineFindOnePredicate } from '@objectstack/metadata-core'; - -// --------------------------------------------------------------------------- -// Fakes -// --------------------------------------------------------------------------- - -interface HookEntry { - event: string; - handler: (ctx: any) => any; - object?: string; - packageId?: string; -} - -/** - * A small engine fake that mirrors the real ObjectQL surface the bridge and - * provenance hook touch: `find({ filter | where })`, `insert`, update-by-id (the - * patch carries `id`, no `where`), a `_registry.listItems(type)` for declared - * metadata, and `beforeUpdate` hooks that run inside `update()`. - */ -class FakeEngine { - rows: Record = {}; - private hooks: HookEntry[] = []; - private declared: Record = {}; - - constructor(seed?: { rows?: Record; declared?: Record }) { - if (seed?.rows) this.rows = JSON.parse(JSON.stringify(seed.rows)); - if (seed?.declared) this.declared = JSON.parse(JSON.stringify(seed.declared)); - } - - // Declared-metadata registry (where manifest decomposition parks stack.webhooks). - // - // [#8378] Items are registered EXACTLY as the real engine registers them — - // the document itself. Boxing each as `{ content: }` made this fake - // the only producer of that envelope in the tree, which is what kept the - // production `i?.content ?? i` looking load-bearing. - get _registry() { - return { - listItems: (type: string) => [...(this.declared[type] ?? [])], - }; - } - - private matches(row: any, cond?: Record): boolean { - if (!cond) return true; - return Object.entries(cond).every(([k, v]) => row[k] === v); - } - - async find(name: string, q?: any): Promise { - const all = this.rows[name] ?? []; - const cond = q?.filter ?? q?.where; - const out = all.filter((r) => this.matches(r, cond)); - return typeof q?.limit === 'number' ? out.slice(0, q.limit) : out; - } - async findOne(name: string, q?: any): Promise { - assertEngineFindOnePredicate(name, q); - return (await this.find(name, q))[0] ?? null; - } - async insert(name: string, data: any): Promise { - const arr = (this.rows[name] = this.rows[name] ?? []); - arr.push({ ...data }); - return data; - } - async update(name: string, data: any, opts?: any): Promise { - // Run beforeUpdate hooks (the provenance stamp lives here). - const id = data?.id ?? opts?.where?.id; - // [#15302] `previous` is bound BEFORE the beforeUpdate dispatch, as the - // real engine binds it (#5574 / #5846 - by-id reads the prior row ahead of - // the dispatch; each per-row context of a predicate write carries its own). - // Withholding it here is what let this fake model a pre-#5574 engine, and - // a hook reading `ctx.previous` would have gone silently unstamped against - // a fake no production caller resembles. - const cond0 = opts?.where ?? (id ? { id } : undefined); - const previous = (this.rows[name] ?? []).filter((r) => this.matches(r, cond0)).map((r) => ({ ...r }))[0]; - const ctx = { input: { id, data }, previous, session: opts?.context }; - for (const h of this.hooks) { - if (h.event === 'beforeUpdate' && (!h.object || h.object === name)) { - await h.handler(ctx); - } - } - const arr = this.rows[name] ?? []; - const cond = opts?.where ?? (id ? { id } : undefined); - for (const r of arr) { - if (this.matches(r, cond)) Object.assign(r, data); - } - return { affected: 0 }; - } - async delete(): Promise { - return { affected: 0 }; - } - async count(name: string): Promise { - return (this.rows[name] ?? []).length; - } - async aggregate(): Promise { - return []; - } - - registerHook(event: string, handler: (ctx: any) => any, options?: Record): void { - this.hooks.push({ event, handler, object: options?.object, packageId: options?.packageId }); - } - unregisterHooksByPackage(packageId: string): number { - const before = this.hooks.length; - this.hooks = this.hooks.filter((h) => h.packageId !== packageId); - return before - this.hooks.length; - } -} - -class FakeRealtime { - private subs = new Map(); - private n = 0; - async publish(event: any): Promise { - for (const sub of this.subs.values()) { - const o = sub.opts ?? {}; - if (o.object && event.object !== o.object) continue; - await sub.handler(event); - } - } - async subscribe(_channel: string, handler: any, opts?: any): Promise { - const id = `s-${++this.n}`; - this.subs.set(id, { handler, opts }); - return id; - } - async unsubscribe(id: string): Promise { - this.subs.delete(id); - } -} - -const ADMIN_CTX = { isSystem: false, positions: [], permissions: [] }; - -function declaredWebhook(over: Record = {}): any { - return { - name: 'task_changed', - label: 'Task Changed', - object: 'showcase_task', - triggers: ['create', 'update'], - url: 'https://hooks.example/task', - method: 'POST', - isActive: true, - ...over, - }; -} - -async function flush() { - await new Promise((r) => setTimeout(r, 0)); -} - -// --------------------------------------------------------------------------- -// Tests -// --------------------------------------------------------------------------- - -describe('bootstrapDeclaredWebhooks', () => { - it('materializes a declared webhook into a sys_webhook row (object→object_name, isActive→active)', async () => { - const engine = new FakeEngine({ declared: { webhook: [declaredWebhook()] } }); - const res = await bootstrapDeclaredWebhooks(engine as any, null); - - expect(res).toEqual({ seeded: 1, skipped: 0 }); - const rows = engine.rows['sys_webhook']; - expect(rows).toHaveLength(1); - const row = rows[0]; - expect(row.name).toBe('task_changed'); - expect(row.object_name).toBe('showcase_task'); // object → object_name - expect(row.active).toBe(true); // isActive → active - expect(row.method).toBe('post'); // lowercased to match the select options - expect(row.managed_by).toBe('package'); - expect(row.customized).toBe(false); - // Full validated envelope stashed for the enqueuer's advanced-config read. - const defn = JSON.parse(row.definition_json); - expect(defn.object).toBe('showcase_task'); - expect(defn.timeoutMs).toBe(30000); // default filled by WebhookSchema.parse - }); - - it('maps isActive:false → active:false so a placeholder webhook ships inactive', async () => { - const engine = new FakeEngine({ declared: { webhook: [declaredWebhook({ isActive: false })] } }); - await bootstrapDeclaredWebhooks(engine as any, null); - expect(engine.rows['sys_webhook'][0].active).toBe(false); - }); - - it('is idempotent — a second boot updates in place, never duplicates', async () => { - const engine = new FakeEngine({ declared: { webhook: [declaredWebhook()] } }); - await bootstrapDeclaredWebhooks(engine as any, null); - await bootstrapDeclaredWebhooks(engine as any, null); - expect(engine.rows['sys_webhook']).toHaveLength(1); - }); - - it('propagates a declared change to a pristine (non-customized) row', async () => { - const engine = new FakeEngine({ declared: { webhook: [declaredWebhook()] } }); - await bootstrapDeclaredWebhooks(engine as any, null); - - engine['declared'].webhook = [declaredWebhook({ url: 'https://hooks.example/task-v2' })]; - await bootstrapDeclaredWebhooks(engine as any, null); - - expect(engine.rows['sys_webhook']).toHaveLength(1); - expect(engine.rows['sys_webhook'][0].url).toBe('https://hooks.example/task-v2'); - }); - - it('seed-not-clobber: an admin edit (customized) survives the next boot', async () => { - const engine = new FakeEngine({ declared: { webhook: [declaredWebhook()] } }); - bindWebhookProvenanceStamp(engine as any); - await bootstrapDeclaredWebhooks(engine as any, null); - - // Admin deactivates the noisy webhook through the CRUD door (non-system). - const id = engine.rows['sys_webhook'][0].id; - await engine.update('sys_webhook', { id, active: false }, { context: ADMIN_CTX }); - expect(engine.rows['sys_webhook'][0].customized).toBe(true); // hook stamped it - - // Redeploy re-runs the seeder — the declared row is still active:true, but - // the admin's active:false must win. - await bootstrapDeclaredWebhooks(engine as any, null); - expect(engine.rows['sys_webhook'][0].active).toBe(false); - }); - - it('never overwrites an admin-authored row that collides by name', async () => { - const engine = new FakeEngine({ - rows: { - sys_webhook: [ - { id: 'admin-1', name: 'task_changed', url: 'https://admin.example', active: true, managed_by: 'admin', customized: false }, - ], - }, - declared: { webhook: [declaredWebhook()] }, - }); - const res = await bootstrapDeclaredWebhooks(engine as any, null); - expect(res).toEqual({ seeded: 0, skipped: 1 }); - expect(engine.rows['sys_webhook']).toHaveLength(1); - expect(engine.rows['sys_webhook'][0].url).toBe('https://admin.example'); // untouched - }); - - it('skips an invalid declared webhook (bad URL) with a warning, without crashing boot', async () => { - const warn = vi.fn(); - const engine = new FakeEngine({ - declared: { webhook: [declaredWebhook({ name: 'good' }), declaredWebhook({ name: 'bad', url: 'not-a-url' })] }, - }); - const res = await bootstrapDeclaredWebhooks(engine as any, null, { warn }); - - expect(res.seeded).toBe(1); // the good one still lands - expect(res.skipped).toBe(1); - expect(engine.rows['sys_webhook'].map((r) => r.name)).toEqual(['good']); - expect(warn).toHaveBeenCalledWith(expect.stringContaining('failed validation'), expect.objectContaining({ name: 'bad' })); - }); - - it('is a no-op when nothing is declared', async () => { - const engine = new FakeEngine(); - const res = await bootstrapDeclaredWebhooks(engine as any, null); - expect(res).toEqual({ seeded: 0, skipped: 0 }); - expect(engine.rows['sys_webhook']).toBeUndefined(); - }); - - it('end-to-end: a declared webhook, once materialized, dispatches on a matching data event', async () => { - const engine = new FakeEngine({ - declared: { - webhook: [ - declaredWebhook({ - triggers: ['create'], - secret: 'shh', - headers: { 'X-Env': 'prod' }, - }), - ], - }, - }); - await bootstrapDeclaredWebhooks(engine as any, null); - - const realtime = new FakeRealtime(); - const calls: any[] = []; - const enqueue: HttpEnqueueFn = async (input) => { - calls.push(input); - return 'id'; - }; - const ae = new AutoEnqueuer(engine as any, realtime as any, enqueue, { refreshIntervalMs: 0 }); - await ae.start(); - - await realtime.publish({ - type: 'data.record.created', - object: 'showcase_task', - payload: { recordId: 't-1' }, - timestamp: '2026-05-24T00:00:00.000Z', - }); - await flush(); - - expect(calls).toHaveLength(1); - expect(calls[0].url).toBe('https://hooks.example/task'); - // Headers come from the definition_json envelope the bridge wrote. The - // signing secret does NOT (#7799) — it goes to the `signing_secret` column, - // which this fake engine (no encrypted-field channel) stores verbatim, so - // the enqueuer reads it back from there. - expect(calls[0].signingSecret).toBe('shh'); - expect(engine.rows['sys_webhook'][0].definition_json).not.toContain('shh'); - expect(calls[0].headers).toEqual({ 'X-Env': 'prod' }); - await ae.stop(); - }); -}); diff --git a/packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts b/packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts deleted file mode 100644 index 2f7f2b817af..00000000000 --- a/packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts +++ /dev/null @@ -1,352 +0,0 @@ -// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. - -/** - * bootstrapDeclaredWebhooks — materialize STACK-declared `webhooks` - * (`defineStack({ webhooks })`) into `sys_webhook` rows so the dispatcher can - * actually see them (closes #3461 for that half — see the next section for the - * half it does not reach). - * - * ## What this path does NOT reach: a connector's nested `webhooks[]` - * This opening sentence used to say "stack/connector-declared", and the - * connector half of it was false. The function's ONE source is - * `readDeclared(engine, metadataService, 'webhook')` — `webhook` METADATA - * ITEMS — and every registrar of that type is keyed on the TOP-LEVEL - * `webhooks:` collection of a manifest / nested-plugin document: - * `registerMetadataCollections` enumerates `METADATA_ARRAY_KEYS` over that - * document (objectql `engine.ts`), and the artifact loader's collection map - * (`packages/metadata/src/plugin.ts`) reads the same spelling table - * (`PLURAL_TO_SINGULAR`, `@objectstack/spec/meta-spelling`). None of them walks - * INTO a `connectors:` entry. A connector's nested `webhooks[]` therefore stays - * inside the connector document, never becomes a `webhook` metadata item, is - * never in `declared`, and is never materialized here — no row, no warning, no - * skip count. The shapes were not even the same: a connector's entries were - * `WebhookConfigSchema` (`WebhookSchema` plus `events` / `signatureAlgorithm`), - * not the `WebhookSchema` this seeder parses. - * - * That gap is now CLOSED BY RETIREMENT, not an oversight to route around: the - * spec retired `connector.webhooks` and the `WebhookConfig` shape under ADR-0049 - * enforce-or-remove (`integration/connector.zod.ts`, "REMOVED: `health`, - * `status` and the nested `webhooks`"), so the authoring doors refuse a nested - * array with a prescription pointing at the top-level collection, and the D2 - * conversion `connector-resilience-keys-removed` strips it from stored rows. - * Widening this seeder to read connectors would resurrect a retired surface. - * The absence is still pinned as behaviour by - * `bootstrap-declared-webhooks.connector-nested.test.ts`, which drives the - * registry below the authoring parse, so this paragraph cannot quietly go - * stale the way the sentence above it did. - * - * ## The disconnect this closes - * The spec authoring surface (`WebhookSchema` — `defineStack({ webhooks })`, - * `@objectstack/spec/automation/webhook`) declares `object` / `isActive`, and - * is generically decomposed into the ObjectQL registry at boot as metadata - * type `webhook`. But the runtime dispatcher ({@link AutoEnqueuer}) reads - * `sys_webhook` DATA rows (`object_name` / `active`), which until now were only - * ever written by hand through the object's CRUD UI. Nothing bridged the two — - * so authoring `webhooks:` on a stack produced metadata artifacts that never - * became dispatchable rows (a silent no-op; ADR-0078). This seeder is that - * missing ingestion path. - * - * ## Shape translation (authoring → runtime row) - * The spec shape diverges from the runtime column names; we map only at this - * boundary and stash the validated envelope in `definition_json` (whence the - * enqueuer reads headers / timeout): - * - `object` → `object_name` - * - `isActive` → `active` - * - `triggers` / `url` / `method` / `label` / `description` → same-named columns - * - `secret` → `signing_secret` (ENCRYPTED — see below) - * - `headers` → `headers_secret` (ENCRYPTED — #7986, same channel) - * - the rest of the parsed {@link Webhook} → `definition_json` (JSON string) - * - * ## Neither credential goes in `definition_json` (#7799, #7986) - * It used to: `definition_json: JSON.stringify(wh)` serialized the whole - * envelope, key included, into an ordinary textarea on an admin-authorable - * object — so `GET /api/v1/data/sys_webhook` returned the receiver's only proof - * of origin to anyone who could read the object. The authored key now goes to - * `sys_webhook.signing_secret`, a `type: 'secret'` column the engine encrypts - * into `sys_secret` and masks on read; `definition_json` carries the same - * envelope MINUS `secret`. Nothing about the authoring surface changes — - * `webhook.zod.ts` still declares `secret` and authors still write it. - * - * Fail-closed: with no CryptoProvider registered the engine REFUSES the write - * (it will not store cleartext), so a secret-bearing webhook is skipped with an - * actionable log line instead of being seeded with an exposed key. - * - * Each item is validated through `WebhookSchema.parse()` first — this gives the - * spec schema a real consumer (defaults for `method`/`isActive`/`timeoutMs` get - * applied) and rejects malformed authoring with a warning instead of crashing - * boot. - * - * ## Seed-not-clobber (mirrors sys_sharing_rule, #2909) - * `sys_webhook` is admin-editable (`managedBy: 'config'`). Declared webhooks - * ship with the app/package, so they seed with `managed_by: 'package'` - * provenance and re-seed on every boot — but a row an admin has created - * (`managed_by: 'admin'`) or edited (`customized: true`, stamped by - * {@link bindWebhookProvenanceStamp}) is never overwritten. Most importantly, - * an admin's `active: false` on a noisy webhook survives redeploys. - * - * MUST run before {@link AutoEnqueuer.start} so the enqueuer's first cache - * refresh already sees the declared rows. - */ - -import type { IDataEngine } from '@objectstack/spec/contracts'; -import { WebhookSchema, type Webhook } from '@objectstack/spec/automation'; -import { - WEBHOOK_SECRET_FIELD, - WEBHOOK_SECRET_REFUSAL_CODE, - WEBHOOK_SECRET_REFUSAL_STATUS, - canResolveSecrets, - isSecretProtectionFailure, - splitWebhookSecret, -} from './webhook-secret.js'; -import { - WEBHOOK_HEADERS_FIELD, - headersPatch, - serializeHeaders, - splitWebhookHeaders, -} from './webhook-headers.js'; - -/** System write context — the boot seeder is not an admin authoring action. */ -const SYSTEM_CTX = { isSystem: true, positions: [], permissions: [] } as const; - -interface Logger { - info?: (msg: string, meta?: unknown) => void; - warn?: (msg: string, meta?: unknown) => void; -} - -/** Random id with a stable prefix — mirrors the sharing-rule seeder. */ -function uid(prefix: string): string { - const g: any = globalThis as any; - if (g.crypto?.randomUUID) return `${prefix}_${g.crypto.randomUUID()}`; - return `${prefix}_${Math.random().toString(36).slice(2, 10)}`; -} - -/** - * Read declared `webhook` items from the ObjectQL registry (where the manifest - * decomposition parks `stack.webhooks`), falling back to the metadata service. - * - * [#8378] Both reads hand back the authoring document itself. The sentence that - * used to stand here — "Items may be wrapped as `{ content }` — unwrap to the - * raw authoring object" — described an envelope with **no producer**: - * `registerMetadataCollections` (objectql `engine.ts`) registers each - * `stack.webhooks` element as-is, `loadMetaFromDb` registers the parsed body - * rather than the `sys_metadata` row, and `MetadataFacade` shed its own copy of - * this unwrap in #7519. `WebhookSchema` declares no `content` key and rejects - * one as unrecognized, so wherever the key did appear the unwrap replaced the - * whole webhook with one of its values — and `''` (falsy, non-nullish) passed - * `??` and then died at `filter(Boolean)`, dropping the webhook silently. - */ -function readDeclared(engine: any, metadataService: any, type: string): any[] { - try { - const reg = engine?._registry; - if (reg?.listItems) { - const items = (reg.listItems(type) ?? []).filter(Boolean); - if (items.length > 0) return items; - } - } catch { - /* fall through to metadata service */ - } - try { - const listed = metadataService?.list?.(type); - const arr = typeof (listed as any)?.then === 'function' ? [] : (listed ?? []); - return Array.isArray(arr) ? arr.filter(Boolean) : []; - } catch { - return []; - } -} - -export interface BootstrapDeclaredWebhooksResult { - seeded: number; - skipped: number; -} - -/** - * Materialize declared webhooks into `sys_webhook`. Idempotent and safe to run - * on every boot. - */ -export async function bootstrapDeclaredWebhooks( - engine: IDataEngine, - metadataService: any, - logger?: Logger, - subscriptionsObject = 'sys_webhook', -): Promise { - const declared = readDeclared(engine, metadataService, 'webhook'); - if (declared.length === 0) return { seeded: 0, skipped: 0 }; - - const now = new Date().toISOString(); - let seeded = 0; - let skipped = 0; - - for (const raw of declared) { - // Validate + fill defaults through the canonical spec schema. A real - // consumer at last — a malformed webhook warns and is skipped, never - // crashing boot. - let wh: Webhook; - try { - wh = WebhookSchema.parse(raw); - } catch (err: any) { - logger?.warn?.('[webhook] declared webhook failed validation — skipped', { - name: (raw as any)?.name, - error: err?.message ?? String(err), - }); - skipped += 1; - continue; - } - - try { - const existing = await engine.find(subscriptionsObject, { - where: { name: wh.name }, - limit: 1, - context: SYSTEM_CTX, - } as any); - const row: any = Array.isArray(existing) ? existing[0] : undefined; - - if (row) { - // Admin owns a same-named row, or has edited this seeded one — never - // clobber. `active: false` on a noisy webhook must survive redeploys. - if (row.managed_by === 'admin') { - logger?.warn?.('[webhook] declared name collides with an admin-authored row — seed skipped', { - name: wh.name, - }); - skipped += 1; - continue; - } - if (row.customized === true) { - skipped += 1; - continue; - } - const patch = { - id: row.id, - ...mapWebhookToRow(wh), - ...(await secretPatch(engine, wh, row, subscriptionsObject)), - ...(await headersPatch( - engine, - splitWebhookHeaders(wh as Record).headers, - row, - subscriptionsObject, - )), - // Adopt pristine/legacy (pre-provenance) rows so future boots - // recognize them as package-managed. - managed_by: 'package', - updated_at: now, - }; - await engine.update(subscriptionsObject, patch, { context: SYSTEM_CTX } as any); - seeded += 1; - continue; - } - - const { secret } = splitWebhookSecret(wh as Record); - const { headers } = splitWebhookHeaders(wh as Record); - const newRow = { - id: uid('whk'), - ...mapWebhookToRow(wh), - // Cleartext goes in exactly once, into the `secret`-typed column; the - // engine's write path wraps it into `sys_secret` and leaves an opaque - // ref behind. Omitted entirely when unauthored, so a webhook with no - // secret costs no crypto and needs no CryptoProvider. - ...(secret ? { [WEBHOOK_SECRET_FIELD]: secret } : {}), - // [#7986] Same channel, same rule, for the header map — omitted when - // unauthored so a header-less webhook still seeds on a host with no - // CryptoProvider wired. - ...(headers ? { [WEBHOOK_HEADERS_FIELD]: serializeHeaders(headers) } : {}), - managed_by: 'package', - customized: false, - created_at: now, - updated_at: now, - }; - await engine.insert(subscriptionsObject, newRow, { context: SYSTEM_CTX } as any); - seeded += 1; - } catch (err: any) { - // [#7799] The engine refuses to persist a `secret` field with no - // CryptoProvider wired, rather than falling back to cleartext. Say so in - // those words — "seed failed" would read as a transient glitch, when what - // actually happened is that the deployment has nowhere safe to put a key. - const protection = isSecretProtectionFailure(err); - logger?.warn?.( - protection - ? '[webhook] declared webhook NOT seeded — its signing secret cannot be stored encrypted, and it is never stored in cleartext instead' - : '[webhook] declared webhook seed failed', - { - name: wh.name, - ...(protection - ? { code: WEBHOOK_SECRET_REFUSAL_CODE, status: WEBHOOK_SECRET_REFUSAL_STATUS } - : {}), - error: err?.message ?? String(err), - }, - ); - skipped += 1; - } - } - - logger?.info?.('[webhook] declared webhooks materialized into sys_webhook', { - seeded, - skipped, - total: declared.length, - }); - return { seeded, skipped }; -} - -/** - * Decide what a RE-SEED should do with an existing row's `signing_secret`. - * - * Re-seeding runs on every boot, and a `secret`-typed write always mints a - * fresh `sys_secret` ciphertext row — so blindly restating the declared key - * would leak one orphan cipher row per webhook per restart. Compare against the - * stored plaintext first (via the engine's privileged dereference) and write - * only on an actual change: - * - * - declared key differs from stored ⇒ write it (rotation in code propagates, - * exactly as it did when the whole envelope was rewritten every boot); - * - identical ⇒ omit the key entirely, leaving the existing ref untouched; - * - declared key removed, row still holds one ⇒ write `null` to CLEAR it - * (code remains the authority for package rows); - * - engine cannot dereference (older engine, or the compare threw) ⇒ fall back - * to writing the declared value. Correct signatures beat tidy storage. - */ -async function secretPatch( - engine: IDataEngine, - wh: Webhook, - row: any, - subscriptionsObject: string, -): Promise> { - const { secret } = splitWebhookSecret(wh as Record); - const hasStored = row?.[WEBHOOK_SECRET_FIELD] != null && row[WEBHOOK_SECRET_FIELD] !== ''; - - if (!secret) return hasStored ? { [WEBHOOK_SECRET_FIELD]: null } : {}; - if (!hasStored || !canResolveSecrets(engine)) return { [WEBHOOK_SECRET_FIELD]: secret }; - - try { - const current = await (engine as any).resolveSecretField( - subscriptionsObject, - String(row.id), - WEBHOOK_SECRET_FIELD, - ); - return current === secret ? {} : { [WEBHOOK_SECRET_FIELD]: secret }; - } catch { - return { [WEBHOOK_SECRET_FIELD]: secret }; - } -} - -/** - * Translate a validated {@link Webhook} into `sys_webhook` column values. - * `object → object_name`, `isActive → active`; the envelope MINUS its two - * credential passengers — `secret` (#7799) and `headers` (#7986) — is stashed - * in `definition_json` for the enqueuer's advanced-config read (`timeoutMs`). - * Both credentials are written separately, into their own encrypted columns. - */ -function mapWebhookToRow(wh: Webhook): Record { - const { envelope: withoutSecret } = splitWebhookSecret(wh as Record); - const { envelope } = splitWebhookHeaders(withoutSecret as Record); - return { - name: wh.name, - label: wh.label ?? wh.name, - object_name: wh.object ?? null, - triggers: wh.triggers ?? [], - url: wh.url, - // Store lowercase to match the object's Field.select option values - // (get/post/…); the enqueuer upper-cases before delivery either way. - method: String(wh.method ?? 'POST').toLowerCase(), - description: wh.description ?? null, - active: wh.isActive !== false, - definition_json: JSON.stringify(envelope), - }; -} diff --git a/packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.connector-nested.test.ts b/packages/plugins/plugin-webhooks/src/declared-webhooks.connector-nested.test.ts similarity index 71% rename from packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.connector-nested.test.ts rename to packages/plugins/plugin-webhooks/src/declared-webhooks.connector-nested.test.ts index 7696c1b82e5..250a07a70db 100644 --- a/packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.connector-nested.test.ts +++ b/packages/plugins/plugin-webhooks/src/declared-webhooks.connector-nested.test.ts @@ -1,12 +1,13 @@ // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. /** - * A connector's nested `webhooks[]` does NOT reach `sys_webhook` — pinned as + * A connector's nested `webhooks[]` does NOT reach the dispatcher — pinned as * behaviour, against a REAL registration path. * * ## Why this file exists * - * `bootstrap-declared-webhooks.ts` used to open with "materialize + * The retired declared-webhook seeder (`bootstrap-declared-webhooks.ts`, which + * `declared-webhooks.ts` replaced: ADR-0131 D2) used to open with "materialize * stack/**connector**-declared `webhooks` into `sys_webhook` rows so the * dispatcher can actually see them". The connector half was false, and it was * false in the one place a reader looks when deciding whether a connector's @@ -26,13 +27,13 @@ * * ⚠️ ANTI-VACUITY CONTROL, on the same subject and in the same corpus: the same * manifest also declares a TOP-LEVEL `webhooks:` entry, and the same assertions - * demand that one IS registered and IS seeded. Without it, every expectation + * demand that one IS registered and IS read as a declared webhook. Without it, every expectation * below would also pass on an engine that registered nothing at all, on a * misspelled metadata type, or on a manifest this engine rejected outright. * * ## The direction this pins is the DECLARED one * - * ⛔ A red here is not a licence to widen the seeder. `packages/spec` RETIRED + * ⛔ A red here is not a licence to widen the declared-webhook reader. `packages/spec` RETIRED * the connector surface under ADR-0049 enforce-or-remove: `connector.webhooks` * is a `retiredKey()` tombstone and the `WebhookConfig` shape left whole * (`integration/connector.zod.ts`, "REMOVED: `health`, `status` and the nested @@ -53,7 +54,7 @@ import { describe, expect, it } from 'vitest'; import { ObjectQL } from '@objectstack/objectql'; -import { bootstrapDeclaredWebhooks } from './bootstrap-declared-webhooks.js'; +import { DeclaredWebhookSource } from './declared-webhooks.js'; const PKG = 'com.acme.billing'; @@ -102,30 +103,16 @@ const namesOf = (registry: any, type: string): string[] => (registry.listItems(type) ?? []).filter(Boolean).map((i: any) => i?.name); /** - * The minimum `IDataEngine` surface the seeder's INSERT path touches, wired to - * a real registry so `readDeclared` sees exactly what boot registered. No - * secret is authored, so no CryptoProvider is needed. - * - * `update` is deliberately ABSENT rather than stubbed. `find` always answers - * empty here, so the seeder always takes the INSERT branch and never dispatches - * an update; a stub for it would be an engine double declaring a verb this file - * does not exercise, which `check:engine-double-contract` correctly treats as - * new surface to pin. A fake that stops where the path stops has nothing to be - * looser than, and a future change that DID reach the update branch would fail - * loudly here instead of passing against a permissive stub. + * The surface the declared-webhook reader touches, wired to a real registry so + * `readDeclaredWebhookItems` sees exactly what boot registered: the registry, + * and a `find` for the activation ledger that holds no row. */ -function seedingEngine(registry: any) { - const rows: any[] = []; +function readingEngine(registry: any) { return { - rows, _registry: registry, async find() { return []; }, - async insert(_object: string, row: any) { - rows.push(row); - return row; - }, }; } @@ -148,19 +135,23 @@ describe("a connector's nested `webhooks[]` never becomes a `webhook` metadata i expect(connector.webhooks?.map((w: any) => w?.name)).toEqual([CONNECTOR_NESTED.name]); }); - it('seeds `sys_webhook` from the top-level collection only — the connector half is not even skipped', async () => { - const engine = seedingEngine(bootedRegistry()); + it('reads the top-level collection only as declared webhooks — the connector half is not even skipped', async () => { + const warn: unknown[] = []; + const reader = new DeclaredWebhookSource(readingEngine(bootedRegistry()), undefined, { + warn: (msg: string) => { warn.push(msg); }, + }); - const result = await bootstrapDeclaredWebhooks(engine as any, undefined); + const declared = await reader.list(); - // CONTROL: the bridge really did run and really did materialize. - expect(result.seeded).toBe(1); - expect(engine.rows.map((r) => r.name)).toEqual([STACK_DECLARED.name]); + // CONTROL: the reader really did run and really did read the declaration. + expect(declared.map((d) => d.webhook.name)).toEqual([STACK_DECLARED.name]); - // SUBJECT: the connector-nested webhook produces NO row — and no `skipped` - // either, because the seeder never sees it at all. A future bridge that - // merely warned about it would move this number, not just the row list. - expect(engine.rows.map((r) => r.name)).not.toContain(CONNECTOR_NESTED.name); - expect(result.skipped).toBe(0); + // SUBJECT: the connector-nested webhook is no declared webhook — and no + // validation warning either, because the reader never sees it at all. A + // future reader that merely warned about it would move this, not just the + // name list. + expect(declared.map((d) => d.webhook.name)).not.toContain(CONNECTOR_NESTED.name); + expect(await reader.declaredWebhookNamed(CONNECTOR_NESTED.name)).toBeNull(); + expect(warn).toEqual([]); }); }); diff --git a/packages/plugins/plugin-webhooks/src/declared-webhooks.test.ts b/packages/plugins/plugin-webhooks/src/declared-webhooks.test.ts new file mode 100644 index 00000000000..134245506bf --- /dev/null +++ b/packages/plugins/plugin-webhooks/src/declared-webhooks.test.ts @@ -0,0 +1,642 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Declared webhooks, read from the registry (ADR-0131 D2/D6) — the dispatcher + * side of the retired `bootstrapDeclaredWebhooks` seeder. + * + * Three things are pinned here: + * + * 1. **The new shape, on a fresh install.** A declared webhook dispatches from + * its declaration — URL, method, triggers, `timeoutMs`, signing secret and + * headers — and nothing writes a `sys_webhook` row for it. + * 2. **The switch (regime C).** A `sys_metadata_activation` row of type + * `webhook` turns a declared webhook off, or on when it shipped inactive, + * and a ledger write refreshes the cache in the request that made it. + * 3. **The dual read, on an 18.0-shaped database.** The rows the 18.0 seeder + * wrote stay. A pristine one is ignored in favour of its declaration — one + * delivery, never two; an edited one (`customized: true`) or an admin row + * of the same name answers as itself and the declaration is withheld, so + * an administrator's `active: false` survives the upgrade. The fixture + * {@link seeded18_0Row} is byte-for-byte the row shape 18.0's seeder wrote + * (its `mapWebhookToRow` plus provenance), which is the shape an upgraded + * database holds. + */ + +import { describe, expect, it, vi } from 'vitest'; +import { AutoEnqueuer, type HttpEnqueueFn } from './auto-enqueuer.js'; +import { + DeclaredWebhookSource, + WEBHOOK_ACTIVATION_TYPE, + declaredWebhookRefId, +} from './declared-webhooks.js'; +import { createWebhookRedeliverGuard } from './redeliver-guard.js'; +import { bindWebhookProvenanceStamp } from './webhook-provenance.js'; +import { WebhookOutboxPlugin } from './webhook-outbox-plugin.js'; +import { assertEngineFindOnePredicate } from '@objectstack/metadata-core'; + +// --------------------------------------------------------------------------- +// Fakes +// --------------------------------------------------------------------------- + +const LEDGER = 'sys_metadata_activation'; + +interface HookEntry { + event: string; + handler: (ctx: any) => any; + object?: string; + packageId?: string; +} + +/** Equality and `$in` — the two predicate shapes the reader and the dispatcher send. */ +function rowMatches(row: any, cond?: Record): boolean { + if (!cond) return true; + return Object.entries(cond).every(([k, v]) => { + if (k.startsWith('$')) throw new Error(`fake engine: the ${k} combinator is not implemented`); + if (v && typeof v === 'object' && Array.isArray((v as any).$in)) return (v as any).$in.includes(row[k]); + return row[k] === v; + }); +} + +/** + * The ObjectQL surface the dispatcher, the reader and the provenance hook + * touch: `find({ where | filter })` with equality and `$in` and the caller's + * `limit`, `insert`, update-by-id with `beforeUpdate` hooks, and a + * `_registry.listItems(type)` for declared metadata. Every write is recorded, so + * "nothing writes a row" is a reading, not an assumption. + */ +function makeFakeEngine(rows: Record, declared: Record) { + const writes: Array<{ op: string; object: string; data: any }> = []; + let hooks: HookEntry[] = []; + return { + rows, + writes, + declared, + get _registry() { + return { listItems: (type: string) => [...(declared[type] ?? [])] }; + }, + async find(name: string, q?: any): Promise { + const matched = (rows[name] ?? []).filter((r) => rowMatches(r, q?.filter ?? q?.where)); + const bounded = q?.limit !== undefined ? matched.slice(0, q.limit) : matched; + return bounded.map((r) => ({ ...r })); + }, + async findOne(name: string, q?: any): Promise { + assertEngineFindOnePredicate(name, q); + return (await this.find(name, q))[0] ?? null; + }, + async insert(name: string, data: any): Promise { + writes.push({ op: 'insert', object: name, data }); + (rows[name] = rows[name] ?? []).push({ ...data }); + return data; + }, + async update(name: string, data: any, opts?: any): Promise { + writes.push({ op: 'update', object: name, data }); + const id = data?.id ?? opts?.where?.id; + const cond = opts?.where ?? (id ? { id } : undefined); + const previous = (rows[name] ?? []).find((r) => rowMatches(r, cond)); + const ctx = { input: { id, data }, previous: previous ? { ...previous } : undefined, session: opts?.context }; + for (const h of hooks) { + if (h.event === 'beforeUpdate' && (!h.object || h.object === name)) await h.handler(ctx); + } + for (const r of rows[name] ?? []) if (rowMatches(r, cond)) Object.assign(r, data); + return { affected: 1 }; + }, + async delete(): Promise { + return { affected: 0 }; + }, + async count(name: string): Promise { + return (rows[name] ?? []).length; + }, + registerHook(event: string, handler: (ctx: any) => any, options?: Record): void { + hooks.push({ event, handler, object: options?.object, packageId: options?.packageId }); + }, + unregisterHooksByPackage(packageId: string): number { + const before = hooks.length; + hooks = hooks.filter((h) => h.packageId !== packageId); + return before - hooks.length; + }, + }; +} + +type FakeEngine = ReturnType; + +function newEngine(seed?: { rows?: Record; declared?: Record }): FakeEngine { + return makeFakeEngine( + seed?.rows ? JSON.parse(JSON.stringify(seed.rows)) : {}, + seed?.declared ? JSON.parse(JSON.stringify(seed.declared)) : {}, + ); +} + +class FakeRealtime { + private subs = new Map(); + private n = 0; + async publish(event: any): Promise { + for (const sub of this.subs.values()) { + const o = sub.opts ?? {}; + if (o.object && event.object !== o.object) continue; + await sub.handler(event); + } + } + async subscribe(_channel: string, handler: any, opts?: any): Promise { + const id = `s-${++this.n}`; + this.subs.set(id, { handler, opts }); + return id; + } + async unsubscribe(id: string): Promise { + this.subs.delete(id); + } +} + +const ADMIN_CTX = { isSystem: false, positions: [], permissions: [] }; + +function declaredWebhook(over: Record = {}): any { + return { + name: 'task_changed', + label: 'Task Changed', + object: 'showcase_task', + triggers: ['create', 'update'], + url: 'https://hooks.example/task', + method: 'POST', + isActive: true, + ...over, + }; +} + +/** + * The row 18.0's `bootstrapDeclaredWebhooks` wrote for `wh` — its + * `mapWebhookToRow` (lower-cased method, `object`→`object_name`, + * `isActive`→`active`, the envelope minus both credentials in + * `definition_json`) plus `managed_by: 'package'` / `customized: false`. The + * credentials went to the encrypted columns; this fake engine has no encrypted + * channel, so the column holds the plaintext the real one would dereference + * to, which is what `resolveWebhookSecret` reads back here. + */ +function seeded18_0Row(id: string, wh: any, over: Record = {}): any { + const { secret, headers, ...envelope } = { method: 'POST', timeoutMs: 30000, isActive: true, ...wh }; + return { + id, + name: wh.name, + label: wh.label ?? wh.name, + object_name: wh.object ?? null, + triggers: wh.triggers ?? [], + url: wh.url, + method: String(envelope.method).toLowerCase(), + description: wh.description ?? null, + active: envelope.isActive !== false, + definition_json: JSON.stringify(envelope), + ...(secret ? { signing_secret: secret } : {}), + ...(headers ? { headers_secret: JSON.stringify(headers) } : {}), + managed_by: 'package', + customized: false, + created_at: '2026-10-01T00:00:00.000Z', + updated_at: '2026-10-01T00:00:00.000Z', + ...over, + }; +} + +function ledgerRow(name: string, active: boolean): any { + return { id: `act_${name}`, metadata_type: WEBHOOK_ACTIVATION_TYPE, name, active }; +} + +function taskCreated(recordId = 't-1'): any { + return { + type: 'data.record.created', + object: 'showcase_task', + payload: { recordId }, + timestamp: '2026-10-19T00:00:00.000Z', + }; +} + +async function flush() { + await new Promise((r) => setTimeout(r, 0)); + await new Promise((r) => setTimeout(r, 0)); +} + +async function startEnqueuer(engine: FakeEngine, logger?: any, deploymentWide?: boolean) { + const realtime = new FakeRealtime(); + const calls: any[] = []; + const enqueue: HttpEnqueueFn = async (input) => { + calls.push(input); + return 'id'; + }; + const declaredWebhooks = new DeclaredWebhookSource(engine, undefined, logger); + const enqueuer = new AutoEnqueuer(engine as any, realtime as any, enqueue, { + refreshIntervalMs: 0, + declaredWebhooks, + ...(deploymentWide !== undefined ? { declaredWebhooksDeploymentWide: () => deploymentWide } : {}), + ...(logger ? { logger } : {}), + }); + await enqueuer.start(); + return { enqueuer, realtime, calls, declaredWebhooks }; +} + +// --------------------------------------------------------------------------- +// 1. The new shape, on a fresh install +// --------------------------------------------------------------------------- + +describe('declared webhooks — a fresh install dispatches from the registry', () => { + it('a declared webhook dispatches on a matching event, with its secret, headers, method and timeout from the declaration', async () => { + const engine = newEngine({ + declared: { + webhook: [declaredWebhook({ + triggers: ['create'], secret: 'shh', headers: { 'X-Env': 'prod' }, method: 'PUT', timeoutMs: 5000, + })], + }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + + await realtime.publish(taskCreated()); + await flush(); + + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ + source: 'webhook', + refId: declaredWebhookRefId('task_changed'), + url: 'https://hooks.example/task', + method: 'PUT', + headers: { 'X-Env': 'prod' }, + signingSecret: 'shh', + timeoutMs: 5000, + }); + expect(calls[0].organizationId).toBeUndefined(); + await enqueuer.stop(); + }); + + it('nothing writes a sys_webhook row: the plugin boot binds the reader and seeds zero rows', async () => { + const engine = newEngine({ declared: { webhook: [declaredWebhook({ secret: 'shh' })] } }); + const services: Record = { objectql: engine, manifest: { register() {} } }; + const hooks: Record Promise>> = {}; + const ctx: any = { + logger: { info() {}, warn() {}, error() {}, debug() {} }, + getService: (n: string) => { + if (!(n in services)) throw new Error(`no ${n}`); + return services[n]; + }, + registerService: (n: string, s: unknown) => { services[n] = s; }, + hook: (event: string, fn: () => Promise) => { (hooks[event] ??= []).push(fn); }, + }; + const plugin = new WebhookOutboxPlugin({ autoEnqueue: false }); + await plugin.init(ctx); + for (const fn of hooks['kernel:ready'] ?? []) await fn(); + + expect(engine.writes.filter((w) => w.object === 'sys_webhook')).toEqual([]); + expect(engine.rows['sys_webhook']).toBeUndefined(); + const reader = services['declaredWebhooks'] as DeclaredWebhookSource; + expect(reader).toBeInstanceOf(DeclaredWebhookSource); + expect((await reader.declaredWebhookNamed('task_changed'))?.webhook.object).toBe('showcase_task'); + await plugin.destroy(); + }); + + it('a declared webhook shipped inactive is not dispatched', async () => { + const engine = newEngine({ declared: { webhook: [declaredWebhook({ isActive: false })] } }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls).toEqual([]); + await enqueuer.stop(); + }); + + it('an invalid declaration is skipped with one warning per name, and a valid one beside it still dispatches', async () => { + const warn = vi.fn(); + const engine = newEngine({ + declared: { webhook: [declaredWebhook({ name: 'good' }), declaredWebhook({ name: 'bad', url: 'not-a-url' })] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine, { warn, debug() {} }); + await enqueuer.refresh(); + await realtime.publish(taskCreated()); + await flush(); + + expect(calls.map((c) => c.refId)).toEqual([declaredWebhookRefId('good')]); + const reports = warn.mock.calls.filter(([m]) => String(m).includes('failed validation')); + expect(reports).toHaveLength(1); + expect(reports[0][1]).toMatchObject({ name: 'bad' }); + await enqueuer.stop(); + }); +}); + +describe('declared webhooks — the organization dimension follows the posture (ADR-0131 D2/D9)', () => { + const orgEvent = (organizationId: string) => ({ + ...taskCreated(), + payload: { recordId: 't-1', organizationId }, + }); + + it('with no wall (`single`) a declared webhook receives an organization-stamped event, its delivery stamped with that organization', async () => { + // Under `single` every record carries the Default Organization (D9), and + // the 18.0 seeded copy carried it too, so it delivered. The declaration + // has no organization of its own; deployment-wide, it delivers as well. + const engine = newEngine({ declared: { webhook: [declaredWebhook()] } }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine, undefined, true); + await realtime.publish(orgEvent('org_default')); + await flush(); + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ refId: declaredWebhookRefId('task_changed'), organizationId: 'org_default' }); + await enqueuer.stop(); + }); + + it('under a wall a declared webhook stays org-less: an organization-walled event is refused, said once', async () => { + const warn = vi.fn(); + const engine = newEngine({ declared: { webhook: [declaredWebhook()] } }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine, { warn, debug() {} }, false); + await realtime.publish(orgEvent('org_a')); + await realtime.publish(orgEvent('org_a')); + await flush(); + expect(calls).toEqual([]); + expect(warn.mock.calls.filter(([m]) => String(m).includes('belongs to NO organization'))).toHaveLength(1); + await enqueuer.stop(); + }); + + it('the plugin reads the posture from the tenancy service: `single` is deployment-wide, an absent service is walled', async () => { + const boot = async (tenancy?: unknown) => { + const engine = newEngine({ declared: { webhook: [declaredWebhook()] } }); + const realtime = new FakeRealtime(); + const calls: any[] = []; + const services: Record = { + objectql: engine, + manifest: { register() {} }, + realtime, + messaging: { + isHttpDeliveryReady: () => true, + enqueueHttp: async (input: any) => { calls.push(input); return 'd'; }, + redeliverHttp: async () => ({ id: 'd', status: 'pending' }), + registerRedeliverGuard() {}, + }, + ...(tenancy ? { tenancy } : {}), + }; + const hooks: Array<() => Promise> = []; + const ctx: any = { + logger: { info() {}, warn() {}, error() {}, debug() {} }, + getService: (n: string) => { + if (!(n in services)) throw new Error(`no ${n}`); + return services[n]; + }, + registerService: (n: string, svc: unknown) => { services[n] = svc; }, + hook: (event: string, fn: () => Promise) => { if (event === 'kernel:ready') hooks.push(fn); }, + }; + const plugin = new WebhookOutboxPlugin({ autoEnqueue: { refreshIntervalMs: 0 } }); + await plugin.init(ctx); + for (const fn of hooks) await fn(); + await realtime.publish(orgEvent('org_default')); + await flush(); + await plugin.destroy(); + return calls; + }; + expect(await boot({ posture: 'single' })).toHaveLength(1); + expect(await boot({ posture: 'isolated' })).toEqual([]); + expect(await boot()).toEqual([]); + }); +}); + +// --------------------------------------------------------------------------- +// 2. The switch (regime C) +// --------------------------------------------------------------------------- + +describe('declared webhooks — the activation ledger switches them (ADR-0131 D6 regime C)', () => { + it('a ledger row with active:false switches a declared webhook off', async () => { + const engine = newEngine({ + declared: { webhook: [declaredWebhook()] }, + rows: { [LEDGER]: [ledgerRow('task_changed', false)] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls).toEqual([]); + await enqueuer.stop(); + }); + + it('a ledger row with active:true switches on a webhook shipped inactive', async () => { + const engine = newEngine({ + declared: { webhook: [declaredWebhook({ isActive: false })] }, + rows: { [LEDGER]: [ledgerRow('task_changed', true)] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls).toHaveLength(1); + await enqueuer.stop(); + }); + + it('another type\'s ledger row of the same name switches nothing', async () => { + const engine = newEngine({ + declared: { webhook: [declaredWebhook()] }, + rows: { [LEDGER]: [{ id: 'a1', metadata_type: 'flow', name: 'task_changed', active: false }] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls).toHaveLength(1); + await enqueuer.stop(); + }); + + it('a ledger write refreshes the cache in the same request — no wait for the periodic refresh', async () => { + const engine = newEngine({ declared: { webhook: [declaredWebhook()] } }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + + engine.rows[LEDGER] = [ledgerRow('task_changed', false)]; + await realtime.publish({ type: 'data.record.created', object: LEDGER, payload: { recordId: 'act_task_changed' } }); + await flush(); + await realtime.publish(taskCreated()); + await flush(); + + expect(calls).toEqual([]); + await enqueuer.stop(); + }); + + it('a ledger read that fails keeps each declared webhook at its declared isActive', async () => { + const warn = vi.fn(); + const engine = newEngine({ declared: { webhook: [declaredWebhook()] } }); + const read = engine.find; + engine.find = async (name: string, q?: any) => { + if (name === LEDGER) throw new Error('ledger down'); + return read.call(engine, name, q); + }; + const { enqueuer, realtime, calls } = await startEnqueuer(engine, { warn, debug() {} }); + await realtime.publish(taskCreated()); + await flush(); + expect(calls).toHaveLength(1); + expect(warn).toHaveBeenCalledWith(expect.stringContaining('sys_metadata_activation read failed'), expect.anything()); + await enqueuer.stop(); + }); +}); + +// --------------------------------------------------------------------------- +// 3. The dual read, on an 18.0-shaped database +// --------------------------------------------------------------------------- + +describe('declared webhooks — an 18.0 database keeps its seeded rows, and nothing fires twice', () => { + it('a pristine seeded row is ignored in favour of its declaration: one delivery, from the declaration', async () => { + const wh = declaredWebhook({ secret: 'shh' }); + const engine = newEngine({ + declared: { webhook: [wh] }, + rows: { sys_webhook: [seeded18_0Row('whk_seeded', wh)] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + + expect(calls).toHaveLength(1); + expect(calls[0].refId).toBe(declaredWebhookRefId('task_changed')); + expect(calls[0].signingSecret).toBe('shh'); + // Expand first: the row is left exactly where 18.0 put it. + expect(engine.rows['sys_webhook']).toHaveLength(1); + expect(engine.writes).toEqual([]); + await enqueuer.stop(); + }); + + it('a pristine seeded row follows its declaration\'s switch: switched off, neither the row nor the declaration fires', async () => { + const wh = declaredWebhook(); + const engine = newEngine({ + declared: { webhook: [wh] }, + rows: { sys_webhook: [seeded18_0Row('whk_seeded', wh)], [LEDGER]: [ledgerRow('task_changed', false)] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls).toEqual([]); + await enqueuer.stop(); + }); + + it('a pristine seeded row follows a changed declaration — the new URL, as the 18.0 seeder would have rewritten it', async () => { + const engine = newEngine({ + declared: { webhook: [declaredWebhook({ url: 'https://hooks.example/task-v2' })] }, + rows: { sys_webhook: [seeded18_0Row('whk_seeded', declaredWebhook())] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls.map((c) => c.url)).toEqual(['https://hooks.example/task-v2']); + await enqueuer.stop(); + }); + + it('a seeded row an administrator switched off (customized) stays off: the declaration is withheld, said once', async () => { + const warn = vi.fn(); + const wh = declaredWebhook(); + const engine = newEngine({ + declared: { webhook: [wh] }, + rows: { sys_webhook: [seeded18_0Row('whk_seeded', wh, { active: false, customized: true })] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine, { warn, debug() {} }); + await enqueuer.refresh(); + await realtime.publish(taskCreated()); + await flush(); + + expect(calls).toEqual([]); + const withheld = warn.mock.calls.filter(([m]) => String(m).includes("declared webhook 'task_changed' is not dispatched")); + expect(withheld).toHaveLength(1); + await enqueuer.stop(); + }); + + it('a seeded row an administrator pointed elsewhere (customized) answers as itself — one delivery, to the row\'s URL', async () => { + const wh = declaredWebhook({ isActive: false }); + const engine = newEngine({ + declared: { webhook: [wh] }, + rows: { + sys_webhook: [seeded18_0Row('whk_seeded', wh, { + active: true, customized: true, url: 'https://real.example/task', + })], + }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ refId: 'whk_seeded', url: 'https://real.example/task' }); + await enqueuer.stop(); + }); + + it('an admin edit through the provenance stamp turns a pristine seeded row into one that answers as itself', async () => { + const wh = declaredWebhook(); + const engine = newEngine({ + declared: { webhook: [wh] }, + rows: { sys_webhook: [seeded18_0Row('whk_seeded', wh)] }, + }); + bindWebhookProvenanceStamp(engine as any); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + + await engine.update('sys_webhook', { id: 'whk_seeded', active: false }, { context: ADMIN_CTX }); + expect(engine.rows['sys_webhook'][0].customized).toBe(true); + await realtime.publish({ type: 'data.record.updated', object: 'sys_webhook', payload: { recordId: 'whk_seeded' } }); + await flush(); + await realtime.publish(taskCreated()); + await flush(); + + expect(calls).toEqual([]); + await enqueuer.stop(); + }); + + it('an administrator-authored row under a declared name answers as itself, and the declaration is withheld', async () => { + const engine = newEngine({ + declared: { webhook: [declaredWebhook()] }, + rows: { + sys_webhook: [{ + id: 'admin-1', name: 'task_changed', object_name: 'showcase_task', triggers: ['create'], + url: 'https://admin.example', method: 'post', active: true, managed_by: 'admin', customized: false, + }], + }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls).toHaveLength(1); + expect(calls[0]).toMatchObject({ refId: 'admin-1', url: 'https://admin.example' }); + await enqueuer.stop(); + }); + + it('a pristine seeded row whose declaration is gone keeps dispatching, as it did on 18.0', async () => { + const engine = newEngine({ + rows: { sys_webhook: [seeded18_0Row('whk_orphan', declaredWebhook({ name: 'retired_hook' }))] }, + }); + const { enqueuer, realtime, calls } = await startEnqueuer(engine); + await realtime.publish(taskCreated()); + await flush(); + expect(calls.map((c) => c.refId)).toEqual(['whk_orphan']); + await enqueuer.stop(); + }); +}); + +// --------------------------------------------------------------------------- +// The redelivery veto reads the same declaration +// --------------------------------------------------------------------------- + +describe('declared webhooks — the redelivery veto', () => { + const guardFor = (engine: FakeEngine) => + createWebhookRedeliverGuard(engine as any, 'sys_webhook', new DeclaredWebhookSource(engine, undefined)); + + it('allows a declared webhook\'s delivery while it is declared and switched on', async () => { + const engine = newEngine({ declared: { webhook: [declaredWebhook({ secret: 'shh' })] } }); + expect(await guardFor(engine)({ source: 'webhook', refId: declaredWebhookRefId('task_changed') })).toBeUndefined(); + }); + + it('refuses one whose declaration is gone, and one switched off through the ledger', async () => { + const gone = newEngine(); + expect(await guardFor(gone)({ source: 'webhook', refId: declaredWebhookRefId('task_changed') })) + .toMatch(/no longer declared/); + + const off = newEngine({ + declared: { webhook: [declaredWebhook()] }, + rows: { [LEDGER]: [ledgerRow('task_changed', false)] }, + }); + expect(await guardFor(off)({ source: 'webhook', refId: declaredWebhookRefId('task_changed') })) + .toMatch(/switched off/); + }); + + it('judges an 18.0 pristine seeded row\'s old delivery by the declaration that now answers for it', async () => { + const wh = declaredWebhook(); + const engine = newEngine({ + declared: { webhook: [wh] }, + rows: { sys_webhook: [seeded18_0Row('whk_seeded', wh)], [LEDGER]: [ledgerRow('task_changed', false)] }, + }); + // The row itself still reads active:true; the declaration's switch governs. + expect(await guardFor(engine)({ source: 'webhook', refId: 'whk_seeded' })).toMatch(/switched off/); + }); + + it('judges a customized seeded row by the row, as before', async () => { + const wh = declaredWebhook(); + const engine = newEngine({ + declared: { webhook: [wh] }, + rows: { + sys_webhook: [seeded18_0Row('whk_seeded', wh, { customized: true })], + [LEDGER]: [ledgerRow('task_changed', false)], + }, + }); + expect(await guardFor(engine)({ source: 'webhook', refId: 'whk_seeded' })).toBeUndefined(); + }); +}); diff --git a/packages/plugins/plugin-webhooks/src/declared-webhooks.ts b/packages/plugins/plugin-webhooks/src/declared-webhooks.ts new file mode 100644 index 00000000000..23e538264e1 --- /dev/null +++ b/packages/plugins/plugin-webhooks/src/declared-webhooks.ts @@ -0,0 +1,255 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * Declared webhooks, read from the registry — ADR-0131 D2 and D6, as the + * ruling on the declared-webhook seeder decided (Q2 A: the seeder retires and + * the dispatcher reads the declaration). + * + * A stack's `webhooks:` are code. They used to be COPIED into `sys_webhook` on + * every boot by `bootstrapDeclaredWebhooks`, one `managed_by: 'package'` row per + * declaration. That seeder is retired: a fresh boot writes no `sys_webhook` row + * for a declared webhook, and the dispatcher ({@link AutoEnqueuer}) reads the + * declaration itself, here, on every cache refresh. `sys_webhook` holds only the + * webhooks an organization authors — plus, on a database an 18.0 install + * created, the mirror rows that release's seeder left behind (below). + * + * ## Where the declaration is read + * + * The two in-process readers the seeder read, in the same order: + * + * 1. ObjectQL's `SchemaRegistry` (`engine._registry.listItems('webhook')`), + * which manifest decomposition fills from every stack's TOP-LEVEL + * `webhooks:` collection. A connector's nested `webhooks[]` is never one of + * these items: that key is retired under ADR-0049 (`integration/ + * connector.zod.ts`), and widening this reader to walk connectors would + * resurrect it. `declared-webhooks.connector-nested.test.ts` pins the + * absence. + * 2. The kernel `metadata` service (`list('webhook')`), only when the registry + * holds no webhook at all — the registry-less host. A promise-returning + * `list` is not awaited, as the seeder did not await it. + * + * Both readers hand back the authoring document itself, so each item is parsed + * through `WebhookSchema` — defaults for `method` / `isActive` / `timeoutMs` + * applied, a malformed item skipped with one warning per name per process, + * never a crashed refresh. + * + * ## The switch: regime C (ADR-0131 D6) + * + * A declared webhook is read-only. It is switched on or off for the deployment + * by one `sys_metadata_activation` row of type {@link WEBHOOK_ACTIVATION_TYPE}, + * written through the activation door (`POST /api/v1/security/_activation/ + * webhook/:name`), and changed by a clone under a new name — an organization's + * own `sys_webhook` row. A ledger row overrides the declared `isActive` in both + * directions, so a webhook shipped inactive is activated by a row with + * `active: true`. No row means the declared default. The read follows the + * sharing-rule reader's rules: a composition with no ledger object issues no + * read, and a read that fails switches nothing for this pass. + * + * ## An 18.0 database: the dual read (expand first) + * + * The 18.0 seeder's rows stay where they are; deleting them, and re-sourcing + * their `sys_http_delivery` tails, is the ownership ceremony's step + * (ADR-0131 D10 fate 2). Until then the reader must not fire a declared webhook + * twice, and must not undo what an administrator did to a seeded row. So a + * `sys_webhook` row holding a declared name is judged by what the 18.0 seeder + * would have done with it on this boot ({@link classifyDeclaredNameHolder}): + * + * - **A pristine mirror** (`managed_by: 'package'`, not `customized`): the + * seeder would have overwritten it from the declaration. The row is ignored + * and the declaration answers. + * - **Any other row** (`customized: true`, or `managed_by: 'admin'`): the + * seeder never touched it, and an administrator's `active: false` on a noisy + * webhook survived every redeploy. The row answers as itself and the + * declaration is withheld, reported once per name per process. + * + * A pristine mirror whose declaration is gone is an ordinary row: 18.0 left it + * in place and dispatching, and so does this release. + */ + +import { WebhookSchema, type WebhookParsed } from '@objectstack/spec/automation'; +import { METADATA_ACTIVATION_TABLE } from '@objectstack/core'; + +/** System read context — the dispatcher reads its own configuration. */ +const SYSTEM_CTX = { isSystem: true } as const; + +/** + * [ADR-0131 D6 regime C] The `sys_metadata_activation.metadata_type` a declared + * webhook's switch is recorded under — one row per webhook name, no tenant + * column. + */ +export const WEBHOOK_ACTIVATION_TYPE = 'webhook'; + +/** + * The prefix of a declared webhook's subscription id, which is also the + * `sys_http_delivery.ref_id` its deliveries carry. A declared name is a + * `SnakeCaseIdentifierSchema` value and cannot hold a colon, and no row id this + * plugin or the engine mints starts this way, so a delivery's `ref_id` says by + * itself whether a declaration or a row produced it. + */ +export const DECLARED_WEBHOOK_REF_PREFIX = 'declared:'; + +/** The subscription id (and delivery `ref_id`) of the declared webhook `name`. */ +export function declaredWebhookRefId(name: string): string { + return `${DECLARED_WEBHOOK_REF_PREFIX}${name}`; +} + +/** The declared webhook name a delivery `ref_id` addresses, or `undefined` for a row's. */ +export function declaredWebhookNameOfRefId(refId: string): string | undefined { + return refId.startsWith(DECLARED_WEBHOOK_REF_PREFIX) + ? refId.slice(DECLARED_WEBHOOK_REF_PREFIX.length) || undefined + : undefined; +} + +interface Logger { + warn?: (msg: string, meta?: unknown) => void; +} + +/** One declared webhook, parsed, with its switch resolved. */ +export interface DeclaredWebhook { + /** The parsed declaration, defaults applied. */ + webhook: WebhookParsed; + /** The package that ships it (the registry item's `_packageId`), when one does. */ + packageId?: string; + /** In force on this deployment: the ledger row's `active` when one exists, else the declared `isActive`. */ + active: boolean; +} + +/** + * The read side the dispatcher, the redelivery veto and the activation door + * share — one reader, so the three cannot disagree about which declared + * webhooks exist or which are switched on. + */ +export interface DeclaredWebhookReader { + /** Every valid declaration, each with its switch resolved. */ + list(): Promise; + /** The declaration named `name`, switched on or off, or `null` when nothing declares it. */ + declaredWebhookNamed(name: string): Promise; +} + +/** + * The raw declared `webhook` items, registry first, metadata service second. + * Never throws: a reader that cannot answer yields nothing from that reader. + */ +export function readDeclaredWebhookItems(engine: unknown, metadataService: unknown): unknown[] { + try { + const reg = (engine as { _registry?: { listItems?: (type: string) => unknown[] } } | undefined)?._registry; + if (typeof reg?.listItems === 'function') { + const items = (reg.listItems('webhook') ?? []).filter(Boolean); + if (items.length > 0) return items; + } + } catch { + /* fall through to the metadata service */ + } + try { + const listed = (metadataService as { list?: (type: string) => unknown } | undefined)?.list?.('webhook'); + const arr = typeof (listed as { then?: unknown } | undefined)?.then === 'function' ? [] : (listed ?? []); + return Array.isArray(arr) ? arr.filter(Boolean) : []; + } catch { + return []; + } +} + +/** + * The reader the plugin binds: the declarations from + * {@link readDeclaredWebhookItems}, the switch from the activation ledger. + */ +export class DeclaredWebhookSource implements DeclaredWebhookReader { + /** Names already reported as failing validation — said once per process. */ + private readonly invalidReported = new Set(); + + constructor( + private readonly engine: unknown, + private readonly metadataService: unknown, + private readonly logger?: Logger, + ) {} + + async list(): Promise { + const parsed = this.parseDeclared(); + if (parsed.length === 0) return []; + const switched = await readWebhookActivation(this.engine, this.logger); + return parsed.map(({ webhook, packageId }) => ({ + webhook, + ...(packageId !== undefined ? { packageId } : {}), + active: switched.get(webhook.name) ?? webhook.isActive !== false, + })); + } + + async declaredWebhookNamed(name: string): Promise { + if (!name) return null; + return (await this.list()).find((d) => d.webhook.name === name) ?? null; + } + + private parseDeclared(): Array<{ webhook: WebhookParsed; packageId?: string }> { + const out: Array<{ webhook: WebhookParsed; packageId?: string }> = []; + const seen = new Set(); + for (const raw of readDeclaredWebhookItems(this.engine, this.metadataService)) { + let webhook: WebhookParsed; + try { + webhook = WebhookSchema.parse(raw); + } catch (err: any) { + const name = String((raw as { name?: unknown })?.name ?? ''); + if (!this.invalidReported.has(name)) { + this.invalidReported.add(name); + this.logger?.warn?.('[webhook] declared webhook failed validation — not dispatched', { + name: name || undefined, + error: err?.message ?? String(err), + }); + } + continue; + } + // Two registrations of one name: the first wins, as the seeder's + // name-keyed upsert let the first one hold the row. + if (seen.has(webhook.name)) continue; + seen.add(webhook.name); + const pkg = (raw as { _packageId?: unknown })._packageId; + out.push({ webhook, ...(typeof pkg === 'string' && pkg ? { packageId: pkg } : {}) }); + } + return out; + } +} + +/** + * [ADR-0131 D6 regime C] The declared webhooks the activation ledger switches, + * by name: `true` on, `false` off; a name with no row is absent. Read as the + * sharing-rule reader reads the same ledger: a composition with no ledger object + * issues no read, and a read that fails switches nothing for this pass. + */ +export async function readWebhookActivation(engine: unknown, logger?: Logger): Promise> { + const out = new Map(); + const ql = engine as { + registry?: { getObject?: (name: string) => unknown }; + find?: (object: string, query: unknown) => Promise; + } | undefined; + if (typeof ql?.find !== 'function') return out; + if (typeof ql.registry?.getObject === 'function' && !ql.registry.getObject(METADATA_ACTIVATION_TABLE)) return out; + try { + const rows = await ql.find(METADATA_ACTIVATION_TABLE, { + filter: { metadata_type: WEBHOOK_ACTIVATION_TYPE }, + fields: ['metadata_type', 'name', 'active'], + limit: 100000, + context: SYSTEM_CTX, + }); + for (const row of (Array.isArray(rows) ? rows : []) as Array<{ metadata_type?: unknown; name?: unknown; active?: unknown }>) { + if (row.metadata_type !== undefined && row.metadata_type !== WEBHOOK_ACTIVATION_TYPE) continue; + if (typeof row.name !== 'string' || !row.name) continue; + out.set(row.name, !(row.active === false || row.active === 0)); + } + } catch (err: any) { + logger?.warn?.( + `[webhook] ${METADATA_ACTIVATION_TABLE} read failed — declared webhooks keep their declared isActive for this pass`, + { error: err?.message ?? String(err) }, + ); + } + return out; +} + +/** + * How a `sys_webhook` row that holds a declared webhook's name is read — what + * the 18.0 seeder would have done with it on this boot (see the module doc): + * `'mirror'` — a pristine seeded copy, ignored in favour of the declaration; + * `'row'` — customized or administrator-authored, answering as itself while the + * declaration is withheld. + */ +export function classifyDeclaredNameHolder(row: { managed_by?: unknown; customized?: unknown }): 'mirror' | 'row' { + return row.managed_by === 'package' && row.customized !== true ? 'mirror' : 'row'; +} diff --git a/packages/plugins/plugin-webhooks/src/migrate-webhook-secrets.ts b/packages/plugins/plugin-webhooks/src/migrate-webhook-secrets.ts index ff82d04f0ad..78976c4ab85 100644 --- a/packages/plugins/plugin-webhooks/src/migrate-webhook-secrets.ts +++ b/packages/plugins/plugin-webhooks/src/migrate-webhook-secrets.ts @@ -6,9 +6,11 @@ * `signing_secret` column. * * ## Why a sweep and not just the seeder - * `bootstrapDeclaredWebhooks` re-seeds package-declared rows on every boot, so - * those heal themselves the moment the new mapping lands. The rows that do NOT - * heal are exactly the ones most likely to hold a real production key: + * The 18.0 seeder `bootstrapDeclaredWebhooks` (retired since: declared webhooks + * are read from the registry, ADR-0131 D2) re-seeded package-declared rows on + * every boot, so those healed themselves the moment the new mapping landed. The + * rows that did NOT heal are exactly the ones most likely to hold a real + * production key: * * - `managed_by: 'admin'` — authored in Setup, never touched by the seeder; * - `customized: true` — a package row an admin edited, deliberately frozen diff --git a/packages/plugins/plugin-webhooks/src/plugin-shutdown-stops-auto-enqueuer.test.ts b/packages/plugins/plugin-webhooks/src/plugin-shutdown-stops-auto-enqueuer.test.ts index a07af1773c3..554610f60b7 100644 --- a/packages/plugins/plugin-webhooks/src/plugin-shutdown-stops-auto-enqueuer.test.ts +++ b/packages/plugins/plugin-webhooks/src/plugin-shutdown-stops-auto-enqueuer.test.ts @@ -6,8 +6,9 @@ * * THE DEFECT, AND HOW FAR IT WENT. At `kernel:ready` the plugin binds two * hooks onto the data engine and starts an {@link AutoEnqueuer}, which holds - * TWO realtime subscriptions, a crypto-provider listener and a `setInterval` - * refresh timer. The teardown that released all of it was spelled `dispose()`. + * TWO realtime subscriptions (three since declared webhooks are read from the + * registry: the activation ledger's is the third), a crypto-provider listener + * and a `setInterval` refresh timer. The teardown that released all of it was spelled `dispose()`. * `Plugin` (`@objectstack/core`'s `types.ts`) declares `init()`, `start?(ctx)` * and `destroy?()` — and NO `dispose()` — and `ObjectKernel.performShutdown()` * / `LiteKernel.destroy()` walk the plugins in reverse calling @@ -121,7 +122,7 @@ describe('#10772 WebhookOutboxPlugin stops its auto-enqueuer on kernel shutdown' // POSITIVE CONTROL — the enqueuer really did subscribe, so the // assertion below measures a release and not an absence. - expect(fixture.realtime.live.size).toBe(2); + expect(fixture.realtime.live.size).toBe(3); await kernel.shutdown(); @@ -162,7 +163,7 @@ describe('#10772 WebhookOutboxPlugin stops its auto-enqueuer on kernel shutdown' it('the retained dispose() alias still tears down for an embedder that calls it directly', async () => { const { plugin, fixture } = await boot(); - expect(fixture.realtime.live.size).toBe(2); + expect(fixture.realtime.live.size).toBe(3); await plugin.dispose(); diff --git a/packages/plugins/plugin-webhooks/src/redeliver-guard.ts b/packages/plugins/plugin-webhooks/src/redeliver-guard.ts index 4a8096099b2..cfbc9451c59 100644 --- a/packages/plugins/plugin-webhooks/src/redeliver-guard.ts +++ b/packages/plugins/plugin-webhooks/src/redeliver-guard.ts @@ -61,9 +61,26 @@ * own presence check because the refusal REASON it returns is written from the * subscription row, and keeps its behaviour byte for byte: an unresolvable key * is refused with the text below, and anything else still propagates. + * + * ## A declared webhook's deliveries (ADR-0131 D2/D6) + * A declared webhook has no `sys_webhook` row: its deliveries carry + * `ref_id = declared:` (`declaredWebhookRefId`), and the guard asks the + * same questions of the declaration, through the reader the dispatcher uses. + * Not declared any more is case 1, switched off (the activation ledger, or a + * declared `isActive: false` no ledger row overrides) is case 2. Case 3 cannot + * arise: the signing key is code, held with the declaration. A pristine 18.0 + * mirror row (`classifyDeclaredNameHolder`) whose name is still declared is + * judged by its declaration too, because the dispatcher reads the declaration + * in its place: replaying one of its old deliveries must answer to the switch + * that now governs the webhook, not to a row nothing reads. */ import type { IDataEngine } from '@objectstack/spec/contracts'; +import { + classifyDeclaredNameHolder, + declaredWebhookNameOfRefId, + type DeclaredWebhookReader, +} from './declared-webhooks.js'; import { WEBHOOK_OBJECT, WEBHOOK_SECRET_FIELD, @@ -99,10 +116,14 @@ export interface RedeliverGuardRow { export function createWebhookRedeliverGuard( engine: IDataEngine, subscriptionsObject: string = WEBHOOK_OBJECT, + declared?: DeclaredWebhookReader, ): (row: RedeliverGuardRow) => Promise { return async (row) => { if (row.source !== 'webhook') return undefined; + const declaredName = declaredWebhookNameOfRefId(row.refId); + if (declaredName !== undefined) return judgeDeclared(declared, declaredName); + const subscription = (await engine.findOne( subscriptionsObject, { where: { id: row.refId } }, @@ -119,6 +140,11 @@ export function createWebhookRedeliverGuard( ); } + if (declared && classifyDeclaredNameHolder(subscription) === 'mirror' && typeof subscription.name === 'string') { + const declaration = await declared.declaredWebhookNamed(subscription.name); + if (declaration) return judgeDeclared(declared, subscription.name, declaration); + } + // [#22804] Case 2: the off switch. `=== true` is the dispatcher's own // predicate (`AutoEnqueuer` loads `{ active: true }`), so a replay is // refused for exactly the rows automatic dispatch skips — `false`, and @@ -165,3 +191,31 @@ export function createWebhookRedeliverGuard( ); }; } + +/** + * The guard's answer for a delivery a declared webhook produced, or a pristine + * 18.0 mirror the declaration now answers for (see the module doc). A lookup + * that throws propagates: "we could not check" never reads as "allowed". + */ +async function judgeDeclared( + declared: DeclaredWebhookReader | undefined, + name: string, + known?: Awaited>, +): Promise { + const declaration = known ?? (declared ? await declared.declaredWebhookNamed(name) : null); + if (!declaration) { + return ( + `the declared webhook '${name}' this delivery belongs to is no longer declared by the application, so ` + + 'there is nothing left to say whether it may still be signed and sent, and it is refused. Declare the ' + + 'webhook again if the endpoint should keep receiving events.' + ); + } + if (!declaration.active) { + return ( + `declared webhook '${name}' is switched off, and a switched-off webhook is skipped by the dispatcher, ` + + 'so this delivery is refused rather than sent to an endpoint that was switched off. Fix: switch it on ' + + `(POST /api/v1/security/_activation/webhook/${name} with { "enabled": true }), then redeliver.` + ); + } + return undefined; +} diff --git a/packages/plugins/plugin-webhooks/src/sys-webhook.object.ts b/packages/plugins/plugin-webhooks/src/sys-webhook.object.ts index 20a0e568e60..18789f7cccb 100644 --- a/packages/plugins/plugin-webhooks/src/sys-webhook.object.ts +++ b/packages/plugins/plugin-webhooks/src/sys-webhook.object.ts @@ -10,17 +10,20 @@ import { ObjectSchema, Field } from '@objectstack/spec/data'; * Studio UI without code changes. The canonical Zod schema for the * `definition_json` envelope lives at `@objectstack/spec/automation/webhook`. * - * ## Two authoring doors, one row - * Rows land here two ways, distinguished by the `managed_by` provenance column: + * ## Rows are the webhooks an organization authors (ADR-0131 D2) + * `managed_by` tells two kinds of row apart: * - **admin** — created/edited directly through this object's CRUD UI. - * - **package** — declared in code (`defineStack({ webhooks })` / - * `defineWebhook()`) and materialized on boot by - * `bootstrapDeclaredWebhooks` (#3461). Re-seeded every boot, but an admin - * edit stamps `customized: true` and freezes the row (seed-not-clobber, - * mirrors `sys_sharing_rule` #2909). + * - **package** — a copy of a webhook declared in code (`defineStack({ + * webhooks })` / `defineWebhook()`), written by the 18.0 boot seeder + * `bootstrapDeclaredWebhooks`. That seeder is retired: a declared webhook + * is read from the registry (`declared-webhooks.ts`) and no boot writes + * such a row any more. The rows an 18.0 install left stay until the + * ownership ceremony deletes them; meanwhile a pristine one is ignored in + * favour of its declaration, and one an administrator edited (`customized: + * true`) answers as itself. * - * One row per `name`. This plugin's {@link AutoEnqueuer} loads active rows on - * boot + on `sys_webhook:changed` events, and turns matching `data.record.*` + * One row per `name`. This plugin's {@link AutoEnqueuer} loads active rows, + * and the declared webhooks, on boot + on `sys_webhook:changed` events, and turns matching `data.record.*` * events into deliveries on the shared `service-messaging` HTTP outbox * (ADR-0018 M3 — `sys_http_delivery`, drained by the messaging dispatcher). * @@ -262,10 +265,10 @@ export const SysWebhook = ObjectSchema.create({ // ── Provenance (#3461 — record-authoritative seed-not-clobber) ── // Mirrors sys_sharing_rule (#2909). Both columns are `readonly`: the // engine strips them from non-system payloads (forge/clear-proof), while - // bootstrapDeclaredWebhooks and the provenance stamp hook write with - // isSystem. Deliberately NOT a write gate: webhooks are a first-class admin - // authoring/tuning surface — admins may edit or deactivate a package row; - // the seeder simply stops overwriting it once `customized` is stamped. + // the provenance stamp hook writes with isSystem (as the retired 18.0 seeder + // did). Deliberately NOT a write gate: webhooks are a first-class admin + // authoring/tuning surface — admins may edit or deactivate a package row, + // which then answers as itself rather than for its declaration. managed_by: Field.select( ['platform', 'package', 'admin'], { @@ -335,7 +338,7 @@ export const SysWebhook = ObjectSchema.create({ * |:---|:---|:---| * | Setup/Studio console — `nav_webhooks` (`webhook-outbox-plugin.ts`), the four list views above, `userActions` create/edit/delete | REST `/api/v1/data/sys_webhook` — the gated data API | `get` `list` `create` `update` `delete` | * | Operator predicate write — "deactivate every webhook on an object" (#4639, for which `AutoEnqueuer.handleSelfHealEvent` carries a `data.records.*` branch built expressly for this gesture) | REST `updateMany` / `deleteMany`, both gated on the `bulk` primitive | `bulk` | - * | `AutoEnqueuer` cache refresh, `bootstrapDeclaredWebhooks`, `stampWebhookProvenance`, `redeliver-guard`, `migrateLegacyWebhookSecrets`, the `headers_secret` write gate | `engine.find/findOne/insert/update` and lifecycle hooks — ObjectQL directly, which never consults `enable.apiMethods` | ungated: unaffected by anything declared here | + * | `AutoEnqueuer` cache refresh, `stampWebhookProvenance`, `redeliver-guard`, `migrateLegacyWebhookSecrets`, the `headers_secret` write gate | `engine.find/findOne/insert/update` and lifecycle hooks — ObjectQL directly, which never consults `enable.apiMethods` | ungated: unaffected by anything declared here | * * ⇒ every primitive is required by a real, measured consumer, so the set is * all six. No consumer outside the admin/operator surface was found. diff --git a/packages/plugins/plugin-webhooks/src/webhook-headers-gate.test.ts b/packages/plugins/plugin-webhooks/src/webhook-headers-gate.test.ts index 6ea136c2f20..b088bd95b75 100644 --- a/packages/plugins/plugin-webhooks/src/webhook-headers-gate.test.ts +++ b/packages/plugins/plugin-webhooks/src/webhook-headers-gate.test.ts @@ -39,7 +39,6 @@ import type { } from '@objectstack/spec/contracts'; import { SysWebhook } from './sys-webhook.object.js'; import { WEBHOOK_HEADERS_FIELD, resolveWebhookHeaders } from './webhook-headers.js'; -import { bootstrapDeclaredWebhooks } from './bootstrap-declared-webhooks.js'; import { WebhookHeadersShapeError, WEBHOOK_HEADERS_SHAPE_REFUSAL_CODE, @@ -386,35 +385,6 @@ describe('sys_webhook.headers_secret shape gate — the write door (#8566)', () ).rejects.toMatchObject({ code: 'VALIDATION_ERROR', status: 400 }); expect(cipherRows(stores)).toHaveLength(0); }); - - it('the declared-webhook seeder still materializes headers through the bound gate', async () => { - // Ruling item 2: the plugin's own write paths inherit this validation - // rather than carrying a second check. That is only safe if they pass - // — `bootstrapDeclaredWebhooks` writes `serializeHeaders()` of an - // already-filtered map, so it does, and this is the pin that keeps it - // true if either side moves. - const { engine, stores } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, { - list: (type: string) => (type === 'webhook' - ? [{ - name: 'crm_hook', object: 'contact', triggers: ['create'], - url: 'https://receiver.example/hook', method: 'POST', - headers: { 'X-Team': 'crm' }, - }] - : []), - } as any); - - const [row] = await engine.find('sys_webhook', { - where: { name: 'crm_hook' }, context: SYSTEM_CTX, - }); - expect(row).toBeDefined(); - // The seeder's write really went through the encrypted channel — it - // was not skipped, and the gate did not stand in its way. - expect(cipherRows(stores)).toHaveLength(1); - await expect(resolveWebhookHeaders(engine, row as any, 'sys_webhook')).resolves.toEqual({ - 'X-Team': 'crm', - }); - }); }); describe('the four values the gate deliberately lets through', () => { diff --git a/packages/plugins/plugin-webhooks/src/webhook-headers-gate.ts b/packages/plugins/plugin-webhooks/src/webhook-headers-gate.ts index 14e6859fc42..a9bdfbd38af 100644 --- a/packages/plugins/plugin-webhooks/src/webhook-headers-gate.ts +++ b/packages/plugins/plugin-webhooks/src/webhook-headers-gate.ts @@ -32,7 +32,7 @@ * * ## Why a hook, and why THIS hook * Maintainer ruling 2026-08-13 (option 2). Validating at the plugin's own write - * paths (`bootstrapDeclaredWebhooks` / `headersPatch` / the migration sweep) + * paths (the since-retired declared-webhook seeder / the migration sweep) * was rejected as insufficient: a direct `PATCH /api/v1/data/sys_webhook` never * goes through any of them, and that is the measured trigger. An engine hook * covers every door at once — the generic data API, the Setup UI, scripts, the @@ -291,8 +291,8 @@ const GATE_PRIORITY = 50; * ADMIN edit; this is a validity verdict on a payload, and a malformed header * map is exactly as unusable when a seeder writes it. Ruling item 2 says the * plugin's own write paths inherit this validation through the hook, which is - * only true if system writes are covered. They pass by construction — - * `bootstrapDeclaredWebhooks` and the migration sweep both write + * only true if system writes are covered. They pass by construction — the + * migration sweep (and the since-retired declared-webhook seeder) writes * `serializeHeaders(...)` of an already `isHeaderMap`-filtered map — so * covering them costs nothing and closes the door for a future write path that * is less careful. diff --git a/packages/plugins/plugin-webhooks/src/webhook-headers.ts b/packages/plugins/plugin-webhooks/src/webhook-headers.ts index 0da824063dd..8bba33268eb 100644 --- a/packages/plugins/plugin-webhooks/src/webhook-headers.ts +++ b/packages/plugins/plugin-webhooks/src/webhook-headers.ts @@ -340,50 +340,3 @@ export async function resolveWebhookHeaders( } return requireHeaderMap(plain, row, `${object}.${WEBHOOK_HEADERS_FIELD}`); } - -/** - * Decide what a RE-SEED should do with an existing row's `headers_secret`. - * - * Same discipline, and the same reason, as `secretPatch` in - * `bootstrap-declared-webhooks.ts`: a `secret`-typed write always mints a fresh - * `sys_secret` ciphertext row and the engine never deletes the superseded one, - * so blindly restating the declared headers on every boot would leak one orphan - * cipher row per webhook per restart. - * - * - declared map differs from stored ⇒ write it (an edit in code propagates); - * - identical ⇒ omit the key entirely, leaving the existing ref untouched; - * - declared headers removed, row still holds some ⇒ write `null` to CLEAR - * (code remains the authority for package rows); - * - engine cannot dereference (older engine, or the compare threw) ⇒ fall back - * to writing the declared value. A correct request beats tidy storage. - */ -export async function headersPatch( - engine: IDataEngine, - declared: WebhookHeaders | undefined, - row: { id: string; [k: string]: unknown }, - object: string, -): Promise> { - const hasStored = row?.[WEBHOOK_HEADERS_FIELD] != null && row[WEBHOOK_HEADERS_FIELD] !== ''; - - if (!declared) return hasStored ? { [WEBHOOK_HEADERS_FIELD]: null } : {}; - - const serialized = serializeHeaders(declared); - const resolver = engine as SecretResolvingEngine; - if (!hasStored || typeof resolver.resolveSecretField !== 'function') { - return { [WEBHOOK_HEADERS_FIELD]: serialized }; - } - - try { - const current = await resolver.resolveSecretField(object, String(row.id), WEBHOOK_HEADERS_FIELD); - // Compared as the CANONICAL serialization on both sides, not as raw - // strings: the stored form was produced by this same function, so key order - // is stable, and a re-parse guards against a hand-edited value that differs - // only in whitespace re-encrypting on every boot. - const stored = parseStoredHeaders(current); - return stored && serializeHeaders(stored) === serialized - ? {} - : { [WEBHOOK_HEADERS_FIELD]: serialized }; - } catch { - return { [WEBHOOK_HEADERS_FIELD]: serialized }; - } -} diff --git a/packages/plugins/plugin-webhooks/src/webhook-outbox-plugin.ts b/packages/plugins/plugin-webhooks/src/webhook-outbox-plugin.ts index 79d1c6bb9e9..c581513b29c 100644 --- a/packages/plugins/plugin-webhooks/src/webhook-outbox-plugin.ts +++ b/packages/plugins/plugin-webhooks/src/webhook-outbox-plugin.ts @@ -19,7 +19,8 @@ import { AutoEnqueuer, type AutoEnqueuerOptions } from './auto-enqueuer.js'; /** What the auth service's in-process session read answers, as `AuthSessionApi` declares it. */ type SessionRead = Awaited>>; import { SysWebhook } from './sys-webhook.object.js'; -import { bootstrapDeclaredWebhooks } from './bootstrap-declared-webhooks.js'; +import { DeclaredWebhookSource } from './declared-webhooks.js'; +import { normalizeTenancyPosture, postureEnforcesWall } from '@objectstack/spec/security'; import { migrateLegacyWebhookSecrets } from './migrate-webhook-secrets.js'; import { createWebhookRedeliverGuard } from './redeliver-guard.js'; import { bindWebhookProvenanceStamp, unbindWebhookProvenanceStamp } from './webhook-provenance.js'; @@ -72,6 +73,15 @@ interface MessagingHttpSurface { */ const WEBHOOKS_SLOT = 'webhooks'; +/** + * [ADR-0131 D6 regime C] The service slot the declared-webhook reader is bound + * to (`DeclaredWebhookReader`), which the activation door + * (`POST /api/v1/security/_activation/webhook/:name`) asks whether a name is + * declared — the reader the dispatcher and the redelivery veto use, so the + * three cannot disagree. The sharing-rule door's `sharingRules` is the pattern. + */ +const DECLARED_WEBHOOKS_SLOT = 'declaredWebhooks'; + /** * [#22756] The slice of a Hono context the redeliver door reads and answers * through, and nothing more: the request's headers (the caller's credentials), @@ -166,6 +176,8 @@ export class WebhookOutboxPlugin implements Plugin { private autoEnqueuer: AutoEnqueuer | undefined; /** Engine the provenance hook was bound to, so `dispose()` can unbind it. */ private boundEngine: any; + /** [ADR-0131 D2/D6] The declared-webhook reader, bound once the data engine is known. */ + private declaredSource: DeclaredWebhookSource | undefined; constructor(private readonly options: WebhookOutboxPluginOptions = {}) {} @@ -221,12 +233,11 @@ export class WebhookOutboxPlugin implements Plugin { if (typeof (ctx as any).hook === 'function') { (ctx as any).hook('kernel:ready', async () => { - // Materialize declared webhooks FIRST — this only needs the data + // The table's write doors, the legacy-secret sweep and the + // declared-webhook reader FIRST — this only needs the data // engine, so it must not be gated behind the auto-enqueue - // dispatch prerequisites (realtime + messaging). Otherwise a - // deployment without realtime would silently fail to materialize - // declared webhooks — the very no-op this bridge closes (#3461). - await this.bootDeclaredWebhooks(ctx); + // dispatch prerequisites (realtime + messaging). + await this.bootWebhookTable(ctx); // [#22756] The redelivery veto, and beside it the `webhooks` // service — gated on the engine and messaging alone, like the // step above, and never on the auto-enqueue prerequisites. It @@ -284,27 +295,35 @@ export class WebhookOutboxPlugin implements Plugin { } /** - * [#3461] Bridge the declarative authoring surface to the dispatcher: - * materialize stack/connector-declared `webhook` metadata into `sys_webhook` - * rows so the auto-enqueuer (and the Studio UI) can see them. + * The `sys_webhook` table's write doors, its legacy-secret sweep, and the + * declared-webhook reader. + * + * [ADR-0131 D2, D6] Declared webhooks are no longer materialized: the boot + * seeder (`bootstrapDeclaredWebhooks`) is retired, a fresh boot writes no + * `sys_webhook` row for one, and the dispatcher, the redelivery veto and the + * activation door read the declaration through one + * {@link DeclaredWebhookSource}, bound here to the `declaredWebhooks` slot. + * The 18.0 seeder's rows stay until the ownership ceremony deletes them; + * `declared-webhooks.ts` says how they are read meanwhile. * * Gated on the DATA ENGINE alone — deliberately independent of the - * auto-enqueue dispatch prerequisites (realtime + messaging). Materializing - * rows is a pure write; a deployment that mounts the webhook plugin without - * realtime must still get its declared webhooks into the table (and the - * Setup UI), even if nothing dispatches them yet. Runs before - * {@link bootAutoEnqueue} so the enqueuer's first cache refresh sees the rows. + * auto-enqueue dispatch prerequisites (realtime + messaging): the write + * doors, the sweep and the activation door serve on a kernel that + * dispatches nothing. Runs before {@link bootAutoEnqueue}, which reads the + * reader bound here. */ - private async bootDeclaredWebhooks(ctx: PluginContext): Promise { + private async bootWebhookTable(ctx: PluginContext): Promise { const engine = this.tryGetService(ctx, ['objectql', 'data']); if (!engine) { - ctx.logger.warn?.('[webhook] declared-webhook bootstrap skipped — no data engine available'); + ctx.logger.warn?.('[webhook] sys_webhook write doors and the declared-webhook reader not bound — no data engine available'); return; } - // Bind the provenance stamp so an admin edit freezes a seeded row. + // Bind the provenance stamp so an admin edit freezes a seeded row: an + // 18.0 mirror an administrator edits stops being one, and answers as + // itself rather than for its declaration (`classifyDeclaredNameHolder`). this.boundEngine = engine; bindWebhookProvenanceStamp(engine as any, ctx.logger as any); - // [#8566] And the headers_secret shape gate, BEFORE the seeder below + // [#8566] And the headers_secret shape gate, BEFORE the sweep below // runs its first write — a validation door that arms after the first // write it is meant to judge is not a door. It covers every write path // at once (the generic data API included, which is the measured @@ -313,24 +332,16 @@ export class WebhookOutboxPlugin implements Plugin { bindWebhookHeadersShapeGate(engine as any, ctx.logger as any); // And the retired credential location: a write that puts `secret` or // `headers` back into `definition_json` is refused at the door, since - // the delivery path refuses such a row too. Bound before the seeder and - // the sweep for the same reason — both pass it by construction. + // the delivery path refuses such a row too. Bound before the sweep for + // the same reason — it passes it by construction. bindWebhookLegacyCleartextGate(engine as any, ctx.logger as any); let metadataService: IMetadataService | undefined; try { metadataService = ctx.getService('metadata'); } catch { /* optional */ } - try { - await bootstrapDeclaredWebhooks(engine, metadataService, ctx.logger as any); - } catch (err: any) { - ctx.logger.warn?.('[webhook] declared-webhook bootstrap failed (dispatcher still serves admin rows)', { - error: err?.message ?? String(err), - }); - } - // [#7799] Then heal the rows the seeder cannot touch. Package rows are - // rewritten above; `managed_by: 'admin'` and `customized: true` rows are - // deliberately frozen against re-seeding, and those are precisely the - // ones holding hand-authored production keys in cleartext. Runs AFTER - // the seeder so a row it just rewrote is already secret-free and the - // sweep is a no-op on it. + this.declaredSource = new DeclaredWebhookSource(engine, metadataService, ctx.logger as any); + ctx.registerService(DECLARED_WEBHOOKS_SLOT, this.declaredSource); + // [#7799] Then heal the rows that still hold a key in cleartext: the + // ones an administrator authored or edited, and any 18.0 seeded row the + // seeder had not yet rewritten. try { await migrateLegacyWebhookSecrets(engine, ctx.logger as any); } catch (err: any) { @@ -371,7 +382,16 @@ export class WebhookOutboxPlugin implements Plugin { engine, realtime, (input) => messaging.enqueueHttp(input), - { ...enqOpts, logger: ctx.logger }, + { + ...enqOpts, + ...(this.declaredSource + ? { + declaredWebhooks: this.declaredSource, + declaredWebhooksDeploymentWide: () => this.postureBuildsNoWall(ctx), + } + : {}), + logger: ctx.logger, + }, ); await this.autoEnqueuer.start(); ctx.registerService('webhook.autoEnqueuer', this.autoEnqueuer); @@ -453,12 +473,29 @@ export class WebhookOutboxPlugin implements Plugin { } messaging.registerRedeliverGuard( 'webhook', - createWebhookRedeliverGuard(engine, subscriptionsObject), + createWebhookRedeliverGuard(engine, subscriptionsObject, this.declaredSource), ); ctx.logger.debug?.('[webhook-outbox] redeliver guard installed for source=webhook'); return true; } + /** + * [ADR-0131 D9] Whether the deployment's tenancy posture builds no + * organization wall — the `tenancy` service's posture, read as the sharing + * plugin reads it for its declared rules: an absent or unreadable service + * is read as walled, never as open. + */ + private postureBuildsNoWall(ctx: PluginContext): boolean { + try { + const probe = ctx.getService<{ posture?: unknown; isolationActive?: boolean }>('tenancy'); + const posture = normalizeTenancyPosture(probe?.posture); + if (posture) return !postureEnforcesWall(posture); + return probe?.isolationActive === false; + } catch { + return false; + } + } + private tryGetService(ctx: PluginContext, names: string[]): T | undefined { for (const n of names) { try { diff --git a/packages/plugins/plugin-webhooks/src/webhook-provenance.ts b/packages/plugins/plugin-webhooks/src/webhook-provenance.ts index 06e6fe8de89..5e542a58497 100644 --- a/packages/plugins/plugin-webhooks/src/webhook-provenance.ts +++ b/packages/plugins/plugin-webhooks/src/webhook-provenance.ts @@ -3,12 +3,14 @@ /** * [#3461] Provenance stamp for `sys_webhook`. * - * `sys_webhook` is RECORD-AUTHORITATIVE: a code-declared webhook is a boot seed - * (`bootstrapDeclaredWebhooks`), and the row — including any admin tuning such - * as flipping a noisy webhook to `active: false` — is the authority. The seeder - * skips rows marked `customized`, so this hook is the half that DETECTS the - * admin edit: any non-system update touching a `package`/`platform`-seeded row - * stamps `customized: true` onto the payload. + * `sys_webhook` is RECORD-AUTHORITATIVE: a row — including any admin tuning such + * as flipping a noisy webhook to `active: false` — is the authority. A + * `package` row is a copy of a declared webhook the 18.0 seeder + * (`bootstrapDeclaredWebhooks`, retired: ADR-0131 D2) wrote; the dispatcher + * ignores a pristine one in favour of its declaration and lets one marked + * `customized` answer as itself (`classifyDeclaredNameHolder`), so this hook is + * the half that DETECTS the admin edit: any non-system update touching a + * `package`/`platform`-seeded row stamps `customized: true` onto the payload. * * Why a data hook (and not a write gate or the REST layer), verbatim to the * sys_sharing_rule rationale (#2909 T1): diff --git a/packages/plugins/plugin-webhooks/src/webhook-secret-at-rest.test.ts b/packages/plugins/plugin-webhooks/src/webhook-secret-at-rest.test.ts index 2075850d4cb..ccac6ce50ea 100644 --- a/packages/plugins/plugin-webhooks/src/webhook-secret-at-rest.test.ts +++ b/packages/plugins/plugin-webhooks/src/webhook-secret-at-rest.test.ts @@ -38,15 +38,21 @@ import { type FetchImpl, } from '@objectstack/service-messaging'; import { AutoEnqueuer } from './auto-enqueuer.js'; -import { bootstrapDeclaredWebhooks } from './bootstrap-declared-webhooks.js'; +import { DeclaredWebhookSource, declaredWebhookRefId } from './declared-webhooks.js'; import { migrateLegacyWebhookSecrets } from './migrate-webhook-secrets.js'; import { SysWebhook } from './sys-webhook.object.js'; import { WEBHOOK_SECRET_FIELD, __objectqlSecretWireForms, resolveWebhookSecret, + splitWebhookSecret, } from './webhook-secret.js'; -import { WEBHOOK_HEADERS_FIELD, resolveWebhookHeaders } from './webhook-headers.js'; +import { + WEBHOOK_HEADERS_FIELD, + resolveWebhookHeaders, + serializeHeaders, + splitWebhookHeaders, +} from './webhook-headers.js'; import { bindWebhookLegacyCleartextGate } from './webhook-legacy-cleartext.js'; /** @@ -243,11 +249,49 @@ function declaredWebhook(overrides: Record = {}) { }; } -/** Metadata service that hands the seeder its declared items. */ +/** Metadata service that hands the declared-webhook reader its declared items. */ function metadataWith(items: unknown[]) { return { list: (type: string) => (type === 'webhook' ? items : []) }; } +/** + * Write the `sys_webhook` row an 18.0 install's boot seeder wrote for each + * declared webhook — the rows an upgraded database still holds (ADR-0131 D2: + * the seeder is retired, its rows stay until the ownership ceremony). The same + * mapping that seeder made, through the REAL engine, so both credentials land + * in their encrypted columns exactly as they did: `object`→`object_name`, + * `isActive`→`active`, a lower-cased method, the envelope minus `secret` and + * `headers` in `definition_json`, `managed_by: 'package'`. This file pins what + * the dispatcher, the sweep and the write doors do with credential-bearing + * rows, and these are the credential-bearing rows that exist without an + * administrator typing one in. + */ +async function seed18_0Rows(engine: any, metadata: { list(type: string): unknown[] }) { + for (const raw of metadata.list('webhook') as Array>) { + const wh: Record = { method: 'POST', timeoutMs: 30000, isActive: true, ...raw }; + const { envelope: withoutSecret, secret } = splitWebhookSecret(wh); + const { envelope, headers } = splitWebhookHeaders(withoutSecret as Record); + await engine.insert('sys_webhook', { + id: `whk_${wh.name}`, + name: wh.name, + label: wh.label ?? wh.name, + object_name: wh.object ?? null, + triggers: wh.triggers ?? [], + url: wh.url, + method: String(wh.method).toLowerCase(), + description: wh.description ?? null, + active: wh.isActive !== false, + definition_json: JSON.stringify(envelope), + ...(secret ? { [WEBHOOK_SECRET_FIELD]: secret } : {}), + ...(headers ? { [WEBHOOK_HEADERS_FIELD]: serializeHeaders(headers) } : {}), + managed_by: 'package', + customized: false, + created_at: '2026-10-01T00:00:00.000Z', + updated_at: '2026-10-01T00:00:00.000Z', + }, { context: SYSTEM_CTX } as any); + } +} + function recordEvent(object: string, record: any): RealtimeEventPayload { const payload = DataEventSchema.parse({ id: randomUUID(), @@ -342,7 +386,7 @@ describe('webhook signing secret at rest (#7799)', () => { it('the secret\'s bytes appear nowhere in the persisted sys_webhook row', async () => { const { engine, stores } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook()])); + await seed18_0Rows(engine, metadataWith([declaredWebhook()])); // ── The read path an ordinary GET /api/v1/data/sys_webhook takes ── // Substring, not a field check: the defect was the key NESTED inside a @@ -383,9 +427,36 @@ describe('webhook signing secret at rest (#7799)', () => { }); }); + it('a declared webhook, read from the registry, signs the same bytes with no row and no cipher stored', async () => { + const { engine, stores } = await buildEngine(); + const realtime = new FakeRealtime(); + const outbox = new MemoryHttpOutbox(); + const enqueuer = new AutoEnqueuer(engine, realtime, (input) => outbox.enqueue(input), { + refreshIntervalMs: 0, + declaredWebhooks: new DeclaredWebhookSource(engine, metadataWith([declaredWebhook()])), + }); + await enqueuer.start(); + await realtime.publish(recordEvent('contact', { id: 'c1', name: 'Ada' })); + await new Promise((r) => setTimeout(r, 0)); + const { impl, calls } = makeFetch(); + await new HttpDispatcher({ nodeId: 'n1', outbox, fetchImpl: impl, partitionCount: 1 }).tick(); + await enqueuer.stop(); + + expect(calls).toHaveLength(1); + const expected = createHmac('sha256', SECRET).update(calls[0].body).digest('hex'); + expect(calls[0].headers['X-Objectstack-Signature']).toBe(`sha256=${expected}`); + expect(calls[0].headers['X-Team']).toBe('crm'); + const [row] = (await outbox.list()) as any[]; + expect(row.refId ?? row.ref_id).toBe(declaredWebhookRefId('crm_hook')); + // The key and the header map are code: nothing at rest holds either. + expect(stores.get('sys_webhook')?.size ?? 0).toBe(0); + expect(stores.get('sys_secret')?.size ?? 0).toBe(0); + expect(JSON.stringify(await outbox.list())).not.toContain(SECRET); + }); + it('signs with the byte-identical signature existing receivers already verify', async () => { const { engine } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook()])); + await seed18_0Rows(engine, metadataWith([declaredWebhook()])); const { calls, outbox } = await deliverOnce(engine); @@ -402,33 +473,6 @@ describe('webhook signing secret at rest (#7799)', () => { expect(JSON.stringify(await outbox.list())).not.toContain(SECRET); }); - it('re-seeding an unchanged webhook does not mint a second sys_secret row', async () => { - const { engine, stores } = await buildEngine(); - const declared = metadataWith([declaredWebhook()]); - - await bootstrapDeclaredWebhooks(engine, declared); - await bootstrapDeclaredWebhooks(engine, declared); - await bootstrapDeclaredWebhooks(engine, declared); - - // Every boot re-seeds package rows; a blind restatement of the key would - // leave one orphan ciphertext row per restart. Two rows, not one, since - // #7986 — one per credential — and the property under test is that the - // count does not GROW across the three boots. - expect(stores.get('sys_secret')!.size).toBe(2); - }); - - it('rotating the declared secret in code re-encrypts and signs with the new key', async () => { - const { engine } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook()])); - - const ROTATED = 'whsec_7799_rotated'; - await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook({ secret: ROTATED })])); - - const { calls } = await deliverOnce(engine); - const expected = createHmac('sha256', ROTATED).update(calls[0].body).digest('hex'); - expect(calls[0].headers['X-Objectstack-Signature']).toBe(`sha256=${expected}`); - }); - it('a webhook authored without a secret still materializes and delivers unsigned', async () => { const { engine, stores } = await buildEngine(); // `headers` dropped alongside `secret` so this pin keeps asserting what @@ -436,7 +480,7 @@ describe('webhook signing secret at rest (#7799)', () => { // at all and therefore needs no CryptoProvider. (Since #7986 a header // map is a credential too; the headers-only case is pinned separately in // the #7986 block below.) - await bootstrapDeclaredWebhooks( + await seed18_0Rows( engine, metadataWith([declaredWebhook({ secret: undefined, headers: undefined })]), ); @@ -527,23 +571,28 @@ describe('legacy cleartext migration (#7799)', () => { }); describe('fail-closed: no CryptoProvider (#7799)', () => { - it('refuses to seed a secret-bearing webhook, reporting an ADR-0112 code + status', async () => { + it('a declared webhook needs no CryptoProvider at all: its secret is code, and nothing is stored', async () => { const { engine, stores } = await buildEngine({ withCrypto: false }); - const warns: Array<{ msg: string; meta: any }> = []; - - const result = await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook()]), { - warn: (msg: string, meta?: unknown) => { warns.push({ msg, meta }); }, + const realtime = new FakeRealtime(); + const outbox = new MemoryHttpOutbox(); + const enqueuer = new AutoEnqueuer(engine, realtime, (input) => outbox.enqueue(input), { + refreshIntervalMs: 0, + declaredWebhooks: new DeclaredWebhookSource(engine, metadataWith([declaredWebhook()])), }); + await enqueuer.start(); + await realtime.publish(recordEvent('contact', { id: 'c1', name: 'Ada' })); + await new Promise((r) => setTimeout(r, 0)); + const { impl, calls } = makeFetch(); + await new HttpDispatcher({ nodeId: 'n1', outbox, fetchImpl: impl, partitionCount: 1 }).tick(); + await enqueuer.stop(); - expect(result).toEqual({ seeded: 0, skipped: 1 }); - // Nothing was written — not the row, not a cleartext fallback column. + // Where the 18.0 seeder refused (no safe place for the key), the + // registry read has nothing to store: no row, no cipher, signed anyway. expect(stores.get('sys_webhook')?.size ?? 0).toBe(0); expect(stores.get('sys_secret')?.size ?? 0).toBe(0); - - // ADR-0112: a consumer branches on the pair, not on the message text. - const refusal = warns.find((w) => w.meta?.code); - expect(refusal?.meta).toMatchObject({ code: 'INTERNAL_ERROR', status: 500 }); - expect(refusal?.msg).toMatch(/cannot be stored encrypted/); + expect(calls).toHaveLength(1); + const expected = createHmac('sha256', SECRET).update(calls[0].body).digest('hex'); + expect(calls[0].headers['X-Objectstack-Signature']).toBe(`sha256=${expected}`); }); it('leaves a legacy row exactly as it was, and reports it as still cleartext', async () => { @@ -592,7 +641,7 @@ describe('boot ordering: the cache is built before the CryptoProvider (#8022)', /** Reboot onto the same rows in the host's real order: kernel first, crypto after. */ async function bootWithoutCrypto() { const first = await buildEngine(); - await bootstrapDeclaredWebhooks(first.engine, metadataWith([declaredWebhook()])); + await seed18_0Rows(first.engine, metadataWith([declaredWebhook()])); // Precondition: the credentials are at rest as ciphertext only — the // exact population #7799 created, and the only one this defect can // reach. Two rows since #7986 (signing secret + header map). @@ -751,7 +800,7 @@ describe('webhook custom headers at rest (#7986)', () => { it('the credential header appears nowhere in the persisted sys_webhook row', async () => { const { engine, stores } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); + await seed18_0Rows(engine, metadataWith([headerBearingWebhook()])); // ── The read path an ordinary GET /api/v1/data/sys_webhook takes ── // `sys_webhook` declares no `enable` block at all, so this is the FULL @@ -784,7 +833,7 @@ describe('webhook custom headers at rest (#7986)', () => { it('still delivers every authored header on the wire, credential included', async () => { const { engine } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); + await seed18_0Rows(engine, metadataWith([headerBearingWebhook()])); const { calls, outbox } = await deliverOnce(engine); @@ -801,42 +850,9 @@ describe('webhook custom headers at rest (#7986)', () => { expect(JSON.stringify(await outbox.list())).not.toContain(SECRET); }); - it('re-seeding an unchanged webhook mints no extra cipher rows for the headers', async () => { - const { engine, stores } = await buildEngine(); - const declared = metadataWith([headerBearingWebhook()]); - - await bootstrapDeclaredWebhooks(engine, declared); - await bootstrapDeclaredWebhooks(engine, declared); - await bootstrapDeclaredWebhooks(engine, declared); - - // A `secret`-typed write ALWAYS mints a fresh `sys_secret` row and the - // engine never deletes the old one, so a blind restatement leaks one - // orphan cipher row per webhook per restart — the reason - // `secretPatch` compares before writing. The headers channel owes the - // same discipline. Exactly two: one signing secret, one header map. - expect(stores.get('sys_secret')!.size).toBe(2); - }); - - it('rotating the declared headers in code re-encrypts and delivers the new value', async () => { - const { engine, stores } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); - await bootstrapDeclaredWebhooks( - engine, - metadataWith([headerBearingWebhook({ headers: { Authorization: 'Bearer rotated_7986' } })]), - ); - - const { calls } = await deliverOnce(engine); - expect(calls[0].headers.Authorization).toBe('Bearer rotated_7986'); - expect(calls[0].headers['X-Team']).toBeUndefined(); - // Code remains the authority for package rows: the superseded map is - // not readable from the row any more either. - const stored = Array.from(stores.get('sys_webhook')!.values())[0] as any; - expect(JSON.stringify(stored)).not.toContain(BEARER); - }); - it('a webhook authored without headers still materializes and delivers', async () => { const { engine, stores } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook({ headers: undefined })])); + await seed18_0Rows(engine, metadataWith([declaredWebhook({ headers: undefined })])); const { calls } = await deliverOnce(engine); expect(calls).toHaveLength(1); @@ -939,7 +955,7 @@ describe('legacy cleartext headers migration (#7986)', () => { describe('fail-closed and re-arm, extended to headers (#7986 × #7799/#8022)', () => { async function bootWithoutCrypto() { const first = await buildEngine(); - await bootstrapDeclaredWebhooks(first.engine, metadataWith([headerBearingWebhook()])); + await seed18_0Rows(first.engine, metadataWith([headerBearingWebhook()])); // Precondition: BOTH credentials are at rest as ciphertext only. expect(first.stores.get('sys_secret')!.size).toBe(2); return buildEngine({ @@ -1112,7 +1128,7 @@ describe('a stored signing secret that resolves to nothing (#8542)', () => { it('refuses to arm when the stored secret decrypts to nothing (historical row — the data-API road is closed by #8559)', async () => { const { engine, stores, driver } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook()])); + await seed18_0Rows(engine, metadataWith([declaredWebhook()])); const [row] = await engine.find('sys_webhook', { where: { name: 'crm_hook' } }); // This state USED to be reachable from the ordinary data API with no @@ -1147,7 +1163,7 @@ describe('a stored signing secret that resolves to nothing (#8542)', () => { it('refuses to arm when the column no longer holds a resolvable ref', async () => { const { engine, stores, driver } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook()])); + await seed18_0Rows(engine, metadataWith([declaredWebhook()])); const [row] = await engine.find('sys_webhook', { where: { name: 'crm_hook' } }); // Written BELOW the engine deliberately, because that is the only route @@ -1165,7 +1181,7 @@ describe('a stored signing secret that resolves to nothing (#8542)', () => { it('refuses when the row is deleted between the cache read and the dereference', async () => { const { engine } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([declaredWebhook()])); + await seed18_0Rows(engine, metadataWith([declaredWebhook()])); // The snapshot the enqueuer's refresh loop holds while it dereferences // each row's credentials, one at a time. @@ -1185,7 +1201,7 @@ describe('a stored signing secret that resolves to nothing (#8542)', () => { it('a webhook authored unsigned still arms and delivers — the refusal is not a blanket', async () => { const { engine } = await buildEngine(); - await bootstrapDeclaredWebhooks( + await seed18_0Rows( engine, metadataWith([declaredWebhook({ secret: undefined })]), ); @@ -1317,7 +1333,7 @@ describe('a stored header map that resolves to nothing (#8558)', () => { it('refuses to arm when the stored map decrypts to nothing (historical row — the data-API road is closed by #8559)', async () => { const { engine, stores, driver } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); + await seed18_0Rows(engine, metadataWith([headerBearingWebhook()])); const [row] = await engine.find('sys_webhook', { where: { name: 'crm_hook' } }); // The '' member of the unusable-spellings family below is special: it @@ -1363,7 +1379,7 @@ describe('a stored header map that resolves to nothing (#8558)', () => { 'refuses to arm when the stored value decrypts fine but is unusable: %s', async (_label, badValue) => { const { engine, stores } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); + await seed18_0Rows(engine, metadataWith([headerBearingWebhook()])); const [row] = await engine.find('sys_webhook', { where: { name: 'crm_hook' } }); // Written through the ORDINARY data API — no driver access, no SQL. @@ -1385,7 +1401,7 @@ describe('a stored header map that resolves to nothing (#8558)', () => { it('refuses to arm when the column no longer holds a resolvable ref', async () => { const { engine, stores, driver } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); + await seed18_0Rows(engine, metadataWith([headerBearingWebhook()])); const [row] = await engine.find('sys_webhook', { where: { name: 'crm_hook' } }); // Written BELOW the engine deliberately: that is the only route measured @@ -1407,7 +1423,7 @@ describe('a stored header map that resolves to nothing (#8558)', () => { it('refuses when the row is deleted between the cache read and the dereference', async () => { const { engine } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); + await seed18_0Rows(engine, metadataWith([headerBearingWebhook()])); // The snapshot the enqueuer's refresh loop holds while it dereferences // each row's credentials, one at a time. @@ -1455,7 +1471,7 @@ describe('a stored header map that resolves to nothing (#8558)', () => { it('a webhook authored with NO headers still arms and delivers — the refusal is not a blanket', async () => { const { engine } = await buildEngine(); - await bootstrapDeclaredWebhooks( + await seed18_0Rows( engine, metadataWith([declaredWebhook({ headers: undefined })]), ); @@ -1485,7 +1501,7 @@ describe('a stored header map that resolves to nothing (#8558)', () => { // written as "refuse whenever headers are stored": the healthy path has // to keep working end to end, credential entry included. const { engine, stores } = await buildEngine(); - await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); + await seed18_0Rows(engine, metadataWith([headerBearingWebhook()])); await expectHeadersGenuinelyStored(engine, stores); const { calls, rows, errors } = await driveOnce(engine); @@ -1634,10 +1650,10 @@ describe('the retired definition_json credential location — refused at the wri expect(calls[0].headers.Authorization).toBe(BEARER); }); - it('the declared-webhook seeder still materializes secret + headers through the bound gate', async () => { + it('an 18.0-seeded row\'s secret + headers still pass the bound gate into their encrypted columns', async () => { const { engine, stores } = await gatedEngine(); - const result = await bootstrapDeclaredWebhooks(engine, metadataWith([headerBearingWebhook()])); - expect(result).toMatchObject({ seeded: 1 }); + await seed18_0Rows(engine, metadataWith([headerBearingWebhook()])); + expect(stores.get('sys_webhook')!.size).toBe(1); expect(JSON.stringify(rowsAtRest(stores))).not.toContain(BEARER); }); }); diff --git a/packages/plugins/plugin-webhooks/src/webhook-secret.ts b/packages/plugins/plugin-webhooks/src/webhook-secret.ts index 1af645c91d8..f7e016e5882 100644 --- a/packages/plugins/plugin-webhooks/src/webhook-secret.ts +++ b/packages/plugins/plugin-webhooks/src/webhook-secret.ts @@ -202,11 +202,6 @@ type SecretResolvingEngine = IDataEngine & { onCryptoProviderChange?(listener: () => void): () => void; }; -/** True when this engine can dereference an encrypted field. */ -export function canResolveSecrets(engine: IDataEngine | undefined): boolean { - return typeof (engine as SecretResolvingEngine | undefined)?.resolveSecretField === 'function'; -} - /** * [#8022] Subscribe to the engine's crypto-provider registration. Returns an * unsubscribe function, or `undefined` when the engine has no such channel. diff --git a/packages/qa/dogfood/test/fixtures/webhook-materialization-fixture.ts b/packages/qa/dogfood/test/fixtures/webhook-materialization-fixture.ts index 93aa278522b..d61a8761110 100644 --- a/packages/qa/dogfood/test/fixtures/webhook-materialization-fixture.ts +++ b/packages/qa/dogfood/test/fixtures/webhook-materialization-fixture.ts @@ -3,17 +3,18 @@ // Webhook materialization fixture — the deterministic ADR-0054 proof for the // `webhook-materialization` high-risk class. // -// A webhook prop is `live` in the ledger because a stack-authored `webhooks:` -// entry is materialized into a dispatchable `sys_webhook` row on boot -// (#3461/#3489) — but "materialized" is not "reads the schema". The authored -// value crosses manifest-decomposition → the ObjectQL registry (type `webhook`) -// → `bootstrapDeclaredWebhooks` → `engine.insert('sys_webhook')`, and the break -// can live in any seam: the bridge was first gated BEHIND the realtime/messaging -// dispatch guard, so a boot without realtime silently materialized nothing (the -// exact silent no-op #3461 set out to kill). This fixture authors ONE webhook -// against ONE object, with ZERO dependence on an example app, so the proof can -// assert the runtime outcome: the sys_webhook row exists with the spec→runtime -// remap applied (`object`→`object_name`, `isActive`→`active`). +// A webhook prop is `live` in the ledger because the dispatcher reads it — but +// "read" is not "reads the schema". Since ADR-0131 D2 retired the boot seeder, +// the authored value crosses manifest-decomposition → the ObjectQL registry +// (type `webhook`) → plugin-webhooks' declared-webhook reader (bound as the +// `declaredWebhooks` service on the data engine alone) → the dispatcher's +// subscription, with NO `sys_webhook` row in between, and the break can live in +// any seam: the old bridge was first gated BEHIND the realtime/messaging +// dispatch guard, so a boot without realtime silently did nothing. This fixture +// authors ONE webhook against ONE object, with ZERO dependence on an example +// app, so the proof can assert the runtime outcome: no row, and a delivery with +// the spec→runtime remap applied (`object`→`object_name`, `isActive`→the +// switch). import { defineStack } from '@objectstack/spec'; import { ObjectSchema, Field } from '@objectstack/spec/data'; @@ -33,8 +34,8 @@ export const WmTask = ObjectSchema.create({ }); /** - * The stack-authored webhook. `object` (spec) must land as `object_name` - * (runtime col) and `isActive` as `active` — the remap the proof asserts. + * The stack-authored webhook. `object` (spec) must land as the dispatcher's + * `object_name` and `isActive` as its switch — the remap the proof asserts. */ export const wmTaskChanged = defineWebhook({ name: 'wm_task_changed', @@ -52,7 +53,7 @@ export const webhookFixtureStack = defineStack({ version: '0.0.0', type: 'app', name: 'Webhook Materialization Fixture', - description: 'Single-object app that authors one webhook to prove stack `webhooks:` entries materialize into dispatchable sys_webhook rows (ADR-0054).', + description: 'Single-object app that authors one webhook to prove stack `webhooks:` entries reach the dispatcher from the registry, with no sys_webhook row (ADR-0054, ADR-0131 D2).', }, objects: [WmTask], webhooks: [wmTaskChanged], diff --git a/packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts b/packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts index 2c9f272f60f..313b9fe89b2 100644 --- a/packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts +++ b/packages/qa/dogfood/test/upgrade-18x-harness.dogfood.test.ts @@ -98,6 +98,47 @@ describe(`[#24522] an 18.0 database (fixture from ${manifest.ref} @ ${manifest.s expect(titles).toContain(`${FIXTURE_NAMES.task} — edited`); }); + // [#24529, step 1] Declared webhooks are read from the registry, and the 18.0 + // seeder's rows stay until the ceremony. The fixture's mirror of + // `showcase_task_changed` was activated in 18.0, which stamped it + // `customized`, so it answers as itself and the declaration of the same name + // is withheld even while the declaration is switched on: one delivery per + // task change, from the row, never a second one from the declaration. + it('2b. the fixture\'s customized declared webhook dispatches exactly once on a task change', async () => { + expect(stack).toBeDefined(); + const adminTok = await stack!.signIn(FIXTURE_PEOPLE.admin.email, FIXTURE_PEOPLE.admin.password); + const hooks = rowsOf(await (await stack!.apiAs(adminTok, 'GET', `/data/sys_webhook?name=${FIXTURE_NAMES.webhook}`)).json()) + .filter((h) => h?.name === FIXTURE_NAMES.webhook); + expect(hooks, 'the 18.0 mirror row is still there').toHaveLength(1); + expect(hooks[0].customized, 'the fixture activated it in 18.0, which marks it customized').toBe(true); + // Switch the declaration ON as well (the showcase ships it inactive), so a + // reader that served both the row and the declaration WOULD deliver twice: + // without this the case could not fail on the rule it pins. + const on = await stack!.apiAs(adminTok, 'POST', `/security/_activation/webhook/${FIXTURE_NAMES.webhook}`, { enabled: true }); + expect(on.status, await on.clone().text()).toBe(200); + const deliveries = async () => rowsOf(await (await stack!.apiAs(adminTok, 'GET', '/data/sys_http_delivery?limit=500')).json()) + .filter((d) => d?.source === 'webhook'); + const before = new Set((await deliveries()).map((d) => String(d.id))); + + const projects = rowsOf(await (await stack!.apiAs(adminTok, 'GET', '/data/showcase_project')).json()); + const created = await stack!.apiAs(adminTok, 'POST', '/data/showcase_task', { + title: 'Upgrade harness task (HEAD)', + project: projects[0]?.id, + }); + expect(created.status, await created.clone().text()).toBeLessThan(300); + + // The enqueue runs off the write path: wait for the first new row, then a + // settle window in which a second (double) delivery would land too. + let fresh: any[] = []; + for (let i = 0; i < 50 && fresh.length === 0; i++) { + await new Promise((r) => setTimeout(r, 100)); + fresh = (await deliveries()).filter((d) => !before.has(String(d.id))); + } + await new Promise((r) => setTimeout(r, 1000)); + fresh = (await deliveries()).filter((d) => !before.has(String(d.id))); + expect(fresh.map((d) => d.ref_id), 'one delivery, from the customized row').toEqual([hooks[0].id]); + }); + it('3. every reshaped table keeps every 18.0 row, with its key columns, once HEAD has stopped', async () => { await stack?.stop(); stack = undefined; diff --git a/packages/qa/dogfood/test/webhook-materialization.dogfood.test.ts b/packages/qa/dogfood/test/webhook-materialization.dogfood.test.ts index cf513b26a3b..c90ace2509e 100644 --- a/packages/qa/dogfood/test/webhook-materialization.dogfood.test.ts +++ b/packages/qa/dogfood/test/webhook-materialization.dogfood.test.ts @@ -9,32 +9,56 @@ // (packages/spec/liveness/webhook.json); the spec liveness gate fails if this // tag is removed. See proof-registry.mts. // -// A webhook prop being `live` means the materializer + dispatcher READ it — -// necessary but not sufficient. This boots the real stack with the webhook + -// messaging plugins, authors a webhook via the app config's `webhooks:` array, -// and asserts the observable runtime outcome: a dispatchable `sys_webhook` row -// materialized on boot with the spec→runtime remap applied (`object`→ -// `object_name`, `isActive`→`active`). It also pins the #3461 integration bug: -// the fixture boots WITHOUT realtime, so a materializer re-gated behind the -// dispatch guard would materialize nothing and this proof would fail. +// A webhook prop being `live` means the dispatcher READS it — necessary but not +// sufficient. This boots the real stack with the webhook + messaging plugins, +// authors a webhook via the app config's `webhooks:` array, and asserts the +// observable runtime outcome of ADR-0131 D2 (the declared-webhook seeder is +// retired): NO `sys_webhook` row is written for the declaration, the plugin +// binds the declared-webhook reader (`declaredWebhooks`) over the registry the +// manifest decomposition filled, and a dispatcher reading through that service +// delivers a matching record event with the spec→runtime remap applied +// (`object`→`object_name`, `isActive`→the switch). The fixture still boots +// WITHOUT realtime: the reader is bound on the data engine alone, so a binding +// re-gated behind the dispatch prerequisites would bind nothing and this proof +// would fail. import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { bootStack, type VerifyStack } from '@objectstack/verify'; import { MessagingServicePlugin } from '@objectstack/service-messaging'; -import { WebhookOutboxPlugin } from '@objectstack/plugin-webhooks'; +import { AutoEnqueuer, WebhookOutboxPlugin } from '@objectstack/plugin-webhooks'; import { webhookFixtureStack } from './fixtures/webhook-materialization-fixture.js'; const SYSTEM_CTX = { isSystem: true, positions: [], permissions: [] }; -describe('objectstack verify WEBHOOK: authored webhook materializes into sys_webhook (#webhook-materialization)', () => { +/** The realtime surface the dispatcher subscribes through, driven by hand. */ +function handRealtime() { + const subs = new Map unknown; opts?: { object?: string } }>(); + let n = 0; + return { + async publish(event: any) { + for (const s of subs.values()) { + if (s.opts?.object && s.opts.object !== event.object) continue; + await s.handler(event); + } + }, + async subscribe(_channel: string, handler: (e: any) => unknown, opts?: { object?: string }) { + const id = `s-${++n}`; + subs.set(id, { handler, opts }); + return id; + }, + async unsubscribe(id: string) { + subs.delete(id); + }, + }; +} + +describe('objectstack verify WEBHOOK: a declared webhook is dispatched from the registry, never materialized (#webhook-materialization)', () => { let stack: VerifyStack; beforeAll(async () => { // `extraPlugins` mirrors the capability plugins `objectstack serve` mounts // for `requires: ['webhooks']` — messaging first (WebhookOutboxPlugin - // depends on it), then the webhook plugin. Deliberately NO realtime service: - // materialization must run on the data engine alone (bootDeclaredWebhooks), - // independent of the auto-enqueue dispatch prerequisites. + // depends on it), then the webhook plugin. Deliberately NO realtime service. stack = await bootStack(webhookFixtureStack, { extraPlugins: [new MessagingServicePlugin(), new WebhookOutboxPlugin()], }); @@ -44,27 +68,48 @@ describe('objectstack verify WEBHOOK: authored webhook materializes into sys_web await stack?.stop(); }); - it('materializes the stack-authored webhook into a sys_webhook row (object→object_name, isActive→active)', async () => { + it('writes no sys_webhook row for the declared webhook, and dispatches it with object→object_name applied', async () => { // eslint-disable-next-line @typescript-eslint/no-explicit-any const engine = (await stack.kernel.getServiceAsync('objectql')) as any; const rows = await engine.find('sys_webhook', { filter: { name: 'wm_task_changed' }, context: SYSTEM_CTX, }); + expect(rows, 'the retired seeder wrote a sys_webhook row for a declared webhook').toEqual([]); - expect(rows, 'authored webhook was NOT materialized into a sys_webhook row').toHaveLength(1); - const row = rows[0]; - - // The remaps that are the whole point of the bridge. - expect(row.object_name, 'spec `object` did not remap to runtime `object_name`').toBe('wm_task'); - expect(row.active, 'spec `isActive:true` did not remap to runtime `active`').toBeTruthy(); - - // Boot-seeded provenance (seed-not-clobber): a package row, not admin. - expect(row.managed_by).toBe('package'); + // The reader the plugin bound on the data engine alone — no realtime here. + // eslint-disable-next-line @typescript-eslint/no-explicit-any + const declaredWebhooks = (await stack.kernel.getServiceAsync('declaredWebhooks')) as any; + expect(declaredWebhooks, 'the declared-webhook reader was not bound').toBeTruthy(); + const declared = await declaredWebhooks.declaredWebhookNamed('wm_task_changed'); + expect(declared?.webhook.object, 'the registry does not hold the authored webhook').toBe('wm_task'); + expect(declared?.active, 'spec `isActive:true` did not become the switch').toBe(true); - // The full envelope is stashed for the dispatcher's advanced-config read. - const defn = JSON.parse(row.definition_json); - expect(defn.object).toBe('wm_task'); - expect(defn.triggers).toEqual(['create', 'update']); + // The dispatcher, reading through that same service, delivers a matching + // record event: the remap reached the subscription it keys on. + const realtime = handRealtime(); + const enqueued: any[] = []; + const enqueuer = new AutoEnqueuer(engine, realtime as any, async (input) => { + enqueued.push(input); + return 'd1'; + }, { refreshIntervalMs: 0, declaredWebhooks }); + await enqueuer.start(); + try { + await realtime.publish({ + type: 'data.record.created', + object: 'wm_task', + payload: { recordId: 't1' }, + timestamp: '2026-10-19T00:00:00.000Z', + }); + await new Promise((r) => setTimeout(r, 0)); + } finally { + await enqueuer.stop(); + } + expect(enqueued, 'spec `object` did not remap to the runtime subscription key').toHaveLength(1); + expect(enqueued[0]).toMatchObject({ + source: 'webhook', + refId: 'declared:wm_task_changed', + url: 'https://hooks.example/wm-task', + }); }); }); diff --git a/packages/runtime/src/domains/activation-gate.ts b/packages/runtime/src/domains/activation-gate.ts index 1c1caf71740..0831d4b0807 100644 --- a/packages/runtime/src/domains/activation-gate.ts +++ b/packages/runtime/src/domains/activation-gate.ts @@ -192,6 +192,19 @@ export const SHARING_RULE_ACTIVATION_SUBJECT: ActivationSubject = { 'organization authors under names of its own stay yours to create and switch.', }; +/** + * [ADR-0131 D6 regime C] The declared-webhook door's wording. A declared webhook + * is customised by a copy under another name plus the declared one switched + * off; an organization still authors its own webhooks, so the remedy names that + * half and the operator for the switch. + */ +export const WEBHOOK_ACTIVATION_SUBJECT: ActivationSubject = { + subject: 'a declared webhook', + remedy: + 'To switch a declared webhook on or off for this installation, ask your platform operator; webhooks your ' + + 'organization authors under names of its own stay yours to create and switch.', +}; + /** * The activation body every `_activation` door reads: `{ enabled?: boolean }`, * unknown keys refused, `enabled` defaulting to `true`. diff --git a/packages/runtime/src/domains/catalog-activation-door.test.ts b/packages/runtime/src/domains/catalog-activation-door.test.ts index f09f3d4191f..f04444a9eac 100644 --- a/packages/runtime/src/domains/catalog-activation-door.test.ts +++ b/packages/runtime/src/domains/catalog-activation-door.test.ts @@ -653,3 +653,76 @@ describe('the catalog activation door — a declared sharing rule', () => { expect(h.tables[LEDGER][0].package_id ?? null).toBeNull(); }); }); + +// [ADR-0131 D2/D6 regime C] A DECLARED webhook is switched through the same door +// and the same ledger: `:type` `webhook`, the name resolved by plugin-webhooks' +// declared-webhook reader (`declaredWebhooks`, `declaredWebhookNamed`), which the +// dispatcher reads its switch through. The effect on dispatch is pinned beside +// that reader (`declared-webhooks.test.ts`). +describe('the catalog activation door — a declared webhook', () => { + const HOOK = 'crm_contact_changed'; + const declaredWebhooks = (declared: Record = { [HOOK]: { packageId: 'crm' } }) => ({ + declaredWebhooks: { + async declaredWebhookNamed(name: string) { + return declared[name] + ? { webhook: { name }, active: true, ...declared[name] } + : null; + }, + }, + }); + + it('`single`: a tenant admin holding `manage_metadata` switches it — one ledger row of type webhook, its package recorded', async () => { + const h = boot('single', {}, declaredWebhooks()); + const res = await flip(h, TENANT_ADMIN(), `/_activation/webhook/${HOOK}`, { enabled: true }); + expect(statusOf(res)).toBe(200); + expect(res.response?.body?.data).toEqual({ type: 'webhook', name: HOOK, enabled: true }); + expect(h.tables[LEDGER]).toEqual([ + expect.objectContaining({ metadata_type: 'webhook', name: HOOK, package_id: 'crm', active: true }), + ]); + }); + + for (const posture of ['group', 'isolated'] as const) { + it(`\`${posture}\`: the switch is the platform operator's — a tenant admin is refused 403 and nothing is written`, async () => { + const h = boot(posture, {}, declaredWebhooks()); + const refused = await flip(h, TENANT_ADMIN(), `/_activation/webhook/${HOOK}`); + expect(statusOf(refused)).toBe(403); + expect(codeOf(refused)).toBe('PERMISSION_DENIED'); + expect(messageOf(refused)).toContain('a declared webhook'); + expect(h.writes).toEqual([]); + + expect(statusOf(await flip(h, PLATFORM_OPERATOR(), `/_activation/webhook/${HOOK}`))).toBe(200); + expect(h.writes).toHaveLength(1); + }); + } + + it('a name no declaration holds is 404 and writes nothing — an organization\'s own webhook has its own switch', async () => { + const h = boot('single', {}, declaredWebhooks()); + const res = await flip(h, TENANT_ADMIN(), '/_activation/webhook/our_own_hook'); + expect(statusOf(res)).toBe(404); + expect(h.writes).toEqual([]); + }); + + it('no declared-webhook reader is 503 SERVICE_UNAVAILABLE, never a write for an unconfirmed name', async () => { + const h = boot('single'); + const res = await flip(h, TENANT_ADMIN(), `/_activation/webhook/${HOOK}`); + expect(statusOf(res)).toBe(503); + expect(codeOf(res)).toBe('SERVICE_UNAVAILABLE'); + expect(h.writes).toEqual([]); + }); + + it('a declared-webhook read that fails is 503, never a write', async () => { + const h = boot('single', {}, { + declaredWebhooks: { async declaredWebhookNamed() { throw new Error('registry down'); } }, + }); + const res = await flip(h, TENANT_ADMIN(), `/_activation/webhook/${HOOK}`); + expect(statusOf(res)).toBe(503); + expect(codeOf(res)).toBe('SERVICE_UNAVAILABLE'); + expect(h.writes).toEqual([]); + }); + + it('a declaration no package names is switched too, its row carrying no package', async () => { + const h = boot('single', {}, declaredWebhooks({ [HOOK]: {} })); + expect(statusOf(await flip(h, TENANT_ADMIN(), `/_activation/webhook/${HOOK}`))).toBe(200); + expect(h.tables[LEDGER][0].package_id ?? null).toBeNull(); + }); +}); diff --git a/packages/runtime/src/domains/catalog-activation.ts b/packages/runtime/src/domains/catalog-activation.ts index ffa07d29c56..6a816ce10cd 100644 --- a/packages/runtime/src/domains/catalog-activation.ts +++ b/packages/runtime/src/domains/catalog-activation.ts @@ -6,7 +6,10 @@ * `{ enabled?: boolean }`, `:type` one of `position` / `permission`, and — since * ADR-0131 D6 charts declared sharing rules under regime C — `sharing_rule`, a * rule a package declares, read from the sharing-rule service (`sharingRules`) - * rather than the security catalog. + * rather than the security catalog; and `webhook`, a webhook a stack declares + * (`defineStack({ webhooks })`), read from the declared-webhook reader + * (`declaredWebhooks`, plugin-webhooks) under the same regime (ADR-0131 D2/D6, + * the ruling that retired the declared-webhook seeder). * * ## Why it exists * @@ -63,6 +66,7 @@ import { POSITION_ACTIVATION_SUBJECT, PERMISSION_SET_ACTIVATION_SUBJECT, SHARING_RULE_ACTIVATION_SUBJECT, + WEBHOOK_ACTIVATION_SUBJECT, type ActivationSubject, } from './activation-gate.js'; import type { HttpProtocolContext, HttpDispatcherResult } from '../http-dispatcher.js'; @@ -76,12 +80,15 @@ export const CATALOG_ACTIVATION_SEGMENT = '_activation'; * types the resolver reads from the ledger; `capability` is code-only on the * regime and has no switch. */ -const CATALOG_ACTIVATION_SUBJECTS: Readonly> = { +const CATALOG_ACTIVATION_SUBJECTS: Readonly> = { position: POSITION_ACTIVATION_SUBJECT, permission: PERMISSION_SET_ACTIVATION_SUBJECT, // [ADR-0131 D6 regime C] A declared sharing rule — read from the sharing-rule // service, which serves no declaration this ledger switches off. sharing_rule: SHARING_RULE_ACTIVATION_SUBJECT, + // [ADR-0131 D2/D6 regime C] A declared webhook — read from the declared-webhook + // reader, which the dispatcher reads its switch through. + webhook: WEBHOOK_ACTIVATION_SUBJECT, }; type CatalogActivationType = keyof typeof CATALOG_ACTIVATION_SUBJECTS; @@ -95,6 +102,7 @@ const NOUN: Readonly> = { position: 'Position', permission: 'Permission set', sharing_rule: 'Declared sharing rule', + webhook: 'Declared webhook', }; /** @@ -113,6 +121,60 @@ function declaredSharingRuleReaderOf(service: unknown): DeclaredSharingRuleReade return typeof candidate?.declaredRuleNamed === 'function' ? (candidate as DeclaredSharingRuleReader) : undefined; } +/** + * [ADR-0131 D2/D6 regime C] The declaration a `webhook` row addresses, as the + * declared-webhook reader (`declaredWebhooks`, plugin-webhooks) reads it: + * declared in the code, whether or not the ledger has it switched off. A + * webhook an organization authored is a `sys_webhook` row with a switch of its + * own (`active`) and is never addressed here. + */ +interface DeclaredWebhookReader { + declaredWebhookNamed(name: string): Promise<{ webhook: { name: string }; packageId?: string } | null>; +} + +function declaredWebhookReaderOf(service: unknown): DeclaredWebhookReader | undefined { + const candidate = service as Partial | null | undefined; + return typeof candidate?.declaredWebhookNamed === 'function' ? (candidate as DeclaredWebhookReader) : undefined; +} + +/** + * The service-backed types — declarations a plugin owns, not catalog entries — + * each with how its declaration is looked up and how it is named in a refusal. + */ +const DECLARATION_LOOKUPS: Readonly | undefined; +}>> = { + sharing_rule: { + service: 'sharingRules', + noun: 'sharing rule', + serviceNoun: 'sharing-rule service', + authoredRemedy: 'A rule your organization authored is switched by its own `active` field.', + lookup: (service, name) => declaredSharingRuleReaderOf(service)?.declaredRuleNamed(name), + }, + webhook: { + service: 'declaredWebhooks', + noun: 'webhook', + serviceNoun: 'declared-webhook reader', + authoredRemedy: 'A webhook your organization authored is switched by its own `active` field.', + lookup: (service, name) => { + const reader = declaredWebhookReaderOf(service); + return reader + ? reader.declaredWebhookNamed(name).then((d) => (d + ? { name: d.webhook.name, ...(d.packageId !== undefined ? { packageId: d.packageId } : {}) } + : null)) + : undefined; + }, + }, +}; + +function isDeclarationType(type: CatalogActivationType): type is keyof typeof DECLARATION_LOOKUPS { + return Object.prototype.hasOwnProperty.call(DECLARATION_LOOKUPS, type); +} + /** * Does this request belong to the door? Every `POST` under `_activation`, with * no upper bound on depth, so the gate is exactly as wide as the arm: a @@ -223,46 +285,55 @@ export async function handleCatalogActivationWrite( return { handled: true, response: deps.error('Data engine not available', 503) }; } let entry: { name: string; packageId?: string } | undefined; - if (type === 'sharing_rule') { - const rules = declaredSharingRuleReaderOf(await deps.resolveService(context, 'sharingRules')); - if (!rules) { + if (isDeclarationType(type)) { + const declaration = DECLARATION_LOOKUPS[type]; + const service = await deps.resolveService(context, declaration.service); + let pending: Promise<{ name: string; packageId?: string } | null> | undefined; + try { + pending = declaration.lookup(service, name); + } catch (err) { + pending = Promise.reject(err); // a reader that throws before its promise is still a failed read + } + if (!pending) { return { handled: true, response: deps.error( - `Cannot verify the declaration of sharing rule '${name}' — no sharing-rule service is bound to this ` + - `deployment. Refusing rather than writing an activation row for a name nobody can confirm exists.`, + `Cannot verify the declaration of ${declaration.noun} '${name}' — no ${declaration.serviceNoun} is ` + + `bound to this deployment. Refusing rather than writing an activation row for a name nobody can ` + + `confirm exists.`, 503, { code: 'SERVICE_UNAVAILABLE' }, ), }; } try { - entry = (await rules.declaredRuleNamed(name)) ?? undefined; + entry = (await pending) ?? undefined; } catch (err: any) { return { handled: true, response: deps.error( - `Cannot verify the declaration of sharing rule '${name}' — the declared rules could not be read ` + - `(${err?.message ?? String(err)}). Refusing rather than writing an activation row for a name nobody ` + - `can confirm exists.`, + `Cannot verify the declaration of ${declaration.noun} '${name}' — the declared ${declaration.noun}s ` + + `could not be read (${err?.message ?? String(err)}). Refusing rather than writing an activation row ` + + `for a name nobody can confirm exists.`, 503, { code: 'SERVICE_UNAVAILABLE' }, ), }; } if (!entry) { + const noun = declaration.noun.charAt(0).toUpperCase() + declaration.noun.slice(1); return { handled: true, response: deps.error( - `Sharing rule '${name}' is not declared by the application — there is nothing to switch ` + - `${enabled ? 'on' : 'off'}. A rule your organization authored is switched by its own \`active\` field.`, + `${noun} '${name}' is not declared by the application — there is nothing to switch ` + + `${enabled ? 'on' : 'off'}. ${declaration.authoredRemedy}`, 404, ), }; } } - const reader = type === 'sharing_rule' ? undefined : securityCatalogReaderOf(ql); - if (type !== 'sharing_rule' && !reader) { + const reader = isDeclarationType(type) ? undefined : securityCatalogReaderOf(ql); + if (!isDeclarationType(type) && !reader) { return { handled: true, response: deps.error( diff --git a/packages/runtime/src/domains/security.ts b/packages/runtime/src/domains/security.ts index 0a2661b6fbc..b5624d7368a 100644 --- a/packages/runtime/src/domains/security.ts +++ b/packages/runtime/src/domains/security.ts @@ -19,8 +19,9 @@ * POST /security/suggested-bindings/:id/confirm → accept it for the deployment * POST /security/suggested-bindings/:id/dismiss → decline it, or revoke an acceptance * POST /security/_activation/:type/:name → switch a position (`position`), a - * permission set (`permission`) or a declared - * sharing rule (`sharing_rule`) on/off + * permission set (`permission`), a declared + * sharing rule (`sharing_rule`) or a declared + * webhook (`webhook`) on/off * for this deployment (`./catalog-activation.ts`) * POST /security/permission-sets/:name/clone → copy a permission set's whole definition * under a new name (`./permission-set-clone.ts`) diff --git a/packages/runtime/src/route-ledger.ts b/packages/runtime/src/route-ledger.ts index b5f3b699d87..46d8d419a16 100644 --- a/packages/runtime/src/route-ledger.ts +++ b/packages/runtime/src/route-ledger.ts @@ -378,7 +378,8 @@ export const ROUTE_LEDGER: readonly RouteLedgerEntry[] = [ { route: 'POST /security/_activation/:type/:name', domain: '/security', disposition: 'server-only', note: 'ADR-0126 §3 regime C, as ADR-0131 D6 amends it — switch ONE position (`:type` = `position`), permission set ' + '(`:type` = `permission`) or declared sharing rule (`:type` = `sharing_rule`, resolved by the `sharingRules` service; ' - + 'its grants reconcile on the ledger write) on/off for this deployment. Writes one `sys_metadata_activation` row through the engine and ' + + 'its grants reconcile on the ledger write) or declared webhook (`:type` = `webhook`, resolved by the `declaredWebhooks` ' + + 'reader; the webhook dispatcher refreshes its subscriptions on the ledger write) on/off for this deployment. Writes one `sys_metadata_activation` row through the engine and ' + 'nothing else (⛔ no definition write, ⛔ no catalog row `active`: ADR-0131 D3 moved the switch into the ledger, and ' + '`resolveUserAuthzGrants` reads only the ledger for these two types). Body `{ enabled?: boolean }`, the same reader as ' + 'the action door. The same two authority tiers as `POST /automation/:name/toggle` and `POST /actions/_activation/:object/:action`: ' diff --git a/packages/spec/liveness/connector.json b/packages/spec/liveness/connector.json index 78b969da998..6d549f301d4 100644 --- a/packages/spec/liveness/connector.json +++ b/packages/spec/liveness/connector.json @@ -135,7 +135,7 @@ "webhooks": { "status": "dead", "verifiedAt": "2026-09-27", - "note": "RETIRED 2026-09-27 (ADR-0049 enforce-or-remove; the connector resilience family) — the `retire` arm of the decision this row asked for. Tombstoned with `retiredKey` (non-strict schema, ADR-0104), carried by `DeclarativeConnectorEntrySchema` too; the nested shape left whole (`WebhookConfig`, `WebhookEvent`, `WebhookSignatureAlgorithm`). Stripped — never moved — from sources and stored rows by the protocol-18 conversion `connector-resilience-keys-removed`: moving a nested webhook into the top-level collection would START deliveries it never made, which is the author's call (the D3 entry `connector-resilience-keys-retired`). The row stays because `retiredKey` keeps the key in the walked shape (the `rls.priority` precedent); it is a LEAF now, so its `connector/webhooks` row in scripts/liveness/undrilled-containers.baseline.json was deleted (the gate reported it stale once the container stopped being one). Why, unchanged from the prior verdict: a connector's NESTED webhook array is not the collection the dispatcher reads — `bootstrapDeclaredWebhooks` (packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts) materializes `sys_webhook` rows only from metadata ITEMS of type `webhook`, which the decomposition registers from the TOP-LEVEL `webhooks:` stack collection, and `connectors: 'connector'` registers a connector entry WHOLE; `bootstrap-declared-webhooks.connector-nested.test.ts` pins that the nested array is not hoisted. Declare a webhook in the top-level `webhooks:` collection. The tombstone is packages/spec/src/integration/connector.zod.ts#ConnectorSchema." + "note": "RETIRED 2026-09-27 (ADR-0049 enforce-or-remove; the connector resilience family) — the `retire` arm of the decision this row asked for. Tombstoned with `retiredKey` (non-strict schema, ADR-0104), carried by `DeclarativeConnectorEntrySchema` too; the nested shape left whole (`WebhookConfig`, `WebhookEvent`, `WebhookSignatureAlgorithm`). Stripped — never moved — from sources and stored rows by the protocol-18 conversion `connector-resilience-keys-removed`: moving a nested webhook into the top-level collection would START deliveries it never made, which is the author's call (the D3 entry `connector-resilience-keys-retired`). The row stays because `retiredKey` keeps the key in the walked shape (the `rls.priority` precedent); it is a LEAF now, so its `connector/webhooks` row in scripts/liveness/undrilled-containers.baseline.json was deleted (the gate reported it stale once the container stopped being one). Why, unchanged from the prior verdict: a connector's NESTED webhook array is not the collection the dispatcher reads — the declared-webhook reader (packages/plugins/plugin-webhooks/src/declared-webhooks.ts, which replaced the retired boot seeder `bootstrapDeclaredWebhooks`) reads only metadata ITEMS of type `webhook`, which the decomposition registers from the TOP-LEVEL `webhooks:` stack collection, and `connectors: 'connector'` registers a connector entry WHOLE; `bootstrap-declared-webhooks.connector-nested.test.ts` pins that the nested array is not hoisted. Declare a webhook in the top-level `webhooks:` collection. The tombstone is packages/spec/src/integration/connector.zod.ts#ConnectorSchema." }, "rateLimitConfig": { "status": "dead", diff --git a/packages/spec/liveness/state-counts/webhook.md b/packages/spec/liveness/state-counts/webhook.md index d052b4768eb..1cf9e8785ee 100644 --- a/packages/spec/liveness/state-counts/webhook.md +++ b/packages/spec/liveness/state-counts/webhook.md @@ -12,4 +12,4 @@ committed anywhere: `check:liveness` sums the shards when it reads them. | Type | live | exp | elsewhere | dead | planned | classified | |---|---|---|---|---|---|---| -| `webhook` | 19 | 0 | 0 | 0 | 0 | 19 | +| `webhook` | 17 | 0 | 0 | 2 | 0 | 19 | diff --git a/packages/spec/liveness/webhook.json b/packages/spec/liveness/webhook.json index e867d8cf543..44531971175 100644 --- a/packages/spec/liveness/webhook.json +++ b/packages/spec/liveness/webhook.json @@ -1,73 +1,73 @@ { "type": "webhook", - "_note": "WebhookSchema (outbound webhook — packages/spec/src/automation/webhook.zod.ts). Governed via a spec-only schema override in the gate (SPEC_ONLY_SCHEMAS): webhook is still NOT a registered metadata type (absent from kernel/metadata-type-schemas.ts) — registering it would turn on Studio webhook CRUD + saveMetaItem overlay + create-seeds; that reassessment is tracked in #3490 and deliberately deferred. TWO THINGS CLOSED THE OLD 'entire surface is dead' classification: (1) #3494 PRUNED the aspirational dead props outright — body / payloadFields / includeSession / retryPolicy / tags / authentication are gone from the schema; (2) the #3461 materializer bridge (PR #3489) makes every REMAINING prop live. `bootstrapDeclaredWebhooks` (plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts) materializes each stack/connector-authored webhook into a `sys_webhook` DATA row on boot — WebhookSchema.parse (:114) → mapWebhookToRow (:191): `object`→`object_name` (:195), `isActive`→`active` (:202), same-named `name`/`label`/`triggers`/`url`/`method`/`description`, `secret`→the encrypted `signing_secret` column (#7799), `headers`→the encrypted `headers_secret` column (#7986), and the REST of the envelope → `definition_json`. The dispatcher (AutoEnqueuer) reads those rows (auto-enqueuer.ts:175 `where:{active:true}`) and fans out on data.record.* events (plus the aggregate data.records.* a predicate write publishes, dispatched under the opt-in bulk_update/bulk_delete triggers — #4639), reading `object_name`/`name`/`url`/`method`/`triggers` off the row, `timeoutMs` back out of `definition_json`, and BOTH credentials through `engine.resolveSecretField()` against their encrypted columns — the signing key from `signing_secret` (#7799) and the custom headers from `headers_secret` (#7986). So all 11 remaining props are LIVE; there is no dead/experimental prop left, so nothing carries an authorWarn (the old per-webhook `url` heads-up is gone — authoring is no longer a no-op). Seed-not-clobber: an admin-edited row (`customized`) is never re-seeded (bootstrap-declared-webhooks.ts:143). The `object` prop carries the ADR-0054 runtime proof for the whole materialization pipeline (bound high-risk class `webhook-materialization`). Field-level line refs: materializer bootstrap-declared-webhooks.ts, runtime object plugins/plugin-webhooks/src/sys-webhook.object.ts, dispatcher auto-enqueuer.ts. 2026-08-28 — WHOLE FILE RE-ANCHORED (commit 9ee2dcfbd, adoption of the #12516 symbol-anchor grammar), and this file was the batch's worst starting position by a distance. Every citation in it was written as a BARE FILENAME with a line (`bootstrap-declared-webhooks.ts:193`, `auto-enqueuer.ts:266`, `sys-webhook.object.ts:160`), which `PATH_RE` cannot parse: a citation must be repo-rooted to be a citation at all. So all eleven `live` entries scanned as ZERO local paths and ZERO line citations — no existence check, no line bound, no key-mention anchoring, nothing. The gate was not passing this file; it was never asked about it, and the ledger's own counters could not say so. Combined with the second fact — no entry carried `verifiedAt`, which the review that accepted the re-anchoring sorts as OLDEST — these rows were unfalsifiable and unclocked at once. Re-closed by hand against c459da6bc: every consumer read on the current tree, no line carried forward. What the re-closure found: the claims were all TRUE and the pointers were all merely unresolvable — the opposite of `object.json` in batch 1, where the pointers resolved and the claims were wrong. Both are the same defect at different ends. Two prose repairs fell out of it: the parenthetical line numbers are gone in favour of the symbols that own each read, and the dispatcher's credential legs are named as `attachSecret` / `attachHeaders`, which REFUSE a subscription whose credential cannot be recovered rather than delivering without it — a fail-closed behaviour the old prose did not record.", + "_note": "WebhookSchema (outbound webhook — packages/spec/src/automation/webhook.zod.ts). Governed via a spec-only schema override in the gate (SPEC_ONLY_SCHEMAS): webhook is still NOT a registered metadata type (absent from kernel/metadata-type-schemas.ts) — registering it would turn on Studio webhook CRUD + saveMetaItem overlay + create-seeds; that reassessment is tracked in #3490 and deliberately deferred. TWO THINGS CLOSED THE OLD 'entire surface is dead' classification: (1) #3494 PRUNED the aspirational dead props outright — body / payloadFields / includeSession / retryPolicy / tags / authentication are gone from the schema; (2) the #3461 materializer bridge (PR #3489) makes every REMAINING prop live. `bootstrapDeclaredWebhooks` (plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts) materializes each stack/connector-authored webhook into a `sys_webhook` DATA row on boot — WebhookSchema.parse (:114) → mapWebhookToRow (:191): `object`→`object_name` (:195), `isActive`→`active` (:202), same-named `name`/`label`/`triggers`/`url`/`method`/`description`, `secret`→the encrypted `signing_secret` column (#7799), `headers`→the encrypted `headers_secret` column (#7986), and the REST of the envelope → `definition_json`. The dispatcher (AutoEnqueuer) reads those rows (auto-enqueuer.ts:175 `where:{active:true}`) and fans out on data.record.* events (plus the aggregate data.records.* a predicate write publishes, dispatched under the opt-in bulk_update/bulk_delete triggers — #4639), reading `object_name`/`name`/`url`/`method`/`triggers` off the row, `timeoutMs` back out of `definition_json`, and BOTH credentials through `engine.resolveSecretField()` against their encrypted columns — the signing key from `signing_secret` (#7799) and the custom headers from `headers_secret` (#7986). So all 11 remaining props are LIVE; there is no dead/experimental prop left, so nothing carries an authorWarn (the old per-webhook `url` heads-up is gone — authoring is no longer a no-op). Seed-not-clobber: an admin-edited row (`customized`) is never re-seeded (bootstrap-declared-webhooks.ts:143). The `object` prop carries the ADR-0054 runtime proof for the whole materialization pipeline (bound high-risk class `webhook-materialization`). Field-level line refs: materializer bootstrap-declared-webhooks.ts, runtime object plugins/plugin-webhooks/src/sys-webhook.object.ts, dispatcher auto-enqueuer.ts. 2026-08-28 — WHOLE FILE RE-ANCHORED (commit 9ee2dcfbd, adoption of the #12516 symbol-anchor grammar), and this file was the batch's worst starting position by a distance. Every citation in it was written as a BARE FILENAME with a line (`bootstrap-declared-webhooks.ts:193`, `auto-enqueuer.ts:266`, `sys-webhook.object.ts:160`), which `PATH_RE` cannot parse: a citation must be repo-rooted to be a citation at all. So all eleven `live` entries scanned as ZERO local paths and ZERO line citations — no existence check, no line bound, no key-mention anchoring, nothing. The gate was not passing this file; it was never asked about it, and the ledger's own counters could not say so. Combined with the second fact — no entry carried `verifiedAt`, which the review that accepted the re-anchoring sorts as OLDEST — these rows were unfalsifiable and unclocked at once. Re-closed by hand against c459da6bc: every consumer read on the current tree, no line carried forward. What the re-closure found: the claims were all TRUE and the pointers were all merely unresolvable — the opposite of `object.json` in batch 1, where the pointers resolved and the claims were wrong. Both are the same defect at different ends. Two prose repairs fell out of it: the parenthetical line numbers are gone in favour of the symbols that own each read, and the dispatcher's credential legs are named as `attachSecret` / `attachHeaders`, which REFUSE a subscription whose credential cannot be recovered rather than delivering without it — a fail-closed behaviour the old prose did not record. 2026-10-19 — THE MATERIALIZER RETIRED (ADR-0131 D2, ruling Q2 A on the declared-webhook seeder): `bootstrapDeclaredWebhooks` no longer exists, so no declared webhook becomes a `sys_webhook` DATA row. The dispatcher reads the declaration from the registry instead (`packages/plugins/plugin-webhooks/src/declared-webhooks.ts`, projected per cache refresh by `auto-enqueuer.ts#projectDeclared`), and switches it through the activation ledger (regime C). Every delivery-bearing prop stays LIVE on the new reader and is re-anchored there; `label` and `description` lost their only reader (the Setup row) and are DEAD, owed enforce-or-remove. The `object` proof now boots the stack and asserts the declared webhook reaches the dispatcher with `object`→`object_name` applied and NO row written.", "props": { "name": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`name: wh.name` — materialized to sys_webhook.name, #3489); packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#bootstrapDeclaredWebhooks (the same value is the idempotency key: `where: { name: wh.name }` decides seed vs patch vs skip); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`name: (row.name as string) ?? row.id` — the dispatcher's dedup/log key)", - "note": "Was dead pre-bridge (admin-CRUD-only). Now fed by authoring. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "evidence": "packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource (`WebhookSchema.parse` of each registry item; the first registration of a name holds it); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared (`name: wh.name`, and `declaredWebhookRefId(wh.name)` as the subscription id and every delivery's `ref_id`); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#resolveDeclared (the name finds the `sys_webhook` row holding it, which an 18.0 database may still carry, and `classifyDeclaredNameHolder` decides which answers); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`name: (row.name as string) ?? row.id` — the dispatcher's dedup/log key)", + "note": "Was dead pre-bridge (admin-CRUD-only). Now fed by authoring. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "label": { - "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`label: wh.label ?? wh.name` — materialized to sys_webhook.label, #3489); packages/plugins/plugin-webhooks/src/sys-webhook.object.ts#SysWebhook (every one of the object's four listViews leads with the `label` column)", - "note": "Display-only — surfaced in Studio, never sent on the wire. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "status": "dead", + "note": "Display-only — surfaced in Studio, never sent on the wire. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: DEAD since ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, which was its only reader — it copied this value into the declared webhook's `sys_webhook` row, where the Setup list showed it. A declared webhook has no row any more, and nothing displays a declaration: the dispatcher reads only what it delivers on (`auto-enqueuer.ts#projectDeclared`), the activation door only the name. Display-only, so the loss is a label nobody sees, never a delivery that changes. Enforce-or-remove (ADR-0049) is owed: a Setup surface that lists declared webhooks (the declared-sharing-rule Disable and Clone surface is the pattern) makes it live again, otherwise it retires.", + "verifiedAt": "2026-10-19", + "authorHint": "A declared webhook's `label` is shown nowhere: declared webhooks are read from the registry and have no Setup row. Remove it — it changes no delivery. A webhook your organization authors in Setup keeps its label." }, "object": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`object_name: wh.object ?? null` — the spec→runtime remap this entry's proof exists for); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`objectName: row.object_name ? String(row.object_name) : undefined` — what the per-object subscription keys on)", + "evidence": "packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared (`object_name: wh.object ?? null` — the spec→runtime remap this entry's proof exists for, now made in memory rather than into a row); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`objectName: row.object_name ? String(row.object_name) : undefined` — what the per-object subscription keys on)", "proof": "packages/qa/dogfood/test/webhook-materialization.dogfood.test.ts#webhook-materialization", - "note": "ADR-0054 high-risk class (webhook-materialization): the spec `object`→runtime `object_name` remap is the representative check for the whole authoring→dispatcher pipeline that crosses manifest-decomposition → materializer → sys_webhook row → dispatcher subscription — the exact multi-layer seam that broke in integration (the materializer was first gated behind the realtime guard) despite green unit tests. The proof authors a stack `webhooks:` entry against the real in-process stack and asserts the sys_webhook row materialized with object_name set. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "note": "ADR-0054 high-risk class (webhook-materialization): the spec `object`→runtime `object_name` remap is the representative check for the whole authoring→dispatcher pipeline that crosses manifest-decomposition → materializer → sys_webhook row → dispatcher subscription — the exact multi-layer seam that broke in integration (the materializer was first gated behind the realtime guard) despite green unit tests. The proof authors a stack `webhooks:` entry against the real in-process stack and asserts the sys_webhook row materialized with object_name set. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "triggers": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`triggers: wh.triggers ?? []`); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (accepts array / comma-string / JSON-string shapes, maps them onto the dispatchable set, warns by name on values the engine never emits (#3196), and refuses a subscription whose set is empty under rule `webhook/without-triggers`)", - "note": "Was dead pre-bridge. Salvaged 1:1. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "evidence": "packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared (`triggers: wh.triggers ?? []`); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (accepts array / comma-string / JSON-string shapes, maps them onto the dispatchable set, warns by name on values the engine never emits (#3196), and refuses a subscription whose set is empty under rule `webhook/without-triggers`)", + "note": "Was dead pre-bridge. Salvaged 1:1. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "url": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`url: wh.url`); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`if (!row?.id || !row?.url) return null` — a row without one is not a subscription at all — then `url: String(row.url)` for delivery)", - "note": "Was dead pre-bridge and carried the per-webhook authorWarn; the bridge (#3489) makes authoring live, so the warning is removed. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "evidence": "packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared (`url: wh.url`); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`if (!row?.id || !row?.url) return null` — a subscription without one is not a subscription at all — then `url: String(row.url)` for delivery)", + "note": "Was dead pre-bridge and carried the per-webhook authorWarn; the bridge (#3489) makes authoring live, so the warning is removed. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "method": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`method: String(wh.method ?? 'POST').toLowerCase()` — stored lowercase to match the object's select option values); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`String(row.method ?? defn.method ?? 'POST').toUpperCase()` — upper-cased back for the wire, so both spellings survive the select change)", - "note": "Was dead pre-bridge. Salvaged 1:1. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "evidence": "packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared (`method: wh.method`, the parsed default `POST` included); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`String(row.method ?? defn.method ?? 'POST').toUpperCase()` — the canonical wire method)", + "note": "Was dead pre-bridge. Salvaged 1:1. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "headers": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (the envelope it returns is the authored one MINUS its two credential passengers — `splitWebhookHeaders` lifts this map out before `definition_json` is serialized, #7986); packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#bootstrapDeclaredWebhooks (`headersPatch` / the `WEBHOOK_HEADERS_FIELD` insert write the map into the encrypted `headers_secret` column); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#attachHeaders (recovers and re-parses it per cache refresh, and REFUSES the subscription when recovery fails rather than delivering header-less)", - "note": "Was dead pre-bridge (definition_json only reachable via hand-authored rows), then materialized from authoring into definition_json as CLEARTEXT — recoverable over the ordinary data API, which is #7986: the same column and the same absent `enable` block as #7799's signing secret, one key of the blob later. It no longer rides in that blob; the authored map itself is unchanged. The WHOLE map moves because only some entries are credentials and the platform cannot tell which. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "evidence": "packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared (`splitWebhookHeaders` lifts the authored map off the declaration onto the cached subscription — held in memory with the code it came from, never written to `sys_webhook` or `sys_secret`); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#attachHeaders (the same map for an organization's own row: recovered from the encrypted `headers_secret` column per cache refresh, the subscription REFUSED when recovery fails rather than delivering header-less, #7986)", + "note": "Was dead pre-bridge (definition_json only reachable via hand-authored rows), then materialized from authoring into definition_json as CLEARTEXT — recoverable over the ordinary data API, which is #7986: the same column and the same absent `enable` block as #7799's signing secret, one key of the blob later. It no longer rides in that blob; the authored map itself is unchanged. The WHOLE map moves because only some entries are credentials and the platform cannot tell which. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "timeoutMs": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`definition_json: JSON.stringify(envelope)` — this key rides the envelope rather than a column of its own); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`timeoutMs: defn.timeoutMs`, read back out of that JSON onto the cached subscription and passed to the outbox on both enqueue paths); packages/services/service-messaging/src/http-outbox.ts#assertEnqueueDeliverable (the outbox side of the same field)", - "note": "Was dead pre-bridge (definition_json only reachable via hand-authored rows). Now materialized from authoring. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "evidence": "packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared (`definition_json: JSON.stringify({ timeoutMs: wh.timeoutMs })`, the parsed default 30000 included); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#parseRow (`timeoutMs: defn.timeoutMs`, read back onto the cached subscription and passed to the outbox on both enqueue paths); packages/services/service-messaging/src/http-outbox.ts#assertEnqueueDeliverable (the outbox side of the same field)", + "note": "Was dead pre-bridge (definition_json only reachable via hand-authored rows). Now materialized from authoring. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "secret": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`splitWebhookSecret` lifts the key out of the envelope before serialization, #7799); packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#bootstrapDeclaredWebhooks (`secretPatch` / the `WEBHOOK_SECRET_FIELD` insert write cleartext exactly once into the `secret`-typed `signing_secret` column, which the engine wraps into sys_secret); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#attachSecret (recovers the plaintext per cache refresh and hands it to the outbox as `signingSecret`, consumed for one HMAC and never persisted, #7722)", - "note": "Was dead pre-bridge (definition_json only reachable via hand-authored rows), then materialized from authoring into definition_json as CLEARTEXT — recoverable over the ordinary data API, which is #7799. It no longer rides in that blob; the authored key itself is unchanged. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "evidence": "packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared (`splitWebhookSecret` lifts the key off the declaration onto the cached subscription, handed to the outbox as `signingSecret`, consumed for one HMAC and never persisted, #7722 — a declared key is code, so no row or cipher holds it); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#attachSecret (the same key for an organization's own row: recovered from the encrypted `signing_secret` column per cache refresh, #7799)", + "note": "Was dead pre-bridge (definition_json only reachable via hand-authored rows), then materialized from authoring into definition_json as CLEARTEXT — recoverable over the ordinary data API, which is #7799. It no longer rides in that blob; the authored key itself is unchanged. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "isActive": { "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`active: wh.isActive !== false` — the NAME MISMATCH remap, spec `isActive` → runtime `active`); packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#bootstrapDeclaredWebhooks (seed-not-clobber: an admin-authored (`managed_by === 'admin'`) or admin-edited (`customized === true`) row is skipped, so an operator's `active: false` survives redeploys)", - "note": "Was dead pre-bridge (NAME MISMATCH spec `isActive` vs runtime `active`, now remapped). Seed-not-clobber preserves an admin's `active:false` across redeploys. The dispatcher gates delivery on the REMAPPED column — `AutoEnqueuer.doRefresh` loads `where: { active: true }`, so only armed rows are ever cached — and that call site is kept here in PROSE rather than as a citation, on the `object.tenancy.organizationField` precedent: a consumer that never names the key belongs in the note, not in `evidence`. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — and this row is the one that PAID for the migration inside the same run. Rewriting this file's bare-filename citations as repo-rooted anchors made them askable for the first time, and the key-mention check immediately answered: `webhook/isActive -> auto-enqueuer.ts` UNANCHORED, because that file names `active` and never `isActive` in any case fold. The check was right — the remap happens at the materializer and nothing downstream of it sees the authored spelling — so the citation moved to prose rather than being exempted. Recording the direction because it is the useful one: an unparseable citation does not fail, it is simply never asked, and eleven of those had been sitting here reading as evidence. Re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "evidence": "packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource (`active: switched.get(webhook.name) ?? webhook.isActive !== false` — the declared default, overridden in either direction by a `sys_metadata_activation` row of type `webhook`, ADR-0131 D6 regime C); packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#resolveDeclared (`if (!d.active) continue` — a declaration switched off is no subscription)", + "note": "Was dead pre-bridge (NAME MISMATCH spec `isActive` vs runtime `active`, now remapped). Seed-not-clobber preserves an admin's `active:false` across redeploys. The dispatcher gates delivery on the REMAPPED column — `AutoEnqueuer.doRefresh` loads `where: { active: true }`, so only armed rows are ever cached — and that call site is kept here in PROSE rather than as a citation, on the `object.tenancy.organizationField` precedent: a consumer that never names the key belongs in the note, not in `evidence`. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — and this row is the one that PAID for the migration inside the same run. Rewriting this file's bare-filename citations as repo-rooted anchors made them askable for the first time, and the key-mention check immediately answered: `webhook/isActive -> auto-enqueuer.ts` UNANCHORED, because that file names `active` and never `isActive` in any case fold. The check was right — the remap happens at the materializer and nothing downstream of it sees the authored spelling — so the citation moved to prose rather than being exempted. Recording the direction because it is the useful one: an unparseable citation does not fail, it is simply never asked, and eleven of those had been sitting here reading as evidence. Re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: RE-ANCHORED to the registry read — ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, so a declared webhook is no longer copied into a `sys_webhook` row; the dispatcher reads the declaration itself (packages/plugins/plugin-webhooks/src/declared-webhooks.ts#DeclaredWebhookSource, projected by packages/plugins/plugin-webhooks/src/auto-enqueuer.ts#projectDeclared). Re-closed by hand against that change.", + "verifiedAt": "2026-10-19" }, "description": { - "status": "live", - "evidence": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts#mapWebhookToRow (`description: wh.description ?? null` — materialized to sys_webhook.description, #3489); packages/plugins/plugin-webhooks/src/sys-webhook.object.ts#SysWebhook (the column the Studio editor renders and edits)", - "note": "Display-only — surfaced/editable in Studio, inert in dispatch. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time.", - "verifiedAt": "2026-08-28" + "status": "dead", + "note": "Display-only — surfaced/editable in Studio, inert in dispatch. 2026-08-28: RE-ANCHORED (commit 9ee2dcfbd) — see the file `_note` for what the re-closure found across this ledger; re-closed by hand against c459da6bc, DATED for the first time. 2026-10-19: DEAD since ADR-0131 D2 retired the boot seeder `bootstrapDeclaredWebhooks`, which was its only reader — it copied this value into the declared webhook's `sys_webhook` row, where the Setup list showed it. A declared webhook has no row any more, and nothing displays a declaration: the dispatcher reads only what it delivers on (`auto-enqueuer.ts#projectDeclared`), the activation door only the name. Display-only, so the loss is a label nobody sees, never a delivery that changes. Enforce-or-remove (ADR-0049) is owed: a Setup surface that lists declared webhooks (the declared-sharing-rule Disable and Clone surface is the pattern) makes it live again, otherwise it retires.", + "verifiedAt": "2026-10-19", + "authorHint": "A declared webhook's `description` is shown nowhere: declared webhooks are read from the registry and have no Setup row. Remove it — it changes no delivery; keep the explanation as a code comment instead." } } } diff --git a/packages/spec/src/automation/webhook.test.ts b/packages/spec/src/automation/webhook.test.ts index 7d8233e22e0..d85d3a29665 100644 --- a/packages/spec/src/automation/webhook.test.ts +++ b/packages/spec/src/automation/webhook.test.ts @@ -224,8 +224,8 @@ describe('WebhookSchema', () => { // #4001 batch 11. The ledger carried this row as `authorable (p)` — "spec-only // (#3461)". Resolving the `(p)` found the opposite: three parse doors, one of -// them a BOOT path. `plugin-webhooks`' `bootstrapDeclaredWebhooks` re-parses -// every declared webhook before materializing it into `sys_webhook`, and a +// them a dispatch path. `plugin-webhooks`' declared-webhook reader re-parses +// every declared webhook before the dispatcher subscribes it, and a // failure there warns and SKIPS the subscription — so a rejection here is the // difference between a webhook that exists and one that does not. describe('unknown keys are rejected, not stripped', () => { diff --git a/packages/spec/src/automation/webhook.zod.ts b/packages/spec/src/automation/webhook.zod.ts index 7adb9e9fbd6..f7b4cd3730d 100644 --- a/packages/spec/src/automation/webhook.zod.ts +++ b/packages/spec/src/automation/webhook.zod.ts @@ -58,16 +58,17 @@ export type WebhookTriggerType = z.input; * Webhook Protocol - Outbound HTTP Integration * Push data to external URLs when events occur in the system. * - * **RUNTIME MATERIALIZATION (#3461):** - * A webhook authored on a stack (`defineStack({ webhooks })`) is not dispatched - * directly from this envelope. On boot, `@objectstack/plugin-webhooks` - * materializes each declared webhook into a `sys_webhook` data row (mapping - * `object → object_name`, `isActive → active`, and stashing the full envelope - * in `definition_json`); the auto-enqueuer dispatches off those rows. Declared - * webhooks re-seed every boot as `managed_by: 'package'`, but a row an admin has - * edited in Setup (`customized: true`) is never clobbered — a deactivated noisy - * webhook survives redeploys. Authoring `webhooks:` is therefore live, not a - * no-op. (A connector's NESTED `webhooks` array never was — it was never + * **RUNTIME (ADR-0131 D2, D6):** + * A webhook authored on a stack (`defineStack({ webhooks })`) is code, and the + * dispatcher reads it from the metadata registry: `@objectstack/plugin-webhooks` + * subscribes each declared webhook on every cache refresh (mapping + * `object → object_name`, with `secret` and `headers` held in memory), and no + * `sys_webhook` row is written for it. `sys_webhook` holds the webhooks an + * organization authors. A declared webhook is read-only: it is switched on or + * off for the deployment through the activation ledger + * (`POST /api/v1/security/_activation/webhook/:name`), which overrides + * `isActive` in either direction, and changed by a copy under a new name. + * Authoring `webhooks:` is therefore live, not a no-op. (A connector's NESTED `webhooks` array never was — it was never * registered as a `webhook` item — and was retired under ADR-0049; declare * every webhook here.) * @@ -113,10 +114,10 @@ export type WebhookTriggerType = z.input; * * 1. {@link defineWebhook} — the authoring factory. * 2. `defineStack({ webhooks })` — `StackSchema` holds `z.array(WebhookSchema)`. - * 3. **Boot** — `plugin-webhooks`' `bootstrapDeclaredWebhooks` re-parses every - * declared webhook before materializing it into a `sys_webhook` row - * (`bootstrap-declared-webhooks.ts`), and a parse failure there warns and - * SKIPS the webhook. So an undeclared key here is not merely stripped — from + * 3. **Dispatch** — `plugin-webhooks`' declared-webhook reader re-parses every + * declared webhook on each subscription-cache refresh + * (`declared-webhooks.ts`, which replaced the boot seeder ADR-0131 D2 + * retired), and a parse failure there warns and SKIPS the webhook. So an undeclared key here is not merely stripped — from * this change on it is the difference between a subscription that exists and * one that does not, which is why the rejection has to carry its fix. * @@ -136,8 +137,8 @@ export const WebhookSchema = lazySchema(() => strictObject({ surface: 'this webhook', aliases: { // Real, in-repo spellings of the same field on the OTHER side of the - // materialization bridge: `sys_webhook`'s columns are `object_name` / - // `active` (`mapWebhookToRow` in plugin-webhooks does the remap). An admin + // dispatcher: `sys_webhook`'s columns are `object_name` / `active` (the + // declared-webhook projection in plugin-webhooks makes the same remap). An admin // or agent reading a row back and re-authoring it from those column names // is the concrete path here, and neither word is within edit distance. // The column spelling alone covers `objectName` too — `aliasProbe` folds @@ -225,7 +226,7 @@ export const WebhookSchema = lazySchema(() => strictObject({ // Declared in #4001 batch 11 for the reason the module header states: both // metadata load paths call `applyProtection` on EVERY type, so a // package-loaded webhook has always carried these keys by the time - // `bootstrapDeclaredWebhooks` re-parses it. `.strip` discarded them; the + // the declared-webhook reader re-parses it. `.strip` discarded them; the // `.strict()` gate above would have rejected them, and the observable failure // would not have looked like a validation problem at all — just a webhook // that stopped existing after a redeploy, with one `warn` to say so. diff --git a/packages/spec/src/migrations/entries/retired-keys/18.integration__Connector__webhooks.ts b/packages/spec/src/migrations/entries/retired-keys/18.integration__Connector__webhooks.ts index bcd7307b190..10b50936bd7 100644 --- a/packages/spec/src/migrations/entries/retired-keys/18.integration__Connector__webhooks.ts +++ b/packages/spec/src/migrations/entries/retired-keys/18.integration__Connector__webhooks.ts @@ -10,7 +10,8 @@ // reads of a connector's own `webhooks` outside `packages/spec`, while // `stack.webhooks` — the lit control, same scan — is read five times; the one // test that authors a nested array -// (`bootstrap-declared-webhooks.connector-nested.test.ts`) exists to pin that it +// (`declared-webhooks.connector-nested.test.ts`, renamed from the retired +// seeder's `bootstrap-declared-webhooks.connector-nested.test.ts`) exists to pin that it // is NOT hoisted. And no code path emits a connector lifecycle event // (`sync.completed`, `auth.expired`, …) for its `events` to subscribe to. // diff --git a/scripts/engine-double-contract.baseline.json b/scripts/engine-double-contract.baseline.json index db86142a923..82b61e0662d 100644 --- a/scripts/engine-double-contract.baseline.json +++ b/scripts/engine-double-contract.baseline.json @@ -481,7 +481,7 @@ "closes": "add @objectstack/objectql to this package's devDependencies (verified acyclic — see `why`), then open the fake's update with assertEngineUpdateDispatch(data, options)" }, { - "file": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.test.ts", + "file": "packages/plugins/plugin-webhooks/src/declared-webhooks.test.ts", "verb": "delete", "unguarded": 1, "kind": "DEBT", @@ -489,7 +489,7 @@ "closes": "add @objectstack/objectql to this package's devDependencies (verified acyclic on this branch — see `why`), then open the fake's delete with assertEngineDeleteDispatch(options)" }, { - "file": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.test.ts", + "file": "packages/plugins/plugin-webhooks/src/declared-webhooks.test.ts", "verb": "update", "unguarded": 1, "kind": "DEBT", diff --git a/scripts/engine-double-contract.pinned.json b/scripts/engine-double-contract.pinned.json index 1bf23ff7c5a..5142b0a8563 100644 --- a/scripts/engine-double-contract.pinned.json +++ b/scripts/engine-double-contract.pinned.json @@ -3737,7 +3737,7 @@ "pinned": 1 }, { - "file": "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.test.ts", + "file": "packages/plugins/plugin-webhooks/src/declared-webhooks.test.ts", "verb": "findOne", "pinned": 1 }, diff --git a/scripts/objectql-double-limit.baseline.json b/scripts/objectql-double-limit.baseline.json index f527efbccde..abd1860f443 100644 --- a/scripts/objectql-double-limit.baseline.json +++ b/scripts/objectql-double-limit.baseline.json @@ -618,9 +618,6 @@ "packages/plugins/plugin-webhooks/src/auto-enqueuer.test.ts": { "unjudged": 1 }, - "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.test.ts": { - "unjudged": 1 - }, "packages/plugins/plugin-webhooks/src/webhook-headers-gate.test.ts": { "blind": 1 }, diff --git a/scripts/query-options-erasure-baseline.json b/scripts/query-options-erasure-baseline.json index a455d5de3d9..c2b55fc8e18 100644 --- a/scripts/query-options-erasure-baseline.json +++ b/scripts/query-options-erasure-baseline.json @@ -43,7 +43,6 @@ "packages/plugins/plugin-auth/src/auth-manager.ts": 14, "packages/plugins/plugin-sharing/src/share-link-routes.ts": 3, "packages/plugins/plugin-sharing/src/share-link-service.ts": 5, - "packages/plugins/plugin-webhooks/src/bootstrap-declared-webhooks.ts": 1, "packages/runtime/src/action-execution.ts": 1, "packages/runtime/src/domains/share-links.ts": 3, "packages/runtime/src/http-dispatcher.ts": 1,