You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
On PostgreSQL every webhook delivery is dropped at enqueue — "[webhook-auto-enqueuer] enqueue failed … Converting circular structure to JSON (Timeout)" — and the loss is logged at WARN #24835
Filing gate: ① product defect with a named location and a reproduction (silent delivery loss on a supported driver).
Category: ① — reach: the public data door (PATCH /api/v1/data/showcase_task/ID) with the showcase's declared webhook switched on through POST /api/v1/security/_activation/webhook/showcase_task_changed, measured on a fresh PostgreSQL 16 database at 307ce4c (and again on an 18.0.0-next.5 database the ceremony upgraded).
Who acts: the seat that owns packages/plugins/plugin-webhooks / packages/services/service-messaging (webhook outbox). Location: packages/plugins/plugin-webhooks/src/auto-enqueuer.ts (the fire-and-forget this.enqueue({... payload ...}), ~line 1167) → MessagingService.enqueueHttp → packages/services/service-messaging/src/sql-http-outbox.ts ~192 payload_json: JSON.stringify(input.payload ?? null).
Dedup: local scan of the 1,237 newest issues (state=all, open and closed) for circular|auto-enqueuer|enqueue failed — 0 hits; webhook.*postgres|postgres.*webhook — 0 hits.
reach: PATCH /api/v1/data/showcase_task/ID on os dev -d postgres://… at 307ce4c, twice (fresh database; upgraded database).
Path: integration-system.webhook-lifecycle
QA-source: #24838 · integration-system.webhook-lifecycle · observed beside the clauses (off-clause probe: driver differences)
Checklist item priority: P1
type: Bug
createdb qa4_fresh; from examples/app-showcase: os dev --seed-admin -p PORT -d postgres://USER:PASS@127.0.0.1:5432/qa4_fresh.
Sign in as admin@objectos.ai / admin123. POST /api/v1/security/_activation/webhook/showcase_task_changed body {"enabled": true} → 200 {"success":true,"data":{"type":"webhook","name":"showcase_task_changed","enabled":true}}.
PATCH /api/v1/data/showcase_task/ID body {"title":"webhook probe"} → 200.
After 4 s, GET /api/v1/data/sys_http_delivery:
Expected: one delivery row, ref_iddeclared:showcase_task_changed (what the same steps give on SQLite: measured three times this run).
Actual: 0 rows. The server log has exactly one line for the event: WARN [webhook-auto-enqueuer] enqueue failed {"webhook":"showcase_task_changed","eventId":"showcase_task:ID:updated:…","err":"Converting circular structure to JSON\n --> starting at object with constructor 'Timeout'\n | property '_idlePrev' -> object with constructor 'TimersList'\n --- property '_idleNext' closes the circle"}.
Same result on an 18.0.0-next.5 PostgreSQL database after os migrate organization-ownership --apply (ceremony complete, marker verified): the PATCH answers 200, no delivery row, the same WARN.
What is wrong
Something in the enqueued payload carries a Node Timeout (a timer handle) on the PostgreSQL path, so JSON.stringify throws and the delivery row is never written. SQLite boots of the same app enqueue normally.
The write that claims to persist the delivery does not land, and the system keeps looking healthy: the record write answered 200, Setup shows the webhook active, and the only trace is a WARN. Under AGENTS.md "Degradation log levels", a dropped durable write is an error, said once, with the consequence and the fix.
Not checked in this run: an admin-authored sys_webhook row on PostgreSQL (the probe's create was refused for its own body shape), and MySQL.
Claim: PM loop round 9 · 2026-10-20T10:14Z
Session: session_011TdZ8GjixNZFLz1xJD9zzK
Account: os-justin (the seat's linked user as GET /user answers it; always the card's assignee)
Branch: claude/issue-24835-pg-webhook-enqueue-circular
Worktree: objectstack-issue-24835
Domain: domain:services
Seat: domain:services#1
File surface: packages/plugins/plugin-webhooks/src/auto-enqueuer.ts (the per-subscription enqueue from ~:1140, projectRecordBodies from ~:1226, and the enqueue failed log line), packages/services/service-messaging/src/sql-http-outbox.ts (the payload_json serialisation at :192) only if the measured fix belongs there, their tests, and one new .changeset/24835-*.md. ⛔ Not auto-enqueuer.ts's mirror and declared-name region (:24–625, PR #24836, the #24583 epic's part 7c), ⛔ not a driver (driver-sql and the drivers are domain:engine's: if the stray Timeout comes from a driver row, stop and report), ⛔ not packages/spec/** (stop on breach; explain in the report)
Container & model: S, mode:subagent, model: default (opus), quoting this dispatch's --tier output "no path-derived mandate: the surface hits none of the 3 declared glob(s) … floor sonnet · default opus · ceiling fable"
Clause-②: no
Responsibility: on PostgreSQL every webhook delivery is lost at enqueue: the outbox's JSON.stringify of the payload throws on a Node Timeout, no sys_http_delivery row is written, and the loss is logged at WARN | the payload the auto-enqueuer hands the messaging outbox, and the degradation log level of a dropped durable write (AGENTS.md) | every deployment on PostgreSQL with an active webhook (18.0 QA wave, off-clause probe)
Thread-read: none
Serial constraints cleared: PR #24836 (the #24583 epic, part 7c) is open on auto-enqueuer.ts at ~:24–625 only (its hunk headers read at this claim), a region disjoint from the enqueue site; whichever lands second merges main. No open PR touches sql-http-outbox.ts (all open PRs' files read at this claim). The 18.x epic's declared territory (#15194) names neither file.
{
"issue": 24835,
"status": "blocked",
"branch": "claude/issue-24835-pg-webhook-enqueue-circular",
"pr": "#24862",
"session": "session_011TdZ8GjixNZFLz1xJD9zzK",
"premise_still_valid": true,
"summary": "Reproduced on a throwaway local PostgreSQL 16.15 cluster with the real ObjectQL engine, the real SqlDriver, and the real AutoEnqueuer wired to the real MessagingService and SqlHttpOutbox. One insert and one by-id update gave 1 sys_http_delivery row on PG (created only) plus the card's exact WARN, against 2 rows on SQLite. The card's loss and its WARN are real, but its location is wrong. The non-plain value is payload.changes.updated_at, a knex Raw (knex.fn.now()) that SqlDriver.update writes INTO THE CALLER'S PATCH (packages/drivers/driver-sql/src/sql-driver.ts:8778; formatInput returns the input object itself when nothing needs coercion, :20336 and :20526). ObjectQL then publishes that same object as DataEvent.changes (packages/objectql/src/engine.ts:18094 passes hookContext.input.data to driver.update; :18530 sets changes to it). The Raw holds the knex client, its pool and a Node Timeout, so sql-http-outbox.ts:192 JSON.stringify throws. Counterfactual: the same PG stack with the driver handed a copy of the patch wrote all 3 rows and logged nothing. That is a driver package, so under the dispatch's hard stop the root cause is NOT patched: it is a fork for domain:engine (open_questions). Delivered: the separable log-level half as draft PR 24862 (Part of, so the card stays open). A failed delivery write is now logged at error, once per webhook, with the consequence and the fix; later losses go to debug with their eventId, and the next recorded delivery re-arms it. Measured on PG after the change: one ERROR, then one DEBUG for the second lost update. No serialisation guard was added to sql-http-outbox.ts: on the decision frame's four axes it does not belong beside the root-cause fix (PR body).",
"tests": "Head 377a129 (after merging origin/main b3fc122, which brought the step-7c rewrite of auto-enqueuer.ts), all under os-verify-lock. pnpm exec turbo run build --concurrency=2 --filter=@objectstack/plugin-webhooks... → BUILD_EXIT=0, 'Tasks: 17 successful, 17 total'. pnpm --filter @objectstack/plugin-webhooks exec vitest run --maxWorkers=2 → TEST_EXIT=0, 'Test Files 21 passed (21) / Tests 226 passed (226)'. pnpm --filter @objectstack/plugin-webhooks typecheck → TYPECHECK_EXIT=0 ('check:test-typecheck: OK … 0 file(s) / 0 error(s)'). Pre-merge head 9a8a7f5: same commands, 21 files / 228 tests passed (2 fewer after the merge, because the merged step-7c change deleted tests in declared-webhooks.test.ts), typecheck 0. Four new pins in src/auto-enqueuer.test.ts assert level, count and named subjects: one error on the first loss naming the webhook and sys_http_delivery with eventId and cause in meta, later losses at debug and never warn; per-webhook; re-arm after a recorded delivery; the bulk path. Ablation, one leg (the changed behaviour), via node scripts/ablation-replace.mjs in WRAP mode. It replaced 'this.writeFailing.add(sub.id);' with the old bare warn plus return: anchor 1→0, blob 81ae0a56ccfa→17c85086162a, on disk. vitest run src/auto-enqueuer.test.ts → 'Tests 4 failed | 40 passed (44)', exactly the 4 new pins. Restore via git checkout HEAD: blob 81ae0a56ccfa equals HEAD and git diff HEAD is empty, both re-checked by hand afterwards. The same leg on 9a8a7f5 read 4 failed and 40 passed with the restore proven. The subject is a relative source import, so there was no dist leg. Live reproduction harness (scratch, deleted; real stack on PG 16.15 and better-sqlite3): base 41646a5 PG gave 'sys_http_delivery rows: 1', 'caller patch.updated_at :: Raw', 'non-plain values: payload.changes.updated_at :: Raw', 'JSON.stringify(payload) THROWS: Converting circular structure to JSON' and the card's WARN; SQLite gave 2 rows and none. Head 9a8a7f5 PG with 2 updates gave 1 row, 'ERROR [webhook-auto-enqueuer] webhook 'probe_task_changed': a delivery could not be recorded in sys_http_delivery, so this event is LOST …' and one DEBUG for the second; SQLite gave 3 rows. Counterfactual (PG, driver.update handed a copy of the patch) gave 3 rows, 'non-plain values: none' and 0 warn/error lines.",
"gates": "Derived with node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack (no paths) at 377a129: 70 families over 3 changed paths. Each command was run with its exit captured before any pipe (cmd > log 2>&1; ec=$?). 67 exited 0. Among them: check:engine-double-contract, check:nul-bytes, check:doc-authoring, check:issue-citations, check:cross-package-test-inputs, check:test-source-alias, check:type-check-coverage, check:published-files, check:dts-closure, check:lean-entry-closure, check-adr-0087-registration --base origin/main, check-changeset-no-major --base origin/main and check-empty-changeset --base origin/main. 2 are NOT MEASURED, exit 3 PREREQUISITE NOT MET: pnpm check:dual-build-cjs-loads (no dist for client-react, client, organizations, verify) and pnpm check:i18n (needs the CLI build closure). 1 is a declared narrowing: pnpm check:type-check-debt (--re-measure rebuilds the whole monorepo under the shared lock). --ran reconciliation with 'cmd :: exit N' lines: 'Run reconciliation — 70 derived, 67 run, 2 NOT-MEASURED, 1 UNRUN' (UNRUN = check:type-check-debt), exit 1. node scripts/check-closing-keyword-parity.mjs --body on the PR body: OK, no parser binds a closing declaration. CI on PR head 377a129 at report time: 24 check-runs. 'No other open PR may claim the same issue' success; 'Packed-tarball smoke (opt-in)' and '18.x upgrade harness (opt-in)' skipped. in_progress: Lint · ESLint, Lint · repo & tooling gates, Lint · docs & code guards, Lint · CI, release & engine gates, Lint · PM dispatch-gates, Lint · scoped · slot lookup & corpus, Lint · scoped · tooling self-tests, Lint · scoped · query-options erasure, Type Check · workspace, Type Check · source gates, Type Check · consumer gates, Type Check · debt ledger, Check Changeset, Check PR Size, Check Documentation Links, Flag docs affected by code changes, Governed Surface Queue Guard, Part-of PR must not also close its card, The card this PR closes must claim this branch, No other open PR may claim the same single-writer path, filter. Test Core and Build Core had not been listed yet. The full-battery 'Temporal Conformance (live PG + MySQL)' does not run on a PR.",
"mcp_calls": "0 — no MCP GitHub tool was called; reads went through gh api (REST GET)",
"api_writes": "2 — two fleet-write relay strokes (POST /repos/objectstack-ai/objectstack/dispatches, executed by fleet-write.yml as objectstack-fleet[bot]): (1) pr_create = POST /repos/objectstack-ai/objectstack/pulls (draft) + POST /repos//issues/24862/assignees [os-justin], read back 8313 bytes identical; (2) comment = POST /repos//issues/24835/comments (this report). Not REST: 3 git pushes (empty branch, 9a8a7f5, merge 377a129). No label written.",
"files_changed": [
"packages/plugins/plugin-webhooks/src/auto-enqueuer.ts (+70/-8; every hunk at line 680 or later, outside the step-7c mirror and declared-name region)",
"packages/plugins/plugin-webhooks/src/auto-enqueuer.test.ts (+114)",
".changeset/24835-webhook-lost-delivery-error.md (patch, @objectstack/plugin-webhooks, carries 'Clause-②: no')"
],
"deviations": [
"The PR's first body line is 'Part of #24835', not the dispatch's 'Fixes #24835'. The delivery loss is not fixed here (driver hard stop), and Fixes would close the card on merge. The PR body says #24835 remains open.",
"The hard stop says 'stop'. I read that as stopping the root-cause half (no driver, spec or outbox edit) and still delivered the separable log-level half in my own file surface. If the intended reading was a full stop, PR 24862 can be closed with no loss.",
"The scratch PG cluster ran at /home/user/pg-issue-24835, a sibling of the worktree rather than inside it: the scratchpad is root-only (0700) and initdb refuses root, so it ran via runuser -u postgres. Port 55835, trust auth. It was stopped (pg_ctl stop -m fast; pg_isready: no response) and deleted before this report.",
"Gate run incident: my first background gate runner was launched as a backgrounded cd-and-run list, so the recorded PID was its subshell and my kill missed the runner. Two runners overlapped, and an orphaned check:type-check-debt started a full monorepo turbo build under the shared lock for a few minutes. I killed every process of mine running in the worktree (by recorded PID and cwd, never by name). That left partial dist in 4 packages of my worktree (hono, cloud-connection, plugin-approvals, plugin-dev). I rebuilt them (42 tasks, 38 cached) and reran the battery once with a single runner; the gate numbers above come from that clean run.",
"Declared narrowing: check:type-check-debt was not run locally. check:dual-build-cjs-loads and check:i18n are NOT MEASURED (exit 3, prerequisite). All three are left to CI.",
"origin/main moved to abd5acd (scripts/engine-double-contract.pinned.json, plugin-email only) after my pre-PR merge. I did not merge again: it does not overlap this diff, and the PR's CI runs on the merge ref.",
"Commit trailers use the model-free pair AGENTS.md requires (Claude-Session plus Co-authored-by: Claude), not the harness's model-named Co-Authored-By line."
],
"open_questions": [
{
"question": "Fork for domain:engine. Where should the PostgreSQL delivery loss be fixed? Measured mechanism: SqlDriver.update stamps updated_at into the caller's patch object (sql-driver.ts:8778, and the same at rotatedUpdateById :10207), because formatInput and applyWriteColumnMap return the caller's object itself when nothing needs coercion (:20336, :20526). ObjectQL reuses that object as DataEvent.changes (engine.ts:18094 → :18530). On PG and MySQL the stamp is knex.fn.now(), a Raw holding the client, pool and Timeout (:8579), so every data.record.updated payload is unserialisable and every webhook update delivery is lost. On SQLite it is an ISO string (:8578) the caller never wrote, which serialises. updateMany already stamps into a fresh object (:10152). stampUpsertUpdatedAt (:8515) writes into formatted too; whether that is ever the caller's object is not measured.",
"options": [
"A: driver-sql (producer). update() and rotatedUpdateById() stamp into a fresh object, as updateMany does. Add a pin that update() leaves its data argument unchanged, on SQLite and on the live-PG/MySQL driver-sql lane (the Temporal Conformance job runs driver-sql against live servers). About 2 lines plus one pin. It removes the knex Raw on PG and MySQL and the stray SQLite updated_at from changes, for every reader: webhooks, afterUpdate hooks, realtime subscribers.",
"B: objectql (consumer). Snapshot hookContext.input.data for the event before driver.update. About 1 line. It protects the event from any driver that mutates, but leaves afterUpdate hooks seeing the mutated patch, and it is tolerance at the consumer for a producer defect.",
"C: both A and B."
],
"recommendation": "A. Business need: measured; every PG or MySQL deployment with an active webhook loses every update delivery, and the card's reach is the public PATCH door. Long-term soundness: contract-first, fixed where the mutation is produced; updateMany already shows the shape. Preventing AI mistakes: nothing new to author; one place stops leaking a dialect value object. Startup focus: no new surface. B only adds a second place that tolerates a producer defect. Whether IDataDriver.update should also say in packages/spec that a driver MUST NOT mutate data is the engine seat's call (spec is off-limits here)."
}
],
"out_of_scope_findings": [
"carrier: the domain:engine fix of the fork above · noted, not filed. Other readers of the same mutated object on PG and MySQL (afterUpdate hooks reading hookContext.input.data after the driver call, realtime subscribers that serialise data.record.updated, a cross-node realtime adapter) would see the same knex Raw. Only the webhook reader was measured. Same family as the fork, so it goes into that fix, not a point card. Dedupe words: SqlDriver update mutates caller data, updated_at knex.fn.now Raw DataEvent changes, Converting circular structure to JSON Timeout, formatInput returns input by identity.",
"carrier: none (承接者:无) · noted in PR 24862 Acceptance notes only. The parked-subscription branch of reportWriteFailure logs every lost record at error with no say-once. Observation; not this card's defect.",
"carrier: none (承接者:无) · noted in PR 24862 Acceptance notes only. check:durability-log-level judges try/catch blocks only, so a promise .catch(cb) or .then(ok, fail) handler around a durable write is invisible to it. Not extended: new gates default to no."
]
}
Claim: PM loop round 11 · 2026-10-20T11:32Z
Session: session_011TdZ8GjixNZFLz1xJD9zzK
Account: os-justin (the seat's linked user as GET /user answers it; always the card's assignee)
Branch: claude/issue-24835-driver-update-no-mutate
Worktree: objectstack-issue-24835-r2
Domain: domain:services
Seat: domain:services#1
File surface: round 2, the root cause measured in report 6289852046 (option A). packages/drivers/driver-sql/src/sql-driver.ts, where update() (:8778) and rotatedUpdateById() (:10207) stamp updated_at into a fresh object instead of the caller's patch, as updateMany does. Also a pin beside them in driver-sql that update() leaves its data argument unchanged, a webhook-delivery pin in plugin-webhooks (a real SqlDriver, an update, one stored delivery whose payload_json parses), and one new .changeset/24835-*.md. Cross-lane: driver-sql is domain:engine's, declared on #6367 (6289887999). ⛔ Not applyTenantScope or any region of PR #24825 (~:7616, :7651, :11481, :15606), ⛔ no other driver method, ⛔ not objectql (option B), ⛔ not packages/spec/** (stop on breach; explain in the report)
Container & model: S, mode:subagent, model: default (opus), quoting this dispatch's --tier output "no path-derived mandate"
Clause-②: no
Responsibility: on PostgreSQL and MySQL every webhook delivery for a record update is lost: SqlDriver.update writes a knex Raw updated_at into the caller's patch, which ObjectQL publishes as DataEvent.changes | the producer of the mutation (driver-sql update / rotatedUpdateById) | every PG or MySQL deployment with an active webhook; afterUpdate hooks and realtime readers of the same object
Thread-read: 6289852046
Serial constraints cleared: Round 1 is PR #24862 (Part of #24835: the lost-delivery log level), on its own branch, in review; round 2 does not touch its file. PR #24825 edits sql-driver.ts in regions disjoint from the stamp site (its hunk headers, read at this claim). No other open PR touches sql-driver.ts.
squash 3fbef41b4a (one parent, 4cef6e2280). auto-enqueuer.ts carries the writeFailing say-once ledger, and the first lost delivery per webhook logs at error
closing references
Part of #24835: no closing keyword
The card stays open and pm:dispatched. Round 2 (claim 6289902041) fixes the root cause in driver-sql: update() stamps a knex Rawupdated_at into the caller's patch. That is cross-lane, declared to domain:engine on #6367 (6289887999). Until round 2 lands, every webhook update delivery on PostgreSQL and MySQL is still lost, but the loss is now reported once at error.
{
"issue": 24835,
"status": "done",
"branch": "claude/issue-24835-driver-update-no-mutate",
"pr": "#24906",
"session": "session_011TdZ8GjixNZFLz1xJD9zzK",
"premise_still_valid": true,
"summary": "Round 2, option A (the producer fix). In packages/drivers/driver-sql/src/sql-driver.ts, update() (base 950f949 :8820) and rotatedUpdateById() (:10249) now stamp updated_at into a fresh object ({ ...formatted, updated_at: this.updatedAtStamp() }, the shape updateMany uses at :10194), never into formatted. formatted can be the caller's own data, because formatInput and applyWriteColumnMap are copy-on-write. stampUpsertUpdatedAt (:8554, assignment :8557) is unchanged, measured: its only caller is upsert() (:9580), where formatted comes from formatInput(object, toUpsert) (:9571) and toUpsert = { ...rest } (:9482) is a fresh object built from const { _id, ...rest } = data (:9481). So it can only write into the driver's own copy; grep finds no other caller in packages/drivers//src. Pins: (1) driver-sql, SQLite, update() leaves data toStrictEqual to a structuredClone on the plain, presumed (column present and column absent) and rotated paths, each proving its stamp ran; (2) plugin-webhooks, a real ObjectQL plus SqlDriver with an active declared webhook through the production wiring: insert plus by-id update store 2 sys_http_delivery rows (ref_id declared:probe_task_changed), each payload_json plain JSON. PG 16.15 throwaway cluster, pin 2's own body: 1 row with the in-place stamp put back, 2 rows on this branch, and the stored update payload's changes equal exactly {"title": "webhook probe"}. Draft PR 24906, Fixes #24835; round 1 (PR 24862) has landed as 3fbef41 and is untouched.",
"tests": "Final head 7c88888 (merge of origin/main 7035682, which brought round 1's landing and a version-packages commit), one os-verify-lock hold, each exit captured: turbo build --filter='@objectstack/plugin-webhooks^...' --filter='@objectstack/driver-sql...' BUILD_EXIT=0 'Tasks: 16 successful, 16 total'; pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2 DRIVER_SQL_TEST_EXIT=0 'Test Files 225 passed | 11 skipped (236) / Tests 3691 passed | 208 skipped (3899)' (the skips are unprovisioned live-PG/MySQL cells); driver-sql typecheck DRIVER_SQL_TYPECHECK_EXIT=0 (include src/**/, so the pin is compiled); plugin-webhooks vitest WEBHOOKS_TEST_EXIT=0 'Test Files 22 passed (22) / Tests 227 passed (227)'; plugin-webhooks typecheck WEBHOOKS_TYPECHECK_EXIT=0 ('check:test-typecheck: OK'). At 5755b9d: pin 1 'Tests 4 passed (4)', pin 2 'Tests 1 passed (1)'. Ablation via node scripts/ablation-replace.mjs in WRAP mode, one leg per changed site, at 5755b9d. Leg A (update(): 'const stamped = { ...formatted, … }' → 'formatted.updated_at = this.updatedAtStamp(); const stamped = formatted;', anchor 1→0, blob ecc2686b44a1→cfad3f1e1a3b): pin 1 PIN1_EXIT=1 'Tests 3 failed | 1 passed (4)' (plain and both presumed cases red; rotated green, since it is the other site); pin 2 SQLite PIN2_SQLITE_EXIT=0 (expected: an ISO string serialises); pin 2 PG PIN2_PG_EXIT=1 'expected [ 'data.record.created' ] to deeply equal [ 'data.record.created', …(1) ]'. Leg B (rotatedUpdateById(): the stamped copy → Object.assign(formatted, { updated_at: … }), anchor 1→0, blob ecc2686b44a1→d6664cc75a61): pin 1 'Tests 1 failed | 3 passed (4)', only the rotated case red. Both restores: blob ecc2686b44a1 equals HEAD and git diff HEAD is empty, re-checked by hand (git hash-object equals git rev-parse HEAD:path, diff 0 bytes). PG: throwaway cluster PostgreSQL 16.15 on 127.0.0.1:55836, with pin 2's body run through a generated, uncommitted copy that changes only the driver config. Before (leg A) 1 row; after (5755b9d, PG_PIN2_EXIT=0) 2 rows, data.record.created and data.record.updated, both declared:probe_task_changed; psql read of the update row's changes: {"title": "webhook probe"}. Round 1's base reading (41646a5) was also 1 row. 05079c5 (between the ablation and the final head) removed two 'as any' option casts from pin 2 only.",
"gates": "node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at the final head 7c88888: 87 families over 7 paths. One runner, each family under its own os-verify-lock acquisition (a re-acquire on queue-timeout), exit captured before any pipe. 84 exited 0. Among them: check:undeclared-dep-imports, check:test-source-alias, check:cross-package-test-inputs, check:engine-double-contract, check:query-options-erasure, check:driver-conformance, check:durability-log-level, check:turbo-task-graph, check:workspace-manifest-cycles, check:override-consistency, check:published-files, check:type-check-coverage, check:nul-bytes, check:doc-authoring, check-adr-0087-registration --base origin/main, check-changeset-fixed, check-changeset-no-major and check-empty-changeset. 2 are NOT MEASURED, exit 3 PREREQUISITE NOT MET: check:dual-build-cjs-loads ('some package has no dist/') and check:i18n ('the workspace CLI is not built'). 1 is a declared narrowing: check:type-check-debt (--re-measure takes the shared lock from inside the gate and rebuilds the whole monorepo). --ran on 'cmd :: exit N' lines: 'Run reconciliation — 87 derived, 84 run, 2 NOT-MEASURED, 1 UNRUN', exit 1 for the declared UNRUN. An earlier battery at 5755b9d found one real red, check:query-options-erasure 'test surface grew 236 → 237' (an 'as any' on engine options in pin 2); 05079c5 removed it. A second battery at 05079c5 read 84 of 86 green with the same 2 prerequisites. check-closing-keyword-parity --body: only 'Fixes #24835' binds; PR 24862 binds nothing. CI on 7c88888 at report time: 25 check-runs. Completed: 'No other open PR may claim the same issue' success; 'Packed-tarball smoke (opt-in)' and '18.x upgrade harness (opt-in)' skipped. in_progress: Lint · ESLint, Lint · repo & tooling gates, Lint · docs & code guards, Lint · CI, release & engine gates, Lint · PM dispatch-gates, Lint · scoped · query-options erasure, Lint · scoped · tooling self-tests, Lint · scoped · slot lookup & corpus, Type Check · workspace, Type Check · source gates, Type Check · consumer gates, Type Check · debt ledger, Check Changeset, Check PR Size, Check Documentation Links, Flag docs affected by code changes, Validate Package Dependencies, Governed Surface Queue Guard, Part-of PR must not also close its card, The card this PR closes must claim this branch, No other open PR may claim the same single-writer path, filter. Test Core and Build Core had not been listed yet; Temporal Conformance (live PG + MySQL) does not run on PRs.",
"mcp_calls": "0 — no MCP GitHub tool was called; reads went through gh api (REST GET)",
"api_writes": "2 — two fleet-write relay strokes (POST /repos/objectstack-ai/objectstack/dispatches, executed as objectstack-fleet[bot]): (1) pr_create = POST /repos/objectstack-ai/objectstack/pulls (draft) + POST /repos//issues/24906/assignees [os-justin], read back 9510 bytes identical, no labels; (2) comment = POST /repos//issues/24835/comments (this report). One push series, not REST, to claude/issue-24835-driver-update-no-mutate: the empty branch, 5755b9d, 05079c5 and merge 7c88888. Nothing pushed to PR 24862's branch.",
"files_changed": [
"packages/drivers/driver-sql/src/sql-driver.ts (+18/-7: update() and rotatedUpdateById() only; applyTenantScope and the PR 24825 regions untouched)",
"packages/drivers/driver-sql/src/sql-driver-24835-update-leaves-caller-data.test.ts (new, pin 1)",
"packages/plugins/plugin-webhooks/src/webhook-delivery-real-sql-driver.test.ts (new, pin 2)",
"packages/plugins/plugin-webhooks/package.json (+1: devDependency @objectstack/driver-sql)",
"packages/plugins/plugin-webhooks/vitest.config.ts (+6: an anchored alias of @objectstack/driver-sql to source)",
"pnpm-lock.yaml (+3: the plugin-webhooks importer entry only)",
".changeset/24835-driver-update-leaves-caller-data.md (patch, @objectstack/driver-sql, 'Clause-②: no')"
],
"deviations": [
"Pin 2 needed three support files the claim does not list by name: the plugin-webhooks devDependency on @objectstack/driver-sql, its vitest source alias (check:test-source-alias's own remedy; its ledger is shrink-only), and the 3-line lockfile importer entry. A plain 'pnpm install' also rewrote two unrelated vitest esbuild peer strings, so I restored the lockfile and inserted only the importer entry by hand. pnpm install --frozen-lockfile passes (exit 0).",
"The PG 'before' count (1 row) was measured on ablation leg A, the in-place stamp put back on this branch, not on an unmodified base. Round 1's base 41646a5 reading is the same 1 row.",
"Gate discipline: one runner, each family under its own lock acquisition. check:type-check-debt was not run locally (declared narrowing): its --re-measure takes the lock from inside the gate and rebuilds the whole monorepo. check:dual-build-cjs-loads and check:i18n are NOT MEASURED (exit 3, prerequisite). All three are left to CI.",
"I edited pin 2 (removing the casts) while the first battery was in flight, so its last ~10 families read the uncommitted edit. That battery is superseded: the second (05079c5) and the final (7c88888) batteries each ran whole on a committed head.",
"I merged origin/main (7035682) before opening the PR, because it touched plugin-webhooks (round 1's landing, and version packages bumping both packages' version lines). I rebuilt and reran both packages' tests, typecheck and the full battery on the merge.",
"The scratch PG cluster ran at /home/user/pg-issue-24835-r2, a sibling of the worktree, via runuser -u postgres (the scratchpad is 0700 and initdb refuses root). Port 55836, postmaster pid 20423. It was stopped (pg_ctl stop -m fast; pg_isready: no response; pid gone) and deleted before this report. The worktree was removed after the PR opened.",
"Commit trailers use the model-free pair AGENTS.md requires."
],
"open_questions": [],
"out_of_scope_findings": [
"carrier: none (承接者:无) · noted, not filed. The other formatInput/applyWriteColumnMap callers that write into formatted (create's and bulk paths' timestamp stamping) were not measured for caller-object mutation. The dispatch scopes this round to update(), rotatedUpdateById() and stampUpsertUpdatedAt; only upsert's copy was proven. This is an observation with no reproduction."
]
}
Reproduces on the published 18.0.0 (npm) · QA-source: #24908 · smoke:npm-18.0.0 · D3
Measured at 2026-10-20T13:56Z without the showcase, which is not on npm. Setup: a create-objectstack@18.0.0 scaffold plus pnpm add pg, a declared webhook on my_app_note (create/update/delete, isActive true), requires automation, triggers, webhooks, realtime. os dev on a fresh PostgreSQL 16.15 database.
POST /api/v1/security/_activation/webhook/my_app_note_changed {"enabled": true} -> 200 enabled true
POST /api/v1/data/my_app_note {"title":"wh probe"} -> 201
PATCH /api/v1/data/my_app_note/ID {"title":"wh probe 2"} -> 200
GET /api/v1/data/sys_http_delivery (after 5 s) -> total 1 (create only)
log: WARN [webhook-auto-enqueuer] enqueue failed {"webhook":"my_app_note_changed","eventId":"my_app_note:ID:updated:...","err":"Converting circular structure to JSON --> starting at object with constructor 'Timeout' ..."}
So on PG the create event enqueues and the update event is the one dropped. The same app on SQLite: 2 rows (create and update).
squash eee65afc89 (one parent, a7199ef974). In sql-driver.ts, update() and rotatedUpdateById() stamp updated_at into { ...formatted, updated_at: … }, never onto the caller's object
closing references
the PR body closes #24835 only. Round 1 (PR #24862, 3fbef41b4a) is named without a keyword
State change in this stroke: pm:dispatched off and the assignee cleared. The card closed completed on the merge.
What changes: on PostgreSQL and MySQL, a record update's webhook delivery is enqueued again. Before, DataEvent.changes carried the knex Raw stamp and the outbox's JSON.stringify threw. Now the update row's changes is exactly the caller's patch (ACCEPT 6292958276). SQLite, creates and deletes are unchanged.
Activity
objectstack-fleet commentedon Oct 20, 2026
Claim: PM loop round 9 · 2026-10-20T10:14Z
Session:
session_011TdZ8GjixNZFLz1xJD9zzKAccount:
os-justin(the seat's linked user asGET /useranswers it; always the card's assignee)Branch:
claude/issue-24835-pg-webhook-enqueue-circularWorktree:
objectstack-issue-24835Domain:
domain:servicesSeat:
domain:services#1File surface:
packages/plugins/plugin-webhooks/src/auto-enqueuer.ts(the per-subscription enqueue from ~:1140,projectRecordBodiesfrom ~:1226, and theenqueue failedlog line),packages/services/service-messaging/src/sql-http-outbox.ts(thepayload_jsonserialisation at:192) only if the measured fix belongs there, their tests, and one new.changeset/24835-*.md. ⛔ Notauto-enqueuer.ts's mirror and declared-name region (:24–625, PR #24836, the #24583 epic's part 7c), ⛔ not a driver (driver-sqland the drivers aredomain:engine's: if the strayTimeoutcomes from a driver row, stop and report), ⛔ notpackages/spec/**(stop on breach; explain in the report)Container & model:
S,mode:subagent,model: default (opus), quoting this dispatch's --tier output "no path-derived mandate: the surface hits none of the 3 declared glob(s) … floor sonnet · default opus · ceiling fable"Clause-②: no
Responsibility:
on PostgreSQL every webhook delivery is lost at enqueue: the outbox's JSON.stringify of the payload throws on a Node Timeout, no sys_http_delivery row is written, and the loss is logged at WARN | the payload the auto-enqueuer hands the messaging outbox, and the degradation log level of a dropped durable write (AGENTS.md) | every deployment on PostgreSQL with an active webhook (18.0 QA wave, off-clause probe)Thread-read: none
Serial constraints cleared:
PR #24836 (the #24583 epic, part 7c) is open on auto-enqueuer.ts at ~:24–625 only (its hunk headers read at this claim), a region disjoint from the enqueue site; whichever lands second merges main. No open PR touches sql-http-outbox.ts (all open PRs' files read at this claim). The 18.x epic's declared territory (#15194) names neither file.Readings taken on
origin/main7c2eab929c.Generated by Claude Code
objectstack-fleet commentedon Oct 20, 2026
os-dev-report
{
"issue": 24835,
"status": "blocked",
"branch": "claude/issue-24835-pg-webhook-enqueue-circular",
"pr": "#24862",
"session": "session_011TdZ8GjixNZFLz1xJD9zzK",
"premise_still_valid": true,
"summary": "Reproduced on a throwaway local PostgreSQL 16.15 cluster with the real ObjectQL engine, the real SqlDriver, and the real AutoEnqueuer wired to the real MessagingService and SqlHttpOutbox. One insert and one by-id update gave 1 sys_http_delivery row on PG (created only) plus the card's exact WARN, against 2 rows on SQLite. The card's loss and its WARN are real, but its location is wrong. The non-plain value is payload.changes.updated_at, a knex Raw (knex.fn.now()) that SqlDriver.update writes INTO THE CALLER'S PATCH (packages/drivers/driver-sql/src/sql-driver.ts:8778; formatInput returns the input object itself when nothing needs coercion, :20336 and :20526). ObjectQL then publishes that same object as DataEvent.changes (packages/objectql/src/engine.ts:18094 passes hookContext.input.data to driver.update; :18530 sets changes to it). The Raw holds the knex client, its pool and a Node Timeout, so sql-http-outbox.ts:192 JSON.stringify throws. Counterfactual: the same PG stack with the driver handed a copy of the patch wrote all 3 rows and logged nothing. That is a driver package, so under the dispatch's hard stop the root cause is NOT patched: it is a fork for domain:engine (open_questions). Delivered: the separable log-level half as draft PR 24862 (Part of, so the card stays open). A failed delivery write is now logged at error, once per webhook, with the consequence and the fix; later losses go to debug with their eventId, and the next recorded delivery re-arms it. Measured on PG after the change: one ERROR, then one DEBUG for the second lost update. No serialisation guard was added to sql-http-outbox.ts: on the decision frame's four axes it does not belong beside the root-cause fix (PR body).",
"tests": "Head 377a129 (after merging origin/main b3fc122, which brought the step-7c rewrite of auto-enqueuer.ts), all under os-verify-lock. pnpm exec turbo run build --concurrency=2 --filter=@objectstack/plugin-webhooks... → BUILD_EXIT=0, 'Tasks: 17 successful, 17 total'. pnpm --filter @objectstack/plugin-webhooks exec vitest run --maxWorkers=2 → TEST_EXIT=0, 'Test Files 21 passed (21) / Tests 226 passed (226)'. pnpm --filter @objectstack/plugin-webhooks typecheck → TYPECHECK_EXIT=0 ('check:test-typecheck: OK … 0 file(s) / 0 error(s)'). Pre-merge head 9a8a7f5: same commands, 21 files / 228 tests passed (2 fewer after the merge, because the merged step-7c change deleted tests in declared-webhooks.test.ts), typecheck 0. Four new pins in src/auto-enqueuer.test.ts assert level, count and named subjects: one error on the first loss naming the webhook and sys_http_delivery with eventId and cause in meta, later losses at debug and never warn; per-webhook; re-arm after a recorded delivery; the bulk path. Ablation, one leg (the changed behaviour), via node scripts/ablation-replace.mjs in WRAP mode. It replaced 'this.writeFailing.add(sub.id);' with the old bare warn plus return: anchor 1→0, blob 81ae0a56ccfa→17c85086162a, on disk. vitest run src/auto-enqueuer.test.ts → 'Tests 4 failed | 40 passed (44)', exactly the 4 new pins. Restore via git checkout HEAD: blob 81ae0a56ccfa equals HEAD and git diff HEAD is empty, both re-checked by hand afterwards. The same leg on 9a8a7f5 read 4 failed and 40 passed with the restore proven. The subject is a relative source import, so there was no dist leg. Live reproduction harness (scratch, deleted; real stack on PG 16.15 and better-sqlite3): base 41646a5 PG gave 'sys_http_delivery rows: 1', 'caller patch.updated_at :: Raw', 'non-plain values: payload.changes.updated_at :: Raw', 'JSON.stringify(payload) THROWS: Converting circular structure to JSON' and the card's WARN; SQLite gave 2 rows and none. Head 9a8a7f5 PG with 2 updates gave 1 row, 'ERROR [webhook-auto-enqueuer] webhook 'probe_task_changed': a delivery could not be recorded in sys_http_delivery, so this event is LOST …' and one DEBUG for the second; SQLite gave 3 rows. Counterfactual (PG, driver.update handed a copy of the patch) gave 3 rows, 'non-plain values: none' and 0 warn/error lines.",
"gates": "Derived with node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack (no paths) at 377a129: 70 families over 3 changed paths. Each command was run with its exit captured before any pipe (cmd > log 2>&1; ec=$?). 67 exited 0. Among them: check:engine-double-contract, check:nul-bytes, check:doc-authoring, check:issue-citations, check:cross-package-test-inputs, check:test-source-alias, check:type-check-coverage, check:published-files, check:dts-closure, check:lean-entry-closure, check-adr-0087-registration --base origin/main, check-changeset-no-major --base origin/main and check-empty-changeset --base origin/main. 2 are NOT MEASURED, exit 3 PREREQUISITE NOT MET: pnpm check:dual-build-cjs-loads (no dist for client-react, client, organizations, verify) and pnpm check:i18n (needs the CLI build closure). 1 is a declared narrowing: pnpm check:type-check-debt (--re-measure rebuilds the whole monorepo under the shared lock). --ran reconciliation with 'cmd :: exit N' lines: 'Run reconciliation — 70 derived, 67 run, 2 NOT-MEASURED, 1 UNRUN' (UNRUN = check:type-check-debt), exit 1. node scripts/check-closing-keyword-parity.mjs --body on the PR body: OK, no parser binds a closing declaration. CI on PR head 377a129 at report time: 24 check-runs. 'No other open PR may claim the same issue' success; 'Packed-tarball smoke (opt-in)' and '18.x upgrade harness (opt-in)' skipped. in_progress: Lint · ESLint, Lint · repo & tooling gates, Lint · docs & code guards, Lint · CI, release & engine gates, Lint · PM dispatch-gates, Lint · scoped · slot lookup & corpus, Lint · scoped · tooling self-tests, Lint · scoped · query-options erasure, Type Check · workspace, Type Check · source gates, Type Check · consumer gates, Type Check · debt ledger, Check Changeset, Check PR Size, Check Documentation Links, Flag docs affected by code changes, Governed Surface Queue Guard, Part-of PR must not also close its card, The card this PR closes must claim this branch, No other open PR may claim the same single-writer path, filter. Test Core and Build Core had not been listed yet. The full-battery 'Temporal Conformance (live PG + MySQL)' does not run on a PR.",
"mcp_calls": "0 — no MCP GitHub tool was called; reads went through gh api (REST GET)",
"api_writes": "2 — two fleet-write relay strokes (POST /repos/objectstack-ai/objectstack/dispatches, executed by fleet-write.yml as objectstack-fleet[bot]): (1) pr_create = POST /repos/objectstack-ai/objectstack/pulls (draft) + POST /repos//issues/24862/assignees [os-justin], read back 8313 bytes identical; (2) comment = POST /repos//issues/24835/comments (this report). Not REST: 3 git pushes (empty branch, 9a8a7f5, merge 377a129). No label written.",
"files_changed": [
"packages/plugins/plugin-webhooks/src/auto-enqueuer.ts (+70/-8; every hunk at line 680 or later, outside the step-7c mirror and declared-name region)",
"packages/plugins/plugin-webhooks/src/auto-enqueuer.test.ts (+114)",
".changeset/24835-webhook-lost-delivery-error.md (patch, @objectstack/plugin-webhooks, carries 'Clause-②: no')"
],
"deviations": [
"The PR's first body line is 'Part of #24835', not the dispatch's 'Fixes #24835'. The delivery loss is not fixed here (driver hard stop), and Fixes would close the card on merge. The PR body says #24835 remains open.",
"The hard stop says 'stop'. I read that as stopping the root-cause half (no driver, spec or outbox edit) and still delivered the separable log-level half in my own file surface. If the intended reading was a full stop, PR 24862 can be closed with no loss.",
"The scratch PG cluster ran at /home/user/pg-issue-24835, a sibling of the worktree rather than inside it: the scratchpad is root-only (0700) and initdb refuses root, so it ran via runuser -u postgres. Port 55835, trust auth. It was stopped (pg_ctl stop -m fast; pg_isready: no response) and deleted before this report.",
"Gate run incident: my first background gate runner was launched as a backgrounded cd-and-run list, so the recorded PID was its subshell and my kill missed the runner. Two runners overlapped, and an orphaned check:type-check-debt started a full monorepo turbo build under the shared lock for a few minutes. I killed every process of mine running in the worktree (by recorded PID and cwd, never by name). That left partial dist in 4 packages of my worktree (hono, cloud-connection, plugin-approvals, plugin-dev). I rebuilt them (42 tasks, 38 cached) and reran the battery once with a single runner; the gate numbers above come from that clean run.",
"Declared narrowing: check:type-check-debt was not run locally. check:dual-build-cjs-loads and check:i18n are NOT MEASURED (exit 3, prerequisite). All three are left to CI.",
"origin/main moved to abd5acd (scripts/engine-double-contract.pinned.json, plugin-email only) after my pre-PR merge. I did not merge again: it does not overlap this diff, and the PR's CI runs on the merge ref.",
"Commit trailers use the model-free pair AGENTS.md requires (Claude-Session plus Co-authored-by: Claude), not the harness's model-named Co-Authored-By line."
],
"open_questions": [
{
"question": "Fork for domain:engine. Where should the PostgreSQL delivery loss be fixed? Measured mechanism: SqlDriver.update stamps updated_at into the caller's patch object (sql-driver.ts:8778, and the same at rotatedUpdateById :10207), because formatInput and applyWriteColumnMap return the caller's object itself when nothing needs coercion (:20336, :20526). ObjectQL reuses that object as DataEvent.changes (engine.ts:18094 → :18530). On PG and MySQL the stamp is knex.fn.now(), a Raw holding the client, pool and Timeout (:8579), so every data.record.updated payload is unserialisable and every webhook update delivery is lost. On SQLite it is an ISO string (:8578) the caller never wrote, which serialises. updateMany already stamps into a fresh object (:10152). stampUpsertUpdatedAt (:8515) writes into formatted too; whether that is ever the caller's object is not measured.",
"options": [
"A: driver-sql (producer). update() and rotatedUpdateById() stamp into a fresh object, as updateMany does. Add a pin that update() leaves its data argument unchanged, on SQLite and on the live-PG/MySQL driver-sql lane (the Temporal Conformance job runs driver-sql against live servers). About 2 lines plus one pin. It removes the knex Raw on PG and MySQL and the stray SQLite updated_at from changes, for every reader: webhooks, afterUpdate hooks, realtime subscribers.",
"B: objectql (consumer). Snapshot hookContext.input.data for the event before driver.update. About 1 line. It protects the event from any driver that mutates, but leaves afterUpdate hooks seeing the mutated patch, and it is tolerance at the consumer for a producer defect.",
"C: both A and B."
],
"recommendation": "A. Business need: measured; every PG or MySQL deployment with an active webhook loses every update delivery, and the card's reach is the public PATCH door. Long-term soundness: contract-first, fixed where the mutation is produced; updateMany already shows the shape. Preventing AI mistakes: nothing new to author; one place stops leaking a dialect value object. Startup focus: no new surface. B only adds a second place that tolerates a producer defect. Whether IDataDriver.update should also say in packages/spec that a driver MUST NOT mutate data is the engine seat's call (spec is off-limits here)."
}
],
"out_of_scope_findings": [
"carrier: the domain:engine fix of the fork above · noted, not filed. Other readers of the same mutated object on PG and MySQL (afterUpdate hooks reading hookContext.input.data after the driver call, realtime subscribers that serialise data.record.updated, a cross-node realtime adapter) would see the same knex Raw. Only the webhook reader was measured. Same family as the fork, so it goes into that fix, not a point card. Dedupe words: SqlDriver update mutates caller data, updated_at knex.fn.now Raw DataEvent changes, Converting circular structure to JSON Timeout, formatInput returns input by identity.",
"carrier: none (承接者:无) · noted in PR 24862 Acceptance notes only. The parked-subscription branch of reportWriteFailure logs every lost record at error with no say-once. Observation; not this card's defect.",
"carrier: none (承接者:无) · noted in PR 24862 Acceptance notes only. check:durability-log-level judges try/catch blocks only, so a promise .catch(cb) or .then(ok, fail) handler around a durable write is invisible to it. Not extended: new gates default to no."
]
}
objectstack-fleet commentedon Oct 20, 2026
Claim: PM loop round 11 · 2026-10-20T11:32Z
Session:
session_011TdZ8GjixNZFLz1xJD9zzKAccount:
os-justin(the seat's linked user asGET /useranswers it; always the card's assignee)Branch:
claude/issue-24835-driver-update-no-mutateWorktree:
objectstack-issue-24835-r2Domain:
domain:servicesSeat:
domain:services#1File surface: round 2, the root cause measured in report
6289852046(option A).packages/drivers/driver-sql/src/sql-driver.ts, whereupdate()(:8778) androtatedUpdateById()(:10207) stampupdated_atinto a fresh object instead of the caller's patch, asupdateManydoes. Also a pin beside them indriver-sqlthatupdate()leaves itsdataargument unchanged, a webhook-delivery pin inplugin-webhooks(a realSqlDriver, an update, one stored delivery whosepayload_jsonparses), and one new.changeset/24835-*.md. Cross-lane:driver-sqlisdomain:engine's, declared on #6367 (6289887999). ⛔ NotapplyTenantScopeor any region of PR #24825 (~:7616,:7651,:11481,:15606), ⛔ no other driver method, ⛔ not objectql (option B), ⛔ notpackages/spec/**(stop on breach; explain in the report)Container & model:
S,mode:subagent,model: default (opus), quoting this dispatch's --tier output "no path-derived mandate"Clause-②: no
Responsibility:
on PostgreSQL and MySQL every webhook delivery for a record update is lost: SqlDriver.update writes a knex Raw updated_at into the caller's patch, which ObjectQL publishes as DataEvent.changes | the producer of the mutation (driver-sql update / rotatedUpdateById) | every PG or MySQL deployment with an active webhook; afterUpdate hooks and realtime readers of the same objectThread-read: 6289852046
Serial constraints cleared:
Round 1 is PR #24862 (Part of #24835: the lost-delivery log level), on its own branch, in review; round 2 does not touch its file. PR #24825 edits sql-driver.ts in regions disjoint from the stamp site (its hunk headers, read at this claim). No other open PR touches sql-driver.ts.Readings taken on
origin/main950f949a9c.Generated by Claude Code
objectstack-fleet commentedon Oct 20, 2026
ACCEPT → PR #24862 comment
6290194485·domain:servicesseat 1 ·session_011TdZ8GjixNZFLz1xJD9zzK· 2026-10-20T11:51ZGenerated by Claude Code
objectstack-fleet commentedon Oct 20, 2026
Round 1 landed ·
domain:servicesseat 1 ·session_011TdZ8GjixNZFLz1xJD9zzK· 2026-10-20T12:22Zadded_to_merge_queue2026-10-20T11:53Z →merged2026-10-20T12:20Zorigin/mainby content3fbef41b4a(one parent,4cef6e2280).auto-enqueuer.tscarries thewriteFailingsay-once ledger, and the first lost delivery per webhook logs aterrorPart of #24835: no closing keywordThe card stays open and
pm:dispatched. Round 2 (claim6289902041) fixes the root cause indriver-sql:update()stamps a knexRawupdated_atinto the caller's patch. That is cross-lane, declared todomain:engineon #6367 (6289887999). Until round 2 lands, every webhook update delivery on PostgreSQL and MySQL is still lost, but the loss is now reported once aterror.Generated by Claude Code
objectstack-fleet commentedon Oct 20, 2026
os-dev-report
{
"issue": 24835,
"status": "done",
"branch": "claude/issue-24835-driver-update-no-mutate",
"pr": "#24906",
"session": "session_011TdZ8GjixNZFLz1xJD9zzK",
"premise_still_valid": true,
"summary": "Round 2, option A (the producer fix). In packages/drivers/driver-sql/src/sql-driver.ts, update() (base 950f949 :8820) and rotatedUpdateById() (:10249) now stamp updated_at into a fresh object ({ ...formatted, updated_at: this.updatedAtStamp() }, the shape updateMany uses at :10194), never into formatted. formatted can be the caller's own data, because formatInput and applyWriteColumnMap are copy-on-write. stampUpsertUpdatedAt (:8554, assignment :8557) is unchanged, measured: its only caller is upsert() (:9580), where formatted comes from formatInput(object, toUpsert) (:9571) and toUpsert = { ...rest } (:9482) is a fresh object built from const { _id, ...rest } = data (:9481). So it can only write into the driver's own copy; grep finds no other caller in packages/drivers//src. Pins: (1) driver-sql, SQLite, update() leaves data toStrictEqual to a structuredClone on the plain, presumed (column present and column absent) and rotated paths, each proving its stamp ran; (2) plugin-webhooks, a real ObjectQL plus SqlDriver with an active declared webhook through the production wiring: insert plus by-id update store 2 sys_http_delivery rows (ref_id declared:probe_task_changed), each payload_json plain JSON. PG 16.15 throwaway cluster, pin 2's own body: 1 row with the in-place stamp put back, 2 rows on this branch, and the stored update payload's changes equal exactly {"title": "webhook probe"}. Draft PR 24906, Fixes #24835; round 1 (PR 24862) has landed as 3fbef41 and is untouched.",
"tests": "Final head 7c88888 (merge of origin/main 7035682, which brought round 1's landing and a version-packages commit), one os-verify-lock hold, each exit captured: turbo build --filter='@objectstack/plugin-webhooks^...' --filter='@objectstack/driver-sql...' BUILD_EXIT=0 'Tasks: 16 successful, 16 total'; pnpm --filter @objectstack/driver-sql exec vitest run --maxWorkers=2 DRIVER_SQL_TEST_EXIT=0 'Test Files 225 passed | 11 skipped (236) / Tests 3691 passed | 208 skipped (3899)' (the skips are unprovisioned live-PG/MySQL cells); driver-sql typecheck DRIVER_SQL_TYPECHECK_EXIT=0 (include src/**/, so the pin is compiled); plugin-webhooks vitest WEBHOOKS_TEST_EXIT=0 'Test Files 22 passed (22) / Tests 227 passed (227)'; plugin-webhooks typecheck WEBHOOKS_TYPECHECK_EXIT=0 ('check:test-typecheck: OK'). At 5755b9d: pin 1 'Tests 4 passed (4)', pin 2 'Tests 1 passed (1)'. Ablation via node scripts/ablation-replace.mjs in WRAP mode, one leg per changed site, at 5755b9d. Leg A (update(): 'const stamped = { ...formatted, … }' → 'formatted.updated_at = this.updatedAtStamp(); const stamped = formatted;', anchor 1→0, blob ecc2686b44a1→cfad3f1e1a3b): pin 1 PIN1_EXIT=1 'Tests 3 failed | 1 passed (4)' (plain and both presumed cases red; rotated green, since it is the other site); pin 2 SQLite PIN2_SQLITE_EXIT=0 (expected: an ISO string serialises); pin 2 PG PIN2_PG_EXIT=1 'expected [ 'data.record.created' ] to deeply equal [ 'data.record.created', …(1) ]'. Leg B (rotatedUpdateById(): the stamped copy → Object.assign(formatted, { updated_at: … }), anchor 1→0, blob ecc2686b44a1→d6664cc75a61): pin 1 'Tests 1 failed | 3 passed (4)', only the rotated case red. Both restores: blob ecc2686b44a1 equals HEAD and git diff HEAD is empty, re-checked by hand (git hash-object equals git rev-parse HEAD:path, diff 0 bytes). PG: throwaway cluster PostgreSQL 16.15 on 127.0.0.1:55836, with pin 2's body run through a generated, uncommitted copy that changes only the driver config. Before (leg A) 1 row; after (5755b9d, PG_PIN2_EXIT=0) 2 rows, data.record.created and data.record.updated, both declared:probe_task_changed; psql read of the update row's changes: {"title": "webhook probe"}. Round 1's base reading (41646a5) was also 1 row. 05079c5 (between the ablation and the final head) removed two 'as any' option casts from pin 2 only.",
"gates": "node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack at the final head 7c88888: 87 families over 7 paths. One runner, each family under its own os-verify-lock acquisition (a re-acquire on queue-timeout), exit captured before any pipe. 84 exited 0. Among them: check:undeclared-dep-imports, check:test-source-alias, check:cross-package-test-inputs, check:engine-double-contract, check:query-options-erasure, check:driver-conformance, check:durability-log-level, check:turbo-task-graph, check:workspace-manifest-cycles, check:override-consistency, check:published-files, check:type-check-coverage, check:nul-bytes, check:doc-authoring, check-adr-0087-registration --base origin/main, check-changeset-fixed, check-changeset-no-major and check-empty-changeset. 2 are NOT MEASURED, exit 3 PREREQUISITE NOT MET: check:dual-build-cjs-loads ('some package has no dist/') and check:i18n ('the workspace CLI is not built'). 1 is a declared narrowing: check:type-check-debt (--re-measure takes the shared lock from inside the gate and rebuilds the whole monorepo). --ran on 'cmd :: exit N' lines: 'Run reconciliation — 87 derived, 84 run, 2 NOT-MEASURED, 1 UNRUN', exit 1 for the declared UNRUN. An earlier battery at 5755b9d found one real red, check:query-options-erasure 'test surface grew 236 → 237' (an 'as any' on engine options in pin 2); 05079c5 removed it. A second battery at 05079c5 read 84 of 86 green with the same 2 prerequisites. check-closing-keyword-parity --body: only 'Fixes #24835' binds; PR 24862 binds nothing. CI on 7c88888 at report time: 25 check-runs. Completed: 'No other open PR may claim the same issue' success; 'Packed-tarball smoke (opt-in)' and '18.x upgrade harness (opt-in)' skipped. in_progress: Lint · ESLint, Lint · repo & tooling gates, Lint · docs & code guards, Lint · CI, release & engine gates, Lint · PM dispatch-gates, Lint · scoped · query-options erasure, Lint · scoped · tooling self-tests, Lint · scoped · slot lookup & corpus, Type Check · workspace, Type Check · source gates, Type Check · consumer gates, Type Check · debt ledger, Check Changeset, Check PR Size, Check Documentation Links, Flag docs affected by code changes, Validate Package Dependencies, Governed Surface Queue Guard, Part-of PR must not also close its card, The card this PR closes must claim this branch, No other open PR may claim the same single-writer path, filter. Test Core and Build Core had not been listed yet; Temporal Conformance (live PG + MySQL) does not run on PRs.",
"mcp_calls": "0 — no MCP GitHub tool was called; reads went through gh api (REST GET)",
"api_writes": "2 — two fleet-write relay strokes (POST /repos/objectstack-ai/objectstack/dispatches, executed as objectstack-fleet[bot]): (1) pr_create = POST /repos/objectstack-ai/objectstack/pulls (draft) + POST /repos//issues/24906/assignees [os-justin], read back 9510 bytes identical, no labels; (2) comment = POST /repos//issues/24835/comments (this report). One push series, not REST, to claude/issue-24835-driver-update-no-mutate: the empty branch, 5755b9d, 05079c5 and merge 7c88888. Nothing pushed to PR 24862's branch.",
"files_changed": [
"packages/drivers/driver-sql/src/sql-driver.ts (+18/-7: update() and rotatedUpdateById() only; applyTenantScope and the PR 24825 regions untouched)",
"packages/drivers/driver-sql/src/sql-driver-24835-update-leaves-caller-data.test.ts (new, pin 1)",
"packages/plugins/plugin-webhooks/src/webhook-delivery-real-sql-driver.test.ts (new, pin 2)",
"packages/plugins/plugin-webhooks/package.json (+1: devDependency @objectstack/driver-sql)",
"packages/plugins/plugin-webhooks/vitest.config.ts (+6: an anchored alias of @objectstack/driver-sql to source)",
"pnpm-lock.yaml (+3: the plugin-webhooks importer entry only)",
".changeset/24835-driver-update-leaves-caller-data.md (patch, @objectstack/driver-sql, 'Clause-②: no')"
],
"deviations": [
"Pin 2 needed three support files the claim does not list by name: the plugin-webhooks devDependency on @objectstack/driver-sql, its vitest source alias (check:test-source-alias's own remedy; its ledger is shrink-only), and the 3-line lockfile importer entry. A plain 'pnpm install' also rewrote two unrelated vitest esbuild peer strings, so I restored the lockfile and inserted only the importer entry by hand. pnpm install --frozen-lockfile passes (exit 0).",
"The PG 'before' count (1 row) was measured on ablation leg A, the in-place stamp put back on this branch, not on an unmodified base. Round 1's base 41646a5 reading is the same 1 row.",
"Gate discipline: one runner, each family under its own lock acquisition. check:type-check-debt was not run locally (declared narrowing): its --re-measure takes the lock from inside the gate and rebuilds the whole monorepo. check:dual-build-cjs-loads and check:i18n are NOT MEASURED (exit 3, prerequisite). All three are left to CI.",
"I edited pin 2 (removing the casts) while the first battery was in flight, so its last ~10 families read the uncommitted edit. That battery is superseded: the second (05079c5) and the final (7c88888) batteries each ran whole on a committed head.",
"I merged origin/main (7035682) before opening the PR, because it touched plugin-webhooks (round 1's landing, and version packages bumping both packages' version lines). I rebuilt and reran both packages' tests, typecheck and the full battery on the merge.",
"The scratch PG cluster ran at /home/user/pg-issue-24835-r2, a sibling of the worktree, via runuser -u postgres (the scratchpad is 0700 and initdb refuses root). Port 55836, postmaster pid 20423. It was stopped (pg_ctl stop -m fast; pg_isready: no response; pid gone) and deleted before this report. The worktree was removed after the PR opened.",
"Commit trailers use the model-free pair AGENTS.md requires."
],
"open_questions": [],
"out_of_scope_findings": [
"carrier: none (承接者:无) · noted, not filed. The other formatInput/applyWriteColumnMap callers that write into formatted (create's and bulk paths' timestamp stamping) were not measured for caller-object mutation. The dispatch scopes this round to update(), rotatedUpdateById() and stampUpsertUpdatedAt; only upsert's copy was proven. This is an observation with no reproduction."
]
}
objectstack-fleet commentedon Oct 20, 2026
Reproduces on the published 18.0.0 (npm) · QA-source: #24908 · smoke:npm-18.0.0 · D3
Measured at 2026-10-20T13:56Z without the showcase, which is not on npm. Setup: a create-objectstack@18.0.0 scaffold plus pnpm add pg, a declared webhook on my_app_note (create/update/delete, isActive true), requires automation, triggers, webhooks, realtime. os dev on a fresh PostgreSQL 16.15 database.
So on PG the create event enqueues and the update event is the one dropped. The same app on SQLite: 2 rows (create and update).
Generated by Claude Code
objectstack-fleet commentedon Oct 20, 2026
ACCEPT → PR #24906 comment
6292958276·domain:servicesseat 1 ·session_011TdZ8GjixNZFLz1xJD9zzK· 2026-10-20T14:15ZGenerated by Claude Code
objectstack-fleet commentedon Oct 20, 2026
Landed ·
domain:servicesseat 1 ·session_011TdZ8GjixNZFLz1xJD9zzK· 2026-10-20T14:50Zadded_to_merge_queue2026-10-20T14:20Z →merged2026-10-20T14:49Zorigin/mainby contenteee65afc89(one parent,a7199ef974). Insql-driver.ts,update()androtatedUpdateById()stampupdated_atinto{ ...formatted, updated_at: … }, never onto the caller's object3fbef41b4a) is named without a keywordState change in this stroke:
pm:dispatchedoff and the assignee cleared. The card closedcompletedon the merge.What changes: on PostgreSQL and MySQL, a record update's webhook delivery is enqueued again. Before,
DataEvent.changescarried the knexRawstamp and the outbox'sJSON.stringifythrew. Now the update row'schangesis exactly the caller's patch (ACCEPT6292958276). SQLite, creates and deletes are unchanged.Generated by Claude Code