Repository navigation
Commit 3089848
fix(core)!: a gracefulShutdown: false kernel no longer exits the process when its teardown times out (#22389)
Fixes #22335
Clause-②: no
A kernel built with `gracefulShutdown: false` no longer calls
`process.exit(1)` when its teardown overruns `shutdownTimeout`. It logs
the timeout at `error`, marks itself `stopped`, and `shutdown()`
resolves, so the host decides whether the process exits. A kernel with
`gracefulShutdown: true` (the default) keeps the hard `exit(1)`, with
the same log line as before.
This implements the maintainer's ruling of record on #22335 (comment
6070759640, letter B):
> **B.** `gracefulShutdown: false` means the kernel does not own the
process: when its teardown times out it logs the error, marks itself
`stopped`, and returns to the host, which decides whether the process
exits. `gracefulShutdown: true` (the default; the kernel installed the
signal listeners itself) keeps today's `exit(1)` on a genuine timeout.
The #5274 pin keeps what it tests, on a `true` fixture; a new pin holds
"`false` does not exit on timeout"; the JSDoc states both meanings of
the option; the changeset names the host that relied on the old exit and
what it must do (exit itself, as it already handles signals). ⛔ Not
taken: A (the docs bent to the code, and one hung environment ends every
environment in the hosted runtime), C (a second public switch for a
split no measured user needs), D (a change to `shutdown()`'s
never-throws contract for every host).
No option, key or export is added, and `shutdown()` still never rejects.
## What changed
- **`packages/core/src/kernel.ts`, `shutdown()`'s genuine-timeout
branch.** The branch is still reached only by the identity-matched
timeout error. It now reads `this.config.gracefulShutdown`, the same
test the constructor uses to install the signal listeners, so a kernel
exits on a timeout exactly when it took the signals itself.
- `true`: the old code, unchanged. It logs `Shutdown timed out — forcing
exit`, destroys the logger and calls `process.exit(1)`.
- `false`: it logs at `error` and falls through to the `finally`. The
log line names the timeout in ms and says that the teardown is still
running in the background, that `gracefulShutdown` is false, that the
process is NOT being exited, and that the host decides.
- The non-timeout branch is unchanged.
- **`ObjectKernelConfig.gracefulShutdown` JSDoc** now states both
meanings: who installs the signal listeners, and who exits the process
when a teardown times out. It also says that the hung teardown is not
cancelled and keeps holding what it has not released.
- **`packages/core/src/kernel.test.ts`.** The #5274 timeout pin ("still
logs the timeout and still forces exit(1) when shutdown genuinely times
out") keeps its title and assertions; only its fixture moves to
`gracefulShutdown: true`. Because a `true` kernel listens on the
worker's own process, the pin now removes any signal listener it added
in its `finally`.
- **New `packages/core/src/kernel-shutdown-timeout-ownership.test.ts`**
(4 tests):
- `false` plus a genuine timeout: no exit, `stopped`, `shutdown()`
resolves, exactly one error-level timeout line carrying an `Error`, and
no `forcing exit` line. The hung teardown is then released: it resumes
in the background (`teardown-resumed`, `destroy`), the kernel stays
`stopped`, there is still no exit, and a second `shutdown()` is the
already-stopped no-op.
- R7's shape: two `false` kernels in one process; stopping the hung one
leaves the sibling `running` with no exit, and the sibling still stops
normally.
- Two controls under `false`: a throwing `destroy()` (isolated inside
the teardown), and an error escaping the teardown (the catch's
non-timeout branch). Neither exits, and neither is reported as a
timeout.
- **`.changeset/22335-graceful-shutdown-false-no-exit.md`**:
`@objectstack/core` `minor` with the **BREAKING** banner, under the
launch-window convention, with ADR-0087 disposition `not-required
(no-migration-prescription)`. It names the host that relied on the old
exit (a `gracefulShutdown: false` host: a server framework with its own
signal handlers, or one kernel per environment in one process) and the
remedy: exit the process yourself once `await kernel.shutdown()`
returns, as you already do for signals.
## Before and after (in process, `process.exit` stubbed)
Probe: two kernels of the same setting per row, `shutdownTimeout: 20`.
The first kernel's teardown either hangs (a `kernel:shutdown` subscriber
that never settles) or throws an error that escapes the teardown. The
host stops the first kernel. "Other kernel" is the second kernel's state
when `exit` was called, or after `shutdown()` returned if it was not.
| Row | Before: exit calls (base `16096e8d7b`) | Before: other kernel |
After: exit calls (`030a795001`) | After: other kernel |
|:--|:--|:--|:--|:--|
| `false`, teardown hangs | 1, `exit(1)` | `running` when the process
exits | 0 | `running`, process alive |
| `true`, teardown hangs | 1, `exit(1)` | `running` when the process
exits | 1, `exit(1)` | `running` when the process exits |
| `false`, teardown throws | 0 | `running` | 0 | `running` |
In every row, before and after, the stopped kernel ends `stopped` and
`shutdown()` resolves.
## Measurements behind the dispatch's assumptions
- **M1, the defect: reproduced.** The first row above, at base
`16096e8d7b`: `exit(1)` fires while the other `false` kernel is
`running`.
- **M2, who relied on the exit.**
- In this repo, no production code builds a kernel with
`gracefulShutdown: false`. `git grep -E "gracefulShutdown:\s*false"
origin/main -- 'packages/**'` at `345d3f3d86` finds 49 files, all tests,
and 0 non-test lines.
- No test except the #5274 pin can reach the timeout branch. The only
fixtures with a short or explicit `shutdownTimeout` are that pin (20 ms,
now on `true`) and the #10604 guard pin (60 s, fake timers, its teardown
completes).
- Every production kernel in the repo takes the default (`true`), so its
behaviour is unchanged:
- CLI `serve` (`packages/cli/src/commands/serve.ts:3173`, `new Runtime({
kernel: { logger } })`);
- every one-shot CLI command through `bootSchemaStack`
(`packages/cli/src/utils/schema-migrate.ts:492`, `new Runtime({ cluster:
false })`), which still calls `stack.shutdown()` and then exits;
- `packages/objectql/src/kernel-factory.ts:35`;
- `packages/verify/src/harness.ts:504`.
- No CLI path can now hang where it used to exit.
- The one measured producer of `false` is objectstack-ai/cloud's hosted
runtime (`packages/objectos-runtime/src/artifact-kernel-factory.ts`, per
the ruling's premise read). That path is not in this repo, and this
session has no read access to objectstack-ai/cloud. **NOT MEASURED: what
that host does after `shutdown()` returns.** The changeset names its
shape and remedy.
- **M3, the `true` path is unchanged.** Same branch body, same log line
(`Shutdown timed out — forcing exit`), same `logger.destroy()` and then
`process.exit(1)`. `handleShutdownSignal` is untouched. The #5274 pin on
a `true` fixture is green, and ablation leg B below shows it depends on
this branch.
- **M4, the abandoned teardown.** After a `false` timeout,
`performShutdown()` keeps running and the kernel does not act on it
again:
- It writes no state; `state` is assigned only in `shutdown()` and
`bootstrap()`.
- It releases no listeners; that happens only in those two methods.
- `raceWithTimeout`'s `Promise.race` keeps a handler on the operation,
so a later rejection is not unhandled.
- The kernel's own logger, already destroyed in `finally`, writes later
lines to stdout and stderr only (`fileStream` is cleared).
- The first new test pins this: release the hung teardown after
`shutdown()` resolved, and the kernel stays `stopped` with no exit.
- The log line and the JSDoc both say that the hung teardown keeps
running and that the host owns what follows.
## Tests (at `77e21f89c2`; the one commit after the code commit
`030a795001` adds only the changeset)
- `@objectstack/core`, `pnpm --filter @objectstack/core test`: 82 files,
2223 tests passed. `test:repo`: 3 files, 48 passed.
- `pnpm --filter @objectstack/core typecheck`: exit 0.
`check:test-typecheck` is OK with the debt ledger unchanged at 4. `tsc
-p tsconfig.test.json --listFiles` compiles both touched test files.
- `@objectstack/runtime` (its dependency closure built first, 32
projects): `vitest run --project local` gives 341 files, 4787 passed and
19 skipped. `--project repo` gives 3 files, 751 passed.
- **Reverse verification (ablation)** on committed `030a795001`. Each
leg ran through `scripts/ablation-replace.mjs` (anchor hit 1 to 0, blob
changed on disk), then restored with `git checkout HEAD`; the restore
was proven by the blob matching HEAD (`0bf80b6dc7ce`) and an empty `git
diff HEAD`. The tests import `./kernel` from source, so no `dist/` is
involved.
- **Leg A**: the new condition forced to `if (true)`. Both `false` pins
go red (`expected [ [ 1 ] ] to deeply equal []`), and the two controls
and the #5274 pins stay green.
- **Leg B**: the condition forced to `if (false)`. Only the #5274
timeout pin goes red (`expected [ Array(1) ] to include 'Shutdown timed
out — forcing exit'`), and the `false` pins stay green.
- **Lint, a declared narrowing.**
- eslint `--no-inline-config --format json` on the three touched TS
files: 3 files, 0 errors, 0 warnings. `--print-config` resolves a config
for each, so they are inside the linted population.
- This narrowing cannot change any verdict on an untouched file:
`eslint.config.mjs` enables no type-aware linting (no
`parserOptions.project` and no `projectService`, which it also states in
prose).
- The full `pnpm lint` belongs to CI.
- **Gates.**
- `node scripts/pm/dispatch-gates.mjs --commands` at `77e21f89c2`
derived 63 commands, and each was run with its exit captured.
- All 63 exited 0. `pnpm check:dual-build-cjs-loads` first exited 3,
PREREQUISITE NOT MET, because it reads every package's `dist/`. It was
re-run after a whole-tree `turbo run build` (72 tasks) at the same head:
107 published require entry points across 66 packages load.
- `--ran` reconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN.
## Serial check with #22354
Re-fetched just before opening: `origin/main` is `117d34de3f`, and
#22354 has not landed. Nothing in `packages/core` moved since base
`16096e8d7b`, so this branch is not merged with main. Both merges are
clean:
- `git merge-tree --write-tree origin/main HEAD` exits 0 (tree
`4c796f278a`).
- `git merge-tree --write-tree` of #22354's head (`4e62f58ea0`) with
this branch exits 0 (tree `8dead452b0`), and the merged `kernel.ts`
carries both changes.
This branch does not touch `bootstrap()` or `bootPhaseHooks`.
## Acceptance notes
- `content/docs/protocol/kernel/lifecycle.mdx` is not edited. Its
sentence (a `false` kernel "stays out of the process-management
business") was false only on the timeout path, and is now true there
too.
- `content/docs/kernel/events.mdx` says `process.exit(1)` "is reserved
for a genuine `shutdownTimeout` overrun". It is still true as an
exclusivity statement, but a reader could take it to mean every overrun
exits. Not edited (not in this card's file surface). Carrier: none.
- The pending note `.changeset/22286-kernel-signal-listeners.md` listed
under "Unchanged" that "A genuine shutdown timeout still calls
`process.exit(1)`". Both notes ship in the same pre-mode release, so
patch round 1 qualifies that sentence (see below).
- `new ObjectKernel({ gracefulShutdown: undefined })`, an explicit
`undefined`, overrides the spread default. Such a kernel installs no
listeners and, after this PR, also does not exit on a timeout. That is
consistent by construction, because both halves read one predicate. No
producer in this repo passes a possibly-undefined value.
- `shutdown()` resolves the same way whether the teardown finished or
timed out; the timeout reaches the host only through the `error` log
line. A host that wants a non-zero exit status after a hung teardown has
no return-value signal, since option D was not taken.
- Read, not measured: if a logger `file` is configured and a plugin logs
through a child logger after a `false` timeout, that write meets the
file stream the parent closed. The logger's own `error` handler disables
file logging with one notice; it is not fatal. Under `true` this was
unreachable, because the process exited.
## Patch round 1: #22286's pending note qualified
Commit `c6c4351a82` changes one sentence in
`.changeset/22286-kernel-signal-listeners.md`, in its "Unchanged"
bullet, and nothing else in that file. It makes no code or test change.
- Before: "A genuine shutdown timeout still calls `process.exit(1)`."
- After: "With `gracefulShutdown: true`, a genuine shutdown timeout
still calls `process.exit(1)`; a `gracefulShutdown: false` kernel no
longer exits (#22335)."
**Why.** `.changeset/pre.json` on `main` is in pre mode (`mode: pre`,
tag `next`) and has consumed no changesets yet. So that note and this
PR's `.changeset/22335-graceful-shutdown-false-no-exit.md` ship in the
same release, and the unqualified sentence would be false there for a
`gracefulShutdown: false` kernel. This is a deliberate correction of
another card's pending note. `Check Changeset` stays red on it by design
(the foreign-correction rule), and the seat's same-head contract review
is the written confirmation.
**Gates at `c6c4351a82`.**
- `dispatch-gates --commands` derived the same 63 commands as before. It
warned that the tree is behind `origin/main` `c8c803c293`, where two
gate inputs changed
(`scripts/migrate/overlay-views-to-sys-view-definition.md` and
`scripts/platform-object-tenancy-census.json`). Nothing in
`packages/core`, the two changesets or `.changeset/pre.json` moved on
`main`, and #22354 has not landed, so this branch is not merged with
main.
- The whole tree was rebuilt first (`turbo run build`, 72 of 72 tasks
cached), then each command ran with its exit captured. 62 exited 0.
- `node scripts/check-empty-changeset.mjs --base origin/main` exited 1,
as expected. It reports ".changeset/22286-kernel-signal-listeners.md:
present on the merge base and CHANGED by this PR", the
deliberate-correction class above.
- `--ran` reconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01EUBvqtauTDmHi2ZgY759p2)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent 11d119a commit 3089848
5 files changed
Lines changed: 240 additions & 13 deletions
File tree
- .changeset
- packages/core/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
| 12 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
966 | 966 | | |
967 | 967 | | |
968 | 968 | | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
969 | 974 | | |
| 975 | + | |
| 976 | + | |
970 | 977 | | |
971 | 978 | | |
972 | | - | |
| 979 | + | |
973 | 980 | | |
974 | 981 | | |
975 | 982 | | |
| |||
1000 | 1007 | | |
1001 | 1008 | | |
1002 | 1009 | | |
| 1010 | + | |
| 1011 | + | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
| 1015 | + | |
| 1016 | + | |
1003 | 1017 | | |
1004 | 1018 | | |
1005 | 1019 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
83 | | - | |
84 | | - | |
85 | | - | |
86 | | - | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
87 | 106 | | |
88 | 107 | | |
89 | 108 | | |
| |||
688 | 707 | | |
689 | 708 | | |
690 | 709 | | |
691 | | - | |
692 | | - | |
693 | | - | |
694 | | - | |
695 | | - | |
696 | | - | |
697 | | - | |
| 710 | + | |
| 711 | + | |
| 712 | + | |
| 713 | + | |
| 714 | + | |
| 715 | + | |
| 716 | + | |
| 717 | + | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
698 | 738 | | |
699 | 739 | | |
700 | 740 | | |
| |||
0 commit comments