Skip to content

Commit 3089848

Browse files
fix(core)!: a gracefulShutdown: false kernel no longer exits the process when its teardown times out (#22389)
Fixes #22335 Clause-②: no A kernel built with `gracefulShutdown: false` no longer calls `process.exit(1)` when its teardown overruns `shutdownTimeout`. It logs the timeout at `error`, marks itself `stopped`, and `shutdown()` resolves, so the host decides whether the process exits. A kernel with `gracefulShutdown: true` (the default) keeps the hard `exit(1)`, with the same log line as before. This implements the maintainer's ruling of record on #22335 (comment 6070759640, letter B): > **B.** `gracefulShutdown: false` means the kernel does not own the process: when its teardown times out it logs the error, marks itself `stopped`, and returns to the host, which decides whether the process exits. `gracefulShutdown: true` (the default; the kernel installed the signal listeners itself) keeps today's `exit(1)` on a genuine timeout. The #5274 pin keeps what it tests, on a `true` fixture; a new pin holds "`false` does not exit on timeout"; the JSDoc states both meanings of the option; the changeset names the host that relied on the old exit and what it must do (exit itself, as it already handles signals). ⛔ Not taken: A (the docs bent to the code, and one hung environment ends every environment in the hosted runtime), C (a second public switch for a split no measured user needs), D (a change to `shutdown()`'s never-throws contract for every host). No option, key or export is added, and `shutdown()` still never rejects. ## What changed - **`packages/core/src/kernel.ts`, `shutdown()`'s genuine-timeout branch.** The branch is still reached only by the identity-matched timeout error. It now reads `this.config.gracefulShutdown`, the same test the constructor uses to install the signal listeners, so a kernel exits on a timeout exactly when it took the signals itself. - `true`: the old code, unchanged. It logs `Shutdown timed out — forcing exit`, destroys the logger and calls `process.exit(1)`. - `false`: it logs at `error` and falls through to the `finally`. The log line names the timeout in ms and says that the teardown is still running in the background, that `gracefulShutdown` is false, that the process is NOT being exited, and that the host decides. - The non-timeout branch is unchanged. - **`ObjectKernelConfig.gracefulShutdown` JSDoc** now states both meanings: who installs the signal listeners, and who exits the process when a teardown times out. It also says that the hung teardown is not cancelled and keeps holding what it has not released. - **`packages/core/src/kernel.test.ts`.** The #5274 timeout pin ("still logs the timeout and still forces exit(1) when shutdown genuinely times out") keeps its title and assertions; only its fixture moves to `gracefulShutdown: true`. Because a `true` kernel listens on the worker's own process, the pin now removes any signal listener it added in its `finally`. - **New `packages/core/src/kernel-shutdown-timeout-ownership.test.ts`** (4 tests): - `false` plus a genuine timeout: no exit, `stopped`, `shutdown()` resolves, exactly one error-level timeout line carrying an `Error`, and no `forcing exit` line. The hung teardown is then released: it resumes in the background (`teardown-resumed`, `destroy`), the kernel stays `stopped`, there is still no exit, and a second `shutdown()` is the already-stopped no-op. - R7's shape: two `false` kernels in one process; stopping the hung one leaves the sibling `running` with no exit, and the sibling still stops normally. - Two controls under `false`: a throwing `destroy()` (isolated inside the teardown), and an error escaping the teardown (the catch's non-timeout branch). Neither exits, and neither is reported as a timeout. - **`.changeset/22335-graceful-shutdown-false-no-exit.md`**: `@objectstack/core` `minor` with the **BREAKING** banner, under the launch-window convention, with ADR-0087 disposition `not-required (no-migration-prescription)`. It names the host that relied on the old exit (a `gracefulShutdown: false` host: a server framework with its own signal handlers, or one kernel per environment in one process) and the remedy: exit the process yourself once `await kernel.shutdown()` returns, as you already do for signals. ## Before and after (in process, `process.exit` stubbed) Probe: two kernels of the same setting per row, `shutdownTimeout: 20`. The first kernel's teardown either hangs (a `kernel:shutdown` subscriber that never settles) or throws an error that escapes the teardown. The host stops the first kernel. "Other kernel" is the second kernel's state when `exit` was called, or after `shutdown()` returned if it was not. | Row | Before: exit calls (base `16096e8d7b`) | Before: other kernel | After: exit calls (`030a795001`) | After: other kernel | |:--|:--|:--|:--|:--| | `false`, teardown hangs | 1, `exit(1)` | `running` when the process exits | 0 | `running`, process alive | | `true`, teardown hangs | 1, `exit(1)` | `running` when the process exits | 1, `exit(1)` | `running` when the process exits | | `false`, teardown throws | 0 | `running` | 0 | `running` | In every row, before and after, the stopped kernel ends `stopped` and `shutdown()` resolves. ## Measurements behind the dispatch's assumptions - **M1, the defect: reproduced.** The first row above, at base `16096e8d7b`: `exit(1)` fires while the other `false` kernel is `running`. - **M2, who relied on the exit.** - In this repo, no production code builds a kernel with `gracefulShutdown: false`. `git grep -E "gracefulShutdown:\s*false" origin/main -- 'packages/**'` at `345d3f3d86` finds 49 files, all tests, and 0 non-test lines. - No test except the #5274 pin can reach the timeout branch. The only fixtures with a short or explicit `shutdownTimeout` are that pin (20 ms, now on `true`) and the #10604 guard pin (60 s, fake timers, its teardown completes). - Every production kernel in the repo takes the default (`true`), so its behaviour is unchanged: - CLI `serve` (`packages/cli/src/commands/serve.ts:3173`, `new Runtime({ kernel: { logger } })`); - every one-shot CLI command through `bootSchemaStack` (`packages/cli/src/utils/schema-migrate.ts:492`, `new Runtime({ cluster: false })`), which still calls `stack.shutdown()` and then exits; - `packages/objectql/src/kernel-factory.ts:35`; - `packages/verify/src/harness.ts:504`. - No CLI path can now hang where it used to exit. - The one measured producer of `false` is objectstack-ai/cloud's hosted runtime (`packages/objectos-runtime/src/artifact-kernel-factory.ts`, per the ruling's premise read). That path is not in this repo, and this session has no read access to objectstack-ai/cloud. **NOT MEASURED: what that host does after `shutdown()` returns.** The changeset names its shape and remedy. - **M3, the `true` path is unchanged.** Same branch body, same log line (`Shutdown timed out — forcing exit`), same `logger.destroy()` and then `process.exit(1)`. `handleShutdownSignal` is untouched. The #5274 pin on a `true` fixture is green, and ablation leg B below shows it depends on this branch. - **M4, the abandoned teardown.** After a `false` timeout, `performShutdown()` keeps running and the kernel does not act on it again: - It writes no state; `state` is assigned only in `shutdown()` and `bootstrap()`. - It releases no listeners; that happens only in those two methods. - `raceWithTimeout`'s `Promise.race` keeps a handler on the operation, so a later rejection is not unhandled. - The kernel's own logger, already destroyed in `finally`, writes later lines to stdout and stderr only (`fileStream` is cleared). - The first new test pins this: release the hung teardown after `shutdown()` resolved, and the kernel stays `stopped` with no exit. - The log line and the JSDoc both say that the hung teardown keeps running and that the host owns what follows. ## Tests (at `77e21f89c2`; the one commit after the code commit `030a795001` adds only the changeset) - `@objectstack/core`, `pnpm --filter @objectstack/core test`: 82 files, 2223 tests passed. `test:repo`: 3 files, 48 passed. - `pnpm --filter @objectstack/core typecheck`: exit 0. `check:test-typecheck` is OK with the debt ledger unchanged at 4. `tsc -p tsconfig.test.json --listFiles` compiles both touched test files. - `@objectstack/runtime` (its dependency closure built first, 32 projects): `vitest run --project local` gives 341 files, 4787 passed and 19 skipped. `--project repo` gives 3 files, 751 passed. - **Reverse verification (ablation)** on committed `030a795001`. Each leg ran through `scripts/ablation-replace.mjs` (anchor hit 1 to 0, blob changed on disk), then restored with `git checkout HEAD`; the restore was proven by the blob matching HEAD (`0bf80b6dc7ce`) and an empty `git diff HEAD`. The tests import `./kernel` from source, so no `dist/` is involved. - **Leg A**: the new condition forced to `if (true)`. Both `false` pins go red (`expected [ [ 1 ] ] to deeply equal []`), and the two controls and the #5274 pins stay green. - **Leg B**: the condition forced to `if (false)`. Only the #5274 timeout pin goes red (`expected [ Array(1) ] to include 'Shutdown timed out — forcing exit'`), and the `false` pins stay green. - **Lint, a declared narrowing.** - eslint `--no-inline-config --format json` on the three touched TS files: 3 files, 0 errors, 0 warnings. `--print-config` resolves a config for each, so they are inside the linted population. - This narrowing cannot change any verdict on an untouched file: `eslint.config.mjs` enables no type-aware linting (no `parserOptions.project` and no `projectService`, which it also states in prose). - The full `pnpm lint` belongs to CI. - **Gates.** - `node scripts/pm/dispatch-gates.mjs --commands` at `77e21f89c2` derived 63 commands, and each was run with its exit captured. - All 63 exited 0. `pnpm check:dual-build-cjs-loads` first exited 3, PREREQUISITE NOT MET, because it reads every package's `dist/`. It was re-run after a whole-tree `turbo run build` (72 tasks) at the same head: 107 published require entry points across 66 packages load. - `--ran` reconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN. ## Serial check with #22354 Re-fetched just before opening: `origin/main` is `117d34de3f`, and #22354 has not landed. Nothing in `packages/core` moved since base `16096e8d7b`, so this branch is not merged with main. Both merges are clean: - `git merge-tree --write-tree origin/main HEAD` exits 0 (tree `4c796f278a`). - `git merge-tree --write-tree` of #22354's head (`4e62f58ea0`) with this branch exits 0 (tree `8dead452b0`), and the merged `kernel.ts` carries both changes. This branch does not touch `bootstrap()` or `bootPhaseHooks`. ## Acceptance notes - `content/docs/protocol/kernel/lifecycle.mdx` is not edited. Its sentence (a `false` kernel "stays out of the process-management business") was false only on the timeout path, and is now true there too. - `content/docs/kernel/events.mdx` says `process.exit(1)` "is reserved for a genuine `shutdownTimeout` overrun". It is still true as an exclusivity statement, but a reader could take it to mean every overrun exits. Not edited (not in this card's file surface). Carrier: none. - The pending note `.changeset/22286-kernel-signal-listeners.md` listed under "Unchanged" that "A genuine shutdown timeout still calls `process.exit(1)`". Both notes ship in the same pre-mode release, so patch round 1 qualifies that sentence (see below). - `new ObjectKernel({ gracefulShutdown: undefined })`, an explicit `undefined`, overrides the spread default. Such a kernel installs no listeners and, after this PR, also does not exit on a timeout. That is consistent by construction, because both halves read one predicate. No producer in this repo passes a possibly-undefined value. - `shutdown()` resolves the same way whether the teardown finished or timed out; the timeout reaches the host only through the `error` log line. A host that wants a non-zero exit status after a hung teardown has no return-value signal, since option D was not taken. - Read, not measured: if a logger `file` is configured and a plugin logs through a child logger after a `false` timeout, that write meets the file stream the parent closed. The logger's own `error` handler disables file logging with one notice; it is not fatal. Under `true` this was unreachable, because the process exited. ## Patch round 1: #22286's pending note qualified Commit `c6c4351a82` changes one sentence in `.changeset/22286-kernel-signal-listeners.md`, in its "Unchanged" bullet, and nothing else in that file. It makes no code or test change. - Before: "A genuine shutdown timeout still calls `process.exit(1)`." - After: "With `gracefulShutdown: true`, a genuine shutdown timeout still calls `process.exit(1)`; a `gracefulShutdown: false` kernel no longer exits (#22335)." **Why.** `.changeset/pre.json` on `main` is in pre mode (`mode: pre`, tag `next`) and has consumed no changesets yet. So that note and this PR's `.changeset/22335-graceful-shutdown-false-no-exit.md` ship in the same release, and the unqualified sentence would be false there for a `gracefulShutdown: false` kernel. This is a deliberate correction of another card's pending note. `Check Changeset` stays red on it by design (the foreign-correction rule), and the seat's same-head contract review is the written confirmation. **Gates at `c6c4351a82`.** - `dispatch-gates --commands` derived the same 63 commands as before. It warned that the tree is behind `origin/main` `c8c803c293`, where two gate inputs changed (`scripts/migrate/overlay-views-to-sys-view-definition.md` and `scripts/platform-object-tenancy-census.json`). Nothing in `packages/core`, the two changesets or `.changeset/pre.json` moved on `main`, and #22354 has not landed, so this branch is not merged with main. - The whole tree was rebuilt first (`turbo run build`, 72 of 72 tasks cached), then each command ran with its exit captured. 62 exited 0. - `node scripts/check-empty-changeset.mjs --base origin/main` exited 1, as expected. It reports ".changeset/22286-kernel-signal-listeners.md: present on the merge base and CHANGED by this PR", the deliberate-correction class above. - `--ran` reconciliation: 63 derived, 63 run, 0 NOT-MEASURED, 0 UNRUN. --- _Generated by [Claude Code](https://claude.ai/code/session_01EUBvqtauTDmHi2ZgY759p2)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 11d119a commit 3089848

5 files changed

Lines changed: 240 additions & 13 deletions

File tree

‎.changeset/22286-kernel-signal-listeners.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,4 +9,4 @@ Clause-②: no
99
- **What was wrong.** With `gracefulShutdown: true` (the default) the constructor added one listener per signal and kept no handle, so `shutdown()` never removed them. Every kernel a process built left three listeners behind, and each one called `process.exit(0)` on the next signal. A process that had built and stopped five kernels and still ran a sixth exited six times on one SIGTERM, and the first exit landed before the live kernel had drained. Eleven built-and-stopped kernels also tripped Node's `MaxListenersExceededWarning`.
1010
- **What a kernel does now.** It holds its listeners from construction until it reaches `stopped`, and removes them at that point, whichever way it got there: `shutdown()` completing, the shutdown timeout, or a failed `bootstrap()`. A stopped kernel has no listener, so it cannot handle a signal. While a kernel is `stopping` its listener stays installed, so a repeated signal is absorbed and does not trigger Node's default action mid-drain.
1111
- **Who exits the process.** Only the handler whose signal started the drain, and only once. A signal that arrives while the host is already stopping the kernel through `shutdown()` is logged and left to the host. The kernel no longer exits the process in that case.
12-
- **Unchanged.** A running kernel still drains and exits with code 0 on SIGINT, SIGTERM or SIGQUIT (code 1 if `shutdown()` throws). `gracefulShutdown: false` still installs nothing. A genuine shutdown timeout still calls `process.exit(1)`.
12+
- **Unchanged.** A running kernel still drains and exits with code 0 on SIGINT, SIGTERM or SIGQUIT (code 1 if `shutdown()` throws). `gracefulShutdown: false` still installs nothing. With `gracefulShutdown: true`, a genuine shutdown timeout still calls `process.exit(1)`; a `gracefulShutdown: false` kernel no longer exits (#22335).
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
---
2+
'@objectstack/core': minor
3+
---
4+
5+
fix(core)!: a `gracefulShutdown: false` kernel no longer exits the process when its teardown times out
6+
7+
Clause-②: no
8+
9+
<!-- adr-0087: not-required (no-migration-prescription) No metadata moves: no spec key, authorable spelling, export, option, type or stored shape is added, removed, renamed or re-shaped, and no stored row is read, rewritten or converted, so there is nothing for `objectstack migrate meta` to rewrite. What changes is a runtime behaviour of `ObjectKernel.shutdown()` under an existing option: on a teardown timeout a `gracefulShutdown: false` kernel returns to its host instead of calling `process.exit(1)`. The other categories are closed on facts: `@objectstack/core` publishes (not unpublished); no ADR-0087 id is named or touched (not registered or already-registered); and no TypeScript declaration moves, only a JSDoc comment (not runtime-interface-only or type-surface-only). -->
10+
11+
**BREAKING** for a host that builds `ObjectKernel` with `gracefulShutdown: false` and counted on the kernel to end the process when a teardown hung: a behaviour change, shipped as `minor` under the launch-window convention for breaking changes.
12+
13+
- **What changes.** When `shutdown()` overruns `shutdownTimeout`, a kernel built with `gracefulShutdown: false` no longer calls `process.exit(1)`. It logs the timeout at `error`, marks itself `stopped`, and `shutdown()` resolves, so the host decides whether the process exits. The hung teardown is not cancelled: it keeps running in the background and holds whatever it has not released until it finishes or the process ends.
14+
- **Why.** `gracefulShutdown: false` already meant that the host, not the kernel, handles the process signals, and the kernel lifecycle docs say such a kernel stays out of the process-management business. The timeout branch did not read the option. So a host that runs several `false` kernels in one process, one per environment, lost all of them when one environment's teardown hung.
15+
- **Unchanged.** With `gracefulShutdown: true` (the default; the kernel installs its own SIGINT, SIGTERM and SIGQUIT listeners) a genuine timeout still logs `Shutdown timed out — forcing exit` and calls `process.exit(1)`. A teardown that throws instead of hanging still never exits the process, under either setting. `shutdown()` still never rejects, and no option, key or export is added. Every kernel `@objectstack/cli` builds takes the default, so its commands behave as before.
16+
- **Who must act.** A host that sets `gracefulShutdown: false` (a server framework with its own signal handlers, or a host running one kernel per environment in one process) and relied on that exit to end a process whose teardown hung. Such a host already handles the signals itself; it now also ends the process itself. Call `process.exit()` in that host once `await kernel.shutdown()` returns, instead of waiting for the event loop to drain: `shutdown()` returns within `shutdownTimeout` whether or not the teardown finished, but a hung teardown can keep the loop alive after it. The timeout is reported in the kernel's `error` log line.
Lines changed: 157 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,157 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* A kernel whose teardown times out exits the process only if it OWNS the
5+
* process (#22335, ruling B): `gracefulShutdown` true — the default, the kernel
6+
* installed the signal listeners itself — keeps the hard `exit(1)` (pinned in
7+
* kernel.test.ts, the #5274 timeout pin). `gracefulShutdown` false means the
8+
* host owns the process: the kernel logs the timeout, marks itself `stopped`,
9+
* and `shutdown()` returns to the host, which decides whether the process ends.
10+
*
11+
* Before this, the timeout branch called `process.exit(1)` without reading the
12+
* option, so a host running several `false` kernels in one process (one per
13+
* environment) lost every one of them when one teardown hung.
14+
*
15+
* `process.exit` is stubbed in every test here: the behaviour under test is
16+
* whether the kernel ends the process, and a real `exit(1)` would end the
17+
* vitest worker with it.
18+
*/
19+
20+
import { describe, it, expect, beforeEach, afterEach, vi } from 'vitest';
21+
import { ObjectKernel } from './kernel';
22+
import type { Plugin } from './types';
23+
24+
type ExitSpy = { mock: { calls: unknown[][] }; mockRestore: () => void };
25+
type LogMethod = (message: string, ...rest: unknown[]) => void;
26+
27+
const SHUTDOWN_TIMEOUT_MS = 20;
28+
29+
describe('ObjectKernel shutdown timeout under gracefulShutdown: false', () => {
30+
let exitSpy: ExitSpy;
31+
32+
beforeEach(() => {
33+
exitSpy = vi.spyOn(process, 'exit').mockImplementation((() => undefined) as never) as unknown as ExitSpy;
34+
});
35+
36+
afterEach(() => {
37+
exitSpy.mockRestore();
38+
});
39+
40+
const hostOwnedKernel = () =>
41+
new ObjectKernel({
42+
logger: { level: 'silent' },
43+
gracefulShutdown: false,
44+
skipSystemValidation: true,
45+
shutdownTimeout: SHUTDOWN_TIMEOUT_MS,
46+
});
47+
48+
const spyOnError = (kernel: ObjectKernel) =>
49+
vi.spyOn((kernel as unknown as { logger: { error: LogMethod } }).logger, 'error');
50+
51+
/** A plugin whose `kernel:shutdown` subscriber holds teardown open until released. */
52+
const hungTeardown = (reached: string[]) => {
53+
let release!: () => void;
54+
const gate = new Promise<void>((resolve) => { release = resolve; });
55+
const plugin: Plugin = {
56+
name: 'hung-teardown',
57+
version: '1.0.0',
58+
init: async (ctx) => {
59+
ctx.hook('kernel:shutdown', async () => {
60+
await gate;
61+
reached.push('teardown-resumed');
62+
});
63+
},
64+
destroy: async () => { reached.push('destroy'); },
65+
};
66+
return { plugin, release: () => release() };
67+
};
68+
69+
it('logs the timeout at error, ends stopped, resolves and does NOT exit the process', async () => {
70+
const reached: string[] = [];
71+
const { plugin, release } = hungTeardown(reached);
72+
const kernel = hostOwnedKernel();
73+
const errorSpy = spyOnError(kernel);
74+
await kernel.use(plugin);
75+
await kernel.bootstrap();
76+
77+
try {
78+
await expect(kernel.shutdown()).resolves.toBeUndefined();
79+
80+
expect(exitSpy.mock.calls).toEqual([]);
81+
expect(kernel.getState()).toBe('stopped');
82+
const timeoutLines = errorSpy.mock.calls.filter((c) => String(c[0]).startsWith('Shutdown timed out'));
83+
expect(timeoutLines).toHaveLength(1);
84+
expect(timeoutLines[0][1]).toBeInstanceOf(Error);
85+
// The `true` line says an exit is being forced; under `false` none is.
86+
expect(errorSpy.mock.calls.map((c) => String(c[0]))).not.toContain('Shutdown timed out — forcing exit');
87+
// The teardown was abandoned, not finished.
88+
expect(reached).toEqual([]);
89+
} finally {
90+
release();
91+
}
92+
93+
// The hung teardown keeps running in the background. When it finishes,
94+
// the kernel does not act on it: still `stopped`, still no exit, and a
95+
// second `shutdown()` is the already-stopped no-op.
96+
await vi.waitFor(() => expect(reached).toEqual(['teardown-resumed', 'destroy']));
97+
expect(kernel.getState()).toBe('stopped');
98+
await expect(kernel.shutdown()).resolves.toBeUndefined();
99+
expect(exitSpy.mock.calls).toEqual([]);
100+
errorSpy.mockRestore();
101+
});
102+
103+
it('two host-owned kernels in one process: stopping the hung one leaves the other running and the process alive', async () => {
104+
const reached: string[] = [];
105+
const { plugin, release } = hungTeardown(reached);
106+
const hung = hostOwnedKernel();
107+
const sibling = hostOwnedKernel();
108+
await hung.use(plugin);
109+
await hung.bootstrap();
110+
await sibling.bootstrap();
111+
112+
try {
113+
await hung.shutdown();
114+
115+
expect(hung.getState()).toBe('stopped');
116+
expect(sibling.getState()).toBe('running');
117+
expect(exitSpy.mock.calls).toEqual([]);
118+
119+
// The sibling is untouched: the host can still stop it normally.
120+
await sibling.shutdown();
121+
expect(sibling.getState()).toBe('stopped');
122+
expect(exitSpy.mock.calls).toEqual([]);
123+
} finally {
124+
release();
125+
}
126+
});
127+
128+
// Control: a teardown that FAILS rather than hangs never exited the process
129+
// under `false`, before this change or after it. Both shapes: a throwing
130+
// `destroy()` (isolated inside the teardown) and an error escaping the
131+
// teardown altogether (the non-timeout branch of `shutdown()`'s catch).
132+
it.each([
133+
['a throwing destroy()', (kernel: ObjectKernel) =>
134+
kernel.use({
135+
name: 'throwing-destroy',
136+
version: '1.0.0',
137+
init: async () => {},
138+
destroy: async () => { throw new Error('destroy boom'); },
139+
})],
140+
['an error escaping the teardown', async (kernel: ObjectKernel) => {
141+
vi.spyOn(kernel as unknown as { performShutdown: () => Promise<void> }, 'performShutdown')
142+
.mockRejectedValue(new Error('teardown boom'));
143+
}],
144+
] as const)('control: %s does not exit the process and is not reported as a timeout', async (_label, arrange) => {
145+
const kernel = hostOwnedKernel();
146+
const errorSpy = spyOnError(kernel);
147+
await arrange(kernel);
148+
await kernel.bootstrap();
149+
150+
await expect(kernel.shutdown()).resolves.toBeUndefined();
151+
152+
expect(exitSpy.mock.calls).toEqual([]);
153+
expect(kernel.getState()).toBe('stopped');
154+
expect(errorSpy.mock.calls.some((c) => String(c[0]).startsWith('Shutdown timed out'))).toBe(false);
155+
errorSpy.mockRestore();
156+
});
157+
});

‎packages/core/src/kernel.test.ts‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -966,10 +966,17 @@ describe('ObjectKernel', () => {
966966
// making progress, so the process would hang holding whatever it failed
967967
// to release. This behaviour is unchanged by #5274 — pinned so the
968968
// narrowing of the catch cannot quietly take it along.
969+
//
970+
// The fixture owns the process (`gracefulShutdown: true`, the default):
971+
// only a kernel that installed the signal listeners itself exits on a
972+
// timeout (#22335). The `false` half — no exit, the host decides — is
973+
// pinned in kernel-shutdown-timeout-ownership.test.ts.
969974
it('still logs the timeout and still forces exit(1) when shutdown genuinely times out (#5274)', async () => {
975+
const signals = ['SIGINT', 'SIGTERM', 'SIGQUIT'] as const;
976+
const listenersBefore = new Map(signals.map((s) => [s, process.listeners(s)]));
970977
const slowKernel = new ObjectKernel({
971978
logger: { level: 'error' },
972-
gracefulShutdown: false,
979+
gracefulShutdown: true,
973980
skipSystemValidation: true,
974981
shutdownTimeout: 20,
975982
});
@@ -1000,6 +1007,13 @@ describe('ObjectKernel', () => {
10001007
} finally {
10011008
exitSpy.mockRestore();
10021009
errorSpy.mockRestore();
1010+
// A `true` kernel listens on the worker's own process; never
1011+
// leave its listeners behind, whatever the test did.
1012+
for (const s of signals) {
1013+
for (const l of process.listeners(s)) {
1014+
if (!listenersBefore.get(s)!.includes(l)) process.removeListener(s, l);
1015+
}
1016+
}
10031017
}
10041018
}, 5000);
10051019

‎packages/core/src/kernel.ts‎

Lines changed: 51 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -80,10 +80,29 @@ export interface ObjectKernelConfig {
8080
defaultStartupTimeout?: number;
8181

8282
/**
83-
* Whether to enable graceful shutdown. When true (the default) the kernel
84-
* listens for SIGINT, SIGTERM and SIGQUIT from construction until it stops:
85-
* a signal drains it and then exits the process. Its listeners are removed
86-
* when it reaches `stopped`, so a stopped kernel never handles a signal.
83+
* Whether this kernel owns the process. It decides two things, always
84+
* together:
85+
*
86+
* - **Who installs the signal listeners.** When true (the default) the
87+
* kernel listens for SIGINT, SIGTERM and SIGQUIT from construction until
88+
* it stops: a signal drains it and then exits the process. Its listeners
89+
* are removed when it reaches `stopped`, so a stopped kernel never
90+
* handles a signal. When false the kernel installs none, and the host
91+
* handles signals itself.
92+
* - **Who exits the process when a teardown times out.** When `shutdown()`
93+
* overruns `shutdownTimeout`, a kernel with this option true logs the
94+
* timeout and calls `process.exit(1)`. A kernel with it false does NOT
95+
* exit the process: it logs the timeout at `error`, marks itself
96+
* `stopped`, and `shutdown()` returns to the host, which decides whether
97+
* the process exits. The hung teardown is not cancelled — it keeps
98+
* running in the background and holds whatever it has not released yet,
99+
* until it finishes or the host ends the process.
100+
*
101+
* Set it false when something else owns the process: a server framework
102+
* that installs its own signal handlers, or a host that runs several
103+
* kernels in one process and must not lose all of them when one teardown
104+
* hangs. Such a host exits the process itself when it decides to.
105+
* `shutdown()` never rejects under either setting.
87106
*/
88107
gracefulShutdown?: boolean;
89108

@@ -688,13 +707,34 @@ export class ObjectKernel {
688707

689708
if (error === shutdownTimeoutError) {
690709
// GENUINE timeout: `performShutdown()` is still running and has
691-
// stopped making progress, so the process would otherwise hang
692-
// holding whatever it failed to release. Hard-exit stays — it
693-
// is the only branch it was ever right for.
694-
this.logger.error('Shutdown timed out — forcing exit', error as Error);
695-
// Flush logger then hard-exit; the process would otherwise hang
696-
await this.logger.destroy();
697-
process.exit(1);
710+
// stopped making progress. Whether that ends the process
711+
// depends on who owns it, and the test is the one the
712+
// constructor uses to install the signal listeners, so a kernel
713+
// exits on a timeout exactly when it took the signals itself
714+
// (`ObjectKernelConfig.gracefulShutdown`).
715+
if (this.config.gracefulShutdown) {
716+
// The kernel owns the process, which would otherwise hang
717+
// holding whatever the teardown failed to release.
718+
// Hard-exit stays — it is the only branch it was ever
719+
// right for.
720+
this.logger.error('Shutdown timed out — forcing exit', error as Error);
721+
// Flush logger then hard-exit; the process would otherwise hang
722+
await this.logger.destroy();
723+
process.exit(1);
724+
} else {
725+
// The host owns the process. Exiting here would end every
726+
// other kernel and workload it runs over one hung
727+
// teardown, so the kernel reports, stays `stopped` and
728+
// returns. Nothing here acts on the abandoned teardown
729+
// again: `performShutdown()` changes no state when it
730+
// finishes, and the kernel's own later log lines from it
731+
// reach the console only, the file stream having been
732+
// closed in `finally`.
733+
this.logger.error(
734+
`Shutdown timed out after ${this.config.shutdownTimeout}ms — the kernel is stopped but its teardown is still running in the background and may still hold what it has not released; gracefulShutdown is false, so the host owns the process and it is NOT being exited — the host decides whether it exits`,
735+
error as Error,
736+
);
737+
}
698738
} else {
699739
// NOT a timeout. `performShutdown()` isolates every teardown
700740
// step it owns (hook dispatch, each destroy(), each shutdown

0 commit comments

Comments
 (0)