Skip to content

fix(serve): shut down gracefully on SIGTERM - #3

Merged
autoantohaki merged 1 commit into
mainfrom
fix/serve-sigterm
Oct 22, 2026
Merged

autoantohaki merged 1 commit into
mainfrom
fix/serve-sigterm

Conversation

@autoantohaki

Copy link
Copy Markdown
Member

Requested by mau

Before: serve only listened for Ctrl+C. docker stop, kill, and service managers send SIGTERM, which skipped the graceful drain entirely. In the container the server runs as PID 1, where an unhandled SIGTERM is ignored, so docker stop hung for its full grace period and then SIGKILLed the process, abandoning in-flight requests and leaving the llama.cpp child to be killed without the bounded shutdown path.

After: on Unix, SIGTERM triggers the same bounded shutdown as Ctrl+C: HTTP and gRPC admission stops, in-flight work drains within --shutdown-timeout-ms, the private runtime is terminated, and the process exits 0.

How: a small shutdown_requested() helper in main.rs waits for either Ctrl+C or SIGTERM (Unix only; Windows keeps Ctrl+C). A new Unix-only binary test starts the real server, sends kill -TERM, and asserts a clean exit. The README container section mentions the behavior.

CI needs #1 merged first (it restores the privacy-scan and supply-chain checks on main).

@autoantohaki
autoantohaki merged commit e3f487e into main Oct 22, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant