TLS: a renewed certificate is only served after a full restart — reload cert/key without dropping connections (SIGHUP or file change) #189

Activity

1. egeominotti

   added a commit that references this issue on Nov 12, 2026

   feat(tls): reload the certificate on SIGHUP without a restart (#189)

   ...

   da28921
2. egeominotti commented on Nov 13, 2026

   

   egeominotti

   on Nov 13, 2026

   Owner

   More actions

   Thanks for the report, and for the Bun probe: it was the basis of the implementation. This shipped in 2.9.22 (release, release post), on npm, in the Docker images and in the binaries.

   What you get

   With TLS configured, SIGHUP re-reads the cert and key from the paths the server was started with and serves them without a restart:

   # replace both files (write a sibling, then rename it over the live path), then
   kill -HUP <pid>

   • Ask 1, reload without a restart: new handshakes on both ports (TCP and HTTP: https://, wss://, SSE) get the renewed certificate. Connections that are already open stay on the socket and the certificate they were opened with; the regression tests keep a TLS TCP connection and a WebSocket open across the swap.
   • Ask 2, keep the current certificate on a bad pair: a pair that cannot be loaded (key does not match, missing or unreadable file, not PEM) is refused before any listener changes, the process keeps running, and it logs TLS reload failed; still serving the previous certificate with the reason. The reload is all-or-nothing across the two listeners.
   • Ask 3, one log line per reload: TLS certificate reloaded with the new certificate's subject, notAfter and fingerprint256, so a deploy hook can confirm it took effect.

   It works the way your probe showed: Bun ignores reload({ tls }), so each listener is replaced by a new one bound on the same port with SO_REUSEPORT, and the old one stops accepting without closing its sockets. Two things on top of that:

   • A TLS listener first makes a plain exclusive probe bind at startup, so SO_REUSEPORT does not let a second broker silently share the port.
   • The HTTP unix-socket variant, which cannot be shared, is rebound in place (keeping the socket file's mode and owner) and goes back to the previous pair if the new bind fails.

   Guide: Renewing certificates.

   Things to know for your setup

   • Docker and the restart policy. The official images declare no stop signal, so Docker records docker kill -s HUP as a manual stop: the reload works, but the restart policy will not restart the container after its next exit. To keep the policy, reload from inside the container (docker exec <container> kill -HUP 1 on alpine, docker exec <container> sh -c 'kill -HUP 1' on debian and slim), or run the container with --stop-signal SIGTERM / Compose stop_signal: SIGTERM (the only option on distroless). Details: Reloading in Docker.
   • Linux, during the swap. Both listeners are bound for an instant; a TCP handshake that completed in the old listener's accept queue but was not accepted before it closes is reset, and the client reconnects. Established connections are never affected.
   • No file watching. The reload is signal-only: nothing changes until the server gets SIGHUP, so a half-written pair is never picked up by itself. A SIGHUP that lands between the two file writes is refused as a mismatch and the next one takes the complete pair.
   • Without TLS, SIGHUP is unchanged. I did not add the explicit handler you suggested as a fallback: installing any handler would override an inherited "ignore" (nohup), and a non-TLS broker started that way would begin to stop on an SSH logout. With TLS the signal now reloads; without it the server installs no handler, exactly as before.
   • Changing the cert or key path still needs a restart.

   The full list of behavior changes is in the 2.9.22 upgrade notes.

   I am leaving this open: if you can run your repro (and your real deploy hook) against 2.9.22, I would like to know whether it behaves as expected in your container setup, in particular the restart-policy point above.
3. egeominotti

   self-assigned this

   on Nov 13, 2026
4. assantech commented on Nov 14, 2026

   

   assantech

   on Nov 14, 2026

   Author

   More actions

   Confirmed on 2.9.22 (image ghcr.io/egeominotti/bunqueue:2.9.22, client 2.9.22, Bun 1.4.4, Docker 29.5.3) — thank you, all three asks are there and the note on the restart policy saved us a mistake: our hook was going to be docker kill -s HUP.

   The repro from the issue, unchanged. Step 3 used to be process exited (code 129, signal SIGHUP):

   1 started with cert A          serves: wire A (old), http A (old)
   2 cert+key replaced on disk    serves: wire A (old), http A (old)   (file on disk: B (new), 5 s later)
     clients so far               producer 70 adds ok / 0 failed, worker 70 jobs / 0 error events
   3 SIGHUP                       process still running; serves: wire B (new), http B (new)
   

   Bare process, a wider check (35/35):

   • The reload is logged 20 ms after the signal, same pid; the line carries the new certificate's fingerprint256 and notAfter.
   • A producer (one add every 100 ms) and a worker connected for the whole run: 0 failed adds, 0 worker error events, every job consumed. A raw TLS socket and a wss://…/ws subscription opened before the reload stay open, and the WebSocket keeps receiving events.
   • Refused, logged, previous certificate on both ports, process alive: the key of another certificate (a SIGHUP between the two writes), a missing key file, a cert file that is not PEM. The next good pair loads.
   • 20 reloads in a row under the same load: one listener per port at the end (ss -ltn), the process's fd count unchanged (16 → 16), still 0 failed adds.
   • A new client pinned to the replaced certificate is refused, one pinned to the served one connects.
   • A second broker on the same ports fails to start (Failed to listen at 0.0.0.0), so SO_REUSEPORT does not let it in. Without TLS SIGHUP still ends the process (129), as documented. SIGTERM after the reloads exits 0 and frees both ports.

   Our container setup (26/26). The broker runs with Compose init: true, so PID 1 is docker-init and bunqueue is its child (we need it to reap the ssl_client orphans of the https healthcheck), plus cap_drop: ALL, no-new-privileges, restart: unless-stopped, the certificate directory mounted read-only:

   • docker exec <container> kill -HUP 1 works in that shape too: docker-init forwards the signal, both listeners serve the new certificate, StartedAt and RestartCount do not change, the healthcheck stays healthy, a producer and a worker connected through the published port lose nothing. Maybe worth one line in the guide: with --init PID 1 is not bunqueue and the command is still the right one.
   • Restart policy, as you describe it. After the reload from inside, a crashed broker (kill -9 of the bunqueue process) is restarted by unless-stopped. After docker kill -s HUP <container> the reload works as well, but the same crash leaves the container exited: Docker took the signal for a manual stop.
   • No zombie and a flat pids count after the reloads and the healthcheck probes; docker stop ends the broker in 0.5 s, exit code 0.

   Our real deploy hook now does this at a renewal: copy cert and key (write aside, rename over), docker exec <container> kill -HUP 1, then wait until both ports serve the certificate that is on disk (openssl s_client on loopback, fingerprint compared with the file) and fail with your TLS reload failed… line otherwise. It repeats the comparison at every run of its timer, so a signal that was lost (SIGHUP ignored: …) is sent again. On a mismatched pair it exits 1 and the broker keeps serving, which is what we wanted from ask 2.

   One thing for whoever updates the hook before the image: on 2.9.20 the same kill -HUP 1 ends the broker and the restart policy brings it back, so the image has to reach the host first.

   The rest of our regression set is green on 2.9.22 as well (state enumeration over TCP and HTTP, #97/#110, #98, #100, #101, #102, #113, #118, #155, TLS, reconnect after a 90 s outage and after a half-open link); first boot of 2.9.22 on a 90 MB database written by 2.9.20: no migration, healthy in 0.44 s. A 2.9.20 client against the 2.9.22 broker, our rollout window, behaves the same. Our pin moves from 2.9.20 to 2.9.22; fine to close from our side.

   👍React with 👍1Reacted by Egeo Minotti