Repository navigation
fix(docker): wait for tenderdash to exit after forwarding a signal - #1607
Conversation
A signal trapped by the entrypoint interrupts `wait`, so the script (PID 1) exited right after forwarding SIGTERM and the kernel killed tenderdash mid-shutdown. `docker stop` returned within milliseconds and the 30 s --shutdown-timeout never ran. The entrypoint now keeps waiting until tenderdash has exited and returns its exit code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
|
✅ Final review complete — no blockers (commit dd97674) · triage: low |
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
thepastaclaw
left a comment
There was a problem hiding this comment.
Final validation — Phase 2 only (queue backlog)
The signal-forwarding change correctly keeps the container process alive during Tenderdash shutdown, and all reported code checks passed. However, the liveness-gated retry has a race that can preserve Bash's signal-interrupted status instead of Tenderdash's final exit status, so the PR still has one blocking issue. The lifecycle-sensitive behavior also lacks a committed regression harness.
🔴 1 blocking | 🟡 1 suggestion(s)
Review provenance
Source: reviewer 1: gpt-6.1-sol (agent: phase2-reviewer, role: general); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
lowbygpt-6.1-sol(effort low) — This is a small, self-contained shell-script behavior fix in the Docker entrypoint, and its correctness is straightforward to verify without touching a critical code surface. - Phase 1 reviewers: not run (skipped for throughput: 13 PRs queued, above the 10 limit)
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort medium); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review (PastaClaw). How to work through them:
- Verify each finding against the code; fix only what is real, as narrowly as the finding.
- If a finding is wrong, decline it on its thread in one or two sentences; the bot answers once and stops.
- To defer one, link a tracking issue on its thread (or reply asking the bot to file one).
- Suggestions are optional: fix them forward in a later PR rather than pushing another round just for them.
- Address every current finding in one push: self-review the full diff first, and don't push piecemeal.
- Keep the PR's scope; don't grow it to satisfy hardening suggestions.
- Carried items were raised in an earlier round: read the thread first, and don't re-fix what you already fixed, declined or deferred there. Disputed and deferred items are already handled and are not listed.
In `DOCKER/docker-entrypoint.sh`:
- [BLOCKING] DOCKER/docker-entrypoint.sh:59-62: Collect the child status even when it exits before the liveness check
When a trapped signal interrupts the first `wait`, `exit_code` contains Bash's signal status (for example, 143), not Tenderdash's status. If Tenderdash exits and Bash reaps the child between that interrupted `wait` and `kill -0`, the liveness check fails and the loop is skipped, leaving 143 as the script's result even when Tenderdash exited successfully or with another status. The same race can occur after a later interrupted wait. Do not use a failed `kill -0` as proof that the preceding `wait` collected the child's final status; track whether each wait was interrupted and perform a final status-bearing wait after shutdown completion.
- [SUGGESTION] DOCKER/docker-entrypoint.sh:53-62: Commit a regression harness for signal-handling behavior
The entrypoint now contains lifecycle-sensitive signal and exit-status logic, but the regression harness described in the PR is not committed. The repository's existing e2e Docker entrypoints do not exercise this production entrypoint. Add a lightweight automated harness covering delayed cleanup, repeated signals, ordinary nonzero exits, and a child that exits immediately after receiving the signal, including the exit-status race identified above, so future changes cannot silently reintroduce premature exits or status loss.
…ignal handler The entrypoint checked `kill -0` to decide whether to wait again after a trapped signal interrupted `wait`. If tenderdash exited, and was reaped, between the interrupted wait and that check, the script exited with the interrupted wait's 143 instead of tenderdash's status. The signal handler now marks the wait as interrupted, and the script waits again until a wait completes uninterrupted; bash returns the saved status of a child that has already exited. DOCKER/test-entrypoint.sh checks a slow shutdown, a repeated signal, an immediate exit on the signal and a plain non-zero exit with a fake tenderdash; the release tooling workflow runs it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
thepastaclaw
left a comment
There was a problem hiding this comment.
Re-review — Final validation — Phase 1 + Phase 2
The entrypoint fix, regression harness, and release-workflow integration are correct, and the supplied CI snapshot shows all code checks succeeding. One test-robustness nitpick remains: a wedged entrypoint can make the harness wait indefinitely instead of failing within a bounded time; PR Hygiene and this review are still pending, but no code checks have failed.
💬 1 nitpick(s)
Approved: nothing here blocks the merge. The non-blocking findings are optional: fix them forward, in this PR or a follow-up, without waiting for another review round.
Review provenance
Source: reviewer 1: glm-5.3-flash (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); reviewer 3: gpt-6.1-sol (agent: phase2-reviewer, role: general); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
lowbygpt-6.1-sol(effort low) — This is a small, self-contained Docker entrypoint signal-handling fix with focused tests and no consensus, funds, cryptography, networking, or storage changes. - Phase 1 reviewers:
glm-5.3-flash— general (completed, effort high); agentphase1-reviewer - Phase 1 model:
glm-5.3-flash— zai quota: 5h 99% left, weekly 91% left; passed overgemini-3.8-flash-high(antigravity quota lookup failed) - Fresh final gate: an independent Phase-2 review ran after iterative findings were reconciled
- Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort medium); agentphase2-reviewer,gpt-6.1-sol— general (completed, effort medium); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review (PastaClaw). How to work through them:
- Verify each finding against the code; fix only what is real, as narrowly as the finding.
- If a finding is wrong, decline it on its thread in one or two sentences; the bot answers once and stops.
- To defer one, link a tracking issue on its thread (or reply asking the bot to file one).
- Suggestions are optional: fix them forward in a later PR rather than pushing another round just for them.
- Address every current finding in one push: self-review the full diff first, and don't push piecemeal.
- Keep the PR's scope; don't grow it to satisfy hardening suggestions.
- Carried items were raised in an earlier round: read the thread first, and don't re-fix what you already fixed, declined or deferred there. Disputed and deferred items are already handled and are not listed.
In `DOCKER/test-entrypoint.sh`:
- [NITPICK] DOCKER/test-entrypoint.sh:23-24: Bound each entrypoint run with timeout so a wedged entrypoint fails instead of hanging CI
`run_entrypoint` backgrounds the production entrypoint and then waits for it without a deadline. If a future signal-handling regression causes the entrypoint or its fake Tenderdash process to remain alive indefinitely, the release-tooling job hangs until its job-level timeout rather than reporting a focused test failure. Wrap the invocation with a generous timeout so a wedged run returns a failure status while preserving the existing signal-forwarding behavior.
|
Ready for review — |
…eout A wedged entrypoint now fails the check instead of hanging the job. --foreground keeps timeout from signalling the fake tenderdash directly, so the test still proves the entrypoint forwards the signal, and --preserve-status reports the entrypoint's own exit status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Waiting for bot review — thepastaclaw not yet. Wait for the missing reviews, or a writer can post |
thepastaclaw
left a comment
There was a problem hiding this comment.
Re-review — Final validation — Phase 1 + Phase 2
The signal-forwarding implementation is correct at the reviewed head, and the committed regression harness is wired into the release workflow. One maintainability suggestion remains: document the Bash saved-status behavior that makes the repeated wait safe after an interrupted wait.
🔴 0 blocking | 🟡 0 suggestion(s) | 💬 0 nitpick(s)
Approved: nothing here blocks the merge. The non-blocking findings are optional: fix them forward, in this PR or a follow-up, without waiting for another review round.
Follow-up candidates (optional) (1)
New non-blocking observations on code unchanged since the last full review. No inline threads are opened for them and later reviews do not repeat them: fix forward or track separately if they are worth it.
🟡 Suggestion: Document Bash's saved-status behavior in the re-wait loop
DOCKER/docker-entrypoint.sh:59-64
The loop is correct today because the entrypoint intentionally does not enable set -e and immediately captures wait's status. Its other non-obvious invariant is that Bash returns the saved child status when wait is called again after the child has already exited; documenting that behavior will help prevent a future maintainer from replacing the loop with a liveness check or otherwise reintroducing the status race.
while :; do
interrupted=false
# Bash re-reports the saved child status when wait is called again after
# the child has exited; the interrupted status is consumed below.
wait "$child"
exit_code=$?
[ "$interrupted" = true ] || break
done
source: glm-5.3-flash (phase1-reviewer: general)
Review provenance
Source: reviewer 1: glm-5.3-flash (agent: phase1-reviewer, role: general); reviewer 2: gpt-6.1-sol (agent: phase2-reviewer, role: general); final verifier: gpt-6.1-sol (agent: sol-verifier, role: final-verifier)
- Triage:
lowbygpt-6.1-sol(effort low) — This is a small, well-contained Docker entrypoint signal-handling change with focused tests and no critical consensus, funds, cryptography, or storage impact. - Phase 1 reviewers:
glm-5.3-flash— general (completed, effort high); agentphase1-reviewer - Phase 1 model:
glm-5.3-flash— zai quota: 5h 96% left, weekly 90% left; passed overgemini-3.8-flash-high(antigravity quota lookup failed) - Code unchanged since the last final review (
8370b72b): 1 new suggestion(s) listed as follow-up candidates, not posted inline - Fresh verifier:
gpt-6.1-sol— final-verifier; agentsol-verifier - Phase 2 reviewers:
gpt-6.1-sol— general (completed, effort medium); agentphase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify the current code and confirm that no unresolved issues remain.
No unresolved findings remain from the prior review on this head.
|
Policy satisfied — this can merge. |
TL;DR: Stopping a Tenderdash container now lets Tenderdash finish its shutdown instead of killing it halfway.
User story
As a node operator (for example through dashmate), I want
docker stopand restarts to shut Tenderdash down cleanly, to avoid every stop, restart or update behaving like a crash.Scenario
Base flow
An operator stops or restarts a Tenderdash container. Docker sends SIGTERM to the container's main process, the entrypoint script, which forwards it to Tenderdash.
Actual behavior
The entrypoint exits right after forwarding the signal, without waiting for Tenderdash. Because it is the container's main process, Docker then ends the container and Tenderdash is killed in the middle of its shutdown.
docker stop -t 40returns in about 0.2 s.Expected behavior
The entrypoint waits until Tenderdash has finished shutting down (up to its 30 s
--shutdown-timeout, within Docker's stop grace period) and exits with Tenderdash's exit code.Detailed discussion
What was done
In
DOCKER/docker-entrypoint.sh, a trapped signal makes bash'swait "$child"return early with status 128+signal while Tenderdash is still running. The signal handler now also setsinterrupted, and the script repeatswaituntil one completes without interruption, so it exits only after Tenderdash does and with Tenderdash's own status. If Tenderdash has already exited by then, bash returns its saved status. A repeated signal is forwarded again, as before. The 10-second sleep after a non-signal failure (#845) is unchanged.DOCKER/test-entrypoint.shruns the entrypoint with a faketenderdashonPATHand checks:The Release tooling workflow runs it when the entrypoint or the test changes.
Testing
DOCKER/test-entrypoint.shpasses; on the original entrypoint fromdevit fails (the entrypoint exits before Tenderdash finishes).shellcheckis clean on both scripts.Breaking changes
None. Containers stop as late as Tenderdash's shutdown takes, bounded by Docker's stop timeout (default 10 s; dashmate sets
stop_grace_period: 40s, dashpay/platform#5836).Prior work
Attribution
🤖 Generated with Claude Code
🤖 Co-authored by Claudius the Magnificent AI Agent
PR Hygiene ·
dd97674github(.github/workflows/release.yml) — approved by lklimekDOCKER/docker-entrypoint.sh,DOCKER/test-entrypoint.sh) — approved by lklimekWhen every merge requirement is met, the
PR Hygienecheck passes. Reviewer limits do not block merging; other required GitHub checks and protections still apply.