Repository navigation
fix(ingest): stop naming signal.SIGKILL where Windows can reach it - #506
Conversation
#505. `signal.SIGKILL` does not exist on Windows, and ingest/textbook_parser.py evaluated it on two paths that both force-kill a MinerU subprocess. Neither path killed anything on Windows: both raised AttributeError, the subprocess kept running with its sandbox output directory open, and the failure surfaced as something unrelated to a timeout. The defect was present TWICE, which is why fixing the obvious half would not have worked: 1. At the call sites (:443 shutdown, :551 wall timeout), `signal.SIGKILL` is an ARGUMENT -- evaluated before the function is entered, so no platform check inside the helper could ever run. 2. Inside the helper itself, `elif sig == signal.SIGKILL` sat on the branch taken when `os.getpgid` is absent, i.e. the Windows branch. The arm written to handle Windows could not execute on Windows. `AttributeError` is not an `OSError`, so the enclosing `contextlib.suppress` did not catch it either. The helper's docstring already reasoned about exactly this hazard for `os.getpgid` one line above where it recurred for `signal.SIGKILL`. FIX: the helper takes an INTENT (`force: bool`), never a signal number, so the constant is named only inside the POSIX branch where it is guaranteed to exist. That makes the Windows arm total by construction rather than by remembering. Rejected alternative, because it is silently wrong rather than merely uglier: a module-level `_SIGKILL = getattr(signal, "SIGKILL", signal.SIGTERM)`. On Windows that collapses to SIGTERM, so the helper's comparison would be true for BOTH phases of terminate_live_mineru's SIGTERM-then-grace-then-SIGKILL escalation and the graceful phase would hard-kill. Also documented, not hidden: on Windows `force` changes nothing, because `Popen.terminate` and `Popen.kill` are both TerminateProcess. A caller that needs a real grace period on Windows does not get one from this helper. The pre-existing accepted gap is separate and unchanged -- on Windows the kill reaches only the direct child, and MinerU's grandchild FastAPI service may survive (security-pdf-sandbox.md, CLAUDE.md gotcha #10). TESTS. The existing test_timeout_falls_back_to_proc_kill_without_getpgid removes `os.getpgid` only. On POSIX `signal.SIGKILL` still resolves, so that test passed in CI while failing on Windows -- the bug shipped because the guard could not see it. Two tests added: * test_timeout_kills_when_the_platform_has_no_sigkill removes BOTH attributes, reproducing Windows on any host. Verified to fail on the pre-fix file with the real `AttributeError` at :551, which is what makes it a regression test rather than a restatement. * test_signal_helper_never_names_sigkill_off_the_posix_branch drives the helper directly for both intents, so a future refactor of the timeout path cannot quietly drop the coverage. It fails pre-fix on the signature, so it guards the helper going forward rather than re-proving the original defect. tests/test_textbook_parser.py and tests/test_parse_tracker.py pass; ruff clean.
|
Full suite, as promised when this opened. Windows 11, CPython 3.11.9,
The 3 remaining failures are not this PR's, and not codeAll three are Where that leaves the platformFor context on how this number was reached, since it moved twice for reasons unrelated to this PR:
The 16-failure drop was |
Closes #505.
signal.SIGKILLdoes not exist on Windows, andingest/textbook_parser.pyevaluated it on two paths that both force-kill a MinerU subprocess. Neither killed anything on Windows: both raisedAttributeError, the subprocess kept running with its sandbox output directory open, and the failure surfaced as something unrelated to a timeout.The defect was present twice
Fixing the obvious half would not have worked.
1 — at the call sites (
:443shutdown,:551wall timeout),signal.SIGKILLis an argument. It is evaluated before the function is entered, so no platform check inside the helper could ever run.2 — inside the helper itself:
That
elifsits on the branch taken whenos.getpgidis absent — which is Windows. The arm written to handle Windows could not execute on Windows.AttributeErroris not anOSError, so the enclosingcontextlib.suppressdid not catch it either.The helper's docstring already reasoned about exactly this hazard for
os.getpgid, one line above where it recurred forsignal.SIGKILL.The fix
The helper takes an intent (
force: bool), never a signal number, so the constant is named only inside the POSIX branch where it is guaranteed to exist:That makes the Windows arm total by construction rather than by remembering.
Rejected alternative, because it is silently wrong rather than merely uglier: a module-level
_SIGKILL = getattr(signal, "SIGKILL", signal.SIGTERM), which #505 floated. On Windows it collapses toSIGTERM, so the comparison would be true for both phases ofterminate_live_mineru's SIGTERM → grace → SIGKILL escalation, and the graceful phase would hard-kill.Documented, not hidden: on Windows
forcechanges nothing, becausePopen.terminateandPopen.killare bothTerminateProcess. A caller needing a real grace period on Windows does not get one here. The pre-existing accepted gap is separate and unchanged — on Windows the kill reaches only the direct child, and MinerU's grandchild FastAPI service may survive (security-pdf-sandbox.md, CLAUDE.md gotcha #10).Why this shipped, and what stops it recurring
The existing
test_timeout_falls_back_to_proc_kill_without_getpgidremovesos.getpgidonly. On POSIXsignal.SIGKILLstill resolves, so that test passed in CI while failing on Windows. The guard could not see the bug it was written next to.Two tests added:
test_timeout_kills_when_the_platform_has_no_sigkillremoves both attributes, reproducing Windows on any host. Verified to fail against the pre-fix file with the realAttributeErrorat:551— that is what makes it a regression test rather than a restatement of the fix.test_signal_helper_never_names_sigkill_off_the_posix_branchdrives the helper directly for both intents, so a future refactor of the timeout path cannot quietly drop the coverage. It fails pre-fix on the signature, so it guards the helper going forward rather than re-proving the original defect.Verification
tests/test_textbook_parser.pyandtests/test_parse_tracker.pypass.ingest/textbook_parser.pytoorigin/mainwhile keeping the new tests makes all three relevant tests fail.