Skip to content

test: count a group of zombies as ended, and kill create once it wrote the manifest - #314

Merged
ThomasK33 merged 3 commits into
mainfrom
test/zombie-leader-load
Nov 4, 2026
Merged

ThomasK33 merged 3 commits into
mainfrom
test/zombie-leader-load

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Why

"destroy ends a child of the program whose main thread exited while another thread runs" needed 2 retries on main and on #311, on this shared host at load averages of 60 to 80. The #311 advisor also found that the early-destroy test still needed the host to start the program before create gave up after about 5 s. Follow-up of #311.

Cause

groupEnds in test/helpers.ts polled kill(-pgid, 0) for 2 s. That call also finds a zombie. A program's background process is an orphan once its parent shell dies, so init reaps it, not the host. On a loaded host, init reaped it after the 2 s deadline. The test then failed although destroy had already ended every process of the group.

I reproduced this with a child subreaper that reaps only every 10 s (slow-reaper.py below). Main fails groupEnds in 2 or 3 of the 5 selected tests per run:

  • "destroy ends a child of the program whose main thread exited while another thread runs"
  • "destroy ends a background process that outlives the program"
  • "destroy ends a background process left by a program that exited"

This branch passes 5 of 5.

Repro wrapper (not committed)
# REAP_DELAY=10 python3 slow-reaper.py npx vitest run --retry=0 test/integration/lifecycle.test.ts -t "whose main thread exited|destroy ends a background process"
import ctypes, os, sys, time
ctypes.CDLL(None).prctl(36, 1, 0, 0, 0)  # PR_SET_CHILD_SUBREAPER
delay = float(os.environ.get('REAP_DELAY', '3'))
child = os.fork()
if child == 0:
    os.execvp(sys.argv[1], sys.argv[1:])
code = None
while True:
    time.sleep(delay)
    try:
        while True:
            pid, status = os.waitpid(-1, os.WNOHANG)
            if pid == 0:
                break
            if pid == child:
                code = os.waitstatus_to_exitcode(status)
    except ChildProcessError:
        sys.exit(code if code is not None else 1)

What changes

Only test helpers and one test change.

  • On Linux, groupEnds now reads the threads of each group member in /proc. A group whose threads are all zombies counts as ended. A live thread, such as the running thread of zombie-leader.py whose main thread exited, still counts as running. Two scans must agree, because a process can fork and end between the /proc listing and its own read. In some cases /proc cannot show the whole group: the listing fails, a read is refused, /proc has hidepid=2, 4, invisible or ptraceable, or mountinfo is unreadable. Then the kill(-pgid, 0) probe decides alone, as before. The probe also decides alone when a scan finds the group id but shows no member at all, as a /proc of another pid namespace does. Off Linux it keeps kill(-pgid, 0), because launchd reaps orphans at once. The deadline is now 5 s, which costs nothing when the group ends sooner.
  • The early-destroy test kills create as soon as the manifest exists. The host reads an existing manifest also after its creator died (waitForManifestFromCreate, src/host/hostMain.ts). So the test no longer needs the host to start the program before create gives up. Its envelope wait is now 30 s instead of 15 s.
  • I checked the advisor's second point with a throwaway test. That test held the host's start-up write and failed on purpose. Afterwards, cleanupHome had killed the held host, because it SIGKILLs the hostPid that the manifest names. No change is needed.

Validation

  • tsc --noEmit, oxlint and oxfmt --check pass.
  • With the 10 s reaper, main failed and this branch passed 5 of 5 (results above).
  • With a delay of 1.5 s and 2.5 s at the start of the host and of each destroy (the test: keep the lifecycle tests stable on a slow runner and cut fixed waits #311 preload), the early-destroy test passes.
  • A throwaway test checked groupEnds against a live zombie-leader.py. It still reads the group as running after 1.5 s, and it reads the group as ended right after a SIGKILL.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: xhigh

…e the manifest

- groupEnds waited for init to reap the program's killed orphans: kill(-pgid, 0)
  finds a zombie. On a loaded host init reaped them after the 2 s deadline,
  so "destroy ends a child of the program whose main thread exited" and the
  tests of a background process failed although destroy had ended every
  process. On Linux groupEnds now reads the threads in /proc and counts a
  group whose threads are all zombies as ended.
- The early-destroy test killed create once the program ran, so it still
  needed the host to start the program before create gave up. It now kills
  create once the manifest exists: the host reads an existing manifest also
  after its creator died. Its envelope wait is 30 s.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-04T01:33:12.152388Z 524b2ba Manual request
🔒 Security Review ✅ Completed 2026-11-04T01:33:58.038975Z 524b2ba Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: f9f11d1180

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f9f11d1180

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread test/helpers.ts Resolved
Comment thread test/helpers.ts Resolved
…proc hides processes

Codex on #314: a process can fork and end between the /proc listing and
its own read, and a /proc that is missing, refuses a read or hides other
users cannot show the whole group.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: c836c62308

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: c836c62308

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

… only ENOENT pass the sessions read

#314 advisor: a /proc of another pid namespace shows no member of the
group, so the probe decides there. Only a missing sessions folder means
that create has not written yet.
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 524b2ba870

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 524b2ba870

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Nov 4, 2026
Merged via the queue into main with commit 8668515 Nov 4, 2026
18 checks passed
@ThomasK33
ThomasK33 deleted the test/zombie-leader-load branch November 4, 2026 02:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant