Skip to content

Commit 38bef8c

Browse files
fix(pm): git-history lets a piped answer drain before it exits, and a no-remote refusal names its floor (#21685)
Fixes #21659 Clause-②: no ## What was wrong `scripts/pm/git-history.mjs` hands its whole answer to one `process.stdout.write`, and the entry ended with `process.exit(main(...))`. On POSIX, Node completes a write to a FILE at once, but a write to a PIPE only as far as the pipe takes it in one go; the rest is queued for the event loop, and `process.exit` drops that queue. So `log ... | wc -l` read one buffer-full, at exit 0. `count` and `touch` print one line, which always fits. The rider: `ensureWindowCovered()` returned from its no-remote branch (a bare-sha or local-branch `--ref`) without the boundaries it had already read, so that refusal printed `shallow floor: unknown`. The refusal itself was correct. ## The change (one file) - The entry sets `process.exitCode` and returns, so the process ends only after its last queued write has drained, on every stdout kind. This is the ruling's second arm. I did not take `fs.writeSync(1, ...)`: once anything touches `process.stdout`, libuv puts a pipe stdout in non-blocking mode, and a sync write to a full non-blocking pipe throws EAGAIN, which would swap the truncation for a crash. - The two `process.exit` calls inside `main()` now return their codes (`return 2` on the window refusal, `return 1` on the self-test no-verdict branch). They are the same defect form in the same function: an exit after a write that may still be queued. `touchMain()` already returned its codes. The self-test handshake still prints its message and exits non-zero. - A `process.stdout` `'error'` listener absorbs EPIPE only, and anything else rethrows. This is new behaviour that the fix makes necessary. Before, `| head -1` exited 0 quietly because `process.exit` threw the queued rest away before EPIPE could surface. With the exit removed, the queued write meets the closed reader, and with no listener Node prints an unhandled `write EPIPE` trace and exits 1 (measured; ablation leg 2 below). A reader that closes early made its own choice, so the answer's exit code stands. - The rider: the no-remote return now carries `shallow` and `boundaries`, so the refusal prints the floor it read. - The header gains a "fifth trap" section with the measurement. ## Reading one: the reproduction, before and after Shared clone (shallow). The floor on `3711e0b763` is 2026-06-22 and the window is covered with no fetch (receipt: `floor 2026-06-22 · tip 2026-09-29 · floor already predates the window (no fetch)`). Command: `node scripts/pm/git-history.mjs log --since=2026-08-14T00:00:00Z --ref=3711e0b763`. | tree | stdout | lines | producer exit | |---|---|---|---| | before (base `7d0781482d`) | piped to `wc -l`, three runs | 346 · 346 · 346 | 0 · 0 · 0 (`PIPESTATUS[0]`) | | before (base `7d0781482d`) | redirected to a file | 5455 (939,998 bytes) | 0 | | after (`d068d3faa1`) | piped to `wc -l`, three runs | 5455 · 5455 · 5455 | 0 · 0 · 0 | | after (`d068d3faa1`) | redirected to a file | 5455, `cmp`-identical to the before file | 0 | | after (`d068d3faa1`) | piped to `head -1` | (1) | 0; stderr is the one-line receipt, no EPIPE trace | Rider, same clone: `count --since=2026-06-01T00:00:00Z --ref=3711e0b763` (no fetch: a bare sha names no remote). Exit 2 with empty stdout in both trees. - before: `shallow floor: unknown (the oldest commit this clone can see on that ref)` - after: `shallow floor: 2026-06-22 (the oldest commit this clone can see on that ref)` ## Reading two: self-test cases, battery and floor | battery | before (cases = pin) | after (cases = pin) | |---|---|---| | pure decisions | 10 | 10 | | real repos | 15 | **17** (+2: the bare-sha refusal, exit 2 with empty stdout, and its floor `2026-07-06`, not `unknown`) | | historyHorizon | 12 | 12 | | bare dates | 16 | 16 | | touch | 26 | 26 | | **piped stdout: the answer a pipe reads is the answer a file holds** (new) | — | **4** | | total printed | 79 | 85 | | `SELF_TEST_BATTERY_FLOOR` (roster size) | 5 | **6** | How the battery and floor were handled: the AGENTS.md rule "Floor — pin battery NAMES, never one total". The new concern gets its own named battery. Each battery's pin rises to the count it now registers. The roster-size floor moves with the roster, the same way commit `55e6f14f` moved it from 4 to 5. The new battery: 1. FIXTURE: on the 40-commit fixture, `log` redirected to a FILE holds the 21 answer lines at exit 0 and is over 1 MiB (1,376,277 bytes). Each line is four 16384-column padded `%H` fields. git 2.43 honours padding up to 16384 per field and prints a wider request unpadded (measured), so the format uses four fields per line. 2. The same `log`, read through a pipe by a reader in its own process, gives the same 21 lines at exit 0. That reader attaches its listener at once, holds the stream paused for 200 ms, then drains it. 3. The piped bytes ARE the file's bytes: same length, same sha256. 4. A reader that closes after its first chunk (the `| head -1` shape, confirmed by reading less than the file holds) leaves exit 0 and a one-line receipt on stderr, with no EPIPE trace. The reader's pipe is a socketpair (Node child_process stdio), not a FIFO. Node treats both as PIPE with async writes. The default UNIX-socket send buffer here is 212,992 bytes, so the answer is about 6.5 of those, and 21 times a 64 KiB FIFO buffer. ## Ablation (`scripts/ablation-replace.mjs`, wrap mode, at `d068d3faa1`) Each leg's mutation landed on disk (anchor x1 to x0, replacement x0 to x1, blob changed). Each was restored and proven: blob `1a54ee0b67d4` equals HEAD's, and `git diff HEAD` is empty. | leg | mutation | self-test | |---|---|---| | 1 | entry back to `process.exit(main(process.argv.slice(2)) \|\| 0);` | **2 FAILED**: the piped reader got 146,176 bytes / 2 lines against the file's 1,376,277 / 21, child exit 0. FIXTURE and early-close stayed green, as expected. | | 2 | EPIPE absorption removed (every stdout error rethrown) | **1 FAILED**: early-close child exit 1 with `Error: write EPIPE`. My first attempt at this leg was a no-op: its replacement `throw err;` was a substring of the anchor, so ablation-replace refused before running anything (exit 1, restored). I re-ran it with a unique replacement. | | 3 | no-remote return without `shallow`/`boundaries` | **1 FAILED**: `shallow floor: unknown` | Every leg went red in the expected direction. No ablation file is left in the tree. ## Gates (at `d068d3faa1`) - `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`, no path, with the change set read from git (1 path). It derives **32** commands, identical to the dispatch's list. - 31 ran outside the lock: all exit 0. The longest were `check-comment-mask-corpus` (116 s) and `check-self-test-workflow-commands` (78 s). `node scripts/pm/git-history.mjs --self-test` took 10 s. - `pnpm check:pm-dispatch-gates` ran under `os-verify-lock.sh`: `VERDICT command-exit 0 · held the lock 1246s (20m46s) · waited 0s`. The battery took 1244.7 s on this box (1976 cases). - `dispatch-gates --ran` with an exit code recorded per line: `32 derived famil(ies) accounted for — 32 run, 0 NOT-MEASURED (a DERIVED zero)`, 0 UNRUN. - Narrowed lint: `eslint --no-inline-config --format json scripts/pm/git-history.mjs` reports 1 file, 0 errors, 0 warnings. `--print-config` for this file shows two rules, `no-restricted-imports` and `comment-swallow/no-code-inside-block-comment`. Neither has `parserOptions.project`, so linting is not type-aware and this diff cannot move any other file's verdict. The full `pnpm lint` is left to CI. ## Acceptance notes - `usage()` keeps its `process.exit(1)`. Every call to it comes before anything is written to stdout, and its stderr text is under 1 KiB, a single write an empty pipe takes whole. - A text grep finds the spelling `process.exit(main(` or `process.exit(await main(` in 27 other files under `scripts/` and `scripts/pm/` (comments included). One probe, `check-entry-guard.mjs --list` (9,589 bytes), reads complete through a pipe. No other wrong answer was measured, so nothing is filed: the truncation needs an answer longer than one buffer-full, and none was found. - No changeset: `scripts/pm/**` publishes nothing. The PR carries `skip-changeset`. - The new battery adds about 0.5 s to the self-test (two 200 ms reader holds). --- _Generated by [Claude Code](https://claude.ai/code/session_01CB6W87z22K2yjUCDyVrJRk)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent cef2bd7 commit 38bef8c

1 file changed

Lines changed: 147 additions & 6 deletions

File tree

‎scripts/pm/git-history.mjs‎

Lines changed: 147 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,33 @@
149149
* what git is handed and what the receipt prints, and a complete instant is
150150
* parsed exactly and never consults the clock.
151151
*
152+
* ## The fifth trap, no git at all: a piped answer is cut at one buffer-full
153+
*
154+
* The answer leaves in ONE `process.stdout.write`. Node completes that write
155+
* at once when stdout is a FILE, but when stdout is a PIPE (on POSIX) it puts
156+
* in only what the pipe takes in one go and queues the rest for the event
157+
* loop. The entry used to end with `process.exit(main(...))`, which tears the
158+
* process down before that queue drains — so `log … | wc -l`, the natural
159+
* spelling of a count taken through `log`, read one buffer-full at exit 0
160+
* (#21659). Measured at 7d0781482d on this repository's shallow clone, the
161+
* same `log --since=2026-08-14T00:00:00Z --ref=3711e0b763` two ways:
162+
*
163+
* | stdout | lines | exit |
164+
* |--------------------------------|-------|------|
165+
* | redirected to a file | 5455 | 0 |
166+
* | piped to `wc -l`, three runs | 346 | 0 |
167+
*
168+
* 346 lines is about one 64 KiB pipe buffer. `count` and `touch` print one
169+
* line, which always fits, so they never showed it. The entry therefore sets
170+
* `process.exitCode` and returns: the process ends when its last queued write
171+
* has drained, on every stdout kind. `main()` and `touchMain()` RETURN their
172+
* exit codes; the one `process.exit` left is `usage()`'s, and every call to it
173+
* comes before anything is written to stdout. A reader that closes early
174+
* (`| head -1`) turns the queued rest into EPIPE; that is the reader's choice,
175+
* not a lost answer, so it is absorbed and the answer's own exit code stands.
176+
* The self-test pins a piped `log`, many buffers long, byte for byte against
177+
* the same `log` sent to a file.
178+
*
152179
* ## Cost, measured — because a tool nobody runs fixes nothing
153180
*
154181
* | case | wall |
@@ -162,7 +189,8 @@
162189
*/
163190

164191
import { execFileSync, spawnSync } from 'node:child_process';
165-
import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
192+
import { createHash } from 'node:crypto';
193+
import { closeSync, mkdirSync, mkdtempSync, openSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
166194
import { tmpdir } from 'node:os';
167195
import { join } from 'node:path';
168196
import { isEntrypoint } from '../invoked-as.mjs';
@@ -185,15 +213,16 @@ import { isEntrypoint } from '../invoked-as.mjs';
185213
// remedy is to find what stopped registering.
186214
const SELF_TEST_BATTERIES = Object.freeze({
187215
'pure decisions': 10,
188-
'real repos': 15,
216+
'real repos': 17,
189217
'historyHorizon: the read-only reading the #9902 adopters call': 12,
190218
'bare dates: the instant the proof reads is the instant git counts': 16,
191219
'touch: the provenance reading a shallow clone fabricates': 26,
220+
'piped stdout: the answer a pipe reads is the answer a file holds': 4,
192221
});
193222

194223
// DELETING an entry silences that battery's floor exactly as effectively as
195224
// zeroing it, so the roster's own size is pinned too.
196-
const SELF_TEST_BATTERY_FLOOR = 5;
225+
const SELF_TEST_BATTERY_FLOOR = 6;
197226

198227
// The key an assertion is filed under when no battery is open. It is not a
199228
// declared battery, so it reds by the same set difference rather than silently
@@ -303,9 +332,14 @@ export function ensureWindowCovered({ cwd, ref, sinceMs, allowFetch = true, allo
303332
const remotes = (git(['remote'], { cwd, allowFail: true }) || '').split('\n').filter(Boolean);
304333
const target = splitRemoteRef(ref, remotes);
305334
if (!target) {
335+
// The boundaries were READ above; a ref that names no remote (a bare sha, a
336+
// local branch) cannot be deepened, but its floor is still known, so the
337+
// refusal prints it rather than `shallow floor: unknown`.
306338
return {
307339
covered: false,
308340
steps,
341+
shallow: shallow0,
342+
boundaries: boundaries0,
309343
reason:
310344
`history is truncated inside the window and '${ref}' names no remote to deepen from ` +
311345
`(remotes here: ${remotes.length ? remotes.join(', ') : 'none'})`,
@@ -751,7 +785,7 @@ function main(argv) {
751785
+ 'so no success line was printed. Exiting 0 here would report a self-test\n'
752786
+ 'that never finished as a self-test that passed.\n',
753787
);
754-
process.exit(1);
788+
return 1;
755789
}
756790
return selfTestCode;
757791
}
@@ -785,7 +819,7 @@ function main(argv) {
785819
` commits are invisible to git log, which reports no error (#9878).\n` +
786820
` Remedy: git -C ${cwd} fetch --unshallow ${splitRemoteRef(opts.ref, ['origin']) ? 'origin' : '<remote>'}\n`,
787821
);
788-
process.exit(2);
822+
return 2;
789823
}
790824

791825
const receipt =
@@ -1055,6 +1089,21 @@ function selfTest() {
10551089
t('the clone is still shallow after that answer — proving the predicate is the floor, '
10561090
+ 'not the shallow flag', isShallow(shallowDeep) === true);
10571091

1092+
// A ref that names no remote — a bare sha, the way a tree is usually quoted —
1093+
// cannot be deepened, so a window crossing its floor is refused. The floor
1094+
// printed beside that refusal is the one already READ, never `unknown`.
1095+
// Nothing is fetched on this path, so `shallowDeep` keeps its depth.
1096+
const shallowDeepTip = g(['rev-parse', 'origin/main'], shallowDeep).trim();
1097+
const bareShaRefusal = runCliAllowFail(['count', `--since=${WINDOW_SINCE}`, `--ref=${shallowDeepTip}`], shallowDeep);
1098+
t('a bare-sha --ref whose window crosses the shallow floor is REFUSED, exit 2 with EMPTY stdout, '
1099+
+ 'because it names no remote to deepen from',
1100+
bareShaRefusal.code === 2 && bareShaRefusal.stdout.trim() === ''
1101+
&& /names no remote to deepen from/.test(bareShaRefusal.stderr), JSON.stringify(bareShaRefusal));
1102+
t('and the refusal prints the floor it READ, 2026-07-06, not `shallow floor: unknown` — the no-remote '
1103+
+ 'branch used to return without the boundaries it had already read',
1104+
/shallow floor: 2026-07-06 /.test(bareShaRefusal.stderr) && !/shallow floor: unknown/.test(bareShaRefusal.stderr),
1105+
bareShaRefusal.stderr);
1106+
10581107
// ── historyHorizon: the read-only reading the #9902 adopters call ───────
10591108
battery('historyHorizon: the read-only reading the #9902 adopters call');
10601109
const hFull = historyHorizon({ cwd: full, ref: 'origin/main', sinceMs: Date.parse(WINDOW_SINCE) });
@@ -1373,6 +1422,90 @@ function selfTest() {
13731422
never.code === 1 && never.stdout.trim() === '' && /no commit on/.test(never.stderr), JSON.stringify(never));
13741423
const noPath = runCliAllowFail(['touch'], full);
13751424
t('touch without --path is usage', noPath.code === 1 && noPath.stdout.trim() === '', JSON.stringify(noPath));
1425+
1426+
// ── piped stdout: the answer a pipe reads is the answer a file holds ────
1427+
// The fifth trap in the header. A file takes the whole answer in one write;
1428+
// a pipe takes what fits and the rest waits on the event loop, which the
1429+
// entry's old `process.exit()` ended. Every answer above fits in one
1430+
// buffer-full, so none of them could see it. This one is padded to many:
1431+
// `%<(16384)` pads a field to 16384 columns (git ignores wider), four to a
1432+
// line, 21 lines. A child's pipe here is a socketpair, so a buffer-full is
1433+
// the socket's send buffer (212992 bytes by default on Linux) rather than a
1434+
// FIFO's 64 KiB; the answer is sized well past both. `full` is read-only
1435+
// here, so this battery's place after the deepening ones changes nothing.
1436+
battery('piped stdout: the answer a pipe reads is the answer a file holds');
1437+
const wideArgs = [
1438+
'log', `--since=${WINDOW_SINCE}`, `--until=${WINDOW_UNTIL}`, `--format=${'%<(16384)%H'.repeat(4)}`, `--cwd=${full}`,
1439+
];
1440+
const newlines = (buf) => buf.reduce((n, b) => n + (b === 10 ? 1 : 0), 0);
1441+
1442+
const wideFile = join(root, 'wide-answer.txt');
1443+
const wideFd = openSync(wideFile, 'w');
1444+
let toFile;
1445+
try {
1446+
toFile = spawnSync(process.execPath, [self, ...wideArgs], { cwd: full, encoding: 'utf8', stdio: ['ignore', wideFd, 'pipe'] });
1447+
} finally {
1448+
closeSync(wideFd);
1449+
}
1450+
const fileBytes = readFileSync(wideFile);
1451+
const fileLines = newlines(fileBytes);
1452+
t('FIXTURE — the wide `log` sent to a FILE holds the 21 answer lines, exit 0, and is over 1 MiB: many times '
1453+
+ 'what a pipe takes in one go. Read first, because an answer that fits one buffer-full passes the piped '
1454+
+ 'pins below with or without the fix',
1455+
toFile.status === 0 && fileLines === 21 && fileBytes.length > 1024 * 1024,
1456+
`exit ${toFile.status} lines ${fileLines} bytes ${fileBytes.length} stderr ${JSON.stringify(toFile.stderr)}`);
1457+
1458+
// The reader is its own process, so the tool's stdout is a real pipe whose
1459+
// far end nothing else drains. It attaches its listener at once (Node
1460+
// discards a child's unread output when the child exits, which would look
1461+
// like a truncation of the reader's own making), holds the stream paused
1462+
// for 200 ms so the tool fills the pipe and has to wait, then reads to EOF —
1463+
// or, in `close-early` mode, takes one chunk and closes, the shape of `| head -1`.
1464+
const PIPE_READER = [
1465+
"const { spawn } = require('node:child_process');",
1466+
"const { createHash } = require('node:crypto');",
1467+
'const [mode, ...cmd] = process.argv.slice(1);',
1468+
"const child = spawn(process.execPath, cmd, { stdio: ['ignore', 'pipe', 'pipe'] });",
1469+
"const hash = createHash('sha256');",
1470+
"let bytes = 0; let lines = 0; let stderr = '';",
1471+
"child.stderr.on('data', (d) => { stderr += d; });",
1472+
'const take = (d) => { bytes += d.length; lines += d.reduce((n, b) => n + (b === 10 ? 1 : 0), 0); hash.update(d); };',
1473+
"if (mode === 'close-early') child.stdout.once('data', (d) => { take(d); child.stdout.destroy(); });",
1474+
"else child.stdout.on('data', take);",
1475+
'child.stdout.pause();',
1476+
'setTimeout(() => child.stdout.resume(), 200);',
1477+
"child.on('close', (code, signal) => process.stdout.write(JSON.stringify({ code, signal, bytes, lines, sha256: hash.digest('hex'), stderr })));",
1478+
].join('\n');
1479+
const readThroughPipe = (mode) => {
1480+
const r = spawnSync(process.execPath, ['-e', PIPE_READER, mode, self, ...wideArgs], {
1481+
cwd: full,
1482+
encoding: 'utf8',
1483+
stdio: ['ignore', 'pipe', 'pipe'],
1484+
timeout: 60_000,
1485+
});
1486+
try {
1487+
return JSON.parse(r.stdout);
1488+
} catch {
1489+
return { code: null, readerExit: r.status, readerSignal: r.signal, readerError: String(r.error ?? ''),
1490+
readerStdout: String(r.stdout).slice(0, 300), stderr: String(r.stderr).slice(0, 300) };
1491+
}
1492+
};
1493+
1494+
const piped = readThroughPipe('slow');
1495+
t('the same `log` through a pipe whose reader holds off reads the SAME 21 lines, exit 0 — where the entry\'s '
1496+
+ 'old process.exit() delivered one buffer-full and still exited 0',
1497+
piped.code === 0 && piped.lines === fileLines, JSON.stringify(piped));
1498+
t('and the piped bytes ARE the file\'s bytes: the same length and the same sha256, not just the same line count',
1499+
piped.bytes === fileBytes.length && piped.sha256 === createHash('sha256').update(fileBytes).digest('hex'),
1500+
`piped ${piped.bytes} bytes ${piped.sha256} · file ${fileBytes.length} bytes`);
1501+
1502+
const early = readThroughPipe('close-early');
1503+
t('a reader that closes after its first chunk (the `| head -1` shape, proved by reading less than the file '
1504+
+ 'holds) costs the tool no `write EPIPE` trace: it exits with the answer\'s code, 0, and its stderr is the '
1505+
+ 'one-line method receipt',
1506+
early.code === 0 && early.bytes > 0 && early.bytes < fileBytes.length
1507+
&& /^method: git log [^\n]*\n$/.test(early.stderr ?? '') && !/EPIPE/.test(early.stderr ?? ''),
1508+
JSON.stringify(early));
13761509
} finally {
13771510
rmSync(root, { recursive: true, force: true });
13781511
}
@@ -1429,5 +1562,13 @@ function selfTest() {
14291562

14301563
const invokedDirectly = isEntrypoint(import.meta.url);
14311564
if (invokedDirectly) {
1432-
process.exit(main(process.argv.slice(2)) || 0);
1565+
// The fifth trap in the header: `process.exit()` here ended the process with
1566+
// most of a piped `log` answer still queued. Setting the exit code and
1567+
// returning lets the queued writes drain first, on every stdout kind.
1568+
process.stdout.on('error', (err) => {
1569+
// A reader that stopped reading (`| head -1`) took what it wanted; the rest
1570+
// of the answer has nowhere to go. Anything else is a real failure.
1571+
if (err?.code !== 'EPIPE') throw err;
1572+
});
1573+
process.exitCode = main(process.argv.slice(2)) || 0;
14331574
}

0 commit comments

Comments
 (0)