fix(smoke): validate remote mktemp -d before using it as an rm -rf target - #33
Conversation
…rget
scripts/smoke-machine-sync-release.mjs set remoteDir from an unchecked remote
`mktemp -d` and later ran `ssh <remote> rm -rf ${shellQuote(remoteDir)}`.
`mktemp -d` prints its error on stderr and nothing on stdout, so a failure left
remoteDir empty. That is inert today only because shellQuote renders '' as ''
-- one appended `/*` at the cleanup site turns it into the 2026-07-24 incident,
where an empty command substitution made `rm -rf "$(cmd)"/*` run as `rm -rf /*`
and destroyed a machine's checkouts. The check belongs at the assignment, not at
the delete, and it must not lean on quoting.
New scripts/lib/remote-temp-dir.mjs exports assertRemoteTempDir, which accepts
only what the template could actually have produced: `mktemp` replaces the
trailing X run in place, so the result must be the template's fixed prefix plus a
single non-empty path component. Checking only the template's parent directory is
not enough -- it would accept /tmp/somebody-elses-dir and delete it. The bound is
derived from the template rather than hardcoded, so it stays correct if the
template moves. Applied at the assignment and re-asserted immediately before the
`rm -rf`.
Packaging: smoke-machine-sync-release.mjs is a published file, so the new module
is registered in all three deliberate allowlists (package.json files,
scripts/validate-public-package.mjs, tests/package-release.test.ts). Without that
the published script would fail with ERR_MODULE_NOT_FOUND for every installed
copy.
Also adds a test asserting that every relative import in a published script is
itself published. The existing allowlists only caught the opposite mistake
(shipping something unreviewed). This one immediately found pre-existing
breakage: scripts/apply-cloud-migrations.mjs:23 imports ../src/storage.ts, but
src/ has never been in `files`, so the published script already throws
ERR_MODULE_NOT_FOUND. Verified against a packed tarball. That path carries a
single exemption named in the test and tracked as todos task 1eb481d5; removing
the exemption is that task's acceptance criterion. It is not fixed here because
choosing between repointing at dist, shipping src, or unpublishing the script is
a separate decision.
Evidence:
bun test tests/machine-smoke-script.test.ts tests/package-release.test.ts
-> 14 pass, 0 fail
node scripts/validate-public-package.mjs
-> passed; public scripts included: 7
npm pack + extract + `bun scripts/smoke-machine-sync-release.mjs --help`
-> resolves and runs from the tarball
full suite (HASNA_KNOWLEDGE_API_* unset) -> 227 pass, 2 skip, 1 fail; that one
failure reproduces on the untouched baseline via git stash, so it is
pre-existing and unrelated.
gitleaks staged scan -> no leaks found
Negative controls, proving the guards fail on bad input rather than only passing
on good: assertRemoteTempDir rejects '', whitespace-only, multi-line output,
relative paths, /, /tmp, /usr, /home/hasna, a sibling dir sharing the temp
parent, a path escaping via .., and a malformed template. Removing the new module
from the allowlist makes the packaging test fail with the exact missing path.
No `rm` is executed by any test.
|
CI on this PR is red. It is not caused by this PR — both failing tests already fail on Failure 1 —
|
| date | fixture age | inside --since 30d? |
observed |
|---|---|---|---|
| 2026-07-24 | 29d | yes | main CI ubuntu-bun passed |
| 2026-07-26 | 31d | no | this PR failed |
| 2026-07-27 | 32d | no | fails locally |
Controlled A/B on untouched main @ 2023a26 in a detached worktree, changing only the fixture date:
A) main as-is, today
bun test tests/cli.test.ts -t "context pack and proposal context"
-> 1 fail
B) main + ONLY 2026-06-25 -> 2026-07-24 in the fixture, nothing else
same command
-> 0 fail
So the calendar is the sole variable. Tracked as todos d1b93867. The fix should seed the fixture relative to now or freeze time — bumping the date only re-arms the bomb.
Failure 2 — knowledge machine sync ledger > sync export redacts local file and workspace refs from bundles (macOS only)
Also fails on main’s own CI run 30106566377, same commit as this PR’s base. Not caused by this PR; needs its own diagnosis.
This PR’s own tests pass in CI
(pass) remote temp dir guard > rejects the empty output a failing mktemp -d leaves behind
(pass) remote temp dir guard > the failure names the path and says why it matters
This PR touches only scripts/lib/remote-temp-dir.mjs, scripts/smoke-machine-sync-release.mjs, scripts/validate-public-package.mjs, the two test files, and package.json. None of them are imported by the CLI or by tests/cli.test.ts.
Adversarial review found the guard silently degraded into the parent-directory-only
check this module exists to avoid, and that the defect narrative described an
unreachable code path. Both fixed.
F1 (high): when a template's final component is nothing but X, the derived prefix
is a bare directory and every sibling under it validated. `/tmp/XXXXXX` accepted
`/tmp/somebody-elses-dir`, `/XXXXXX` accepted `/etc` and `/home`, and
`/home/hasna/XXXXXX` accepted `/home/hasna/.ssh` -- all reproduced. `/tmp/XXXXXX`
is an ordinary mktemp template. The guard that should have caught this,
`prefix.includes('/') === false`, could never fire, because '/tmp/'.includes('/')
is true; mutation testing confirmed that line was both ineffective and untested.
Templates without a fixed prefix in their final component are now refused.
F2 (medium): the stated defect was not reachable. runRemote goes through
runChecked, which throws on a non-zero remote exit, so a FAILING `mktemp -d`
(exit 1, empty stdout) aborts the run and never assigns "". Verified against GNU
coreutils 9.4. The reachable hole is the remote exiting ZERO with stdout that is
not the path -- an ssh Banner, a chatty login profile, or a ForceCommand wrapper.
Verified: `echo banner; mktemp -d ...` exits 0 and yields a two-line value the
guard rejects. The claim appeared in four places (commit, PR body, module
docstring, test comment); the docstring and test comment are corrected here and
the PR body separately. An incident-remediation artifact that misstates the defect
it fixes corrupts the incident record, which is worse than the bug.
F3 (medium): the bound is derived from a template built from a caller-supplied
version string, and templates containing `..` passed validation with a meaningless
bound. Templates must now be normalized.
F4 (medium): the re-assertion before the `rm -rf` sits in a `finally`, where a
throw REPLACES any in-flight exception from the try block and skips the cleanup --
masking a real smoke failure and guaranteeing a remote leak. It is now contained:
it reports and skips the delete, and can never mask. Its actual purpose (catching a
future refactor that makes the value reassignable) is stated plainly.
F5 (medium): the new "published imports must be published" test only scanned
relative specifiers, so it missed that scripts/apply-cloud-migrations.mjs has a
SECOND unresolvable import -- `@hasna/contracts/auth`, a devDependency only. Task
1eb481d5 would have closed on a false green with the published script still
throwing. The test now checks bare specifiers against runtime dependencies too,
both are exempted by name, and both must be removed for that task to be done.
F6 (low-medium): the import regex missed side-effect imports (`import './x.mjs'`)
and fired inside comments. Comments and template literals are now stripped, and
side-effect imports are matched. Runtime builtins (node:, bun:) are excluded --
found by the new check flagging bun:sqlite.
F7/F8 (low): the GNU suffix form is documented as deliberately refused; the local
temp dir can no longer leak on the guard's throw path, because the remote dir is
now created first.
Evidence: 18 pass, 0 fail on the touched tests (was 14). The three mutations that
SURVIVED the reviewer's mutation testing (parent-only bound, bare prefix accepted,
absolute check dropped) are now all CAUGHT, as is dropping the new normalization
check. Package validator passes, packed tarball still resolves and runs. Full suite
231 pass / 1 fail, that failure being the pre-existing dated time bomb tracked as
d1b93867. No rm executed by any test.
Round 2 — review findings fixed in
|
…d-rail test itself
Re-review returned APPROVE-WITH-FIXES and found that round 2 introduced a defect in
the same class it was closing.
N1 (new, mine): the import test's comment stripper was regex-based, so
`/\*[\s\S]*?\*/` treated the `/*` inside an ordinary glob string like '**/*.ts' as a
comment opener and deleted everything up to the next `*/`. One innocuous glob line
before a broken import made the check go green -- precisely the
passes-while-broken failure that test exists to catch. Reproduced, then replaced
with a character scanner that tracks string and template literals, so a `/*` or
`//` inside a string is no longer a comment. Verified by planting
`const g = '**/*.ts'` + a broken import: now fails with the exact missing specifier.
N9: the bare-specifier check false-positived on legitimate forms. Builtins imported
without the `node:` prefix, the package's own name (valid via `exports`), and
`#`-prefixed `imports` subpaths are now allowed. The self-reference case was live,
not hypothetical: scripts/smoke-machines-adapter.mjs already imports
'@hasna/knowledge' and was passing only by accident of the old template-literal
strip.
N3a: the docstring claimed the guard accepts "only what this exact template could
have produced", which was false -- any suffix of any length and charset was
accepted. mktemp replaces the X run with exactly that many [A-Za-z0-9] characters,
so that is now enforced and the claim is true. Rejects one-char, over-long,
`$(id)`, `..` and space-containing suffixes.
N3b: a comment presented a paraphrase as a verbatim transcript -- `echo banner`
cannot print "Welcome to linux-node-a", and mktemp never prints the unexpanded
XXXXXX. Replaced with the actual measured output. This is the same defect F2 was
raised for, in text rewritten to fix F2.
N3c: "interpolated into --peer-workspace" was loose; that site passes a separate
argv element. The real shell interpolations are the remote `cd <value> && ...` and
`rm -rf <value>`. Corrected.
N4/N5/N6: the finally block is now fully contained. Round 2 wrapped only the
assertion -- the statement that cannot currently fail -- while leaving rmSync and
the ssh spawn able to throw, replace the in-flight exception from the try block,
and skip the remaining cleanup. All three are individually contained now, and a
non-zero remote cleanup exit is reported instead of discarded. Both temp dirs are
created inside the try with null guards in finally, so whichever one exists is
cleaned up; round 2's reordering had only moved the leak from local to remote.
N7/N8: added the missing fixtures -- the banner-first shape the docstring calls the
live case, the `X{3,}` POSIX boundary, and suffix shape. Bare `.toThrow()`
assertions now pin a message pattern.
Evidence: mutation retest of all 10 checks -- every one CAUGHT, including M8
(`X{3,}` -> `X{1,}`) which survived the reviewer's run, and M5 which my own charset
check had subsumed until a message-ordering test pinned it. Touched-file tests 22
pass / 0 fail (was 18). Validator passes, packed tarball resolves and runs. Full
suite 235 pass / 2 skip / 1 fail, that failure being the pre-existing dated time
bomb (task d1b93867).
CI note: PR #33 is red with TWO failures, not one. Both reproduce on main's own CI
for the same base commit, and `git diff origin/main...HEAD -- src/ tests/cli.test.ts`
is empty, so neither is caused by this PR. The macOS-only redaction failure is
recorded on d1b93867 alongside the time bomb.
Round 3 — re-review findings fixed in
|
…k to itself Round-3 review REJECTED a15850f: my fix for N1 reintroduced N1, and nothing in the suite pinned either round-3 change. THE REAL DEFECT WAS STRUCTURAL, not the parsing bug. Three versions of this check shipped in a row, each replaceable with something that reports nothing while the whole suite stayed green, because the tests meant to pin them exercised a private copy of the logic instead of the logic itself. Round 3's mutation run proved it: stripComments could be replaced with `s => ""`, or reverted to the exact regex N1 condemned, and every test passed. Moving the implementation never fixed the operating mode, which was silent-green. Fixed by extracting `unresolvableImportsIn(script, source)` and having BOTH the enforcement test and the pinning tests call it. Mutation retest, all now CAUGHT and previously all SURVIVED: isAlwaysResolvable always true; extractor reports nothing; exemptions widened to everything; relative check disabled; bare-dep check disabled; `bun:` blessed as a class again. Parsing: replaced the hand-written scanner with Bun.Transpiler().scan(). Two hand-rolled strippers had the same defect - a `/*` inside an ordinary string opened a phantom comment and deleted real imports. The second only moved the trigger: a regex literal containing a quote (`/['"]/`) shifted it into string state and everything after it vanished, which the reviewer demonstrated on the real published smoke script with two innocuous added lines. Extracting imports is a parsing problem; anything short of a parser is a guess that fails silently. The parser also removed the reason for three allowances, so all three are deleted rather than justified: - `ownName`: existed only to mask a phantom the regex stripper invented from a template literal that GENERATES a child script. A real parser does not report it. The allowance also waved through `@hasna/knowledge/<subpath>` for subpaths absent from `exports`, which fail with ERR_PACKAGE_PATH_NOT_EXPORTED. - unprefixed builtins: `builtinModules` under `bun test` is BUN's list and contains `ws`, `undici` and `bun` - real npm names, NOT Node builtins. Verified: all three are absent from Node 22's list and throw ERR_MODULE_NOT_FOUND from the tarball. - `#`-prefixed: package.json has no `imports` map, so every one is guaranteed to fail. It could only ever hide breakage. `bun:` is no longer blessed as a class either. Verified from the packed tarball: `node scripts/smoke-open-files-installed-boundary.mjs` throws ERR_UNSUPPORTED_ESM_URL_SCHEME. Exempted by name and filed as task 104f993d. F6/F7 - containment was correct in runSyncSmoke but violated one frame up. Its two callers' finally blocks used a raw rmSync, discarding the exception runSyncSmoke takes care to preserve, in exactly the --no-machines-sync paths the release smoke runs. And `report` itself was unguarded: process.stderr.write is synchronous on a pipe and throws EPIPE when the consumer exits, masking the in-flight error and skipping the remote cleanup. Both are now shared, individually-contained helpers (reportCleanupProblem, removeTempDirSafely) used at every finally; zero raw rmSync remains in any finally. F8 - the two validator checks that survived round 3 are now pinned: a lone \r (the multi-line check could be narrowed to \n unnoticed) and the suffix charset, with 6-character fixtures so only the charset rule can reject them - the earlier fixtures were all the wrong length, leaving the shell-metacharacter backstop pinned by nothing. Evidence: touched-file tests 24 pass / 0 fail (was 22). All 6 import-check mutations CAUGHT (all 6 previously survived); both validator mutations CAUGHT. Validator passes; packed tarball resolves and runs UNDER NODE, not just Bun. Full suite 239 pass / 2 skip / 1 fail, that failure being the pre-existing dated time bomb (task d1b93867). gitleaks clean.
Round 4 — REJECTED at
|
The most important change came out of a review accident, not a finding. Importing
scripts/smoke-machine-sync-release.mjs executed it: `main()` ran at module scope, so
merely loading the PUBLISHED script performed `bun install -g @hasna/knowledge` on
the local machine and attempted `ssh <remote> bun install -g` - before any argument
was inspected. It fired for real during review of this PR. Any tool that imports the
file for linting or dependency analysis - including the packaging check this PR adds
- triggers it.
`main()` now runs only when invoked directly, compared against `process.argv[1]`
rather than `import.meta.main`, which Node did not support until v24 and which would
have silently turned the script into a no-op there. Verified: still works under both
bun and node when invoked; importing is inert in 9ms with no install and no ssh.
Incident recorded with blast radius (local install only; the remote host has no DNS
or ssh-config entry, so nothing on the fleet was touched).
F-1: the exemption list was duplicated - the test meant to pin it iterated a
hardcoded literal array rather than the Set - so ADDING an exemption was checked by
nothing. A genuinely broken import plus a matching entry left the whole file green.
Same silent-green class as round 3, one level down. The loop now derives from the
Set, and every entry must name a published script. Verified: that exact combined
mutation is now CAUGHT.
F-2: deleting the allowances over-corrected and created two false positives, both
proven against an extracted tarball under node rather than argued:
import { sep } from 'path' RESOLVES (unprefixed Node builtin)
@hasna/knowledge/storage RESOLVES (declared in exports)
Both are restored, narrowly: unprefixed builtins are matched against a hardcoded
NODE list (not `builtinModules`, which under `bun test` returns BUN's list including
ws/undici/bun), and self-reference only for subpaths the `exports` map declares.
Still refused, also verified: @hasna/knowledge/<undeclared>
(ERR_PACKAGE_PATH_NOT_EXPORTED), `#`-prefixed (ERR_PACKAGE_IMPORT_NOT_DEFINED),
ws/undici (ERR_MODULE_NOT_FOUND). The previous comment claimed each deletion was
"verified against this repo"; the self-reference one was assumed, and was wrong.
F-3: the relative branch checked a hand-maintained 7-script list instead of what the
package ships, so `../package.json` and `../dist/index.js` - both published, both
verified resolvable - were reported as breakage. Now checked against `files`, with
package.json treated as always shipped since npm includes it regardless.
F-4: documented the extractor's real scope. It reports import/import()/export-from,
not require(), import.meta.resolve() or new URL(). None are used today; saying so
beats letting the test name imply more than it covers.
F-5: the EPIPE rationale was factually wrong. Measured on Linux, writing to a stderr
pipe whose reader has exited raises an asynchronous 'error' event that kills the
process - not a synchronous throw - so the try/catch cannot intercept it. The
comment now says what the guard does and does not cover.
Evidence: touched-file tests 26 pass / 0 fail. Mutation battery all CAUGHT, including
X1 (broken import + matching exemption) which SURVIVED round 5, plus the bogus-extra-
exemption, check-returns-nothing, always-resolvable, bare-dep-disabled and
builtins-widened mutations. Validator passes; full suite 239 pass / 2 skip / 1 fail,
that failure being the pre-existing dated time bomb (task d1b93867).
Corrected evidence claim: the touched-file count is 26, not the 24 stated last round.
…both fixes with tests Closes the three round-6 review findings, all of the same class: the two fixes this PR exists to make were not observable by any test, and the guard added for one of them introduced a third silent failure. 1. The direct-invocation guard compared `resolve(process.argv[1])` with `fileURLToPath(import.meta.url)`. Node resolves the main module through symlinks before deriving `import.meta.url`; `resolve()` is purely lexical. So through any symlinked path - a pnpm-style `node_modules/@hasna/knowledge -> <store>/package`, or macOS `/tmp -> /private/tmp`, which is the PR's own documented `npm pack && tar xzf` evidence procedure - `main()` never ran, the script printed nothing and exited 0. A release gate checking only the exit status would have recorded a green smoke that did nothing. Compare realpaths instead, falling back to the lexical comparison only when a path cannot be resolved at all. Verified against the packed tarball: through a pnpm-style symlink under node, `--dry-run --json` now produces output and `--totally-bogus` raises "Unknown argument"; before, both exited 0 silently. 2. Nothing pinned that `assertRemoteTempDir` was called before the remote `rm -rf`; deleting both calls left the whole suite green. `createRemoteTempDir` and `removeRemoteTempDir` now live in the side-effect-free `scripts/lib` module and take their ssh work as a parameter, so tests drive the functions that decide whether the validator runs, including the negative that matters - no delete is attempted when the value does not validate. They are kept out of the smoke script so this test file never has to import it. 3. Nothing pinned "does not execute on import"; reverting to a bare `main();` left the suite green. A new test imports the script in a child process whose PATH puts recorders in front of bash/bun/ssh, with a control proving the recorders are reachable, and asserts nothing was installed and nothing was sent to the remote. Both argv shapes are covered (`node -e`, and an ordinary importer module). Also corrects the guard's comment, which claimed Node had no `import.meta.main` until v24; measured, v22.22.3 defines it as a boolean. Mutation-verified - each fix reverted individually, each caught by exactly one test: lexical compare restored -> symlink test fails; `main()` unconditional -> import test fails; create-side assert deleted -> createRemoteTempDir test fails; delete-side re-check deleted -> "never attempts a delete" test fails; helpers bypassed in the script -> wiring test fails. Tests: 249 pass, 2 skip, 1 fail (bun test), up from 239 pass, 2 skip, 1 fail. The one failure is tests/cli.test.ts:2680, reproduced identically at base main 2023a26 (66 pass / 1 fail) and unrelated to this PR.
59d1d65 Reconciles an independent adversarial review of the previous commit. Kept as a second commit rather than an amend: the previous commit's message contains a wrong sentence (see 2 below), and rewriting history to hide it would be the same mistake as editing a merged PR body instead of commenting on it. 1. the corrected claim survived in a third file (review F1, also self-found) tests/cli.test.ts:722-725 still carried the unreachable comma justification that this PR corrected in README.md and src/cli.ts. Repo-wide, the claim had four sites, not three. Comment-only; the assertions were already correct and are unchanged. 2. "verify:generated still exits 0" was wrong (review F2) The previous commit message and the PR body both said it. Measured: bun scripts/verify-generated-artifacts.mjs -> exit 0 bun run verify:generated -> exit 1 package.json:58 makes verify:generated rebuild first, and the rebuild does not reproduce the committed bin/knowledge-mcp.js or dist/index.js. Two causes, both pre-existing on main and neither introduced here: package.json is inlined into the bundles and gained a `files` entry in #33 that the committed mcp bundle predates, and zod's codegen has drifted. bin/knowledge.js is unaffected because it is minified. Confirmed not a regression of this branch: git diff --exit-code 64d05ae 59d1d65 -- bin/knowledge-mcp.js dist/index.js exits 0, and this PR does not touch package.json or src/mcp.js. Tracked separately; main ships a stale mcp bundle. 3. the replacement for the old overclaim was itself an overclaim (review F3) "one tag named `p q` reads exactly like a pair" is false: a pair renders `p, q`. Since no not_found entry can hold a comma, joining on ', ' is injective, so a plain space is a legibility problem, not a collision — and calling it an "ambiguity" contradicted the injectivity the same comment had just established. The review found the case that IS load-bearing, and it is stronger: trim() strips only the ends, so whitespace inside a name survives. `-t $'p\nq'` yields not_found ["p\nq"], and joined raw that breaks the single-line message in two: unquoted: 'Removed 1 tag from k_w (not found: p\nq)' -> 2 lines quoted: 'Removed 1 tag from k_w (not found: "p\\nq")' -> 1 line Tab behaves the same. Now cited instead, and pinned by a new assertion rather than left as prose — the one added test assertion in this PR. 4. dependency-state caveat on the rebuild-and-compare check (review F4) @hasna/events is imported by src/cli.ts and is NOT in the build's --external list, so it is bundled; it is declared ^0.1.3 and this repo commits no lockfile. 0.1.14 reproduces the committed bundle, 0.1.13 does not (1040485 vs 1045325 stripped). The CHANGELOG now carries that caveat, since it recommends the check. Tracked separately. 5. smaller review items The `list` block's comment said "repeated -t still accumulates there" while sitting inside `list`, where repeated -t NARROWS (measured: 3 -> 2 -> 1 matches as values are added). Names the command explicitly now, in both README and source. The untag example block now says its three lines are each run against the same starting state rather than as a sequence. The 9-line bullet added to the unrelated "inventory paths block fix" section is back to a 4-line pointer. bin/knowledge.js is deliberately NOT rebuilt: every source edit here is a comment, and rebuilding to a temp outfile reproduces the committed bundle byte-for-byte after stripping (sha256 0630038c6ab4c200), which is the evidence that no shipped behaviour moved.
…e the check say so `bun run verify:generated` was RED on main at 64d05ae while `bun scripts/verify-generated-artifacts.mjs` was GREEN. One check, two names, two answers — and the passing one had been cited in CHANGELOG.md as evidence that a rebuilt bundle matched its source, which it never checked. ## The stale artifacts bun scripts/verify-generated-artifacts.mjs -> exit 0 bun run verify:generated -> exit 1 package.json:58 prefixed a `bun run build`, and the rebuild did not reproduce the committed files. Rebuilding from clean main leaves exactly two artifacts modified — `bin/knowledge-mcp.js` and `dist/index.js` — and the whole diff is 9 lines: * `"scripts/lib/remote-temp-dir.mjs"` added. package.json is inlined into the bundles and its `files` array gained that entry in 354a855 (#33), so the committed mcp bundle predates #33. Occurrences of `remote-temp-dir` in the committed artifacts at 64d05ae: bin/knowledge.js 1, bin/knowledge-mcp.js 0. * three zod codegen collapses of empty else blocks, `} else if (ctx.target === "openapi-3.0") {} else {}` -> `} else {}` removed. Dead-code elimination, semantically identical, absent from bin/knowledge.js only because that bundle is --minify'd. Both are absorbed rather than pinned around. The rebuild is deterministic: two consecutive builds produce byte-identical output for all six bundles (`cmp` clean), and the four unaffected bundles reproduce the committed bytes exactly. `bin/knowledge.js` is also rebuilt here, for one reason worth stating: package.json is inlined, so changing the `verify:generated` script line below changes that bundle too. Its only content change is that script string. CI never caught this because .github/workflows/ci.yml runs "Verify generated artifacts" AFTER "Run tests", and the pre-existing context-pack test failure aborts the job first. ## Making the check mean something The old script never rebuilt. On a clean checkout its exit 0 meant only "the files I did not touch are untouched". Measured against it earlier: exit 0 with src/cli.ts diverged from bin/knowledge.js, and exit 0 with bin/knowledge.js corrupted by 30 junk bytes. * ONE ENTRY POINT. The build moved inside the script, so it cannot be run without the rebuild and read as a sync check. `verify:generated` is now the script alone, and CI calls `bun run verify:generated` instead of reassembling the two halves. * NO HARDCODED ARTIFACT LIST. The diff gate covers the `bin` and `dist` directories instead of two paths, and the stale-pattern scan derives its files from `git ls-files`. The old four-file list had already drifted: it omitted bin/knowledge-serve.js and dist/serve.js. * A DIRTY-BEFORE PRECONDITION. A post-rebuild `git diff` only means something if the generated paths matched the index beforehand. It now fails with the offending paths named instead of reporting drift it cannot attribute. * THE PATTERNS PROVE THEMSELVES BEFORE A CLEAN RESULT IS TRUSTED. Each stale pattern carries a fixture it must match and a counter-fixture it must not; if either fails the script exits 1 rather than reporting every artifact clean. A regex that can no longer match anything reports "clean" forever. That self-check earned its place immediately: the first fixture written for `/path:\s*decodeURIComponent\([^)]*\.pathname\)/` used `decodeURIComponent(new URL(uri).pathname)` and the script rejected it, because `[^)]*` cannot cross a nested `)`. The fixture is now verbatim the line 2bea200 removed from src/source-ref.ts, and the known bound is documented at the pattern rather than left as a surprise. tests/generated-artifacts.test.ts covers the pieces: directory-wide gate, derived file list containing all six shipped bundles and nothing outside bin/dist, every pattern firing on its fixture and rejecting its counter-fixture, a synthetic stale bundle in a temp root producing one problem per pattern while the counter-fixtures come back clean, and `verify:generated` staying a single command. End-to-end script behaviour is deliberately not tested there — it would need a real build, which overwrites bin/ and dist/ while other test files run.
…e the check say so `bun run verify:generated` was RED on main at 64d05ae while `bun scripts/verify-generated-artifacts.mjs` was GREEN. One check, two names, two answers — and the passing one had been cited in CHANGELOG.md as evidence that a rebuilt bundle matched its source, which it never checked. ## The stale artifacts bun scripts/verify-generated-artifacts.mjs -> exit 0 bun run verify:generated -> exit 1 package.json:58 prefixed a `bun run build`, and the rebuild did not reproduce the committed files. Rebuilding from clean main leaves exactly two artifacts modified — `bin/knowledge-mcp.js` and `dist/index.js` — and the whole diff is 9 lines: * `"scripts/lib/remote-temp-dir.mjs"` added. package.json is inlined into the bundles and its `files` array gained that entry in 354a855 (#33), so the committed mcp bundle predates #33. Occurrences of `remote-temp-dir` in the committed artifacts at 64d05ae: bin/knowledge.js 1, bin/knowledge-mcp.js 0. * three zod codegen collapses of empty else blocks, `} else if (ctx.target === "openapi-3.0") {} else {}` -> `} else {}` removed. Dead-code elimination, semantically identical, absent from bin/knowledge.js only because that bundle is --minify'd. Both are absorbed rather than pinned around. The rebuild is deterministic: two consecutive builds produce byte-identical output for all six bundles (`cmp` clean), and the four unaffected bundles reproduce the committed bytes exactly. `bin/knowledge.js` is also rebuilt here, for one reason worth stating: package.json is inlined, so changing the `verify:generated` script line below changes that bundle too. Its only content change is that script string. CI never caught this because .github/workflows/ci.yml runs "Verify generated artifacts" AFTER "Run tests", and the pre-existing context-pack test failure aborts the job first. ## Making the check mean something The old script never rebuilt. On a clean checkout its exit 0 meant only "the files I did not touch are untouched". Measured against it earlier: exit 0 with src/cli.ts diverged from bin/knowledge.js, and exit 0 with bin/knowledge.js corrupted by 30 junk bytes. * ONE ENTRY POINT. The build moved inside the script, so it cannot be run without the rebuild and read as a sync check. `verify:generated` is now the script alone, and CI calls `bun run verify:generated` instead of reassembling the two halves. * NO HARDCODED ARTIFACT LIST. The diff gate covers the `bin` and `dist` directories instead of two paths, and the stale-pattern scan derives its files from `git ls-files`. The old four-file list had already drifted: it omitted bin/knowledge-serve.js and dist/serve.js. * A DIRTY-BEFORE PRECONDITION. A post-rebuild `git diff` only means something if the generated paths matched the index beforehand. It now fails with the offending paths named instead of reporting drift it cannot attribute. * THE PATTERNS PROVE THEMSELVES BEFORE A CLEAN RESULT IS TRUSTED. Each stale pattern carries a fixture it must match and a counter-fixture it must not; if either fails the script exits 1 rather than reporting every artifact clean. A regex that can no longer match anything reports "clean" forever. That self-check earned its place immediately: the first fixture written for `/path:\s*decodeURIComponent\([^)]*\.pathname\)/` used `decodeURIComponent(new URL(uri).pathname)` and the script rejected it, because `[^)]*` cannot cross a nested `)`. The fixture is now verbatim the line 2bea200 removed from src/source-ref.ts, and the known bound is documented at the pattern rather than left as a surprise. tests/generated-artifacts.test.ts covers the pieces: directory-wide gate, derived file list containing all six shipped bundles and nothing outside bin/dist, every pattern firing on its fixture and rejecting its counter-fixture, a synthetic stale bundle in a temp root producing one problem per pattern while the counter-fixtures come back clean, and `verify:generated` staying a single command. End-to-end script behaviour is deliberately not tested there — it would need a real build, which overwrites bin/ and dist/ while other test files run.
Secondary remediation for the 2026-07-24 data-destruction incident. The primary fix is
hasna/hooks#10; this closes the same class of
defect in this repo. Separate PR by design.
The defect
scripts/smoke-machine-sync-release.mjssetremoteDirfrom an unchecked remotemktemp -d, then later deleted it over ssh:mktemp -dprints its error on stderr and nothing on stdout, so a failure leavesremoteDirempty — the same failure mode asbun pm cache, which is what turnedrm -rf "$(bun pm cache)"/*intorm -rf /*on station02.This is inert today only because
shellQuote('')renders as'', so the delete degradesto
rm -rf ''. It is one appended/*away from catastrophic, and the value is alsointerpolated into
--peer-workspace. Relying on a quoting helper to neutralise a value thatshould never have been accepted is exactly the assumption this incident punished.
The fix
New
scripts/lib/remote-temp-dir.mjsexportingassertRemoteTempDir, applied at theassignment and re-asserted immediately before the
rm -rf.It accepts only what the template could actually have produced.
mktempreplaces thetrailing run of
Xin place, so a valid result is the template's fixed prefix followedby a single non-empty path component. Checking only the template's parent directory is not
enough — that would accept
/tmp/somebody-elses-dirand delete it. The bound is derived fromthe template rather than hardcoded, so it stays correct if the template moves.
Rejected: empty, whitespace-only, multi-line output, non-absolute paths,
/,/tmp,/usr,/home/hasna, a sibling directory sharing the temp parent, a path escaping via.., and amalformed template (relative, or without at least three
X).Packaging — a second, real bug this surfaced
smoke-machine-sync-release.mjsis a published file. Adding an import to it withoutregistering the new module would ship a package that throws
ERR_MODULE_NOT_FOUNDfor everyinstalled copy. The new module is therefore registered in all three deliberate allowlists
(
package.jsonfiles,scripts/validate-public-package.mjs,tests/package-release.test.ts)— that triplication is a review tripwire, so it is honoured rather than worked around.
The existing allowlists only catch the opposite mistake (shipping something unreviewed).
Nothing caught "a published script imports something unpublished", so this PR adds that test.
It immediately found pre-existing breakage.
scripts/apply-cloud-migrations.mjs:23statically imports
../src/storage.ts, butsrc/has never been infiles. Verified againsta packed tarball:
That path carries a single exemption named in the test, tracked as todos task
1eb481d5;removing the exemption is that task's acceptance criterion. It is not fixed here because
choosing between repointing at
dist, shippingsrc, or unpublishing the script is aseparate decision that deserves its own review — and silently deleting the test to make it
green would be precisely the "guard that passes while protecting nothing" trap.
Evidence
Full suite with
HASNA_KNOWLEDGE_API_URL/_KEYunset: 227 pass, 2 skip, 1 fail. That onefailure (
tests/cli.test.ts— "context pack and proposal context commands return boundedagent JSON") reproduces on the untouched baseline via
git stash, so it is pre-existing andunrelated. With those env vars set, 99 tests fail because the cloud-API flip disables the
local sqlite catalog — also environmental, also unrelated.
Negative controls
Guards are proven to fail on bad input, not merely to pass on good:
assertRemoteTempDirthrows for every rejected value listed above, each an explicit test case.missing path:
No
rmis executed by any test, at any scope. Only the validator is exercised.Scope
grepconfirms this was the only remotemktemp -dand the onlyssh … rm -rfin the repo.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.