Repository navigation
Commit 398e907
fix(scripts): error-status conformance walk skips an entry that vanished between readdir and stat (#20267)
Fixes #20225
Clause-②: no
## What changed
One file: `scripts/check-error-status-conformance.mjs`.
- `walk()` now stats each listed entry through `statListedEntry()`. That
function tolerates exactly one error: `ENOENT` from the stat of an entry
that the parent's `readdirSync` just listed, and only when an `lstat`
cannot find the entry either. In that case the entry VANISHED and is
skipped. Every other error still throws.
- `--self-test` gains battery 28, with 10 cases on real disk. The roster
floor goes from 28 to 29 and the case total from 60 to 70.
- `deriveWireFace()` keeps its signature, so
`check-error-status-conformance.d.mts` is unchanged.
- No package source changes and no published surface moves. Root scripts
live in the private root package, so this PR carries no changeset.
## Why: the mechanism, measured
- `packages/spec/src/api/error-catalog-docs.test.ts` calls
`deriveWireFace()` from `@objectstack/spec#test:repo`. CI's `Test Core`
job runs `pnpm turbo run test test:repo` (`.github/workflows/ci.yml`).
That turbo invocation also builds packages while the walk runs, because
the test tasks depend on `^build`.
- tsup 8.5.1 loads its config through bundle-require 5.1.0.
bundle-require writes `tsup.config.bundled_ID.mjs` next to the config,
imports it and unlinks it (`getOutputFile` and `unlink` in
`bundle-require/dist/index.js`). Vite 8.0.16 writes its bundled config
under `node_modules/.vite-temp/` instead, and the walk skips
`node_modules`.
- The old `walk()` called `readdirSync` and then a bare `statSync` on
each entry. An entry the listing named could be gone before its stat
ran.
## Reproduced first, on `origin/main`
The probe lives in the scratchpad and is not committed. A sibling
process loops create and unlink of
`packages/spec/tsup.config.bundled_probe.mjs`, while
`deriveWireFace(worktree)` runs in the parent. The "before" module is a
byte-identical copy of the script at `443b2f4f` (blob `e31c273b`). The
"after" module is a copy of this PR's head `1c249be6` (blob `309db59a`,
equal to `git rev-parse
HEAD:scripts/check-error-status-conformance.mjs`). Both legs ran under
`scripts/pm/os-verify-lock.sh`.
| leg | module | runs | ok | ENOENT |
|---|---|---|---|---|
| control, no churn | `443b2f4f` | 1 | 1 | 0 |
| churn | `443b2f4f` | 8 | 4 | **4**: `ENOENT: no such file or
directory, stat '…/packages/spec/tsup.config.bundled_probe.mjs'` at
`walk (check-error-status-conformance.mjs:1832:15)` |
| churn | `1c249be6` (this PR) | **24** | **24** | **0** |
Every run that finished read **2674** sources, in all three legs. The
churner was stopped after each leg, and no probe file was left behind
(`leftBehind: false` in both churn legs).
## The rule: what it skips and what still throws
| case | answer | pinned by |
|---|---|---|
| a listed entry that is gone (stat ENOENT, then lstat ENOENT) | skipped
| 28, 28b, 28c |
| absent at the stat, back under the same name by the lstat (not a
symlink) | scanned as present; lstat IS its stat | 28d |
| a dangling symlink (stat ENOENT, lstat finds the link) | **throws**
`ENOENT` from `stat`. Skipping it would shrink the corpus, the same case
`check-startup-registry-verdict.mjs` pins as "an error, not a smaller
corpus" | 28e |
| `ENOTDIR` | **throws** (produced for real: the listed directory is
replaced by a file) | 28f |
| `EACCES` on the stat | **throws** (injected: root bypasses permission
bits, and CI and this container run as root) | 28g |
| `ENOENT` on the stat, then any other error on the lstat | **throws**
that error (injected) | 28h |
| a missing scan ROOT | **throws** `ENOENT` naming the root | 28i |
| nothing vanishing | every source-shaped file is read, including the
one the churn deleted | 28j |
- The rule does not match on names, because the next transient file will
have a different one.
- `statSync(p, { throwIfNoEntry: false })` is not this rule. Measured on
Node v22.22.2, it also answers `undefined` for `ENOTDIR` and for a
dangling symlink.
- The header states what is **not** covered. A listed directory that
vanishes before its own `readdirSync` still throws, and so does a pushed
`.ts` file that vanishes before `readFileSync`. No producer of either
was found under the scan root.
The self-test's success line gains this clause, quoted verbatim:
> … and the corpus walk skips an entry that VANISHED between readdir and
stat (every present file still scanned) while a dangling symlink,
ENOTDIR, a permission error and a missing root still throw.
### The pins can fail: five ablations
For each ablation the fix was committed first. The mutation went through
`node scripts/ablation-replace.mjs` in wrap mode, where the anchor must
hit once and the blob must change. After each run the file was restored
to the HEAD blob `309db59a`, and `git diff HEAD` came back empty.
| ablation | expected red | observed |
|---|---|---|
| 1 vanished → rethrow (no tolerance) | 28, 28b, 28c | `3/70 case(s)
failed`: 28, 28b (`enoent=1`), 28c (`no corpus`) |
| 2 drop the dangling-symlink throw | 28e | `1/70`: 28e (`ENOENT open
…/dangling.ts`: it failed at the read, not at the stat) |
| 3 swallow every stat error | 28f, 28g | `2/70`: 28f, 28g |
| 4 naive ENOENT swallow (no lstat) | 28d, 28e, 28h | `3/70`: 28d, 28e,
28h |
| 5 tolerate ENOENT on the ROOT listing | 28i | `1/70`: 28i |
## Local verification (head `1c249be6`)
- `node scripts/check-error-status-conformance.mjs --self-test`: `✓ … 70
cases pass`.
- `pnpm --filter @objectstack/spec exec vitest run --project repo
--maxWorkers=2 --reporter=verbose src/api/error-catalog-docs.test.ts`:
`Test Files 1 passed (1)`, `Tests 5 passed (5)`. All five
`error-catalog.mdx ↔ the published wire face` cases are green.
- The gate families derived by `node scripts/pm/dispatch-gates.mjs
--commands --repo objectstack-ai/objectstack` on this branch are
identical to the dispatch's list (30 commands).
- All 30 were run at `1c249be6`. Every exit code was written to disk
before it was read, and all 30 are `0`. `node
scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --ran
FILE` answers `Run reconciliation — 30 derived, 30 run, 0 NOT-MEASURED,
0 UNRUN` and `✓ … (a DERIVED zero — all 30 recorded an exit code and
none of them is 3)`. The list includes `pnpm
check:error-status-conformance` (self-test plus the real run),
`check:entry-guard`, `check:parse-guard`, `check:declaration-mirrors` (+
`--self-test`), `check:self-test-wired` (+ `--self-test`) and
`check:nul-bytes`.
- The full CI farm is CI's job; this list is the derived subset, not an
enumeration of `lint.yml`.
## Acceptance notes
### Census: every readdir-then-stat walker in `scripts/**` and
`packages/*/scripts/**`
Method: `git ls-files` over both globs gives 474 script files. Of those,
183 call `readdirSync`/`readdir(`, and 85 also call a stat. Every hit
was read. A walker **can race a transient file** only when two things
hold: it scans a tree that a build writes transient entries into (a
package root or `dist/`), **and** it runs concurrently with that build.
In CI, concurrency exists only inside a turbo invocation that mixes
build and test tasks. So the second half was measured on these surfaces:
package tests that import or spawn a repo script, package `build`/`test`
npm scripts, and turbo `inputs`.
**Fixed here (1).** `scripts/check-error-status-conformance.mjs`
`walk()`. It scans `packages/` and runs inside
`@objectstack/spec#test:repo`, and its race was reproduced above.
**Same shape, over a tree a build writes to, but never concurrent with a
build. Recorded, not fixed.** On every path that runs them, each is a
sequential CI step (`lint.yml`, the type-check job) or a manual command.
No package test or build script runs any of them on the real tree: the
package tests that reach `check-regen-pending.mjs` pass sandbox
directories. They would race only if someone ran the gate while a build
ran in the same worktree.
- `packages/` walks with a bare per-entry `statSync`:
- `packages/spec/scripts/check-error-code-provenance.ts`
`collectSourceFiles`
- `scripts/check-dispatcher-error-vocabulary.mjs` `walkManifests`,
`walkSources`
- `scripts/check-doc-authoring.mjs` `collectPackageProseFiles`
- `scripts/check-error-code-casing.mjs` `walk`
- `scripts/check-init-service-contract.mjs` `walk`
- `scripts/check-objectql-double-limit.mjs` `testFilesUnder`
- `scripts/check-route-envelope.mjs` (two walks)
- `scripts/check-runtime-services-index.mjs` `walk`
- `scripts/check-settings-bind-window.mjs` `walk`
- `scripts/check-single-authz-resolver.mjs` `walk`
- `scripts/check-startup-registry-verdict.mjs` `collectSourceFiles`
- `scripts/check-where-matcher-conformance.mjs` `testFilesUnder`
- `scripts/check-wildcard-fallthrough.mjs` `walk`
- Whole-repo walks:
- `scripts/check-parse-guard.mjs` `walkOutside`
- `scripts/check-watch-hint-literal.mjs` `walk`
- Dirent walks that stat each matched file for its mtime:
- `scripts/check-type-check-coverage.mjs` `newestSourceMtime`
- `scripts/check-regen-pending.mjs` `newestMtime` (spec `dist/`,
`json-schema/`, `src/`). Its real-tree callers are `build-docs.ts`,
`check-generated.ts` and `lib/dist-freshness.ts`, and each runs after a
build.
- `packages/spec/scripts/check-generated.ts` `declarationFiles`. It
reads `dist/` after the forced build it performs itself, in the same
process.
**Same shape, over trees no build writes to. Cannot race a build
transient.**
- `src`-only roots:
- `packages/spec/scripts/build-docs.ts:136`
- `packages/spec/scripts/category-title.test.ts:47`
- `packages/spec/scripts/check-duration-unit-keys.ts`
- `scripts/check-driver-conformance.mjs` `walkTsInto`
- `scripts/check-kernel-hook-pairs.mjs`
- `scripts/check-meta-type-normalized.mjs`
- `scripts/check-sdui-lockstep.mjs`
- `scripts/check-spec-parsed-alias.mjs`
- `scripts/check-doc-authoring.mjs` `collectSpecSourceFiles`
- the sibling-directory `find`s at
`check-objectql-double-limit.mjs:1283` and
`check-where-matcher-conformance.mjs:1018`
- `packages/spec/scripts/check-llms-txt.ts` `readWorkspacePackages`,
which stats listed package directories
- Docs, skills, `.claude` and `.github` roots:
- `scripts/check-corpus-claim-drift.mjs`
- `scripts/check-doc-authoring.mjs` `walk`, `collectPublishedSkillFiles`
- `scripts/check-doc-route-spelling.mjs`
- `scripts/check-docs-spec-enumerations.mjs`
- `scripts/check-role-word.mjs`
- `scripts/check-skill-compatibility-version.mjs`
- `scripts/check-skill-frame-sync.mjs`
- `scripts/check-workflow-step-name-quoting.mjs`
- `scripts/check-self-test-wired.mjs` `walkActionFiles`
- `scripts/pm/check-skill-id-lint.mjs`
- `scripts/pm/dispatch-gates.mjs` `tierValueSites`
- `scripts/symbol-anchors.mjs`
- `scripts/` roots:
- `scripts/check-entry-guard.mjs`
- `scripts/check-parse-guard.mjs` `walk`
- `scripts/check-self-test-wired.mjs` `walkScripts`
- `scripts/pm/measurement-claim-triage.mjs`
- A manual codemod, run on a root the user names:
`scripts/codemod/apimethods-legacy-to-primitives.mjs`.
**Already tolerant, through a catch-all around the stat.** This is
broader than `ENOENT`, which is a different concern from this card's:
- `scripts/audits/14744-before-update-per-row-value-census.mjs` (two
walks)
- `scripts/check-cli-command-ids.mjs` (two walks)
- `scripts/check-durability-degradation-log-level.mjs`
- `scripts/check-logger-receiver-detach.mjs`
- `scripts/check-optional-error-sink-contract.mjs`
- `scripts/check-page-declaration-shape.mjs`
- `scripts/codemod/view-to-viewitem.mjs`
- `scripts/report-test-timings.mjs`
- `scripts/publish-smoke.sh`
- `packages/spec/scripts/analyze-bundle-size.ts` (`getFileSize`)
**Not the shape.** The remaining files list entries with `readdirSync(…,
{ withFileTypes: true })`, so no stat runs per entry, or they stat only
a root or a fixed path. That includes
`packages/spec/scripts/lazify-schemas.ts`,
`scripts/check-dts-closure.mjs` and
`scripts/check-sourcemap-no-sources-content.mjs`, whose only stat is of
the root before listing it.
**Shared helper (Zone 2.4).** Only one walker needed the tolerance, so
the three-walker threshold was not met. No new shared module was
created, and `scripts/lib/**` holds no readdir/stat helper today (none
of its files is among the 85).
### Outside this card's file surface (noted, not fixed)
- Test files under `packages/**/src` that walk the repo with a bare
readdir-then-stat are outside the census globs and outside the claim,
which allows no `packages/**/src` change. 12 test files call both
`readdirSync` and `statSync` on a repo-rooted path. One of them walks
the whole of `packages/` from inside a turbo test task:
`packages/lint/src/lint-startup-registry-verdict.corpus.test.ts`
`collectSourceFiles(packagesDir)`, in `@objectstack/lint#test`. I
extracted that function verbatim from the file, transpiled it with
esbuild and ran it over the real `packages/` under the same churn. It
threw `ENOENT … stat '…/packages/spec/tsup.config.bundled_probe.mjs'` in
**4 of 8** runs, while the other 4 read 2688 files. It is the same
mechanism as this card. It is reported to the seat for routing, and it
is not fixed here.
- `check-error-status-conformance.mjs`'s own `scanSources` reads each
pushed file with `readFileSync`, and the recursion calls `readdirSync`
on each listed directory. Neither is covered, as the header states,
because no producer was found. The measured one
(`tsup.config.bundled_ID.mjs`) is `.mjs` and is filtered out before the
read.
---
_Generated by [Claude
Code](https://claude.ai/code/session_01Rjy9MeetSfq34PKn81CRiN)_
Co-authored-by: Claude <noreply@anthropic.com>1 parent 1207baf commit 398e907
1 file changed
Lines changed: 263 additions & 11 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
153 | 153 | | |
154 | 154 | | |
155 | 155 | | |
156 | | - | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
157 | 161 | | |
158 | | - | |
| 162 | + | |
159 | 163 | | |
160 | 164 | | |
161 | 165 | | |
| |||
203 | 207 | | |
204 | 208 | | |
205 | 209 | | |
| 210 | + | |
206 | 211 | | |
207 | 212 | | |
208 | 213 | | |
209 | 214 | | |
210 | | - | |
| 215 | + | |
211 | 216 | | |
212 | 217 | | |
213 | 218 | | |
| |||
1751 | 1756 | | |
1752 | 1757 | | |
1753 | 1758 | | |
1754 | | - | |
| 1759 | + | |
| 1760 | + | |
| 1761 | + | |
| 1762 | + | |
| 1763 | + | |
| 1764 | + | |
| 1765 | + | |
| 1766 | + | |
| 1767 | + | |
| 1768 | + | |
| 1769 | + | |
| 1770 | + | |
| 1771 | + | |
| 1772 | + | |
| 1773 | + | |
| 1774 | + | |
| 1775 | + | |
| 1776 | + | |
| 1777 | + | |
| 1778 | + | |
| 1779 | + | |
| 1780 | + | |
| 1781 | + | |
| 1782 | + | |
| 1783 | + | |
| 1784 | + | |
| 1785 | + | |
| 1786 | + | |
| 1787 | + | |
| 1788 | + | |
| 1789 | + | |
| 1790 | + | |
| 1791 | + | |
| 1792 | + | |
| 1793 | + | |
| 1794 | + | |
| 1795 | + | |
| 1796 | + | |
| 1797 | + | |
| 1798 | + | |
| 1799 | + | |
| 1800 | + | |
| 1801 | + | |
| 1802 | + | |
| 1803 | + | |
| 1804 | + | |
| 1805 | + | |
| 1806 | + | |
| 1807 | + | |
| 1808 | + | |
| 1809 | + | |
| 1810 | + | |
| 1811 | + | |
| 1812 | + | |
| 1813 | + | |
| 1814 | + | |
| 1815 | + | |
| 1816 | + | |
| 1817 | + | |
| 1818 | + | |
| 1819 | + | |
| 1820 | + | |
| 1821 | + | |
| 1822 | + | |
| 1823 | + | |
| 1824 | + | |
| 1825 | + | |
| 1826 | + | |
| 1827 | + | |
| 1828 | + | |
| 1829 | + | |
| 1830 | + | |
| 1831 | + | |
| 1832 | + | |
| 1833 | + | |
| 1834 | + | |
| 1835 | + | |
| 1836 | + | |
| 1837 | + | |
| 1838 | + | |
| 1839 | + | |
| 1840 | + | |
| 1841 | + | |
| 1842 | + | |
| 1843 | + | |
| 1844 | + | |
| 1845 | + | |
| 1846 | + | |
| 1847 | + | |
| 1848 | + | |
| 1849 | + | |
| 1850 | + | |
| 1851 | + | |
| 1852 | + | |
| 1853 | + | |
| 1854 | + | |
| 1855 | + | |
| 1856 | + | |
| 1857 | + | |
| 1858 | + | |
| 1859 | + | |
| 1860 | + | |
| 1861 | + | |
| 1862 | + | |
| 1863 | + | |
| 1864 | + | |
| 1865 | + | |
| 1866 | + | |
| 1867 | + | |
| 1868 | + | |
| 1869 | + | |
| 1870 | + | |
| 1871 | + | |
| 1872 | + | |
| 1873 | + | |
| 1874 | + | |
| 1875 | + | |
| 1876 | + | |
| 1877 | + | |
| 1878 | + | |
| 1879 | + | |
| 1880 | + | |
| 1881 | + | |
| 1882 | + | |
| 1883 | + | |
| 1884 | + | |
| 1885 | + | |
| 1886 | + | |
| 1887 | + | |
| 1888 | + | |
| 1889 | + | |
| 1890 | + | |
| 1891 | + | |
| 1892 | + | |
| 1893 | + | |
| 1894 | + | |
| 1895 | + | |
| 1896 | + | |
| 1897 | + | |
| 1898 | + | |
| 1899 | + | |
| 1900 | + | |
| 1901 | + | |
| 1902 | + | |
| 1903 | + | |
| 1904 | + | |
| 1905 | + | |
| 1906 | + | |
| 1907 | + | |
| 1908 | + | |
| 1909 | + | |
| 1910 | + | |
| 1911 | + | |
| 1912 | + | |
| 1913 | + | |
| 1914 | + | |
| 1915 | + | |
| 1916 | + | |
| 1917 | + | |
| 1918 | + | |
| 1919 | + | |
| 1920 | + | |
| 1921 | + | |
| 1922 | + | |
| 1923 | + | |
| 1924 | + | |
| 1925 | + | |
| 1926 | + | |
| 1927 | + | |
| 1928 | + | |
| 1929 | + | |
| 1930 | + | |
| 1931 | + | |
1755 | 1932 | | |
1756 | 1933 | | |
1757 | 1934 | | |
| |||
1814 | 1991 | | |
1815 | 1992 | | |
1816 | 1993 | | |
1817 | | - | |
| 1994 | + | |
| 1995 | + | |
| 1996 | + | |
1818 | 1997 | | |
1819 | 1998 | | |
1820 | 1999 | | |
| |||
1825 | 2004 | | |
1826 | 2005 | | |
1827 | 2006 | | |
1828 | | - | |
1829 | | - | |
| 2007 | + | |
| 2008 | + | |
| 2009 | + | |
| 2010 | + | |
| 2011 | + | |
| 2012 | + | |
| 2013 | + | |
| 2014 | + | |
| 2015 | + | |
| 2016 | + | |
| 2017 | + | |
| 2018 | + | |
| 2019 | + | |
| 2020 | + | |
| 2021 | + | |
| 2022 | + | |
| 2023 | + | |
| 2024 | + | |
| 2025 | + | |
| 2026 | + | |
| 2027 | + | |
| 2028 | + | |
| 2029 | + | |
| 2030 | + | |
| 2031 | + | |
| 2032 | + | |
| 2033 | + | |
| 2034 | + | |
| 2035 | + | |
| 2036 | + | |
| 2037 | + | |
| 2038 | + | |
| 2039 | + | |
| 2040 | + | |
| 2041 | + | |
| 2042 | + | |
| 2043 | + | |
| 2044 | + | |
| 2045 | + | |
| 2046 | + | |
| 2047 | + | |
| 2048 | + | |
| 2049 | + | |
| 2050 | + | |
| 2051 | + | |
| 2052 | + | |
| 2053 | + | |
| 2054 | + | |
| 2055 | + | |
| 2056 | + | |
| 2057 | + | |
| 2058 | + | |
| 2059 | + | |
| 2060 | + | |
| 2061 | + | |
| 2062 | + | |
| 2063 | + | |
| 2064 | + | |
| 2065 | + | |
| 2066 | + | |
| 2067 | + | |
| 2068 | + | |
| 2069 | + | |
| 2070 | + | |
| 2071 | + | |
| 2072 | + | |
| 2073 | + | |
| 2074 | + | |
| 2075 | + | |
| 2076 | + | |
| 2077 | + | |
| 2078 | + | |
| 2079 | + | |
1830 | 2080 | | |
1831 | 2081 | | |
1832 | | - | |
1833 | | - | |
| 2082 | + | |
| 2083 | + | |
| 2084 | + | |
1834 | 2085 | | |
1835 | 2086 | | |
1836 | 2087 | | |
| |||
1840 | 2091 | | |
1841 | 2092 | | |
1842 | 2093 | | |
| 2094 | + | |
1843 | 2095 | | |
1844 | 2096 | | |
1845 | | - | |
| 2097 | + | |
1846 | 2098 | | |
1847 | | - | |
| 2099 | + | |
1848 | 2100 | | |
1849 | 2101 | | |
1850 | 2102 | | |
| |||
0 commit comments