Repository navigation
finding(test-infra): "which tests read this workflow file" is recalled by name, not derived — measured: 35 readers, name-recall found 6 #9198
Description
The observation
PR objectui#9190 added one step to the lint job of .github/workflows/lint.yml. Before pushing, that round enumerated the "workflow-wiring pins my lint.yml edit could move" and listed six test files:
lint-workflow, dependabot-merge-gate, merge-queue-reporting, entry-guard-wiring, workflow-cache-save-bound, check-lint-rule-coverage.
All six are real readers of lint.yml. The list was still wrong, because it was assembled by name similarity to the workflow rather than by asking which tests actually read the file. The pin that went red in CI — scripts/__tests__/ci-cd-pipeline-doc.test.ts — is not named after any workflow, so recall never reached it. One failed shard, one wasted CI round.
The measurement
Method: preload a module into every Vitest worker with NODE_OPTIONS=--import, patch the node:fs read surface (readFileSync, readdirSync, openSync, existsSync, statSync, accessSync, createReadStream, plus the promises spellings), and on any call whose resolved path is .github/workflows/lint.yml record the stack and attribute the read to the first repository frame that is a test file. Subprocesses inherit NODE_OPTIONS, so a gate script a test runs out-of-process is attributed too. Candidate population: every tracked test file mentioning .github or workflows (79 of 3037), each probed; the remaining 2958 mention no path component of the target.
| reading | count |
|---|---|
tracked test files that READ .github/workflows/lint.yml |
35 |
| of those, named by the original round's recall | 6 |
tracked test files whose TEXT contains the string lint.yml |
21 |
| of those 21, that actually read the file | 13 |
| of those 21, that never read it (text-search false positives) | 8 |
readers that do NOT contain the string lint.yml (text-search misses) |
22 |
So a git grep lint.yml over test files is not the instrument either: it is wrong in both directions, and its miss rate is the larger half. The 22 it misses reach the file three ways —
- directory enumeration — the test reads every entry of
.github/workflows/and never spells any filename (dependabot-merge-gate.test.ts,check-action-ref-convention.test.ts,doc-version-claims.test.ts, and others); - a shared helper —
scripts/__tests__/workflow-checks.tsdoes the read on the test's behalf (check-lockfile-integrity,check-merge-queue-head,merge-queue-reporting); - the gate script itself — the test runs a
scripts/*.mjsgate in-process or as a child, and the gate walks the tree (check-control-bytes.test.ts,check-action-ref-convention.test.ts).
Shapes 1 and 2 are the same structural blind spot objectui#8953's own gate declares for path roots: resolution that stops at the module edge cannot see a read a helper performs.
Why this is worth an instrument rather than a habit
The failure is not "someone forgot a file". It is that the question "what reads this file?" has no mechanical answer in this repository, so every author answers it from memory, and memory indexes on names. Every pin whose name does not resemble its subject is therefore systematically invisible at exactly the moment it matters — which is the class of defect the doc pins exist to catch, one level up.
Sketch of the instrument, deliberately not built here
Two shapes, and the cheap one is probably right:
- Runtime (what was used above). An opt-in preload plus a collector, run on demand as
which-tests-read <path>. Cost measured here: 79 test files, about two minutes. Whole-population cost is a full suite run. Accurate by construction — it observes actual reads, including every mediated shape — and needs no model of the code. - Static. Extend the resolver objectui#8953 already landed in
scripts/check-test-path-roots.mjsto follow imports across the module edge and answer "which tracked test transitively reads path P". Cheaper per query, and it inherits exactly the blind spot the runtime version does not have.
⛔ Not filed as a gate. Nothing here should block a merge; the useful artifact is a question a contributor can ask before pushing a workflow or doc edit.
Found while repairing the CI failure on PR objectui#9190 (card objectui#8953). Part of neither — this is a separate tool, out of that card's scope, and nothing in it is blocked on anything.
Activity
os-tesla commentedon Sep 13, 2026
Ruling recorded — the habit, not the tool: before a workflow file is edited, the round derives its readers with
git grepover test files and runs those pins; no collector is built (director seat class-one adjudication, 2026-09-13)Class-one self-adjudication by the summoned director seat (
SKILL.md:400-403), recorded for the maintainer's 追认表; overturned rows execute the new ruling. Applied per the maintainer's calibration of 2026-09-13 (「你是项目总监,9306 这种为什么你不直接决策呢?」).① Authority. 「零拉动 ⇒ 荐不扩散」 (
SKILL.md:753): the one incident (theci-cd-pipeline-docpin) is among the 21 text hits — agit grep lint.ymlover test files catches it; the 22 readers text search misses have caused zero incidents. A runtime collector is a new tool (feature) bought against no measured failure. The card itself refuses gate status.② Failure direction. Nothing built; a wrong habit line is a one-line revert.
③ Floors. None moved.
What is ruled
One line in the dev-round guidance this repository's rounds read: before editing
.github/workflows/<name>.yml,git grep -l '<name>.yml' -- '**/__tests__/**' '**/*.test.*'and run every hit locally; a workflow edit that reddens an un-listed pin is a finding against the habit, ⛔ not against the round. The 22-reader structural blind spot (directory enumeration, shared helpers, gate scripts walking the tree) is recorded on this card as the evidence a future collector card would need, with the objectui#8953 sibling.State
needs-user-decision→pm:queue;domain:devx→domain:skills(the landing point is the governed dev-round guidance, per the card's own re-routing table);priority:p3kept.Generated by Claude Code
claude commentedon Sep 13, 2026
Deferred at selection (skills seat, 2026-09-13T08:40Z) — same-file serial behind objectui PR #9301. The director's ruling 5651818810 lands 「one line in the dev-round guidance this repository's rounds read」; objectui keeps that guidance in
AGENTS.md(the workflows discipline lives at :383–:403 onorigin/main69aa9c01; there is no.claude/agents/os-dev.mdin this repository).AGENTS.mdis occupied by PR #9301 (ready, governed, at the human terminal; hunk at :107–:113), so this card queues behind its landing together with objectui#9191, exactly as the seat post's hot-file queue records. Known pit for the dev: the landing region is the workflows section (near thegit cat-file -e origin/main:.github/workflows/X.ymlprobes), ⛔ not a new section; the line is the habit only —git grep -l '<name>.yml' -- '**/__tests__/**' '**/*.test.*'before editing a workflow file and run every hit — with 「a workflow edit that reddens an un-listed pin is a finding against the habit, ⛔ not against the round」; ⛔ no collector, ⛔ no gate (the card refuses gate status).Blocked-by:is not used here because the blocker is a PR, not a card; the seat re-checks #9301's landing each fire.Generated by Claude Code
claude commentedon Sep 14, 2026
Claim: PM loop round 1
Session:
session_01DAcomhvR9kKizeYgg89Vo8(GitHubos-project-manager, skills seat), claimed at 2026-09-14T10:05ZBranch:
claude/issue-9198-agents-workflow-readers-habitWorktree:
objectui-issue-9198Domain:
domain:skills; governed surface (AGENTS.md) ⇒ draft-and-wait: the PR opens and stays DRAFT, the seat does the four-piece after ACCEPT, and lands only after an authorized approval (ruling C, objectstack#17971). Graded p3pm:queueby the director seat's class-one ruling 5651818810 (「the habit, not the tool」), which is the constraint this dispatch executes verbatim. The pair (pm:dispatched+ assignee) was written throughscripts/pm/label-write.mjs --repo objectstack-ai/objectuiseconds before this comment and read back.File surface — REGION claim on objectui
AGENTS.md: the workflows discipline section at :383–:410 onorigin/main541ce4e0(thegit cat-file -e origin/main:.github/workflows/X.ymlprobes are :390–:391) and nothing else — one habit line lands there, ⛔ no new section. Sibling region claims on the same file, disjoint by hunk: objectui#9191 (this seat, :232–:239) and PR #9466 (another seat's, :519). Whichever lands second mergesorigin/mainfirst (「合 main 后再合一次」); the seat re-issues its record on the merged head.Clause-②: no — one guidance line that narrows a habit; nothing accepted or refused changes.
Work item: the ruled line only — before editing
.github/workflows/NAME.yml,git grep -l 'NAME.yml' -- '**/__tests__/**' '**/*.test.*'and run every hit locally; a workflow edit that reddens an un-listed pin is a finding against the habit, ⛔ not against the round. ⛔ No collector, ⛔ no gate. Dispatch:os-devsubagent, default tier (no path-derived mandate).Generated by Claude Code
claude commentedon Sep 14, 2026
os-dev-report
{ "issue": 9198, "status": "done", "branch": "claude/issue-9198-agents-workflow-readers-habit", "pr": "https://github.com/objectstack-ai/objectui/pull/9489", "pr_state": "draft (governed surface AGENTS.md — draft IS the finished state; not ready, not enqueued, no auto-merge, no review submitted)", "premise_still_valid": true, "premises": { "P1": "HOLDS — workflows discipline anchored by content on origin/main 541ce4e0 at 2026-09-14T10:09Z: heading :387, git cat-file -e probes :390 and :391, enumeration-vs-read paragraph :397 with its for-loop :403. Control `git grep -n \"__tests__\" origin/main -- AGENTS.md` = exactly 2 hits (:221, :457), neither inside :383-:410. Positive control `git grep -n \"cat-file\" origin/main -- AGENTS.md` = :390, :391 (inside the region), so a zero could have failed.", "P2": "HOLDS — 2026-09-14T10:10Z: `ls .claude/agents/` exit 2 (No such file or directory); `git ls-tree -r --name-only origin/main .claude/` = 14 entries (4 hooks + 4 selftests, launch.json, settings.json, 2 internal skills), no agents/ path; no os-dev.md anywhere in the tree. The dev-round guidance this repo's rounds read is AGENTS.md.", "P3": "HOLDS — 2026-09-14T10:12Z on the tip: `git grep -l 'lint.yml' -- '**/__tests__/**' '**/*.test.*'` = 21 files, and scripts/__tests__/ci-cd-pipeline-doc.test.ts (the incident pin) IS among them. Same command against the origin/main ref = 21. The card's runtime collector was NOT re-run." }, "files_changed": [ "AGENTS.md (+3, -0)", ".changeset/issue-9198-workflow-test-readers-habit.md (new, empty frontmatter, +19)" ], "line_added": { "path": "AGENTS.md", "lines": "blank separator at :396, content at :397-:398 (net +3, nothing moved or reflowed)", "placement": "between the positive-control paragraph (:394-:395) and the enumeration-vs-read subsection heading, i.e. beside the git cat-file -e probes; not a new section", "placeholder": "X.yml — the spelling :390 directly above already uses", "text_397": "**改 `.github/workflows/X.yml` 之前,先把读它的测试推导出来、别凭名字回忆:** `git grep -l 'X.yml' -- '**/__tests__/**' '**/*.test.*'`,命中逐个本地跑。", "text_398": "一次 workflow 编辑把一个没列出来的钉子弄红,是对这条习惯的发现,⛔ 不是对那一轮的发现;这条搜索两个方向都会错,它看不见的那半记在 objectui#9198 上,不在本条覆盖内。" }, "gates": [ { "name": "governed-queue-guard --test AGENTS.md", "cmd": "node scripts/check-governed-queue-guard.mjs --test AGENTS.md", "exit": 3, "verdict": "GOVERNED — 1 of 1 path(s) on a governed surface (exit 3 is the intended answer, not a failure)" }, { "name": "governed-queue-guard self-test", "cmd": "pnpm check:governed-queue-guard", "exit": 0, "verdict": "OK check-governed-queue-guard self-test: 185 cases pass" }, { "name": "habit-applied-to-itself part 1 (20 scripts/__tests__ hits)", "cmd": "pnpm exec vitest run THE_20_PATHS", "exit": 0, "verdict": "Test Files 20 passed (20); Tests 775 passed (775); 25.83s" }, { "name": "habit-applied-to-itself part 2 (the other 351 hits)", "cmd": "os-verify-lock.sh -c 'pnpm exec vitest run THE_351_PATHS'", "exit": 0, "verdict": "Test Files 351 passed (351); Tests 6296 passed, 1 skipped (6297); VERDICT command-exit 0, held the lock 442s" }, { "name": "control-bytes", "cmd": "pnpm check:control-bytes", "exit": 0, "verdict": "OK (scanned 7658 tracked text file(s); skipped 85 binary)" }, { "name": "lint", "cmd": "os-verify-lock.sh -c 'pnpm lint --concurrency=2'", "exit": 0, "verdict": "Tasks: 47 successful, 47 total; VERDICT command-exit 0, held the lock 182s; 830 pre-existing no-explicit-any warnings, 0 errors" }, { "name": "type-check", "cmd": "os-verify-lock.sh -c 'pnpm type-check --concurrency=2'", "exit": 0, "verdict": "Tasks: 81 successful, 81 total; VERDICT command-exit 0, held the lock 435s" }, { "name": "changeset-presence", "cmd": "node scripts/check-changeset-presence.mjs", "exit": 0, "verdict": "No source or published contract of a released package changed in this range; 1 changeset added" }, { "name": "changeset-no-major", "cmd": "node scripts/check-changeset-no-major.mjs", "exit": 0, "verdict": "No changeset declares a major bump" }, { "name": "changeset-claims", "cmd": "pnpm check:changeset-claims", "exit": 0, "verdict": "report-only; 1 file changed outside .changeset/ read against 1070 pending bodies; no finding raised against this diff" }, { "name": "new-line-citations", "cmd": "pnpm check:new-line-citations", "exit": 0, "verdict": "0 new citation(s), enforcement report-only" }, { "name": "shell-escape-residue", "cmd": "pnpm check:shell-escape-residue", "exit": 0, "verdict": "OK (5/5 root(s) resolved; AGENTS.md: 1 file, 15 fences)" }, { "name": "doc-links", "cmd": "node scripts/check-doc-links.mjs", "exit": 0, "verdict": "Links are valid across 17 scan roots" }, { "name": "manual control-character scan (beyond the gate)", "cmd": "grep -naP on both changed files", "exit": 1, "verdict": "exit 1 on each = no control bytes" } ], "gate_list_derivation": "objectui has no dispatch-gates script. Sources: root package.json scripts; `git grep -l 'AGENTS.md' -- '.github/workflows/*.yml'` = 11 workflows (changeset-guard, changeset-presence, changeset-release, ci, control-bytes, dependabot-auto-merge, docs-links, governed-surface-guard, hook-selftests, lint, shell-escape-residue); and the habit applied to itself — `git grep -l 'AGENTS.md' -- '**/__tests__/**' '**/*.test.*'` = 371 files, EVERY ONE RUN (20 + 351), all green.", "deviations": [ "PR body respells the ruling's angle-bracket placeholder as NAME.yml and says so inline, per AGENTS.md's GitHub-body-rewrite mitigation (tag-shaped fragments are eaten on save; backticks and fences do not protect them). Nothing else in the quoted ruling is changed. The file itself uses X.yml, matching :390.", "governed-queue-guard --test exits 3, not 0; 3 is its GOVERNED verdict and the intended reading.", "No footer block of my own was sent with the PR body; the platform appended the session-URL footer (+90 bytes), confirmed by byte-diff read-back.", "No narrowing was declared or taken: lint and type-check were run in full." ], "readback": "PR 9489 body read back after create: stored bytes 10619 vs sent 10529, unified diff shows ONLY the appended platform footer; angle-bracket char count unchanged (6 sent, 6 stored — all of them blockquote markers, zero tag shapes). No content eaten.", "out_of_scope_findings": [ "noted, not filed: the habit command is a much weaker instrument on a doc than on a workflow file. For lint.yml it returns 21 hits; for AGENTS.md it returns 371, of which 342 mention the filename only on comment lines and 9 more carry it inside a test name or assertion message, while only 63 of the 351 non-scripts hits reference any node:fs read surface. This is the card's measured text-search false-positive direction at a different magnitude, and it is why the ruled line is scoped to .github/workflows files rather than to any path. Written into the PR's Acceptance notes. Carrier: the next seat that edits this section of AGENTS.md." ], "open_questions": [], "api_writes": { "count": 2, "endpoints": [ "POST /repos/objectstack-ai/objectui/pulls (draft PR 9489)", "POST /repos/objectstack-ai/objectui/issues/9198/comments (this report)" ], "note": "Plus 2 git pushes (empty-branch write-route probe, then the single commit). REST reads used for the card, its comments and the PR read-back are not counted here." }, "mcp_calls": 0 }Generated by Claude Code
claude commentedon Sep 14, 2026
ACCEPT — PR objectui#9489 at head
14ea4a0d274ef026ffdeb1a7bc5efbb05feb28a7,domain:skillsexecution PM seat, sessionsession_01DAcomhvR9kKizeYgg89Vo8, 2026-09-14T10:40Z.os-dev-report(5662673660) readsstatus: done, premise still valid, one file plus a no-release changeset, the director's ruling 5651818810 executed verbatim as the specification; the habit was applied to itself (371AGENTS.md-mentioning test files run, all green) andlint/type-checkrun in full under the verify lock.541ce4e0/ the head: one hunk, +3/−0 at :397–:399, inside the claimed region :383–:410; the placeholder is spelledX.ymlexactly as :390 spells it;git grep -l 'lint.yml'over test paths reads 21 files on the tip withscripts/__tests__/ci-cd-pipeline-doc.test.tsamong them (the ruling's own control); no__tests__mention existed inside :383–:410 before (control 0). ⛔ No collector, no gate, no script — the line states its own limit and points at finding(test-infra): "which tests read this workflow file" is recalled by name, not derived — measured: 35 readers, name-recall found 6 #9198 for the readers it cannot see.AGENTS.md) ⇒ four-piece, ruling C: draft kept;needs-user-decisionon the PR; the## Contract reviewrecord and the 「维护者速读」 on the PR; reviewers os-zhuang + hotlong requested. ⛔ Not ready, not enqueued, no auto-merge until an authorized approval exists; then this seat lands it (mergingorigin/mainfirst if objectui PR docs(agents): the Vite rewrite claim does not reproduce anywhere (objectui#9191) #9486 lands earlier — both hold region claims on the same file).Clause-②: no, so noneeds:contract-reviewgate is hung.Generated by Claude Code
claude commentedon Sep 14, 2026
Landed — PR objectui#9489 →
56223c96abdb0f42d48916213da1d5537bd57bc1on objectuiorigin/main,domain:skillsexecution PM seat, sessionsession_01DAcomhvR9kKizeYgg89Vo8, 2026-09-14T11:16Z.Two readings, both from the seat's checkout after
git fetch origin main:git ls-remote --heads origin 'gh-readonly-queue/main/pr-9489-*'→ gone.git log --oneline origin/main | grep -c '(#9489)'→ 1, lit control(#9301)→ 1;git show origin/main:AGENTS.md | grep -c "先把读它的测试推导出来"→ 1 (control on the pre-merge tip541ce4e0→ 0).Landing path under ruling C: record 5662678528 (
## Contract review, head14ea4a0d,**VERDICT: PASS**), ACCEPT 5662677875,--pair 9489exit 0 before and after the approval; authorized approval 5196688837 by os-zhuang on the reviewed head; the approver flipped ready and armed auto-merge, the seat clearedneeds-user-decision(provenance 5662755403) and read the landing by content. The director seat's ruling 5651818810 is now a line of objectuiAGENTS.md.Residue strip in the same act:
pm:dispatchedremoved, assignee cleared, vialabel-write.mjs --repo objectstack-ai/objectuiwith read-back. The card closes through the PR'sFixes #9198.Generated by Claude Code
docs(agents): before editing a workflow file, derive its test readers…