From 432fbba7fa24946f3d568c338710acd216526e94 Mon Sep 17 00:00:00 2001 From: Garrett Allen <98648590+Gerrrt@users.noreply.github.com> Date: Wed, 16 Sep 2026 18:30:08 +0000 Subject: [PATCH] fix(check-docs): saying a host runs no Alloy counted it as running one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit count_alloy_agents tested `"alloy" in cell`, which cannot tell "Alloy agent (Docker)" from "runs no Alloy". Seven rows of docs/architecture.md say the second thing. The bug was survivable only by accident: every one of them is also marked Not built yet, so NOT_BUILT masked the negation — and it would have surfaced on the commit that dropped that marker, which is the commit already busy changing this count for a real reason, and where a wrong count reads as a stale sentence in hardware.md rather than as a miscount. Found 2026-09-16 when smaug's row said "No Alloy agent — this host is scraped" and pushed the count to five. bahamut, leviathan, titan, ramuh, carbuncle and siren all carry "runs no Alloy" and were queued to do the same, one at a time, as the lab domain gets built. Negations are stripped before the substring test rather than the affirmative being matched, and that choice is the part worth recording. Matching "alloy agent" instead looks tidier and is wrong: prometheus names Alloy in a service list — "docker-socket-proxy, Alloy" — and never says "Alloy agent" at all, so that test drops a host which genuinely runs one and quietly reports three. An over-count fails loudly against hardware.md's sentence; an under-count is a checker agreeing with a document that has gone stale, which is the failure this whole file exists to prevent. Verified: deleting prometheus's Alloy mention takes the count 4 -> 3, so it was being counted by that bare word alone. Measured rather than asserted, by building each row in memory and counting both ways: scenario OLD NEW today, as committed 4 4 build bahamut (runs no Alloy) 5 4 old overcounts build smaug (runs no Alloy) 5 4 old overcounts build titan (runs no Alloy) 5 4 old overcounts build trinity (has an agent) 5 5 both rise, correct The last line is the one that matters for not weakening the assertion: a host that really does declare an agent still moves the count the moment its marker comes off, which is exactly when hardware.md's sentence should be forced to change. All seven negative rows hold at four; trinity and odin both rise to five. The regex knows two shapes, "runs no Alloy" and "No Alloy agent", and the comment says plainly that a negation it does not know would overcount — loudly, but it should be extended here rather than worked around by reaching for different words in the table. smaug's row was reworded to dodge this on 2026-09-16 (c938c37); that reword is now belt and braces rather than the fix. Refs #88 Co-Authored-By: Claude Opus 5 --- scripts/check_docs.py | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/scripts/check_docs.py b/scripts/check_docs.py index 0c92d89..22ec3b1 100755 --- a/scripts/check_docs.py +++ b/scripts/check_docs.py @@ -360,6 +360,18 @@ def count_notifying_receivers() -> int: ) +# A Contents cell saying a host runs NO Alloy agent. Stripped before the +# substring test below, so that saying so does not read as saying the opposite. +# +# "no Alloy" covers both shapes the table uses: "runs no Alloy" on the six lab +# machines ADR-0029 scrapes rather than instruments, and "No Alloy agent" on a +# host that is scraped for a different reason. A negation this does NOT know +# would overcount — the failure is loud rather than silent, because the count +# is asserted against hardware.md's sentence, but it is worth extending here +# rather than reaching for a different phrasing in the table. +NO_ALLOY = re.compile(r"\bno\s+alloy\b", re.I) + + def count_alloy_agents() -> int: """Hosts the architecture table says run an Alloy agent. @@ -375,6 +387,24 @@ def count_alloy_agents() -> int: convenience: dropping the marker on the commit that builds the host pushes this count to four and fails hardware.md's "three Alloy agents" in the same run, which is exactly when that sentence should be forced to change. + + SAYING A HOST RUNS NO ALLOY USED TO COUNT IT AS RUNNING ONE. The test was a + bare `"alloy" in cell`, which cannot tell "Alloy agent (Docker)" from "runs + no Alloy". It was survivable only by accident: every row that says so is + also marked NOT_BUILT, so the negation was masked by the exclusion above — + and would have surfaced on the commit that dropped the marker, which is the + commit already busy changing this count for a real reason. Found 2026-09-16 + when `smaug`'s row said "No Alloy agent" and pushed the count to five; six + further rows (`bahamut`, `leviathan`, `titan`, `ramuh`, `carbuncle`, + `siren`) carry "runs no Alloy" and are waiting to do the same. + + Negations are STRIPPED rather than the affirmative being matched, and that + is the part worth writing down. Matching `alloy agent` instead looks + tidier and is wrong: `prometheus` names Alloy in a service list — "… + docker-socket-proxy, Alloy" — and never says "Alloy agent" at all, so that + test would drop a host which genuinely runs one and quietly report three. + An over-count fails loudly against hardware.md; an under-count would have + been a checker agreeing with a stale sentence. """ tables = tables_under( ARCH_MD.read_text(encoding="utf-8"), @@ -385,7 +415,7 @@ def count_alloy_agents() -> int: return sum( 1 for row in tables[0][1:] if len(row) > 3 - and "alloy" in strip_md(row[3]).lower() + and "alloy" in NO_ALLOY.sub("", strip_md(row[3])).lower() and not NOT_BUILT.search(row[3]) )