tests/scripts/test_check_site.py:158 asserts the number of rendered benchmark detail links against a literal:
self.assertEqual(len(detail_hrefs), 10)
Ten is the count of docs/benchmarks/*.md today. It is a measurement of one file set stored inside another file, which this repository's records rule already forbids for records and which behaves the same way here: the next change that adds or removes a benchmark detail page reds a test that has nothing to do with it, in a job (agent-record) that owns none of the benchmark surfaces.
Why it is worth raising now rather than later
Until #1754 the literal was inert in CI: the agent-record runner had no Hugo, so the case never reached line 158. Installing Hugo on that lane is what arms it. The lane fix is therefore the change that converts a latent drift lock into a live one, and it should not land without the drift lock being owned.
The relationship is a bijection, and it is derivable
Measured on d60692c89 with Hugo 0.146.3+extended, rendering website/ and parsing public/docs/benchmarks/index.html:
docs/benchmarks/*.md stems: 10 — at-a-glance, dwarfstar-gguf, how-we-measure, llama-cpp-cpu, memory, mlx-lm-apple-m4, open-gaps, reproduce, speculative-decoding, vllm-online-serving.- Table hrefs in the rendered index: 10, all of them detail links.
- Duplicated targets: none. Slugs with no link: none.
So the literal 10 is standing in for len(detail_slugs), and the stronger statement — the set of linked basenames equals the set of on-disk slugs — holds exactly today. Deriving it at read time removes the drift lock and catches a case the literal cannot: a page that gains a file and loses its index row keeps the count at 10.
A derived assertion needs its own non-vacuity floor. sorted(x) == sorted(y) is satisfied by two empty sets, so an empty docs/benchmarks/ would turn the case into a pass. Whatever lands must assert the slug set is non-empty in the same change.
Not fixed in flow, deliberately
This changes what the case asserts, not only how it spells a number, and AGENTS.md routes a semantic change to a checker through the normal row, spec and fresh-review path rather than the in-flow rule. It also touches a file that PR #1726 has open against the same case.
Owner
GATE-CI-SITE-HUGO-LANE, listed under ## Owed in .agents/specs/gate-ci-site-hugo-lane.md.
tests/scripts/test_check_site.py:158asserts the number of rendered benchmark detail links against a literal:Ten is the count of
docs/benchmarks/*.mdtoday. It is a measurement of one file set stored inside another file, which this repository's records rule already forbids for records and which behaves the same way here: the next change that adds or removes a benchmark detail page reds a test that has nothing to do with it, in a job (agent-record) that owns none of the benchmark surfaces.Why it is worth raising now rather than later
Until #1754 the literal was inert in CI: the
agent-recordrunner had no Hugo, so the case never reached line 158. Installing Hugo on that lane is what arms it. The lane fix is therefore the change that converts a latent drift lock into a live one, and it should not land without the drift lock being owned.The relationship is a bijection, and it is derivable
Measured on
d60692c89with Hugo 0.146.3+extended, renderingwebsite/and parsingpublic/docs/benchmarks/index.html:docs/benchmarks/*.mdstems: 10 —at-a-glance,dwarfstar-gguf,how-we-measure,llama-cpp-cpu,memory,mlx-lm-apple-m4,open-gaps,reproduce,speculative-decoding,vllm-online-serving.So the literal
10is standing in forlen(detail_slugs), and the stronger statement — the set of linked basenames equals the set of on-disk slugs — holds exactly today. Deriving it at read time removes the drift lock and catches a case the literal cannot: a page that gains a file and loses its index row keeps the count at 10.A derived assertion needs its own non-vacuity floor.
sorted(x) == sorted(y)is satisfied by two empty sets, so an emptydocs/benchmarks/would turn the case into a pass. Whatever lands must assert the slug set is non-empty in the same change.Not fixed in flow, deliberately
This changes what the case asserts, not only how it spells a number, and
AGENTS.mdroutes a semantic change to a checker through the normal row, spec and fresh-review path rather than the in-flow rule. It also touches a file that PR #1726 has open against the same case.Owner
GATE-CI-SITE-HUGO-LANE, listed under## Owedin.agents/specs/gate-ci-site-hugo-lane.md.