Goal
Add a repository test that holds conformance/manifest.json to two structural invariants: case ids are unique, and every fixture on disk is named by exactly one case.
Why
The manifest is the authoritative index of the document-conformance corpus, and the count statements around the repository are now derived from it. Two ways it can drift quietly:
- a duplicated id — a copy-paste when adding a case. The suite would run one of them, or both under one name, and the count would still look right.
- an orphaned fixture — a file under
conformance/ that no case names. It is dead weight that looks like coverage, and a contributor adding a sibling case may reasonably assume it is exercised.
Neither is currently checked. The second is the more interesting one, because an unreferenced fixture is indistinguishable from a covered case when you are reading the directory.
Scope
- Add a test to
tests/test_repository.py.
- Assert that case ids are unique, naming any duplicate.
- Assert that every case's
path exists.
- Assert that every fixture file under the corpus directories is named by exactly one case, naming any that is not.
- Use
subTest so a failure identifies the id or the path rather than only the count.
- Do not add, remove, or rename any fixture or case in this issue. If the test finds an orphan, report it in the pull request and let a maintainer decide.
Acceptance criteria
Contributor learning
Treating an index as an artifact with its own invariants, and the difference between a file existing and a file being exercised.
Goal
Add a repository test that holds
conformance/manifest.jsonto two structural invariants: case ids are unique, and every fixture on disk is named by exactly one case.Why
The manifest is the authoritative index of the document-conformance corpus, and the count statements around the repository are now derived from it. Two ways it can drift quietly:
conformance/that no case names. It is dead weight that looks like coverage, and a contributor adding a sibling case may reasonably assume it is exercised.Neither is currently checked. The second is the more interesting one, because an unreferenced fixture is indistinguishable from a covered case when you are reading the directory.
Scope
tests/test_repository.py.pathexists.subTestso a failure identifies the id or the path rather than only the count.Acceptance criteria
python -m unittest discover -s tests -vpasses on the unmodified tree.git commit -s).Contributor learning
Treating an index as an artifact with its own invariants, and the difference between a file existing and a file being exercised.