Skip to content

Say where the source comes from, so anyone can reproduce this - #41

Merged
kurtseifried merged 1 commit into
mainfrom
docs/source-provenance
Aug 12, 2026
Merged

Say where the source comes from, so anyone can reproduce this#41
kurtseifried merged 1 commit into
mainfrom
docs/source-provenance

Conversation

@kurtseifried

Copy link
Copy Markdown
Contributor

The repository claimed reproducibility and gave nobody the means to exercise it.

"Everything is generated from published source releases by the scripts in tools/" and "regenerate AICM controls from a published release" both name the commands and never name the release, or where to get it. The input was public the whole time; the pointer to it was not written down anywhere.

That single omission is what made "only one person can regenerate this" true — not access, documentation.

It also left an assertion in a document that exists to be trusted with no way for a reader to check it. That's the same defect class as the idempotency claim (now exercised, #38) and the YAML view's implied durability (now stated, #31): a true claim that cannot be verified is still a defect, and a worse one than a false claim, because the reader has no way to discover the gap.

The chain, which is entirely public

CSA artifact page  ->  AICM workbook (.xlsx)
    ->  parse_aicm.py / parse_caiq.py    [dataset-public-laws-regulations-standards]
    ->  aicm-1.1.0.json  (normalised, ~5.5 MB)
    ->  tools/generate_aicm_*.py         [this repository]
    ->  objects/

README.md now carries that, names the dataset repository holding the normalised extraction alongside the upstream workbook, the extraction scripts, a changelog and metadata recording source URL and licence — and draws the boundary: that repository owns extraction, this one owns the STIX rendering. A misread workbook is an issue there; a badly built object is an issue here.

And why the version segment is not cosmetic

Two different AICM datasets are published upstream under the label "v1.1", distinguishable only by the workbook filename and a stamp in cell A1 of every worksheet. This catalog is built from the extraction stamping 1.1.0, and every object says so in its SecID:

secid:control/cloudsecurityalliance.org/aicm@1.1.0#MDS-01

A bare "AICM v1.1" does not identify an extraction. aicm@1.1.0 does — which is what makes a mapping claim here resolvable to the exact data it came from.

CLAUDE.md and AGENTS.md carried the same unpointed claim and now carry the pointer.

No object, schema or tool changes. Doc-discipline checks, coverage.py --check and the YAML round-trip all pass.

The repository claimed reproducibility and gave nobody the means to exercise it.
"Everything is generated from published source releases by the scripts in tools/"
and "regenerate AICM controls from a published release" both name the commands and
never name the release or where to get it. The input was public the whole time; the
pointer to it was not written down anywhere.

That single omission is what made "only one person can regenerate this" true. Not
access -- documentation. It also left an assertion in a document that exists to be
trusted with no way for a reader to check it, which is the same defect class as the
idempotency claim (now exercised) and the YAML view's implied durability (now
stated): a true claim that cannot be verified is still a defect, and a worse one
than a false claim, because a reader has no way to discover the gap.

The whole chain is public:

  CSA artifact page -> AICM workbook (.xlsx)
    -> parse_aicm.py / parse_caiq.py   [dataset-public-laws-regulations-standards]
    -> aicm-1.1.0.json
    -> tools/generate_aicm_*.py        [this repository]
    -> objects/

README now carries that, names the dataset repository holding the normalised
extraction along with the upstream workbook, the extraction scripts, a changelog and
metadata recording source URL and licence, and draws the boundary: that repository
owns extraction, this one owns the STIX rendering, so a misread workbook is an issue
there and a badly built object is an issue here.

Also records why the version segment is not cosmetic. Two different AICM datasets are
published upstream under the label "v1.1", distinguishable only by the workbook
filename and a stamp in cell A1 of every worksheet. This catalog is built from the
extraction stamping 1.1.0, and every object says so in its SecID --
secid:control/cloudsecurityalliance.org/aicm@1.1.0#MDS-01. A bare "AICM v1.1" does not
identify an extraction; aicm@1.1.0 does, which is what makes a mapping claim here
resolvable to the exact data it came from.

CLAUDE.md and AGENTS.md carried the same unpointed claim and now carry the pointer.

No object, schema or tool changes.
@kurtseifried
kurtseifried merged commit c8d29b5 into main Aug 12, 2026
6 checks passed
@kurtseifried
kurtseifried deleted the docs/source-provenance branch August 12, 2026 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant