Skip to content

Close the three silver criteria the tree could not answer - #75

Merged
HackTuah merged 2 commits into
mainfrom
docs/silver-readiness
Sep 23, 2026
Merged

HackTuah merged 2 commits into
mainfrom
docs/silver-readiness

Conversation

@HackTuah

Copy link
Copy Markdown
Member

I assessed all 55 OpenSSF Best Practices criteria at the silver level against this tree, reading the criteria from the badge project's own criteria.yml and en.yml rather than from memory. Most are already met by work already done. Three were genuinely unmet, and each was a real gap rather than a wording problem.

documentation_roadmap — MUST

The project had no public roadmap at all. The planning material is deliberately private, and the README's milestone table says what each milestone means without saying what the project intends to do, or not do, over the next year — which is what the criterion asks for.

ROADMAP.md says both: where the project is, the next four milestones in order, the three things that matter most in the next year, and five things this project will not build. The refusals are the half usually missing from a roadmap, and here they are decisions with reasons rather than omissions.

assurance_case — MUST

The criterion requires four things by name: a description of the threat model, clear identification of trust boundaries, an argument that secure design principles have been applied, and an argument that common implementation security weaknesses have been countered. docs/10-assurance-case.md had a strong claim structure (C1–C10 with evidence and limits) and none of those four under those names.

It now opens with:

  • The threat model — five named adversaries, each with the capability assumed for it. The primary one is not a person but a text stream: an agent reads pages, files, tool results and messages, any of which can carry instructions written by someone who wants the agent acting on their behalf. Also what is explicitly outside the model, and why.
  • Trust boundaries — restated here rather than pointed at, because an assurance case that sends you elsewhere for its own boundaries is asking to be read twice and understood once.
  • Secure design principles — the classical set, each mapped to the specific place in this tree where it is realised, so "we applied it" can be checked.
  • Common implementation weaknesses — thirteen classes with the counter for each, and the limit where the counter is incomplete.

One row of that table was wrong, and was corrected before this commit

The first draft said network egress is gated. It is not. Trinity.NetworkGuard lives in test/support/ and blocks outbound connections during the test suite; it is not a production control. The SSRF row now says exactly that, and calls itself the weakest row in the table. Writing a security document is the place where the rules of evidence matter most, not least.

Two other rows were tightened against what the tree actually does (the Markdown renderer's unsafe: false plus sanitizer, and the exact Sobelow invocation).

test_policy_mandated and tests_documented_added — both MUST

The project has behaved this way since slice 000 but had written it down only as practice, and a habit cannot be pointed at in a review. docs/03-conventions.md gains a Tests policy of four numbered rules; CONTRIBUTING.md states in its change-proposal instructions that major new functionality comes with tests in the same change.

Evidence

mix gate exit 0; 575 passed, 18 excluded; coverage 80.31%.

Assessed all 55 OpenSSF Best Practices criteria at the silver level against
this tree. Most were already met by work already done. Three were not, and each
was a real gap rather than a wording problem.

documentation_roadmap (MUST). The project had no public roadmap at all: the
planning material is deliberately private, and the README's milestone table says
what each milestone means without saying what the project intends to do or not
do over the next year. ROADMAP.md says both, including the three things that
matter most in the next year and the five things this project will not build.
The refusals are the half that is usually missing, and they are decisions rather
than omissions.

assurance_case (MUST). The criterion requires four things by name: a threat
model, clear identification of trust boundaries, an argument that secure design
principles were applied, and an argument that common implementation weaknesses
were countered. docs/10 had none of them under those names. It now opens with
five named adversaries and the capability assumed for each, the trust boundaries
restated rather than pointed at, the classical design principles each mapped to
the place in this tree where it is realised, and a table of weakness classes with
the counter for each.

Three rows of that table were checked against the tree and corrected before this
commit. The first draft said network egress is gated; it is not.
Trinity.NetworkGuard lives in test/support and blocks outbound connections during
the suite, which is not a production control, and the SSRF row now says so and
calls itself the weakest row in the table. Writing a security document is exactly
where the rules of evidence matter most.

test_policy_mandated and tests_documented_added (both MUST). The project has
behaved this way since slice 000 but had written it down only as practice.
docs/03 gains a Tests policy of four numbered rules; CONTRIBUTING.md states in
its change-proposal instructions that major new functionality comes with tests
in the same change.

mix gate exit 0; 575 passed; coverage 80.31%.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
The silver criterion build_repeatable asks whether repeating the build from the
same sources gives the same bit-for-bit result. The honest answer is "partly",
and a criterion answered partly deserves a measurement rather than an adjective.

Two forced prod compiles of the same tree, sha256 over every .beam, and each
differing file compared chunk by chunk: 219 of 243 modules are byte-identical,
24 differ, and the differences are entirely Dbgi (18) and ExCk (3) plus three in
container layout alone. The Code chunk, which is the executable bytecode, is
identical in all 243.

So the compiled behaviour is reproducible and the compiler's metadata is not.
That is a known BEAM property rather than something this project introduced, and
this project already had to account for it: Trinity.CorePolicy hashes stripped
beams precisely because Dbgi varies between builds and between virtual machines.

The packaged binary is explicitly excluded. Burrito wraps the release with a
launcher and embeds build-time information, and no claim is made about two
package runs producing the same bytes; that claim belongs with the signed
release pipeline, where a third party might want to check it.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
@HackTuah
HackTuah merged commit 7cb78a3 into main Sep 23, 2026
8 checks passed
@HackTuah
HackTuah deleted the docs/silver-readiness branch September 23, 2026 19:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant