Repository navigation
Close the three silver criteria the tree could not answer - #75
Merged
Merged
Conversation
Assessed all 55 OpenSSF Best Practices criteria at the silver level against this tree. Most were already met by work already done. Three were not, and each was a real gap rather than a wording problem. documentation_roadmap (MUST). The project had no public roadmap at all: the planning material is deliberately private, and the README's milestone table says what each milestone means without saying what the project intends to do or not do over the next year. ROADMAP.md says both, including the three things that matter most in the next year and the five things this project will not build. The refusals are the half that is usually missing, and they are decisions rather than omissions. assurance_case (MUST). The criterion requires four things by name: a threat model, clear identification of trust boundaries, an argument that secure design principles were applied, and an argument that common implementation weaknesses were countered. docs/10 had none of them under those names. It now opens with five named adversaries and the capability assumed for each, the trust boundaries restated rather than pointed at, the classical design principles each mapped to the place in this tree where it is realised, and a table of weakness classes with the counter for each. Three rows of that table were checked against the tree and corrected before this commit. The first draft said network egress is gated; it is not. Trinity.NetworkGuard lives in test/support and blocks outbound connections during the suite, which is not a production control, and the SSRF row now says so and calls itself the weakest row in the table. Writing a security document is exactly where the rules of evidence matter most. test_policy_mandated and tests_documented_added (both MUST). The project has behaved this way since slice 000 but had written it down only as practice. docs/03 gains a Tests policy of four numbered rules; CONTRIBUTING.md states in its change-proposal instructions that major new functionality comes with tests in the same change. mix gate exit 0; 575 passed; coverage 80.31%. Signed-off-by: Ayla Croft <aylacroft@proton.me>
The silver criterion build_repeatable asks whether repeating the build from the same sources gives the same bit-for-bit result. The honest answer is "partly", and a criterion answered partly deserves a measurement rather than an adjective. Two forced prod compiles of the same tree, sha256 over every .beam, and each differing file compared chunk by chunk: 219 of 243 modules are byte-identical, 24 differ, and the differences are entirely Dbgi (18) and ExCk (3) plus three in container layout alone. The Code chunk, which is the executable bytecode, is identical in all 243. So the compiled behaviour is reproducible and the compiler's metadata is not. That is a known BEAM property rather than something this project introduced, and this project already had to account for it: Trinity.CorePolicy hashes stripped beams precisely because Dbgi varies between builds and between virtual machines. The packaged binary is explicitly excluded. Burrito wraps the release with a launcher and embeds build-time information, and no claim is made about two package runs producing the same bytes; that claim belongs with the signed release pipeline, where a third party might want to check it. Signed-off-by: Ayla Croft <aylacroft@proton.me>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I assessed all 55 OpenSSF Best Practices criteria at the silver level against this tree, reading the criteria from the badge project's own
criteria.ymlanden.ymlrather than from memory. Most are already met by work already done. Three were genuinely unmet, and each was a real gap rather than a wording problem.documentation_roadmap— MUSTThe project had no public roadmap at all. The planning material is deliberately private, and the README's milestone table says what each milestone means without saying what the project intends to do, or not do, over the next year — which is what the criterion asks for.
ROADMAP.mdsays both: where the project is, the next four milestones in order, the three things that matter most in the next year, and five things this project will not build. The refusals are the half usually missing from a roadmap, and here they are decisions with reasons rather than omissions.assurance_case— MUSTThe criterion requires four things by name: a description of the threat model, clear identification of trust boundaries, an argument that secure design principles have been applied, and an argument that common implementation security weaknesses have been countered.
docs/10-assurance-case.mdhad a strong claim structure (C1–C10 with evidence and limits) and none of those four under those names.It now opens with:
One row of that table was wrong, and was corrected before this commit
The first draft said network egress is gated. It is not.
Trinity.NetworkGuardlives intest/support/and blocks outbound connections during the test suite; it is not a production control. The SSRF row now says exactly that, and calls itself the weakest row in the table. Writing a security document is the place where the rules of evidence matter most, not least.Two other rows were tightened against what the tree actually does (the Markdown renderer's
unsafe: falseplus sanitizer, and the exact Sobelow invocation).test_policy_mandatedandtests_documented_added— both MUSTThe project has behaved this way since slice 000 but had written it down only as practice, and a habit cannot be pointed at in a review.
docs/03-conventions.mdgains a Tests policy of four numbered rules;CONTRIBUTING.mdstates in its change-proposal instructions that major new functionality comes with tests in the same change.Evidence
mix gateexit 0; 575 passed, 18 excluded; coverage 80.31%.