From 331d76ad9fc68ff98b98b4c4b6c41a8d9e126242 Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Wed, 23 Sep 2026 15:28:20 -0400 Subject: [PATCH 1/2] docs: close the three silver criteria the tree could not answer Assessed all 55 OpenSSF Best Practices criteria at the silver level against this tree. Most were already met by work already done. Three were not, and each was a real gap rather than a wording problem. documentation_roadmap (MUST). The project had no public roadmap at all: the planning material is deliberately private, and the README's milestone table says what each milestone means without saying what the project intends to do or not do over the next year. ROADMAP.md says both, including the three things that matter most in the next year and the five things this project will not build. The refusals are the half that is usually missing, and they are decisions rather than omissions. assurance_case (MUST). The criterion requires four things by name: a threat model, clear identification of trust boundaries, an argument that secure design principles were applied, and an argument that common implementation weaknesses were countered. docs/10 had none of them under those names. It now opens with five named adversaries and the capability assumed for each, the trust boundaries restated rather than pointed at, the classical design principles each mapped to the place in this tree where it is realised, and a table of weakness classes with the counter for each. Three rows of that table were checked against the tree and corrected before this commit. The first draft said network egress is gated; it is not. Trinity.NetworkGuard lives in test/support and blocks outbound connections during the suite, which is not a production control, and the SSRF row now says so and calls itself the weakest row in the table. Writing a security document is exactly where the rules of evidence matter most. test_policy_mandated and tests_documented_added (both MUST). The project has behaved this way since slice 000 but had written it down only as practice. docs/03 gains a Tests policy of four numbered rules; CONTRIBUTING.md states in its change-proposal instructions that major new functionality comes with tests in the same change. mix gate exit 0; 575 passed; coverage 80.31%. Signed-off-by: Ayla Croft --- CONTRIBUTING.md | 5 ++ README.md | 3 + ROADMAP.md | 73 ++++++++++++++++++++++++ docs/03-conventions.md | 21 +++++++ docs/10-assurance-case.md | 117 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 219 insertions(+) create mode 100644 ROADMAP.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 769803f..a742ba7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -39,6 +39,11 @@ evidence. The short version: wrong line stays and is corrected below it, saying what it supersedes. - **A failing test comes before the fix.** A test for a claimed property is committed failing first, by name, and the fix commit refers to it. +- **Major new functionality comes with tests for it, in the same change.** This is a policy, not a + preference: a change that adds behaviour and no test for that behaviour is incomplete, and the + question a reviewer will ask is which test fails if the change is reverted. A fixed bug gets a + regression test naming the defect. The full policy is the *Tests* section of + `docs/03-conventions.md`. ## Commit messages diff --git a/README.md b/README.md index ffffc41..04f7d5a 100644 --- a/README.md +++ b/README.md @@ -210,6 +210,9 @@ rather than described by it. M0 to M5a are approved as of 2026-09-22, and 070 of M5b on 2026-09-23. Slice numbers have gaps on purpose (000, 001, 010, 011 and so on) so that a slice can be inserted later without renumbering anything. +**[`ROADMAP.md`](ROADMAP.md)** sets out what the project intends to do and what it intends not to +do over the next year, and why the order is what it is. + ## What is in the repository | Path | Purpose | diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..2f570b0 --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,73 @@ + +# Roadmap + +What this project intends to do, and what it intends not to do, over the next year. Written for +someone deciding whether to depend on Trinity, review it, or contribute to it. + +The unit of work is a **slice**: one increment small enough to review in a sitting, which ends with +a tag and an engineering record of what was built and what was proven. Milestones group slices. +Dates are deliberately absent, because a date this project could not hold would be worth less than +an order it can. What is here is the order, the meaning of each step, and what has to be true +before a step counts as done. + +## Where the project is + +Pre-alpha, and usable from source. Milestones **M0 through M5a** are approved, and the first slice +of M5b with them. There is no released binary yet: packaged binaries are built and smoke-tested on +Linux, macOS and Windows on every change that touches the packaging inputs, but nothing is +published for general use until the release pipeline in M6 signs it. + +## Next, in order + +| Milestone | What it means | Status | +|---|---|---| +| **M5b Reaches** | Messaging gateways, so a conversation can start from a channel rather than the desktop, and subagents | in progress | +| **M6 Ships** | Observability and a cost ledger, the native desktop shell, and a **signed release pipeline** | next | +| **M7 Sandboxed** | Skills that execute, inside an in-VM sandbox rather than on trust | after M6 | +| **M9 Donatable** | Open-source hygiene audited end to end, the supply chain signed, the shared libraries extracted for reuse | continuous, closing after M7 | + +Milestones M0 to M5a are complete and are described in the [Milestones](README.md#milestones) +table in the README, together with what each one had to demonstrate before it was accepted. + +### The three things that matter most in the next year + +1. **A signed release.** Today the project produces build provenance for every packaged artifact, + attested through the build's own identity and verifiable by whoever holds the binary. It does + not yet sign a release with a project-held key, and it publishes no release. Both arrive in M6. + Until then, the honest description of this project is "buildable from source", not "shipped". +2. **An in-VM sandbox for executable skills.** The skills system can already extend itself behind + approval and a scanner, but a skill that executes is a different class of risk from a skill that + is read, and M7 is where that risk is contained by construction rather than by review. +3. **Extraction of the reusable parts.** The receipt chain, the permission gate and the MCP + authorization layer are written to be liftable out of this tree. M9 lifts them, so that the + useful parts of this work do not require adopting the whole of it. + +## What this project will not do + +These are decisions, not omissions, and they are not expected to change within the year. + +- **No hosted multi-tenant service.** Trinity is single-user and local-first. The design does not + preclude a hosted version; this project is not building one. +- **No training or reinforcement-learning pipeline.** Exporting trajectories may come later. + Training on them is somebody else's project. +- **No mobile applications.** +- **No breadth of integrations at launch.** Depth first: a small number of gateways and tools that + work completely, rather than many that work partly. A long list of half-supported integrations is + a liability in a system whose whole argument is about what it will and will not do on your behalf. +- **No feature matched to another agent's feature list for its own sake.** Scope grows from what + the security and assurance argument can support, not from comparison. + +## How the order is decided, and how it changes + +Slices are ordered by dependency, not by preference, and the order is recorded before the work +starts. Anything that changes the architecture, the stack, the data model or the process is +recorded as an architecture decision record in [`docs/adr/`](docs/adr) **before** it is +implemented, with a status of proposed, accepted, or superseded. Decision records are appended to +and never rewritten, so a decision that turned out wrong stays visible next to the one that +replaced it. + +Detailed per-slice plans are working material and are not published; what is published is the +result: a tag, a commit that explains what was built, and the record of what was proven. See +[`GOVERNANCE.md`](GOVERNANCE.md) for who decides, and +[`docs/06-risk-register.md`](docs/06-risk-register.md) for the risks that could change this order, +each with what would have to happen for it to be closed. diff --git a/docs/03-conventions.md b/docs/03-conventions.md index bb812e1..058f566 100644 --- a/docs/03-conventions.md +++ b/docs/03-conventions.md @@ -15,6 +15,27 @@ ## Tests +### The policy + +Stated as a policy rather than left as a habit, because a habit cannot be pointed at in a review. + +1. **Major new functionality is accompanied by tests for it in the automated suite, in the same + change.** Not afterwards, not in a follow-up. A change that adds behaviour and no test for that + behaviour is incomplete, and the reviewer's question is "which test fails if this is reverted?" +2. **A fixed bug gets a regression test naming the defect**, so the same defect cannot return + silently. The test is committed failing first, by name, and the fix commit refers to it. +3. **A claimed property has a test that could fail.** A test that passes whatever the code does + proves nothing; where it is not obvious that a test discriminates, it is demonstrated red once + and the output recorded. +4. **The population comes from the tree.** Any claim about "every X" is tested against a list the + command produces, not against a list someone typed. + +The gate enforces what can be enforced mechanically: the suite runs on every commit and every push, +and the coverage floor in `coverage.tsv` refuses a drop. The rest is enforced in review, and this +section is what review points at. + +### The practice + - Unit tests for pure modules; process tests for GenServers/gen_statem (start under a test supervisor, send messages, assert state via public API, not `:sys.get_state` except in recovery tests). - LiveView tests use `Phoenix.LiveViewTest`; prefer `element/3` with text filters. diff --git a/docs/10-assurance-case.md b/docs/10-assurance-case.md index 4de68b7..f3bfb88 100644 --- a/docs/10-assurance-case.md +++ b/docs/10-assurance-case.md @@ -21,6 +21,123 @@ deployment supplies, each of which is named as an assumption at the end. --- +## The threat model + +Who this system is defended against, what they want, and what they are assumed to be able to do. +Claims C1 to C10 are the argument that these adversaries do not get what they want; naming them +first is what makes those claims checkable rather than decorative. + +**A1. The model, manipulated.** The primary adversary is not a person but a text stream. An agent +reads web pages, files, tool results, messages from chat platforms and responses from external +tool servers, and any of those can carry instructions written by someone who wants the agent to act +on their behalf rather than the user's. This adversary does not need to breach anything: it only +needs the system to treat what it reads as what it was told. **Capability assumed:** full control +of the content of anything the agent reads from outside the machine, including the ability to +imitate the user, the system, or an internal record. + +**A2. A confused deputy inside the loop.** An adversary who cannot get the agent to act directly +may instead get it to ask for something reasonable and then change what that request means: widen a +path after it is approved, alter arguments between approval and execution, or reuse a decision made +for one thing to authorise another. **Capability assumed:** the ability to influence the arguments +of a call between the moment a decision is made and the moment it runs. + +**A3. A caller without authority.** Anything that speaks to Trinity's own interfaces: a client of +the MCP server, a message arriving at a gateway from an unknown account, a process on the machine +that finds an open port. **Capability assumed:** the ability to send well-formed requests, to +present a token it obtained somewhere, and to retry. + +**A4. The supply chain.** A dependency, a build action, or an artifact substituted between build +and use. **Capability assumed:** publishing a malicious version of a package the project depends +on, or moving a tag an action was referenced by. + +**A5. An observer after the fact.** Someone disputing what the system did, or altering the record +of it. Included because a system that acts on a user's behalf is worth little if what it did cannot +be established later. **Capability assumed:** read and write access to the machine's filesystem, +including the records themselves. + +**Explicitly outside the model, and therefore outside every claim below.** A compromised host +operating system or kernel; a user acting against their own interest with full local privileges; a +model provider that is itself hostile; physical access; and the correctness of any external +authorization server a deployment chooses to trust. These are listed again under **Assumptions** +with the reason each is excluded. + +## Trust boundaries + +The boundaries are drawn in `docs/07-security-model.md` and restated here, because an assurance +case that points elsewhere for its own boundaries is asking to be read twice and understood once. + +``` +[The user] trusted +[Persona, user-authored skills, configuration] trusted, but validated +[Model output] UNTRUSTED +[Tool results, web pages, files, MCP servers, + gateway messages, skills from a hub] UNTRUSTED +``` + +Three of those lines do the work. + +**Model output is untrusted.** Not because the provider is assumed hostile, but because the model +reads untrusted things and its output is downstream of them. A system that trusts model output has +no boundary at all, only a longer path to the same place. + +**Everything crossing inward is labelled at the crossing, not later.** Every content part carries +its origin, a source reference, a digest and a taint. Summaries and compactions inherit the maximum +taint of their inputs, so nothing launders its provenance by being summarised. + +**Authority never crosses a boundary with data.** A credential establishes who is calling; it never +decides what may happen. That decision belongs to the permission gate and the selected authority +adapter, on the trusted side, and a session carries no authority that survives a reset. + +The boundary that matters most at runtime is the one effects cross: a compile-time effect catalogue +with a single execution path, described in C1. + +## Secure design principles, and where each one is applied + +The principles are the classical ones (Saltzer and Schroeder, as the OpenSSF criteria use them). +Each row names where in this tree the principle is realised, so that "we applied it" can be +checked rather than believed. + +| Principle | Where it is applied | +|---|---| +| **Economy of mechanism** | One side-effect membrane. Every effect passes a single compile-time catalogue and one execution path, rather than each subsystem deciding for itself. A small mechanism is one a reviewer can hold in their head. | +| **Fail-safe defaults** | A tool with no explicit tier is treated as asking for approval. An approval whose arguments no longer match the ones it was granted for is denied, not repaired. No approved cryptographic algorithm available means the effect is refused, never signed with a weaker one and never written unsigned. | +| **Complete mediation** | The gate is in the execution path, not beside it. An approval is re-derived from a canonical hash of the exact call at the moment of execution rather than checked once and remembered, which is what closes A2. | +| **Least privilege** | Channel trust caps: what a messaging gateway may approve is bounded below what the desktop may, and a capped request is refused in words and receipted rather than dropped. Tokens are audience-bound. Workflow tokens grant read, with writes declared by the one job that needs them. | +| **Separation of privilege** | Identity and authority are separate concerns with separate modules (C2). Holding a valid credential is not sufficient to cause an effect. | +| **Open design** | The whole tree is public, including this document, the risk register with its unfixed entries, and the engineering records of changes that went wrong. Security does not rest on any of it being unread. | +| **Defence in depth** | Untrusted content is framed *and* the gate decides *and* effects pass one membrane *and* what happened is receipted. No single one of these is load-bearing alone. | +| **Psychological acceptability** | A refusal says what was refused and why, in the channel where it was asked. A silent refusal teaches a user that the system is broken, and a user who believes that routes around it. | +| **Work factor** | Argument-bound approvals and audience-bound tokens are chosen over name-based checks specifically because the cheap attacks against the latter are the ones that actually occur. | + +## Common implementation weaknesses, and how each is countered + +The classes below are the ones that apply to what this project is: an Elixir application on the +BEAM with a web interface, a database, an HTTP client and a protocol server. Each names the counter +and, where the counter is incomplete, says so. + +| Weakness | Counter in this tree | +|---|---| +| **Prompt injection** (the injection class that actually applies here) | Untrusted content is framed at the boundary with its origin and taint and is never treated as instruction; model output is itself untrusted; and no instruction in content can reach an effect without passing the gate. This is the weakness this project is mostly *about*. | +| **Memory-safety defects** (CWE-787, CWE-125, CWE-416) | Do not arise in the application tree: Elixir on the BEAM, and Rust in the desktop shell. There is no C or C++ in the project's own code (C9). | +| **SQL injection** (CWE-89) | All database access goes through Ecto's parameterised queries. Sobelow, which includes checks for interpolated SQL, runs on every commit as a gate step (`sobelow --exit`). | +| **Cross-site scripting** (CWE-79) | Phoenix templates escape by default. Rendered Markdown goes through `TrinityWeb.Markdown`, which sets `unsafe: false` so raw HTML in the source is not emitted, and applies the renderer's sanitizer on top of that. The previous renderer was dropped at slice 013 for a retired package with an open XSS advisory, which the dependency audit refuses. Sobelow runs on every commit. | +| **Cross-site request forgery** (CWE-352) | Phoenix's CSRF protection on the endpoint; the protocol server's own transport is authenticated separately and does not use cookie authority. | +| **Path traversal** (CWE-22) | Filesystem tools validate against configured roots, and a manifest path that escapes its root is rejected rather than normalised. | +| **Unsafe deserialisation** (CWE-502) | No `:erlang.binary_to_term` on external input. External data arrives as JSON and is decoded to plain data, never to atoms from untrusted keys. | +| **Hard-coded and leaked secrets** (CWE-798, CWE-532) | A secret scan runs in the gate; tests assert that no token material appears in records or model context; keys live in files separate from configuration and can be replaced without recompilation (C10). | +| **Broken authentication of the protocol server** (CWE-287, CWE-863) | Tokens are validated for issuer, audience, expiry and not-before against an algorithm allow-list, with `none` and symmetric algorithms refused; the scope check precedes the permission gate; a token minted by the non-default personal profile is refused in production. | +| **Weak cryptography** (CWE-327) | No SHA-1, no ECB, no unauthenticated cipher modes; a test sweeps the tree for them. Algorithms are measured inside a FIPS-mode container on a dedicated build leg rather than asserted (C7). | +| **Missing authorization on an approved action** (CWE-862) | Approvals bind a canonical hash of the exact call, re-derived at execution. | +| **Server-side request forgery** (CWE-918) | Every web fetch passes the permission gate, so a person decides each one, and the decision binds the exact arguments rather than the tool's name. **This is the weakest row in the table and it is not counted as solved.** There is no egress allow-list, and no restriction by resolved address, so an approved fetch to a host that resolves to a private or link-local address is not blocked by construction. `Trinity.NetworkGuard` blocks outbound connections during the test suite and is not a production control; describing it as one would be an overstatement, and it was one in the first draft of this table. | +| **Denial of service through unbounded work** (CWE-400) | Iteration, token and wall-clock caps are owned by the code rather than passed in as arguments; reaching one is a recorded outcome rather than a crash. | + +Static analysis for these classes runs on every commit (Sobelow for the Phoenix-specific ones, +Credo in strict mode, the compiler with warnings as errors, plus the dependency and licence +audits). What none of them cover is the first row of this table, which is why the argument for it +is structural rather than a tool. + +--- + ## C1. No effect happens without a decision **Argument.** Every tool call is decided before it runs. The decision is made by one component, From 66738bb7e55ff97e62fa01027fcc872a3db9ac3a Mon Sep 17 00:00:00 2001 From: Ayla Croft Date: Wed, 23 Sep 2026 15:32:44 -0400 Subject: [PATCH 2/2] docs: measure build reproducibility rather than assert it The silver criterion build_repeatable asks whether repeating the build from the same sources gives the same bit-for-bit result. The honest answer is "partly", and a criterion answered partly deserves a measurement rather than an adjective. Two forced prod compiles of the same tree, sha256 over every .beam, and each differing file compared chunk by chunk: 219 of 243 modules are byte-identical, 24 differ, and the differences are entirely Dbgi (18) and ExCk (3) plus three in container layout alone. The Code chunk, which is the executable bytecode, is identical in all 243. So the compiled behaviour is reproducible and the compiler's metadata is not. That is a known BEAM property rather than something this project introduced, and this project already had to account for it: Trinity.CorePolicy hashes stripped beams precisely because Dbgi varies between builds and between virtual machines. The packaged binary is explicitly excluded. Burrito wraps the release with a launcher and embeds build-time information, and no claim is made about two package runs producing the same bytes; that claim belongs with the signed release pipeline, where a third party might want to check it. Signed-off-by: Ayla Croft --- docs/packaging.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/docs/packaging.md b/docs/packaging.md index 099b94c..09960ea 100644 --- a/docs/packaging.md +++ b/docs/packaging.md @@ -212,3 +212,37 @@ on the gate's `fips` job: OTP built from source with `--enable-fips` against a U whole gate run with the mode on, and the algorithms the mode removes listed in the tree. `docs/fips-leg.md` has the image, how the mode is entered, the diff, and the findings (two of them about reaching hex.pm and GitHub from inside the mode). + +## Build reproducibility, measured + +Measured 2026-09-23, on this machine, for the OpenSSF Best Practices criterion `build_repeatable`, +which asks whether repeating the build from the same sources gives the same bit-for-bit result. +Recorded here because the answer is "partly", and a criterion answered "partly" is worth a +measurement rather than an adjective. + +**Method.** `MIX_ENV=prod mix compile --force` twice from the same tree, with the sha256 of every +`.beam` in `_build/prod/lib/trinity/ebin` compared between the two runs, and each differing file +then compared chunk by chunk with `:beam_lib.all_chunks/1`. + +**Result.** + +| | | +|---|---| +| Modules compiled | 243 | +| Byte-identical across two builds | 219 | +| Differing | 24 | +| Chunks responsible | `Dbgi` (debug info) in 18, `ExCk` (Elixir compile metadata) in 3; three more differ in container layout alone | +| **`Code` chunk, the executable bytecode** | **identical in all 243** | + +**What that means.** The compiled behaviour of this project is reproducible: recompiling the same +sources produces the same instructions every time. What is not reproducible is the metadata the +compiler attaches beside them, which is a known property of the BEAM toolchain rather than +something this project introduced, and which this project already had to account for elsewhere: +`Trinity.CorePolicy` hashes **stripped** beams precisely because the `Dbgi` chunk varies between +builds and between virtual machines, so a policy hash taken over unstripped beams would change +without the policy changing. + +**What is not measured here.** The packaged binary. A Burrito artifact wraps the release with a +launcher and embeds build-time information, and no claim is made that two package runs produce the +same bytes. That claim belongs with the signed release pipeline, where reproducibility becomes +something a third party might want to check rather than an internal property.