Skip to content

feat: a regulated profile that refuses to boot on a configuration it cannot support - #110

Merged
HackTuah merged 7 commits into
mainfrom
feat/regulated-boot
Sep 30, 2026
Merged

HackTuah merged 7 commits into
mainfrom
feat/regulated-boot

Conversation

@HackTuah

@HackTuah HackTuah commented Sep 27, 2026 •

Copy link
Copy Markdown
Member

Five refusals, gated on TRINITY_PROFILE=regulated (or config :trinity, :profile, :regulated). Under the default profile nothing here does anything: every check takes the profile as its first argument and answers :ok for :default without looking at the value.

Refusal
AC1 the MCP authorization profile must be :production
AC2 the personal issuer must not be the one in force
AC3 every model must name an endpoint on TRINITY_REGULATED_LLM_ENDPOINTS
AC4 receipts that cannot append at boot stop the node
AC5 the authority in force must not be Trinity.Authority.Local

AC5 supplies no adapter and invents none. If TRINITY_AUTHORITY is unset or resolves to Local, the boot is refused and the operator has to supply one. The refusal is the point.

AC3 refuses a model with no base_url. Such a model goes to its provider's default endpoint, which the configuration does not state, and an endpoint that is not stated cannot be shown to be on the list.

The two files the rules protected, and they are not in the diff

$ git diff --stat HEAD~1
 lib/trinity.ex                            |   8 ++
 lib/trinity/application.ex                |  74 ++++++++++-
 lib/trinity/profile.ex                    | 196 ++++++++++++++++++++++++++++++
 test/trinity/profile_signer_down_test.exs |  59 +++++++++
 test/trinity/profile_test.exs             | 174 ++++++++++++++++++++++++++
 5 files changed, 510 insertions(+), 1 deletion(-)
  • lib/trinity/receipts/supervisor.ex keeps its fail-open shape for every profile that is not :regulated. A missing signer still logs, sets the alarm, and lets the tree start. AC4 is a separate call that :default never makes.
  • lib/trinity/mcp/boot.ex keeps its rescue. AC2 is decided before any child starts, which is before MCP.Boot runs, rather than by narrowing a rescue so a warning could become a refusal.

The commit-msg hook confirms it independently: it reports both paths as named in the message but not touched.

Rule 5's pair, raw

$ mix test test/trinity/profile_signer_down_test.exs --trace
Trinity.ProfileSignerDownTest [test/trinity/profile_signer_down_test.exs]
  * test the DEFAULT profile still boots with the signer down (0.3ms) [L#41]
  * test the REGULATED profile does not boot with the signer down, and says why (0.2ms) [L#49]
  * test regulated boots when the signer is available (0.1ms) [L#56]

Finished in 0.01 seconds (0.00s async, 0.01s sync)

Result: 3 passed

Both use a real failure: the setup writes a file where a directory should be and calls KeyCustody.boot!/1 beneath it, getting {:error, {:key_file, :enotdir}}, and asserts it really failed before either test runs.

The mistake that rule caught

The first version of those tests lived in an async file and called KeyCustody.boot!/1 directly. That function writes to :persistent_term on both paths, including {:unavailable, reason} on failure, so inducing a failure poisoned the signer for the whole run:

Result: 891/939 passed (20/20 properties, 871/919 tests), 18 excluded

1) test AC2: the signed payload's keys are slice 024's ten plus exactly what slice 026 added
   ** (MatchError) no match of right hand side value:
       {:error, {:signer_unavailable, {:key_file, :enotdir}}}

48 unrelated failures from one test. Moved to its own async: false file, which ExUnit runs only after every async module finishes, restoring the selection in on_exit.

Design

Trinity.Profile is pure. Nothing in it starts, stops or touches a process, and it reads the environment in two named functions only. The caller reads the world and hands the values in, so every refusal is unit testable and the Trinity boundary grows no dependency on Authority or Receipts to ask a question about a value.

One export added: Trinity already exported Receipts.KeyCustody but not Authority.Selection, and the application needs the selection rule before the selection process exists. Adding both boundaries to the application's deps was my first attempt and the compiler refused it, correctly.

Housekeeping

No public ROADMAP.md row: plan_check rule 4 iterates ROADMAP rows to slices and never the reverse, so a slice with no row is never visited. Branch is not named slice/*, because rule 11 requires slice/NNN-. No numeric id: the private ROADMAP reserved none.

mix gate ; echo EXIT=$?
prod_check: PASS
3781 mods/funs, found no issues.
trinity.names: OK over 672 tracked files
Result: 938 passed (20 properties, 918 tests), 18 excluded
trinity.coverage: 042 80.31% vs 005 80.31%: OK
EXIT=0

bash process/plan_check.sh | tail -1
plan_check: PASS

Three consecutive full runs, all 938, so the new synchronous file does not flake by ordering.

AC1 to AC5 at a boot, not at a function call

The first version of this branch proved the decisions in Trinity.Profile and called that AC1 to AC5. That was the easy half. AC1 to AC5 are about a boot, and start/2 cannot be re-run in a VM where the application is already up, so test/trinity/regulated_boot_node_test.exs spawns a separate OS process per case and asserts on what that process did.

case environment expected
1 :default, key file is a directory starts (BOOT_OK, status 0)
2 :regulated, production mcp_auth, non-Local authority, no allow-list refuses, regulated_llm_endpoints_unset
3 :regulated, production mcp_auth, allow-list set, TRINITY_AUTHORITY unset refuses, regulated_refuses_local_authority
4 :regulated, every condition met starts (BOOT_OK, status 0)

Case 1 is rule 1 written as a test: lib/trinity/receipts/supervisor.ex stays fail-open for every profile that is not :regulated, and if case 1 ever goes red that shape has been changed. Case 4 exists so that 2 and 3 cannot be satisfied by a profile that refuses everything.

Isolation: XDG_DATA_HOME per case, so each child has its own data directory, lock and database files. MIX_ENV=dev rather than test, deliberately: the test environment puts both repos on the Ecto sandbox, and a child with no sandbox owner blocks on checkout instead of booting. mix run --no-start, so the script sets application environment and defines the non-Local authority fixture before the code under test runs. That fixture is defined inside the child script rather than added to lib/, because no adapter other than Local is supplied by this tree. Nothing writes :persistent_term in the suite VM, and there is no collect/3.

mix test test/trinity/regulated_boot_node_test.exs --trace
  * test 2. ... does not start (886.2ms)
  * test 4. ... production mcp_auth, a non-Local authority, an allowed endpoint and a signer (1200.2ms)
  * test 3. ... everything else correct, but TRINITY_AUTHORITY unset, does not start (904.6ms)
  * test 1. ... a broken key store does not stop a default node (1196.5ms)
Result: 4 passed
EXIT=0

Run three times in a row, 4 passed each time.

The cases were checked against planted violations

A passing test proves nothing until it has been seen to fail for its own reason.

plant effect
check_receipts(:default, {:error, _}) returns an error no case changed. Correct: under :default the application never calls it, which is rule 2 holding
the receipts check runs for every profile, not only :regulated case 1 red, reporting that the default profile refused to boot with the signer down
check_llm_endpoints(:regulated, nil, _) returns :ok case 2 red, the node started with no endpoint allow-list
check_authority(:regulated, Local) returns :ok case 3 red, the node started on the local authority

git diff --stat -- lib/ is empty after the plants were removed, so the reviewed code is the code that was tested.

mix gate ; echo EXIT=$?
Result: 942 passed (20 properties, 922 tests), 18 excluded
trinity.coverage: 042 80.31% vs 005 80.31%: OK
EXIT=0

One earlier mix gate run reported 941/942 with a single failure that was not captured before the output scrolled; the two full runs after it, and three runs of the new file alone, are green. The failure is unidentified rather than explained, and it is recorded here as unidentified.

lib/trinity/receipts/supervisor.ex and lib/trinity/mcp/boot.ex are untouched on this branch: git diff origin/main...HEAD -- lib/trinity/receipts/supervisor.ex lib/trinity/mcp/boot.ex is empty.

Cases 5 and 6

case environment expected
5 :regulated, TRINITY_MCP_AUTH_PROFILE = personal, then local, otherwise case 4 refuses, regulated_requires_production_mcp_auth and the profile refused
6 case 4's configuration with case 1's broken key store refuses, regulated_boot_refused / regulated_requires_receipts / key_file

Case 5 sets the variable and lets config/runtime.exs map it, rather than calling Application.put_env/3: that is the path an operator uses, and the one that rejects a value outside the three.

Case 6 is the first case to reach the error branch of the receipts check at an actual boot. Cases 1 to 4 never did: case 1 is :default so the check is not called, cases 2 and 3 fail earlier at configuration, and case 4 succeeds. Whether the reason survived Supervisor.stop/2 called from inside start/2 was therefore untested. It does:

BOOT_REFUSED {:trinity, {{:regulated_boot_refused, {:regulated_requires_receipts, {:key_file, :eisdir}}}, {Trinity.Application, :start, [:normal, []]}}}
child EXIT=3

Case 1 still passes, so the same broken key store does not stop a :default node. That pair is the whole of AC4.

A defect in the earlier cases, found while writing case 5

The report tail inspected the reason with limit: 3. It printed:

BOOT_REFUSED {:trinity, {{...}, ...}}

The reason was hidden completely. Cases 2 and 3 asserted that the reason appeared somewhere in the captured output, and what they matched was the crash report OTP writes to stderr, not the line the node printed. The assertion was true; it was not testing what it claimed to test. Now inspected with limit: :infinity, and cases 5 and 6 assert against the BOOT_REFUSED line alone through refusal_line/1.

Planted breaks, each reverted

plant effect
skip the receipts gate in verify_regulated_receipts case 6 red, case 1 green: exactly the pair the profile exists to create
check_mcp_auth(:regulated, _) always returns :ok case 5 red. It also showed :personal is still caught by AC2's check_embedded_as/2, with the different reason regulated_refuses_personal_issuer. Case 5 catches it only because it asserts the specific reason rather than that some refusal happened. The local iteration did not run, so this plant does not isolate that half

git diff --stat -- lib/ empty after each revert.

mix test test/trinity/regulated_boot_node_test.exs --trace
Result: 6 passed
EXIT=0

mix gate
prod_check: PASS
3788 mods/funs, found no issues.
trinity.names: OK over 673 tracked files
Result: 944 passed (20 properties, 924 tests), 18 excluded
trinity.coverage: 042 80.31% vs 005 80.31%: OK
GATE_EXIT=0

Only test/trinity/regulated_boot_node_test.exs changed for these two cases. lib/ is untouched by this commit.

Every refusing case now asserts on the printed line

Cases 2 and 3 have been pointed at refusal_line/1, as 5 and 6 already were. No unscoped out =~ match is left on any refusal: cases 1 and 4 match BOOT_OK, and 2, 3, 5 and 6 match the reason on the BOOT_REFUSED line only.

Proved by planting the original limit: 3 back into the report tail:

with limit: 3
cases 2, 3, 5, 6 red, e.g. BOOT_REFUSED {:trinity, {:bad_return, ...}}
cases 1, 4 green (they assert BOOT_OK)

And in that same configuration, run by hand, the reason is still in the captured output:

--- the line the node printed (limit: 3) ---
BOOT_REFUSED {:trinity, {:bad_return, ...}}
--- and where the reason actually is ---
9:  ** (RuntimeError) TRINITY_PROFILE=regulated refuses to boot: {:regulated_llm_endpoints_unset, "TRINITY_REGULATED_LLM_ENDPOINTS"}

out =~ "regulated_llm_endpoints_unset" passes there. refusal_line(out) =~ "regulated_llm_endpoints_unset" fails. That is the whole difference, and it is now the one being asserted.

mix test test/trinity/regulated_boot_node_test.exs --trace
Result: 6 passed
EXIT=0

mix test
Result: 944 passed (20 properties, 924 tests), 18 excluded
TEST_EXIT=0

Only test/trinity/regulated_boot_node_test.exs changed: 13 insertions, 6 deletions.

mix gate is red on this branch, and on main, for an unrelated reason

mix deps.audit now reports three advisories against mint 1.10.1, published upstream since the last green run:

mint 1.10.1 - EEF-CVE-2026-91043 (HIGH)    HPACK-indexed cookie fields bypass max_header_list_size and exhaust client memory
mint 1.10.1 - EEF-CVE-2026-94194 (MEDIUM)  chunked framing applied when chunked is not the final transfer coding
mint 1.10.1 - EEF-CVE-2026-92103 (MEDIUM)  HTTP/2 frames buffered to 16 MiB before max_frame_size is enforced
Found packages with security advisories
** (exit) 1

mint is transitive, through finch. mix.exs and mix.lock are byte-identical to main on this branch, so this is a main condition and not something this PR introduced. mix hex.outdated mint says 1.11.0 is available and satisfies finch's ~> 1.8, so a lock bump looks sufficient, but that is a dependency change and this task was scoped to one test file. Not done, and flagged for your call.

prod_check, credo and the suite all pass; the gate exits at the audit step before reaching them in a single run, which is why the numbers above come from mix test on its own.

Not merged.

…cannot support

Five refusals, gated on TRINITY_PROFILE=regulated, or :trinity, :profile set to
:regulated. Under the default profile nothing here does anything: every check
takes the profile as its first argument and answers :ok for :default without
looking at the value.

  AC1  the MCP authorization profile must be :production
  AC2  the personal issuer must not be the one in force
  AC3  every model must name an endpoint on TRINITY_REGULATED_LLM_ENDPOINTS
  AC4  receipts that cannot append at boot stop the node
  AC5  the authority in force must not be Trinity.Authority.Local

AC5 supplies no adapter and invents none. If TRINITY_AUTHORITY is unset or
resolves to Local, the boot is refused and the operator has to supply one. The
refusal is the point.

AC3 refuses a model with no base_url rather than letting it through. Such a
model goes to its provider's default endpoint, which the configuration does not
state, and an endpoint that is not stated cannot be shown to be on the list.

Trinity.Profile is pure. Nothing in it starts, stops or touches a process, and
it reads the environment in two named functions only. The caller reads the world
and hands the values in, so every refusal is unit testable and the Trinity
boundary grows no dependency on Authority or Receipts to ask a question about a
value.

Two things deliberately not changed, because the owner's rules forbade it and
the rules were right:

- lib/trinity/receipts/supervisor.ex keeps its fail-open shape for every profile
  that is not :regulated. A missing signer still logs, sets the alarm, and lets
  the tree start, because every effect is denied until a signer returns. AC4 is
  a separate call that :default never makes.
- lib/trinity/mcp/boot.ex keeps its rescue. AC2 is decided before any child
  starts, which is before MCP.Boot runs, rather than by narrowing a rescue so a
  warning could become a refusal.

Neither file appears in this diff.

One export added: Trinity exported Receipts.KeyCustody already but not
Authority.Selection, and the application needs the selection RULE before the
selection PROCESS exists. Adding both boundaries to the application's deps was
the first attempt and the compiler refused it, correctly.

The mistake worth recording: the first version of AC4's tests called
KeyCustody.boot!/1 from an async file. That function writes its answer to
persistent_term on both paths including the failure, so inducing a failure
poisoned the signer for the whole run and 48 unrelated tests failed with
{:signer_unavailable, {:key_file, :enotdir}}. Moved to its own async: false
file, which ExUnit runs only after every async module has finished, restoring
the selection in on_exit.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
The profile suite proves the decisions Trinity.Profile makes. It does not prove a
boot, and start/2 cannot be re-run in a VM where the application is already up,
so calling that suite AC1 to AC5 proves the easy half.

Four cases, each a separate OS process with its own environment:

  1. default + a broken key store -> starts. This is rule 1 as a test: the
     receipts supervisor stays fail-open for every profile that is not
     :regulated, and if this case ever goes red that shape has been changed.
  2. regulated + production mcp_auth + a non-Local authority + no endpoint
     allow-list -> does not start, naming regulated_llm_endpoints_unset.
  3. the same, with the authority left to resolve to Local -> does not start,
     naming regulated_refuses_local_authority.
  4. every condition met -> starts, then stops. Without this case, 2 and 3 pass
     for a profile that refuses everything and prove nothing.

Isolation: XDG_DATA_HOME per case, so each child gets its own data directory,
lock and database files. MIX_ENV=dev rather than test, because the test
environment puts both repos on the Ecto sandbox and a child with no owner blocks
on checkout instead of booting. mix run --no-start, so the script can set
application environment and define the non-Local authority fixture before the
code under test runs; that fixture stays in the child rather than going into
lib/, because no adapter is supplied by this tree.

Nothing here writes :persistent_term in this VM, and there is no collect/3.

Verified by planting three violations and watching the right case go red:
returning :ok from check_receipts/2 under :default changed nothing, which is
rule 2 holding; running the receipts check for every profile turned case 1 red;
short-circuiting the endpoint check turned case 2 red; short-circuiting the
authority check turned case 3 red. lib/ is byte-identical to before the plants.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
Case 5: TRINITY_PROFILE=regulated with TRINITY_MCP_AUTH_PROFILE set to personal
and then to local, everything else as case 4. Neither starts, and the printed
reason names regulated_requires_production_mcp_auth and the profile it refused.
The variable goes through config/runtime.exs rather than Application.put_env,
because that is the path an operator uses and the one that rejects a value
outside the three.

Case 6: case 4's configuration with case 1's broken key store. Does not start,
and the printed reason is
{:regulated_boot_refused, {:regulated_requires_receipts, {:key_file, :eisdir}}}.
Case 1 keeps passing, so the same key store still does not stop a default node,
which is the whole difference AC4 is about. This is also the first case to reach
the error branch of the receipts check at an actual boot: cases 1 to 4 never did,
so whether the reason survived Supervisor.stop/2 inside start/2 was untested
until now. It does.

Found while writing case 5: the report tail inspected the reason with limit: 3,
which printed BOOT_REFUSED {:trinity, {{...}, ...}} and hid it completely. The
earlier cases asserted the reason appeared somewhere in the captured output, and
matched the crash report OTP writes to stderr rather than the line the node
printed. Now inspected with limit: :infinity, and cases 5 and 6 assert against
the BOOT_REFUSED line alone via refusal_line/1.

Planted breaks, each reverted, lib/ left byte-identical:

  skipping the receipts gate entirely (verify_regulated_receipts) turned case 6
  red and left case 1 green, which is the pair the profile exists to create.

  making AC1's check_mcp_auth/2 always pass turned case 5 red, and showed that
  :personal is still caught by AC2's check_embedded_as/2 with a different reason
  (regulated_refuses_personal_issuer). Case 5 catches that only because it
  asserts the specific reason rather than that a refusal happened.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
Cases 2 and 3 asserted the reason appeared somewhere in the captured output.
With stderr_to_stdout: true that output also carries OTP's crash report, which
repeats the same words, so those assertions passed whether or not the node ever
printed a reason. They now go through refusal_line/1, as cases 5 and 6 already
did, and there is no unscoped match left on any refusal.

Proved by planting the original inspect limit back:

  with limit: 3 the node prints BOOT_REFUSED {:trinity, {:bad_return, ...}} and
  all four refusing cases go red, while cases 1 and 4 (which assert BOOT_OK)
  stay green.

  in the same configuration, run by hand, the reason is still present in the
  captured output, on the crash report line:
    ** (RuntimeError) TRINITY_PROFILE=regulated refuses to boot:
       {:regulated_llm_endpoints_unset, "TRINITY_REGULATED_LLM_ENDPOINTS"}
  so the old assertion would have passed exactly where the new one fails. That
  is the difference the change is for.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
The FIPS leg went red on all six cases at 05f444d, and none of them was red
because a refusal failed. The child runs `mix run` under MIX_ENV=dev; the leg has
no prebuilt dev tree, so the child compiled the dev dependency tree from scratch
and reached the network for the `tokenizers` precompiled NIF. OTP's TLS client
cannot finish that download: it fails the HelloRetryRequest middlebox assertion
(erlang/otp#8470, docs/fips-leg.md finding 2, owned by slice 002). The child died
at `mix compile`, so every assertion about what a node booted was really an
assertion about a failed download.

The module now carries :needs_dev_compile, and test_helper.exs excludes that tag
only when TRINITY_FIPS_LEG=1, which is the variable the fips job already sets.
gate and postgres run all six cases unchanged. No workflow change was needed: the
leg already declares itself and the helper already branches on it.

MIX_ENV=test was tried first, as the preferred mechanism, and is written up in the
moduledoc. It does not block on the Ecto sandbox: test_helper.exs never runs in
the child, so the pool stays in :auto mode and the app boots. It was rejected for
a different reason. config/test.exs fixes the database path and does not derive it
from XDG_DATA_HOME, so six children write into the suite's own database; measured,
a child appended its boot receipt at seq 360 on the live chain. Overriding both
repos' paths and migrating inside the child fixes that on SQLite, measured at
1.45 s with 25 and 4 tables created and the boot receipt back at seq 1. It does
not carry to the postgres job, where :database names a database rather than a
file and each case would need one provisioned and dropped. That code is not
written here because it could not be tested here.

Nothing under lib/. No case deleted. No product change: this is a limit of the
harness on one leg.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
@HackTuah
HackTuah merged commit 918bad6 into main Sep 30, 2026
8 checks passed
@HackTuah
HackTuah deleted the feat/regulated-boot branch September 30, 2026 15:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant