Skip to content

Commit e0e96ed

Browse files
committed
test: prove AC1 to AC5 at a boot, in a spawned OS process
The profile suite proves the decisions Trinity.Profile makes. It does not prove a boot, and start/2 cannot be re-run in a VM where the application is already up, so calling that suite AC1 to AC5 proves the easy half. Four cases, each a separate OS process with its own environment: 1. default + a broken key store -> starts. This is rule 1 as a test: the receipts supervisor stays fail-open for every profile that is not :regulated, and if this case ever goes red that shape has been changed. 2. regulated + production mcp_auth + a non-Local authority + no endpoint allow-list -> does not start, naming regulated_llm_endpoints_unset. 3. the same, with the authority left to resolve to Local -> does not start, naming regulated_refuses_local_authority. 4. every condition met -> starts, then stops. Without this case, 2 and 3 pass for a profile that refuses everything and prove nothing. Isolation: XDG_DATA_HOME per case, so each child gets its own data directory, lock and database files. MIX_ENV=dev rather than test, because the test environment puts both repos on the Ecto sandbox and a child with no owner blocks on checkout instead of booting. mix run --no-start, so the script can set application environment and define the non-Local authority fixture before the code under test runs; that fixture stays in the child rather than going into lib/, because no adapter is supplied by this tree. Nothing here writes :persistent_term in this VM, and there is no collect/3. Verified by planting three violations and watching the right case go red: returning :ok from check_receipts/2 under :default changed nothing, which is rule 2 holding; running the receipts check for every profile turned case 1 red; short-circuiting the endpoint check turned case 2 red; short-circuiting the authority check turned case 3 red. lib/ is byte-identical to before the plants. Signed-off-by: Ayla Croft <aylacroft@proton.me>
1 parent 2f5aa7d commit e0e96ed

1 file changed

Lines changed: 218 additions & 0 deletions

File tree

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
# SPDX-FileCopyrightText: Sudo Apt Holdings LLC
2+
# SPDX-License-Identifier: Apache-2.0
3+
defmodule Trinity.RegulatedBootNodeTest do
4+
@moduledoc """
5+
AC1 to AC5 at the only place they are real: a boot.
6+
7+
`Trinity.ProfileTest` proves the decisions. It does not prove the boot, and a suite that proves
8+
`Profile.check_*` and calls that AC1 to AC5 is proving the easy half. `start/2` cannot be re-run
9+
in this VM, where the application is already up, so each case here spawns a **separate OS
10+
process** with its own environment and asserts on what that process did.
11+
12+
## How each case is isolated
13+
14+
`XDG_DATA_HOME` per case, which `Trinity.Paths.data_dir/2` reads on Linux, so the child gets its
15+
own data directory, its own `Trinity.DataDir.Lock` and its own database files and cannot collide
16+
with this suite or with another case.
17+
18+
`MIX_ENV=dev`, deliberately, and not `test`. The test environment puts both repos on
19+
`Ecto.Adapters.SQL.Sandbox`, and a child process with no sandbox owner blocks on checkout rather
20+
than booting: that is the trap slice 126 was opened for and then retracted over.
21+
22+
`mix run --no-start`, so the script can set application environment and define a fixture module
23+
**before** `Application.ensure_all_started/1` runs the code under test.
24+
25+
Nothing here touches `:persistent_term` in this VM. Every mutation happens in the child.
26+
"""
27+
use ExUnit.Case, async: false
28+
29+
@moduletag timeout: 180_000
30+
31+
# A signer failure the child can reproduce: the key file exists as a directory, so `File.read/1`
32+
# answers `{:error, :eisdir}` and `KeyCustody.boot!/1` answers `{:error, {:key_file, :eisdir}}`.
33+
# `keys_dir` itself is a real directory, so nothing raises on the way there.
34+
defp broken_keys_dir(root) do
35+
dir = Path.join(root, "broken-keys")
36+
File.mkdir_p!(Path.join(dir, "receipts-ed25519.key"))
37+
dir
38+
end
39+
40+
defp tmp!(name) do
41+
dir = Path.join(System.tmp_dir!(), "regboot-#{name}-#{System.unique_integer([:positive])}")
42+
File.mkdir_p!(dir)
43+
on_exit(fn -> File.rm_rf(dir) end)
44+
dir
45+
end
46+
47+
# One child boot. Returns `{output, exit_status}`; the script prints BOOT_OK or BOOT_REFUSED and
48+
# halts, so the status distinguishes the two without parsing prose.
49+
defp boot(env, script) do
50+
base = [
51+
{"MIX_ENV", "dev"},
52+
{"TRINITY_FAKE_PROVIDER", "1"},
53+
# Never inherit this suite's values for anything the code under test reads.
54+
{"TRINITY_PROFILE", nil},
55+
{"TRINITY_AUTHORITY", nil},
56+
{"TRINITY_REGULATED_LLM_ENDPOINTS", nil},
57+
{"TRINITY_MCP_AUTH_PROFILE", nil}
58+
]
59+
60+
System.cmd("mix", ["run", "--no-start", "-e", script],
61+
env: base ++ env,
62+
stderr_to_stdout: true
63+
)
64+
end
65+
66+
# The tail of every script: start the application and say what happened.
67+
@report """
68+
case Application.ensure_all_started(:trinity) do
69+
{:ok, _} -> IO.puts("BOOT_OK"); System.halt(0)
70+
{:error, reason} -> IO.puts("BOOT_REFUSED " <> inspect(reason, limit: 3)); System.halt(3)
71+
end
72+
"""
73+
74+
# A module that implements Trinity.Authority and is not Local. Defined in the child rather than
75+
# added to lib/, because AC5 says no adapter is supplied by this tree and none is invented.
76+
@fixture """
77+
defmodule RegulatedBootFixture do
78+
@behaviour Trinity.Authority
79+
def stage(staged, _ctx), do: {:ok, staged}
80+
def decide(_staged, decision, _ctx), do: {:ok, decision, "fixture"}
81+
def execute(_staged, _decision, _ctx), do: {:error, :not_used}
82+
def receipt(_kind, _attrs), do: {:error, :not_used}
83+
end
84+
"""
85+
86+
defp production_mcp_auth do
87+
"""
88+
Application.put_env(:trinity, :mcp_auth, profile: :production)
89+
"""
90+
end
91+
92+
defp allowed_models do
93+
"""
94+
llm = Application.get_env(:trinity, :llm, [])
95+
Application.put_env(:trinity, :llm, Keyword.put(llm, :models, [
96+
%{id: "allowed", provider: :req_llm, model: "openai:x",
97+
base_url: "https://models.internal/v1", api_key_env: "NONE",
98+
caps: [:stream], price: %{input: 0.0, output: 0.0}}
99+
]))
100+
"""
101+
end
102+
103+
describe "1. the default profile boots with the signer down" do
104+
test "a broken key store does not stop a default node" do
105+
root = tmp!("default-signer-down")
106+
107+
script = """
108+
r = Application.get_env(:trinity, :receipts, [])
109+
Application.put_env(:trinity, :receipts, Keyword.put(r, :keys_dir, "#{broken_keys_dir(root)}"))
110+
#{@report}
111+
"""
112+
113+
{out, status} = boot([{"XDG_DATA_HOME", root}], script)
114+
115+
assert status == 0,
116+
"""
117+
the default profile refused to boot with the signer down. That is rule 1: \
118+
lib/trinity/receipts/supervisor.ex is fail-open for every profile that is not \
119+
:regulated, and this is the test that says so.
120+
121+
#{String.slice(out, -3000, 3000)}
122+
"""
123+
124+
assert out =~ "BOOT_OK"
125+
end
126+
end
127+
128+
describe "2. regulated refuses an unset endpoint allow-list" do
129+
test "production mcp_auth and a non-Local authority, but no allow-list, does not start" do
130+
root = tmp!("regulated-no-endpoints")
131+
132+
script = """
133+
#{@fixture}
134+
#{production_mcp_auth()}
135+
#{@report}
136+
"""
137+
138+
{out, status} =
139+
boot(
140+
[
141+
{"XDG_DATA_HOME", root},
142+
{"TRINITY_PROFILE", "regulated"},
143+
{"TRINITY_AUTHORITY", "RegulatedBootFixture"}
144+
],
145+
script
146+
)
147+
148+
assert status == 3, "the node started with no endpoint allow-list:\n#{out}"
149+
assert out =~ "BOOT_REFUSED"
150+
151+
assert out =~ "regulated_llm_endpoints_unset",
152+
"it refused, but not for the reason under test:\n#{String.slice(out, -3000, 3000)}"
153+
154+
assert out =~ "TRINITY_REGULATED_LLM_ENDPOINTS"
155+
end
156+
end
157+
158+
describe "3. regulated refuses the local authority" do
159+
test "everything else correct, but TRINITY_AUTHORITY unset, does not start" do
160+
root = tmp!("regulated-local-authority")
161+
162+
script = """
163+
#{production_mcp_auth()}
164+
#{allowed_models()}
165+
#{@report}
166+
"""
167+
168+
{out, status} =
169+
boot(
170+
[
171+
{"XDG_DATA_HOME", root},
172+
{"TRINITY_PROFILE", "regulated"},
173+
{"TRINITY_REGULATED_LLM_ENDPOINTS", "https://models.internal"}
174+
],
175+
script
176+
)
177+
178+
assert status == 3, "the node started on the local authority:\n#{out}"
179+
180+
assert out =~ "regulated_refuses_local_authority",
181+
"it refused, but not for the reason under test:\n#{String.slice(out, -3000, 3000)}"
182+
end
183+
end
184+
185+
describe "4. regulated boots when every condition is met" do
186+
test "production mcp_auth, a non-Local authority, an allowed endpoint and a signer" do
187+
root = tmp!("regulated-good")
188+
189+
script = """
190+
#{@fixture}
191+
#{production_mcp_auth()}
192+
#{allowed_models()}
193+
#{@report}
194+
"""
195+
196+
{out, status} =
197+
boot(
198+
[
199+
{"XDG_DATA_HOME", root},
200+
{"TRINITY_PROFILE", "regulated"},
201+
{"TRINITY_AUTHORITY", "RegulatedBootFixture"},
202+
{"TRINITY_REGULATED_LLM_ENDPOINTS", "https://models.internal"}
203+
],
204+
script
205+
)
206+
207+
assert status == 0,
208+
"""
209+
a correctly configured regulated node refused to boot. If this fails while 2 and 3 \
210+
pass, the profile refuses everything and proves nothing.
211+
212+
#{String.slice(out, -4000, 4000)}
213+
"""
214+
215+
assert out =~ "BOOT_OK"
216+
end
217+
end
218+
end

0 commit comments

Comments
 (0)