Skip to content

Slice 110: Lua in the BEAM, under limits the machine enforces - #96

Merged
HackTuah merged 5 commits into
mainfrom
slice/110-luerl-sandbox
Sep 24, 2026
Merged

HackTuah merged 5 commits into
mainfrom
slice/110-luerl-sandbox

Conversation

@HackTuah

Copy link
Copy Markdown
Member

A run_lua tool and skill-owned scripts run Lua inside this process: no operating system process, no filesystem handle, no socket. A model asked to total a column produces a number that looks right; a script produces one that is right and leaves the script in the transcript for anyone to check.

The limits are the VM's, not a promise

Limit Enforced by Binds?
Wall clock monitor + timeout Yes
Heap max_heap_size with kill: true, set by the runner on itself Yes, the VM kills it
Concurrency max_children on the runners' supervisor Yes, refused by name
Reductions reported as a statistic No — see below

One limit is reported rather than claimed, and the measurement is published. luerl_sandbox enforces max_reductions by polling with a 100 ms sleep between checks:

cap=1000     -> reductions=21721012 in 101ms
cap=1000000  -> reductions=34373451 in 101ms

A cap of a thousand and a cap of a million behave identically because neither is ever binding. So reductions are a statistic here and the bound that stops a runaway script is time. AC1 is restated rather than met as written — asserting a small reduction bound would be asserting something false.

No second door

trinity.tool calls Trinity.Effects.Runner.execute/3, the executor a model's own call uses, so it is decided by the same gate and leaves the same receipts. A test asserts a denied call gives the script nothing, because that is exactly what would make Lua a second door.

A call needing a person fails immediately rather than holding the sandbox open — asserted with stats.time_ms < 1_500 against a 2,000 ms budget. A run is bounded in seconds and a person is not; letting a script block would turn every approval into a timeout.

Three corrections to this slice's own spec, all from measuring first

  • luerl_sandbox is not a package. It is a module inside luerl, so this adds one dependency, not two.
  • lua_entry already existed, carried by the skills parser since slice 040.
  • AC1's reduction bound is not one this library delivers, as above.

Two tests that would have proved nothing

  • The path-traversal test re-implemented the check and asserted against its own copy. script_path/2 is public now and the test calls the real function.
  • It expected /etc/passwd to be refused. It isn't: Path.join drops the leading slash and neutralises it into the skill's directory. The test asserts the true safe behaviour, because a test stating the wrong safe behaviour goes red the day someone makes the code stricter.

One thing I got wrong and undid

mix sobelow --mark-skip-all marked eighty-odd findings across the whole codebase, widening a suppression from 5 entries to 85. Restored from a copy and only this slice's one fingerprint added, with its reason. Sobelow reports exactly one finding in this tree, which is the only reason the damage was visible in a diff.

Verification

mix gate
Result: 794 passed (14 properties, 780 tests), 18 excluded
trinity.coverage: 014 80.31% vs 004 80.31%: OK

README, CHANGELOG, docs/sandbox.md and the effect catalogue are updated. AC7 (a recording of the agent writing Lua) and AC8 (a person reading the threat model) remain manual.

🤖 Generated with Claude Code

Parked while slice 014 fixes the prompt-window defect this slice exposed
(NOTES F4). Not a pull request yet: two tests unrelated to the sandbox are red
because adding one tool pushed the test window past the point where the session's
prompt is built from the oldest 500 rows rather than the most recent.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
A run_lua tool and skill-owned scripts run Lua inside this process: no operating
system process, no filesystem handle, no socket. A model asked to total a column
produces a number that looks right; a script produces one that is right and leaves
the script in the transcript.

The limits are the virtual machine's rather than a promise. The runner sets its own
max_heap_size and is killed at the boundary without this code being involved, a
wall clock bounds the run, and the pool refuses an over-limit run by name rather
than queueing it invisibly, because a run that waits an unbounded time to start has
no wall-clock bound at all.

One limit is reported rather than claimed. luerl_sandbox enforces max_reductions by
polling with a 100 ms sleep between checks, so caps of 1,000 and of 1,000,000 both
stop a tight loop at 20 to 34 million reductions. Reductions are a statistic here
and the bound that stops a runaway script is time; docs/sandbox.md carries the
measurements rather than the claim.

What a script wants done it asks for through Effects.Runner.execute/3, the executor
a model's own call uses, so it is decided by the same gate and leaves the same
receipts. A test asserts a denied call gives the script nothing, because that is
what would make Lua a second door. A call needing a person fails immediately rather
than holding the sandbox open: a run is bounded in seconds and a person is not, and
letting a script block would turn every approval into a timeout.

A skill may ship a script under its own scripts/ directory. Where it may read is
checked by resolving the path rather than trusting it, so a traversal is caught by
where it lands. script_path/2 is public so the test exercises the real function; the
first version re-implemented the check and asserted against its own copy, which is
the one way to make a security test meaningless.

The skill scanner gained a sandbox_escape rule, and both the code and the test say
it is a reviewer's signal rather than a control: the control is that the globals are
absent at run time, which the sandbox tests assert one by one.

Three corrections to the slice's own assumptions, all from measurement before code:
luerl_sandbox is a module of luerl rather than a package, lua_entry has existed in
the skills parser since slice 040, and AC1's reduction bound is not one this library
delivers.

mix gate: 794 passed (14 properties, 780 tests), 18 excluded.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
…ole VM

CI failed on gate, postgres and fips at the same test while it passed locally every
time. It compared Process.list/0 before and after ten killed runs and asserted the
delta was under ten. On a quiet machine that reads as the runners going away; on a
CI runner the rest of the suite moves that number far more than ten runners do, so
it was measuring noise and calling it a leak.

It counts Task.Supervisor.children on the sandbox's own supervisor now, which is
exactly the population the assertion is about, polled to empty against a bound
rather than slept at.

The first version had the shape of a good test: a before, an after and a threshold.
What it lacked was a population it controlled, and a threshold over a population you
do not control is a coin toss that usually lands the right way up locally.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
Trinity.Telemetry.CostsTest failed with Exqlite.Error: Database busy on the gate
leg of pull request 96 while the sibling run of the identical commit passed. The
test is untouched by that branch, the error is the row's original wording, and the
failure is on a write.

Filed as R26 only after reading the stack trace, which is what the counter-example
entry above it exists to insist on: three failures this week wore that signature
and two of them were real defects in this tree.

What it adds to the row: the three candidate fixes have now gone untaken across
four blocking occurrences in two days, which is a decision rather than an
oversight. It should be a slice rather than a re-run.

Signed-off-by: Ayla Croft <aylacroft@proton.me>
@HackTuah
HackTuah merged commit 9e9259a into main Sep 24, 2026
8 checks passed
@HackTuah
HackTuah deleted the slice/110-luerl-sandbox branch September 24, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant