A reservation ledger, for the case where the floors fit and the caches do not

[marcobambini]

Narrowed from #31, which closed when gate 1 ran. This is the residue that
survived it, and the residue is smaller and better defined than the original
question.

What #31 settled, and is not up for re-litigation

• Two K3 opens on a 64 GB machine cannot be fixed by any budget policy.
  docs/LEARNED.md §64: K3's floor is 29.19 GB, almost entirely the resident
  trunk, so two floors are 58.38 GB against a 51.54 GB ceiling. A ledger would
  drive the second process to its floor and the pair would still oversubscribe.
  Gate 1 confirmed the failure mode empirically — neither process reached
  decode, and the machine needed a power cycle.
• Bounding the ceiling by an instantaneous sample stays refused, for the
  reason in src/memory.c.
• Explicit budgets stay authoritative. Warn, never clamp.
• Capacity is solved. waste_usable_ram() is min(physical, cgroup limit).

Where the case actually exists

A ledger only has something to divide when the floors fit and the caches do
not. That is not a corner — it is what a larger machine running K3 looks
like, and it is the opposite of where #31 was written:

host	ceiling 3/4	one open picks	two opens	two at the floor
64 GB	51.54 GB	46.39 GB	92.78 GB — 135%	58.38 GB — 85%, over the ceiling for one
128 GB	96.00 GB	80.77 GB (floor + 3x)	161.54 GB — 126%	58.38 GB — 46%, comfortable

On 64 GB there is nothing to engineer: the floors alone bust the budget. On
128 GB the second process has 37 GB of headroom above its floor that a ledger
could hand it deliberately instead of leaving it to be discovered by paging.

This is the row #31's table called "K3 + Kimi-Linear, different containers" —
generalised. The predicate is not container identity and never was; it is
resident bytes against a ceiling somebody else has already spent, which is
arithmetic waste_open is one input short of doing.

What is already argued about the shape

From #31's thread, so it is not re-derived:

• kill(pid, 0) is the wrong liveness primitive. Pid reuse leaves a dead
  reservation unprunable, and every future budget on that machine is
  permanently smaller — a symptom nobody attributes to a stale file. Closing it
  with process start times costs three platform paths.
• flock per reservation is the right one, and is already in the tree. One
  file per reservation held under LOCK_EX for the life of the process; the
  kernel releases it on any exit. A reader takes LOCK_SH | LOCK_NB and a file
  it can lock is dead. The refcounted, atfork-safe, CLOEXEC implementation
  is at src/waste.c:117, landed as exclusive_open in Add opt-in exclusive model-container ownership #29 — same technique,
  different key.
• Placement has no answer that is both safe and complete. World-writable
  means any local user can publish a 10 TB reservation and starve the machine;
  per-user is safe and blind exactly where the harm is real, on a shared box.
  This is a choice, not a cost, and it comes before a file format.
• Namespaces are worse. Two containers on one host each see one process,
  believe they are alone, and share one kernel's RAM. Silent under-counting in
  a safety mechanism is worse than no mechanism, because the printed budget
  line then looks like it was checked against something.
• waste.h says no global state. A cross-process ledger is the first thing
  this project would write outside a container directory. That is a decision
  about what the library is.
• The "capacity, not pressure" defence is weaker than Two auto-budget opens on one machine each size against the whole machine #31 stated. The sum
  of live reservations is not fixed for the life of the process. What survives
  is that its volatility is declared and attributable: a smaller budget is
  always matched by a real, still-live claim, rather than by a transient that
  has already gone. That is enough to admit the input; it is not determinism,
  and the cost has to be accepted openly.

Gates, and the first one must not be run the way #31's was

#31's gate 1 cost a reboot because the unmanaged arm is the dangerous one.
That is unavoidable here too, so it has to be bounded by the host rather than
by hope.

1. Does the floors-fit case degrade at all? Two auto-budget K3 opens on a

   64 GB host, against a single-process baseline. Run under a memory cap
   that bounds the damage — Linux with memory.max on a cgroup, where the
   OOM killer ends the arm instead of the machine. If the OS degrades
   gracefully when the floors fit, this closes with a dated entry and nothing
   is built.

2. Would the ledger's outcome have been better? Same pair, both pinned by
   hand to the budget a ledger would have picked — the cap split between them.
   Unlike Two auto-budget opens on one machine each size against the whole machine #31's gate 2 this is a safe arm by construction, since two floors are
   46% of a 128 GB host. If two processes sharing the ceiling are meaningfully
   faster than two contending for all of it, there is something to engineer.
3. Only then, pick the shape, against the constraints above.

@Lrrr908 has the machine this needs — 128 GB, Linux, and a K3 container, from
#37 — and gate 1 is the arm a 64 GB host can never select. No obligation
attached; noting it because it is the only host in this project's reach where
the question is answerable safely.

Non-goals

• Any change to explicit --budget.
• Anything that refuses an open that would otherwise have succeeded. Refusal
  is what exclusive_open already does, opt-in, and Two auto-budget opens on one machine each size against the whole machine #31's criticism of its key
  stands — it just turns out the right predicate has no non-refusing answer in
  the 64 GB case.
• The 64 GB two-K3 case. §64 closed it; the only things that would cover it are
  refusing, or not duplicating the trunk, which is a different engine shape.

Done when

docs/LEARNED.md has a dated entry with the gate-1 numbers, and either a
mechanism behind them or the reason there is none — the second outcome being as
good as the first.

[marcobambini]

@Lrrr908 — the issue names your machine, so here is the concrete version, plus
the scoping I can give you now and could not give you from a results table
afterwards. No obligation, same as last time.

Your Gate 7 dataset became docs/LEARNED.md §63 and closed the last open gate
in GATES.md, so this is the second thing only your host can answer.

Why the cgroup is not just a safety net

waste_usable_ram() is min(physical, cgroup limit), reading memory.max and
memory.high hierarchically (src/memory.c, §40). So putting both processes
in one limited cgroup does not merely contain the damage — it re-creates
the experiment at a bounded scale, because each process then sizes itself
against the cgroup rather than against the machine. Two processes each taking
3/4 of what they can see is 1.5x what exists, which is the whole phenomenon.

That is why this is runnable on your host and was not on mine. On a 64 GB
laptop the unmanaged arm has nowhere to be contained to — #31's gate 1 froze
the machine and needed a power cycle, with neither process reaching decode.

The recipe

cgroup v2, hard limit, and swap off inside it — otherwise you get thrash
instead of a crisp bound, which is exactly what killed my run:

sudo mkdir -p /sys/fs/cgroup/warpgate
echo $((80 * 1024**3)) | sudo tee /sys/fs/cgroup/warpgate/memory.max
echo 0                 | sudo tee /sys/fs/cgroup/warpgate/memory.swap.max

Then move a shell in and work from it:

echo $$ | sudo tee /sys/fs/cgroup/warpgate/cgroup.procs

First, confirm what the engine sees — this replaces any arithmetic I would
otherwise be doing on your behalf from an ambiguous "128 GB":

./waste plan ~/models/k3.waste

Report that output. At memory.max = 80 GiB the resolver's ceiling is 60 GiB,
so it should select floor + 1x — the same arm that killed my machine,
here with 48 GiB of host RAM outside the cgroup keeping the box alive. If you
would rather exercise floor + 2x, use 96 GiB instead; the ceiling becomes
72 GiB and floor + 2x at 63.58 fits.

Baseline, then the pair:

./waste bench ~/models/k3.waste -n 30                 # one process
./waste bench ~/models/k3.waste -n 30 &               # then two, together
./waste bench ~/models/k3.waste -n 30 &
wait

Worth reporting: the plan output, per-arm tok/s and peak RSS, the cgroup's
memory.peak and memory.events (the oom and oom_kill counters), any
dmesg OOM lines, and — if one process is killed — which one and how far in.
Medians over repeats if the arms complete; if they do not, the elapsed time and
resident set at death is the measurement.

The two things worth knowing before you spend the time

1. A "no collapse" result is a real one and closes this issue. If the
cgroup reclaims gracefully and both processes finish, that kills the premise
and nothing gets built — which this issue says in advance is as good an outcome
as a mechanism. Do not treat a boring result as a failed run.

2. On your machine a ledger can at best hand both processes the floor, and
this is the part I would not have been able to tell you afterwards. The
resolver steps in whole multiples of a working set. For two K3 opens to each
get floor + 1x you need 2 × 46.39 ≤ 3/4 × physical, which is 123.7 GiB of
RAM — more than you have. So the ledger's best case for this workload is
"both at the floor", and gate 2 measures the entire value of the mechanism,
not a sample of it. Two at the floor is 58.38 GiB, comfortable inside an 80 GiB
cgroup, so that arm is safe by construction.

If gate 2 says two at the floor are not meaningfully better than two contending
for everything, there is nothing to build here either, and this issue closes the
same way #31 did.

One datapoint §64 explicitly says is missing

My run was macOS, which grew swap to 40 GB and froze rather than killing
anything. §64 records that a Linux OOM killer would have shot one process and
the survivor might well have completed — a different result, and one nobody has.
Your run produces it as a by-product whether or not the rest is interesting.

[mfethe1]

A reservation ledger needs a feasibility-precedence gate in front of its policy rules — and the interesting corner is inside your scope, not outside it.

This is evidence for the narrowing, not a competing design. #31 closing on "no budget policy fixes it" is what prompted us to check whether the inverse rule we had derived was safe, and it was not.

What we had

Working the miss-time side of this problem, we had derived a funding rule for capacity restore (recovering cache after a pressure episode rather than staying shrunk):

fund restore iff beta*(1-hit) > spine0 — miss-share of token time above ~50%

T = spine0 + beta*(1-hit), fitted over four regime corners from third-party rigs. It survived every corner we threw at it, which is exactly why §64 made us nervous: it is a value rule stated over an operating point where restore is assumed implementable.

What §64 breaks

Miss-share saturates toward 100% as capacity collapses. So the rule fires hardest precisely where there is nothing to restore. On S64-2×K3 it reads 91.7% and says fund restore into −6.84 GB of headroom.

That corner is yours and already settled — we are not re-litigating it. It is the negative control.

The corner that is not settled

Using your numbers (K3 floor 29.19 GB, ceilings 3/4 physical per §57/§63 — thank you for the 7/8 correction in #37, we had inherited it too), with quantum = one token's working set = 17.19 GB from waste plan @ d13493a:

host	floor	ceiling	headroom	feasible	max miss-share	rule fires	misfire
S64-1×K3	29.19	51.54	22.35	✅	87.6%	1/4	0/4
S64-2×K3	58.38	51.54	−6.84	❌	98.9%	4/4	4/4
S128-1×K3	29.19	96.00	66.81	✅	87.6%	1/4	0/4
S128-2×K3	58.38	96.00	37.62	✅	87.6%	1/4	0/4
S128-3×K3	87.57	96.00	8.43	❌	96.8%	2/4	2/4

S128-3×K3 is the load-bearing cell. The floors fit — 87.57 against 96.00, which is precisely the case this issue narrows to — and the ledger still has nothing to divide: 8.43 GB against a 17.19 GB quantum. Below one token's working set the cache cannot hold a single token's experts, so on our own replay data the hit rate is 0.00% at any policy, not merely poor. A ledger there splits a pool too small for any participant to use, and the value rule endorses it at 96.8% miss-share.

"Floors fit" is necessary but not sufficient. The sufficient condition is ceiling − floor ≥ quantum, per participant.

Scope check, because the subtraction is only meaningful if they compose

Before trusting any of the above: floor is resident trunk, cache excluded (§64: "almost entirely the resident trunk"); ceiling is waste_usable_ram(), cache included. Same units, same machine, and the residual is the cache pool. That was a pre-registered kill condition for this analysis — if the two had been defined on different scopes the whole framing would have been wrong.

Proposed ledger ordering

1. hard runtime + liveness reservations (floors)
2. safety margin
3. minimum restore quantum ← the gate, evaluated before any policy rule
4. cache / prefetch allocation
5. policy optimisation within what remains

A high miss share is a symptom of capacity starvation, not authorisation to spend capacity that does not exist.

Claim discipline

• MEASURED (yours): floors, ceilings, one-token working set, the 64 GB liveness failure.
• RECOMPUTED (ours): headroom, feasibility, miss-share, monotonicity — $0 from your pinned numbers.
• ESTIMATED: the hit-vs-headroom curve.
• NOT ESTABLISHED: tok/s, latency, speedup. No throughput claim anywhere in this.

Three caveats worth stating rather than burying:

1. The hit curve is Kimi-Linear-48B, not K3 — 256 experts / 26 layers vs 896. The dead zone below one token's working set is arithmetic and architecture-general; the specific hit values are not K3's. This is why the verdict rests on the feasibility inequality, which needs no hit curve at all, rather than on the hit column.
2. Multi-open floors assume no shared resident pages between K3 opens. If the trunk is shared they stack sublinearly and S128-3×K3 might clear. Unverified — flagged rather than assumed away.
3. quantum = one token's working set is a choice with a stated reason, not a measured threshold. A smaller quantum shrinks the misfire set; it does not rescue S64-2×K3, which is negative regardless.

Script, pinned inputs and SHA256SUMS are on our side and reproducible — the whole thing is a $0 recompute over your published numbers, no new measurement. Happy to post it if useful.

On gate 1: agreed the unmanaged arm is the dangerous one, and the cgroup point is sharper than a safety net — since waste_usable_ram() reads the cgroup limit, both processes size against it, which reproduces the phenomenon at bounded scale rather than merely containing it. We do not have a >64 GB Linux host to run it on, so this is analysis, not a claim on the gate.