diff --git a/.github/ISSUE_SPECS/1106-spectrum-hop-probe-waits-on-the-clock.md b/.github/ISSUE_SPECS/1106-spectrum-hop-probe-waits-on-the-clock.md deleted file mode 100644 index de10b1e2f..000000000 --- a/.github/ISSUE_SPECS/1106-spectrum-hop-probe-waits-on-the-clock.md +++ /dev/null @@ -1,86 +0,0 @@ -# Bound the browser spectrum hop probe by its deadline, not a poll count - -## Problem - -The SDK-driven browser qualification's configured spectrum hop probe -(`hosts/host-web/qualification/sdk-response-entry.ts`, `runConfiguredSpectrumHopQualification`) -waits for two published windows after it resumes the `AudioContext`: - -```ts -const deadline = performance.now() + 10_000; -for (let attempt = 0; attempt < 512 && publications.length < 2; attempt += 1) { - await subscription.pump(); - if (performance.now() >= deadline) break; -} -``` - -`pump()` can return almost at once, so the loop can use up its 512 attempts long before the -10-second deadline. If the audio device starts slowly, the second 1,024-sample window (about -21 ms at 48 kHz) has not been published yet. The probe then fails with -`H1024 browser probe published 1 windows`. - -Evidence: -- It failed once in Chromium on PR #1105 (run 36719596271, job 109902172521). -- It failed again in Chromium on PR #1238 (run 37115329466, attempt 1, job 111181516316, head - `8c6268967`), and passed when the failed jobs were re-run. The same attempt failed Firefox's - `sdk-spectrum-continuous` gate, which is *Name the failed predicate and bound the waits of the - browser continuous-spectrum gate by a deadline* (#1248). -- It passed when the failed job was re-run, and in every local three-browser run. -- It failed again in Chromium on PR #1625 (controls batch 9, head `097b9151e`) after passing on that PR's first run and in two local three-browser runs on the same code; batch 9 changed nothing the probe reaches (the qualification page's spectrum code and the spectrum service are unchanged by it). -- The probe's code has been unchanged since `756ed813`. - -## Smallest closable slice - -Bound the wait by the deadline only, and yield to the event loop between pumps (for example -`await new Promise((r) => setTimeout(r, 5))`), so the probe cannot finish before the audio clock -has advanced. Keep the 10-second deadline, the two-window requirement and the typed failure. -Apply the same fix to any sibling probe in the qualification entry points that bounds a -wait by an attempt count. - -Authorized paths: `hosts/host-web/qualification/sdk-response-entry.ts`, any sibling entry in -`hosts/host-web/qualification/`, and this spec. - -## Objective gates - -1. The browser qualification with `--check-matrix --self-test-mutations` passes in Chromium, - Firefox and WebKit. -2. A planted slow start (for example a 500 ms delay before the first pump) still passes, while the - same delay with the old attempt bound fails. Record both as PR evidence. -3. A probe that never publishes still fails with its typed message within the deadline. - -## Decision record: hop probe wait (batch 8) - -Shipped. `runConfiguredSpectrumHopQualification` waits through `waitForCount`: it polls `pump()` -and then sleeps 5 ms, and stops at a 10 s `performance.now()` deadline. There is no poll count. -The failure still names what it saw: the windows seen with their captured samples, the elapsed -time, the poll count, the notification count and the context state. -The only sibling bounded by an attempt count was `submitBlocks` in the collection probe (200 -retries of 5 ms on source backpressure). It now stops at the probe's 10 s readiness budget. - -Departures from the text, and why: -- The wait starts at the click, not when `resume()` resolves. Firefox resolves `resume()` only - when its audio has started, so a wait that began after the resolve could not observe a slow start. -- The wait ends on two windows one hop (1,024 samples) apart, not on the first two windows. A - late start leaves the first reads behind the audio and the capture ring drops windows (WebKit - published windows 7,168 samples apart under the planted delay). The hop is the distance between - two consecutive publications, so a hop bug still times out and fails; the row reports that pair. -- The slow start is planted by holding the native `startSpectrumStream` for 750 ms after the - click. The context is running and every `pump()` returns at once with nothing published. A - `resume()` outside the click handler is not possible (the `resume-clicks` gate), and a worklet - that stalls the audio thread did not work: it blocks `pump()` too, so the old loop survived it. - -Gates (`run.mjs`, row `sdk-spectrum-hop-wait`): the first publication is at least 650 ms after the -click; the wait ended before its deadline with a bounded poll count; a wait that can never be -satisfied (250 ms deadline, nothing published) ends at its own deadline and polls no faster than -its timer. Self-test mutations: `sdk-spectrum-hop-wait-fast-start`, `-early`, `-spin`, -`-never-unbounded`. - -Measured (first publication / second publication after the click, polls): Chromium 813 / 836 ms, -162 polls; Firefox 836 / 889 ms, 150 polls; WebKit 1289 / 1344 ms, 165 polls. The never-publishing -wait ended at 254 ms (deadline 250 ms) with 35 to 50 polls. - -Red evidence: under the same 750 ms plant, the old 512-poll loop (restored temporarily after the -resume, then removed) fails in Chromium (`published 0 windows` after 512 polls in 13 ms) and WebKit -(512 polls in 10 ms). It passes in Firefox, whose `resume()` resolves only after the delay. With a -3 s plant and a 1 s deadline the new probe fails in bounded time with its typed message (`published -0 windows in 1001 ms (deadline 1000 ms, 197 polls ...)`). diff --git a/.github/ISSUE_SPECS/1280-carry-live-controlled-effect-lanes-across-a-plan-swap.md b/.github/ISSUE_SPECS/1280-carry-live-controlled-effect-lanes-across-a-plan-swap.md deleted file mode 100644 index cd4aeb01e..000000000 --- a/.github/ISSUE_SPECS/1280-carry-live-controlled-effect-lanes-across-a-plan-swap.md +++ /dev/null @@ -1,489 +0,0 @@ -# Carry live-controlled effect lanes across a plan swap - -Stream A of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-2, D15-7, D15-8). -Slice 11 of *Swap a rebuilt plan without an audio gap* (#1269). Code anchors verified on `main` at -`6fb211594`. - -## Product outcome - -A banked effect lane that a live control drives keeps its exact state through a plan swap. The -browser always attaches live controls, and the C ABI does since #1263. The lane keeps its live -state too: its live bypass, its dry shunt, its unread live values and its pending Observe -records. No acknowledged live edit is lost and none is refused: a value committed but not yet -rendered at the swap is rendered by the successor, at any quantum, including a ramp in flight -across the swap. A structural transaction that also changes a -carried lane's live parameter or bypass sounds exactly like "live edit, then structural edit" -(D15-7). - -## Context - -- A live-controlled bank slot is `LiveControlEffectBankStage` (`crates/rack/src/lib.rs:937`, impl - `:1151`). Its `drain` (`:1257`, run from `begin_block`, `:1211`) only **stages** values. Each - lane's `EffectControlLane::stage` (`crates/effect-contract/src/live.rs:341`) packs automation - spans or prepared targets, and `process_inner` (`rack/src/lib.rs:1294`) applies them in the same - block. Draining at the carry would stage values into a plan that never renders again (#1269's - blocker X3). -- On `main` before #1345, `EffectControlLane` (`live.rs:125`) held a queue consumer, a - prepared-target FIFO, the live `bypass` flag and the live symmetry terms. **After #1345 - (root note, 2026-10-12, stream B batch-3 verdict): the observation queue's consumer is no longer - in `EffectControlLane`. It lives in `LiveControlEffectBankStage::observe_queues` - (`crates/rack/src/lib.rs` ~1162, taken out of each lane at bind, drained at ~1530).** Anything - below that says the carry reaches the observation FIFO through `EffectControlLane` reaches it - through that stage field. Carrying a queue would need an inherited - consumer and an admission limit on the successor, which is a new BACKPRESSURE for a live value. - Decision 15 forbids that (D15-2). So this slice lands after *Hold effect parameter, bypass and - EQ-target values in latest-target cells* (#1345), and carries **cells**: - - #1345 D1 gives each lane one cell per `(parameter_index, channel)` of a block-rate parameter, - one bypass cell, and for the EQ one cell per `(target slot, channel)`. Observe records stay in a - small FIFO (#1345 D2). - - #1312 D1-D2 make each cell a revision-bounded three-slot cell (#1432 D1, Amendment 1). Each - slot carries its revision and a sequence. Render applies, per cell, the newest value whose - revision is `<=` the block's snapshot `S`, and keeps the last sequence it applied (`p`). - Applying sequence `s` adds `s - p - 1` to `live_values_superseded`. - - #1432 D1 also gives the reader `peek_unread`. It returns the newest unread words and their - sequence without consuming them, and is valid once the writer is quiescent. - - The control plane writes only the newest plan's cells: the pending candidate's if there is - one (D15-17; #1053 D7). So no control write reaches a predecessor's cells once its successor is - published, and render reads them whole. -- The slot's dry shunt is one interleaved AoSoA line for the whole bank, with one shared cursor - (`shunt`, `rack/src/lib.rs:956-958`; `BypassShunt`, `live.rs:868-890`). Observation taps are - `ObservationLane` (`live.rs:588`), one per bank lane (`rack/src/lib.rs:961`). -- The console-lane carry through the payload scratch, the effect rule D1 and the refusal counter - come from *Carry console effect lanes across a plan swap* (#1279). The base, the retarget pattern - and the restart set come from #1277 (D2-D6). - -## Decisions frozen for this slice - -- **D1. Rule.** As #1279 D1, with live controls attached in both plans or in neither. Live - parameter values and a lowered bypass are live, because the lane exists. Everything #1279 D1 - lists as prepared stays prepared. -- **D2. Carried cells.** `EffectControlLane::carry_from(&mut self, predecessor: - &EffectControlLane)` takes the predecessor by shared reference and writes only the successor - lane: - - it copies the applied live `bypass` flag and the live symmetry terms; - - for every predecessor cell where `peek_unread` (#1312 D1) returns `Some((words, s))`, it stores - the words in the successor lane's render-owned **carried slot** for that cell, with the unread - count `n = s - p`, where `p` is the predecessor cell's last applied sequence. The writer is - quiescent: no control write reaches a predecessor's cells once its successor is published; - - it consumes nothing: the predecessor never renders again, so nothing reads its cells after the - carry; - - it writes no cell, so the control thread stays each cell's only writer. - - Each carried slot resolves at the **successor's first rendered block**, before that block stages - anything. The carried slot is render-owned state, like a ramp in flight. The predecessor would - apply the same value at its next block, which is the same boundary B, so the adopted output is - exact. It resolves in #1345 D3's order: - - if the successor's own cell's read under the adoption block's snapshot is `Applied` (an - unread value whose revision is `<= S`), that value is newer (written by D4's retarget or after - the successor's publication). It applies; - - otherwise the carried value applies, and a successor value whose revision is `> S` stays - pending for a later block (Amendment 1). - - Every predecessor value is `<= S`: every write to the predecessor carries a revision below the - successor's revision (#1312 D6). - - Counting (#1312 D2, one unit, each superseded value once): the predecessor never reads again, - so the successor counts for it. It adds `n` when its own newer value wins, and `n - 1` when the - carried value applies. A successor's own cell counts only its own values, as every cell does. - - The carried slots are preallocated with the lane, one per cell, and cleared after that block. - Nothing is refused and nothing is dropped. -- **D3. Pending Observe records.** (After #1345 the FIFO's consumer is - `LiveControlEffectBankStage::observe_queues`, not `EffectControlLane`; root note, 2026-10-12.) - The carry drains the predecessor's observation FIFO with the - bounded drain and applies each record to the predecessor's `ObservationLane`, at the boundary - where the predecessor's next block would apply it. Observation output is not audio, and the - observer carry (*Carry meter and effect observation state across a plan swap*, #1327) then - carries the tap state. -- **D4. Carry, then retarget (D15-7).** The per-effect value derivation inside - `classify_live_delta` (`crates/host-core/src/live_delta.rs:211`) is extracted into one per-owner - function, as #1277 D5 did for strips. For every carried lane whose live values differ, the - successor entry points write its cells before they return: the bypass cell, the parameter cells, - or an EQ's target cells designed from the base values. A cell write cannot fail (#1345 D5). The - writes happen exactly once, at preparation (#1277 D5). -- **D5. Shunt.** A carried lane's shunt words move with it, as a copy of that lane's words of the - interleaved line, taken relative to the shared cursor of each bank. -- **D6. Acked-batch question: can an ack ever precede a drop? No.** This slice adds no admission - check and no refusal. Every acked value is in a cell of the plan it was written to. A carried - value reaches the successor through D2, and a value superseded there is counted exactly. - -## Deliverables - -1. D2 and D3 in `crates/effect-contract/src/live.rs`. D4 in `crates/host-core`. D5 in - `crates/rack`. The location table and the program section in `crates/graph`. The inventory rows - and the join in `crates/host-core`. -2. Gap-free tests in `crates/host-core/tests/successor_swap.rs`. - -## Authorized paths - -- `crates/effect-contract/src/live.rs` -- `crates/rack/src/lib.rs` -- `crates/graph/src/lib.rs`, `crates/graph/src/runtime.rs` -- `crates/host-core/src/prepare.rs`, `crates/host-core/tests/successor_swap.rs`, - `crates/host-core/tests/support/successor.rs` -- `crates/host-core/src/live_delta.rs`: D4's extraction only. This is stream B's file. - -## Non-goals - -- No bank to per-node flips (#1281), and no per-node moves (#1282). -- No admission change on either host. Cells (#1345) already admit every live value. -- No observer window carry (#1327, #1287). - -## Objective gates - -1. **No acked value lost.** Prepare with live controls and a console EQ. An EQ gain target - `v0 → v1` is rendered in A. A target `v1 → v0` is written to A's cells just before the swap and - not yet rendered. B is prepared from the base (`v0`). Every block equals the reference A plus the - muted track, fed the same values at the same blocks. Repeat with a compressor threshold. -2. **Ramp across the swap at a small quantum.** At quantum 32, a compressor threshold ramp of 64 - samples in flight at the swap is bit-identical to the reference. -3. **Newest value wins, counted exactly.** Write 40 values of one compressor parameter to A's cell - after A's last block, then publish B, then write 40 values of the same parameter to B's cell. - Every write succeeds. B's first block equals a twin that wrote only the last value, and - `live_values_superseded` grows by exactly 79 (A's 40 carried values and 39 of B's). A - second case writes nothing to B: the carried value applies and the counter grows by 39. -4. **Live bypass and shunt.** A true-peak limiter insert on four tracks (a full bank at - `Backend::Simd4`; eight tracks at `Backend::Simd8`), with one lane bypassed live before the swap - and the cursor at a nonzero offset. The bypassed lane keeps its delayed dry signal across the - swap. -5. **Carry, then retarget.** B also changes a carried lane's compressor threshold and an EQ band - gain (live values). Run 1 is the structural swap with D4's retarget. Run 2 writes the same edits - to A just before the swap and prepares B from a base that holds them. The two runs are - bit-identical in every block, and the strip is not in the restart set. -6. **Realtime.** Every swap block makes zero allocations and frees. -7. Commands: - - `cargo test --locked -p effect-contract -p rack -p graph -p host-core --features rack/test-support,graph/test-support,host-core/test-support` - - `cargo build --locked --release -p audit -p bench && bash scripts/trace-graph-audit.sh target/release/audit && bash scripts/trace-effect-contract-audit.sh target/release/bench` - - `cargo run --locked --release -q -p realtime-policy`, `bash scripts/check-workspace-policy.sh`, - `cargo fmt --all -- --check`, - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings` - -## Test value - -- Gate 1: a carry that drops the predecessor's unread cells, or stages them into the retired plan, - renders `v1` forever. It turns red. -- Gate 2: a payload that re-derives a ramp's step moves bits when the ramp crosses the swap. It - turns red at quantum 32; every other gate runs at 128. -- Gate 3: a carried value that overrides the successor's newer cell applies an old value, and a - miscounted carry moves the exact count. Either turns it red. -- Gate 4: a shunt copy that ignores the shared cursor turns it red. -- Gate 5: a retarget that loses to the carried value applies the old value last. A missing retarget - keeps the old threshold. Either turns it red. - -## Amendment 1 (root, 2026-10-05): revision-bounded cells - -Root's binding requirement (2026-10-05) makes every live value of one revision take effect in one -block (#1432 Amendment 1, #1312 Amendment 2). Two parts change: - -- the Context bullet on #1312 D1-D2; -- D2's resolution, which now reads the successor's own cell under the adoption block's snapshot. - -The counting rule is unchanged. - -## Dependencies - -- *Carry console effect lanes across a plan swap* (#1279). -- *Hold live values in latest-target cells on both hosts* (#1312): `peek_unread`, which D2 reads. -- *Hold effect parameter, bypass and EQ-target values in latest-target cells* (#1345): this slice - carries the cells #1345 creates, so #1345 lands first. - -Under a latency growth, the warm successor adopts in move mode at its first ready block `S` -(*Grow latency during playback by adopting a primed warm successor*, #1287). That is an ordinary -move-mode carry at a later boundary, so D2's carried slots resolve at `S` by the same rule, and this -slice's gates cover the rule. - -## Amendment (decision 17, root 2026-10-13) - -**Split.** Both pieces stay in this issue. - -- **(a)** D2 (carried cells) and D4 (carry, then retarget). Gates 1, 2, 3, 5 and 6. -- **(b)** D5 (shunt) and D3 (pending Observe records). Gate 4 and the new gate 8. Dependency: (a). - -**Gate 8 (D3), new.** In `crates/host-core/tests/successor_swap.rs`: a compressor insert with -live controls. After A's last block and before B is published, admit Observe records to the -lane's observation queue (`LiveControlEffectBankStage::observe_queues`): one subscribe, and in a -second case the queue filled to its capacity. At the carry, every record is applied to A's -`ObservationLane` with the bounded drain: A's queue reads empty, and A's lane holds the state that -A's next block would have reached by applying the same records. Zero allocations and frees on the -swap block. It turns red if the carry leaves records in the retired queue (an acknowledged -subscribe is lost) or drains past the queue's count at entry. - -## Amendment (controls coordinator, batch 10, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Anchors moved.** `LiveControlEffectBankStage` is `crates/rack/src/lib.rs:1154` (`new` `:1218`, - `begin_block` `:1459`, `drain` `:1526`, `process_inner` `:1574`; fields `observe_queues` `:1165`, - `shunt` `:1178`, `observations` `:1184`, `designed` `:1194`). `EffectControlLane` is - `crates/effect-contract/src/live.rs:748` (`bypass` `:770`, `mix` `:773`, `mix_position` `:776`, - `symmetry` `:792`), `stage` `:1035`, `ObservationLane` `:1321`, `BypassShunt` `:1621`. - `classify_live_delta` is `crates/host-core/src/live_delta.rs:324`; the per-instance derivation D4 - extracts is `effect_records` (`:1689`), `parameter_records` (`:1743`) and `design_targets` - (`:1793`). #1279's carry is `graph::bank_effect_lanes` and `install_effect_carry` - (`crates/graph/src/lib.rs:3229`, `:3246`), `Runtime::carry_effect_lanes` - (`crates/graph/src/runtime.rs:4726`), `EffectBankStage::carry_lane_from` - (`crates/rack/src/lib.rs:945`), `join_chains` and `resolve_effect_carry` - (`crates/host-core/src/prepare.rs:3596`, `:3683`). -- **This slice is now the production effect carry.** Since #1555 and #1381a, both hosts attach - `controls::live_lanes(model)` on every plan, and `LiveLanes::effects` is true for any session with - an effect that has a live row. So every effect lane of a production plan is a - `LiveControlEffectBankStage` lane or a per-node `LiveControlEffect`. #1279's carry - (`EffectBankStage` only) is reached only by test entries that attach no effect lanes. Today every - structural transaction on either host restarts each strip that holds a console slot or an insert. -- **D1 for decision 17.** A difference is live exactly when the per-instance function of D4 returns - records for it, and prepared when it refuses it (a rebuild-only row: the delay and multiband - bypass, a `Sample`-rate parameter, the link mode; a structure change; a domain error). The join - calls that one function, as the strip join calls `fader_records` (#1277 D5). `same_console_entry` - (`prepare.rs:3550`) stays for the non-live stage only. A move also needs the same stage kind in - both plans (D1's "in both or in neither"): `GraphEffectLane` and `EffectLaneInventoryRow` - (`prepare.rs:723`) gain the kind. -- **Banked inserts are in this slice.** No slice carries an insert that banks in both plans: #1281 - is bank to per-node, #1282 is per-node to per-node, and `resolve_effect_carry` joins console - owners only. `bank_effect_lanes` already lists banked insert lanes under `RackId::Dynamic`, and - gate 4 uses a banked insert. So the join adds the owner key (strip, `Dynamic`, insert ID) for a - surviving insert that is banked in both plans. -- **D2 gains #1341 D7.** `carry_from` also copies `mix` and `mix_position` (#1341a1, landed in batch - 9), so a swap inside a bypass crossfade continues the ramp. #1341's gate 6 runs here. -- **D2, where the carried slots resolve.** `stage` returns early when no dirty bit is set - (`live.rs:1070`). The carried slots resolve before that return. Carried parameter and target - words go through the same per-group `applied` arrays as cell reads, so the #1004 pairing rule - and the witness admit them exactly as cell reads. -- **D2, the count.** Counting a carried value needs one engine method, which the owner approved on - 2026-10-14 (one attempt; if the permission system refuses it, stop and report to root): - `pub fn add_carried(&self, count: u64)` on `LiveValuesSuperseded` - (`crates/engine/src/realtime/latest_cell.rs:946`), in the realtime-policy region beside the - private `add` (`:982`), with the same body (load, saturating add, store; zero changes nothing). - The doc of `add` changes to name it as the second render path that writes the count. It is the - only `crates/engine/src/realtime` edit of this issue. Its unit test goes in `latest_cell.rs`'s - tests. -- **D2, keys after #1603.** A cell slot is keyed by publication (`GateKey`), and a monitor tick - publishes the committed revision again under a new publication. "Every predecessor value is - `<= S`" reads "every predecessor write was published on the predecessor's gate before the - successor was published". The carried slot resolves with no key compare across the two gates, - so the rule does not change. -- **D4, the write.** The retarget writes through the successor's `EffectControlProducer` with - `PreparedRenderPlan::stamp_held_gate(base.revision)` (`crates/engine/src/realtime/plan.rs:813`), - in the block that writes the strip retargets (`prepare.rs:2644`). A bypass goes as a - `BypassSwitch` with the session's `mute_ms` ramp (#1341 D1): `write_bypass` with - `bypass_ramp_samples` until #1622 folds the ramp into the record. Parameter ramps come from - `SuccessorBase::edit_ramps` (#1394) where the edit gave one. -- **The live-stage carry.** `LiveControlEffectBankStage` gains `layout()`, `can_carry_lane_from` and - `carry_lane_from`: the payload through the plan's scratch as `EffectBankStage` does, then - `EffectControlLane::carry_from`, then the shunt words (D5), then the lane's `designed` flag - recomputed after the restore, as `EffectBankStage` refreshes its witness. `effect_stage_mut` and - `effect_lane_mut` (`runtime.rs:4319`, `:4328`) reach both stage types. -- **Split (replaces the decision-17 split; root confirms).** Each piece stays in this issue. - - **#1280a:** D2 (with #1341 D7), D5, the live-stage carry, the join for console slots and banked - inserts, and `add_carried`. A live difference is treated as prepared until #1280c: the strip - restarts, as today, and no value is lost, because the successor is prepared from the committed - model. Gates 1, 2, 3, 4, 6 and #1341 gate 6. - - **#1280c:** D4 (the extraction and the retarget). Gate 5. - - **#1280b:** D3 (pending Observe records). Gate 8. - - Reason: D5 moves into (a) because a carried crossfade on a latency effect reads the dry line - in its first blocks after the swap, so (a) without D5 is not exact for #1341 gate 6 with the - limiter. D4 can stand apart, because before it a live difference restarts the strip, which is - today's behaviour. -- **Gate 7** gains `cargo test --locked -p engine` (the `add_carried` test). In gates 1, 3 and 5, - "write to A's cells" means through A's `EffectControlProducer`, stamped on A's gate: the control - plane is the only writer in production (decision 17). -- **Dependencies.** #1279, #1312 and #1345 are closed. #1341a1 and #1341a2 landed in batch 9. - #1622 (open) edits the same bypass write in `live.rs` and `live_delta.rs`: if both are in one - batch, #1622 lands before #1280c. - -## Ruling (root, 2026-10-14) - -The batch-10 split is approved: #1280a (D2 with #1341 D7, D5, the live-stage carry, the console and -banked-insert join, `add_carried`; gates 1-4, 6 and #1341's gate 6), #1280c (D4, gate 5), #1280b -(D3, gate 8). #1341 closes when #1280a lands. #1622 follows #1280a in batch 10 lane K and rebases -on it. - -## Decision record: #1280a (batch 10) - -**Shipped.** D2 with #1341 D7, D5, the live-stage carry, the join for console slots and banked -inserts, and `add_carried`. - -- `LiveValuesSuperseded::add_carried` (`crates/engine/src/realtime/latest_cell.rs`): the same body as - `add`, public, in the realtime-policy region. One attempt, accepted by the permission system. Its - unit test is `add_carried_adds_saturates_and_ignores_zero`. -- `EffectControlLane::carry_from` (`effect-contract/src/live.rs`) copies `bypass`, `mix`, - `mix_position` and `symmetry`, and gives each cell a render-owned carried slot (`CarriedCell`: - the newest unread words and the unread count `s - p`). `stage` resolves the slots before its - `any == 0` early return, in `Reads::read`: the successor's own `Applied` read wins and adds `n`; - otherwise the carried words apply and add `n - 1`. A slot the predecessor had not resolved yet is - merged (its count is added), so a successor that is replaced before its first stage loses nothing. - `can_carry_from` compares the two cell shapes (`EffectLaneShape` equality). The slots are - preallocated with the lane and counted in `retained_resources`. -- `BypassShunt::carry_line_lane_from` (D5): one lane's samples of the interleaved line, walked from - each ring's own cursor in age order, so another width and another cursor are correct. It copies - nothing when neither bank keeps a latency line, and refuses a line of another latency. -- `LiveControlEffectBankStage` gains `layout`, `lanes`, `lane_kind`, `can_carry_lane_from`, - `carry_lane_from`, `carry_refused`, `as_any_mut`, and a `RESTORED` bit per lane (a byte, no new - allocation). The payload move is one function (`carry_payload`) shared with `EffectBankStage`. - `EffectLaneKind { Plain, Uncontrolled, BypassOnly, Cells }` is the "kind" of the join: a pair of - unlike kinds does not carry. -- `graph`: `EffectStageMut` reaches both stage types in `effect_stage_mut` / `effect_lane_mut`; - `GraphEffectLane` gains `kind`; the swap block's check and carry run for both. -- `host-core`: `EffectLaneInventoryRow` gains `kind`. The join adds the owner key (strip, `Dynamic`, - insert ID) for a surviving insert (`InsertOwnerJoin`, bit-equal declaration in both models). A - live difference is a prepared one until #1280c: the strip restarts, as today. - -**Departures.** - -- The live stage had no `RESTORED` witness term. A restore whose left and - right sections differ must decline the mono collapse as `EffectBankStage` does, so the live - stage holds the term now (not in the text of the issue). -- A lane kind that is not `Cells` carries its payload (and a lowered bypass) and no cells; the - swap block never refuses the hand-over for a shunt that exists on one side only. -- Gate 3's count is per cell. The compressor threshold is a `PerLane` parameter with a left and a - right cell, so the test writes both and the counts are 2 x 79 and 2 x 39. - -**Gates and tests** (`crates/host-core/tests/successor_swap.rs`): - -| Gate | Test | -|---|---| -| 1 | `unrendered_live_writes_survive_the_swap_at_{eight,four}_lanes` (EQ band gain target and compressor threshold) | -| 2 | `a_live_ramp_in_flight_finishes_after_the_swap_at_{eight,four}_lanes` (quantum 32, 64-sample ramp half done) | -| 3 | `the_newest_value_wins_and_the_count_is_exact_at_{eight,four}_lanes` (158 and 78, i.e. 79 and 39 per cell) | -| 4 | `a_bypassed_lane_keeps_its_delayed_dry_signal_at_{eight,four}_lanes` (banked limiter insert, nonzero cursor) | -| 6 | `assert_clean_swap` in every case above: allocator and render audit zero over the swap block | -| #1341 6 | `a_bypass_crossfade_in_flight_continues_after_the_swap_at_{eight,four}_lanes` (comp, and limiter with latency) | -| join | `a_changed_control_kind_carries_no_effect_lane_at_{eight,four}_lanes` | -| engine | `add_carried_adds_saturates_and_ignores_zero` | -| shunt | `carry_tests::a_shunt_lane_moves_to_a_bank_of_another_width_and_cursor` | - -Red under mutation (each run once, reverted): no cell carry (gate 1), no `add_carried` and a -carried value that overrides the successor's (gate 3), no shunt copy and a copy that ignores the -cursors (gate 4 and #1341 6), no `mix` copy (#1341 6). The four-lane cases are `ignore`d on this -AVX2 host (a console bank binds at the width its build executes, #1017). -`a_console_slot_with_live_controls_restarts_its_strip` is deleted: a live console slot now carries. - -**Found, not fixed (successors).** - -- A route's PDC delay line is a state of its own that no carry moves. A session where one strip has - an insert with latency and another has none gets a delay line on the second strip's route, and a - swap restarts it: the first limiter-insert case failed on exactly that until every track had the - insert. The gates use sessions without unequal latency. -- Pending Observe records of a carried lane are still lost at the swap (D3, #1280b). -- `cargo run -p realtime-policy` fails at the base on `hosts/host-web/tests/control_command_heap.rs` - (`unsafe` outside the approved files); it is not from this slice. - -## Decision record: #1280c (batch 10) - -**Shipped.** D4 (the extraction and the retarget) and gate 5. After this slice a live difference on a -carried effect lane no longer restarts the strip: the lane carries, then the successor retargets it -to the new committed value. - -- **One per-instance function.** `effect_instance_records` (`host-core/src/prepare.rs`, beside the - strip derivation it mirrors) holds what `live_delta.rs` had in `effect_records`'s body, - `parameter_records` and `design_targets`, and `same_params`; `live_delta.rs` keeps `locate` and - `effect_records`, which now only adds the live address. It returns `Ok(None)` for no difference, - `Ok(Some(EffectInstanceRecords))` for a live one and `Err(LiveRebuild)` for a prepared one - (another identity, quality, sidechain or link mode; a bypass of an effect that does not lower it, - by `lowers_session_bypass`; a parameter that is not `Block`-rate; a parameter that does not - resolve; a ramp the indexed law refuses). The join calls it per surviving instance through - `join_chains`, so a difference is live in the carry exactly when it is live on the live path. - `same_console_entry`, `same_insert` and `committed_chain` are deleted: the lowered pair - (`SessionModel::lower_strip`) of both models is compared by that function. -- **Join states.** `ConsoleOwnerJoin` and `InsertOwnerJoin` hold `FamilyJoin` - and the instance's `LiveEffectAddress`. A live difference of a slot or insert at another position - is prepared (another address); a `Same` one still carries. -- **The write.** `resolve_effect_carry` takes the producer step as a callback, called only for a - lane whose layout and kind already match: `prepare_effect_retarget` preflights every record on the - successor's own producer (a record the lane has no cell for refuses the lane, and the strip - restarts), and, for the EQ, begins the owner transaction, applies the edits and preflights the - targets. The writes happen after `stamp_held_gate(base.revision)`, in the block that writes the - strip retargets: `write_bypass` with a `BypassSwitch` at the session's `mute_ms` ramp - (`bypass_ramp_samples`), `write` for a parameter record, `publish_candidate_targets` and - `commit_owner` for the EQ. Nothing is written when preparation fails before that block. - -**Departures.** - -- Parameter ramps from `SuccessorBase::edit_ramps`: an effect parameter record has no ramp (the - effect smooths it itself) and `EditRampRow` has no effect-parameter row, so only the bypass has a - length, and it is the session's. Nothing to read from `edit_ramps` here. -- The test gate's successor helper (`LivePlan::successor`) now takes the commit's gate steps (stamp - the revision again, publish it), as `CellPlan::successor_at` already did. Without them a retarget - stamped at preparation is never published on the successor's gate. - -**Gate and tests** (`crates/host-core/tests/successor_swap.rs`, `crates/host-core/src/prepare.rs`): - -| Gate | Test | -|---|---| -| 5 | `a_live_difference_on_a_carried_lane_is_retargeted_at_{eight,four}_lanes`: a compressor threshold, an EQ band gain and a compressor bypass, each on three carried lanes. Run 1 is the structural swap with the retarget, run 2 writes the same edit to A before the swap and prepares B from a base that holds it. Bit-identical in every block, `restarted` empty, a clean swap block (zero allocations and frees), and not vacuous against a swap that leaves the lane at its old value. | -| join | `the_per_instance_function_separates_live_from_prepared`, `a_live_difference_carries_only_when_the_successor_accepts_its_records` | -| classifier | `a_changed_effect_row_without_a_record_is_a_rebuild` (unchanged, through the moved function) | - -The four-lane case is `ignore`d on this AVX2 host (#1017), as the #1280a cases are. - -**Found, then fixed in the batch 10 review fold.** `ChainJoin::may_carry` ignored surviving -inserts. The carry program is built only when some owner may carry, and `install_effect_carry` -refuses a plan without one (`NoEffectCarryProgram`, surfaced as `host.carry.install`), so the -earlier line here that nothing was wrong was wrong: a transaction whose only carryable owners were -inserts would have been refused. See "Fold (batch 10 review)" below. - -### `audit capi` digest (batch 10) - -The C ABI render audit commits one structural transaction after its first call (it adds a muted track, `tools/audit/src/capi.rs`). Before #1280a that swap restarted every strip with a console EQ slot; now the slots carry their state across it. So `audit capi`'s `pcm_digest` moves from `76525e7ef9e78cf4` (every batch through 9) to `9bc5ec3d8a4c223c`, measured at `f0e387307` and its parent `869ffb78b`; allocations, frees and render errors stay 0. CI checks the digest's format only. - -## Decision record: #1280b (batch 10) - -*Superseded in part: the `apply_pending_observations` / `settle_observations` description below is replaced by the "Fold (root ruling B, batch 10)" section, and the stage's `next_block_sample` by the "Fold (batch 10 review)" section.* - -**Shipped.** D3 (pending Observe records) and gate 8. (D5 shipped in #1280a.) - -- `LiveControlEffectBankStage::apply_pending_observations(lane) -> u64` (`crates/rack/src/lib.rs`): - pops lane `lane`'s observation queue to the count it held at entry and applies each record to the - lane's `ObservationLane` (`apply_observe`), at the first sample of the block the stage would have - rendered next. It has the same counted-drain shape as `drain` and counts a record with no tap in - `unbound_observations` as `drain` does. The two loops are written twice, not shared: the - realtime-policy call rule refuses a call to a popping function from inside a marked loop. -- `Runtime::carry_effect_lanes` (`crates/graph/src/runtime.rs`) calls it on the predecessor's lane - through `EffectStageMut::settle_observations` (a stage with no control lane has no queue) right - after each lane's hand-over, so it runs for every moved effect lane. Bounded by the queue's count - at entry and by the number of moves; allocation-, lock- and syscall-free. The wrapper is named - apart from the popping method on purpose: the carry's move loop may not call a popping function - by name, and `begin_block` over `drain` is the same two-level shape (the tool's stated limit of a - helper two calls deep). Root may want to rule on whether that shape is the intended one. -- The sample: `adopt_predecessor` gets no clock, and the engine trait is outside this slice (no edit - under `crates/engine/src/realtime`). The stage keeps `next_block_sample`, the first sample of the - block after the last one it drained (`first_sample + quantum`, one saturating add in `drain`), and - the carry applies at it. A stage that never drained a block applies at sample 0. - -**Departures.** - -- Every shipped compressor, gate, multiband and limiter declares one tap, so a queue's capacity is - one record. "The queue filled to its capacity" is therefore every lane's queue full (a further push - is refused), and "one subscribe" is one lane's queue holding a record while the others are empty. -- The tap state is observed by the armed count and by the window a further block of A closes: the - window opens at the sample A's next block starts at, equal to the reference's. - -**Gate 8** (`crates/host-core/tests/successor_swap.rs`, compressor insert on a full bank with live -controls and one observation tap): -`a_pending_subscribe_is_applied_at_the_carry_at_{eight,four}_lanes` and -`full_observe_queues_are_applied_at_the_carry_at_{eight,four}_lanes`. Each admits the records after -A's last block, hands over, and checks: zero allocations, reallocations and frees on the swap block -(allocator and render audit), no restart, no refused restore; every retired queue reads empty; A's -armed-tap count equals the reference's (the same session that rendered the swap block itself with -the same records); and after one more block of A the published windows (`first_sample`, -`end_sample`, `blocks`) equal the reference's, subscribed lanes only. Red under mutation (each run -once, reverted): no call in the carry (queues not empty), and the carry applying at sample 0 (window -opens at 0 instead of 768). The four-lane cases are `ignore`d on this AVX2 host (#1017). - -**Found, not fixed.** The tap state A now holds is not carried to B: that is the observer carry -(#1327). Until it lands, an acknowledged subscribe armed on A's retired tap is not armed on B. - -## Fold (root ruling B, batch 10) - -Root ruled that the `settle_observations` wrappers of #1280b put the pop two calls deep, which the realtime policy does not see, and that this routes around the policy. Rewritten so the policy checks the pop directly: - -- `Runtime::carry_effect_lanes` (`crates/graph/src/runtime.rs`) no longer drains in its move loop. After that loop, a second marked loop (`for step in moves.iter()`) resolves each predecessor site and pops the queue itself: `let mut available = queue.available_at_entry(); while available != 0 { available -= 1; queue.try_pop() .. }`, the count read where the loop is, one pass per move. Each record goes through `effect_contract::apply_observe` at the sample as before (a bank stage's own `next_block_sample`, a per-node instance's `predecessor_next_sample`). -- Deleted: `LiveControlEffectBankStage::apply_pending_observations` (the duplicate of `drain`'s loop), `EffectStageMut::settle_observations` and `LiveControlEffect::settle_observations`. `drain` and the per-node `drain_observations` stay, one per caller, each with its bound at its loop. One function cannot serve `drain` and the carry: `drain` walks every queue of the stage, the carry one lane. -- New, non-popping accessors that hand the carry the parts: `rack::ObserveTarget { queue, tap, first_sample, unbound }`, `LiveControlEffectBankStage::observe_target_mut(lane)`, `EffectControlLane::observe_queue_mut()` (effect-contract), and in graph `observe_target_at` over `EffectStageMut::into_observe_target` and `LiveControlEffect::observe_target_mut`. A record with no tap still counts in `unbound_observations` of a bank stage. -- Departure: no new "pending lanes" table. The carry program `moves` is the plan-sized table, it does not change between the two loops, and the second loop reads the queue's count and the sample from the instance itself. So there is no new storage and the resource estimate does not move. One behavior difference: the old code settled only a move whose successor resolved; the new loop settles every move whose predecessor resolves (the predecessor never renders again either way). `can_carry_effect_lanes` already refuses a program whose successor does not resolve, so no reachable case differs. -- Policy note: the loop iterates `moves.iter()`, not `moves`. The slice-parameter form requires that nothing names the parameter earlier in the function body, and the first loop does. `.iter()` is the same finite collection form. With `available -= 1;` removed the tool refuses the loop at `runtime.rs` (`marked realtime unbounded try_pop drain`), so the pop is judged here. - -## Fold (batch 10 review) - -- `ChainJoin::may_carry` (`crates/host-core/src/prepare.rs`) now also counts a surviving insert that is not prepared, so a strip whose only carryable owner is an insert builds the carry program that `install_effect_carry` requires. Unit test `a_surviving_insert_alone_makes_a_chain_join_carry`. Not reproducible through a swap today: a strip present in both models also joins its fader and matrix, which are `Same` or `Live` (never prepared) whenever live controls attach, and that alone builds the program. So the join, not a swap, owns the check, and the earlier "nothing is wrong today" line is corrected above. -- One clock for the carry's Observe drain. `LiveControlEffectBankStage::next_block_sample` (a per-block store that duplicated `GraphExecutor::next_block_sample`) is deleted, and so is `rack::ObserveTarget::first_sample`. `Runtime::carry_effect_lanes` applies every retired queue's records at `predecessor_next_sample`, for a bank lane and a per-node instance alike. A stage that skipped a block's drain can no longer hold a stale clock. The #1280b window-sample assertions stay green. -- The per-node instance keeps `unbound: None` in its `ObserveTarget`: the control plane refuses a subscription the plan cannot hold before it reaches a queue, so nothing is unbound there, and a counter nothing reads would be dead state. (A first attempt at this fold added one and was reverted.) diff --git a/.github/ISSUE_SPECS/1281-carry-an-insert-lane-that-moves-between-a-bank-and-a-per-node-instance.md b/.github/ISSUE_SPECS/1281-carry-an-insert-lane-that-moves-between-a-bank-and-a-per-node-instance.md deleted file mode 100644 index e6636ffce..000000000 --- a/.github/ISSUE_SPECS/1281-carry-an-insert-lane-that-moves-between-a-bank-and-a-per-node-instance.md +++ /dev/null @@ -1,214 +0,0 @@ -# Carry an insert lane that moves between a bank and a per-node instance - -Stream A of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-7, D15-8). -Slice 11b of *Swap a rebuilt plan without an audio gap* (#1269). Code anchors verified on `main` at -`6fb211594`. - -## Product outcome - -An insert whose lane changes representation at a swap keeps its exact state. Adding the eighth -compressor insert makes a group of seven per-node instances bank, and removing one makes a full bank -fall apart. Both directions continue bit for bit, live state included. The carry moves the state at -the swap block, and the predecessor never renders again. - -## Note (root, 2026-10-12, stream B batch-3 verdict) - -After #1345 a bank lane's observation queue consumer lives in -`LiveControlEffectBankStage::observe_queues` (`crates/rack/src/lib.rs` ~1162), taken out of its -`EffectControlLane` at bind. A per-node `LiveControlEffect` keeps its consumer in its -`EffectControlLane` (`crates/graph/src/runtime.rs` ~1230, drained by `drain_observations`). A -lane's move between a bank and a per-node instance therefore moves the consumer between those two -homes as well as the cells; #1280's D3 states the carry rule. - -## Context - -- An insert group banks only when it is full. The planner pads only console groups (`pads`, - `crates/graph-compiler/src/banks.rs:76`). So at 8 lanes, seven tracks with the same insert render - per node and eight bank; at 4 lanes, three and four. #888 and #889 (open) would bank partial - groups. Whichever lands later keeps this slice's gates. -- A per-node insert is `NodeKind::Effect(GraphPreparedEffect)` or `NodeKind::LiveControlEffect` - (`crates/graph/src/runtime.rs:1136`, `:1143`; struct `LiveControlEffect` `:1184`). Both hold a - `GraphPreparedEffect` (`crates/graph/src/lib.rs:881`) with `processor: Box` (`:884`). A bank lane is a lane of an `EffectBankStage` or a - `LiveControlEffectBankStage` (`crates/rack/src/lib.rs:750`, `:937`). -- Per-node and bank payloads share one layout: `snapshot_state_payload` and `restore_state_payload` - (`crates/effect-contract/src/lib.rs:1930`, `:1975`) and the bank pair (`:2070`, `:2079`). Both - are render-safe and exact mid-ramp (#1278, on `main`). Both snapshots take `&self`. -- #1069 (*Multiband compressor: a ramp's cut moves a lane's bits in a bank*, open): a multiband lane - with a ramp in flight is not bit-identical between a bank and a per-node instance until #1069 - lands. The gates use the compressor and the limiter. -- The effect rule, the retarget, the carried cells and the shunt copy come from *Carry console - effect lanes across a plan swap* (#1279, D1) and *Carry live-controlled effect lanes across a - plan swap* (#1280, D2-D5). - -## Decisions frozen for this slice - -- **D1. Rule.** As #1279 D1, and #1280 D1 for live-controlled owners. Live differences retarget as - #1280 D4. Prepared differences restart the owner and add its strip to the restart set (#1277 D6). -- **D2. Move mode.** The state goes through the payload scratch in one of two directions: - - per-node `snapshot_state_payload` → bank `restore_track_state_payload`; - - bank `snapshot_track_state_payload` → per-node `restore_state_payload`. - - The control state follows #1280 D2's `carry_from`. The shunt words move between the per-node - `BypassShunt` and the lane's words of the bank line, relative to the bank's shared cursor. -- **D3. A collapsed bank is desymmetrized first.** Before a bank snapshot, a collapsed predecessor - bank is desymmetrized, once per bank (`desymmetrize_channels`, - `crates/effect-contract/src/lib.rs:2173-2182`). This is bit-neutral: it writes into the right - channel the state a dual run would hold, and collapse never changes bits. So the bank snapshot is - exactly the payload a dual run would write, and the per-node restore continues it bit for bit - (#1279 D4 states the same rule for bank-to-bank lanes). - -## Deliverables - -1. D2-D3 in `crates/graph` and `crates/rack`, and the join in `crates/host-core`. -2. Gap-free tests in `crates/host-core/tests/successor_swap.rs`. - -## Authorized paths - -- `crates/rack/src/lib.rs` -- `crates/graph/src/lib.rs`, `crates/graph/src/runtime.rs` -- `crates/host-core/src/prepare.rs`, `crates/host-core/tests/successor_swap.rs`, - `crates/host-core/tests/support/successor.rs` - -## Non-goals - -- No per-node to per-node move (#1282). -- No multiband gate before #1069. - -## Objective gates - -1. **A group becomes a bank, both widths.** At `Backend::Simd8`, seven tracks carry a compressor - insert (per node), and B adds an eighth, muted, so the group banks. Every block is bit-identical - to the reference. At `Backend::Simd4`, the same with three and four tracks. Repeat with the - limiter insert. -2. **A bank falls apart.** The reverse edit (remove one track from a full group) is bit-identical. -3. **Live state.** Gate 1 with live controls and a compressor value unread in A's cell at the swap - is bit-identical to the reference fed the same value. A second case also changes the threshold of - a lane that changes representation (a live value): it is bit-identical to "live edit, then - structural edit" (#1280 gate 5's shape). -4. **Realtime.** Each swap block makes zero allocations and frees. -5. Commands: - - `cargo test --locked -p rack -p graph -p host-core --features rack/test-support,graph/test-support,host-core/test-support` - - `cargo build --locked --release -p audit && bash scripts/trace-graph-audit.sh target/release/audit` - - `cargo run --locked --release -q -p realtime-policy`, `bash scripts/check-workspace-policy.sh`, - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings` - -## Test value - -- Gate 1: a carry that handles only bank-to-bank leaves the new bank's lanes at rest. It turns red. -- Gate 2: the reverse direction is a separate code path (a bank snapshot into a per-node restore). - A carry that handles only gate 1's direction turns it red. -- Gate 3: control state lost across a representation change, or a retarget routed to the old - representation's producer, turns it red. - -## Dependencies - -- *Carry live-controlled effect lanes across a plan swap* (#1280). - -## Amendment (controls coordinator, batch 10, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Anchors moved.** `NodeKind::LiveControlEffect` is `crates/graph/src/runtime.rs:1161`, struct - `LiveControlEffect` `:1228`; `GraphPreparedEffect` is `crates/graph/src/lib.rs:909`; the per-node - payload pair is `crates/effect-contract/src/lib.rs:2325` and `:2370`, the bank pair `:2468` and - `:2483`; `desymmetrize_channels` is the trait method at `:2588`; `pads` is still - `crates/graph-compiler/src/banks.rs:76`. `LiveControlEffectBankStage` is - `crates/rack/src/lib.rs:1154`. #888, #889 and #1069 are still open, so the Context and gate - choices stand. -- **Every production lane is live-controlled** (see #1280's batch 10 amendment): a bank lane is a - `LiveControlEffectBankStage` lane and a per-node insert is a `LiveControlEffect`. The non-live - forms are reached only by test entries. The gates use the live entries. -- **D3.** Use `BankChain::disengage_for_carry`, once per predecessor chain, as - `Runtime::carry_effect_lanes` does (`runtime.rs:4726`), not a direct `desymmetrize_channels` - call: the chain owns the collapse state. -- **The observation consumers do not move.** Each plan keeps its own queue consumers (a bank's in - `observe_queues`, a per-node lane's in its `EffectControlLane`). The control plane writes only the - newest plan's queue, and #1280b applies the predecessor's pending records to its - `ObservationLane` at the carry; #1327b then moves the tap state. So the Note's "moves the - consumer" reads "applies the pending records, then moves the tap state". -- **Live retarget.** D1's "live differences retarget as #1280 D4" is #1280c (the per-instance - function). Gate 3's second case needs it. -- **Shunt copy.** The per-node `BypassShunt` holds `dry_*`, `line_*` and one `cursor`; a bank's line - is interleaved by lane. The copy helper of #1280a (D5) serves both directions; if it does not, - this slice adds a `BypassShunt` lane-word copy in `crates/effect-contract/src/live.rs` (added to - the authorized paths for that helper only). -- **Dependencies.** #1280a and #1280c (gate 3's second case). Not #1280b. - -## Decision record: #1281 (batch 10) - -**Shipped.** An insert instance that changes representation at a swap (a per-node live-controlled -effect to a lane of a `LiveControlEffectBankStage`, and back) carries its exact state, its control -lane (bypass, crossfade, symmetry terms, unread cells), its shunt words and its pending Observe -records. D1 is the one per-instance function of #1280c: a live difference retargets the moved -instance, a prepared one restarts the strip. - -- `rack`: `EffectInstance` / `EffectInstanceMut` (processor, control lane, dry shunt of a per-node - instance), `EffectLaneKind::of_instance`, and on `LiveControlEffectBankStage` - `can_carry_instance`, `carry_lane_from_instance` (per node to bank: the instance's - `snapshot_state_payload` into the bank's `restore_track_state_payload`, `carry_from`, the shunt - line as lane `lane` of `lanes`, then `designed` and the `RESTORED` bit as for a lane carry) and - `carry_lane_into_instance` (bank to per node, `&self`: the bank's - `snapshot_track_state_payload` into `restore_state_payload`, `carry_from`, the shunt words as - lane 0 of 1). D5 needed no new helper: `BypassShunt::carry_line_lane_from` serves a per-node - shunt as a bank of one lane. -- `graph`: `GraphEffectSite { Lane(GraphLaneLocation), Node(unit) }` replaces the location of - `GraphEffectMove` and `GraphEffectLane`; `bank_effect_lanes` also lists each - `NodeKind::LiveControlEffect` (layout `None`); `Runtime::{has_effect_site, can_carry_effect_lanes, - carry_effect_lanes}` handle the four site pairs (node to node is refused at install, - `EffectMoveBetweenNodes`, and is #1282). D3: a bank predecessor is disengaged once per chain by - `BankChain::disengage_for_carry` before any snapshot, as the lane carry does; a per-node - predecessor has no collapse. `install_effect_carry` takes the joined scratch size - (`scratch_bytes`), since a per-node instance has no layout in a bound plan. -- `host-core`: `EffectLaneInventoryRow.location` is a `GraphEffectSite`. A per-node row takes its - layout from the prepared entries (`InstanceLayout`, `EffectLaneLayout::of(program_key())`), so it - compares equal to a bank row exactly when the payloads match. `resolve_effect_carry` still - restarts a strip when both sides of an owner are per-node (#1282). - -**Departures.** - -- Pending Observe records of a per-node predecessor are applied at the carry too (the amendment - says the consumers do not move; applying them is #1280b's rule, so both directions follow it). - A per-node instance has no clock, and a field on `LiveControlEffect` would break the estimate's - field sum (`graph-compiler/src/estimate.rs`), so `GraphExecutor` keeps `next_block_sample` (the - first sample after its last rendered block) and passes it to the carry. -- A per-node successor that refuses a restore has no stage to count it, so `GraphExecutor` keeps - `node_carry_refused`, added to `carry_refusal_count`. -- Only the live forms move across representations: a per-node `NodeKind::Effect` (no control lane) - has no row and a plain `EffectBankStage` pairs with no node. Both are reached only by test - entries (amendment, batch 10). A strip with such an insert restarts, as before. -- `LiveControlEffectBankStage::carry_lane_into_instance` takes `&self`; the refusal is counted by - the caller. - -**Gates and tests** (`crates/host-core/tests/successor_swap.rs`; the four-lane cases are `ignore`d -on this AVX2 host, #1017): - -| Gate | Test | -|---|---| -| 1 (per node to bank, compressor and limiter) | `a_group_of_inserts_becomes_a_bank_at_{eight,four}_lanes` | -| 2 (bank to per node) | `a_bank_of_inserts_falls_apart_at_{eight,four}_lanes` | -| 3 (unread value; live difference on moved lanes, both directions) | `live_values_cross_the_form_change_at_{eight,four}_lanes` | -| 4 (zero allocations and frees on the swap block) | `assert_clean_swap` in every case above | -| Observe records of a moved lane, both directions | `pending_observe_records_cross_the_form_change_at_eight_lanes` (the #1280b test, parameterised by session pair) | - -Each form test first checks that A is per node and B banked (or the reverse) through -`bank_effect_lanes`, and that a swap whose strips restart renders other bits. Results: -`cargo test --locked --all-features` for `host-core` (422), `effect-contract` (87), `rack` (67), -`graph` (103), `graph-compiler` (178) and `capi` (153), and `host-web --features -host-web/test-support` (127), all green; clippy `-D warnings` and fmt clean; -`realtime-policy` ok. - -## Fold (batch 10 gates): the per-node layout comes from the bound plan - -The browser boot-budget gate failed on the batch 10 integration branch: wasm high-water growth -21,102,592 > 21,037,056 (pin 20 MiB plus one page). #1280a's carried slots put growth at exactly -20,971,520; the compile-time `instance_layouts` table this issue added in `host-core` (one 88-byte -row with two `Box` names for each of the 768 effect entries, alive through the compile peak) -cost two more pages, and serves only per-node instances. The fold deletes the table (and -`InstanceLayout`, `instance_layout()`). `graph`'s per-node `LiveControlEffect` now keeps a `layout: -rack::EffectLaneLayout` (`Copy`) read off the effect metadata's program key in -`LiveControlEffect::new`; `bank_effect_lanes` returns it, `GraphEffectLane.layout` is a plain -`EffectLaneLayout` again (not `Option`), and the inventory row takes `lane.layout`. The -graph-compiler per-node live-control owner charge (`estimate.rs`) and its independent test sum add -`size_of::()`, so retained == charged in every `bypass_resources` case. -Measured growth: 21,102,592 -> 20,971,520 (back to the #1280a figure; the gate passes). diff --git a/.github/ISSUE_SPECS/1282-carry-per-node-effect-instances-across-a-plan-swap.md b/.github/ISSUE_SPECS/1282-carry-per-node-effect-instances-across-a-plan-swap.md deleted file mode 100644 index 3e19997f4..000000000 --- a/.github/ISSUE_SPECS/1282-carry-per-node-effect-instances-across-a-plan-swap.md +++ /dev/null @@ -1,202 +0,0 @@ -# Carry per-node effect instances across a plan swap - -Stream A of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-7, D15-8). -Slice 12 of *Swap a rebuilt plan without an audio gap* (#1269). Code anchors verified on `main` at -`6fb211594`. - -## Product outcome - -An effect that renders per node in both plans, and whose prepared values are unchanged, keeps its -exact state through a plan swap. Examples are a delay's echoes and feedback tail, a multiband -compressor, and any insert the planner leaves unbanked in both plans. Its live state and unread live -values come along, so no acknowledged edit is lost. The carry moves the instance by pointer at the -swap block, and the predecessor never renders again. It costs a few words per effect, whatever the -state size; a delay holds two seconds per channel. There is no payload scratch and no copy. A -latency-growth swap (*Grow latency during playback by adopting a primed warm successor*, #1287) -adopts in move mode at its first ready block too, so it moves the instance the same way. - -## Context - -- A per-node effect is `NodeKind::Effect(GraphPreparedEffect)` or - `NodeKind::LiveControlEffect(Box)` (`crates/graph/src/runtime.rs:1136`, - `:1143`). - - `GraphPreparedEffect` (`crates/graph/src/lib.rs:881-891`) carries `id: EffectNodeId`, - `metadata`, `processor: Box` (`:884`), `response_snapshot_declared` - and `native_id`. The state lives behind the `processor` box, so swapping two boxes moves the - whole state by pointer. - - `LiveControlEffect` (`runtime.rs:1184-1197`) adds its `EffectControlLane`, a span window, a - `BypassShunt` and optional observation. Its records are applied inside `execute_op` (`:3510`). -- The delay never banks (`NEVER_BANKED_EFFECTS`, `crates/effect-compiler/src/prepare.rs:244`), and - the multiband keeps a prepared bypass (`PREPARED_BYPASS_EFFECTS`, `:260`; `lowers_session_bypass`, - `:266`). Their session bypass sits inside the processor. A live bypass is the lane's `bypass` flag - and the shunt. -- The per-node payload pair is render-safe and exact mid-ramp for every effect, the delay included - since `e0e4e8a20` (#1278 D2a). A payload copy of a delay at 96 kHz would still pass about 1.5 MiB - through a scratch buffer twice. -- The effect rule, the retarget and the carried control state come from *Carry live-controlled - effect lanes across a plan swap* (#1280). The bank and per-node flips come from #1281. - -## Decisions frozen for this slice - -- **D1. Rule.** As #1279 D1. The inventory's prepared layout includes the processor's prepared - bypass. - - A committed bypass change on the delay or the multiband is a prepared change. The owner - restarts and its strip goes into the restart set, so *Duck-swap a strip whose state cannot - continue across a plan swap* (#1324) gives it a transition (D15-13 E4). - - When *Give the delay a live bypass shunt* (#1339) and *Give the multiband compressor a live - bypass shunt* (#1340) land, that bypass becomes live and retargets under #1280 D4. They rewrite - gate 3. - - For an effect whose bypass is lowered, a live bypass is lane state and carries. -- **D2. Move mode.** When both plans hold the owner per node, swap their `processor` boxes - (`core::mem::swap`). For a `LiveControlEffect`, also swap the `BypassShunt` (same frames and - latency by D1). The successor's lane takes the predecessor's control state with - `EffectControlLane::carry_from` (#1280 D2), so unread live values render in the successor. No - payload, no copy, no allocation. The predecessor's plan then holds the successor's at-rest box, - and it retires and is reclaimed off the render thread with that plan, as today. No - `PreparedNativeEffect` method is added: the move needs none. -- **D3. Mixed wrappers.** If one plan has live controls on the owner and the other does not (no - host does this today), the owner does not carry, and the join records it in the restart set. - -## Deliverables - -1. D1-D3 in `crates/graph` (the location table and the program section), and the inventory rows - and the join in `crates/host-core`. -2. Gap-free tests in `crates/host-core/tests/successor_swap.rs`. -3. A test-support counter of bytes the carry passes through the payload scratch. - -## Authorized paths - -- `crates/graph/src/lib.rs`, `crates/graph/src/runtime.rs` -- `crates/host-core/src/prepare.rs`, `crates/host-core/tests/successor_swap.rs`, - `crates/host-core/tests/support/successor.rs` - -## Non-goals - -- No graph delay lines (#1283, #1284). -- No live bypass shunt for the delay or the multiband (#1339, #1340). - -## Objective gates - -1. **Gap-free acceptance: a delay tail.** Session A has three tracks: one with a delay insert at - 50 ms and high feedback, and one with a multiband compressor insert. The source plays for 4 - blocks, then drops to a deterministic noise floor near -80 dBFS (never an exact zero). Session B - adds a muted track whose ID sorts first. Swap after block 6, while the tail rings. The swapped - run and a fresh B fed the same PCM from frame 0 are bit-identical for 64 blocks. With this - slice's section of the program disabled, the tail stops at block 7. -2. **Unread values and live bypass.** Prepare with live controls. Write a delay-time value to A's - cell just before the swap (not yet rendered), and a live bypass of a per-node compressor insert a - block earlier (one track, so it does not bank). Prepare B from the base. Every block equals the - reference fed the same values at the same blocks. -3. **A prepared bypass change restarts.** A committed bypass change on the delay: the join has no - pair for it, its successor instance starts at rest, and `restarted_strips()` is exactly that - strip. -4. **Cost shape.** At 96 kHz, the payload-bytes counter reads zero for the delay's owner (it - moved). The swap block makes zero allocations and frees. -5. Commands: - - `cargo test --locked -p graph -p host-core --features graph/test-support,host-core/test-support` - - `cargo build --locked --release -p audit && bash scripts/trace-graph-audit.sh target/release/audit` - - `cargo run --locked --release -q -p realtime-policy`, `bash scripts/check-workspace-policy.sh`, - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings` - -## Test value - -- Gate 1: a carry that leaves per-node effects at rest cuts the tail. It turns red. -- Gate 2: a move that keeps the successor's own lane state drops the acknowledged live bypass and - the unread value. It turns red: the compressor's bypass is a lowered lane bypass, so the gate - sees it. -- Gate 3: a rule that ignores the prepared bypass moves an unbypassed delay into a plan whose - committed bypass is on. It turns red. -- Gate 4: a carry that copies a delay through the payload scratch (correct bits, about 1.5 MiB - copied twice on the render thread) turns it red. - -## Dependencies - -- *Carry an insert lane that moves between a bank and a per-node instance* (#1281). - -## Amendment (controls coordinator, batch 10, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Anchors moved.** `NodeKind::LiveControlEffect(Box)` is - `crates/graph/src/runtime.rs:1161`, struct `:1228` (fields `effect`, `control`, `spans`, `shunt`, - `observation`); `GraphPreparedEffect` is `crates/graph/src/lib.rs:909`; `NEVER_BANKED_EFFECTS` is - `crates/effect-compiler/src/prepare.rs:291` and `PREPARED_BYPASS_EFFECTS` `:307`. #1339 and #1340 - are open, and the registry keeps the delay and multiband bypass rebuild-only - (`lowers_session_bypass`), so D1 stands. -- **D3 for decision 17.** Both hosts attach effect lanes from `controls::live_lanes`, which depends - on the session's shape only, so a mixed pair arises only when a transaction adds the first or - removes the last effect with a live row. D3 stands. -- **D2.** Swap `effect`'s processor box and the `shunt`; the `spans` window is not moved (each plan - sized its own from the same metadata; #1621, open, sizes it by live cells, which this move does - not read). The lane's control state goes through `EffectControlLane::carry_from` with its carried - slots (#1280a). The `observation` lane is #1327b's. -- **Live retarget.** A live difference on a per-node owner retargets by #1280c's per-instance - function. -- **Gate 2.** A live bypass now crossfades over `mute_ms` (#1341a1). The reference gets the same - record at the same block, so the gate stands. -- **Edit order with #1621** (`runtime.rs`, the per-node window): whichever is scheduled first lands - first; the other rebases. They change different lines. -- **Dependencies.** #1281, #1280c (for D1's retarget). - -## Decision record: #1282 (batch 11) - -**Shipped.** A live-controlled per-node effect instance that survives a swap with its prepared -layout unchanged moves by pointer: D1-D3 as frozen. - -- `graph`: `LiveControlEffect::can_move_from` / `move_from` (`runtime.rs`, in a realtime-policy - region). The check compares the prepared layout, the control kind and cell shape (the successor - holds its carried slots, #1633) and the shunt shape. The move takes the control state through - `EffectControlLane::carry_from` (unread cells, bypass crossfade, symmetry terms, #1280a), then - exchanges the two processor boxes and the two `BypassShunt`s with `core::mem::swap`. The `spans` - window and the `observation` lane stay (#1621 sizes the window; the tap state is #1327b's), and - the pending `Observe` records of the predecessor are applied at the carry by the second marked - loop, as for every other move. `Runtime::can_carry_effect_lanes` and `carry_effect_lanes` gain the - node-to-node arm. `install_effect_carry` no longer refuses a node-to-node move and - `GraphCarryInstallError::EffectMoveBetweenNodes` is deleted. -- `host-core`: `resolve_effect_carry` drops the node-to-node exclusion, and a node-to-node move - adds nothing to the payload scratch (`carry.scratch_bytes` counts only moves that go through it), - so the carry's retained charge for a session of delays is the move table and the carried slots. - D3 needs no code: a per-node effect without live controls has no inventory row, so its owner - restarts its strip. -- `effect-contract` (outside the authorized list, one mechanical helper): `BypassShunt::same_shape`. -- `graph` test-support: `test_only_carry_payload_bytes` / `test_only_reset_carry_payload_bytes`, a - thread-local count of the layout payload each lane move passes through the scratch. -- Test harness: `effect_run_blocks` (a run of more than `BLOCKS` blocks), `Feed::with_blocks`, - `EffectRun::payload_bytes`; `LivePlan::submit` submits at the plan's own sample rate. - -**Departures.** None from the frozen decisions. D1's "prepared bypass" needs no new code: the -inventory layout already carries `bypass`, and `effect_instance_records` refuses a bypass change of -an effect that does not lower it. - -**Gates and tests** (`crates/host-core/tests/successor_swap.rs`): - -| Gate | Test | -|---|---| -| 1 | `a_delay_tail_rings_on_through_the_swap_at_every_width` (64 blocks; restart run loses the echo) | -| 2 | `unread_values_and_a_live_bypass_cross_the_swap_at_every_width` | -| 3 | `a_prepared_bypass_change_restarts_its_strip_at_every_width` (`restarted_strips() == ["eq0"]`) | -| 4 | `a_delay_moves_by_pointer_at_96_khz_at_every_width` (payload bytes 0, swap block clean); `the_payload_counter_counts_a_lane_move` (the counter reads nonzero for a bank move) | -| D3 | `an_owner_without_live_controls_on_one_side_restarts_at_every_width` | -| D1 live | `a_live_difference_on_a_moved_instance_is_retargeted_at_every_width` (delay time, compressor bypass) | - -Red under mutation (each run once, reverted): no processor exchange (gates 1, 2, 4 red), no -`carry_from` (gate 2 red). - -**Consequence outside the slice.** `capi`'s `a_prepared_bypass_change_rebuilds_and_renders_the_committed_model` -(#1266 gate 3) used a control that rebuilt by content edits, which restarted the per-node multiband -as a side effect. A content edit now moves the multiband, so the control's rebuilds also replace -the multiband by a fresh instance of the same declaration under a new ID (a prepared change of -that owner), which restarts it and its strip as the bypass rebuild does. The oracle is otherwise -unchanged. - -**Measured.** Browser boot high-water growth 20,447,232 bytes (before: 20,447,232; the pin is -20,971,520). No resource estimate or pin moved: `cargo test -p host-core --all-features` (428), -`effect-contract`, `rack`, `graph`, `graph-compiler`, `capi`, `engine`, `host-web --features -host-web/test-support` green; clippy `-D warnings` on `host-core` (no features, default, -all features) and `host-core graph effect-contract rack capi --all-targets --all-features`; -`realtime-policy`, `check-bench-policy.sh`, `check-builtins-fixtures.sh`, `check-cross-targets.sh`, -`check-web-audioworklet.sh`, `trace-graph-audit.sh` (1,000,000 blocks) and rustdoc `-D warnings` pass. - -### `audit capi` digest (batch 11) - -The C ABI render audit commits one structural transaction after its first call (it adds a muted track). Its session also holds a per-node live effect instance (the audit editor reaches a non-EQ effect lane). Before this slice that instance restarted at the swap; now it moves by pointer and keeps its state. So `audit capi`'s `pcm_digest` moves from `9bc5ec3d8a4c223c` to `bf7dcedd1d92621f`, measured by building the audit at `405221f8d` (#1633, the parent: `9bc5ec3d8a4c223c`) and at `067d145ec` (this slice: `bf7dcedd1d92621f`); the later batch 11 slices leave it at `bf7dcedd1d92621f`. Allocations, frees and render errors stay 0. diff --git a/.github/ISSUE_SPECS/1283-carry-compensation-lines-across-a-plan-swap.md b/.github/ISSUE_SPECS/1283-carry-compensation-lines-across-a-plan-swap.md deleted file mode 100644 index 7bb656cbd..000000000 --- a/.github/ISSUE_SPECS/1283-carry-compensation-lines-across-a-plan-swap.md +++ /dev/null @@ -1,211 +0,0 @@ -# Carry compensation lines across a plan swap - -Stream A of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-7, D15-8, D15-9). -Slice 13 of *Swap a rebuilt plan without an audio gap* (#1269). Code anchors verified on `main` at -`6fb211594`. - -## Product outcome - -In a session with plug-in delay compensation, adding a track or a zero-latency effect no longer -empties the compensation lines. Every compensated path keeps its delayed audio through the swap, at -the output and inside submixes alike. A route that the transaction -re-points does not carry its line under its route ID, and its source strip is never ducked for it: -a re-pointed route is a route removed plus a route added, which *Ramp a route that a plan swap adds -to or removes from a surviving strip* (#1363) ramps at route level (D15-9). - -## Context - -- PDC delays each edge into a node by that node's latest incoming arrival minus the edge's own - arrival (`timings`, `crates/graph-compiler/src/pdc.rs:38`; `max` at `:61-65`, the per-edge delay - at `:66-88`). It does this at every node with several inputs: the output, every submix input, - and every effect with a sidechain. Each delay is an `InsertedDelay { node, edge_id, samples }` - (`crates/graph/src/lib.rs:344`). -- At runtime a line is two rings and one cursor (`CompensationDelay`, - `crates/graph/src/runtime.rs:940-944`), held in `Runtime::delays` (`:2509`) and reached through - `StagedInput::line` (`:1295-1302`). -- Edge IDs are stable: `GraphEdgeId::{TrackMain, RouteSource, RouteDestination, EffectSidechain}` - (`graph/src/lib.rs:306-311`), keyed by node, route or effect. -- The program scaffold is today's move-mode `GraphCarryProgram` (`crates/graph/src/lib.rs:2757`), - run at the swap block; the predecessor then retires. The restart set and the per-node carries - come from *Carry fader, mute and pan ramps across a plan swap* (#1277, D6) and *Carry per-node - effect instances across a plan swap* (#1282). - -## Decisions frozen for this slice - -- **D1. Key and rule.** A line carries when its `GraphEdgeId` exists in both plans and the edge's - source and destination nodes are the same in both. A line has no live value, so there is no - retarget. - - A re-pointed route keeps its route ID but gets a new source or destination. Its line does not - carry: the new route's line starts at rest. Its source strip does **not** go into the restart - set. #1363 D1(b) and D4 keep the old route as a fading route under its own graph ID and fade - the new route in. - - The lookup is by successor edge ID, so *Ramp a route that a plan swap adds to or removes from a - surviving strip* (#1363 D4) can extend it with an alias: a fading route's edge carries from the - predecessor's edge of the original route ID, and a predecessor line named by an alias carries - only into the alias (the successor's edge under the original ID then starts at rest, even with - unchanged endpoints). This slice's lookup maps each successor edge to the predecessor edge - it reads, which is the same ID here; #1363 D4 supplies the alias entries. - - An effect owner that restarts (#1279 D1) does not stop the lines around it from carrying. -- **D2. Equal length, move mode.** Swap the two rings and the cursor. -- **D3. Only edge lines.** This slice carries only lines keyed by a `GraphEdgeId`. A source-claim - line (a raw-source line on a source-reading node, *Grow latency during playback by adopting a - primed warm successor*, #1287 D1) is not an edge. *Carry source-claim lines across a plan swap - and fill a grown line for a prime* (#1402) carries it, keyed by claiming node and source, and - fills it by #1287's L2 and L3. -- **D4. Different lengths: a head-aligned copy.** The successor emits the predecessor's pending - samples first, in the order the predecessor would have emitted them, truncated to fit or followed - by `+0.0`. - - So a path whose compensation grew by `δ` has a gap of `δ` samples after its pending audio, and one - whose compensation shrank by `δ` skips `δ` samples. With no surviving node's arrival changing, an - unchanged edge always has equal length. *Keep every node's latency from dropping during - playback* (#1285) keeps arrivals from dropping. In a warm adoption (#1287) every edge between - carried nodes keeps its length (its L1), so growth never reaches this gap. -- **D5. One routine.** One allocation-free copy routine serves D4 in both directions and both - channels. - -## Deliverables - -1. D1-D5 in `crates/graph` (the location table and the carry routine), and the inventory rows and - the join in `crates/host-core`. -2. Unit tests of D4's copy in `crates/graph`: equal, longer and shorter successor lengths; a - predecessor cursor at every position modulo the block; a wrapping source cursor. -3. Gap-free tests in `crates/host-core/tests/successor_swap.rs`. - -## Authorized paths - -- `crates/graph/src/lib.rs`, `crates/graph/src/runtime.rs` -- `crates/host-core/src/prepare.rs`, `crates/host-core/tests/successor_swap.rs`, - `crates/host-core/tests/support/successor.rs` - -## Non-goals - -- No change to how PDC computes delays. No floors (#1285), no source-claim lines (#1287 D1) and no - claim-line carry (#1402). -- No strip or submix input delay lines, and no live send ramps (#1284). -- No route-level ramp for a re-pointed route, and no fading-route line alias (#1363 D4). - -## Objective gates - -1. **Gap-free acceptance at the output.** Session A has three tracks. The first has a true-peak - limiter insert (latency `Fs/100 + 6`), so the other two edges into the output carry lines. - Session B adds a muted track with no latent effect, so no node's arrival changes. The swapped run - (swap after block 6) and a fresh B fed from frame 0 are bit-identical in every block, at 44.1, - 48, 88.2 and 96 kHz. With this slice's section disabled, the compensated tracks drop out for the - line's length. -2. **Inside a submix.** Session A also routes two tracks into a submix, one of them through a - limiter insert, so the other's edge into the submix carries a line. B adds a muted zero-latency - track to that submix. Bit-identical in every block. -3. **Head alignment** (deliverable 2) passes. -4. **A re-pointed route does not duck its strip.** B re-points one route to another submix. Its - line under the route ID starts at rest, every other line carries, and `restarted_strips()` is - empty. -5. **Realtime.** The swap block makes zero allocations and frees. -6. Commands: - - `cargo test --locked -p graph -p graph-compiler -p host-core --features graph/test-support,host-core/test-support` - - `bash scripts/check-graph-determinism.sh` - - `cargo build --locked --release -p audit && bash scripts/trace-graph-audit.sh target/release/audit` - - `cargo run --locked --release -q -p realtime-policy`, `bash scripts/check-workspace-policy.sh`, - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings` - -## Test value - -- Gate 1: a line keyed by delay-vector index (which shifts when a track is added) instead of edge - ID, or a line left at rest, turns it red. -- Gate 2: a carry that handles only edges into the output leaves submix lines at rest. It turns red. -- Gate 3: a copy that starts from the predecessor's write position instead of its read position - turns it red. -- Gate 4: a rule keyed only by route ID carries a re-pointed route's line into the wrong bus. A - rule that still restarts the source strip for a re-pointed route puts it in the restart set. - Either turns it red. - -## Dependencies - -- *Carry per-node effect instances across a plan swap* (#1282). - -## Amendment (controls coordinator, batch 10, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Anchors moved.** `timings` is still `crates/graph-compiler/src/pdc.rs:38`; `InsertedDelay` is - `crates/graph/src/lib.rs:344` and `GraphEdgeId` `:306-311`; `CompensationDelay` is - `crates/graph/src/runtime.rs:958`, `Runtime::delays` `:2603`, `StagedInput` `:1359`; - `GraphCarryProgram` is `crates/graph/src/lib.rs:3005`. The carry runs in - `GraphExecutor::adopt_predecessor` (`lib.rs:3598`): every section's check runs before any section - moves, so this section's check joins that list. -- #1285, #1287, #1363 and #1402 are open; D1-D5 and the gates stand. -- **Dependency on #1282.** The lines hold no effect state; the dependency orders the shared files - (`graph/src/{lib.rs,runtime.rs}`, `host-core/src/prepare.rs`, `successor_swap.rs`) only. This - slice can land before #1327 or after it. - -## Decision record: #1283 (batch 11) - -**Shipped.** D1-D5 as frozen: a compensation line carries when its edge ID exists in both plans -with the same endpoints; equal lengths swap the rings and the cursor, other lengths take a -head-aligned copy. - -- `graph/program.rs`: `DelaySpec` gains `edge` (the delayed edge's position in `GraphSpec::edges`), - so the lowered program says which edge each line delays. -- `graph/runtime.rs` (policy regions): `CompensationDelay::adopt` (D2 swap, else D4 copy through - `carry_head_aligned`, one routine for both channels and both directions), and - `Runtime::can_carry_delay_lines` / `carry_delay_lines`, bounded by the move table. -- `graph/lib.rs`: `GraphDelayLine` (edge, source, destination, line, length), `GraphDelayMove`, - `take_compensation_lines`, `install_delay_carry`, `delay_carry_retained_bytes`, three install - errors, `GraphExecutor::carry_delays`, and the check and the carry in `adopt_predecessor` (the - check joins the "every section checks before any section moves" list). `carry_program_retained_bytes` - counts the move table. -- `host-core/prepare.rs`: `PlanStateInventory::delay_lines` (counted in `retained_bytes`), - `resolve_delay_carry` (the join), the table charged against the graph cap, and a program that - names the predecessor and moves no source when a line is the only thing that carries. - -**Decisions taken where the text left room.** - -- A route's two edges (`RouteSource`, `RouteDestination`) report the route's own endpoints (its - source strip and its destination node), not the edge's. The destination edge's own endpoints do - not change when only the route's source is re-pointed, so the text's "same source and destination - nodes" would have carried the old line there. -- The identity of the lines (`GraphDelayLine`, with ID strings) is not retained in the plan: the - executor holds it until `take_compensation_lines` is called, once, right after bind, and the - inventory (control side, counted) keeps it. The plan retains only the move table. -- The lookup is by successor edge to the predecessor edge of the same ID; #1363 D4 adds the alias. - -**Gates and tests.** - -| Gate | Test | -|---|---| -| 1 | `compensated_output_lines_keep_their_audio_at_every_rate_and_width` (44.1, 48, 88.2, 96 kHz; each asserts two compensated lines in the predecessor) | -| 2 | `compensated_submix_lines_keep_their_audio_at_every_rate_and_width` | -| 3 | `runtime::tests::a_successor_line_continues_the_predecessors_pending_samples_head_aligned` (equal, longer, shorter, one-word lines; a cursor at every position and wrapped) | -| 4 | `a_repointed_route_does_not_carry_its_line_at_every_rate_and_width` (`restarted_strips()` empty; muted re-pointed strip equals B, audible one differs); `prepare::tests::delay_lines_join_by_edge_and_endpoints`; `tests::a_route_edge_names_the_endpoints_of_its_route` | -| 5 | `assert_clean_swap` in every case above: allocator and render audit zero over the swap block | - -Red under mutation (each run once, reverted): no `install_delay_carry` (gates 1, 2, 4 red), join -without endpoints (gate 4 red), copy from index 0 instead of the cursor (gate 3 red). - -**Consequence outside the slice.** Two `capi` live tests compared a rebuild with another history -and relied on the lines restarting at rest: `a_live_mute_survives_a_later_rebuild` (live mute one -block before the swap against a mute in the swap transaction) and -`a_prepared_bypass_change_rebuilds_and_renders_the_committed_model` (carried source ring against a -fresh one in the control). With carried lines the audio a line holds at the swap plays out after -it, and in both pairs the held audio differs by one block. They now compare from the block after -the lines flush (the session's latency, in blocks); the oracle is otherwise unchanged. Found, not -from this slice: `host-web` `meter_delivery_restarts_on_the_successor_and_never_publishes_a_misshaped_frame` -fails at the base (lane W's file). - -**Measured.** Browser boot high-water growth 20,447,232 bytes (before: 20,447,232; the pin is -20,971,520). No pin moved. `host-core` (442 with all features), `graph` (105), `graph-compiler` -(178), `rack` (67), `capi` (153), `control-plane` (5) green; clippy `-D warnings` on `host-core` (no -features, default, all features) and `host-core graph capi` with all targets; `realtime-policy`, -`check-bench-policy.sh`, `check-workspace-policy.sh`, `check-builtins-fixtures.sh`, -`check-graph-determinism.sh`, `check-cross-targets.sh`, `check-web-audioworklet.sh`, -`trace-graph-audit.sh` (1,000,000 blocks) and rustdoc `-D warnings` pass. - -## Fold (batch 11 review) - -- The `#1283 D1` join doc sat on the test helper `strip_delay`; it now sits on - `delay_lines_join_by_edge_and_endpoints`, the test it describes. -- `a_live_mute_survives_a_later_rebuild` and `a_prepared_bypass_change_rebuilds_and_renders_the_committed_model` - compared two of six blocks after the lines flushed. Both rigs, the control and the rebuild run now render - `flushed + 6` blocks after the seek and compare `[flushed..]`, so six blocks are compared. The gate-3 comment - now says what the pre-flush window holds: the source ring's audio and audio rendered under the predecessor's - bypass in the carried lines, which no control built from the committed model reproduces. diff --git a/.github/ISSUE_SPECS/1284-carry-strip-delay-lines-and-live-send-ramps-across-a-plan-swap.md b/.github/ISSUE_SPECS/1284-carry-strip-delay-lines-and-live-send-ramps-across-a-plan-swap.md deleted file mode 100644 index 7689bdaca..000000000 --- a/.github/ISSUE_SPECS/1284-carry-strip-delay-lines-and-live-send-ramps-across-a-plan-swap.md +++ /dev/null @@ -1,218 +0,0 @@ -# Carry strip delay lines and live send ramps across a plan swap - -Stream A of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-7, D15-8, D15-9). -Slice 14 of *Swap a rebuilt plan without an audio gap* (#1269). Code anchors verified on `main` at -`6fb211594`. - -## Product outcome - -Three kinds of state keep their exact values through a plan swap. The carry moves them at the swap -block, and the predecessor never renders again: -- a strip's input alignment delay (`delay_samples`, used for multi-mic alignment); -- a submix's input delay; -- a live send whose level is ramping at the swap. - -After this slice every DSP state family of a plan carries. A structural edit that changes no -node's latency is then bit-continuous for every unchanged path. Meters follow in #1327. -A changed `delay_samples` restarts its line, and the strip is reported for the duck-swap transition -(D15-9). A re-pointed send never ducks its source strip: it is a route removed plus a route added, -which *Ramp a route that a plan swap adds to or removes from a surviving strip* (#1363) ramps at -route level (decision 15, D15-9). The retarget of a carried send whose live value changed is -*Deliver value-only send edits to the running C ABI plan* (#1225) D8. - -## Context - -- A strip's `delay_samples` is a `TrackDelayLine` (`crates/graph/src/runtime.rs:1021`). It is driven - by `NodeKind::TrackDelay` on a track input or `NodeKind::SumDelay` on a submix input - (`:1106-1132`), and held in `Runtime::track_delays` (`:2513`). - - It is not latency (`crates/graph-compiler/src/pdc.rs:1-24`). - - A strip whose delay is 0 on both lanes has no line. -- A live send is `NodeKind::LiveRoute(Box)` (`runtime.rs:861-874`): an `IndexedRamp`, - its position, its mute and its control lane. At bind it is settled on the prepared coefficients. - Its `drain` (`:896`) applies the records present at entry directly to the ramp. -- The send records the classifier emits arrive with #1225 and *Let C ABI sends follow their - source strip's mute live* (#1226); decision 15 also makes `follows_mute` live in the browser - (#1342). Until they land the classifier emits no route record, so every send value change is a - prepared difference. #1225 D8 adds the carried send's retarget when it makes them live; this slice - does not depend on it. -- The base, the retarget pattern and the restart set come from #1277 (D2-D6) and *Carry - compensation lines across a plan swap* (#1283). The carry program and its all-or-nothing check - are on `main` (#1270-#1276). - -## Decisions frozen for this slice - -- **D1. Strip delay lines.** The key is the strip ID and the lane. A line carries when the lane's - `delay_samples` is equal in the base and in the successor's model. The lengths are then equal. - - At the swap block the carry swaps the two rings and the cursor. - - A changed `delay_samples` is a prepared value (decision 14; D15-9). Its line starts at rest, and - the strip goes into the restart set, so #1324's duck-swap removes the click. -- **D2. Live sends.** The key is the route ID. The route's source, tap and destination are - structural. - - If any of them differs, the route is re-pointed (decision 15, D15-9): its - ramp state does not carry, and its source strip does **not** go into the restart set. #1363 - D1(b) ramps it at route level, as the old route removed plus the new route added. - - Gain, matrix, mute and `follows_mute` are live exactly when the classifier emits a record for - their change (#1277 D3). A send whose only differences are live carries. - - A send with any other value difference does not carry and starts at its prepared value. Its - source strip does not go into the restart set; #1363 handles it like a re-pointed route. Once - #1225, #1226 and #1342 have landed, every value of a send into a submix is live, so this case no - longer arises. - - At the swap block, drain the predecessor's route lane (the records present at entry), then copy - the ramp, its position and its mute. The drain applies the records at the boundary where the - predecessor's next block would apply them, so the successor continues exactly. -- **Moved out.** The former D3 (carry, then retarget for sends) and its gate are #1225 D8 and its - gate 7. This slice writes no route record and edits no classifier code. - -## Deliverables - -1. D1-D2 in `crates/graph`, and the inventory rows and the join in `crates/host-core`. -2. Gap-free tests in `crates/host-core/tests/successor_swap.rs`. - -## Authorized paths - -- `crates/graph/src/lib.rs`, `crates/graph/src/runtime.rs` -- `crates/host-core/src/prepare.rs`, `crates/host-core/tests/successor_swap.rs`, - `crates/host-core/tests/support/successor.rs` - -## Non-goals - -- No change to delay or ramp kernels. No floors (#1285). -- No duck-swap itself (#1324), and no route-level ramp (#1363). -- No send retarget (#1225 D8). - -## Objective gates - -1. **Gap-free acceptance.** Session A has: - - one track with `delay_samples` on both lanes (different values); - - two tracks routed into a submix with `delay_samples`; - - a live send from a third track into that submix. - - Prepare with live controls. Admit a send-level record with a multi-block ramp between blocks 5 - and 6. Prepare B (A plus a muted track whose ID sorts first) from the base. Every block equals - the reference fed the same record at the same block. -2. **Prepared changes.** A strip whose `delay_samples` changed starts its line at rest. A send - re-pointed to another submix starts at its prepared value, and its ramp is not carried. - `restarted_strips()` is exactly the delay-changed strip; the re-pointed send's source strip is - not in it. -3. **Realtime.** The swap block makes zero allocations and frees. -4. Commands: - - `cargo test --locked -p graph -p host-core --features graph/test-support,host-core/test-support` - - `cargo build --locked --release -p audit && bash scripts/trace-graph-audit.sh target/release/audit` - - `cargo run --locked --release -q -p realtime-policy`, `bash scripts/check-workspace-policy.sh`, - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings` - -## Test value - -- Gate 1: any of three defects turns it red: - - a strip or submix delay line left at rest drops the delayed strip for its delay length; - - a send ramp restarted from its target steps the level; - - a record is lost because the copy ran before the drain. -- Gate 2: carrying a line whose `delay_samples` changed plays the old alignment. A send carried into - a new destination mixes the old ramp into the wrong bus. A re-pointed send that still ducks its - source strip puts it in the restart set. Each turns it red. - -## Dependencies - -- *Carry compensation lines across a plan swap* (#1283). -- None on #1225: the send retarget lives there (D8) and #1225 depends on this slice. - -## Amendment (controls coordinator, batch 10, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Anchors moved.** `LiveRoute` is `crates/graph/src/runtime.rs:861` (its `drain` `:902`), the - `NodeKind::LiveRoute` variant `:1166`, `TrackDelayLine` `:1039`, `NodeKind::TrackDelay` `:1124`, - `NodeKind::SumDelay` `:1144`, `Runtime::track_delays` `:2607`. -- **Send records are live now.** #1225, #1226 and #1557 are closed. For a route into a submix the - registry's gain, matrix, mute and `follows_mute` rows are `Live` (`crates/controls/src/route.rs`) - on both hosts, and the browser reaches them through the core since the flip (#1342 is still open - but has nothing left to make live). A route into the output has no `LiveRoute` lane and stays - prepared. So the Context bullet "Until they land the classifier emits no route record" and D2's - case "a send with any other value difference" apply only to routes into the output. -- **#1225 D8 moves back here.** #1225 closed as superseded by #1557, and its decision 17 amendment - moves its D8 (carry, then retarget for sends) to this issue. The bullet "Moved out", the non-goal - "No send retarget (#1225 D8)" and the Dependencies line on #1225 are void. New: - - **D3. Carry, then retarget (D15-7; #1225 D8).** The classifier's per-route derivation - (`route_records`, `crates/host-core/src/live_delta.rs:1416`) becomes one per-route function that - `classify_live_delta` and the successor join both call (the pattern of #1277 D5). It works on - effective values: a route's effective gate follows its source strip's effective mute (#1558 - D1). For every carried send whose live values differ between `committed` and the successor's - model, the successor entry writes the route's record into the successor's route cell - (`RouteControlProducer::write`, `crates/host-core/src/route_controls.rs:237`) with - `stamp_held_gate(base.revision)`, in the retarget block with the strips (`prepare.rs:2644`), - with the ramp from `SuccessorBase::edit_ramps`. The write cannot fail (#1347). - - **Gate 5 (new; #1225 gate 7).** On gate 1's session, B is A plus a muted track whose ID sorts - first, and B also changes the carried send's gain. Run 1 is the structural swap with D3's - retarget. Run 2 writes the same value to A just before the swap and prepares B from a base that - holds it. Every block of the two runs is bit-identical. It turns red if a carried send keeps - the predecessor's level (a missing retarget) or if the retarget applies a block early or late. -- **D2's drain with cells.** A route lane holds a latest-target cell since #1347. "Drain the - predecessor's route lane (the records present at entry)" reads: apply the predecessor's pending - cell value under `LiveSnapshot::ALL` with `CellReader::read_counted`, as the strip carry does - (#1277 D8), then copy the ramp, its position and its mute. No engine edit. -- **Authorized paths** gain `crates/host-core/src/live_delta.rs` (D3's extraction only). -- **Dependencies.** #1283 (shared files). #1225 is removed. - -## Decision record: #1284 (batch 8) - -**Shipped.** D1, D2 and the amended D3 (carry, then retarget; #1225 D8), as frozen. - -- `graph`: `TrackDelayLine::lanes_match` / `adopt` (equal rings and cursors swap, lane by lane), - `LiveRoute::adopt` (ramp, position, mute), `Runtime::{can_carry,carry}_{strip_delays,live_routes}`, - all in policy regions and bounded by the move table. The route carry first applies the - predecessor's pending cell value under `LiveSnapshot::ALL`, then copies. New API: - `GraphStripDelayLine` / `GraphStripDelayMove` (lane mask `STRIP_DELAY_LEFT|RIGHT`), - `GraphLiveRoute` / `GraphRouteMove` (by runtime unit), `take_strip_delay_lines`, `take_live_routes` - (read once after bind, as `take_compensation_lines`), `install_strip_delay_carry`, - `install_route_carry`, the two `*_carry_retained_bytes`, six install errors; both moves join the - "every section checks before any section moves" list. `build_sequential` now also returns the bind - identities. -- `host-core`: `PlanStateInventory::{strip_delays, live_routes}`; `join_strip_delays` and - `resolve_strip_delay_carry` (the key is strip and lane; a lane that carries has an equal - `delay_samples` and a line in both plans; a strip with a changed lane, or whose line one plan lacks, - goes into `restarted`); `join_routes` and `resolve_route_carry` (a route into a submix with the same - ID, source (strip and tap) and destination, and a live route in both plans). D3: the per-route - function is `route_record_between` (+ `RouteEditRamps`, moved from `live_delta.rs`); the classifier - and the join both call it, on effective values (`EffectiveDiff::route_gates`, `gate_length`, - `source_strips`, un-gated from `control-provider`). The retarget is written with - `stamp_held_gate(base.revision)` in the strips' block. `join_strips` takes the `EffectiveDiff`, built - once per preparation. The late carry program (names the predecessor, moves no source) now also - covers a plan whose only carry is a strip delay or a send. - -**Departures.** - -- `LiveRoute::drain` is renamed `apply_pending`. The call rule of the realtime policy refused - `from.drain(..)` in the move loop, because other functions named `drain` pop a queue. `apply_pending` - holds no pop (it reads a latest-target cell); its one other caller is the block drain in `execute_op`. -- A route carries only if its source and destination are equal. The text says "source, tap and - destination"; the tap is part of `RouteSource`. -- Test harness: `EffectValue::SendGain` and `CellRecord::Route` write a send record. The two - `a_followed_mute_change_*` oracles (the "written to A" run) now also write the send's gate record, as - a live commit does (#1558 D1): with sends carried, a strip mute written to A without the following - send's record is not the state the committed model describes. - -**Gates and tests** (`crates/host-core/tests/successor_swap.rs` unless noted): - -| Gate | Test | -|---|---| -| 1 | `strip_delays_and_a_send_ramp_cross_the_swap_at_{eight,four}_lanes` (track lanes 37/53, a lane with no delay, submix 41/29, a ramp from block 4 and a record pending at the swap) | -| 2 | `a_changed_lane_delay_restarts_only_its_lane_at_*` (restart set `["eq0"]`, left lane carried, right plane silent), `a_repointed_send_starts_at_its_prepared_value_at_*` (no restart, equals B without records), `only_the_delay_changed_strip_is_restarted_at_*` | -| 3 | `assert_clean_swap` in every case: zero allocations and frees on the swap block | -| 5 | `a_carried_send_is_retargeted_to_the_successors_gain_at_*` (run 1 retarget equals run 2 live write; not vacuous against a stale level) | -| unit | `graph::runtime::tests::a_strip_delay_line_hands_over_the_lanes_of_its_mask`, `prepare::tests::strip_delay_lanes_join_by_strip_and_lane`, `prepare::tests::live_routes_join_by_route_id` | - -Red under mutation (each run once, reverted): no strip delay move (gates 1, 2 red), no route move -(gates 1, 2b red), no predecessor drain (gates 1, 5 red), no retarget write (gate 5 red), route join -without the destination (gate 2b red), lanes keyed together (gate 2a red). - -**Measured.** Browser boot high-water growth 20,447,232 bytes (before: 20,447,232). No pin moved. -`realtime-policy`, `check-bench-policy.sh`, `check-workspace-policy.sh`, `check-builtins-fixtures.sh`, -`check-graph-determinism.sh`, `check-cross-targets.sh`, `check-web-audioworklet.sh`, rustdoc -`-D warnings` and clippy `-D warnings` (host-core no features, default, all features; graph) pass. -`trace-graph-audit.sh` was not run. - -## Fold (batch 11 review) - -- `LiveRoute::apply_pending` doc: "The drain runs in every block" is now "The read runs in every block" - (#1347 D3 reads a cell, it drains no queue). -- The two capi oracles that cover the carried lines now compare six blocks after the flush (see #1283). diff --git a/.github/ISSUE_SPECS/1285-keep-every-nodes-latency-from-dropping-during-playback.md b/.github/ISSUE_SPECS/1285-keep-every-nodes-latency-from-dropping-during-playback.md index 1639650bd..1582e5d47 100644 --- a/.github/ISSUE_SPECS/1285-keep-every-nodes-latency-from-dropping-during-playback.md +++ b/.github/ISSUE_SPECS/1285-keep-every-nodes-latency-from-dropping-during-playback.md @@ -120,3 +120,79 @@ permanent latency reserve. ## Dependencies - *Carry strip delay lines and live send ramps across a plan swap* (#1284). + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Moved anchors.** +- `timings` is `crates/graph-compiler/src/pdc.rs:37-160`. `max` is `:61-65`, the edge loop + `:66-99`, the arrival `max + latency` `:100-104`, the output latency `:143-150`. It is called + from `crates/graph-compiler/src/compile.rs:422`, inside `compile_with_builtins` (`:78`). +- `GraphBuiltinsCompileRequest` is `crates/graph-compiler/src/lib.rs:41`. Its fields are + `plan_id`, `effects`, `builtins`, `caps` and `dispatch`. No floor field exists yet. +- `HostPrepareReport::latency_samples` is `crates/host-core/src/prepare.rs:273`, filled at + `:3134`. The C ABI copies it at `crates/control-plane/src/compile.rs:1003` into + `PlanResources::latency_samples` (`crates/control-plane/src/adapter.rs:104`). The ABI field is + `crates/capi/src/abi.rs:283`. +- The inventory is `PlanStateInventory` (`prepare.rs:711`). It already holds the compensation + lines (`delay_lines`, #1283 D1), the strip delays and live sends (#1284) and the restart set + `restarted` (#1277 D6, `prepare.rs:744`; read by `restarted_strips`, `:688`). D3's "restart set + (#1279 D1)" is this field now. +- Successors are prepared by the control plane: `prepare_runtime` + (`crates/control-plane/src/compile.rs:901`) is called with a `SuccessorBase` at + `crates/control-plane/src/control.rs:1662-1683`. `SuccessorBase` is `prepare.rs:1070`. The floors + travel in the inventory, so D2 needs no control-plane change. +- The header's latency text is `crates/capi/include/miso_engine_v1.h:54` and `:81` (D6's place). + +**Authorized paths.** +- `crates/builtins-compiler` does not build the graph request. The one product site is + `crates/host-core/src/prepare.rs:2466`. Deliverable 1's "plumbed through the builtins graph + compile request" means that site. `crates/builtins-compiler/src/lib.rs` leaves the list. +- The request is a struct literal in 21 files. A new field breaks each one. Outside the list are + `crates/graph-compiler/src/bin/graph_fixture.rs`, `crates/host-core/tests/live_routes.rs`, + `route_mute.rs`, `submix_strip.rs`, `tools/audit/src/builtins_graph.rs`, `fixture_builtins.rs`, + `graph.rs` and `tools/console-workload/src/lib.rs`. Proposed: add them for the neutral field only + (an empty floor map). Root confirms at filing. +- **B15-R9, root decision under owner delegation (2026-10-15): the eight files above are allowed, for a neutral-field-only + addition to the graph request struct literal.** No other edit to them is authorized. + +**Decisions.** +- D1-D6 hold against the code and against decision 17. Decision 17 does not touch latency (C11 + keeps the render rules), and a floor is not a control. +- Both hosts: since #1381a the browser runs `control_plane::SessionState` + (`hosts/host-web/src/lib.rs:6-16`), so a browser rebuild gets floors through the same code. A + browser boot is still a no-floor case. +- The browser reports no plan latency. Decision 17 C13 lists "plan latency and tail" as a one-host + difference for root to file; no issue is filed yet. So D4 and gate 3 cover the C ABI only. +- Empty boot (#1682): an empty plan shares only the output node with its successor, at arrival 0. + Its floor changes nothing. +- Stored automation: #1637's node time is `τ(r) = timeline(r + ΣP - a(n))`. With floors, `a(n)` + must be the floored arrival, the one the plan renders. #1647 (open) reads `a(n)`. Whichever of + #1285 and #1647 lands second makes it read the floored value. + +**Dependencies.** +- #1284 is closed. #1283, #1279 and #1277 are closed. +- Batch order (root, 2026-10-15): batch 15a is the controls lane, #1344, #1398 and #1310. + #1324 left batch 15a (root ruling, 2026-10-15); it lands in a later batch after #1288, #1391, + #1363 and #1325. The rest of batch 15 follows. #1324 is not ordered before this slice. This + slice keeps its no-duck fallback until #1324 lands: a restarted strip starts at rest with no + duck. +- #1324 is open and not in batch 15/16. D3's duck-swap needs it. Until it lands, a restarted + strip starts at rest with no duck. +- #1287 is open. D2 names it as later work only. #1323 is in the batch. + +**Gates and commands.** +- Every named file and script exists: `crates/host-core/tests/successor_swap.rs`, + `crates/capi/src/runtime/tests.rs`, `docs/C_ABI_V1_QUALIFICATION.md`, + `scripts/check-graph-determinism.sh`, `scripts/check-capi-abi.sh`, + `scripts/check-workspace-policy.sh`, `tools/console-workload`. +- Gate 6's command is valid. Its features exist. + +## Amendment (controls coordinator, batch 15a fold, 2026-10-16) + +- **Browser latency.** "No issue is filed yet" (batch-15 amendment, Decisions) is void. It is + #1688 (*Report the plan latency on the browser*), which lands after this slice and adds the + browser's parity row on the floored latency. D4 and gate 3 still cover the C ABI only here. +- **Anchor.** The `restarted` field of `PlanStateInventory` is `prepare.rs:750` at `228fca26c` + (the batch-15 amendment says `:744`). On `main` at `c5b5c88b2` it is `:757`. diff --git a/.github/ISSUE_SPECS/1293-export-transaction-apply-and-anchored-seek-from-the-browser-engine-module.md b/.github/ISSUE_SPECS/1293-export-transaction-apply-and-anchored-seek-from-the-browser-engine-module.md index 44d2acd46..3431253d2 100644 --- a/.github/ISSUE_SPECS/1293-export-transaction-apply-and-anchored-seek-from-the-browser-engine-module.md +++ b/.github/ISSUE_SPECS/1293-export-transaction-apply-and-anchored-seek-from-the-browser-engine-module.md @@ -182,3 +182,85 @@ module* (#1386), which uses this slice's staging and outcome record. **Decision 17 amendment (2026-10-13).** #1562 adds one generic command-frame entry on the control half (`miso_engine_web_v1_control_request_ptr`, `_control_command`, `_control_response_ptr`, `_control_response_bytes`) that runs `SessionState::command` for any frame, transactions included. D2 (`miso_engine_web_v1_apply`) and D3 (the outcome record) are dropped: a transaction is applied through that entry and its outcome is the response frame (revision, `edit_path`, diagnostics). D4-D7 (committed document, anchored seek, seek report, mirrors) and D1's staging for #1386's document replace stay. **Decision 17 amendment (2026-10-13), second line.** D4 is withdrawn too: the committed document reaches the browser as `SESSION_SNAPSHOT_GET` frames through #1562's generic command entry, with the same bytes as on the C ABI, and the SDK reads it with `engine.committedDocument()` (#1571). No `committed_document_ptr` or `committed_document_bytes` export is added. D5-D7 (the anchored seek, the seek report and the mirrors) stay. + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Moved anchors.** +- `miso_engine_web_v1_document_ptr` is `hosts/host-web/src/ffi.rs:2959`; its live refusal is + `:2960-2970`. +- `AudioWorkletEngineHost::diagnostic` is `hosts/host-web/src/lib.rs:3175`. + `miso_engine_web_v1_buffer_ptr` is `ffi.rs:2996`, and its `BUFFER_DIAGNOSTIC` arm `:723`. +- `miso_engine_web_v1_source_seek` is `ffi.rs:3251`. Its host method `seek_source` is + `lib.rs:3322`. It calls `ready.control.seek_unserviced` and then `plan.prepare_source_seek` + (`:3340-3352`). +- `PcmSourceConsumer::prepare_seek` is `crates/source/src/lib.rs:1049` (doc `:1040-1048`). The + D4 trap still holds. +- The producer-side anchored seek is `SessionState::seek_at`, + `crates/control-plane/src/control.rs:3028` (#1309 moved it). It runs the service step first. +- The mirrors: `expected_exports` is `scripts/check-web-audioworklet.sh:274`, and its call of + `check-abi-layout-v1.py` is `:635`. `EXPORTS` is `tools/parameter-metadata/src/abi_layout.rs:125` + (102 names) and `scripts/check-abi-layout-v1.py:120`. +- #1562's generic entry exists: `control_request_ptr` (`ffi.rs:3347`), `control_command` + (`:3366`), `control_response_ptr` (`:3399`) and `control_response_bytes` (`:3412`). `service` is + `:3336`. + +**The control half today.** The browser runs one instance. Since #1381a it holds +`control_plane::SessionState` (`hosts/host-web/src/lib.rs:6-16`). Today, "no control half" means +a host that is not ready: `control_event` returns `RESULT_REFUSED_LIFECYCLE` for it +(`lib.rs:2397-2415`). That code equals `RESULT_WRONG_STATE` (`lib.rs:146`). Gate 5 uses that case +until #1332 adds `worker` mode. + +**After the two decision 17 amendments.** +- D2, D3 and D4 are dropped. Gates 1 and 2 test only those, so they go too. #1562's gates cover + apply through the generic entry. +- Deliverable 1 is now D1, D5 and D6. D7's "eight exports" is now four: `edit_ptr`, + `source_seek_at`, `source_seek_report` and `source_seek_report_ptr`. The one record is D6's. If + D1 goes (below), three. +- D5 and D6 hold against decision 17. The C ABI has both as C exports: + `miso_engine_v1_source_seek_at` (header `:425`) and #1316 D6's + `miso_engine_v1_source_seek_report`. So these exports put the same function on the second host + (C1). They are not a browser-only feature, and source transport is not a control. + +**B15-R8, root decision under owner delegation (2026-10-15): drop D1 from this slice. Re-scope #1386 under C1.** #1386's diff +goes in `control-plane` as a protocol frame through `submit_command` and `control_command` on both +hosts. Its staging is `control_request_ptr`, and its outcome is the response frame. #1386 and #1643 +record this in their own amendments. Deliverable 1 is now D5 and D6. D7's exports are three: +`source_seek_at`, `source_seek_report` and `source_seek_report_ptr`. Only D1's staging export +`edit_ptr` goes. D1's other rule stays: each remaining export runs on the control half and refuses +with `RESULT_REFUSED_LIFECYCLE` (a pointer export returns 0) on an instance with no control half. +The finding follows. D1's staging stays only for #1386's document replace. But #1386 (open) also +uses D3's outcome record, which is dropped. And #1386's diff sits in `hosts/host-web` with no C ABI +form, which is a browser-only feature under decision 17 C1. The generic entry's staging +(`control_request_ptr`, bounded by the boot's `maximum_control_frame_bytes`) already carries a +frame. Proposed: drop D1 from this slice. Re-scope #1386 under C1: the diff goes in the core +(`control-plane`) as a protocol frame through `submit_command` and `control_command` on both hosts. +Its staging is `control_request_ptr`, and its outcome is the response frame. + +**Hazard on D5.** The SDK PCM feed calls the plain seek from `process()` too: `applySharedSeek` is +"shared by render and the between-block control handler" +(`sdk/src/browser-assets/miso-engine-v1-pcm-feed-worklet.js:465-470`). That is why `seek_source` +uses `seek_unserviced`. `SessionState::seek_at` runs the service step, which `process()` forbids +(`control.rs:2983-2990`). D1's rule (control half only) holds for `source_seek_at`, so the SDK +calls it from the message handler only (#1294, #1297). A render-side caller would need a +`seek_at_unserviced` in `crates/control-plane`, which is outside this slice's paths. + +**Dependencies.** +- Closed: #1309, #1313, #1381 (as #1381a) and #1562. +- In the batch: #1316 and #1319. +- #1290 (open) was needed only for D2. #1313 was needed only for D3. Proposed: remove both. +- #1332 and #1387 are open and not in batch 15/16. In `single` mode, #1381a already routes every + source export through `SessionState`, and #1387's amendment only moves that routing to the + Worker. Proposed: make #1332 and #1387 ordering-only, for the `worker`-mode case of gate 5. D5 and + D6 in `single` mode do not wait for them. Root confirms. +- **B15-R8, root decision under owner delegation (2026-10-15):** #1293's dependencies drop #1290 and #1313. #1332 and #1387 + are ordering-only. + +**Other specs that name dropped exports.** #1386 names `miso_engine_web_v1_edit_outcome_ptr`. +#1643 gate uses `miso_engine_web_v1_apply` and cites this slice for it. Root fixes those specs. +Done under B15-R8: #1386 and #1643 each carry an amendment of 2026-10-15. + +**Gates and commands.** Every named script and file exists. `build-web-audioworklet.sh +--named-twin`, `check-sdk-generated.sh [DIR]`, the parameter-metadata `--check` +(`tools/parameter-metadata/src/lib.rs:1143`) and `check-abi-layout-v1.py --self-test` are valid. diff --git a/.github/ISSUE_SPECS/1310-supersede-an-unadopted-candidate-plan-by-compare-and-swap.md b/.github/ISSUE_SPECS/1310-supersede-an-unadopted-candidate-plan-by-compare-and-swap.md index c153a69b8..4a7d50046 100644 --- a/.github/ISSUE_SPECS/1310-supersede-an-unadopted-candidate-plan-by-compare-and-swap.md +++ b/.github/ISSUE_SPECS/1310-supersede-an-unadopted-candidate-plan-by-compare-and-swap.md @@ -238,3 +238,372 @@ a revision word. The decisions above keep their meaning, read through the follow - *Prepare a successor across a withdrawn candidate plan* (#1344). - *Publish an applied-revision watermark and complete edits asynchronously* (#1314). - *Size the C ABI's plan capacities and resource admission for a superseding candidate* (#1398). + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `e2f6cfb24`. + +**Batch order (root, 2026-10-15).** Batch 15a is the controls lane: #1344, #1398, #1310, in +that order. #1324 left batch 15a (root ruling, 2026-10-15); it lands in a later batch after #1288, +#1391, #1363 and #1325, in that order. The rest of batch 15 follows. #1319 and #1323 need this +slice (root rulings B15-R4 and B15-R5 of 2026-10-15). + +**Moved anchors.** Each fact in Context still holds; only the lines moved. +- The one-candidate refusal is `crates/control-plane/src/control.rs:1742-1744`. The #1042 lag + check is `:1579-1586`. The reservation refusals are `:1751-1765` (four arms since #1343 and + #1311). The full report table or pending list is `:1781-1788`. +- The base: `prepare_runtime` with its `SuccessorBase` literal is `control.rs:1662-1683`. + `SuccessorBase` is `crates/host-core/src/prepare.rs:1070-1097`. The carry program names its + predecessor at `prepare.rs:2131-2134` (and `:3067-3073`). +- The persisting producers move at `control.rs:1846-1849` (`adopt_persisting`, still + `crates/host-core/src/source.rs:286`). +- `SessionStore::commit_prepared` is `crates/protocol/src/model.rs:1384`. The controller's + structural commit is `commit_prepared_structural`, `crates/protocol/src/controller.rs:2260`. +- The exchange: `PlanPublisher::withdraw` `crates/engine/src/realtime/plan_exchange.rs:552`, + `republish` `:586`, `PlanReplacementReservation::set_superseded` `:642`, `UnadoptedCandidate` + `:251`. +- Gate 3's fault test is `crates/capi/src/runtime/tests.rs:2364`. `TestStructuralFaultPhase` is + `control.rs:161-175`. +- Gate 5: the structural tolerance in `plan_swap_race.rs` is now `:420` (a bounded wedge window, + `WEDGE_BLOCKS`); the source-submit tolerance is `:323`. +- Gate 7's assertions are in `crates/capi/tests/resource_lifecycle.rs` (`paused_bursts`, + `:3311-3365`, and the test at `:3078`). + +**Code notes for the implementer.** +- D3: since #1613 a value-only transaction can commit in place (`commit_prepared`'s + `TransactionForm::InPlace`, `model.rs:1397-1400`), with no displaced compilation. A rebuild + never does: the structural arm reads `prospective_session().compiled()` (`control.rs:1662-1667`). + So D3's displaced model always exists on the rebuild path. A live commit while A waits swaps + owners in A's committed model, never in P0's kept one. +- D4 and D5: `check_donation` reads A's plan through `UnadoptedCandidate::plan()`. After the + commit A is never republished, so step 2 takes A's plan with `into_plan()`. That returns A's + credit early, which only frees a credit. No `plan_exchange.rs` edit is needed. +- D4: `republish` panics if a newer epoch was committed after the withdrawal. B's reservation + takes no epoch until its commit (step 4), so a refusal before the commit can republish A. +- D6: `r_P0` is the revision last published on P0's gate (Amendment 1). P0's `ProviderEpoch` + (`control.rs:37`) holds that gate's handle, and `PlanRevisionGate::last_published` + (`crates/engine/src/realtime/plan.rs:663`) reads it. No new field is needed. + +**Decisions.** D1, D4-D7 hold against the code and decision 17. Two do not hold as written. +- **ROOT RULING (2026-10-15): B's base keeps P0's monitor overlay with P0's model, and the edit + ramps committed after `r_P0`, merged with B's own (B wins on a shared row).** This is the + proposal below; D2 and D3 read with it. The finding follows. The rest of B's base (D2, D3). + D2 writes `SuccessorBase { inventory, + committed }`. `SuccessorBase` now also has `monitor` (#1559), `edit_ramps` (#1394) and + `observation` (#1584). `monitor.committed` is "the overlay the predecessor's cells hold"; today + the control plane passes its one `self.monitor` (`control.rs:1677-1681`). While A waits, a solo + edit goes to A's cells (Amendment 1), not P0's. So for B the overlay P0's cells hold is not + `self.monitor`. `edit_ramps` is B's own transaction's. A row that only A (or a live edit while + A waited) changed then ramps with the session's `control_smoothing` in B, and gate 1's twin + (one combined transaction) ramps it with A's length. Proposed: D3 also keeps P0's monitor + overlay beside P0's model, and B's `monitor.committed` is that kept overlay. The control plane + keeps the edit ramps of every edit committed after `r_P0` and drops them with the epoch; B's + `edit_ramps` are those merged with B's own, B's ramp winning on a row both set. `observation` + is the host's bindings as now. +- **ROOT RULING (2026-10-15): B arms its taps from the host's bindings (#1584 D6a). A record kind + the bindings cannot express moves to B before publication, with room checked in D4. A kind that + fits neither stops the implementer, who reports to root.** D8 reads with this. Where the + proposal below says "before step 4", the ruling's "before publication" governs. The finding + follows. Records queued on A (D8). Each epoch's effect producers own the + instance's observation queue (`EffectControlProducer`, `crates/effect-compiler/src/prepare.rs:619`). + A record admitted between A's commit and its adoption waits in A's queue: see + `hosts/host-web/src/tests.rs:7020-7032` ("the record is queued on the successor"). D5 step 6 + drops A with that queue, so an acked record can be lost. D8 says nothing about it. Proposed: D8 + adds that B's preparation arms each tap from the host's bindings as they stand after every + admitted record (#1584 D6a), so B holds what a record queued on A asked for, and the record + drops with A. The implementer shows this for each record kind the queue holds. A kind that the + bindings do not express moves to B's matching instance queue before step 4, with room checked + in D4. If neither holds for a kind, stop and report to root. + +**The browser (non-goal).** "The browser (it adopts the control plane in #1332)" is stale. Since +#1381a the browser runs `control_plane::SessionState` in its one instance +(`hosts/host-web/src/lib.rs:6-16`). Supersession reaches it through the same code, which decision +17 C1 asks for. No browser export changes. Browser tests that expect a second structural edit to +be refused while a candidate waits are superseded with gate 7. Add `hosts/host-web/src/tests.rs` +to Authorized paths for that only, and run `cargo test --locked -p host-web --features +host-web/test-support` in gate 9. + +**Dependencies on GitHub.** +- Closed: #1502, #1309, #1343, #1314, #1311, #1312. +- Open, batch 15a before this slice: #1344, #1398. +- #1403 (open, not in batch 15) adds `PrimedCandidate`; D5 step 6's drop of it is #1403's. #1325 + (open, not in batch 15) is cited only for D5's reason. +- #1482 (open) also edits `plan_exchange.rs`'s area; this slice does not edit that file. + +**Gates and commands.** Every named command, package, feature and script exists +(`scripts/check-protocol-control-policy.sh`, `scripts/check-capi-abi.sh`, protocol and +control-plane `test-support`). + +**Realtime-path edits.** None planned under `crates/engine/src/realtime`. The exchange's +`withdraw`, `republish` and `set_superseded` exist. + +## Decision record: #1310 (batch 15a) + +**Shipped.** +- D1-D6 in `crates/control-plane/src/control.rs`: the structural arm withdraws the newest pending + candidate (`withdraw_pending_candidate`), then `rebuild` prepares, admits, reserves, checks the + donation and the observation forwards, checks the protocol predicate and commits; a refusal + leaves the candidate in `withdrawn` and `command` republishes it. After the commit, in D5's + order: `apply_donation` and `adopt_persisting` from A, B's `superseded` word + (`r_B - 1 - last_published(P0's gate)`) and revision, publication, `adopt_persisting` of A's + remaining producers back into P0's epoch, then A's plan and epoch drop. The one-candidate + `Backpressure` check is deleted. `Taken`/`Nothing` take the old path with A as base. +- D2/D3 (root ruling B15-Q3): `ProviderEpoch::carried_base` (from `PreparedHost::carried_base`, charged + as a capi epoch row with `carried_base_retained_bytes`) and `ProviderEpoch::kept: KeptBase + { model, monitor, edit_ramps }`, set on the epoch a rebuild succeeds, extended by every later + live or superseding commit (`merged_ramps`, the later edit winning), dropped when that epoch is + promoted out. B's base across a withdrawn A is `{ A.carried_base, kept.model, merged ramps, + MonitorBase { kept.monitor, next, ramp } }`. `held_models()` charges every kept model in both + admissions (`compiled_model_admission_over`); the two-model helpers are deleted. +- D3 protocol: `SessionStore::commit_prepared_displacing` and + `CommittedCommandFrame::take_displaced` (`crates/protocol`). +- B15-Q4 (root ruling): an `ObservationSlot::queued` flag marks a tap whose record waits on an + unadopted plan; `observation_forwards` re-queues the tap's bound state on B's matching instance + before publication, room checked before the commit. The observation queue holds only `Observe` + records, which the bindings express; but a tap B carries from P0 takes P0's state at the swap + and never sees a binding, so the record moves (route 2). +- Header, `CONTROL_PROTOCOL_SEMANTICS.md`, `C_ABI_V1_QUALIFICATION.md`: deliverables 3 and 4. + +**Departures and findings.** +- `crates/engine/src/realtime/plan_exchange.rs` (one attempt, accepted): the reservation owns B's + plan, so D4's check and D5's donation need `PlanReplacementReservation::{plan, plan_mut}` + (control side, outside every realtime-policy region). The amendment's "no edit needed" missed it. +- Found by gate 5: with A taken but not promoted, B committed on it leaves two adopted candidates + pending; when the atomic names B, `promote_pending_provider` now retires every older pending + epoch with the running one. Before the fix a race reached `capi.source.epoch` and + `source.ring.vacated`. +- Gate 1: "insert quality" does not prepare (#1344); T2 replaces the insert's effect. Gate 1 adds a + fourth track whose fader T1 moves with its own ramp length, so a base of the current model or a + lost ramp merge is red. +- Gate 3: the cap is `maximum_effect_state_bytes` one byte below B's own; `ProtocolCommit` is in + the fault list (returns `INTERNAL`). +- Gate 5: a new back-to-back race (`structural_edits_back_to_back_supersede_or_swap_and_never_ + backpressure`) with an unlocked control thread; the old races keep their pacing and now fail on + any structural `BACKPRESSURE`. Typical run: 32 withdrawn, 31 taken of 64. +- Gate 7: `paused_bursts` now expects the second paused edit to commit; the swap-window, epoch, + lifecycle, double-live oracle and pressure tests were updated (`resource_lifecycle.rs`, + `runtime/tests.rs`). The live peak with a pending candidate gains the kept model. +- Browser gap (B15-Q4), closed in the follow-up below. + +**Gates** (all pass): `supersede_tests::{a_superseding_replacement_renders_as_one_combined_ +transaction, a_donated_source_keeps_its_pcm_and_its_held_seek, a_refused_replacement_restores_the_ +withdrawn_candidate, a_replacement_over_the_three_plan_ceiling_is_refused_and_the_candidate_kept, +a_removed_sources_producer_returns_to_the_running_epoch}` (gates 1-4, 6; red checked for: step 5 +removed, donation skipped, superseded word 0, base model current, base inventory A's, ramps not +merged); `plan_swap_race` (gate 5, four tests); `resource_lifecycle` (gate 7); +`model::tests::a_compiled_commit_returns_the_compilation_it_displaced` (D3); +`host-web structural_apply::a_structural_edit_behind_an_unadopted_candidate_supersedes_it`. + +**Follow-up (coordinator, same batch).** +1. *Browser copy of forwarded records (B15-Q4).* The core keeps the records the last rebuild queued + again (`SessionState::observation_forwarded() -> &[ObservationForward]`, set by the + superseding commit, empty otherwise). The browser's `run_command` copies each onto the new + plan's `PlanShape` through `note_observe`, the same note `record_observe` makes for an + admitted record (armed bit, arm sample at the command's `next_absolute_sample`, commanded + bit), so the hand-over keeps it; the browser decides nothing itself. Test + `a_subscription_queued_on_a_superseded_candidate_publishes_on_its_replacement` (host-web): + red without the browser copy (mask 0) and red without the core's re-queue (no window). No + parity row: the C ABI admits no observation record, so it cannot reach the case. +2. *B wins on every row it writes (B15-Q3).* `session::EditRamps` now also records rows written with + no length and targets rewritten whole (`clear_target`); `EditRamps::overlay` lays a later + transaction over an earlier one with the later winning on every row it wrote; `get` and + `entries` are unchanged. `merged_ramps` uses it, and the live path's merge runs for any edit + that writes a live row (`is_untouched`). Tests: `edit_ramp::tests::an_overlay_lets_the_later_ + transaction_win_on_every_row_it_wrote` and capi `a_superseding_edit_without_a_ramp_length_ + takes_the_default_on_its_row` (red with the old merge). One transaction may not give a row two + lengths (`control.ramp.conflict`), so that test's reference is the transaction holding T1's + structural edit and T2. +3. *Kept monitor overlay.* `KeptBase::monitor_ramp` keeps the ramp of the latest overlay change + committed since the epoch's last revision; a replacement that writes no monitor row renders + the change between the kept overlay and its own with it. Test `a_solo_committed_while_a_ + candidate_waits_holds_on_its_replacement` (capi): equal to one transaction of T1, the solo and + T2; red with the current overlay as the base and red without the kept ramp. + +**Review fold (batch 15a).** +- *B1 (host difference).* A paused browser page refused the fifth structural edit with + `BACKPRESSURE` (probe: eight add-track transactions without render gave `[OK,OK,OK,OK,6]`) + where the C ABI commits: a superseded candidate's `PendingShape` stayed until a later adoption. + The core now says which plans it holds (`SessionState::holds_plan_epoch`), and the browser drops + the tables and spectrum captures of every unadopted entry whose epoch the core no longer holds, + after each publishing command and in `reclaim_shapes` (`drop_superseded_shapes`); it decides + nothing itself. The `PENDING_SHAPES` comment is corrected. Tests: host-web + `structural_apply::paused_structural_edits_all_commit_and_the_newest_renders_as_a_fresh_boot` + (eight paused edits all OK, the adopted plan holds all eight tracks, superseded count +7, equal + to a fresh boot of the committed model from the adoption block) and parity row + `control_parity::paused_structural_edits_commit_alike_on_both_hosts` (48 and 96 kHz); both red + with the drop disabled (`[0,0,0,0,6,6,6,6]`; parity: `control.plan.backpressure`). +- *M1 (with #1398 D3).* Render claims A and is preempted before P0's retirement reaches control; + C finds A `Taken` and publishes B on it; render adopts B; D finds B `Taken` with `[A, B]` + pending and hit the D3 accounting fault. `withdraw_pending_candidate` now promotes every pending + epoch up to the newest on `Taken`, or on an empty mailbox while one is pending (render adopts in + publication order), so a rebuild meets at most the one candidate it may supersede; the dead + older-epoch loop in the non-superseding branch is a `debug_assert`. New control-plane seam + `test_hold_reclamation` (test-support) keeps displaced plans in the retirement queue. Test capi + `supersede_tests::edits_after_render_took_two_candidates_unretired_each_commit`: two `Taken`, + blocks bit-equal to a paced run, every owner balanced; red without the promotion (the D3 + `debug_assert`). +- *Item 2 (coordinator default, root approved).* A three-plan refusal is `BACKPRESSURE` when the + successor fits beside the newest held plan alone and the committed model (what a retry meets + after render adopts the candidate), and `COMPILE_REJECTED` only when it cannot fit even then. + The second check charges the candidate's rows as prepared, so a ring it takes by donation is + counted twice there: it errs toward the permanent refusal by at most those rings' bytes, never + toward an admission. Tests: gate 4 now expects `Backpressure` at a cap between the two- and + three-plan peaks and adds a cap between P0+A and A+B that gives `graph.resource.limit` before + and after A is adopted; `runtime::tests::a_valid_edit_inside_the_swap_window_is_admitted_ + against_the_running_plan` expects `Backpressure` for the pending case (each red against the + other behavior). Header, `C_ABI_V1_QUALIFICATION.md` and `CONTROL_PROTOCOL_SEMANTICS.md` say so. +- *m3.* Test capi `a_live_ramp_committed_while_a_candidate_waits_holds_on_its_replacement`: T1 + pending, a live `t4` fader edit with a one-sample ramp on A's lanes, T2 supersedes; equal to one + transaction of all three; red with the live path's kept-ramps merge deleted. +- *m4.* Gate 3 adds a case whose withdrawn candidate is B (superseded word 1): a fault before the + protocol commit republishes B, whose adoption completes T1 as superseded and T2 as exact; a + retried T3 completes two as superseded. Red with `PlanPublisher::republish` writing a zero word + (checked in a scratch copy outside the worktree; no edit under `crates/engine/src/realtime/`). +- *m5.* The `newest_providers` doc moved off `kept_holder`; `held_models` refuses a kept model with + no row (`debug_assert!` and `INTERNAL`) instead of skipping it; the `synchronize_plan_epochs` and + `promote_pending_provider` comments name every pending candidate. +- *Header gap.* The header's "neither refeeds nor reseeks" holds for every source the new plan + keeps from the waiting replacement or from the running plan, except one case: a source or owner + that A restarted or removed and B reverts to its P0 form starts fresh in B. That gap closes with + the successor issue "Carry an owner or source a superseding candidate keeps unchanged from P0 + from P0's state". +- *Edits outside the original authorized paths, and who accepted them.* + `crates/session/src/edit_ramp.rs` (coordinator, under B15-Q3); `hosts/host-web/src/lib.rs` + (coordinator, under B15-Q4); `crates/engine/src/realtime/plan_exchange.rs`, the reservation + accessors (one attempt, accepted by root on 2026-10-16); `crates/capi/src/runtime/mod.rs` + (mechanical). + +**Review fold 2 (batch 15a).** +- *B-1 (C ABI wedge).* Two `Taken` outcomes while render lagged its retirements parked two epochs + (`retired [0,1] current 2 pending [3]`); render then adopted C and published epoch 3, and every + later call promoted from the atomic before it drained, found `2 + 1 > 2` and answered + `INTERNAL` forever. Fix, both halves: `RETIRED_PROVIDERS = RETIREMENT_CAPACITY` (3; each parked + epoch's plan holds one retirement credit), and `synchronize_plan_epochs` reclaims every + reclaimable plan before it promotes from the atomic. Test capi + `supersede_tests::two_taken_candidates_and_a_published_adoption_never_wedge_the_session` + (hold-reclamation and unpublished-render seams; asserts the reported state, then four blocks of + service and a fourth edit commit, owners balanced). Red: with both halves reverted the test + fails at the held service step, before the hold clears (the step that reclaims nothing answers + `Internal`; fold 3, m-A, corrected the earlier text, which placed it at the first call after the + hold clears); with only the capacity reverted, the test's step that reclaims nothing cannot park + a third epoch (`Internal`). +- *M-1 (a displaced plan uncharged after a `Taken` promotion).* `ProviderEpoch::parked: + Option` captures an epoch's report row, capi resources and carried bytes + when it is parked (the report table drops the row; no later commit changes it), and + `held_epochs` now returns every parked epoch first, then the running plan, then the candidates, + so both admissions charge a plan until it is reclaimed. Ordering keeps M2: an epoch carries only + from the one before it in that list or from a plan already reclaimed, so "every held epoch but + the oldest leaves its carried bytes" counts each ring once. The `held_epochs` and admission docs + say so. Test capi `a_displaced_plan_not_yet_reclaimed_is_charged_after_a_taken_promotion`: + gate 4's cap (269,263, three-plan peak 311,372) gives `BACKPRESSURE` with P0 parked, one service + clears it and the same T2 commits; a source-total cap at the counted-once peak (24,430 = 11,394 + + 22,478 - 10,640 + 22,478 - 21,280) admits and one byte below refuses. Red: parked epochs left + out gives `Ok` at the graph cap and `Ok` at 24,429; parked epochs listed after the running plan + (double count) refuses at 24,430. +- *m-1.* The header (supersession paragraph and the sizing rule), `CONTROL_PROTOCOL_SEMANTICS.md` + and `C_ABI_V1_QUALIFICATION.md` no longer promise that a retry after a render commits: + `BACKPRESSURE` means the edit fit against the newest held plan alone, so a retry normally + commits, within a small band (about +64 graph and +158 capi bytes between B prepared across A + and B' on adopted A; a donated ring counted twice in the second check). The sizing rule notes + that a plan render has just displaced counts until reclaimed. +- *m-5 (browser).* Test host-web `structural_apply::a_superseded_candidates_spectrum_handover_ + drops_with_the_superseding_command` (single capture, stream running): A's entry and hand-over + drop with T2's command, B's waits, the service after adoption promotes it and the stream + publishes windows; red with the command's `drop_superseded_shapes` disabled. Test + `structural_apply_realtime::a_superseded_candidate_with_a_spectrum_hand_over_costs_no_render_ + call_an_allocation`: render-locked allocator calls 0 over every block, no attachment drop in + render; red (7 calls) when the adoption block drops the superseded entry's captures. +- *Found with m-5, fixed (outside the listed items).* `promote_spectrum` evaluated + `entry.spectrum.take()` for every entry before checking `adoption`, so any service step or + command between a rebuild's commit and its adoption block dropped the waiting candidate's + spectrum hand-over. It now takes only an adopted entry's. The m-5 test asserts the hand-over + survives a service step before adoption (red without the fix). A stream test with a single + carried capture stayed green on the bug (the service's carried reader keeps serving), so the + user-visible effect is unproven for that case and the test was not kept; a non-carried capture, + a one-shot or a collection may differ (not investigated). +- *Re-pin.* Reference session `capi_retained_bytes` 165,922 -> 168,810 (x86-64, +2,888: one more + 1,648-byte provider-epoch slot and the 248-byte parked row in each of five slots), within the + 175,296 ceiling, which stays (`resource_lifecycle.rs`, `C_ABI_V1_QUALIFICATION.md`). +- *Acked PCM on a reverted source (root ruling of 2026-10-16, the acked-batch rule).* The header + gap was worse than "starts fresh": when A redeclared a source X, the host fed A's restarted X + ring (acked), and B declares X as P0 does, X is not donated (its declaration differs between A + and B), B prepares it fresh, and A's acked chunks dropped with A. Guard: + `host_core::drops_acked_reverted_source(B's model, A's donation side, P0's kept model)` (true + when such an X's producer on A is no longer unfed: a chunk, a seek or an end of region), called + in `rebuild` after preparation and before the admission; it answers `BACKPRESSURE`, so the + caller republishes A unchanged, no event is queued and the revision stays. After render adopts + A, the retry commits on A. It is small (one read-only function, one call; no state), so it was + coded. Tests: capi `supersede_tests::a_supersession_that_would_drop_acked_pcm_on_a_redeclared_ + source_is_backpressure` (refusal, A still the candidate, no event, revision 44; every block + bit-identical to the sequential run; the adoption block differs from a run whose host never fed + A's ring, so the acked chunk is heard) and parity row `control_parity::a_supersession_that_would_ + drop_acked_pcm_is_backpressure_alike_on_both_hosts` (48 and 96 kHz: both hosts `6`, same + revision, no browser event, retry commits alike, same bits after). Red without the guard: capi + `Ok(80)`; parity `(0, 0)` against `(6, 6)`. The header, `CONTROL_PROTOCOL_SEMANTICS.md` and + `C_ABI_V1_QUALIFICATION.md` now say such a supersession is refused with `BACKPRESSURE` until A + adopts; #1698 lifts the guard (a line added to its spec). +- *Edits outside the original authorized paths.* `hosts/host-web/src/lib.rs` (`promote_spectrum` + only), `hosts/host-web/tests/structural_apply_realtime.rs` (the m-5 cost test), + `hosts/host-web/tests/control_parity.rs` (the guard's parity row), + `.github/ISSUE_SPECS/1698-carry-an-owner-or-source-a-superseding-candidate-keeps-unchanged-from-p0.md` (one line). No edit under + `crates/engine/src/realtime/`. +- *Parity row's silent adoption block (harness, not engine).* A plain run of this session (no + edit) first outputs a non-zero sample at frame 459 at 48 kHz and 939 at 96 kHz (the plan's + compensated latency, which grows with the rate), so its first playing block is 3 / 7 and block 2 + was silent on both hosts for that reason alone (probe, not kept). The row now commits T1 at block `lead + 1` (`lead` = a plain run's first + playing block), asserts the adoption block plays, renders to block `adopt + lead` before the + retry, and asserts there that the output plays and differs from a control run whose host never + fed A's `src-t1` ring (blocks before it bit-identical), bits equal on both hosts at both rates. + Red: the control run also fed the acked chunk (`assert_ne`, block 7 at 48 kHz); adoption at + block 2 ("the adoption block plays"). + +**Review fold 3 (batch 15a).** +- *M-A (fold 3; root ruling M-A of 2026-10-16).* The fold-2 guard covered one of the ways A's fed + ring drops with A. Widened: B is refused with `BACKPRESSURE` before its commit when some source + of A has a fed producer (a chunk, a seek or an end of region) on a ring A created and the + donation `check_donation` returns for A -> B does not move it. Cases: (a) A redeclares X, B + declares X as P0 does (the fold-2 case); (b) A redeclares X, B redeclares it a third way; (c) A + redeclares X, B removes it; (d) A adds X, B removes or redeclares it. Not in scope: a source A + carries unchanged from P0 (its chunks are in P0's ring, and B treats it as a direct P0 -> B swap + would). "A created the ring" is `donor.plan.can_prepare_source_seek(index)` on X's index in A's + inventory: a carried source is prepared vacant (its ring arrives at the swap), so this is false + for it, the same freshness test `check_donation` applies. "Not donated": no pair in the + donation's consumer half has X's donor index. The function is renamed + `host_core::drops_acked_fresh_source(A's donation side, &DonationPlan)` (read-only, + allocation-free: a linear scan of the plan's pairs per source of A). It reads the donation plan, + so its call moved in `rebuild` from before the admission to right after `check_donation`, which + runs after the plan reservation; the reservation is cancelled by the return (it returns its + credit and consumes no epoch, as the `AfterPlanReservation` fault path already relies on), so + the refusal still commits nothing, queues no event, and the caller republishes A. A host that + does not render keeps getting `BACKPRESSURE` for such an edit until it renders. m-C is moot: the + guard no longer reads `self.providers.kept`. Tests: capi `supersede_tests::a_supersession_that_ + would_drop_acked_pcm_on_a_fresh_source_is_backpressure` (renamed from `..._on_a_redeclared_ + source_...`; five cases (a), (b), (c), (d) removed, (d) redeclared, each: `BACKPRESSURE`, + revision unchanged, A still the candidate, no event, every block bit-identical to the sequential + run, the adoption block differs from a run that never fed A's ring); host-core + `withdrawn_successor::the_guard_refuses_exactly_a_fed_fresh_ring_the_donation_leaves_behind` + (refused: (a)-(d); not refused: a fed `c` B keeps (donated), A's fresh `c` unfed, a source A + carries from P0 fed through A's producer, a reverted source unfed); parity row + `control_parity::a_supersession_that_would_drop_acked_pcm_is_backpressure_alike_on_both_hosts` + extended to (a), (b) and (c) (B removes `src-t1`, `t1` and its routes) at 48 and 96 kHz. Red + with the fold-2 guard restored (this fold's tests unchanged): capi (a) passes and (b), (c), (d) + removed and (d) redeclared each commit (`Ok(80)`); parity `(0, 0)` against `(6, 6)` at 48 kHz + case (b). Docs: the header supersession paragraph, `CONTROL_PROTOCOL_SEMANTICS.md` and + `C_ABI_V1_QUALIFICATION.md` state the widened rule, the paused-host cost, and that a source the + caller's own edit removes discards its queued audio. #1698 now lifts only case (a). +- *Removal contract (root ruling, fold 3).* A source the caller's own edit removes discards its + queued audio: a direct P0 -> B removal, and B removing a source A carried unchanged from P0. The + caller asked for it, so it is not an unintended drop; the header and + `CONTROL_PROTOCOL_SEMANTICS.md` say so. No counter or telemetry field reports those frames + (source telemetry is per live source and leaves with it; the watermark outcome flags and the + counters name none): #1699 (placeholder, no spec yet) is the successor issue for a counter of + the frames such a removal discards. +- *m-A.* B-1's red text corrected above: with both halves reverted, the wedge test fails at the + held service step. +- *m-B.* The drain-before-promotion order in `synchronize_plan_epochs` (B-1's second half) is + defence in depth, not a tested gate: at capacity 3 each parked plan holds a retirement credit, + so the parked list has room whichever order runs, and no test is red without the order. Code + kept; its comment says so. The B-1 test defends the capacity half only. +- *Task B-A (doc job).* Five private intra-doc links in `crates/host-core/src/source.rs` + (`donate_producers`, `donate_consumers`, `unfed`) are plain code spans; + `RUSTDOCFLAGS='-D warnings' cargo doc --locked --workspace --no-deps` passes. diff --git a/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md b/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md index 9113f8e53..cdffb0a52 100644 --- a/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md +++ b/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md @@ -153,3 +153,88 @@ stem's frame exactly, instead of guessing one or two quanta (round-1 finding D1- - *Publish an applied-revision watermark and complete edits asynchronously* (#1314): the record module this generalizes. - *Report held source blocks apart from underruns* (#1318): the held counter this reports. + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Moved anchors.** Each fact in Context still holds; only the lines moved. + +- `begin_block_at` is `crates/source/src/lib.rs:1115` (doc `:1106-1114`). It and + `begin_block_play` (`:1135`) now take a private `SeekClock` (`:1030-1038`, the enum at `:1031`). D1's rename is + `SeekClock::At`'s field and `begin_block_at`'s parameter. +- `observe_seek_at_block_boundary` is `:1323-1365`. The acquire step can re-observe a command + inside `begin_block_play` (`acquire_current_block`, `:1394-1440`, #1274 MINOR-3). D3's + publication at the end of `begin_block_play` sees both. +- The graph driver's block entry is `:1752-1785`; the call is `:1772`. +- The producer flips `active_generation` in `try_seek` at `:778` (`try_seek` `:753-787`). The + C ABI submit report copies it at `crates/capi/src/ffi.rs:501`. +- `SourceResourceReport` is `:209-236`; `resource_report` is `:490-546`. The row-sum test of + gate 6 is `report_separates_session_pcm_from_source_overhead` (`:2056`). +- `prepare_seek` is still `:1049`. The carry's `mem::swap` is `:1931` (`adopt_sources`, + `:1902-1938`). +- `SourceControlSet::seek` is `crates/host-core/src/source.rs:200`, `seek_at` `:223`, its doc + `:209-222`. +- `newest_providers` is `crates/control-plane/src/control.rs:2557`, `newest_providers_mut` + `:2578`. `SessionState::seek` is `:3003`, `seek_at` `:3028`. +- The header's anchor text is `crates/capi/include/miso_engine_v1.h:204-216` (the clock clause + `:205-207`). The session-thread table is `:18-22`. + +**Dependencies on GitHub.** + +- #1314 is closed. `crates/engine/src/realtime/watermark.rs` is on this tree. +- #1309 is closed. `crates/control-plane/src/control.rs` is correct in Authorized paths. +- #1318 is open. It is in the batch-15/16 list. +- #1348 is closed. Its service step exists (`SessionState::service`, `control.rs:1486`). +- Downstream, not dependencies: #1350, #1293, #1332, #1387, #1537, #1640 and #1641 (open). #1641 + plans a seek-report row for the timeline, which has no ring. D2's record shape must not stop a + later non-ring owner from publishing the same words. + +**Decisions.** + +- D1, D3, D4, D5, D7 and D8 hold against the code. +- D6 and #1348. The header (`:148-153`) says every session call that passes its argument checks + runs the service step first. Now that #1348 has landed, the new call follows that rule: + `SessionState::seek_report` calls `self.service()` first, as `seek` does, and the header's + thread table and service paragraph list the new call. This applies #1348 D2. It does not change + D6's results. +- D6 needs a re-export in `crates/capi/src/runtime/mod.rs` and `crates/control-plane/src/lib.rs`. + Add both to Authorized paths for re-exports only. +- D6's feature bit: the mask is 255 today (`MISO_ENGINE_V1_FEATURE_SERVICE` is 128). The rule + "next free bit when it merges, by symbol" stands. +- **B15-R1, root decision under owner delegation (2026-10-15): one shared seqlock module, `crates/engine/src/realtime/seqlock.rs`. + #1316 lands first and creates it. #1482 D3 later moves `observe.rs` onto it.** (D2 against + #1482 D3.) D2 puts the generic part in `seqlock.rs`, not in `watermark.rs`, and `watermark.rs` + uses it. Add `crates/engine/src/realtime/seqlock.rs` (new) to Authorized paths. The finding + follows. D2 makes #1314's record a generic `PublishedWords` in `watermark.rs`. *Attribute each claim's revisions exactly in + the watermark and share one seqlock* (#1482, open, not in the batch-15/16 list) D3 puts the + seqlock protocol in a new `crates/engine/src/realtime/seqlock.rs` that `observe.rs` and + `watermark.rs` both use. Done in either order without a rule, the two slices build two shared + abstractions or a third copy of the seqlock. Proposed resolution: one shared seqlock module. + If #1482 lands first, D2 builds the seek report on `seqlock.rs` and leaves `watermark.rs` + untouched. If this slice lands first, D2 puts the generic part in `seqlock.rs` (added to + Authorized paths) and #1482 D3 then moves `observe.rs` onto it. Root picks the order. +- **B15-R2, root decision under owner delegation (2026-10-15): keep D6 as a direct C export, `miso_engine_v1_source_seek_report`.** + (D6 against decision 17 C13.) The logic is one core function, + `SourceControlSet::seek_report`, behind both hosts. Source submission, seeks and the seek report + are transport under decision 17 C1, so their direct exports are not C13 exceptions. Decision 17's + C13 line records this (`docs/rulings/one-control-abstraction-2026-10-13.md`). The finding + follows. C13 says a new feature gets "its C ABI + form (a protocol message through `miso_engine_v1_submit_command`, no new C export)". D6 adds + a new C export, `miso_engine_v1_source_seek_report`. Root's decision-17 amendments to #1293 kept + that slice's browser seek-report export (#1293 D6), so root seems to treat source seeks and + their reports as host transport (C1: "frame staging"), like `seek` and `seek_at`. Proposed + resolution: keep D6 as a direct export. Record that source submission, seeks and the seek + report are transport under C1, with one core function (`SourceControlSet::seek_report`) behind + both hosts. The alternative is a protocol message, which would need a new message ID and a + source-ID field in the protocol. + +**Gates and commands.** Every named script, package and feature exists. `crates/source/tests/` +holds `randomized.rs` and `seek_schedule_model.rs`. Gate 6's "`crates/source` resource test" is +the unit test named above. Moved pins from the new per-ring row can reach +`crates/capi/tests/resource_lifecycle.rs` and `hosts/host-web/tests/browser-v1/expected.json`. +Re-pin from the measurement, with the reason. + +**Realtime-path edits.** D2 edits `crates/engine/src/realtime/watermark.rs` and `mod.rs` and adds +the new `seqlock.rs` (B15-R1). D3 adds a render-side publication in the source consumer. That slice gets +one attempt. diff --git a/.github/ISSUE_SPECS/1317-document-the-seek-contract-and-the-c-abi-growth-rule-in-the-header.md b/.github/ISSUE_SPECS/1317-document-the-seek-contract-and-the-c-abi-growth-rule-in-the-header.md index e93ee425c..9e37f6ac5 100644 --- a/.github/ISSUE_SPECS/1317-document-the-seek-contract-and-the-c-abi-growth-rule-in-the-header.md +++ b/.github/ISSUE_SPECS/1317-document-the-seek-contract-and-the-c-abi-growth-rule-in-the-header.md @@ -117,3 +117,76 @@ test stops modelling the `==` comparison it tells hosts never to write. - *Anchor every seek on the plan's source-read clock* (#1316): the clock name and the seek report the text documents. - *Report held source blocks apart from underruns* (#1318): the held reporting the text documents. + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Moved anchors.** The header grew by about 110 lines (the live-edit, solo, response-preview, +completion, service and engine-memory paragraphs). Each behaviour in Context still holds; only +the lines moved. + +- D1 replaces the paragraph "Starting an added stem in time" at + `crates/capi/include/miso_engine_v1.h:204-216` (was `:96-108`). Its clock clause is `:205-207`; + the false clause that #1318 deletes is `:214-215`. +- D2 goes beside `MISO_ENGINE_V1_ABI_VERSION` at `:219` (was `:111`). The feature bits are + `:244-252`; the mask is 255. +- Displacement is the paragraph "Sources across a structural transaction" at `:193-202` (was + `:85-94`). +- Clock origin: a new plan's clock is 0 at `crates/engine/src/realtime/plan.rs:834`; + `render_contiguous` is `:1172-1185`; a swapped-in plan continues the clock at + `crates/engine/src/realtime/plan_exchange.rs:779`. Anchors must be quantum multiples at + `crates/source/src/lib.rs:771-775`. +- Held: `begin_block_at` doc `crates/source/src/lib.rs:1106-1114`. Replacement: + `observe_seek_at_block_boundary`, `:1324-1346`. In flight: `try_seek`'s push and its + `Backpressure`, `:776-786`; the string is still `crates/host-core/src/source.rs:99`. +- Far past: `:1350-1365`. The region-end tests are `:2264-2310` + (`a_late_anchored_seek_past_the_region_end_is_the_end_of_region` and the on-time short-block + case). +- Rate: still `crates/host-core/src/source.rs:167-172`. +- Growth rule: `docs/C_ABI_V1_QUALIFICATION.md:112-113` is unchanged. The `==` model in + `crates/capi/tests/c/abi_smoke.c` is now `:221` (the block is `:215-223`). The Rust exact-mask + tests are `version_and_capabilities_are_exact` (`crates/capi/src/ffi.rs:1558`) and + `crates/capi/src/abi.rs:550-554`. + +**Code notes for the implementer.** + +- The smoke test also pins header constants at compile time (`abi_smoke.c:22-27`, for example + `ABI_ASSERT(MISO_ENGINE_V1_FEATURE_MASK == 255)`). They pin the header, not the library's + report, and `scripts/check-capi-abi.sh --self-test` relies on the fixtures' pins (`:87-88`). + D4 covers the runtime check only. Keep the static pins. +- Gate 2: the checker's self-test mutates the staged header and the `nm` output. It cannot + change the library's capability report. Use the "local patch, recorded in the PR" route. +- Since #1348 the seek calls run the service step first (header `:148-153`). D1's `seek` + paragraph should not contradict that paragraph. A cross-reference is enough. +- "Equal to the render clock in this version" stays true after #1396: only a warm successor + (#1355) makes the offset nonzero in production. #1355 must amend this text when it lands. + Record that in #1355. + +**Dependencies on GitHub.** #1316 and #1318 are open. Both are in the batch-15/16 list. #1275, +#1314 and #1348 are closed. No dependency is outside the batch. + +**Decisions.** + +- D1, D3 and D4 hold against the code and decision 17. +- **B15-R3, root decision under owner delegation (2026-10-15): D2 keeps its text and adds two sentences: "A feature added as a + protocol message is announced by its protocol capability flag in the CAPABILITIES reply, not by + a feature bit. The registry, not this rule, governs protocol flags."** (D2 against decision + 17 C13.) The header states the two sentences and points to the registry for the rest. The + finding follows. Since decision 17, new features reach + the C ABI as protocol messages through `miso_engine_v1_submit_command`, with no new export + (C13). Each is announced by a protocol capability flag in the `CAPABILITIES` reply, not by a C + feature bit (header `:105`, `RESPONSE_GET`, "capability flag bit 14"). The protocol decoder + refuses an unknown capability flag (`docs/CONTROL_PROTOCOL_REGISTRY.md:13`), which is the + opposite of D2's "tolerate unknown bits" for `feature_mask`. D2 says nothing about protocol + flags. So a host author who reads only D2 can apply the C rule to protocol flags. Proposed + resolution: D2 keeps its text and adds two sentences: "A feature added as a protocol message + is announced by its protocol capability flag in the CAPABILITIES reply, not by a feature bit. + The registry, not this rule, governs protocol flags." Root decides whether the header states + the protocol rule or points to the registry. + +**Gates and commands.** `scripts/check-capi-abi.sh`, its `--self-test`, both C fixtures and +`scripts/check-workspace-policy.sh` exist. `cargo test --locked -p capi` runs both Rust +exact-mask tests. + +**Realtime-path edits.** None. This slice is header text, one doc and one C test. diff --git a/.github/ISSUE_SPECS/1318-report-held-source-blocks-apart-from-underruns.md b/.github/ISSUE_SPECS/1318-report-held-source-blocks-apart-from-underruns.md index 23d789965..ecad3f826 100644 --- a/.github/ISSUE_SPECS/1318-report-held-source-blocks-apart-from-underruns.md +++ b/.github/ISSUE_SPECS/1318-report-held-source-blocks-apart-from-underruns.md @@ -107,3 +107,53 @@ spectrum windows carry. ## Dependencies - none + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Moved anchors.** Each fact in Context still holds; only the lines moved. + +- `held_seek` is still `crates/source/src/lib.rs:1018`; `HeldSeek` is `:1022-1027`. + `observe_seek_at_block_boundary` is `:1323-1365`. The `begin_block_at` doc is `:1106-1114`. +- The missing-PCM branch of `begin_block_play` is `:1154-1178` (the function is `:1135-1190`). + `SourceReadReport` is `:404-423`, `SourceConsumerTelemetry` `:439-448`, `telemetry()` + `:1290-1299`. +- The graph source driver's block entry is `:1752-1785`. The vacant-entry branch is `:1765-1769`; + the fold of `report.underrun_event` is `:1773`. +- `copy_after_disarm_telemetry` is `:1854-1875`. It still writes five slots per source. +- `GraphObservationValidity` is `crates/graph/src/lib.rs:2927-2940`. +- Spectrum windows moved into the core service (#1591): the flag is retained in + `crates/host-core/src/spectrum_service.rs` (`:496`, `:517`, `:1206`), not `spectrum.rs`. The + browser copies it into its window record at `hosts/host-web/src/ffi.rs:1048` and `:1174`. No + code changes there: a held block simply stops setting the flag. +- The header clause is `crates/capi/include/miso_engine_v1.h:214-215` ("and those blocks / count + as source underruns"). +- The host-core facade's `seek_at` doc is `crates/host-core/src/source.rs:209-222`; "reports an + underrun" is `:222`. +- Tests: `prepare_seek_holds_an_anchored_seek` is `:2245`. The anchored-seek tests are now + `:2160-2445`. The held-block underrun assertion that this slice changes is `:2364` + (`assert_eq!(consumer.underrun_events, u64::from(anchored))` in + `a_seek_whose_pcm_arrives_inside_the_block_window_keeps_its_pcm`). The region-end tests are + `:2264-2310`. `graph_driver_forwards_underrun_and_seek_generation_facts` is `:2748`. + +**Decisions.** + +- D1-D6 hold against the code and decision 17. +- D1, made precise. `begin_block_play`'s acquire step can re-observe a command and set or apply + a held seek (`acquire_current_block`, `:1394-1440`, #1274 MINOR-3). So read `held_seek` in the + missing-PCM branch, after the acquire. That is the "after this block's observe step" of D1. +- The missing-PCM branch also advances `next_frame` by one quantum while held. D2 keeps that, so + no bit moves. + +**Dependencies on GitHub.** None. Downstream and open: #1316, #1317, #1320, #1387, #1537 and +#1640. `crates/source/src/lib.rs` is a hot file: #1640's hazard note orders stream B's source +slices (#1316, #1318, #1320) before it. + +**Gates and commands.** Gate 4's tests exist (`crates/host-core/tests/successor_swap.rs`, +`anchored_swap_run`, `:629-830`). That file also compares spectrum windows' `source_underrun` +between a run and its reference (`:9438`). Both runs hold the same way, so it stays green. Every +named command exists. + +**Realtime-path edits.** None under `crates/engine/src/realtime`. The change is on the source +consumer's render path in `crates/source/src/lib.rs`. diff --git a/.github/ISSUE_SPECS/1319-test-held-seeks-across-swaps-and-supersession-and-add-a-seek-to-audit-capi.md b/.github/ISSUE_SPECS/1319-test-held-seeks-across-swaps-and-supersession-and-add-a-seek-to-audit-capi.md index f31c386bc..405ce85f7 100644 --- a/.github/ISSUE_SPECS/1319-test-held-seeks-across-swaps-and-supersession-and-add-a-seek-to-audit-capi.md +++ b/.github/ISSUE_SPECS/1319-test-held-seeks-across-swaps-and-supersession-and-add-a-seek-to-audit-capi.md @@ -130,3 +130,80 @@ lateness assertion, test-only untimed reads) are #1350; see Non-goals. withdrawal-outcome counters. - *Add miso_engine_v1_service for bounded control work between edits* (#1348), for D3's promotion before T2. + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Moved anchors.** Each fact in Context still holds; only the lines moved. + +- The decision-15 handoff files are unchanged: the adversary's "MINOR — untested ack path" is + `docs/handoffs/decision-15-2026-10-05/PLAN-2026-10-05-adversary-round1.md:460`, and the plan + item is `PLAN-2026-10-05-agreed.md:191`. +- The host-core helper's assertion is `crates/host-core/tests/successor_swap.rs:638` + (`anchored_swap_run`, `:631`). +- In `crates/capi/src/runtime/tests.rs`: `an_added_source_starts_at_its_anchor_at` is + `:1333-1455`, `an_added_c_abi_source_starts_at_its_anchored_render_sample` `:1456`, + `assert_bit_identical` `:1513`. `test_plan_carry_counts` is `crates/capi/src/ffi.rs:1256`. +- The carry's `mem::swap` is `crates/source/src/lib.rs:1931` (`adopt_sources`, `:1902-1938`). + `held_seek` is still `:1018`. +- `SessionState::seek_at` is `crates/control-plane/src/control.rs:3028-3041`; it runs the + #1348 service step first, then calls `newest_providers_mut` (`:2578`). `adopt_persisting` is + still `crates/host-core/src/source.rs:286`; the control plane calls it at `control.rs:1846-1849`. +- A second structural transaction while a candidate waits is still refused, now at + `control.rs:1742-1744` (`pending_providers` is a `Vec`, but this check keeps it at one). +- `tools/audit/src/capi.rs`: `CALLS_BEFORE_TRANSACTION` is `:52`, `apply_structural_transaction` + `:178-211`, `submit_source_chunk` `:213-251`, the run loop `:480-560`. The audit also commits a + live edit every 64 calls (`LIVE_EDIT_PERIOD`, `:39`, #1258). CI runs it at + `.github/workflows/qualification.yml:777-785` and validates the JSON from `:786`. + +**Code notes for the implementer.** + +- D2 and D3: decision 17 and its slices make more values live (trim, polarity, input filters, + sends, VCA). A value edit can take the live path and publish no candidate. So T1 and T2 must be + structural edits (insert quality, or adding or removing an insert or a track). Assert edit-path + 3 (rebuild) in each response, so that a later live flip turns the test red, not vacuous. +- D4: `submit_source_chunk` hard-codes generation 1 (`:220`). Give it a generation argument. The + structural transaction already submits the generation-1 block at frame 128 before the seek, so + the order stays right. The fixture source has 48000 frames, so frame `A = 512` is in its region. +- D4 runs the seek and the submit outside every render scope, like the transaction. The + seek call's service step is control work there too. + +**Dependencies on GitHub.** + +- #1348 is closed. D3's promotion runs in `SessionState::service` (`control.rs:1486`). +- #1343 is closed: the engine's withdrawal primitive exists + (`PlanPublisher::withdraw`, `crates/engine/src/realtime/plan_exchange.rs:552`). The control + plane does not use it yet. +- *Supersede an unadopted candidate plan by compare-and-swap* (#1310) is open and is not in the + batch-15/16 list. #1310 depends on *Prepare a successor across a withdrawn candidate plan* + (#1344), also open and not in the list. D2, D3 and the withdrawal counters need both. +- Batch order (root, 2026-10-15): batch 15a is the controls lane, #1344, #1398 and #1310. + #1324 left batch 15a (root ruling, 2026-10-15); it lands in a later batch after #1288, #1391, + #1363 and #1325. + The rest of batch 15 follows, so #1310 and #1344 land before this slice. + +**Decisions.** + +- D1 and D4 hold against the code and decision 17. +- D2 and D3 hold as designs, but they cannot run before #1310. +- **B15-R4, root decision under owner delegation (2026-10-15): keep #1319 whole. #1310 and #1344 land first, in batch 15a.** + (Scope against #1310.) No successor issue is split off. #1319 closes on D1-D4 after #1310, + in the rest of batch 15. + The finding follows. Deliverable 1 says D2 and D3 come "after #1310 + has landed", and #1310 is outside the batch-15/16 list. Proposed resolution: split D2 and D3 + (with gate 3's D2/D3 records) into a successor issue that depends on #1310. #1319 then closes on + D1 and D4 in batch 15. The alternative is to keep #1319 open across batches, which the + "smallest closable slice" rule in AGENTS.md argues against. + +**Gates and commands.** Every named command, package and file exists. Gate 4's +`cargo test --locked --release -p audit` is also run by CI (`qualification.yml:775-776`). + +**Realtime-path edits.** None under `crates/engine/src/realtime`. D4 adds a held seek to the +audited render, so a defect on that path shows up as an audit violation, not as an edit here. + +## Amendment (controls coordinator, batch 15a fold, 2026-10-16) + +- **Batch placement.** "#1310 ... is not in the batch-15/16 list" and "#1344, also open and not in + the list" (Dependencies on GitHub) are void. #1344 and #1310 are in batch 15a (with #1398), which + lands before this slice. diff --git a/.github/ISSUE_SPECS/1320-let-a-source-consumer-check-and-replay-its-next-blocks-for-a-prime.md b/.github/ISSUE_SPECS/1320-let-a-source-consumer-check-and-replay-its-next-blocks-for-a-prime.md index 2cdb08d06..b49ca8b47 100644 --- a/.github/ISSUE_SPECS/1320-let-a-source-consumer-check-and-replay-its-next-blocks-for-a-prime.md +++ b/.github/ISSUE_SPECS/1320-let-a-source-consumer-check-and-replay-its-next-blocks-for-a-prime.md @@ -199,3 +199,60 @@ and `k = 3`. `crates/source/src/lib.rs` land first. - *Report held source blocks apart from underruns* (#1318). - *Test held seeks across swaps and supersession, and add a seek to audit capi* (#1319). + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Anchors.** +- Unchanged: `TransferBlock` (`crates/source/src/lib.rs:450`), `prepare_at_source_frame` + (`:568`), `RETAINED_TRANSFER_BLOCKS` (`:654`), `validate_submission_metadata` (`:924`), + `PcmSourceConsumer` (`:993`), `held_seek` (`:1018`), `begin_block_at` (`:1115`), + `begin_block_observe` (`:1127`), `begin_block_play` (`:1135`) and + `observe_seek_at_block_boundary` (`:1323`). +- Moved: `acquire_current_block` is `:1394`. Its early return is `:1395-1397`, its pop of an + unobserved generation `:1408-1411`, and the observe after it `:1431`. It pops at most + `transfer_block_count` blocks per call (`:1398-1401`). +- The underrun branch is `:1154-1177`; the `next_frame` advance is still `:1164-1168`. The + short-block zeroing in `play` is `:1518-1527`. +- Graph driver: `begin_block` is `:1752`, its `consumer.begin_block_at` `:1772`, and + `played_planes` `:1844`. +- SPSC (`crates/engine/src/realtime/spsc.rs`): `Consumer` `:217`, `bounded_spsc_move` `:250`, + `available_at_entry` `:421`, `try_pop` `:441`. The consumer's realtime region is `:296-460`. +- `SeekClock` (`:1031`) already has `Hold`, and `prepare_seek` (`:1049`) uses it. D3 needs no new + clock. +- The loom CI step is now `.github/workflows/qualification.yml:734-735`. The command is unchanged. + +**Already done.** `bench-support` is a `crates/source` dev-dependency +(`crates/source/Cargo.toml:18`). Deliverable 4 is done. `crates/source/Cargo.toml` and +`Cargo.lock` need no change. + +**What landed around this slice.** +- #1311 (closed) added the plan-side hook. `PlanAdoption::Primed { not_before, lead_blocks }` + (`spsc.rs:532`) asks `PreparedPlanExecutor::prime_ready(block_start, lead_blocks)` + (`crates/engine/src/realtime/plan.rs:349`, default `false`). This slice's + `PcmSourceConsumer::prime_ready` is the per-ring check that #1355 calls from that hook. + `block_start` is a render sample, and #1355 adds the source-read offset `O` (#1396). +- Since #1639 (code landed; issue still open), the graph's block entry reads the jump lengths + (`crates/graph/src/lib.rs:4824-4826`) and then calls `source_set.begin_block` (`:4844`). Where + the prime goes in that entry is still #1355's. +- #1681: a ring has one or two channels. No effect here. + +**Decisions.** D1-D5 hold against the code. Decision 17 changes nothing here: source transport +is not a control. + +**Authorized paths and gates.** +- `scripts/check-realtime-policy.sh` is gone. Gate 7's `cargo run ... -p realtime-policy` still + works. If D1-D3 add a marked region instead of using an existing one, `marked_regions_floor` + (`tools/realtime-policy/src/policy.rs:227`) goes up. That file joins the paths for that line + only. +- The browser's resource record carries `source_overhead_bytes` (`hosts/host-web/src/lib.rs:1252`). + D4's flag changes it. `hosts/host-web/src/tests.rs` and `hosts/host-web/tests/` join the paths + for any assertion of that row. +- `crates/source/tests/prime.rs` is new. Every other named file and script exists. + +**Later work, not this slice.** #1640 (open) factors the seek state machine after this slice. +#1641 (open) adds a timeline consumer that carries like a source. Whether a warm adoption primes +the timeline consumer is for #1355 or #1641 to decide. + +**Dependencies.** #1316, #1318 and #1319 are open and in the batch. Nothing outside the batch. diff --git a/.github/ISSUE_SPECS/1323-reset-latency-floors-at-a-host-declared-discontinuity.md b/.github/ISSUE_SPECS/1323-reset-latency-floors-at-a-host-declared-discontinuity.md index 1b6911702..5feb98b4f 100644 --- a/.github/ISSUE_SPECS/1323-reset-latency-floors-at-a-host-declared-discontinuity.md +++ b/.github/ISSUE_SPECS/1323-reset-latency-floors-at-a-host-declared-discontinuity.md @@ -227,3 +227,108 @@ The discontinuity successor publishes the *committed* revision on a fresh gate, revision 0 would be refused by `stamp(0)` (and by host-core's entry check, `host.successor.revision`, which requires `1 <= revision`). Root decides how this slice handles a session at revision 0 when #1323 is next. + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Moved anchors.** +- "render.time.discontinuity" is a render diagnostic, now at + `crates/capi/include/miso_engine_v1.h:182`. The seek prototypes are `:420-430`. +- The last feature bit is now `MISO_ENGINE_V1_FEATURE_SERVICE` (`1 << 7`, + `crates/capi/src/abi.rs:70`, header `:251`). The mask is `abi.rs:72-79` and header `:252`. #1316 + and #1642 also add bits. D1's "next free bit when this slice merges" still governs. +- The mask test at `crates/capi/src/abi.rs:554` asserts `FEATURE_MASK == 0xff`, a literal. Gate 6 + replaces that assertion in this slice. +- `SuccessorBase` is `crates/host-core/src/prepare.rs:1070-1097`. +- The control plane's `prepare_runtime` call is `crates/control-plane/src/control.rs:1662`, with + its `SuccessorBase` literal at `:1670`. + +**`SuccessorBase::new` does not exist.** #1277 closed without it, so this slice adds it. +- `SuccessorBase` now has six fields: `inventory`, `committed`, `revision`, `edit_ramps` (#1394), + `monitor` (#1559) and `observation` (#1584). +- Each new field has a neutral value: `EditRamps::new()` is `const` + (`crates/session/src/edit_ramp.rs:102`), `MonitorBase::NONE` (`prepare.rs:1125`) and + `NO_OBSERVATION` (`prepare.rs:1100`). `new` sets those and `discontinuity: false`. + `edit_ramps` is a reference, so `new` needs a static neutral `EditRamps` beside + `NO_OBSERVATION`. + +**Construction sites at `228fca26c`.** These replace D3's list: +- `crates/control-plane/src/control.rs:1670`; +- `crates/host-core/tests/support/successor.rs:98`; +- `crates/host-core/tests/successor_swap.rs:1043`, `:2663`, `:3926`, `:3950`, `:3980`, `:7159`, + `:9331`; +- `crates/host-core/tests/browser_successor.rs:284`; +- `crates/host-core/tests/effect_observation.rs:544`; +- `crates/capi/tests/resource_lifecycle.rs:1805`; +- `hosts/host-web/src/tests.rs:5938`. + +`crates/host-core/tests/withdrawn_successor.rs` does not exist (#1344 is open). Add to the +authorized paths, for the construction sites only: `crates/host-core/tests/browser_successor.rs`, +`crates/host-core/tests/effect_observation.rs` and `hosts/host-web/src/tests.rs`. + +**Dependencies.** +- #1285, #1317 and #1396 are in the batch. +- #1309 is closed: the control plane is `crates/control-plane/src/`. +- #1311 is closed: `PlanAdoption::Primed` is `crates/engine/src/realtime/spsc.rs:532`. +- #1314 is closed: the outcome flags are header `:255-256`. +- #1310 is open and not in batch 15/16. D2's supersede, D4's completion and gate 5 need it. The + engine's `withdraw` exists (`crates/engine/src/realtime/plan_exchange.rs:552`, #1343), but #1310's + compare-and-swap and its D2, D5 and D6 do not. #1310 also waits on #1344 and #1398, both open. +- #1403 (open) adds `PrimedCandidate` later. It is not a dependency. +- Batch order (root, 2026-10-15): batch 15a is the controls lane, #1344, #1398 and #1310. + #1324 left batch 15a (root ruling, 2026-10-15); it lands in a later batch after #1288, #1391, + #1363 and #1325. + The rest of batch 15 follows. This slice is in the rest (B15-R5). + +**B15-R5, root decision under owner delegation (2026-10-15): order #1310, #1344 and #1398 before #1323. No placeholder +refusal.** #1344, #1398 and #1310 are in batch 15a; #1323 follows in the rest of batch 15. +The finding follows. D2 supersedes a pending candidate +through #1310, which is not planned for this batch. A declaration while a candidate is pending has +no rule without it. Proposed: order #1310, #1344 and #1398 before #1323. Do not ship a placeholder, +such as a typed backpressure refusal while a candidate is pending. + +**B15-R6, root decision under owner delegation (2026-10-15): no revision-0 special case. At revision 0 the call returns OK and +publishes nothing.** D2 adds that sentence. Gate 3 adds a revision-0 case: a declaration +on a host at revision 0 returns OK, publishes no candidate, and leaves the next render and the +reported latency unchanged. The finding follows. Proposed answer: no special +case. Floors and source-read offsets come only from a successor. A successor needs +`1 <= revision` (`host.successor.revision`, `prepare.rs:1915`, `:3200`), and the committed revision +only grows. So when D2's test holds, the committed revision is at least 1. At revision 0 the call +returns OK and publishes nothing. Add that sentence to D2 and a revision-0 case to gate 3. + +**B15-R7, root decision under owner delegation (2026-10-15): this slice also adds `miso_engine_web_v1_declare_discontinuity(handle) +-> u32`.** It forwards to the same control-plane function. It adds the export-list mirrors +of #1293 D7 and a native host-web gate equal to gate 2 (in `hosts/host-web/src/tests.rs`). +#1643 calls it before its seek. Add `hosts/host-web/src/ffi.rs`, `hosts/host-web/src/lib.rs` and +the mirror files of #1293 D7 to Authorized paths. The finding follows. D5 leaves the browser to #1332 and +#1296. Since #1381a the browser runs `control_plane::SessionState` in its one instance +(`hosts/host-web/src/lib.rs:6-16`), so the function is reachable there now. Only a browser entry +is missing. Decision 17 C1 says a behaviour on one host only is a defect. #1642 makes a C ABI host +declare before a session seek, but #1643 (the browser session seek) does not, so browser floors +would never reset. Proposed: this slice also adds a control-half export +`miso_engine_web_v1_declare_discontinuity(handle) -> u32` that forwards to the same control-plane +function. It adds the export-list mirrors of #1293 D7 and a host-web native gate equal to gate 2. +#1643 then calls it before its seek. Alternative: carry the declaration as a protocol frame +through `submit_command` and `control_command`, C13's form with no new export. But D1, #1317 and +#1642 already name the C export. + +**Other notes.** +- `TRANSPORT_SET` now carries the state only (#1635, code landed; issue still open). A transport + stop is not a declaration. The host calls D1 itself. +- #1641 (open) gives every plan a timeline that "carries like a source". D2's successor "carries + only the source consumers". If #1641 lands first, the timeline consumer carries too, or the + playhead and stored automation (#1647) restart at the declaration. Whichever lands second adds + it to the discontinuity join. +- Decisions D1-D4 otherwise hold against the code and decision 17. The declaration is not a + control. + +**Gates and commands.** +- The named files exist, except `withdrawn_successor.rs` (#1344, new there). +- `tools/audit/src/capi.rs` exists, and `scripts/check-capi-abi.sh` has `--self-test`. +- Gate 7's commands are valid. Its features exist. + +## Amendment (controls coordinator, batch 15a fold, 2026-10-16) + +- **Batch placement.** "#1310 is open and not in batch 15/16" (Dependencies) is void. #1310, #1344 + and #1398 are batch 15a, which lands before this slice (B15-R5). diff --git a/.github/ISSUE_SPECS/1324-duck-swap-a-strip-whose-state-cannot-continue-across-a-plan-swap.md b/.github/ISSUE_SPECS/1324-duck-swap-a-strip-whose-state-cannot-continue-across-a-plan-swap.md index 06170d71d..f0fa1fed7 100644 --- a/.github/ISSUE_SPECS/1324-duck-swap-a-strip-whose-state-cannot-continue-across-a-plan-swap.md +++ b/.github/ISSUE_SPECS/1324-duck-swap-a-strip-whose-state-cannot-continue-across-a-plan-swap.md @@ -312,3 +312,73 @@ Through the exported C entries, quantum 128, 48 kHz, `N = 2000`, single-threaded between a bank and a per-node instance* (#1281) and *Carry per-node effect instances across a plan swap* (#1282): gates 5, 7(b) and 9 assume that an untouched insert carries. Until these land, a strip that holds an insert is in `restarted_strips()` and is ducked (root, 2026-10-12). + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `e2f6cfb24`. + +**Batch order (root, 2026-10-15).** Batch 15a is the controls lane: #1344, #1398, #1310, in +that order. The rest of batch 15 follows. This slice is not in batch 15a; see the root ruling below. + +**Moved anchors.** +- `SuccessorBase` is `crates/host-core/src/prepare.rs:1070-1097`, with six fields: `inventory`, + `committed`, `revision`, `edit_ramps` (#1394), `monitor` (#1559) and `observation` (#1584). + `prepare_host_runtime_with_live_lanes_successor` is `:1311`. `restarted_strips()` is + `:686-690`; its doc already names this issue as its consumer. The restart set is built at + `:3101-3105` from the strip, effect and strip-delay joins. +- The classifier no longer has `Prepared` and `PreparedBypass`. Decision 17's control rows + replaced them: `LiveRebuild` is `prepare.rs:4230` (`Structure`, `Domain`, + `RebuildOnly(reason)`, `AutomationTarget`), and the steps are documented at + `crates/host-core/src/live_delta.rs:259-280`. A non-`Block` parameter and a rebuild-only builtin + are `RebuildOnly` with the row's reason. Each trigger of gate 3 still rebuilds: insert add, + remove, order and quality are `Structure`; `delay_samples` is a rebuild-only row (the row at + `crates/session/src/visit.rs:122`, field 5; `crates/host-core/src/control_provider.rs:3108-3113` + finds it as "a rebuild-only value"); the limiter's `lookahead_ms` is `AutomationRate::None` with a + prepared reason (`crates/true-peak-limiter/src/lib.rs:212-228`). +- `SendTap` is `crates/session/src/model.rs:957-972`. +- The sidechain edge is built at `crates/graph-compiler/src/compile.rs:377-384`; the tap's stage + is `crates/graph-compiler/src/ids.rs:198` and `:210-220`. +- `render_sample` is `crates/control-plane/src/plan.rs:13`, read at `:39`. +- The compressor's zero latency is `crates/compressor/src/lib.rs:306`. + +**Code that does not exist yet.** `crates/host-core/src/transition.rs`, +`plan_strip_transition`, the duck overlay and `crates/capi/tests/strip_transitions.rs` are +#1325's. The arm entry point is #1288's. `SuccessorBase::forced_restart` is #1397's; until #1397 +lands, D1's forced-restart clause has nothing to read. + +**Dependencies on GitHub.** +- Closed: #1277, #1280, #1281 and #1282. So per-node and moving inserts carry now. The hazard + and the dependency note on gates 5, 7(b) and 9 no longer apply: an untouched insert carries. +- Open, in batch 15a before this slice: #1310. +- Open and not in batch 15: #1288 (arm), #1325 (duck, `S`, overlay, `transition.rs`), #1363 (route + arm, D3) and #1391 (pre-fader route lanes). D1-D5 and every gate need all four. +- Forward references, not dependencies: #1397, #1354, #1287, #1402 and #1392 (open). +- **Root decision under owner delegation (2026-10-15; not one of B15-R1 to B15-R9): #1324 leaves batch 15a. It lands in a later batch after #1288, + #1391, #1363 and #1325, in that order. Batch 15a is #1344, #1398, #1310.** + The finding follows. It needs #1288, #1325, #1363 and #1391, + which are open and outside batch 15. None of their code is on `main`. Proposed: move #1324 out + of 15a to the batch that lands #1288, #1391, #1363 and #1325, in that order, with #1324 after + them. Batch 15a is then #1344, #1398 and #1310. #1285's batch-15 note (D3's duck-swap) then + keeps its fallback: until #1324 lands, a restarted strip starts at rest with no duck. The + alternative is to add the four to batch 15a before #1324. + +**Decisions.** +- D1-D7 hold against the code. Decision 17 does not change which strips restart: the restart set + comes from the carry joins, not from the classifier's rows. +- D3's "channels the model leaves unmuted": since #1559 a successor bakes the effective values + (`controls::effective`: the model, VCA mutes and the monitor overlay, C8). An armed strip must + stay muted when the effective value is muted, so a solo elsewhere does not fade it in. Read + D3 with "effective" for "model". #1288's arm entry point must take the same input. +- D5's structural edit during the window is #1310's supersession with #1325 D6's overlay. #1310's + base (its batch-15 amendment) keeps P0's model; the overlay mutes go on top of it. + +**Authorized paths.** Deliverable 2 calls #1325's path from the control plane. Add +`crates/control-plane/src/control.rs` (the call only). The control plane is +`crates/control-plane/src/` since #1309. + +**Gates and commands.** The commands, packages and scripts exist, except +`crates/capi/tests/strip_transitions.rs`, which #1325 creates. `--test strip_transitions` fails +until then. + +**Realtime-path edits.** None planned under `crates/engine/src/realtime`. The fire and the ramped +mutes are #1288's and #1325's render-side work. diff --git a/.github/ISSUE_SPECS/1327-carry-meter-and-effect-observation-state-across-a-plan-swap.md b/.github/ISSUE_SPECS/1327-carry-meter-and-effect-observation-state-across-a-plan-swap.md deleted file mode 100644 index 21270c23d..000000000 --- a/.github/ISSUE_SPECS/1327-carry-meter-and-effect-observation-state-across-a-plan-swap.md +++ /dev/null @@ -1,425 +0,0 @@ -# Carry meter and effect observation state across a plan swap - -Stream A of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-14, D15-7). -Code anchors verified on `main` at `6fb211594`. - -This slice carries builtin meters and effect observation taps. The spectrum observer is *Carry -spectrum capture state across a plan swap* (#1395), which reuses D1-D4's rules for its own key -and its shared mode word. - -## Product outcome - -An app that reads a strip's level or an effect's gain reduction sees no reset when a track is -added or another strip is edited. A meter or observation tap is unchanged when its owner's key and -configuration are the same in both plans. Such an owner keeps its open -window, its counters and its sequence through a plan swap. The host's existing reader keeps -receiving, with no generation bump, no counted loss, and no window relabelled. This replaces -#1269's P10 ("restart at the swap"), which decision 15 rejects as a shortcut (D15-14). Only -changed or added owners start fresh. - -## Context - -- **Builtin meters.** - - A meter is a graph observer, `MeterObserver(MeterAccumulator)` - (`crates/builtins-compiler/src/lib.rs:4820`, impl `:4821`). - - `MeterAccumulator` (`crates/builtins/src/lib.rs:4553`) holds the window state (`start`, - `frames`, `sequence`, two `MeterLane`s and the cumulative counters) and the - `Producer`. - - The host reads through `MeterConsumer` (`builtins-compiler/src/lib.rs:289`), returned in - `HostLiveControlHandles::meters` (`crates/host-core/src/prepare.rs:445`) and requested by - `HostMeterRequest` (`:399`). - - A snapshot carries `reset_generation` and `window_sequence` (`builtins/src/lib.rs:4505-4518`). -- **Effect observation taps.** - - `ObservationLane` (`crates/effect-contract/src/live.rs:588`) is held per node by - `LiveControlEffect::observation` (`crates/graph/src/runtime.rs:1196`) and per bank lane by - `LiveControlEffectBankStage::observations` (`crates/rack/src/lib.rs:961`). - - Readers reach the host as `HostLiveControlHandles::effect_observations` (`prepare.rs:452`). -- **Spectrum** is #1395. `SpectrumCaptureObserver` (`crates/host-core/src/spectrum.rs:1441`) is a - graph observer too, so #1395 registers it in this slice's observer location table (D2). -- **The browser** bumps `meter_generation` and counts a loss when a snapshot's `reset_generation` - changes (`hosts/host-web/src/lib.rs:3545-3561`), which is what a swap to fresh meters causes. -- **The C ABI** has no strip meter, observation tap or spectrum capture. Its telemetry is the - output peak each render publishes (`crates/control-plane/src/plan.rs:261`, read by - `collect_render_activity`, `crates/control-plane/src/control.rs:625`). That is continuous across a - swap already, so this slice has nothing to carry there. -- The carry scaffold, the effect carries and the source-producer precedent - (`SourceControlSet::adopt_persisting`, `crates/host-core/src/source.rs:286`) are on `main` or come - from #1279-#1284. - -## Decisions frozen for this slice - -- **D1. Keys and rule (P1).** There are two kinds of owner here: - - A **meter** carries when its strip ID, tap, metric set and `MeterConfig` are equal in both - plans. - - An **observation tap** carries when its effect owner carried (#1279-#1282) and its tap - descriptor is the same. - - Nothing in them is a live value, so there is no retarget. A non-carried owner starts fresh and - is not a strip restart: it changes no audio. -- **D2. Move mode.** At the swap block, swap each carried observer's whole state between the plans, - its producer included. A reader that already holds the carried producer's consumer therefore - stays connected to the observer that renders on. The successor's fresh observer goes to the - retiring predecessor. - - The observer location table records each observer's owner kind and key, not its handle, so - #1395 adds the spectrum kind without changing the section. - - Within one kind, the table's entries are in the order of that kind's reader vector in - `HostLiveControlHandles` (`meters`, `effect_observations`), so an entry's index is its - reader's index. -- **D3. Carry record and reader pairing, at the swap block.** No reader is swapped on the control - side: a commit runs before adoption, and a candidate it names can be superseded (#1310) and - never adopted. - - The successor's table holds a preallocated carry record, one slot per entry. D2's section - writes it in the swap block: for each carried entry, `ObserverCarry { kind, successor, - predecessor }` (the two entry indices). The record is allocation-free and written once, since - a plan is adopted at most once. - - The engine gets the record type and one accessor: `ObserverCarry` and `ObserverKind` - (`Meter`, `Observation`; #1395 adds `Spectrum`) in `crates/engine/src/realtime/plan.rs`, a - `PreparedPlanExecutor::observer_carries(&self) -> &[ObserverCarry]` method whose default is - empty, and `PreparedRenderPlan::observer_carries()`. The graph executor (`impl PreparedPlanExecutor for - GraphExecutor`, `crates/graph/src/lib.rs:3124`) returns its table's record. - - host-core gives two pairing functions, `pair_carried_meters(successor: &mut [MeterConsumer], - predecessor: &mut [MeterConsumer], carries: &[ObserverCarry])` and - `pair_carried_observations` (same shape, over the observation handles). For each record of - its kind, it swaps `successor[s]` with `predecessor[p]`. It is allocation-free, drops nothing, - and may run on the render thread. - - A host whose readers live inside the plan pairs them in the swap block, from the plan's host - attachment hook. The browser is that host: *Swap and retire browser plans through the - Worker's service loop* (#1381) D3 calls `pair_carried_meters` (its meter vector is in host-core - order) and pairs observation handles through its own host-core-index-to-slot map, because the - browser keeps observations in gapped per-slot storage. After the swap block the successor holds - the carried readers, and the retiring predecessor holds the fresh ones. -- **D4. A warm successor carries by the same move.** A warm successor (*Grow latency during - playback by adopting a primed warm successor*, #1287) renders no block before its adoption at - `S` (D15-8 (round-5 amendment)). D2's section runs in that adoption block, as at any swap, so its - observers carry with no overlapping window and nothing published twice. This slice adds no other - mode. -- **D5. Both hosts.** Everything is in engine (the record type and accessor only), host-core, graph, rack, builtins and effect-contract. - - The browser's readers live in its plan's host attachment. #1381 D3 calls D3's pairing functions from the attachment hook in the swap block, so the browser has no other carry code. *Replace the running browser session in the Rust host* (#1290) gates the result: the browser's `meter_generation` does not advance across a structural apply that carries every meter. - - **Decision 17 amendment (2026-10-13).** The C ABI now has state to carry: since #1581 every C ABI plan holds one strip meter per strip, and since #1584 its effects' observation taps. Its readers live on the control thread, so #1587 calls D3's two pairing functions there, when the control plane promotes the adopted plan. This issue adds no C ABI code; its gates stay as written. -- **D6. A fresh meter joins the carried window grid.** A host that folds meters as one group - needs every meter's windows to match: the browser requires equal `window_sequence`, - `start_sample`, `end_sample` and `frames` across the group (`hosts/host-web/src/lib.rs:3485-3500`). - A fresh `MeterAccumulator` opens its first window at its first observation (`start: None`, - `crates/builtins/src/lib.rs:4683`), so an added strip's meter would sit off the grid forever. - So, in the swap block, a fresh meter (no carry record) whose `MeterConfig` equals a carried - meter's takes the open window position of the first such carried meter (lowest successor entry - index): its `start`, `frames` and `sequence`, with its lanes and cumulative counters at zero. - That is exactly what it would hold had it observed silence since the window opened, and the - strip did not exist before adoption. With no carried meter of that config, it starts as today. - It is a scalar copy in the move section: no allocation. - -## Deliverables - -1. D2 in `crates/builtins` (`MeterAccumulator` swap), `crates/builtins-compiler` (`MeterObserver`), - `crates/effect-contract/src/live.rs` (`ObservationLane`), `crates/rack` and `crates/graph` (the - observer location table and the program section). -2. D1 and D3's pairing functions in `crates/host-core/src/prepare.rs`. D3's record type, the - executor method and the plan accessor in `crates/engine/src/realtime/plan.rs`. -3. Tests in `crates/host-core/tests/successor_swap.rs`, through the browser's preparation entry - points (`prepare_host_runtime_with_selected_meters_between_render_calls`, - `crates/host-core/src/prepare.rs:1051`, and, for the observation taps, - `prepare_host_runtime_with_live_controls_successor`, `:926`). - -## Authorized paths - -- `crates/builtins/src/lib.rs`, `crates/builtins-compiler/src/lib.rs`, `crates/rack/src/lib.rs` -- `crates/effect-contract/src/live.rs` -- `crates/graph/src/lib.rs`, `crates/graph/src/runtime.rs` -- `crates/host-core/src/prepare.rs`, `crates/host-core/tests/successor_swap.rs`, - `crates/host-core/tests/support/successor.rs` -- Outside stream A's ownership; root sequences it: `crates/engine/src/realtime/plan.rs` - (`ObserverCarry`, `ObserverKind`, the executor method and the plan accessor only). - -## Non-goals - -- No browser code: the attachment hook is #1381, the browser gate #1290. No change to meter or - spectrum kernels. -- No spectrum carry (#1395). - -## Objective gates - -1. **Meters carry, both widths.** Prepare A with meters at every tap of three strips, with a window - several blocks long, so a window is open at the swap. B adds a muted track whose ID sorts first. - Read through A's consumers before and after the swap. The snapshot sequence has no gap, keeps - `reset_generation`, has `cumulative_discontinuities` 0, and every snapshot equals the reference - (a fresh B with the same meters, fed from frame 0) field by field. The added muted track's - meter shares the carried meters' window boundaries and sequence numbers from its first - snapshot, and each of its snapshots equals the reference's. -2. **A changed meter starts fresh.** B changes one meter's window length. That meter's successor - reader starts a new sequence, and every other meter carries as in gate 1. -3. **Pairing follows the record.** In gate 1's script, after the swap block, call - `pair_carried_meters` and `pair_carried_observations` on the successor's and predecessor's - handle vectors with the successor's `observer_carries()`. Every carried strip's snapshots then - continue through the successor's vector at that strip's new index, as in gate 1. The added - track's meters have no record, and their successor readers receive the fresh observers' - snapshots. -4. **Observation taps.** With a live-controlled compressor insert and its gain-reduction tap armed, - the tap's readings continue across the swap, equal to the reference. A compressor that restarts - (its threshold changed, prepared on a lane without live controls) starts its tap fresh. -5. **Realtime.** The swap block and the two pairing calls make zero allocations and frees, - measured with `bench_support::alloc`'s thread-scoped counters after warm-up. -6. Commands: - - `cargo test --locked -p engine -p builtins -p builtins-compiler -p effect-contract -p rack -p graph -p host-core --features builtins-compiler/test-support,rack/test-support,graph/test-support,host-core/test-support` - - `cargo build --locked --release -p audit && bash scripts/trace-builtins-graph-audit.sh target/release/audit && bash scripts/trace-graph-audit.sh target/release/audit` - - `cargo run --locked --release -q -p realtime-policy`, `bash scripts/check-workspace-policy.sh`, - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings` - -## Test value - -- Gate 1 turns red on any of five defects: - - a carry that copies the window but keeps the successor's producer leaves the host's reader on a - dead queue; - - a meter keyed by handle index instead of `(strip, tap)` reads the wrong strip after the lane - shift; - - a window restarted at the swap shows a counted discontinuity; - - a lost open window shows a sequence gap; - - a fresh meter that opens its window at adoption (D6 missing) is off the group's grid, so its - sequence and boundaries differ from the reference's. -- Gate 2: a rule that ignores the configuration carries a window of the wrong length. It turns red. -- Gate 3: a pairing that swaps by position instead of by the record's two indices puts a reader - on the wrong strip's producer once the added track shifts every index. It turns red. -- Gate 4: observation state that does not follow its effect owner restarts the tap on an unchanged - effect. It turns red. - -## Dependencies - -- *Carry strip delay lines and live send ramps across a plan swap* (#1284), and through it every - effect carry that observation taps follow. - -## Amendment (decision 17, root 2026-10-13) - -Dependencies gain *Prepare every browser preparation branch concurrently, as a successor too, in -host-core* (#1401), for D2's meter successor entry. - -## Amendment (controls coordinator, batch 10, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Anchors moved.** `MeterObserver` is `crates/builtins-compiler/src/lib.rs:5943`, - `MeterConsumer` `:942`; `MeterAccumulator` is `crates/builtins/src/lib.rs:5130`; - `HostMeterRequest` is `crates/host-core/src/prepare.rs:436`, `HostLiveControlHandles::meters` - `:482` and `effect_observations` `:489`; `ObservationLane` is - `crates/effect-contract/src/live.rs:1321`; a bank's `observations` is - `crates/rack/src/lib.rs:1184`; `impl PreparedPlanExecutor for GraphExecutor` is - `crates/graph/src/lib.rs:3579`; `PreparedRenderPlan::carry_from` is - `crates/engine/src/realtime/plan.rs:1158`. The entries are - `prepare_host_runtime_with_selected_meters_between_render_calls` (`prepare.rs:1487`), its - successor form (`:1544`, #1401) and `prepare_host_runtime_with_live_controls_successor` (`:1295`). -- **#1609's observer hook.** `GraphRuntimeObserver::block_end` exists now, and - `GraphNodeObserverBinding` reads `accepts_block_end` once at bind (`graph/src/lib.rs:2881-2890`). - D2 swaps the boxed observer between two bindings of the same kind and never swaps a binding, so - the cached flag stays right. -- **D5's C ABI note is ahead of the code.** #1581, #1584 and #1587 are open: a C ABI plan has no - strip meter and no observation tap yet. The gates are host-core gates and do not change. -- **#1584 D6 (open).** A successor is prepared with the committed taps armed, and a tap whose lane - carries takes the predecessor's state (D2). That is exact only when #1280b has applied the - predecessor's pending Observe records at the carry. So #1327b depends on #1280b. -- **The browser's readers today.** Since #1381a the meter consumers and observation handles are in - `RenderCompanions`, the plan's attachment. The spectrum capture is not: it is in host-core's - `SpectrumService`, on the control side (see #1395's batch 10 amendment). -- **Split (as plan v2).** #1327a: meters (D1's meter key, D2, D3 for meters, D6; gates 1, 2, 3 and - 5). #1327b: observation taps (D1's tap key, D3 for observations; gate 4). #1327a needs no carry - chain slice (the observer table needs only the carry program, on `main`); #1327b needs #1280b, - #1281 and #1282, whose owners the taps follow. The filed dependency on #1284 is file order only. -- **Engine edit (one-attempt rule).** D3's `ObserverCarry`, `ObserverKind`, - `PreparedPlanExecutor::observer_carries` and `PreparedRenderPlan::observer_carries` are an edit of - `crates/engine/src/realtime/plan.rs`. Recommendation: make it the chain's one `plan.rs` edit, in - #1327a: declare `ObserverKind::Spectrum` here too (so #1395 has no engine edit), and add #1381 - D3's hook here, `HostAttachment::adopt_predecessor(&mut self, predecessor: &mut dyn - HostAttachment, carries: &[ObserverCarry])` with an empty default, called by - `PreparedRenderPlan::carry_from` after the executor hand-over when both plans have an attachment - (so #1381b has no engine edit). Gate 3 then pairs the meters from a host-core test attachment's - hook in the swap block (the pattern of `crates/host-core/tests/host_attachment_render.rs`), which - is how the browser calls it. If root rules against the fold, #1395 and #1381b each carry their own - `plan.rs` edit under its own one-attempt rule. -- **If the hook folds in:** the call is a new indirect call in the browser's render closure, so - the pinned indirect-site table of `scripts/check-web-audioworklet-callgraph.py` (#1333 D4) gains - it in the same slice, with #1381 D3's reason; that script joins this slice's authorized paths for - that row only. - -## Ruling (root, 2026-10-14): one plan.rs edit - -#1327a makes the one combined edit to `crates/engine/src/realtime/plan.rs` the chain needs: the -observer carries of #1327 (`ObserverCarry`, `ObserverKind`, `observer_carries`), the -`ObserverKind::Spectrum` variant of #1395 and `HostAttachment::adopt_predecessor` with its call in -`PreparedRenderPlan::carry_from` for #1381b, with the callgraph pin in -`scripts/check-web-audioworklet-callgraph.py`. One attempt; if the permission system refuses it, -the slice stops and root is told at once. #1395 and #1381b make no edit of their own in -`crates/engine/src/realtime`. - -## Ruling (root, 2026-10-14): the acknowledged subscribe - -Since #1280b, a pending Observe record of a carried lane is applied at the carry to the -predecessor's `ObservationLane`, but the tap it arms is not carried to the successor until this -issue: an acknowledged subscribe can be dropped at a swap (an ack that precedes a drop). #1327b's -gates include that case: a subscribe admitted and acknowledged before a swap is armed on the -successor's tap after the swap, and the successor publishes the window the predecessor would have -published, with zero allocations and frees on the swap block. - -## Decision record: #1327a meters (batch 8, worktree b8k) - -**Shipped.** - -- **Engine, one `plan.rs` edit (root ruling).** - - `ObserverKind { Meter, Observation, Spectrum }` and `ObserverCarry { kind, successor, predecessor }` - (`usize` entry indices). - - `PreparedPlanExecutor::observer_carries()` with an empty default, and - `PreparedRenderPlan::observer_carries()`. - - `HostAttachment::adopt_predecessor(&mut self, predecessor: &mut dyn HostAttachment, carries: - &[ObserverCarry])` with an empty default. `PreparedRenderPlan::carry_from` calls it after the - executor hand-over when both plans have an attachment, with the successor executor's record. - Both the exchange's swap block and `adopt_predecessor_plan` reach it. - - The two new types are re-exported from `crates/engine/src/realtime/mod.rs` (one line). They - cannot be named outside `engine` without it. -- **`scripts/check-web-audioworklet-callgraph.py`.** `enter_block` pins 4 indirect sites, from 2: - the attachment hook and the `observer_carries()` read are new, with #1381 D3's reason. -- **builtins.** `MeterWindowPosition`, `MeterAccumulator::{config, window_position, join_window, - rebind_handle}`. -- **builtins-compiler.** `MeterObserver` answers the four new `GraphRuntimeObserver` methods. -- **graph.** - - `GraphRuntimeObserver::{carry_kind, adopt_binding_handle, window_position, join_window}` and - `ObserverWindow`. - - `GraphObserverLocation`, `GraphObserverRow`, `GraphObserverMove`, `GraphObserverJoin`, - `observer_rows`, `install_observer_carry`, `observer_carry_retained_bytes`. - - The executor's observer section is one boxed program (moves, joins, preallocated record), so - `GraphExecutor` grows by one pointer. - - The swap block checks every move first (all or nothing), then exchanges the boxed observers - between the two bindings, then runs the D6 joins. -- **host-core.** - - `MeterInventoryRow` and `MeterConfigKey` in `PlanStateInventory`, `resolve_meter_carry`, - `pair_carried_meters`, and the carry program now exists when a meter may carry. - - Test seam `test_only_prepare_host_runtime_with_selected_meters_on`. - - The meter request and its configuration are built by `requested_meter_owners` and - `requested_meter_config`, shared with the carry decision. - -**Departures from the text.** - -- **Only the reader's consumer is swapped.** `pair_carried_meters` exchanges the `consumer` field, - not the whole `MeterConsumer`. Handles are `index + 1`, so an added strip shifts every handle. - The swapped-in observer takes its binding's handle (`adopt_binding_handle`), so a snapshot - published after the swap names the reader that receives it. A snapshot published before the - swap keeps the predecessor's handle. -- **`pair_carried_observations` is not here.** It is #1327b. -- **Gate 2 is split.** A meter's window length is the preparation's one `meter_period_frames`, so no - caller can change one meter's configuration alone. - - Test A changes the length for every meter: nothing carries, the record is empty and each - meter's first window opens at the adoption block (sequence 0). - - Test B changes one meter's metric set: it alone is missing from the record. By D6 it joins the - carried grid, so its first snapshot after the swap is window 1, not a new sequence. -- **Gate 1's added muted track.** Its meters share the carried boundaries and sequence numbers - at every tap. They equal the reference field by field at the post-fader and post-matrix taps - only. A pre-fader tap sees the source in the reference from sample 0, while D6 holds the added - strip as silent before adoption, so that first window's energy cannot equal the reference's. - This is what D6 says (the strip did not exist before adoption). The gate text over-claimed. - -**Gates -> tests** (`crates/host-core/tests/successor_swap.rs`, module `meter_carry`): - -| Gate | Tests | -|---|---| -| 1 | `meters_carry_through_the_predecessors_readers_at_{eight,four}_lanes` | -| 2 | `a_changed_meter_starts_fresh_and_the_others_carry` | -| 3 | `paired_readers_follow_the_record_at_{eight,four}_lanes` | -| 5 | `the_meter_swap_block_allocates_and_frees_nothing` | - -- Gate 1 reads A's own consumers. -- Gate 3 pairs in a test attachment's hook, so the swap block includes the pairing. -- With the D6 join removed, gates 1 and 3 turn red at the added strip's first snapshot (checked). - -**Measured.** - -- Web AudioWorklet module: 3,924,130 B shipped, 4,491,023 B named twin. -- Boot high-water growth: 20,447,232 B, unchanged. -- Callgraph: the one pin row above. - -**Known interim state.** `host-web` test -`structural_apply::meter_delivery_restarts_on_the_successor_and_never_publishes_a_misshaped_frame` -is red. The browser's successor meter consumers now sit on fresh observers that the swap hands to -the retiring plan until #1381b's attachment hook pairs them, so no window folds after a swap. -#1381b and #1290d replace that test's "restart" expectation with "no restart". - -**Open item for root.** `HostAttachment::adopt_predecessor`'s doc in `plan.rs` links -`[PreparedRenderPlan::carry_from]`, which is `pub(crate)`. `RUSTDOCFLAGS='-D warnings' cargo doc -p -engine` fails on that link (`rustdoc::private_intra_doc_links`). It needs a one-line doc fix to -plain backticks in `crates/engine/src/realtime/plan.rs`. That is a second edit under the -one-attempt rule, so it was not made. - -### Known red test until #1381b (batch 11) - -`hosts/host-web` `tests::structural_apply::meter_delivery_restarts_on_the_successor_and_never_publishes_a_misshaped_frame` fails after this slice: the browser's successor meter readers sit on observers the swap hands to the retiring plan until #1381b's attachment hook pairs them (the browser rebuild path is test-support only until #1562b). #1381b, in this batch, turns the test from "restart" to "carry"; the batch lands only with it green (root, 2026-10-15). Root accepted the `crates/engine/src/realtime/mod.rs` re-export and a doc-only fix of `plan.rs` (a link to the crate-private `PreparedRenderPlan::carry_from` is plain code). - -## Decision record: #1327b observation taps (batch 8, worktree b8k) - -**Shipped.** No edit under `crates/engine/src/realtime`: #1327a's `ObserverKind::Observation`, `ObserverCarry`, -`observer_carries` and `HostAttachment::adopt_predecessor` are enough. - -- **effect-contract.** `ObservationLane::{same_taps, swap_state}`. `swap_state` exchanges the two lanes' tap - tables (arm state, open window, sequence, peaks, publisher) and armed count by pointer, and refuses unless both - lanes declare the same number of taps with the same fold at each index. Each lane keeps its own - `default_window_blocks`: a carried tap keeps the window length it was armed with. -- **rack.** `LiveControlEffectBankStage::observation_mut(lane)`. -- **graph.** - - `GraphTapMove` (the sites of one effect move, plus the two reader entries) and `install_observation_carry`. - A tap move binds to its effect move in the sorted effect table, so it is installed after - `install_effect_carry`, and refused if it names no effect move, if its successor holds no observation lane, or - if a site or entry repeats. - - `ObserverCarryProgram` gains the tap table. Its record holds one slot per meter move and per tap move, and - `observer_carry_retained_bytes` takes a third argument, the tap count. - - The effect section marks each tap move `taken` where its effect move actually carried (lane to lane, lane to - per-node, per-node to lane, per-node to per-node; a refused restore marks nothing). - - `Runtime::carry_taps` runs after the section's own pending-`Observe` loop (#1280 D3), so the predecessor's tap - already holds every acknowledged subscribe as its next block would have applied it. It then swaps the two - lanes' tap state and appends one `ObserverCarry { kind: Observation, successor, predecessor }` per swap after the - meter records. -- **host-core.** - - `ObservationInventoryRow` (reader entry, effect site, tap count) in `PlanStateInventory`, in the order of - `HostLiveControlHandles::effect_observations`. - - `resolve_observation_carry`: a reader set carries when its effect instance's move is in the effect carry and both - plans' sets have as many taps. A restarting effect, or one a rebuild removes, carries nothing. - - `pair_carried_observations(successor, predecessor, carries)`: swaps the `readers` of the two - `EffectObservationHandle`s the record names. Allocation-free. - -**The acknowledged subscribe.** A subscribe admitted before the swap sits in the predecessor's observe queue. The -carry pops it into the predecessor's tap at `predecessor_next_sample`, the sample the successor's first block -starts at, and the tap then moves to the successor with its window and publisher. The successor arms nothing -itself and publishes the window the predecessor would have. Today's red test `pending_observe_swap` asserted the -tap armed on A; it now asserts it armed on the successor, with the successor's reader (paired) equal to the -reference's window field by field, per-node and bank forms in both directions. - -**Departures from the text.** - -- **The tap is keyed by its effect owner, not by its own table.** D1 says the same, but the carried state moves in - the effect section's loop (a tap is a field of the effect instance, not a graph observer), so it has its own - small table beside the meters' instead of rows in `observer_rows`. -- **A restarted or removed owner's subscribe is dropped, as #143 D7 says.** The fresh tap starts unarmed. Arming the - successor's taps at preparation is #1584 D6 (open). The removed strip's tap stays armed on the retiring - predecessor. -- **Gate 4's "restart" case uses a prepared change (link mode).** With live controls attached, a threshold change - on an insert is a live difference and carries (#1280c), so it cannot start the tap fresh. - -**Gates -> tests.** - -| Gate | Tests | -|---|---| -| 4 and the acknowledged subscribe, per-node, bank, both form changes, eight lanes | `successor_swap`: `observation_carry::taps_follow_their_effect_through_every_form_at_eight_lanes`, `pending_observe_records_are_applied_at_the_carry_at_eight_lanes`, `full_observe_queues_are_applied_at_the_carry_at_eight_lanes`, `pending_observe_records_cross_the_form_change_at_eight_lanes` | -| 4, the same at four lanes | `..._at_four_lanes` twins: ignored where the build does not execute four lanes (#1017), as every console-effect test | -| 4, restart starts fresh | `observation_carry::a_restarted_effect_starts_its_tap_fresh_at_{eight,four}_lanes` | -| 5, swap block and the hook's pairing | `observation_carry::the_tap_swap_block_allocates_and_frees_nothing` (and the allocator/audit assertion inside the form tests) | -| lane exchange | `effect-contract/tests/observation_lane`: `a_lane_swap_carries_the_open_window_and_the_publisher`, `a_lane_swap_refuses_another_menu` | - -**Known state.** `host-web` `structural_apply::meter_delivery_restarts_on_the_successor_...` is still red, as the -#1327a record says (#1381b). No other `host-web` test changed: the browser's rebuild path is test-support only and -no host-web test swaps observation taps. #1381b pairs the browser's observation handles with -`pair_carried_observations` from the attachment hook, through its host-core-index-to-slot map. - -## Fold (batch 11 review) - -- `host_core::pair_carried_observations` and `pair_carried_meters` run on the render thread (host-web's - `adopt_predecessor` calls them) and now sit inside `REALTIME_POLICY_BEGIN`/`END` regions; `realtime-policy` - passes. -- Observe records left in the queue of a predecessor owner that did not move (a restarted or removed owner) are - never applied and never counted: `carry_effect_lanes` visits only the effect moves. No counter that a host - reports covers them (`unbound_observations` is read by tests only), and the rule is not to add an unread - counter. This goes to #1584 D6 (batch 12), which owns the reported observation counters. -- `cargo clippy -p host-core --no-default-features --all-targets` already passes at this head: the file the - review named, `tests/vca_live.rs`, no longer exists, and every host-core test that needs `control-provider` - carries a crate-level `cfg`. No change. diff --git a/.github/ISSUE_SPECS/1341-crossfade-the-bypass-switch-over-the-session-ramp.md b/.github/ISSUE_SPECS/1341-crossfade-the-bypass-switch-over-the-session-ramp.md deleted file mode 100644 index ab7ef222f..000000000 --- a/.github/ISSUE_SPECS/1341-crossfade-the-bypass-switch-over-the-session-ramp.md +++ /dev/null @@ -1,506 +0,0 @@ -# Crossfade the bypass switch over the session ramp - -Stream E of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-1, D15-13 E5). -Code anchors verified on `main` at `6fb211594`. - -## Product outcome - -A live bypass toggle on any effect that has a bypass shunt no longer steps the output by -`wet - dry`. It crossfades linearly between the effect's wet output and its latency-matched dry -signal, over the session's mute ramp (`control_smoothing.mute_ms`, or the default table). In this slice -the render paths (per node and banked) crossfade, a plan swap carries a crossfade in flight, and -the C ABI's bypass transactions use it. The effect's latency does not change. From the first sample -after the ramp, the output is bit-identical to today's whole-block select. A session whose -`mute_ms` is 0 renders exactly today's bits. The browser's `COMMAND_EFFECT_BYPASS` gets the session -ramp in *Crossfade the browser's live bypass command over the session ramp* (#1393). - -## Context - -- **The shunt** (`crates/effect-contract/src/live.rs:822-990`). The wet path always runs. The dry - signal is delayed by exactly the declared latency. "Selection is whole-block, never per sample" - (`:841-843`), done by copies, so `-0.0` survives (`apply`, `:979-984`; `dry`, `:987-990`). - `capture` (`:948-976`) must run on every block when the shunt has a latency line (`feeds_line`, - `:941-946`). At zero latency it is skipped on blocks that no reader needs. -- **The bypass state.** `EffectControlRecord::Bypass(bool)` (`live.rs:95`) sets - `EffectControlLane::bypass` at the block's `stage` (`:398-401`). `bypassed()` (`:245`) is the - only reader. The witness term `UNBYPASSED` is seeded from it (`:175`, `:201`) and declines mono - collapse while an upstream stage is bypassed (`crates/effect-contract/src/symmetry.rs:38`, - `:99-106`). -- **Two render paths read it:** - - **Per node**, `NodeKind::LiveControlEffect` (`crates/graph/src/runtime.rs:3510`): - `capture_dry = bypassed || shunt.feeds_line()` (`:3537`), and `shunt.apply` after `process` - when bypassed (`:3582-3585`). - - **Banked**, `LiveControlEffectBankStage` (`crates/rack/src/lib.rs:937`): `any_bypassed` - (`:1321-1323`) gates the capture (`:1348-1360`). The restore copies the dry words of exactly - the bypassed lanes, strided by `lane_count` (`:1441-1462`). -- **Producers of a bypass record:** - - the C ABI classifier (`crates/host-core/src/live_delta.rs:391-393`); - - the browser decode (`hosts/host-web/src/lib.rs:4489-4501`), which today refuses a non-zero - `smoothing_samples` word as `MALFORMED`; - - the SDK's `bypass(enabled)` (`sdk/src/core/live-controls.ts:943`), which has no ramp option. -- **A drift-free, vectorisable law already exists.** `IndexedRamp` - (`crates/lane/src/kernels.rs:1073-1100`) computes `c(k) = round(round(k * step) + start)` for - `1 <= k < length`, and assigns `target` exactly for `k >= length`. A step is `length == 0`. It - is a pure function of the frame index, so it vectorises over frames, and a settled ramp is - exactly the target. The live routes already use it. -- **The ramp length.** #1054 D3 maps the bypass crossfade to `mute_ms`, through - `LiveRamps::mute_samples` (`crates/host-core/src/live_delta.rs:33-58`). The C ABI qualification - document says "The switch is a step: there is no bypass crossfade (decision 14, F7)" - (`docs/C_ABI_V1_QUALIFICATION.md:450-452`). -- **Hot file, and the order.** `crates/effect-contract/src/live.rs` is edited first by - *Hold effect parameter, bypass and EQ-target values in latest-target cells* (#1345), which turns - the live bypass into a one-word cell drained first (#1345 D1, D3). Then *Carry live-controlled - effect lanes across a plan swap* (#1280) carries those cells and the shunt words in move mode - (cells #1280 D2, retarget D4, shunt words D5). This slice lands after both (plan risk 3) and - extends that carry. - -## Decisions frozen for this slice - -- **D1. The bypass value and its ramp travel as one unit.** #1345's bypass cell widens from one - word to two, `bypassed` and `ramp_samples`, written and drained as one unit (D15-2: a cell holds - its target words and its ramp together). The producer fills `ramp_samples` from - `LiveRamps::for_session(model).mute_samples` (#1054 D4). The bypass is the one live row with no - per-edit length (D15-1 recorded resolution): a host asks for a step by setting `mute_ms` to 0. - The browser decode, which this slice does not change in behaviour, writes `ramp_samples = 0`; - *Crossfade the browser's live bypass command over the session ramp* (#1393), which needs #1364's - browser `LiveRamps`, gives it the session ramp. Its `smoothing_samples` word stays required - zero. -- **D2. Lane state.** `EffectControlLane` holds: - - the target (`bypass`); - - a one-word mix ramp `m`, where `m = 1` is wet and `m = 0` is dry, as an `IndexedRamp` that uses - word 0; - - its frame position `k`. - - A record whose target equals the current settled state changes nothing. Otherwise it starts - `IndexedRamp::new([m_now, 0, 0, 0], [target_m, 0, 0, 0], ramp_samples)` with `k = 0`. `m_now` - is the mix at the current position, so a reversal mid-ramp continues from where the output is. - Every rendered frame advances `k`, which saturates at the length. A length above - `INDEXED_RAMP_LENGTH_MAXIMUM` is refused where the record is produced (C ABI: `Domain` in the - classifier; browser: `DOMAIN`). At the launch bound (1000 ms at 96 kHz, 96,000 samples) this - cannot happen. -- **D3. The mix.** For a ramping lane and frame `k`: - `out = dry + (wet - dry) * m(k)`, as a separate subtract, multiply and add. There is no fused - multiply-add, as base Wasm SIMD requires. - - A settled-wet lane keeps the wet words untouched. - - A settled-dry lane copies the dry words, as today. - - Settled lanes are never computed through the formula, so the after-ramp bits are today's. - - The same `m(k)` applies to left and right, so the switch never makes the channels differ. - - Linear in amplitude, the engine's one ramp law family (#1055 `FINDINGS.md`, section 3): dry - and wet are the same signal, latency-matched, and are highly correlated, so an equal-gain - crossfade keeps the level where an equal-power one would bulge by up to 3 dB at the midpoint. -- **D4. Vectors, not scalars.** - - **Per node:** `m(k)` is computed for a whole run of frames by the indexed law, and the mix runs - lane-wide over frames. - - **Banked:** the interleaved frame holds `lane_count` contiguous words. The mix runs lane-wide - on one vector of per-lane `m`. A select then keeps the wet words of settled-wet lanes and the - dry words of settled-dry lanes, and takes the mix for ramping lanes. - - The kernels live next to `BypassShunt` in `live.rs` and use the `lane` crate's vector types. - The same kernel shape serves every target; only the lane width differs. -- **D5. Capture.** Both paths capture the dry block when any lane is bypassed or ramping, or when - the shunt feeds a line. -- **D6. Mono collapse.** A ramping lane counts as bypassed for the witness: `UNBYPASSED` is false - from the record until the ramp settles at wet. Collapse is declined during a ramp exactly as it - is under a bypass. -- **D7. Carry.** The mix ramp and `k` are live lane state, carried with the bypass cell by #1280's - move-mode lane carry: the swap block moves them, and the predecessor then retires. A swap in the - middle of a ramp continues the ramp. -- **D8. Acked-batch question.** No queue or admission rule changes, and a record still applies at - its block boundary. A bypass record is never acked without effect: its ramp starts at the next - block. - -## Deliverables - -1. D1-D4 and D6 in `crates/effect-contract/src/live.rs` (record or cell, lane state, kernels), with - the shunt's doc rewritten ("Selection is whole-block" becomes the crossfade rule). -2. D4-D5 at the two call sites: `crates/graph/src/runtime.rs:3530-3585` and - `crates/rack/src/lib.rs:1318-1462`. -3. D1 producers: the classifier (`live_delta.rs:391-393`) with the session mute ramp; the browser - decode (`hosts/host-web/src/lib.rs:4489-4501`) with `ramp_samples = 0` (D1); every other - construction of the record or cell (mechanical). -4. D7 in #1280's move carry. -5. Docs: `docs/C_ABI_V1_QUALIFICATION.md:450-452` and `docs/EFFECT_CONTRACT_V1.md`'s bypass - text. -6. The tests below. - -## Authorized paths - -- `crates/effect-contract/src/live.rs`, `crates/effect-contract/src/symmetry.rs` -- `crates/graph/src/runtime.rs`, `crates/rack/src/lib.rs` (stream A owns them; root sequences the - merge) -- `crates/host-core/src/live_delta.rs` (stream B owns it), `crates/host-core/tests/live_delta.rs`, - `crates/host-core/tests/successor_swap.rs` -- `crates/graph-compiler/tests/bypass_shunt_identity.rs`, `crates/graph-compiler/tests/bypass_cohorts.rs` -- `crates/capi/src/runtime/live_tests.rs`, `docs/C_ABI_V1_QUALIFICATION.md` -- `docs/EFFECT_CONTRACT_V1.md` -- Mechanical record-shape updates in any file that builds a bypass record or cell, including - `hosts/host-web/src/lib.rs` (D1's zero only; stream H owns it) - -## Non-goals - -- **The browser's bypass command.** *Crossfade the browser's live bypass command over the session - ramp* (#1393): host-web fills `ramp_samples` from the `LiveRamps` it keeps for the running - session (*Resolve an absent live ramp to the session default on the browser and in the SDK*, - #1364, D2), the `COMMAND_EFFECT_BYPASS`, shipped-host and SDK `bypass` docs, and the gate that a - C ABI bypass transaction and a browser bypass command render the same bits. It depends on this - issue and #1364. -- No crossfade for a prepared bypass. The delay and the multiband compressor get shunts in - *Give the delay a live bypass shunt* (#1339) and *Give the multiband compressor a live bypass - shunt* (#1340). Once they land, D3 covers them with no further edit. -- No equal-power or raised-cosine shape. #1055's section 5.5 what-if is research, not a decision. -- No ghost-strip crossfade for a plan swap (D15-9 defers it). -- No change to latency, PDC, bank cohorts or the prepared bypass path. - -## Hazards - -- **Existing equality tests.** A test that pushes a live bypass and compares the next block with - a prepared-bypass or authored control now sees a ramp. Where the claim is the switch itself, - author `control_smoothing.mute_ms = 0` (#1054 D6). Otherwise compare after the ramp. Do not - weaken a comparison. -- **Zero-latency effects** skip the capture on un-bypassed blocks. A ramp toward wet ends on a - block whose capture must still run (D5). - -## Objective gates - -1. **The exact law, per node** (`bypass_shunt_identity.rs`, new). Render one per-node live - instance three times from the same input, as the existing per-node harness does (`:538-608`): - - W, never bypassed; - - D, bypassed from preparation; - - X, bypassed by a record at block 3 with a 480-sample ramp at quantum 128, so the ramp ends - mid-block. - - Every sample of X before block 3 equals W. Inside the ramp it equals - `D + (W - D) * m(k)`, with `m` computed independently by `IndexedRamp::new([1,0,0,0], - [0,0,0,0], 480)`. From the ramp's end it equals D, bit for bit. Run it for a zero-latency effect - and for one with latency (the true-peak limiter). -2. **Bank equals node** (same file; extend `:610-760`'s live-toggle test). Four lanes (eight on an - AVX2 width), with toggles on lanes 1 and 3 at different blocks and one reversal mid-ramp. Each - lane equals the per-node render of the same track bit for bit, and untouched lanes equal W. - Banking never moves a lane's bits. -3. **Reversal.** In gate 2's reversal, the bypass record lands at block 3 and the un-bypass record - at block 5, at `k = 256` of the 480-frame ramp. From block 5 the output follows - `IndexedRamp::new([m(256), 0, 0, 0], [1, 0, 0, 0], 480)`, with no jump, and equals W exactly - from 480 frames later. -4. **Zero ramp.** With `mute_ms = 0`, every existing shunt-identity and bypass-cohort test passes - unchanged, and gate 1's X equals D from the first sample of block 3. -5. **C ABI** (`crates/capi/src/runtime/live_tests.rs`). A bypass transaction on a playing session - follows gate 1's law with the session's `mute_ms`, then equals a control booted bypassed. The - ramp blocks allocate nothing (`bench_support::alloc` thread counters, statics warmed). -6. **Carry** (`crates/host-core/tests/successor_swap.rs`). A swap at a block inside a ramp renders - bit-identically to the same session without the swap. -7. **Cross-target.** `bash scripts/check-cross-targets.sh` and the browser legs pass. The mix uses - no fused multiply-add, so the native, AArch64 and Wasm bits match. -8. **Commands:** - - `cargo test --locked -p effect-contract`, - `cargo test --locked -p graph-compiler --test bypass_shunt_identity --test bypass_cohorts` - - the workspace debug leg and the DSP-crates debug leg, as `.github/workflows/qualification.yml` - runs them (`test-debug-a`, `test-debug-b`) - - `cargo build --locked --release -p audit && bash scripts/trace-graph-audit.sh target/release/audit && ./target/release/audit capi` - - `bash scripts/build-web-audioworklet.sh --named-twin target/ci/qualification-named-twin target/ci/qualification-artifacts`, - `bash scripts/check-web-audioworklet.sh --without-metadata-regeneration target/ci/qualification-artifacts target/ci/qualification-named-twin/miso-engine-v1-audio-worklet.simd128.named.wasm`, - `bash scripts/test-web-audioworklet.sh` - - `cargo run --locked --release -q -p realtime-policy`, `bash scripts/check-effect-runtime-policy.sh`, - `bash scripts/check-workspace-policy.sh`, `bash scripts/check-cross-targets.sh` - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` -9. **Digests.** A checked-in digest of a render that toggles a live bypass moves. Each one is - listed in the PR with the reason "bypass crossfade, decision 15 E5" and re-pinned on its own - (decision 15, risk 4). - -## Test value - -- Gate 1 turns red if the mix uses `m` from an accumulated sum, or an FMA, if the ramp is - mis-anchored (off by a frame at the block boundary), or if the after-ramp path computes the - formula instead of copying. No test checks a ramped bypass today. -- Gate 2 turns red if a bank lane reads a neighbour's `m`, if the select keeps a settled lane on - the formula path, or if the bank and per-node kernels diverge. -- Gate 3 turns red if a reversal restarts from the settled endpoint (a jump) instead of from the - current mix. -- Gate 4 turns red if a zero length is not a step, which would move today's bits for an explicit - 0. -- Gate 5 turns red if the C ABI producer forgets the session ramp (the toggle steps). -- Gate 6 turns red if the carry drops the ramp position, so the successor jumps to the endpoint. - -## Dependencies - -- *Session `controlSmoothing`: configurable ramp lengths for live mute, fader and pan changes* - (#1054), for `LiveRamps::mute_samples` and D3's row table. -- *Research: default ramp lengths for live mute, fader and pan changes (cited, measured, listened)* - (#1055), whose decision-15 addition confirms `mute_ms` for the bypass row. -- *Hold effect parameter, bypass and EQ-target values in latest-target cells* (#1345): D1 widens - its bypass cell. -- *Carry live-controlled effect lanes across a plan swap* (#1280): D7 extends its carries. - -Dependent: *Crossfade the browser's live bypass command over the session ramp* (#1393) depends on -this issue. - -## Amendment (decision 17, root 2026-10-13, Q2) - -- This slice gains one gate taken from #1563B gate 3: the SDK's `bypass` builder (a `SET_CONTROL` - on control `0` of an effect section) renders, on the headless engine and through the browser's - core path, the same crossfade over the session ramp as the C ABI, bit for bit per block. - #1563B no longer waits for this slice; only that row does. - -## Decision record (decision 17, lane K, batch 5) - -### The a1 / a2 / b split - -The slice ships in three coherent commits, each with its own gates. - -| Part | Content | Gates | -|---|---|---| -| **a1** (this commit) | Per node: D1 (the cell and the producers), D2, D3 and D5 (per node), D6; the C ABI producer; the docs; the per-node render and its tests. The bank steps unchanged. | 1, 3 (per node), 4, 5, 7, 9 | -| **a2** (next) | D4's banked kernel: the interleaved mix with a per-lane select, the bank's capture and restore. | 2, 3 (banked) | -| **b** (after #1280) | D7: the mix ramp and its position carried across a plan swap in #1280's move-mode lane carry. | 6 | - -The SDK `bypass` builder row (Q2 amendment) is **pending**: its browser leg runs only after #1562a -(the browser core path), and its headless leg needs the browser's `LiveRamps` of #1364. Nothing in -a1 builds it. - -### Adaptations to today's code - -- **The cell is already a latest-target cell** (#1345 landed): `EffectCells::bypass` is a - `LatestCell<2>` now (`bypassed`, `ramp_samples`), read in the one drain. -- **The record keeps its shape.** `EffectControlRecord::Bypass(bool)` is matched exhaustively by the - browser host, whose source another lane owns, so it stays the *step* spelling (the same write with - a ramp of 0, which is D1's browser value). The ramp-carrying unit is `BypassSwitch { bypassed, - ramp_samples }`, written by `EffectCellWriter::write_bypass` and - `EffectControlProducer::write_bypass` (preflight `check`/`preflight_bypass`, refusal - `EffectWriteRefusal::Ramp` above `INDEXED_RAMP_LENGTH_MAXIMUM`). The classifier's - `LiveEffectRecords::bypass_ramp_samples` carries the session ramp beside the `Bypass` record, and - the control plane writes the pair as one `BypassSwitch`. Folding the ramp into the record itself, - and giving the browser's `COMMAND_EFFECT_BYPASS` the session ramp, is #1393 with the browser lane. -- **Registry.** The bypass row's motion is `Motion::Fixed` (no per-edit length, so `SET_CONTROL` - on control 0 cannot carry a length) and `LiveRampRow::BypassCrossfade` maps to `mute_ms`; the - delay and the multiband bypass stay `rebuild_only` through `lowers_session_bypass`. The control - list now reports the bypass row's `default_ramp_samples` as the session's mute ramp (was 0). -- **Ramp index.** Frame `f` of a block at ramp position `p` takes `k = p + f + 1`, the convention of - `route_mix_ramp_block`: the first ramp frame is already one step along, the ramp reaches its target - on the `length`-th frame, and `m_now` at a reversal is `m(p)`, the last rendered frame's mix. The - settled frames are never computed. -- **Lane API.** `needs_dry()` (bypassed or ramping, D5), `bypass_ramping()`, `bypass_mix()`, - `apply_bypass::(shunt, left, right)` (mix and advance) and `advance_bypass_mix(frames)` on - `EffectControlLane`; `BypassShunt::crossfade::` is the kernel, with the outlined `f32` tail. - D6: a record admits `Bypass(true)` into the witness whenever the target is dry or the ramp is - longer than 0; the advance that settles a lane at wet admits `Bypass(false)`. -- **The bank, until a2.** `LiveControlEffectBankStage` still restores dry words by the lane's - target (`bypassed()`), so a banked lane steps at its record exactly as before. It calls - `advance_bypass_mix(frames)` once per block so each lane's ramp state, and the `UNBYPASSED` term it - restores, stay in step with the timeline. A console slot always banks, so on the C ABI only a - per-node insert (a remainder group) crossfades until a2 lands; a2 deletes the interim. -- **Test adaptations (hazard 1).** The C ABI tests that compare a live bypass with a rebuild wait - `window(latency, mute ramp, quantum)`, and the two that compare with the browser's stepping lane - author `control_smoothing = hard switch`. - -### Evidence - -- `bypass_shunt_identity.rs`: the exact law per node (gate 1) for a zero-latency effect (compressor) - and one with latency (true-peak limiter), at the host width and at scalar width; the reversal at - `k = 256` and a repeated target (gate 3, per node); the zero ramp (gate 4). -- `live_control.rs`: the witness, `needs_dry`, the repeated target, a zero ramp and the refusal. -- `live_delta.rs`: the record's ramp is the session's `mute_ms` (no per-edit length) and a ramp above - the law's bound is `Domain`. -- C ABI (gate 5, `live_tests.rs`): a bypass transaction follows the law with the session's - `mute_ms` and lands on a control booted bypassed, for a zero-latency effect and one with - latency, at `mute_ms` 10 and 0, with no allocation in a ramp block. The law holds there to a few ulps - of the block peak, because the strip's gain stages follow the effect and round; gate 1 pins it - exactly at the effect's output. - -## Decision record: #1341a1 (written in batch 8, lands in batch 9) - -The per-node crossfade (commit `b6bbc44e3`, built on a pre-flip base) was re-applied onto `main` at -`32cee58d0`, after the batch 6-7 flip made the browser a thin client of the core's `SessionState`. - -### What shipped - -The a1 content of the batch 5 decision record above, unchanged: `BypassSwitch { bypassed, -ramp_samples }` as the two-word bypass cell, the per-node lane crossfade (`EffectControlLane`, -`BypassShunt::crossfade`), `LiveEffectRecords::bypass_ramp_samples` filled from the session's -`mute_ms`, the control plane's `write_effect_record`, the bypass row's `default_ramp_samples`, and -the docs. A bank (a console slot, or any banked cohort) still steps; that is a2. - -### Departures from the batch 5 text, and why - -- **The browser needs no separate producer.** `COMMAND_EFFECT_BYPASS` and the browser's own live - lane are gone from `main`: the browser sends `SET_CONTROL` frames into the core, whose control - plane writes the `BypassSwitch` (with the session ramp) for every host. So #1341a1 crossfades the - browser too, and #1393 (already superseded by #1563) has no work left. D1's "browser writes a ramp - of 0" no longer applies. The stale comments that said so were corrected - (`EffectControlRecord::Bypass`, `live_tests.rs`, `C_ABI_V1_QUALIFICATION.md`). -- **D7 (carry) does not apply now.** The #1280 carries are not on `main`, and - `crates/engine/src/realtime/latest_cell.rs` is not touched. The mix ramp and its position `k` are - carried by #1280's move-mode lane carry; that extension moves with #1341b. - -### Amendment gate (SDK `bypass` builder, from #1563B gate 3) - -The browser leg is reachable now, as the browser is the core's `SESSION_STATE` through frames. The -gate is three tests, one per layer, because the C ABI, the headless engine and the browser host run -one core: - -- C ABI: `capi` `a_live_bypass_crossfades_over_the_session_mute_ramp_and_lands_on_the_bypassed_control` - (the law, at `mute_ms` 10 and 0, with zero allocation in a ramp block). -- C ABI against the browser host, bit for bit per block, at 44.1, 48, 88.2 and 96 kHz: - `hosts/host-web/tests/control_parity.rs` `a_live_bypass_crossfades_alike_on_both_hosts` (new), plus - the scripted insert and console-slot bypass rows of - `both_hosts_give_the_same_result_for_the_same_control_edits`. The new test fails on a step (a step - spans at most one block, a ramp spans `ceil(mute_ms * rate / 128)` or one more) and on a wrong - length, and requires the render to equal the booted-bypassed one from the ramp's end. -- SDK: `sdk/test/live-controls-evals.mjs` "the SDK bypass builder crossfades an insert over the - session ramp": the builder's `SET_CONTROL`, through the headless engine and through the browser - transport (`createBrowserLiveControls` over `engineHost`), renders the same bits in every block, - with the same count of crossfade blocks as the law, and settles on the booted-bypassed document. - The "effect bypass on a console slot" row left `todo` and gates (it settles on the booted - document; the console slot steps until a2). - -### Test adaptations (hazard 1) - -- `sdk/test/console-evals.mjs`: the two evals that compare a live insert bypass with the authored - one author `controlSmoothing` of zero (`HARD_SWITCH`); the claim there is addressing, not fading. -- `hosts/host-web/tests/control_admission.rs`: the insert "bypass" row compares with the fresh plan - from block 16 (after the ramp and the strip's delay compensation) instead of from block 0. - -### Conflicts met on `main` - -`crates/control-plane/src/control.rs` (the effect-record write loop moved under the `if let -Some(stamp)` publication block: kept main's shape, both write sites call `write_effect_record`) and -`crates/host-core/src/control_provider.rs` (`control_row` became `Shape::of`: the bypass arm of the -`Motion::Fixed` match moved there). - -## Decision record: #1341a2 (written in batch 8, lands in batch 9) - -### What shipped - -D4's banked kernel, and with it the deletion of the a1 interim. A bank now renders each lane's -bypass crossfade exactly as the per-node path renders it, so a placement change (a console slot, a -cohort of 4-7 inserts on a 4-lane target, a remainder insert) moves no rendered bit. - -- `BypassShunt::crossfade_bank::` (`crates/effect-contract/src/live.rs`, a realtime-policy - region): the interleaved block holds `L::WIDTH` words per frame, one per bank lane. Per frame and - channel one vector op computes each lane's `m = fma(k, step, start)` from a vector of per-lane ramp - indices `k = position + f + 1` (one `add` per frame advances all of them; exact below `2^24`) and - the mix `dry + (wet - dry) * m` as a separate subtract, multiply and add, the per-node order. - `select(k < length, mix, select(settled_dry, dry, wet))` then keeps the wet words of a settled-wet - lane and the dry words of a settled-dry lane (bit selects: `-0.0`, NaN payloads and subnormals pass - untouched, and a settled lane never goes through the formula). A bank with no ramping lane skips - the mix and only selects, which replaces the old scalar strided restore loop with a vector select - over the same words. -- State: `BankCrossfade` (five `[f32; 8]` arrays and a flag, inline in `LiveControlEffectBankStage`, - no allocation, nothing new in the heap resource report) is rewritten every block from - `EffectControlLane::bypass_crossfade_lane()`; a lane with no control lane is `BankCrossfadeLane::WET`. -- The stage (`crates/rack/src/lib.rs`): the dry block is captured when any lane `needs_dry()` (D5, - was `bypassed()`), the kernel runs after the bank and the taps, and the lanes then advance by the - block (`advance_bypass_mix`, kept: it is the lane's state step and restores the witness term). The - interim, "restore dry words by the target and only advance", is gone. The collapsed (mono) body - shares the code: a ramping lane counts as bypassed for the witness, so a collapsed block has no - ramping lane. - -### Gates and tests - -- Gate 2 (bank equals node) and gate 3 banked: - `graph-compiler` `bypass_shunt_identity` `a_bank_crossfades_every_lane_exactly_as_the_node_does`. - The real `LiveControlEffectBankStage` through `BankChain` at the build's bank width, with eight - lanes' schedules (four on a 4-lane build): a bypass over 480 frames that ends inside a block (the - per-node kernel's scalar tail), a reversal at `k = 256`, a lift over 300, a step, a lane bypassed - without a channel, a lane without a control lane, a never-switched lane, and a lane whose ramp is - exactly one block followed by a bypass. Every lane equals its own per-node render (`needs_dry`, - capture, effect, `apply_bypass`) bit for bit in every block, for the compressor (no latency) and the - true-peak limiter (latency). Red when the bank steps: first diff is lane 1, block 3, a `-0.0` where - the node has `+0.0`. -- Widths. An effect binds only the build's own bank width (no runtime dispatch, D4), so the stage - test runs at the host width (8 lanes on this AVX2 host). The 4-lane and 8-lane kernels are both - pinned lane by lane without a processor in `effect-contract` - `live_control::a_bank_kernel_crossfades_every_lane_exactly_as_the_node_kernel_does`: eight lane - schedules (ramp, reversal, lift, step, never, session-bypassed, short ramps, a ramp ending - mid-block), wet words with `-0.0`, infinities and NaNs, at `Simd4`, `Simd8` and the scalar lane - type `f32` (`WIDTH = 1`; a one-lane bank run once per schedule row, added in batch 9 on root's - requirement that gate 2 show all three widths) against the per-node `apply_bypass` at the host - width. The per-node kernel's own scalar tail is exercised by the stage test (ramps that end inside - a block); the bank has no tail (its frame is exactly one vector). The 4-lane build itself runs the stage test under - `scripts/run-aarch64-tests.sh` (its debug leg tests every product crate of `capi`'s closure with - `--all-targets`, which holds `effect-contract`, `graph-compiler`, `graph`, `rack`, `host-core` and - `capi`, so the NEON `Simd4` and scalar kernel rows and the stage test run there; `host-web` is not - in the closure) and, as simd128, the SDK headless evals below. -- M1 (lift through the real runtime): `capi` - `a_live_bypass_crossfades_over_the_session_mute_ramp_and_lands_on_the_bypassed_control` now runs, - per effect (compressor with no latency, soft clip with 31), per `mute_ms` (10, and 0), a bypass and - a lift, and the effect as an insert (a remainder, rendered per node) and as a console slot (always - banked). The lift boots the document with the effect bypassed, commits the lift, and asserts - `D + (W - D) * m(k)` with `m` running 0 to 1 frame by frame, then W exactly, with no allocation in - a ramp block. Mutation: a lift ramping the wrong way, and a bank that steps, each turn it red. -- `render_per_node` in `bypass_shunt_identity.rs` is **kept**: the C ABI test holds the law to a few - ulps of the block peak (the strip's gain stages round after the effect), and it is the only test - that pins the law exactly at the effect's output and at scalar kernel width. -- SDK: `sdk/test/live-controls-evals.mjs` "the SDK bypass builder crossfades ..." now has an insert - case and a console-slot case (headless and browser transport, bit for bit per block, the ramp's - block count, settles on the booted bypassed document). The console-slot row of - `liveEqualsBooted` keeps its settle check, its comment now says the bank crossfades. -- `hosts/host-web/tests/control_parity.rs`: the scripted console-slot bypass rows already compare the - two hosts bit for bit; the module doc no longer says the bank steps. A console-slot variant of - `a_live_bypass_crossfades_alike_on_both_hosts` was tried and dropped: this session's strips hold - effects after the console slot, whose history outlasts the ramp, so the render is not equal to the - booted bypassed one for 55 blocks (a property of the fixture, not of the crossfade). - -### Hazard 1 adaptation - -`capi` `a_live_eq_bypass_beside_a_gain_change_renders_like_the_browsers_lane`: the console EQ banks -and stepped before a2; it compares with a reference lane pushed a bare `Bypass` record (a step), so -its session authors the hard switch, as the insert test beside it already does. - -### Docs - -`docs/C_ABI_V1_QUALIFICATION.md` (the bank crossfades, no per-node-only sentence), -`docs/EFFECT_CONTRACT_V1.md` (the bank kernel paragraph), the rack stage's docs. - -### Evidence - -No checked-in digest moved: `check-effect-runtime-fixtures.sh` and `check-console-fixtures.sh` pass -unchanged (none of them toggles a live bypass). - -## Decision record: the browser qualification's live-bypass row (batch 9 fold) - -The row `sdk-live-bypass` (`hosts/host-web/qualification/run.mjs` `validateSdkLiveBypass`, rendered by -`sdk-response-entry.ts` `runSdkLiveBypassQualification`) was written for a stepping bypass: it held -each live render digest equal to the authored control's (`consoleOffLive === consoleOff`), and it -went red on the browser build at this batch ("a live bypass toggle on the post_insert console slot did -not render its authored bypass state"), because the live bypass now crossfades over the session's -`mute_ms`. The row's own header said it would move to this issue. - -**New contract (strict, per quantum, four edits).** Each live render is compared with two controls -the same browser renders: the starting one (`base` for a bypass, the authored bypass for a lift) and -the target. Per 128-frame quantum: the render equals the starting control until the ramp reaches the -output (found, not assumed, as `control_parity.rs` `a_live_bypass_crossfades_alike` finds `start`); -then it differs from both controls for `ceil(ramp / quantum)` or one more quanta; from there it equals -the target bit for bit to the end of the render. A step (at most one quantum), a no-op (never leaves -the start), a render that falls back to its start inside the ramp, a ramp that does not settle, and a -lift that does not land are each red. Nothing is "eventually similar". - -**The fixture states its ramp.** The session now declares `controlSmoothing: { muteMs: 10, faderMs: -10, panMs: 10 }` (the fader and pan lengths are unused here) instead of leaning on the default, and -the response carries `muteRampSamples` (`round(48000 * 10 / 1000)` = 480) and `quantumFrames` (128), -from which the gate derives its block count (4). The render is 16 quanta (2,048 frames, was 8) so a -settled stretch follows the ramp. The response carries a SHA-256 per quantum for the seven renders -(`blocks`); the three whole-render digests that remain are the audible-and-distinct check. - -**Measured (chromium, firefox, webkit agree).** Console slot (`desk-hi`, after the insert, nothing -audible follows it) bypass: quanta 0-3 between the controls, 4 on equal the bypassed control; lift: -quanta 0-3 between, 4 on equal the document with none. Insert (`ins-mid`; the console EQ after it -re-filters the changed input) bypass: quanta 0-4 between (5 = ramp + 1), 5 on equal the bypassed -control; lift: 0-3 between, 4 on. Both directions settle exactly, so the strict settle holds on the -insert row too, and the gate keeps no tolerance. - -**Self-test mutations (three new, two moved).** `sdk-live-bypass-console` and `-insert` now replace -the live blocks with the wrong control; new `sdk-live-bypass-step` (one quantum of start then the -target), `-unsettled` (the last quantum back at the start) and `-lift` (the lift left bypassed). All -five are proven red in each browser by `--self-test-mutations`. - -## Amendment (controls coordinator, batch 10, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **#1341b folds into #1280a.** D7 is a copy of `mix` (`crates/effect-contract/src/live.rs:773`) - and `mix_position` (`:776`) in `EffectControlLane::carry_from`, beside `bypass` and the symmetry - terms that #1280 D2 copies. A separate slice would add one field copy to code that #1280a - writes. #1280a runs gate 6, for a zero-latency effect and for the true-peak limiter (which also - needs #1280 D5's shunt copy, now in #1280a). -- This issue closes when #1280a's evidence is upstream: a1, a2 and the Q2 SDK builder row are done - (batch 9 records above). -- #1622 (open) folds the ramp into the bypass record; D7 does not change. diff --git a/.github/ISSUE_SPECS/1344-prepare-a-successor-across-a-withdrawn-candidate-plan.md b/.github/ISSUE_SPECS/1344-prepare-a-successor-across-a-withdrawn-candidate-plan.md index 21ec2fa6e..977765b27 100644 --- a/.github/ISSUE_SPECS/1344-prepare-a-successor-across-a-withdrawn-candidate-plan.md +++ b/.github/ISSUE_SPECS/1344-prepare-a-successor-across-a-withdrawn-candidate-plan.md @@ -168,3 +168,238 @@ it needs; on its own it changes no host behaviour. - *Carry fader, mute and pan ramps across a plan swap* (#1277): stream A's open edit of `prepare.rs` lands first. Its new carry section is then reported by `installed_carry_reads` (D1). + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `e2f6cfb24`. + +**Batch order (root, 2026-10-15).** Batch 15a is the controls lane: #1344, #1398, #1310, in +that order. #1324 left batch 15a (root ruling, 2026-10-15); it lands in a later batch after #1288, +#1391, #1363 and #1325, in that order. The rest of batch 15 follows. This slice is first. + +**Moved anchors.** Each fact in Context still holds, except as noted. +- `SuccessorBase` is `crates/host-core/src/prepare.rs:1070-1097`. It now has six fields: + `inventory`, `committed`, `revision` (#1277 D10), `edit_ramps` (#1394), `monitor` (#1559) and + `observation` (#1584). D2's literal `SuccessorBase { inventory, committed }` must set all six. + Tests here use the neutral values: `EditRamps::new()`, `MonitorBase::NONE` (`prepare.rs:1125`), + `NO_OBSERVATION` (`:1100`) and a revision of at least 1 (`host.successor.revision`). The real + values for a superseding successor are #1310's. +- The carry program is built at `prepare.rs:2131-2134` and installed at `:2744`. A second + install, with no sources, is at `:3067-3073` (a plan whose only carry is a line or a send). +- `carried_inputs` is no longer a function. It is a local at `prepare.rs:2016`; the input moves + are installed at `:2772`. `carried_sources` is the local at `:1953`. +- `PreparedHost` is `prepare.rs:659-675`. `SourceInventoryRow` is `:695-703` (`index` at `:702`). + `InputSectionInventoryRow` is `:871` (its `location` at `:876`). `PlanStateInventory` is + `:711-751`. +- The executor's carry fields are `crates/graph/src/lib.rs:3211-3244`. The layout witness + `GraphExecutorWithoutSplitPairTable` repeats them at `:4445-4456`. The installers are + `install_builtin_input_carry` `:3390` and `install_carry_program` `:3662`. +- `GraphPreparedSourceSetDriver` is `graph/src/lib.rs:2565`; its `adopt_sources` is `:2629`. +- In `crates/source/src/lib.rs`: `adopt_sources` is `:1902-1938`, `held_seek` `:1018`, + `begin_block` `:1103`, `begin_block_at` `:1115`, `begin_block_with` `:1119`. `begin_block_with` + now takes a private `SeekClock` (`:1030-1038`). +- `PreparedRenderPlan::adopt_predecessor_plan` (gate 1's twin) is + `crates/engine/src/realtime/plan.rs:1201`. +- `SourceControlSet::adopt_persisting` is still `crates/host-core/src/source.rs:286`. + +**D1: the carry families at HEAD.** The executor now installs nine carry sections, not two: +`carry` (sources), `carry_inputs`, `carry_strips` (#1277), `carry_effects` with `carry_scratch` +(#1279-#1282), `observer_carry` (meters and observation taps, #1327), `carry_delays` (#1283), +`carry_strip_delays` and `carry_routes` (#1284). Host-core also carries spectrum captures +(#1395): the pairing is `PlanStateInventory::spectrum_pairing`, built in host-core, not in the +executor. D1 holds as a design. Its scope follows from it: +- `installed_carry_reads` reports a predecessor key for every executor section above. +- The restriction keeps the inventory rows each section reads: `sources`, `input_sections`, + `strip_stages`, `effect_lanes`, `meters`, `observations`, `delay_lines`, `strip_delays` and + `live_routes`. It keeps the `spectrum` rows that A's own `spectrum_pairing` names. +- The scalar fields keep P0's values (`plan_identity`, `sample_rate_hz`, `quantum_frames`, + `spectrum_collection`). `spectrum_pairing` and `restarted` describe A against P0, not P0, so + the restricted base holds them empty. +- Gate 2 covers each family above, one row each, not only sources and input moves. Its test + value covers the same defect for every family. +- The restart set still works across a withdrawn plan. A strip A restarted has no row in the + base, but the joins come from the committed models. `resolve_strip_carry` (`prepare.rs:4799`) + and `resolve_effect_carry` (`:5197`) mark a strip restarted when its predecessor row is + missing. So B lists it in `restarted_strips()` and #1324 ducks it. +- Size: nine families make this slice larger than its brief assumed. The rows are mechanical; + the attempt budget stands. + +**D2-D6 hold** against the code. Decision 17 does not touch them: the base and donation are +host-core internals, with no host surface. +- D3's donor is an `UnadoptedCandidate` (#1343, `crates/engine/src/realtime/plan_exchange.rs:251`). + It gives `plan()` (shared) and `into_plan()`. `check_donation` needs only `plan()`. + `apply_donation` needs `&mut` on the donor; #1310 calls it after the commit, when A is never + republished, so `into_plan()` serves. No `plan_exchange.rs` edit is needed. +- D5's flag is one render-side store in `begin_block_with`. The non-goal "Any render-thread + change" means no change beyond that store. `realtime-policy` covers it. + +**Dependencies on GitHub.** +- #1277 is closed, and so are #1279-#1284, #1311 and #1343. +- #1325 (open) is named in gate 3 for scope only. It is not a dependency. +- Downstream: #1310 (batch 15a), #1358 and #1397 (open, not in batch 15). + +**Authorized paths.** Unchanged. The spectrum rows are read in `prepare.rs`, so +`crates/host-core/src/spectrum.rs` is not needed. + +**Gates and commands.** Every named package, feature and script exists: host-core features +`control-provider` and `test-support`, graph `test-support`, `scripts/check-graph-policy.sh`, +`scripts/check-host-core-policy.sh`, `tools/realtime-policy`. `withdrawn_successor.rs` is new here. + +**Realtime-path edits.** None under `crates/engine/src/realtime`. D5 adds one `bool` store to the +source consumer's block entry (`crates/source/src/lib.rs`, render-side). D4's swap runs on the +control thread. + +## Decision record: #1344 (batch 15a) + +**Shipped.** +- D1: `graph::installed_carry_reads(&PreparedRenderPlan) -> Option` names every + executor field without `..` and reports a predecessor key for all nine carry sections (sources, + input lanes, `(owner, lane)` strip stages, effect sites, `(kind, location)` observers for meters + and spectrum, observation tap sites, delay lines, strip delay lines, live routes; joins and the + scratch read none). `PreparedHost::carried_base` is the base inventory restricted by + `PlanStateInventory::restricted_to` (destructures without `..`), keeping P0's identity, rate, + quantum and capture kind, with `spectrum_pairing` and `restarted` empty. New report row + `carried_base_retained_bytes`; new accessor `PlanStateInventory::spectrum_count`. +- D2: doc on `SuccessorBase`; no new entry point. +- D3: `check_donation(DonationSide, DonationSide) -> Result` + (read-only), `donate_producers`, `donate_consumers`, `apply_donation(DonationSideMut, + DonationSideMut, DonationPlan) -> usize` in `host-core/src/source.rs`. `DonationPlan { producers, + consumers }` holds the two halves; it borrows neither side. +- D4: `GraphPreparedSourceSetDriver::{can_swap_sources, swap_sources}` (source crate implements + them, whole-entry swap, all or nothing); `graph::{can_donate_sources, donate_sources}`. +- D5: `PcmSourceConsumer::has_begun_block`, one `bool` store in `begin_block_with`. + +**Departures.** +- The amendment's "check_donation needs only `plan()`" and "no edit under + `crates/engine/src/realtime`" cannot both hold: no shared executor downcast existed. Added + `PreparedPlanExecutor::as_any(&self)` (default `None`) and `PreparedRenderPlan::executor_any(&self)` + in `crates/engine/src/realtime/plan.rs` (control-side, no render code; one attempt, accepted). +- The driver trait also gained `as_any(&self)` and `source_has_begun_block`, needed to downcast the + donor through a shared reference and to type `DonationError::Consumed`. +- `installed_carry_reads` takes `&PreparedRenderPlan`, not `&mut`. +- Spectrum rows are kept by the observer section's installed `Spectrum` moves (D1's rule), with a + debug assertion that they equal the predecessor entries of A's `spectrum_pairing` (the + amendment's wording). +- A successor with no section installed gets `Some` (P0's identity, no rows), not `None`: its + replacement still needs a base, and nothing carries. +- Gate 5's "declaration or ring-config mismatch is refused": D3 selects only sources with equal + declaration and ring, so a mismatched `c` is not selected; the check returns an empty plan and + nothing moves. A typed refusal would block every other donation and refuse a legitimate + transaction that redeclares a source; the redeclared source's old PCM is not owed, exactly as in + the existing carry rule (#1272 gate 3). `DonationError` is `Consumed` and `Unswappable`. +- Gate 1's "insert quality" edit does not prepare (no launch effect declares a second quality: + `effect.quality.unsupported`); the tests replace the insert's effect (compressor to + gate-expander) under the same insert ID. Gate 1/3/4/5 sessions have no console slot, because on + an AVX2 target a console slot at `Backend::Simd4` fails `console.slot.unbanked` even for the + unmodified fixture (see successor work). + +**Gates** (`crates/host-core/tests/withdrawn_successor.rs`, all pass): +1. `a_successor_prepared_across_a_withdrawn_plan_renders_as_its_twin` (Simd8 and Simd4): 16 + blocks bit-identical to the twin, `a` carried (B's reads name P0's index), `b` absent, + `restarted_strips() == [t1, t2]`; a fresh B differs. +2. `the_carried_base_holds_exactly_the_rows_the_hand_over_reads`: all ten inventory families + non-empty in A's reads, base count equals read count per family, P0 identity, source rows equal + P0's, effect lanes and observations strictly fewer than P0's; a first plan has `None`. +3. `a_source_absent_from_the_running_plan_is_never_carried_from_the_withdrawn_one`. +4. `a_donated_ring_plays_its_acked_pcm_and_held_seek` (Simd8 and Simd4): one pair, apply returns 1, + bit-identical to the twin; without the donation it differs; every rendered block makes zero + allocator calls (`bench_support::alloc` thread counters). +5. `a_refused_or_mismatched_donation_moves_nothing`. +6. `cargo test --locked -p host-core --all-features` (41 targets ok), `-p graph --features + test-support`, `-p source`, `-p host-web --all-features`: pass. `-p capi`: passes with the + mechanical one-line fix below applied (the coordinator later committed it; see below). `audit capi`: `pcm_digest + bf7dcedd1d92621f`, 0 violations. `realtime-policy`: ok, 150 regions (none added). + `check-graph-policy.sh`, `check-host-core-policy.sh`, `check-workspace-policy.sh`, fmt, clippy + on `engine`, `graph`, `source`, `host-core` (`--all-targets --all-features -D warnings`): pass. + +**Edits outside the original authorized paths, and who accepted them.** +- `crates/engine/src/realtime/plan.rs`: the read-only `PreparedPlanExecutor::as_any` and + `PreparedRenderPlan::executor_any` (control-side, no render code); one attempt, accepted by root + on 2026-10-16. +- `crates/host-core/src/lib.rs` (re-exports) and `crates/host-core/src/render_session.rs` (the + `carried_base` field): mechanical. +- `crates/capi/tests/resource_lifecycle.rs`: the coordinator added the `carried_base,` + destructure and `assert!(carried_base.is_none())` (mechanical). This corrects the attempt's + record, which said the file was not edited: the attempt did not edit it, the coordinator did. + +**Review fold (batch 15a).** +- M3, gate 2: the session now has track `t` and source `a`, which sort first, and A removes both, + so A's keys are not P0's; A also redeclares `c` and replaces `t0000`'s insert. Under live input + lanes an input-section or strip-stage value edit carries and retargets, so those families (and + meters, delay lines, strip delays, live routes) restart by removal only; the plan's one spectrum + capture carries. The test compares sets, not counts, through two test-support views + (`test_only_inventory_rows`, `test_only_carry_read_keys` in `prepare.rs`): base keys equal read + keys, and base owners equal the owners both plans hold less those A restarted. Red evidence: + `restricted_to` returning the full inventory fails "base keys equal read keys"; + `installed_carry_reads` reporting successor keys fails for each section on its own (sources, + delay lines and strip delays on the owner set; input lanes, strip lanes, meters and live + routes on the key set; effect sites and observation taps on "base rows are P0 rows"; spectrum + on prepare's own pairing assertion). +- M3, gate 1: gate 1's plans are now prepared with live controls, so inserts bank with live + lanes and can carry. `t3` carries a transient shaper (audible state, its own bank), and every + input section carries its high-pass state. Red evidence: dropping the effect-lane or + input-section rows from the restriction fails gate 1. The reviewer's "an unrestricted base is + red" assertion is not added. That assertion and root's item 7 cannot both hold: the restricted + base and P0's full inventory differ only in rows A restarted or removed, and an owner from + those rows carries from P0 only when B reverts it to its P0 form. So the assertion would lock in + the behaviour that root ruled a defect (an owner or source that A restarts and B reverts starts + fresh in B). Item 7 is not small, and it is not coded here, for three reasons. (1) The fix + removes the restriction: the control plane passes P0's inventory as the base, which reverses D1 + and D2. (2) It needs a P0 producer pass before the donor pass: for a source that A redeclared and + B reverts, the donor's producer feeds A's ring and not the P0 ring that B reads. (3) It must + decide where host submits made between A's commit and B's swap go. Successor placeholder (the + coordinator writes it): *B carries an owner or source it keeps unchanged from P0 from P0's state*. +- m1: `donate_producers` now moves the donor's producer into the successor and leaves the donor + vacant, so a later submit or seek to the donor is refused (`Vacated`) and never acked. The + successor's own fresh producer is dropped. `check_donation` refuses a successor producer that + has already been fed with the new `DonationError::SuccessorFed`, so that drop never loses acked + PCM. `apply_donation` first re-reads both halves' preconditions (producer pairs resolve, + `can_donate_sources`) and moves nothing if either fails. It then runs the consumer swap (all or + nothing) and, only if that moved, the producers, so a donated ring and its producer are never + separated. Test: `a_donation_never_separates_an_acked_ring_from_its_producer`, and gate 4 now + asserts the donor's `Vacated`. Red evidence: restoring the producer swap fails gate 4; removing + the fed check fails the new test; the old producers-first order fails the new test. +- m2: `crates/source` tests `a_source_swap_refuses_every_unswappable_pair_and_moves_nothing` + (every `swappable` refusal: either index out of range, either entry vacant, channel counts, quanta, + either index repeated, a foreign driver; nothing moves) and + `a_source_swap_carries_a_prepared_seek_into_the_next_block`. Each `swappable` condition removed + on its own and the `pending_generation_change` OR removed are each red. The OR stays set-wide. + A comment at the OR records why this is safe. It can only over-report one block's + generation-change validity and never hides a real change. It cannot be reached in production, + because a donor never had `prepare_source_seek`. A per-entry flag would add reads on render. + +**Open, outside this slice's paths (resolved).** `crates/capi/tests/resource_lifecycle.rs:810` +destructures `PreparedHost` exhaustively and needed `carried_base,` plus +`assert!(carried_base.is_none())`. The attempt did not edit it (batch 15a rule for `crates/capi`). +The coordinator added it, as recorded above. + +**Review fold 2 (batch 15a).** +- *m-2.* A release-build probe fed B's producer between `check_donation` and `apply_donation` and + lost the acked chunk: `producers_resolve` did not re-read `unfed`. It now requires the + successor's producer to be unfed, so the apply refuses and moves nothing. `donate_producers` and + `donate_consumers` are `pub(crate)` (no caller outside `host-core`) and no longer re-exported. + Test `a_successor_fed_between_the_check_and_the_apply_moves_nothing`: apply returns 0, both + sides keep their rings, B's producer continues after its acked chunk and A's is not vacated. + Red with the `unfed` re-read removed: the `debug_assert!` in `donate_producers` (debug) and + "a fed successor refuses the apply" (release). +- *m-3.* `a_seek_or_an_empty_end_of_region_on_the_successor_refuses_the_donation`: a seek alone + (generation 2) and a seek to the region's end closed by a zero-frame end-of-region chunk each + give `SuccessorFed`. Red with `unfed` reading only the written frames ("seek only"). A + zero-frame end-of-region chunk at generation 1 is not reachable: the validator refuses a + zero-frame source and the ring refuses the chunk as non-contiguous, so the chunk follows a seek. + The removed pre-check is caught by the m-2 test above. +- *m-4.* The doc of `a_donation_never_separates_an_acked_ring_from_its_producer` no longer claims + the old producer swap; it names gate 4's `Vacated` assert + (`a_donated_ring_plays_its_acked_pcm_and_held_seek`) as the catch. +- *Reverted-source guard (root ruling of 2026-10-16).* `host_core::drops_acked_reverted_source` + (`source.rs`, beside `check_donation`, reusing `unfed`) reports a source the donor redeclared + from the running plan, fed since, that the successor declares as the running plan does: the one + case the donation cannot move and the acked chunks would drop. #1310's rebuild refuses it as + `BACKPRESSURE` (see #1310's fold 2); #1698 lifts it. +- *Fold 3 (M-A, root ruling of 2026-10-16).* The guard is widened and renamed + `host_core::drops_acked_fresh_source(donor side, &DonationPlan)`: any fed ring the donor created + (`can_prepare_source_seek` on the donor, as in `check_donation`) that the donation does not move + is refused, so B reverting, redeclaring a third way or removing the source, and A adding it, are + all covered; a source the donor carried from P0 is not. Unit test + `withdrawn_successor::the_guard_refuses_exactly_a_fed_fresh_ring_the_donation_leaves_behind`; + see #1310's fold 3. diff --git a/.github/ISSUE_SPECS/1360-check-the-warm-successor-deadline-in-miso-engine-v1-service-and-report-its-outco.md b/.github/ISSUE_SPECS/1360-check-the-warm-successor-deadline-in-miso-engine-v1-service-and-report-its-outco.md index 78173e244..56103d79d 100644 --- a/.github/ISSUE_SPECS/1360-check-the-warm-successor-deadline-in-miso-engine-v1-service-and-report-its-outco.md +++ b/.github/ISSUE_SPECS/1360-check-the-warm-successor-deadline-in-miso-engine-v1-service-and-report-its-outco.md @@ -209,3 +209,9 @@ A mobile host that adds a latent effect during playback gets a seamless swap thr - *Publish an applied-revision watermark and complete edits asynchronously* (#1314). - *Report each configured counter's own value in the C ABI counter snapshot* (#1351): the `TRANSITION_REPREPARE_REFUSALS` ID. + +## Amendment (controls coordinator, batch 15a, 2026-10-15) + +- `AdmissionPeak::WithReprepare` named above did not ship with #1398 (root ruling: code with no + production caller is a placeholder). #1403 adds the reprepare admission peak together with its + first caller; see its batch 15a amendment. diff --git a/.github/ISSUE_SPECS/1381-swap-and-retire-browser-plans-through-the-workers-service-loop.md b/.github/ISSUE_SPECS/1381-swap-and-retire-browser-plans-through-the-workers-service-loop.md deleted file mode 100644 index 357e09db3..000000000 --- a/.github/ISSUE_SPECS/1381-swap-and-retire-browser-plans-through-the-workers-service-loop.md +++ /dev/null @@ -1,561 +0,0 @@ -# Swap and retire browser plans through the Worker's service loop - -Stream H of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-3, D15-10, D15-11, D15-17). -Code anchors verified on `main` at `6fb211594`. - -This slice follows *Run the browser control plane in a Worker and keep the AudioWorklet -render-only* (#1332) and *Move browser source submission and seeks into the Worker* (#1387). It -builds on the adapter preparation hook of *Prepare through an adapter-supplied preparer in the -control-plane crate* (#1400). - -## Product outcome - -The browser engine holds its plan the way the C ABI does. -- The Worker owns the control plane's session state (`control_plane::SessionState`: the committed - model at revision 0, the source producers, the plan publisher and the retirer). -- The worklet renders through `RealtimePlanOwner`. A published successor is adopted at a block - boundary, together with its render-side companions, and the clock continues. -- The Worker runs the service step continuously through a new export. Every retired plan and its - companions are reclaimed in the Worker, never on the audio thread. -- The Worker reads the applied-revision watermark and its counters through a new export. - -Rendered bits do not change. This is the substrate that *Replace the running browser session in -the Rust host* (#1290) publishes through. - -## Context - -- **The browser renders the plan directly.** `render_next` calls `ready.host.plan.render` - (`hosts/host-web/src/lib.rs:3212`, `:3234-3239`). There is no publisher, no candidate and no - retirement. -- **The exchange carries only the plan.** - - `plan_exchange` returns publisher, owner and retirer - (`crates/engine/src/realtime/plan_exchange.rs:194`). The owner holds - `active: (PlanEpoch, PreparedRenderPlan)` (`:80-92`). - - `RealtimePlanOwner::enter_block` (`:375`) adopts a candidate, continues the clock, runs - `carry_from`, and commits the old plan to the retirement queue. - - `PlanRetirer::try_reclaim` (`:512`) returns `(PlanEpoch, PreparedRenderPlan)` and nothing - else. `PlanReplacementReservation::cancel` (`:321`) hands an unpublished plan back. -- **The browser's render-side companions live beside the plan**, in `ReadyOwnership` - (`hosts/host-web/src/lib.rs:1509`): `spectrum_capture` (`:1578`), `meters` (`:1582`), - `effect_observations` (`:1586`) and their bookkeeping. Nothing carries them with a plan. -- **The control plane is capi's today.** *Extract the C ABI control plane into a portable crate - both hosts call* (#1309) moves it to `crates/control-plane` (lib `control_plane`): - - `SessionState` (today `crates/control-plane/src/control.rs:321`) owns the controller, - provider epochs, publisher, retirer and render diagnostics; - - its source routing is `newest_providers`, `submit`, `seek` and `seek_at` (today - `control.rs:1628`, `:1641`, `:1655`, `:1671`); - - `PlanState` (today `crates/control-plane/src/plan.rs:57`) wraps `RealtimePlanOwner` and - `SharedPlanState` for render. -- **Sources.** After #1387 the producer half of every source ring (`SourceControlSet`, the - `sources` field of host-core's prepared runtime, `crates/host-core/src/prepare.rs:529`) lives in - the Worker, which calls it directly. The worklet holds only ring consumers. -- **The render-activity code.** #1309 D7 keeps `RENDER_DIAGNOSTIC_CODE` as - `capi.render.activity`, because `host-core` sizes the slot by that literal - (`crates/host-core/src/control_provider.rs:117`). -- **No browser export exists for the service step or the watermark.** #1348 D1 defines - `SessionState::service()`. #1314 D4 defines the record and `PlanPublisher::watermark_reader()`. - The browser's frozen export lists are `expected_exports` - (`scripts/check-web-audioworklet.sh:203`), `EXPORTS` - (`tools/parameter-metadata/src/abi_layout.rs:135`) and `EXPORTS` - (`scripts/check-abi-layout-v1.py:117`, run by `check-web-audioworklet.sh:562`). - -## Decisions frozen for this slice - -- **D1. Boot.** - 1. The Worker's boot builds `control_plane::SessionState` from the boot document through the - browser's `RuntimePreparer`, `WebRuntimePreparer` (#1400 D5). The committed model is at - revision 0. - 2. `SessionState` takes the Worker's `SourceControlSet` (#1387) into its provider epochs. It is - the one owner of the producers from then on. - 3. Boot moves the render half (`control_plane::PlanState`, whose plan carries the companions of - D3) into #1332's transfer token. - 4. Browser-only boot steps run as today, in the same order: the document budgets, the VCA - bound, the bridge buffers. -- **D2. Source routing.** The Worker's source entry points (`miso_engine_web_v1_source_submit`, - `_source_seek`, and the native `seek_at` and seek report) call `SessionState`'s source routing, - never the set directly. A source that the newest committed session removed therefore follows - *Remove a strip in two phases: ramp out, then a scheduled swap* (#1325) D4 on the browser as on - the C ABI. Feeds, messages and replies are unchanged. -- **D3. Companions travel inside the plan.** - - `PreparedRenderPlan`'s `host_attachment: Option>`, with - `set_host_attachment` and `host_attachment_mut`, comes from #1400 D4. The browser preparer - sets it at preparation, before publication. - - `RealtimePlanOwner` gains `active_attachment_mut::() -> Option<&mut T>`, a - downcast of the active plan's attachment. It is render-side and allocation-free. - - The browser's attachment is #1400 D5's `RenderCompanions` struct: the meter consumers, the - effect observation handles and the spectrum capture. This slice adds to it the per-plan meter - and observation bookkeeping that `ReadyOwnership` holds today. The Worker prepares it with the - plan. - - **Pairing hook.** `HostAttachment` gains `fn adopt_predecessor(&mut self, predecessor: &mut - dyn HostAttachment, carries: &[ObserverCarry]) {}` (empty default). `PreparedRenderPlan:: - carry_from` (`crates/engine/src/realtime/plan.rs:917`) calls it after the executor hand-over - when both plans have an attachment, with the successor's `observer_carries()` (*Carry meter - and effect observation state across a plan swap*, #1327 D3). It therefore runs on the render - thread, in the swap block, after #1327's and #1395's sections moved the carried observers, - before the successor renders, and before the predecessor is committed to retirement. - - `RenderCompanions::adopt_predecessor` downcasts the predecessor through `as_any_mut`; another - type, or no carry record, pairs nothing. Otherwise it pairs the three reader kinds below. Every - step is a swap of two values or a scalar copy: no allocation, no drop. - - **Meters.** The browser's `meters` vector is host-core's `HostLiveControlHandles::meters` in - its own order (`hosts/host-web/src/lib.rs:7188`), so the carry record's meter indices address - it directly. It calls #1327 D3's `pair_carried_meters`, and swaps each pair's `meter_pending` - entry (`:1632`, one per meter) the same way. - - **Observations: two index spaces.** The carry record's observation indices are positions in - host-core's dense `effect_observations` vector (`crates/host-core/src/prepare.rs:452`). The - browser stores the handles in a gapped box indexed by its dense `effect_slot` - (`hosts/host-web/src/lib.rs:1586`), filled by `dense_effect_slot` at preparation (`:7031-7058`), - so the two indices differ and shift when a strip is added. So the companions gain - `observation_reader_slots: Box<[u32]>`: entry `i` is the browser slot that host-core reader `i` - was stored at, written in that same preparation loop. The browser does not call - `pair_carried_observations` (its slices are host-core's dense order). For each `Observation` - record `{ successor: s, predecessor: p }` it maps `s` and `p` through its own and the - predecessor's tables and swaps the two slots' `effect_observations`, `observation_armed` and - `observation_arm_samples` entries. `observation_tracks` and `observation_present` are never - moved: they describe the successor's own strips, so a carried reader folds into its strip's - new index. - - **Spectrum.** #1395 D3's `pair_carried` for its single capture or collection. - - **Meter delivery state.** The state that belongs to the meter stream, not to one reader, is - taken from the predecessor's companions: - - copied: `meter_generation`, `meter_snapshot_generation`, `meter_snapshot_drops_seen`, - `meter_loss_count` and `meter_windows`; - - from the meter header, only the delivery words: `windows`, `first_sample`, `end_sample`, - `sequence`, `reserved[0]` and `reserved[1]`, and `master_gr_present` (it describes the last - published frame). The shape words (`struct_size`, `abi_version`, `track_count`, - `submix_count`, `master_track_plus_one`) stay the successor's (`WebMeterHeader`, - `:1128-1156`). The `f32` frame is not copied: its shape may differ, and the next fold writes - it whole in the successor's shape; - - the master window state is output-level and continuous across the swap, so it moves: the - `master_windows` ring is swapped (both are sized by the first meter's queue capacity, - `:7107-7110`), and `master_read`, `master_write`, `master_count`, `master_peak`, - `master_start_sample` and `master_end_sample` (`:1619-1629`) are copied. Without them the - first fold after the swap finds no master window for its span and fails `master_valid` - (`:3600-3603`). - - So after the swap block the active companions hold the readers of the observers that render. - Carried readers keep their sequence, and the browser's `meter_generation` does not advance - (#1290 gates it on a structural apply). - - Adoption: the companions become active at the same block as their plan, because they are part - of it. A block whose `enter_block` returned `Applied` reads the new attachment before it - renders. - - Freeing: `enter_block` moves the displaced plan, attachment included, into the retirement - queue. The Worker's service step reclaims it with `try_reclaim` and drops it there. A candidate - that is never adopted returns to the control side (`cancel`, supersession #1310, withdrawal - #1343) and drops there. Render never drops an attachment. - - The vtable calls (`as_any_mut`, `Any::type_id`, `adopt_predecessor`) are new `call_indirect` - sites in render's closure, added to #1333 D4's pinned table with this reason. -- **D4. Render.** `render_next` renders through `PlanState`'s `RealtimePlanOwner`. -- **D5. Two new exports.** Only the control side calls them: the Worker, or in `single` mode the - worklet's control handler. Neither is in the render-locked set. - - `miso_engine_web_v1_service(handle: u32) -> u32` runs `SessionState::service()` once - (#1348 D1). Results: `RESULT_OK`; `RESULT_INVALID_ARGUMENT` for a wrong handle; an epoch - failure as `RESULT_INTERNAL` with the session diagnostic. It commits and acknowledges nothing. - - `miso_engine_web_v1_watermark_read(handle: u32) -> u32` copies the watermark and its counters - into a fixed record, `WebPlanWatermark`, found at `miso_engine_web_v1_watermark_ptr(handle) -> - u32`. The record is 96 bytes and mirrors `miso_engine_v1_watermark` (#1314 D6) field for - field, in its order: `struct_size`, `reserved0`, `revision`, `first_sample`, `outcome_flags`, - `exact_count`, `transition_fallback_count`, `superseded_count`, `reserved[5]`. It has no - pre-roll counter (D15-8 (round-5 amendment)). It lives in the control half and is allocated - at boot. A read that gives up after #1314 D4's attempts returns `RESULT_BACKPRESSURE` and - leaves the record untouched: a read to retry, never a refused edit. - - The read calls `SessionState::watermark()`, the accessor that *Publish the applied-revision - watermark in the browser status* (#1349) D3 names: a read of #1314's `PlanWatermarkReader`, - cloned from `PlanPublisher::watermark_reader()` when the session is built. Root orders this - slice before #1349, so this slice adds the accessor and #1349 reuses it. - - All three names join every frozen export list. The record joins the ABI layout. The generated - SDK copies are regenerated. -- **D6. Service loop.** - - `worker` mode: the Worker calls `miso_engine_web_v1_service` after every message it handles, - and on the repeating one-quantum timer that #1387 starts for the source drain. The browser may - clamp the timer; correctness does not depend on the period, because deadlines are counted in - render samples (D15-17). - - Both modes run the same service step, so any step a later slice adds to it runs the same with - or without cross-origin isolation. The Worker runs no catch-up: a warm successor is adopted by - render (D15-8 (round-5 amendment)), and its deadline check is #1361's. - - `single` mode: no control work runs inside `process()`. The worklet's control handler calls - the export after every control message. The main-realm host also posts an internal - `miso.service.v1` message to the worklet, on #1332 D7's control-plane-to-worklet port, from a - one-quantum `setInterval`, and the handler services on it. The worklet may not use timers. - `check-web-audioworklet.sh` bans `setInterval` in the main-realm host today (`:512-515`); this - slice changes that rule to exactly one pinned `setInterval(` site in the main-realm host, the - single-mode service tick, cleared at dispose, with a red self-test mutation (a second site). - The worklet's ban is unchanged. This runs outside the - render-locked window, and its allocations and frees count in #1332's - `singleModeControlAllocations`. - - The loop starts when boot succeeds and stops before dispose. -- **D7. Render-activity code.** The browser uses the crate's `capi.render.activity` code - unchanged. Renaming it would change a protocol-visible code on both hosts and the literal that - host-core sizes by. -- **D8. Acked-batch question: can an ack ever precede a drop? No.** This slice adds no queue that - carries an acknowledged command. The service step acknowledges nothing (#1348 D6). A retired - plan is already displaced. An unadopted candidate drops only on the control side, under the - completion rules of the issue that displaced it. - -## Deliverables - -1. D3's owner accessor in `crates/engine/src/realtime/plan_exchange.rs`; D3's hook in - `crates/engine/src/realtime/plan.rs`; `RenderCompanions::adopt_predecessor` in host-web. -2. D5's `SessionState::watermark()` accessor in `crates/control-plane`. -3. D1, D2, D4-D6 in `hosts/host-web/src/lib.rs`, `hosts/host-web/src/ffi.rs`, and the control - Worker, worklet and main-realm host scripts. -4. The three export names in every frozen export list, the record in the ABI layout, and the - regenerated SDK copies. -5. Test-support API, public only under `host-web/test-support`: #1387's `boot_split_for_test` - now returns a `ControlHalf` that holds the `SessionState`; add `ControlHalf::service()` and - `ControlHalf::republish_committed_for_test()`, which prepares a successor of the committed model - through the browser preparer and publishes it, and `ControlHalf::publish_successor_for_test( - document)`, which prepares a given document through the browser preparer and publishes it - without committing it. They exist only for gates 2-5; no export reaches them. -6. The integration test binary `hosts/host-web/tests/plan_exchange_threads.rs`. - -## Authorized paths - -- `hosts/host-web/src/{lib.rs,ffi.rs,tests.rs}`, `hosts/host-web/tests/plan_exchange_threads.rs` - (new), `hosts/host-web/Cargo.toml` (the `bench-support` dev-dependency if no earlier slice - added it; #1400 adds the `control-plane` dependency), `hosts/host-web/web/` -- `scripts/check-web-audioworklet.sh` (`expected_exports`, the capability rule for the new - message, and D6's one pinned main-realm `setInterval(` site with its self-test mutation), - `scripts/check-web-audioworklet-callgraph.py` (the pinned indirect sites) -- `scripts/test-web-audioworklet.mjs` (the fake exports and the single-mode service tick) -- `sdk/assets/miso-engine-v1-abi-layout.json`, `sdk/src/generated/abi.ts` (regenerated only) -- These are outside stream H's ownership; root sequences them after #1309, #1314 and #1348: - - `crates/engine/src/realtime/plan_exchange.rs`: D3's owner accessor only; - - `crates/engine/src/realtime/plan.rs`: D3's `adopt_predecessor` trait method and its call in - `carry_from` only; - - `crates/control-plane/src/`: the `watermark()` accessor only; - - `tools/parameter-metadata/src/abi_layout.rs` (`EXPORTS` and the record's fields), - `scripts/check-abi-layout-v1.py` and `scripts/fixtures/abi-layout-v1-self-test.json`. - -## Non-goals - -- No structural edit (#1290). No live edit in the Worker (*Admit browser live edits in the Worker - through the committed model*, #1562). -- No warm-successor deadline check (*Check the warm-successor deadline in the browser Worker's - service loop and report its outcome*, #1361). -- No watermark words in the browser status block (#1349). -- No adapter preparation hook and no plan attachment field (#1400). - -## Objective gates - -1. **Same bits.** The browser legs' native-digest gate and the `direct-oracle.mjs` parity pass - unchanged in Chromium, Firefox and WebKit, on the isolated leg and the non-isolated leg. -2. **Swap and reclaim, two threads.** Integration test binary - `hosts/host-web/tests/plan_exchange_threads.rs`. Decision 15 rules that host-web's native - allocation-count gates live in an integration binary, never in `src/tests.rs`, which already - registers a `#[global_allocator]`. It links `bench_support::alloc` and calls - `assert_installed()` first. - - The control half runs on thread A, the render half on thread B. Boot nine tracks with meters, - one observed effect and a spectrum capture. Render 6 blocks. - - On A, call `republish_committed_for_test()`. Render 6 more blocks on B. - - The block after publication reports `SwapOutcome::Applied`. The clock continues. Every block - is bit-identical to a run without the republish. - - Every observer carries (the model is unchanged), so the hook pairs every reader. A meter - poll, an observation read and a spectrum read after the swap are served by the new plan's - attachment and continue the old sequences: no sequence gap, `meter_generation` unchanged, - no counted meter loss, no spectrum `Failed` epoch. - - On B, the thread-scoped counters read `allocations == 0 && frees == 0` around every render - call. - - The next service step on A reclaims exactly one plan. The old attachment is dropped on A - (a drop counter on `RenderCompanions`, test-support only). -3. **A shifted strip keeps its readers and its master window.** Same binary and threads as - gate 2. Boot tracks `b` and `c`, meters on both, `c` designated master, and a live-controlled - compressor insert on `c` with its gain-reduction tap armed. Render until a meter window is open. - Publish, with `publish_successor_for_test`, the same document plus a muted track `a` whose ID - sorts first, so `c` moves from strip 1 to strip 2 and its effect slot moves too. Render on. - - The first meter poll after the swap folds a window: its header has `track_count` 3 and - `master_track_plus_one` 3 (the successor's shape), its `sequence` is the last pre-swap - sequence plus one, `reserved[0]` is unchanged, and `reserved[1]` has `METER_VALID_MASTER` set - and a loss count of 0. - - From that poll on, every frame's peak and gain-reduction words equal those of a fresh boot of - the successor document fed the same input from frame 0. `c`'s gain reduction is in strip 2's - word, strip 0's (`a`) and strip 1's (`b`) are 0, and `masterGrDb` is present. -4. **Single mode.** Gate 2 on one thread gives the same blocks. The retired plan and its - attachment are reclaimed by the first service call after the swap block. -5. **Watermark export.** In gate 2's script, `miso_engine_web_v1_watermark_read` returns - `RESULT_OK` and a record equal, field for field, to `SessionState::watermark()`. A wrong handle - returns `RESULT_INVALID_ARGUMENT` and writes nothing. -6. **Exports are frozen.** `bash scripts/check-web-audioworklet.sh` (which runs - `check-abi-layout-v1.py` over the shipped layout) and `bash scripts/check-sdk-generated.sh - target/ci/qualification-artifacts` pass. `python3 -B scripts/check-abi-layout-v1.py --self-test` - passes and turns red if one list omits an export or the record omits a field. - `check-web-audioworklet.sh`'s self-test turns red on a second main-realm `setInterval(` site - and on a `setInterval` in the worklet. -7. **C ABI unchanged.** `cargo test --locked -p capi` passes. `target/release/audit capi` reports - 0 allocations, locks and syscalls, with the same `pcm_digest` as the base (PR evidence). -8. **Commands:** - - `cargo test --locked -p host-web --features host-web/test-support` (runs gates 2-5) - - `cargo test --locked -p control-plane --features test-support` - - `cargo test --locked -p engine --features engine/realtime-audit` - - `cargo test --locked -p capi` - - `cargo build --locked --release -p audit -p capi && target/release/audit capi` - - the artifact build and `bash scripts/check-web-audioworklet.sh` in both forms that #1332 - gate 7 spells; `bash scripts/test-web-audioworklet.sh` - - `npm run qualify -- --artifacts ... --sdk-root ... --browser --check-matrix --self-test-mutations` - in `hosts/host-web/qualification`, for chromium, firefox and webkit - - `bash scripts/check-cross-targets.sh`, `bash scripts/check-host-core-policy.sh` - - `cargo fmt --all -- --check`, - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `bash scripts/check-workspace-policy.sh` - -## Test value - -- Gate 2: turns red if the browser still renders the plan directly (no adoption), if the clock - restarts on adoption, if a swapped plan is served by the old companions, if the hook is not - called or pairs nothing (the new attachment's readers would sit on producers that left with the - retired plan, so every meter and the spectrum stream go silent or restart), if the render - thread drops a retired plan or attachment, or if the Worker never reclaims them. -- Gate 3: turns red if the swap copies the whole meter header (the successor would report the old - `track_count` and master index), if the master window state is not moved (the first fold fails - `master_valid`, so `METER_VALID_MASTER` is clear and a loss is counted), or if observation - readers are paired by host-core index or by `observation_tracks` moved with them (`c`'s gain - reduction lands in the wrong strip's word). Gate 2 cannot catch these: its successor has the - same shape and the same indices. -- Gate 4: turns red if single mode leaks retired plans, or reclaims them inside the render - export. -- Gate 5: turns red if the export reads anything but #1314's record (for example the status - words, which carry no counters), or writes for a wrong handle. -- Gate 6: its export-list self-test is not new. The new mutation turns red if the timer rule is - relaxed beyond the one single-mode service tick, which would let control timers into the - main realm unpinned. -- Gates 1 and 7 are not new. They hold the bits of both hosts across the move. - -## Dependencies - -- *Run the browser control plane in a Worker and keep the AudioWorklet render-only* (#1332). -- *Move browser source submission and seeks into the Worker* (#1387). -- *Extract the C ABI control plane into a portable crate both hosts call* (#1309). -- *Add miso_engine_v1_service for bounded control work between edits* (#1348). -- *Publish an applied-revision watermark and complete edits asynchronously* (#1314). -- *Prepare through an adapter-supplied preparer in the control-plane crate* (#1400). -- *Carry meter and effect observation state across a plan swap* (#1327). -- *Carry spectrum capture state across a plan swap* (#1395). - -**Decision 17 amendment (2026-10-13).** This slice lands as filed. #1596 then removes the spectrum capture from `RenderCompanions` and D3's **Spectrum** pairing bullet: the browser's spectrum readers move to the control plane in the Worker and are paired at promotion, as on the C ABI (#1593 D5). - -## Amendment (decision 17, root 2026-10-13) - -**Single instance first.** This slice is split. Both pieces stay in this issue. - -- **#1381a** lands first, in `single` mode only, on today's one-instance module (no #1332, #1387 or - #1380). - - D1: the one instance's boot builds `SessionState` through `WebRuntimePreparer`. It takes the - instance's own `SourceControlSet` (D1.2), revision gate and cell producers (D1.3: the render - half stays in the instance's `LiveHost`; there is no transfer token). - - D2: the instance's source exports route through `SessionState`. - - D4 as filed. - - D5 as filed, except that the watermark read calls `PlanQueries::watermark()` - (`crates/control-plane/src/plan.rs:111`), not `SessionState::watermark()` (root Q5). - - D6: `single` mode only. The main-realm service tick posts `miso.service.v1` on the node's - existing port. - - D3: companions travel inside the plan as the attachment, with `active_attachment_mut`; there - is no `adopt_predecessor`. - - #1381a also takes #1332 D1's control counter: `CONTROL_ALLOCATIONS` in `render_lock.rs`, the - export `miso_engine_web_v1_control_allocation_count`, and `singleModeControlAllocations` in - the host status. - - It adds one `SessionState` accessor that lets the adapter reach an effect producer's - observation queue for observe kinds 7 and 8 (ruling C9). The later browser-feature phase - deletes it with those kinds. - - Gates: 1, 2 (swap, clock, bits, reclaim; no reader pairing), 4, 5, 6 and 7, on the - non-isolated leg. - - Dependencies: #1400, #1309, #1314, #1348. -- **#1381b**: D3's `adopt_predecessor` pairing, gate 2's companion assertions and gate 3. - Dependencies: #1381a, #1327, #1395. - -The Worker deployment of D1-D6 is #1332 and #1387. It adds no logic: the same exports run in the -Worker. - -**Flip batch 6W (root ruling A1).** #1381a, #1562a, #1570b, #1563B and #1565A land together, in one batch of lane W. Each slice keeps its own commits and tests and is green at its own boundary; the batch runs the full CI-equivalent gate set once. If one slice fails, the batch stops: `main` never gets a partial flip, so it never has two writers of one plan's revision gate and cells. No transitional API lends `SessionState`'s producers or revision to the old command path. - -**Catalog held once (batch 6 review fold, root 2026-10-14).** The browser holds its control catalog in `PreparedHost.control_catalog`, which `ReadyOwnership.host` keeps for the session, and the browser report charges it (#1400 amendment). #1381a must not hold the catalog twice: when `SessionState` (through `SessionControlProvider`) takes the session's catalog, `ReadyOwnership` stops holding the prepared one, and the report charges one catalog, not two. - -## Evidence: #1381a (flip batch 6W, attempt 1) - -Landed on `codex/controls-batch-7`, single mode only, on today's one-instance module. - -- **D1.** Boot builds `control_plane::SessionState` through `compile_children` and - `WebRuntimePreparer`, with the instance's own sources, revision gate and cell producers. The - browser's early checks (budgets, shape, the VCA bound) run first, as before; their compiled model - and parse are dropped before the control plane compiles the session again, so the boot's transient - peak is one compile's. `SessionState` retains the one compiled model, read through the new - `SessionState::committed_session()`. -- **D2.** `source_submit` and `source_seek` call `SessionState::submit` and `seek`. -- **D3.** The companions are the plan's attachment. The meter poll and the observation reads reach - the active plan's `RenderCompanions` through `RealtimePlanOwner::active_attachment_mut` - (`PlanState::active_attachment_mut`). There is no `adopt_predecessor`: readers are not paired - (#1381b), so after a swap the host reads the successor's fresh readers. - Spectrum is the exception to "travels in the attachment": `SpectrumService` owns its captures - (host-core), so boot moves the capture out of the attachment into it, until #1596 moves the - spectrum readers to the control plane. -- **D4, D5, D6.** `render_next` renders through `PlanState`. `miso_engine_web_v1_service`, - `_watermark_read` and `_watermark_ptr` (the 96-byte `WebPlanWatermark`, read through - `PlanQueries::watermark()`), `_control_allocation_count` (`CONTROL_ALLOCATIONS`, every allocator - call outside a render-locked window) and `singleModeControlAllocations` in the status. The - worklet's control handler runs one service step after every message, and the main realm posts - `miso.service.v1` (no request ID, no reply) from the one pinned `setInterval(`. -- **Ruling C9.** `SessionState::effect_producers_mut()` lets observe kinds 7 and 8 reach an effect - producer's observation queue. - -### Deviations from the filed text - -1. **Two owner edits beyond D3's accessor.** `RealtimePlanOwner::can_prepare_source_seek`, - `prepare_source_seek` and `copy_response_snapshot` forward to the active plan (outside the - policy region of `plan_exchange.rs`), with thin `PlanState` wrappers. Without them the paused - seek (`seek_source` prepares the consumer so new PCM enters a full ring) and the live response - capture lose their only route to the plan once the owner holds it. -2. **`PARSE_TRANSIENT_MULTIPLIER` 20 to 26, open for root.** The worst 1 MiB boot document - (192 tracks, 4 inserts each, 768 effects) grows the wasm high-water to 25,427,968 bytes (batch 6 - final head: 16,121,856), where the root-ruled pin is 20,971,520 plus one page (21,037,056). The - growth is 4,390,912 bytes over it and is 24.25 MiB, under root's 25 MiB line. The pin stays 26 - only so the rest of the AudioWorklet gate runs; root rules on it. Nothing is held twice: the - catalog (`ControlCatalog`, 887,888 bytes in this document) is built once by the preparer and the - control plane's provider takes it, `ReadyOwnership` holds no `PreparedHost`, and the compiled - model is the control plane's one canonical copy (527,071 bytes) read through - `committed_session()`. Measured on a native allocator (peak 14,691,515 to 19,810,096; retained - after boot 6,450,366 to 13,449,640, so +6,999,274), by allocation site: the 768 effects' live - lanes from `attach_effect_live_controls` hold 4.0 MB (5.2 KB each: 2,304 + 720 + 720 + 672 + - 384 + 192 + 96 + 96 + 60), the render side's bank live lanes hold 2.4 MB (14,080 and 1,760 - and 1,280 bytes for each of 96 bank effects, and 4,096 bytes for each of 192 strips twice), and - the control plane's own fixed rows hold the rest (0.4 MB: the provider's 128 KiB diagnostics, the - 64 KiB replay cache, four 32 KiB queues, two 16 KiB frames). All of it is the lanes and protocol - machinery decision 17 C6 gives every session; none of it duplicates a browser structure. -3. **Retained budgets re-pinned** (`retained_ceilings.rs`, `expected.json` ceilings): the bridge - rows charge the control plane's `control_retained_bytes` (427,534 bytes for the one-strip - fixture), `builtin_retained_bytes` rises because the control plane attaches the strip input - lanes. The 427,534 bytes are the control plane's protocol machinery for one session, and the - catalog (9,556 bytes, charged since batch 6 as its own bridge term) is inside it, charged once: - the provider's diagnostics (131,072), the replay cache (65,536), four queue rings (32,768 each), - two control frames (16,384 each) and a 16,432-byte ring, the queue index rows (10,240 and 6,144), - the plan exchange, the epoch rows and the catalog. Nothing in it is a second copy of a browser - structure. Measured on x86-64 after the rebase onto batch 6's final head: the two bridge rows - are 1,584,073 and 1,604,684 (ceilings 1,637,632 and 1,660,288); the shipped module's are - 1,576,061 and 1,596,672 (ceilings 1,628,864 and 1,651,520). No ceiling moved by the rebase. -4. **The integration binary is `plan_exchange_threads.rs`, single-thread form** (gate 4), with the - module's own counting allocator instead of `bench_support::alloc` (the workspace has no such - crate). Its session has empty console sections, no insert and filters off: console slots, - inserts and input filters carry no state across a swap yet (#1281, #1282, #1277 D6), so a - session that holds one is not bit-identical after the adoption. Gate 3 and the pairing - assertions of gate 2 are #1381b's. -5. **Lanes follow the control plane** (decision 17 C6), not the boot words. The preparer widens the - effect lane when observation taps are requested, so a session with no live effect still boots. - -### Old-path behaviours that stop working until the next slices (#1562a, #1570b, #1565A) - -- Command kinds 1 to 6 and 9 to 17 (pan, matrix, fader, mute, effect parameter and bypass, solo, - trim, polarity, input filters, route gain, mute and matrix, VCA fader and mute) are refused whole - with `COMMAND_REASON_UNSUPPORTED_KIND`. Kinds 7 and 8 (observe) work. -- The prepared commands (`prepared_command_submit`: EQ targets, builtin input filters) are refused - whole as unsupported, the EQ and input-filter configuration copies answer `RESULT_UNSUPPORTED`, - and the live route count and IDs read zero. -- The live response capture (`track_response_capture`) works: on the nine-track EQ fixture it - answers `RESULT_OK` for `eq0`, `eq4` and `eq8` under the default options, with 64 command - records, with observation taps and with meters, and the two live-response FFI tests pass. The earlier note was wrong about the capture. What the SDK evals that - query a response (`live-response-evals.mjs` 312, `response-subscription-evals.mjs` 83) fail on is - their edit leg: the band-1 gain edit is refused `unsupportedKind`, so the response after the - edit equals the one before it. #1562a, which carries the edit kinds, restores them. -- A hazard for #1290: `SessionState::submit` and `seek` run the service step first, so the first - source feed after a published swap would reclaim the displaced plan inside the source export's - render-locked window (and inside the PCM-feed worklet's `process()`). Nothing publishes a plan - in the browser yet, so nothing triggers it today. - **Resolved (batch 7 review B1):** the browser's `submit_source` and `seek_source` call - `SessionState::submit_unserviced` and `seek_unserviced`, which run no service step; the C ABI keeps - `submit` and `seek` (service first). `render_locked_source.rs` measures the source exports after a - render with counter telemetry configured: zero render-locked allocations. -- `check-browser-expected-resources.py` stops at the command timeline digest (the identity digest - and the resource rows are checked first and hold); `check-sdk-headless.sh` fails 25 subtests, all - of them command-driven; 113 host-web unit tests and the two `control_parity.rs` tests are - `#[ignore]`d with one reason string, for #1565A to delete. - -### Decision record: boot budget (root, 2026-10-14) - -- The 1 MiB worst document's wasm high-water growth rose from 16,121,856 bytes (batch 6 head) to - 25,427,968 bytes (24.25 MiB) when the single instance began to own `SessionState` with decision - 17 C6's always-attached live lanes. Nothing is held twice (catalog 887,888 bytes and model 527,071 - bytes held once each). Native retained after boot: 6,450,366 -> 13,449,640 bytes. Breakdown: - per-effect live lanes 4.0 MB (768 effects x about 5.2 KB), render-side bank live lanes 2.4 MB, - control-plane fixed rows about 0.4 MB, per-strip input lanes 0.17 MB. This is the browser half of - the C6 evidence (#1555 recorded the C ABI half). -- Root ruled the multiplier 20 -> 25 (25 MiB = 26,214,400 bytes, the tightest whole multiplier - above the measurement). A CI measurement above 25 MiB stops the batch and goes to root. The - successor "Shrink the always-attached per-effect live lanes" (#1611) aims to return the pin to 20. - -## Amendment (controls coordinator, batch 10, 2026-10-14): #1381b - -Checked against `12d4a4f9d`. - -- **Spectrum leaves the hook.** The capture is in host-core's `SpectrumService`, not in - `RenderCompanions` (#1381a's D3 evidence). D3's **Spectrum** bullet is void now, not only after - #1596: spectrum continuity across a swap is #1395's control-side pairing at promotion, which - #1290 (d) calls. Gate 2's spectrum assertions ("a spectrum read ... no spectrum `Failed` epoch") - move to #1290 (d+e) gate 7. -- **Dependencies of #1381b:** #1381a, #1327a and #1327b. #1395 is removed. -- **Engine edit.** D3's `HostAttachment::adopt_predecessor` and its call in - `PreparedRenderPlan::carry_from` (now `crates/engine/src/realtime/plan.rs:1158`; the filed `:917` - moved). #1327's batch 10 amendment recommends that #1327a makes this edit with its own; then - #1381b edits no engine file and only implements `RenderCompanions::adopt_predecessor`. Otherwise - it is #1381b's own `plan.rs` edit, under its own one-attempt rule. D3's owner accessor - (`RealtimePlanOwner::active_attachment_mut`, `plan_exchange.rs:708`) landed with #1381a. -- **Master window.** #1581 (open) D5 deletes the browser's master ring. If it lands first, D3's - master-window bullets and gate 3's `METER_VALID_MASTER` part go with it. #1582 (open) moves the - meter fold into host-core's `MeterDelivery`, still in the render half; D3's delivery-state copy - then targets that struct. The readers stay in the render half in both cases, so the swap-block - hook stays. -- **Gates.** Gate 2's two-thread form needs #1332 (open); #1381b runs gate 2's pairing assertions - in the single-thread form, as #1381a ran gate 2, and gate 3. `bench_support::alloc` exists - (`tools/bench-support`, `assert_installed`); host-web may add the dev-dependency, as the - authorized paths allow. -- **Anchors in `hosts/host-web/src/lib.rs` moved** (the attachment is reached at `:1846` and - `:2652`); lane W re-reads them at implementation. - -## Decision record: #1381b the pairing hook (batch 11, worktree b8w) - -**Shipped.** No engine edit: #1327a's `HostAttachment::adopt_predecessor(predecessor, carries)` is enough. All edits are in -`hosts/host-web/src/lib.rs` and `src/tests.rs`. - -- **`RenderCompanions::adopt_predecessor`** (render thread, swap block, inside a realtime-policy region). It downcasts the - predecessor through `as_any_mut`, calls host-core's `pair_carried_meters` and `pair_carried_observations` on the two - companions' vectors (the browser keeps both in host-core's order, so the record's indices address them directly), and notes each - paired reader in `carried: Box<[CarriedReader]>`, sized at preparation to one entry per reader and charged in - `RenderCompanions::retained_bytes`. Another attachment type, or an empty record, pairs and notes nothing. -- **Table hand-over.** `adopt_shape(epoch, carried)` runs in the same block and follows the pairs with the state the tables hold: - `meter_pending` entries swap per carried meter, and `PlanShape::carry_tap_state` moves the armed bits and arm samples of - each carried observation reader between the two shapes' slots. The slot of reader `r` is `PlanShape::reader_slots[r]`, the - inverse of `EffectInstance::reader`, built in `build_plan_shape` and charged in `arm_table_bytes`. A tap whose arm sample in - the successor shape is not zero was named by a command admitted after the commit and keeps that state; the others take the - predecessor's. -- **Delivery.** A swap whose record carries at least one meter no longer restarts the meter delivery: `meter_generation`, - the loss count, the master windows and the published header are the stream's and stay. A swap that carries no meter (the meter - window length changed, #1327a gate 2 test A) restarts as before. - -**Departures from the text.** - -- The spec's per-plan bookkeeping inside the companions (D3: generation, master windows, `observation_armed` in the attachment) - was not moved: #1381a/#1290a left them in `ReadyOwnership` and `PlanShape` (swapped at the adoption block), and the delivery - fields are continuous without a copy. The hook plus the table hand-over give the same result. -- `pair_carried_observations` is called (the spec said the browser would not): the browser's `effect_observations` is - already in host-core's order. The slot map is for the per-slot tap state only. -- Gate 3's published frame with a changed strip count (`track_count` 3, the shifted strip's word) needs the frame to grow with - the plan: that is #1290d. Here a strip-count change still publishes nothing and counts every window it leaves; the - strip-count-keeping swap and the shifted observation slot are tested. -- The spectrum pairing is not in the hook (batch 10 amendment). -- `meter_windows_a_strip_count_change_leaves_unpublished_are_counted`: the windows waiting at the swap are now counted by the poll that finds them - unpublishable, not at the swap; the count is not zeroed. - -**Gates -> tests** (`hosts/host-web/src/tests.rs`, `structural_apply`): - -| Gate | Test | -|---|---| -| 2 (meters carry, no epoch change, no loss) and the misshaped frame | `meter_delivery_carries_through_a_swap_and_never_publishes_a_misshaped_frame` (was `meter_delivery_restarts_...`) | -| 2/3 (acknowledged subscription, shifted slot) | `an_armed_tap_and_its_windows_follow_the_effect_through_a_swap` | -| loss counted | `meter_windows_a_strip_count_change_leaves_unpublished_are_counted` | -| allocation-free swap block | `tests/structural_apply_realtime.rs` (render-locked count 0 across the adoption block, hook included) | - -With the hook returning early, all three unit tests turn red (checked). - -## Fold (batch 11 review) - -- `PlanShape::carry_tap_state` no longer reads arm sample `0` as "no command". A command admitted against the - successor at sample zero (before the first block) was overwritten by the predecessor's armed bit. The shape now - holds `observation_commanded`, one bit per tap per effect slot, set where `push_observe` admits a tap command and - cleared once the hand-over has read it; the table is charged in `arm_table_bytes`. Gate: - `a_tap_command_admitted_at_sample_zero_survives_the_hand_over` (red with the sentinel, checked). -- The `adopt_shape` doc over-claimed that the poll finds the carried pending windows where it left them. That holds - only while the meter handles are unchanged: a swap that removes one strip and adds another shifts handles, the - carried pending snapshots fail the poll's handle check, and they are dropped and added to the meter loss count. diff --git a/.github/ISSUE_SPECS/1386-diff-a-replacement-document-against-the-committed-model-and-export-replace-from.md b/.github/ISSUE_SPECS/1386-diff-a-replacement-document-against-the-committed-model-and-export-replace-from.md index 36d6d9f2a..95f8d3b15 100644 --- a/.github/ISSUE_SPECS/1386-diff-a-replacement-document-against-the-committed-model-and-export-replace-from.md +++ b/.github/ISSUE_SPECS/1386-diff-a-replacement-document-against-the-committed-model-and-export-replace-from.md @@ -114,3 +114,493 @@ transaction apply. The SDK's `replaceSession` (*Apply session transactions from - *Replace the running browser session in the Rust host* (#1290). - *Export transaction apply and anchored seek from the browser engine module* (#1293). + +## Amendment (controls coordinator, root rulings, 2026-10-15) + +- **Re-scoped under decision 17 C1** (root ruling B15-R8, 2026-10-15, recorded in #1293). The diff and + `apply_document` go in the core, `crates/control-plane/src/`, not in `hosts/host-web`. A document + replace is a protocol frame through `miso_engine_v1_submit_command` on the C ABI and + `miso_engine_web_v1_control_command` on the browser (#1562's generic entry). Its staging is + `miso_engine_web_v1_control_request_ptr`, bounded by the boot's `maximum_control_frame_bytes`. Its + outcome is the response frame (revision, `edit_path`, diagnostics). +- **Stale references.** `miso_engine_web_v1_edit_ptr` and `miso_engine_web_v1_edit_outcome_ptr` + are not built: #1293 dropped D1's staging (B15-R8) and D3's outcome record (decision 17). D3's export + `miso_engine_web_v1_replace` and D4's mirrors go with them. D2's "#1290's `apply`" is the control + plane's transaction apply. +- The new message ID, its fields, both hosts' gates and the new authorized paths are re-briefed + before implementation. That re-brief is root's. + +## Re-brief (controls coordinator, batch 15 docs, 2026-10-15; root reviews at the push) + +Code anchors verified on `c43b65870` (the staging anchors on `bed928efb`). This section is the +brief. Where it and the filed text above differ, this section governs. + +**Superseded filed text (kept above, not deleted).** +- Title and Product outcome, "export replace from the browser engine module": superseded; the + replace is a protocol command on both hosts (decision 17 C1, root ruling B15-R8). +- Context bullets 4 and 5 (#1290's `apply` and D8, #1293's staging, outcome record and mirrors): + superseded; both hosts already run the core's transaction path. +- D1's path `hosts/host-web/src/document_diff.rs`: superseded by `crates/control-plane/src/document.rs`. + D1's edit order stays, with the additions below. +- D2 (`AudioWorkletEngineHost::apply_document`): superseded by D2 below. +- D3 (`miso_engine_web_v1_replace`) and D4 (export mirrors): superseded; no replace export is + added. Each host gains one document transport export only (D7). +- Deliverables, Authorized paths, Objective gates 3, 5 and 6, Test value and Dependencies: + superseded by the sections below. Gates 1, 2 and 4 are restated below. + +### Product outcome + +An app replaces the whole running session with a new session document while it plays. The engine +diffs the document against the committed model and applies the result as one transaction. The +outcome is the same as the equivalent transaction: the same revision, edit path, events and +rendered bits. The C ABI and the browser serve it through the same core function, as the protocol +command `SESSION_REPLACE` (`0x0013`). The SDK's `replaceSession` (#1296) sends this frame and +nothing else, so there is still one edit path (D15-11). A replace accepts every document the boot +accepts, up to the boot's `maximum_document_bytes` (1 MiB on the browser), on both hosts. The +document does not go in the frame: the frame names a document the host passes beside it (D7), and +the control plane validates and parses it off the render thread. + +### Context (verified code anchors) + +- **Message IDs.** `MessageId` is `crates/protocol/src/wire.rs:96-134`; the last command is + `ResponseGet = 0x000e` (`:120`). `from_raw` is `:165-193`. `RETIRED_MESSAGE_IDS` is `:142`: + `0x0004`, `0x0005`, `0x0006` and event `0x8002`. The registry is + `docs/CONTROL_PROTOCOL_REGISTRY.md` (retired IDs `:11-15`, the command table `:31-45`). +- **IDs that open specs already claim.** `0x000f` is `OBSERVATION_READ` (#1586 D1, open). + `0x0010` is `SPECTRUM_CONFIGURE` (#1594 D1, open). `0x0011` and `0x0012` are `SPECTRUM_CAPTURE` + and `SPECTRUM_STREAM` (#1595 D1, open). Event `0x8040` is `RESPONSE_READY` (#763). Capability + bits 14, 15 and 16 are the response, observation and spectrum slices'. +- **Capabilities.** `CapabilityFlags` is `crates/protocol/src/message_wire.rs:52-61`. + `check_capabilities_invariants` is `:4095-4131`. `allocated_id` is `:4134-4143` and covers + commands `1..=14`. `capability_registry` is `crates/protocol/src/controller.rs:3614`; its command + array is `[0_u16; 12]` and its loop is `for id in 1_u16..=14` (`:3620`). +- **The transaction path.** `ControlCommand` is `controller.rs:934-966`. The controller's + structural preparation is `plan_structural_command` (`:1880`); it accepts only + `SessionTransactionApply` (`:1889`), and it runs the replay preflight, the exact-revision check, + the decode and the edit-count check before it prepares. `prepare_command_frame` is `:1685` and + `prepare_command_frame_in_place` is `:1717`. `ProtocolControllerConfig` is `:60-70`; the control + plane builds it in `crates/control-plane/src/compile.rs:1153-1166`. +- **The core entry both hosts call.** `SessionState::command` + (`crates/control-plane/src/control.rs:1510`) runs the controller's preparation, then the live + commit (`commit_live`) or the rebuild. A rate or quantum change is refused there before anything + is prepared, with `CommandError::CompileRejected` and `control.apply.reprepare_required` + (`:1559-1577`, the constant `:859`). The C ABI calls it from `miso_engine_v1_submit_command` + (`crates/capi/src/ffi.rs:681`). The browser calls it from `miso_engine_web_v1_control_command` + (`hosts/host-web/src/ffi.rs:3366`), with the frame staged at + `miso_engine_web_v1_control_request_ptr` (`:3347`). +- **Frame bounds.** The control plane's codec takes `max_frame_bytes` and `max_string_bytes` from + the boot's `maximum_control_frame_bytes` (`compile.rs:1085-1091`). The replay cache stores each + new request's whole bytes (`controller.rs:336-366`). The browser's frame bound is 16 KiB + (`CONTROL_FRAME_BYTES`, `hosts/host-web/src/lib.rs:5087`); its document bound is 1 MiB + (`MAXIMUM_DOCUMENT_BYTES`, `:74`). The C ABI's are caller-chosen (`CompileLimits`). So a + document cannot go in a frame: 16 KiB would cap the browser's replace far below its boot. +- **The browser's boot document staging** (anchors verified on `bed928efb`). It is the + module's thread-local `BOOT_STAGING` (`hosts/host-web/src/ffi.rs:566`), a `BootStaging` + (`:105-115`) whose `document: ManuallyDrop>` is freed only by assignment, never by a + thread-local destructor (#1333 D5). `miso_engine_web_v1_document_ptr(len)` (`:2957-2992`) + refuses a live instance with `RESULT_REFUSED_LIFECYCLE` (`:2964-2969`), refuses + `len > MAXIMUM_DOCUMENT_BYTES` (`:2971`), then `try_reserve_exact`s a `Vec` of exactly `len` + bytes, assigns it over the old one and returns its address (`:2976-2990`). `boot_staged` + (`:3006-3124`) parses the staged bytes and, on success, clears `document_valid` but keeps the + bytes (`:3051-3055`): after boot the instance retains a copy of the boot document, not charged + in the budget's bridge rows (`hosts/host-web/src/lib.rs:4774-4790`); dispose frees it + (`ffi.rs:526-554`, `reset_after_dispose`). The page hands the document over in the worklet's + message handler (`port.onmessage`, `hosts/host-web/web/miso-engine-v1-audio-worklet.js:287`): + boot writes it at `document_ptr` and calls `boot` (`:360-369`), and every control frame goes + through `miso.control.v1` (`CONTROL_FIELDS = ["tag", "requestId", "frame"]`, `:93`; + `receiveControl` `:1699`; `readControlCommand` `:1727-1735`). `process()` (`:1911`) never + stages or parses. The live-instance refusal of `document_ptr` is pinned by + `sdk/test/boot-evals.mjs:325-350`. The frame path is `control_request_mut` (`lib.rs:2305`, + bounded by `CONTROL_FRAME_BYTES`), `control_command` (`lib.rs:2345`) and `run_command` + (`lib.rs:1604`), which calls `SessionState::command` (`:1615`). +- **The C ABI's boot document.** It has no staging. `miso_engine_v1_compile_session` + (`crates/capi/src/ffi.rs:311`) takes `document: *const u8, document_bytes: u64`, refuses + `document_bytes > limits.maximum_document_bytes` (`:347`) and borrows the bytes for the call + only (`borrowed_bytes`, `:361`). `CompileLimits::maximum_document_bytes` is + `crates/capi/src/abi.rs:122`. `miso_engine_v1_submit_command` (`ffi.rs:681`) borrows only the + frame. The bound reaches the control plane as `ControlLimits::maximum_document_bytes` + (`crates/control-plane/src/adapter.rs:35`, set at `crates/capi/src/runtime/mod.rs:187` and + `hosts/host-web/src/lib.rs:5057`), but the control plane does not read it today. +- **Parse.** `host_core::parse_host_session` (`crates/host-core/src/prepare.rs:1145`) is what the + boot uses (`compile.rs:1054`). +- **The model.** `SessionModel` is `crates/session/src/model.rs:89-127`. Every keyed list is sorted + by stable ID in the compiled model (`crates/session/src/compile.rs:140-164`). + `control_smoothing` (`model.rs:116-120`) is `None` for `{ "kind": "default" }`. `SessionEdit` is + `crates/protocol/src/model.rs:154`. No edit sets `control_smoothing` whole: `SET_CONTROL` of the + session rows `fader_ms`, `mute_ms` and `pan_ms` writes the explicit arm, and no transaction + writes the default arm back (registry `:139`). +- **Classification.** `classify_live_delta` (`crates/host-core/src/live_delta.rs:344`) compares + models (steps `:259-280`), so a whole-entity upsert that changes only values classifies as live. +- **Response.** `TransactionApplied` (`message_wire.rs:198-201`) is `applied_operations` and + `edit_path`. The response header carries the revision (`wire.rs:330-344`). +- **Pins.** `COMPLETE_SCHEMA_HASH` is `crates/conformance/src/protocol_corpus.rs:928`, with its + re-pin history in the doc comment above it. Its other spellings are + `scripts/check-protocol-wasm-parity.sh:172-173`, `fuzz/corpus/complete-schema-manifest.md:10` + and `docs/CONTROL_PROTOCOL_CONFORMANCE.md:3`. The protocol allocation audit lists one frame per + command (`tools/audit/src/protocol.rs:142-200`). +- **SDK.** The SDK encodes its frames by hand (`sdk/src/core/control-frame.ts:32` for `0x0003`). + No generated SDK table lists message IDs (`sdk/src/generated/`: `abi.ts`, `catalog.ts`, + `control-builders.ts`, `controls.ts`, `provenance.ts`, `session-keys.ts`). +- **Parity test.** `hosts/host-web/tests/control_parity.rs` boots the same session on the C ABI + (`CAbi`, `:899`) and the browser (`Rig`, `hosts/host-web/tests/common/mod.rs:349`) and sends the + same frames to both. + +### Decisions frozen for this slice + +- **D1. The message.** `MessageId::SessionReplace = 0x0013` (`SESSION_REPLACE`), command and + response. Why `0x0013`: it is the lowest command ID that is not allocated, not retired and not + claimed by an open spec. `0x0001`-`0x0003` and `0x0007`-`0x000e` are allocated; `0x0004`-`0x0006` + are retired and never reallocated; `0x000f`-`0x0012` are claimed by #1586, #1594 and #1595 + (open). A search of the code, the docs and `.github/ISSUE_SPECS/` finds no other use of `0x0013` + as a message ID. If this slice lands before those three, `0x000f`-`0x0012` stay unallocated and + decode as `UNSUPPORTED_MESSAGE` until they land. + - Command: `1:document-bytes U32 R`, the exact length of the document the host passes beside + the frame (D7), and `2:document-sha256 BYTES R`, exactly 32 bytes, the SHA-256 of that + document. The frame carries no document bytes. Exact revision only + (`requires_exact_revision`). `document-bytes` 0 or a digest that is not 32 bytes is + `INVALID_FIELD`. Why the digest: the replay cache compares frame bytes, and the document is + not in the frame. With the digest, the same request ID with another document is other frame + bytes, so it is `REQUEST_ID_REUSE`, never a cached reply for a document it did not apply. + - Success response: the `0003` payload, `1:applied-operations U32 R`, `2:edit-path U8 R` + (`TransactionApplied`). `edit_path` is live, model_only or rebuild. It is never monitor: a + document does not carry solo, and the monitor overlay is kept. + - Capability: no new flag bit. `SESSION_REPLACE` is listed only with the session family + (`0003` and `8001`, bit 3). The decode invariant gains: `has_command(0x0013)` implies the + session family. `allocated_id` and the registry loop gain `0x0013`, and the registry's command + array grows to fit. +- **D2. The diff, in the core.** `crates/control-plane/src/document.rs` (new): + `pub fn diff(committed: &SessionModel, next: &SessionModel) -> Vec`. It emits the + filed D1 order (session-level setters; removals in the order routes, automation, VCAs, tracks, + submixes, outputs, sources; `SetConsole`; upserts in the order sources, outputs, submixes, + tracks, VCAs, routes, automation), then the `control_smoothing` step below. Rules: + - An entity differs when its bits differ, not when `PartialEq` says so: a zero's sign counts, + as in the classifier's step 1. One edit per differing entity, never a finer edit. + - `revision` and `schema_version` are not diffed. The header's exact revision is the guard, and + a document of another schema version is refused at parse. + - `control_smoothing`: when `next` holds an explicit arm whose lengths differ from the + committed lengths, the diff emits one `SetControl` per changed session row (`fader_ms`, + `mute_ms`, `pan_ms`). When `next` holds the default arm and the committed model holds an + explicit arm, no edit can express it. Decided (root review, 2026-10-15): a replace-only + preparation step applies the document's `control_smoothing` to the candidate (an absent key + is the session default, `None`) after the edits and before the final validation, through one + new `SessionStore` preparation entry that only the replace path calls. No wire edit is + added. The change is model-only (no prepared plan reads the key). +- **D3. The path.** The controller does the frame work; the control plane owns parse and diff. + - The core entry. `SessionState` gains `command_with_document(&mut self, request: &[u8], + document: &[u8], output_capacity: u64)`; `command(request, output_capacity)` is + `command_with_document(request, &[], output_capacity)`. The document is borrowed for the call + only; nothing in the control plane or the protocol crate keeps it. The controller's frame + preparation (`prepare_command_frame`, `prepare_command_frame_in_place`) threads the same + borrowed slice to `plan_structural_command`. `SessionState` keeps `maximum_document_bytes` + from its `ControlLimits` (today unread) as the replace's bound. + - `ProtocolControllerConfig` gains `document_replace: Option`, a plain `fn` + pointer that takes the committed `SessionModel`, the borrowed document, the frame's + `document-bytes` and `document-sha256`, and the document byte bound, and returns the edits + plus the replace's `control_smoothing` value, or a typed refusal (status and diagnostic). The + control plane sets it to its own function in `compile_children`. The protocol crate holds no + diff, parse or hash code. `0x0013` is advertised only when it is set and session + transactions are on. + - `document_replace` checks, in this order: the document length equals `document-bytes`; the + length is at most `maximum_document_bytes`; the SHA-256 of the document equals + `document-sha256` (`sha2`, the workspace pin, added to `crates/control-plane/Cargo.toml`); + then it parses with `parse_host_session` and diffs. All of it is control work, never render. + - `plan_structural_command` accepts `SessionReplace` beside `SessionTransactionApply`. It runs + the same replay preflight, exact-revision check and output reservation first, then calls + `document_replace`, then the same edit-count check and the same preparation. A replayed + request returns its cached reply before the document is read, so a retry needs only the + same frame bytes. Every refusal goes through the controller's non-OK response, so the replay + cache holds it like any other reply. + - An empty diff (and no smoothing change) commits nothing. The reply is OK with + `applied_operations` 0, `edit_path` model_only and the unchanged revision. No + `SESSION_COMMITTED` is queued and no candidate is published. + - Otherwise the prepared token goes to `SessionState::command` exactly as a transaction's does: + the live commit or the rebuild, the rate and quantum refusal, the plan admission and the + `SESSION_COMMITTED` event (operation count = `applied_operations`). The reply's message ID is + `0x0013`. + - Both hosts call `SessionState::command_with_document`. Neither host decodes the frame or + runs replace code; each host adds only the document transport of D7. +- **D4. Refusals, all before commit.** Each one leaves the revision, the committed document and + the rendered bits unchanged, and queues no event: + - no document beside the frame (the plain `submit_command` or a browser command with nothing + staged), or a document whose length is not `document-bytes`: `INVALID_FIELD`, + `session.replace.document_length`; + - a document longer than the boot's `maximum_document_bytes`: `LIMIT_EXCEEDED`, + `session.replace.document_too_large` (the browser's D7 staging refuses this length first); + - a document whose SHA-256 is not `document-sha256`: `INVALID_FIELD`, + `session.replace.document_digest`; + - a document that does not parse or validate: `VALIDATION_FAILED` with the parser's + diagnostics; + - more edits than `maximum_transaction_edits`: `LIMIT_EXCEEDED`, as for a transaction; + - a stale revision: `REVISION_CONFLICT`, as for a transaction; + - a rate or quantum change: `CompileRejected`, `control.apply.reprepare_required`, as for a + transaction (`control.rs:1559-1577`); + - a structural replace while a candidate is pending: whatever a transaction gets at that time + (today `BACKPRESSURE`; after #1310, supersession). +- **D5. The acked-batch question.** The OK reply is written only after the commit. A refusal is + returned before any state changes. No queue is added. An ack can never precede a drop. +- **D6. Docs.** `docs/CONTROL_PROTOCOL_REGISTRY.md` (the command row, the appended-ID note, the + capability rule), `docs/CONTROL_PROTOCOL_SEMANTICS.md` (the replace rules of D2-D5), + `docs/CONTROL_PROTOCOL_SIZING.md` (the request is the header plus two fixed fields, under + every frame bound; the document's only bound is `maximum_document_bytes`), the C ABI header + prose for D7's export and `docs/C_ABI_V1_QUALIFICATION.md`. +- **D7. The document transport: one core rule, each host's existing document shape.** The + core rule is D3's: the host passes the document as a borrowed `&[u8]` beside the frame, and + the control plane checks, hashes, parses and diffs it on the control side. A host differs only + in how the bytes reach that borrow, as it already does for frames (the browser stages frames, + the C ABI borrows them). Neither host keeps the document after the call. + - **Browser: reuse the boot document staging.** New export + `miso_engine_web_v1_replace_document_ptr(handle: u32, len: u32) -> u32`. It returns the + address of the same `BOOT_STAGING.document` buffer the boot uses, sized exactly as + `document_ptr` sizes it (`try_reserve_exact`, assignment over the old `Vec`), and sets + `document_valid`. It is the live twin of `document_ptr`: it returns 0, and stages nothing, + for a wrong handle, a host that is not ready, `len` 0, `len > MAXIMUM_DOCUMENT_BYTES` or a + failed reservation. It does not touch the boot result word. `document_ptr` stays pre-boot + only (its lifecycle refusal is pinned, `sdk/test/boot-evals.mjs:325-350`). + `miso_engine_web_v1_control_command(handle, len)` keeps its signature: it passes + `&BOOT_STAGING.document` when `document_valid` (else `&[]`) to + `SessionState::command_with_document` through `run_command`, and after every command, + whatever its message, it frees the staging (`*document = Vec::new()`) and clears + `document_valid`. So a staged document serves exactly the next command and is never + retained; the first command after boot also frees the boot document copy that boot kept. + The worklet's `miso.control.v1` message accepts an optional `document` field (a + `Uint8Array`, the same checks as `frame`); `readControlCommand` stages it with + `replace_document_ptr` before it stages the frame. This is the message handler's work + (`single` mode) or the control side's; `process()` never stages, hashes or parses. + **ROOT APPROVED (2026-10-15): a new browser export.** Reason: `document_ptr` refuses a live instance and + that refusal is a pinned contract; relaxing it would change the single-live-handle + lifecycle rule. The export moves the ABI layout mirrors (D7 paths). + - **C ABI: borrow the document beside the frame.** The C ABI has no staging a frame can name + (Context). New export `miso_engine_v1_submit_document_command(session: *mut Session, + request: *const u8, request_bytes: u64, document: *const u8, document_bytes: u64, response: + *mut BytesOut) -> u32`, Thread: control. It is `miso_engine_v1_submit_command`'s contract + plus the document borrowed for this call only under `compile_session`'s pointer-plus-length + rule (`borrowed_bytes`, `ffi.rs:361`); `document` must be nonnull when `document_bytes` is + nonzero, else `RESULT_INVALID_ARGUMENT`. It calls `command_with_document` and maps results + exactly as `submit_command` does. `submit_command` is unchanged; a `SESSION_REPLACE` frame + sent through it is the D4 `session.replace.document_length` refusal. The C ABI retains + nothing. A session-owned copy staging (a stage export plus a buffer) was rejected: it would + retain up to `maximum_document_bytes` per session and add a stale-staging state. + **ROOT APPROVED (2026-10-15): a new C export.** + +### Deliverables + +1. D1 in `crates/protocol` (wire, schema, codec, typed frame, controller, capabilities) and the + corpus row and re-pin in `crates/conformance`. +2. D2 in `crates/control-plane/src/document.rs`; D3 in `crates/protocol/src/controller.rs`, + `crates/protocol/src/model.rs` (the replace preparation entry only) and + `crates/control-plane/src/{compile.rs,control.rs,lib.rs}`. +3. D7: the browser's export, staging consumption and worklet message field; the C ABI's export. +4. D6 and the gates below. + +### Authorized paths + +- `crates/protocol/src/{wire.rs,message_wire.rs,message_wire/tests.rs,schema.rs,typed_frame.rs,controller.rs,controller/tests.rs,model.rs,lib.rs}`. +- `crates/conformance/src/protocol_corpus.rs`, `crates/conformance/tests/conformance_corpus.rs`, + `fuzz/corpus/complete-schema-manifest.md`, `scripts/check-protocol-wasm-parity.sh` (the + self-test hash strings only). +- `crates/control-plane/src/{document.rs,compile.rs,control.rs,lib.rs}`, + `crates/control-plane/Cargo.toml` (`sha2` only), `Cargo.lock`, + `crates/control-plane/tests/session_replace.rs` (new). +- `crates/capi/src/ffi.rs` (D7's export only), `crates/capi/tests/session_replace.rs` (new), + `crates/capi/include/miso_engine_v1.h` (D7's declaration and prose), + `crates/capi/tests/c/abi_smoke.c` (the declaration's link check, if the smoke lists every + export), `scripts/check-capi-abi.sh` (the export list), `tools/audit/src/protocol.rs` (one + frame). +- `hosts/host-web/src/{ffi.rs,lib.rs}` (D7 only), `hosts/host-web/web/miso-engine-v1-audio-worklet.js` + (the optional `document` field only), `hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts` + (the field's type, if the message is typed there), + `hosts/host-web/web/miso-engine-v1-audio-worklet-artifact.sha256` (re-pinned by + `scripts/build-web-audioworklet.sh`), `scripts/check-web-audioworklet.sh` (the export list), + `tools/parameter-metadata/src/abi_layout.rs`, `scripts/check-abi-layout-v1.py`, + `scripts/fixtures/abi-layout-v1-self-test.json` and the generated mirrors + `sdk/assets/miso-engine-v1-abi-layout.json` and `sdk/src/generated/abi.ts` (the export name + only, regenerated, never hand-edited), `scripts/test-web-audioworklet.mjs` (the fake module's + export and the message-field cases). +- `hosts/host-web/tests/control_parity.rs`. +- `docs/CONTROL_PROTOCOL_{REGISTRY,SEMANTICS,SIZING,CONFORMANCE}.md`, + `docs/C_ABI_V1_QUALIFICATION.md`. + +### Non-goals + +- No hand-written SDK change. `replaceSession`, its frame encoder and its use of the worklet's + `document` field are #1296's (`sdk/src/core/control-frame.ts`, `sdk/src/core/boundary.ts`). + No generated SDK table lists message IDs, so none moves; only the generated ABI export list + gains D7's browser export. +- No browser replace export and no change to the browser's boot path or `document_ptr`. +- No change to classification, the live commit or the rebuild. +- No change to the browser's frame bound (`CONTROL_FRAME_BYTES` stays 16 KiB) and no chunked + frame: D7 carries the document. +- No supersession of a pending candidate (#1310). + +### Objective gates + +Commands: +- `cargo test --locked -p protocol --all-features`, `cargo test --locked -p conformance` +- `cargo test --locked -p control-plane --features control-plane/test-support` +- `cargo test --locked -p capi` +- `cargo test --locked -p host-web --features host-web/test-support` +- `cargo build --locked --release -p audit && bash scripts/run-protocol-allocation-audit.sh target/release/audit` +- `bash scripts/check-protocol-wasm-parity.sh`, `bash scripts/check-protocol-control-policy.sh`, + `bash scripts/check-conformance-boundaries.sh`, `bash scripts/check-capi-abi.sh`, + `bash scripts/check-workspace-policy.sh` +- The web artifact gates with the arguments `.github/workflows/qualification.yml` gives them: + `scripts/build-web-audioworklet.sh` (`:132`), `scripts/check-web-audioworklet.sh` (`:342`), + `scripts/test-web-audioworklet.mjs` (`:356`); then `bash scripts/check-sdk-generated.sh`, + `python3 scripts/check-abi-layout-v1.py` and + `node scripts/check-web-boot-budget.mjs .wasm` +- `CARGO_TARGET_DIR=target/ci/wasm-simd RUSTFLAGS='-C target-feature=+simd128' cargo check --locked --target wasm32-unknown-unknown -p target-smoke -p protocol` +- `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, + `cargo fmt --all -- --check` + +1. **Diff coverage (control-plane test).** For each keyed kind (source, track, submix, output, + route, automation, VCA), one case adds, one removes and one changes an entity; one case changes + the console; one the session ID; one a fader from `0.0` to `-0.0`; one each way between the + default and an explicit `control_smoothing`. For each, `diff` returns exactly one edit per + differing entity (or the smoothing rows), and a replace commits a canonical snapshot equal to + that of a session booted from the document, except the revision. +2. **Dependent removal.** A document that removes a source and the track that reads it commits. +3. **Same as a transaction (C ABI, `crates/capi/tests/session_replace.rs`).** Through + `miso_engine_v1_submit_document_command`: a replace with the boot document plus a muted track returns + OK, path rebuild, revision +1, and renders the same blocks as the equivalent + `SESSION_TRANSACTION_APPLY` on a twin session. A document that changes one fader returns path + live. An identical document returns OK, model_only, the unchanged revision; no + `SESSION_COMMITTED` is dequeued and no candidate is published. +4. **Refusals before commit (C ABI).** One case for each refusal of D4 except the pending + candidate. Each returns its typed status (or result) and diagnostic. After each, the revision + and the `SESSION_SNAPSHOT_GET` bytes are unchanged, no event is dequeued, and the next 8 blocks + are bit-identical to a twin that got no command. +5. **Replay.** The same request ID with the same frame bytes returns the cached reply and commits + once, also when the retry passes no document. The same request ID with another document (so + another `document-sha256`) is `REQUEST_ID_REUSE` and commits nothing. +6. **Both hosts (parity row).** A new test in `hosts/host-web/tests/control_parity.rs` sends the + same `SESSION_REPLACE` frames and documents to the C ABI (D7's export) and the browser (D7's + staging, then `control_command`): one rebuild, one live, one identical document and one + refusal. The two hosts return the same reply bytes (request ID aside), the same revision, the + same `SESSION_SNAPSHOT_GET` bytes and bit-identical blocks, at 44.1 and 48 kHz. +7. **Wire and pins.** Codec round trips of the command and the reply. Decode refuses a missing + field 1 or 2, a digest that is not 32 bytes, and a capability record that lists `0x0013` + without the session family. The corpus gains `command.session_replace` and + `response.session_replace`. `COMPLETE_SCHEMA_HASH` is re-pinned once, with a history line + naming #1386, and its other spellings (parity script, manifest, conformance doc) move with + it. The allocation audit includes the new frame. The web artifact digest and the ABI layout + mirrors move once, regenerated by their own tools. +8. **A document larger than the frame (parity row, root review a).** A new test in + `hosts/host-web/tests/control_parity.rs` boots a small session on both hosts and replaces it + with a document of more than 64 KiB (four times the browser's 16 KiB frame bound; a valid + session whose size comes from more tracks, not padding), then with a second large document + that changes one fader (path live). Both hosts return OK, the same reply bytes (request ID + aside), revision, snapshot bytes and bit-identical blocks. A browser case at exactly + `MAXIMUM_DOCUMENT_BYTES` stages and commits, and one byte more gets 0 from + `replace_document_ptr`. The gate records, per host, from a measurement in the test: + - bytes the host retains for the document after the reply: browser, the capacity of + `BOOT_STAGING.document` (expected 0, D7 frees it, including the boot copy); C ABI, 0 + (borrowed, nothing kept); + - bytes the host holds for the document during the call: browser, the staged length; C ABI, + none of its own (the caller's buffer); + - on the browser, the wasm linear memory growth across one replace of the 1 MiB boot ceiling + document (`scripts/check-web-boot-budget.mjs` method), against the boot's projection + `document_bytes * PARSE_TRANSIENT_MULTIPLIER` (`hosts/host-web/src/lib.rs:76-98`). + Any retained byte row these numbers move (`exact_retained_bytes`, the boot budget script's + pin) is re-pinned from the measurement, with the measured figure in its history comment. A + replace growth above the boot projection is reported to root, not pinned. + +### Test value + +- Gate 1: a diff that misses a kind, emits an unchanged entity or compares with `PartialEq` + (the sign of zero) turns red; so does a replace that cannot return to the default smoothing. +- Gate 2: a diff whose removal order makes an intermediate check refuse turns red. +- Gate 3: a replace that takes a second edit path, reboots, or commits on an identical document + turns red. +- Gate 4: a refusal that moves the model, the revision or a rendered bit before it fails turns + red. +- Gate 5: a replace path that skips the replay preflight commits twice; a frame without the + digest answers a cached reply for another document. +- Gate 6: a host that runs its own replace code, or answers other bytes, turns red. +- Gate 7: a frame shape or capability rule that differs from the registry turns red. +- Gate 8: a replace bounded by the frame instead of `maximum_document_bytes`, a browser staging + that keeps the document after the command, or a host whose large-document reply differs, + turns red. + +### Dependencies (checked with `gh issue view` on 2026-10-15) + +- Closed and needed: #1309 (the control plane both hosts call), #1562 (the browser's generic + command entry), #1381 (the browser runs `SessionState`). +- Not dependencies any more: #1290 (open; its `apply` is now the core's transaction path) and + #1293 (open; B15-R8 dropped the staging this slice used). +- Ordering only: #1586, #1594 and #1595 (open) hold `0x000f`-`0x0012`. #1310 (open, batch 15a) + changes what a structural replace gets while a candidate is pending; this slice inherits it. +- Downstream: #1296 (open) sends this frame from `replaceSession`. + +### Root review (2026-10-15) + +Root approved D1 (`0x0013` `SESSION_REPLACE`), the diff in the control plane (D2), the +`TransactionApplied` reply and the gates, and ruled the two open points. The sections above +already carry both rulings. +- **(a) Document size.** A replace accepts every document the boot accepts, up to + `maximum_document_bytes`, on both hosts; it is not capped by the control frame. The frame names + the document and does not carry it; the bytes are validated and parsed off the render thread, + by one rule on both hosts. Applied as D1's fields, D3's core entry, D4's refusals, D7 and + gate 8. D7 adds one export per host; root approved both (2026-10-15). +- **(b) `control_smoothing`.** Approved: a replace-only preparation step applies the document's + `control_smoothing` (absent = the session default), with no new wire edit. Applied in D2. +- **Second review (root, 2026-10-15).** Both exports are approved + (`miso_engine_web_v1_replace_document_ptr`, `miso_engine_v1_submit_document_command`), with the + frame naming the document by length and SHA-256, and `SessionState::command_with_document` with a + borrowed document. Freeing the browser's uncharged boot-document copy at the first command is + kept: gate 8 records the browser's retained bytes before and after that first command. Gate 8 + adds one case: on a page without cross-origin isolation, the 1 MiB replace runs in the worklet's + message handler, so it is recorded as a counted blocking rebuild (the documented exception), and + `process()` still allocates and frees nothing (the render-locked count stays 0). + +## Amendment (controls coordinator, batch 15a fold, root ruling 2026-10-16): the split + +**#1386 is now part (a), core and C ABI.** The browser transport moves to the placeholder +*Export the browser session replace transport* (#1697, `.github/ISSUE_SPECS/1697-export-the-browser-session-replace-transport.md`). + +- **(a) stays here.** D1 (`SESSION_REPLACE` `0x0013` in `crates/protocol`), D2 (the diff in + `crates/control-plane/src/document.rs`), D3 (`SessionState::command_with_document`, the one core + path both hosts call), D4, D5, D6 and D7's C ABI half + (`miso_engine_v1_submit_document_command`). The browser keeps calling `SessionState::command`, + which is `command_with_document(request, &[], ...)`, so a `SESSION_REPLACE` frame sent through + `miso_engine_web_v1_control_command` today gets D4's `session.replace.document_length` refusal + through the same core path. That is the both-host core path of (a). +- **#1697 takes** D7's browser half (`miso_engine_web_v1_replace_document_ptr`, the staging + consumption in `control_command`, the worklet message's `document` field), both `.d.ts` twins, + the ABI layout mirrors of that export, gate 6 and gate 8's browser half. +- **Gates of (a).** Gates 1-5 and 7 as filed. Gate 6 becomes a C ABI and browser parity row for + the refusal only: the same `SESSION_REPLACE` frame with no document gives the same reply bytes + (request ID aside) on both hosts, and changes nothing. Gate 8 keeps its C ABI half: a document of + more than 64 KiB and the large live replace through `miso_engine_v1_submit_document_command`, + with 0 retained bytes after the reply. Gate 7's "the ABI layout mirrors move" is #1697's. +- **Authorized paths of (a).** As filed, without `hosts/host-web/src/{ffi.rs,lib.rs}`, the + worklet, the `.d.ts`, the artifact digest, `scripts/check-web-audioworklet.sh`, the ABI layout + tools and mirrors and `scripts/test-web-audioworklet.mjs` (all #1697's). Added: see the feature + bit below. `hosts/host-web/tests/control_parity.rs` stays, for gate 6's refusal row. + +**Corrections to the re-brief (it predates #1310, which ships in batch 15a).** + +- **D4, last bullet.** "Today `BACKPRESSURE`" is void. Since #1310, a structural replace while a + candidate is pending gets what a structural transaction gets: it supersedes the unadopted + candidate by compare and swap. Gate 4 adds that case: a replace while a candidate is pending + commits once and its plan supersedes the candidate, as #1310's transaction case does. +- **Non-goal "No supersession of a pending candidate (#1310)"** (`:433`) is void. The replace + inherits #1310's supersession through the shared path and adds no supersession code of its own. +- **Non-goal "No browser replace export"** (`:429`) contradicted D7 and its root approval. It now + reads: no browser export in (a); the browser export is #1697's. +- **The C export needs a feature bit** (#1317 D2: each added symbol gets its own bit, the next + free one when it merges). `miso_engine_v1_submit_document_command` gets + `MISO_ENGINE_V1_FEATURE_SESSION_REPLACE`, and `FEATURE_MASK` grows to include it. Added to + Authorized paths: `crates/capi/src/abi.rs` (the bit and the mask, and its unit-test pins at + `abi.rs:548-553`), `crates/capi/src/ffi.rs` (also `version_and_capabilities_are_exact`, + `:1558`), `crates/capi/include/miso_engine_v1.h` (the bit and `MISO_ENGINE_V1_FEATURE_MASK`, + `:265`) and `crates/capi/tests/c/abi_smoke.c` (the mask pin `:23` and a bit check, no longer + conditional). The mask value is the one at merge (#1316 may take a bit first). +- **D3, the identical document.** The re-brief answers `model_only` with no commit. That conflicts + with the wire: `EditPath::ModelOnly` says "the transaction committed" + (`crates/protocol/src/message_wire.rs:161-165`), and the registry's `model_only` is a commit + that render need not act on. Proposed: an identical document is a refusal-free no-op with a + dedicated path, `EditPath::Unchanged = 5` (registry row, decode, corpus, schema re-pin in gate + 7): OK, `applied_operations` 0, the unchanged revision, no `SESSION_COMMITTED`, no candidate. + The alternative is to commit it (`model_only`, revision + 1, one `SESSION_COMMITTED` with 0 + operations). Gate 3's identical case and gate 6 follow the ruling. + ROOT DECISION (2026-10-16): an identical-document replace replies with a new `EditPath::Unchanged` (5): OK, 0 operations, unchanged revision, no event. Nothing changed, so the revision does not move. The `edit_path` vocabulary (protocol, registry, the vocabulary gates), the SDK decoder and the parity row all include 5. diff --git a/.github/ISSUE_SPECS/1393-crossfade-the-browsers-live-bypass-command-over-the-session-ramp.md b/.github/ISSUE_SPECS/1393-crossfade-the-browsers-live-bypass-command-over-the-session-ramp.md deleted file mode 100644 index be6e0dd41..000000000 --- a/.github/ISSUE_SPECS/1393-crossfade-the-browsers-live-bypass-command-over-the-session-ramp.md +++ /dev/null @@ -1,160 +0,0 @@ -# Crossfade the browser's live bypass command over the session ramp - -Stream E of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-1, D15-13 E5). -Code anchors verified on `main` at `6fb211594`. - -Split from *Crossfade the bypass switch over the session ramp* (#1341), which delivers the -crossfade in render, the plan-swap carry and the C ABI producer. This issue owns the browser -decode, the browser parity gate, and the host-web and SDK docs. - -## Product outcome - -A browser app that toggles an effect's bypass live (`COMMAND_EFFECT_BYPASS`, the SDK's -`bypass(enabled)`) hears the same linear crossfade as a C ABI bypass transaction: over the -session's mute ramp (`control_smoothing.mute_ms`, or the default table), from the next block. A -session whose `mute_ms` is 0 renders exactly today's step. For the same session and the same -toggles, the browser and the C ABI render the same bits. - -## Context - -- **What #1341 delivers first.** The live bypass travels as one two-word latest-target cell, - `bypassed` and `ramp_samples` (#1341 D1, on the cell of *Hold effect parameter, bypass and - EQ-target values in latest-target cells*, #1345). Render crossfades over `ramp_samples` (#1341 - D2-D4); a length of 0 is today's whole-block select. The C ABI classifier fills the length from - `LiveRamps::for_session(next).mute_samples`. The browser decode writes `ramp_samples = 0` (#1341 - D1), so the browser still steps. -- **The browser decode.** `CommandRecord::into_effect_records` - (`hosts/host-web/src/lib.rs:4484-4501`) admits `COMMAND_EFFECT_BYPASS` only with channel 255, - parameter 0, a zero `smoothing_samples` word and zero trailing values, and refuses anything else - as `COMMAND_REASON_MALFORMED`; a value other than 0 or 1 is `COMMAND_REASON_DOMAIN`. It emits - `EffectControlRecord::Bypass(bool)` (`:4500`). -- **The session's lengths in the browser.** *Resolve an absent live ramp to the session default on - the browser and in the SDK* (#1364, D2) has host-web compute `LiveRamps::for_session` from the - session it prepared and keep it with the live controls. It keeps the bypass word's zero rule: a - sentinel or any non-zero word on bypass is `MALFORMED`. #1364 also adds the cross-host parity - test `hosts/host-web/tests/control_smoothing_parity.rs`, with `capi` as a host-web - dev-dependency. -- **Docs that still describe a step.** `COMMAND_EFFECT_BYPASS` (`hosts/host-web/src/lib.rs:836-844`), - the `EffectBypass` kind in `sdk/src/browser/shipped-host.d.ts:202-209`, and the SDK's `bypass` - (`sdk/src/core/live-controls.ts:931-943`). -- **The bypass has no per-edit length.** Decision 15, D15-1 recorded resolution: - the bypass crossfade alone always uses the session mute ramp. *Carry an optional per-edit ramp - length on live session edits* (#1394) gives no length to `SetEffectBypass`. - -## Decisions frozen for this slice - -- **D1. The length.** The browser's bypass admission writes the cell's `ramp_samples` from the - `LiveRamps` #1364 keeps for the running session: `mute_samples`. It is read on the control - thread at admission, never in render. -- **D2. The word stays reserved.** The command's `smoothing_samples` word stays required zero. A - non-zero word, the `SESSION_DEFAULT_RAMP` sentinel included, is `MALFORMED`. The SDK's `bypass` - keeps writing 0 and gains no ramp option. -- **D3. Where admission lives.** If *Admit browser live edits in the Worker through the committed - model* (#1562) has merged when this lands, the bypass command lowers to `SetEffectBypass` and - the shared classifier's #1341 producer already fills the length. This slice then changes no - admission code and delivers D4 and the gates. Otherwise D1 goes in `into_effect_records`. -- **D4. Docs.** `COMMAND_EFFECT_BYPASS`, the shipped-host `EffectBypass` kind and the SDK's - `bypass` say: the toggle crossfades linearly between the wet output and the latency-matched dry - signal over the session's mute ramp; `mute_ms = 0` is a step; the effect's latency never - changes. -- **D5. Acked-batch question.** No queue, admission order or ack changes; the record carries a - different length. A bypass is never acked without effect: its ramp starts at the next block. - -## Deliverables - -1. D1 and D2 in `hosts/host-web/src/lib.rs` (or nothing there under D3). -2. D4 in `hosts/host-web/src/lib.rs`, `sdk/src/browser/shipped-host.d.ts` and - `sdk/src/core/live-controls.ts`. -3. The tests below. - -## Authorized paths - -- `hosts/host-web/src/lib.rs`, `hosts/host-web/src/tests.rs`, - `hosts/host-web/tests/control_smoothing_parity.rs` (stream H owns host-web; root sequences the - merge) -- `sdk/src/browser/shipped-host.d.ts` and `sdk/src/core/live-controls.ts` (docs only), - `sdk/test/live-controls-evals.mjs` - -## Non-goals - -- The render crossfade, the carry and the C ABI producer (#1341). -- A per-edit bypass length (D15-1: none exists). -- Resolving absent lengths for the other rows (#1364). - -## Hazards - -- **Session replacement.** If *Replace the running browser session in the Rust host* (#1290) has - merged, the `LiveRamps` read in D1 must be the replaced session's. Read it from where #1364 keeps - it for the running session; never cache a boot copy. -- **Existing browser tests** that toggle a live bypass and compare the next block with a prepared - or authored control now see a ramp. Where the claim is the switch, author `mute_ms = 0`; - otherwise compare after the ramp. Do not weaken a comparison. - -## Objective gates - -1. **Admission** (`hosts/host-web/src/tests.rs`, new). At 48 kHz with no `control_smoothing`, an - admitted bypass command carries `bypassed` and `ramp_samples = 480`; with `mute_ms = 0` it - carries 0. A word of 1 or of `u32::MAX` is `MALFORMED`. (Under D3 the same claims are made on - the lowered transaction's committed record.) -2. **Parity** (`hosts/host-web/tests/control_smoothing_parity.rs`, extended). One session with two - tracks and a live-controlled compressor insert on one, at 44.1 and 96 kHz: a bypass at a fixed - block, a reversal 64 frames into the ramp, and a later bypass. The browser (native host-web, - bypass commands) and the C ABI (`capi`, `SetEffectBypass` transactions) render bit-identical - output on every block. The same script with `mute_ms = 0` is bit-identical between the hosts - too. -3. **Allocation.** The render blocks of gate 2 allocate nothing on the browser host (the existing - host-web allocation gate passes). -4. **Commands:** - - `cargo test --locked -p host-web --features test-support`, `cargo test --locked -p capi` - - the workspace debug leg (`test-debug-a` in `.github/workflows/qualification.yml`) - - `mkdir -p target/ci/qualification-artifacts target/ci/qualification-named-twin && bash scripts/build-web-audioworklet.sh --named-twin target/ci/qualification-named-twin target/ci/qualification-artifacts` - - `bash scripts/check-web-audioworklet.sh --without-metadata-regeneration target/ci/qualification-artifacts target/ci/qualification-named-twin/miso-engine-v1-audio-worklet.simd128.named.wasm` - - `bash scripts/test-web-audioworklet.sh` - - `bash scripts/check-sdk-generated.sh target/ci/qualification-artifacts`, - `bash scripts/check-sdk-types.sh`, - `bash scripts/check-sdk-headless.sh target/ci/qualification-artifacts` - - `bash scripts/check-workspace-policy.sh`, `bash scripts/check-cross-targets.sh` - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` -5. **Digests.** A checked-in browser digest of a render that toggles a live bypass moves. Each one - is listed with the reason "browser bypass crossfade, decision 15 E5" and re-pinned on its own. - -## Test value - -- Gate 1 turns red if the browser keeps writing 0 (the toggle steps), reads the fader or pan key, - or lets a non-zero word through as a length. -- Gate 2 turns red if the two hosts disagree on the length, on the block the ramp starts, or on - the reversal's anchor. No test compares a bypass across hosts today. - -## Dependencies - -- *Crossfade the bypass switch over the session ramp* (#1341). -- *Resolve an absent live ramp to the session default on the browser and in the SDK* (#1364): the - `LiveRamps` host-web keeps, and the parity test this slice extends. - -**Decision 17 amendment (2026-10-13).** Superseded by #1563. The SDK's `bypass` becomes a `SET_CONTROL` on the effect section's control `0`, which the core crossfades on both hosts (#1341). `COMMAND_EFFECT_BYPASS` is deleted by #1565. Order note: if #1341 lands before #1563, the cross-host parity test (#1560) shows the browser's step against the C ABI's crossfade; root orders #1341 accordingly. - -## Amendment (controls coordinator, batch 9, 2026-10-14) - -Checked against `main` at `7feb27800`, and against #1341a1 and #1341a2 on `codex/controls-batch-9k` -(`3b6e63354`, `4108ed463`). - -- **No work is left. This issue closes with #1341a1** (batch 9), with #1341a2 for the banked case. -- **Why.** #1563B and #1565A deleted the browser's own bypass path: `COMMAND_EFFECT_BYPASS`, - `into_effect_records` and the shipped-host `EffectBypass` kind are not on `main`. The SDK's - `bypass(enabled)` is a `SET_CONTROL` on control `0` (`sdk/src/generated/control-builders.ts:963-969`), - and its doc already says that the core crossfades it on both hosts. The browser sends that frame to - the core's `SessionState`, and #1341a1's control plane writes `BypassSwitch { bypassed, - ramp_samples }` with the session's mute ramp for every host. So D1-D3 have no browser code to - change, and D2's command word no longer exists. -- **The gates are met by #1341a1/a2.** Gate 1: host-core `tests/live_delta.rs` (the record's ramp is - the session's `mute_ms`; a ramp above the law's bound is refused). Gate 2: - `hosts/host-web/tests/control_parity.rs` `a_live_bypass_crossfades_alike_on_both_hosts` (the C ABI - and the browser host, bit for bit per block, at 44.1, 48, 88.2 and 96 kHz) and the scripted insert - and console-slot bypass rows of the same file; the reversal is pinned per node and banked in - `bypass_shunt_identity.rs`, and both hosts run that one core path. Gate 3: #1341a1's capi test - asserts no allocation in a ramp block; the host-web allocation gate and gate 4's commands run in - batch 9's gates. - Gate 5: no checked-in digest moved (#1341a2's record). D4's docs: the command and the shipped-host - kind are deleted, and the SDK builder's doc states the crossfade. -- The coordinator closes #1393 as completed by #1341a1 when batch 9 is upstream. diff --git a/.github/ISSUE_SPECS/1395-carry-spectrum-capture-state-across-a-plan-swap.md b/.github/ISSUE_SPECS/1395-carry-spectrum-capture-state-across-a-plan-swap.md deleted file mode 100644 index 8817f7748..000000000 --- a/.github/ISSUE_SPECS/1395-carry-spectrum-capture-state-across-a-plan-swap.md +++ /dev/null @@ -1,346 +0,0 @@ -# Carry spectrum capture state across a plan swap - -Stream A of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-14, D15-7). -Code anchors verified on `main` at `6fb211594`. - -The spectrum half of the former *Carry meter, observation and spectrum state across a plan swap*, -split for size in the decision-15 fix round. The meter and observation half is *Carry meter and -effect observation state across a plan swap* (#1327); this slice reuses its rules, its observer -location table and its program section. - -## Product outcome - -A browser page that shows a strip's or the output's spectrum sees no reset when a track is added -or another strip is edited. A spectrum capture whose target and channels are the same in both -plans keeps its analysis history, its open window, its stream epoch, its sequence and its mode -through a plan swap. The host's existing reader keeps receiving, with no `Failed` epoch, no `Gap`, -no `Warming` restart, and no window relabelled. This replaces #1269's P10 ("restart at the -swap"), which decision 15 rejects (D15-14). Only a changed or added capture starts fresh. - -## Context - -- **Observer.** `SpectrumCaptureObserver` (`crates/host-core/src/spectrum.rs:1441-1449`) holds the - `Producer`, two `SPECTRUM_WINDOW_FRAMES` planes (2048 frames, `:24`), its - `SpectrumChannels`, the shared `mode` word, the one-shot state (`SpectrumOneShotState`, `:1301`, - with its shared `state` word) and the continuous state (`SpectrumContinuousState`, `:1311`, with - its history ring, `expected_block_sample`, `next_window_start`, `sequence` and the shared - `SpectrumContinuousShared`, `:1260`). It is a graph observer (`impl GraphRuntimeObserver`, - `:2216`), bound by `GraphNodeObserverBinding { node, handle, observer: Box }` - (`crates/graph/src/lib.rs:2653-2657`). -- **Control side.** `SpectrumCapture` (`spectrum.rs:457-472`) holds the consumer, the same three - shared words (`state`, `mode`, `shared`), its read counters, its `target` and `channels`. - `SpectrumCaptureCollection` (`:803-808`) holds several captures, the `selected` index and a - `selection_epoch`; `select` (`:885`) finds an entry by `(target, channels)`, and preparation - refuses duplicate entries (`SpectrumPrepareError::DuplicateEntry`). -- **Keys.** `SpectrumTarget` (`:166-174`) names a track's post-input or post-matrix boundary, or an - output, by ID. Observer handles are positional (`SPECTRUM_OBSERVER_HANDLE - index`, - `:1420-1424`), so they are not keys. -- **A fresh observer restarts the stream.** A discontinuity resets the history and starts a new - epoch (`SpectrumContinuousReadError::Failed`, `:1242-1246`). That is what a swap to a freshly - prepared observer shows today. -- **Preparation.** `prepare_host_runtime_with_live_controls_and_spectrum` (`crates/host-core/src/prepare.rs:843`) - returns a `SpectrumCapture`, and `..._and_spectrum_collection` (`:870`) a - `SpectrumCaptureCollection`. Both build their observers at `:1676-1700`. Neither has a successor - form on `main`: *Prepare every browser preparation branch concurrently, as a successor too, - in host-core* (#1401) adds them. -- **The browser** holds the capture in its own `Single`/`Collection` wrapper - (`hosts/host-web/src/lib.rs:1482-1487` reads `stream_epoch` through it). -- **The C ABI** has no spectrum capture, so it has nothing to carry. - -## Decisions frozen for this slice - -- **D1. Key and rule (P1).** A spectrum observer carries when its `SpectrumTarget` and - `SpectrumChannels` are equal in both plans, and both plans have the same sample rate and render - quantum. A collection entry is matched by `(target, channels)`, never by index or handle. - Nothing in it is a live value, so there is no retarget. A non-carried observer starts fresh. It - changes no audio and is not a strip restart. -- **D2. Move mode.** Register spectrum observers in #1327's observer location table under D1's key, - with the new `ObserverKind::Spectrum`. A single capture is entry 0; a collection's entries are - in the collection's entry order, so an entry's index is its capture's index. At the swap block, - swap each carried observer between the plans: the whole `SpectrumCaptureObserver`, its producer - and its three shared words included. Swapping the boxed observers is enough and - allocation-free. A reader that already holds the carried capture therefore stays connected to - the observer that renders on, and its mode word still controls it. The successor's fresh - observer goes to the retiring predecessor. The section writes a `Spectrum` record in #1327 - D3's carry record for each carried observer. -- **D3. Reader pairing, at the swap block.** No capture is swapped on the control side, for - #1327 D3's reason: a commit precedes adoption and can be superseded. host-core gives two - pairing calls, allocation-free, dropping nothing, callable on the render thread: - - `SpectrumCapture::pair_carried(&mut self, predecessor: &mut SpectrumCapture, carries: - &[ObserverCarry])`, for a single capture: when the record has a `Spectrum` entry, the - successor's capture and the predecessor's swap. - - `SpectrumCaptureCollection::pair_carried(&mut self, predecessor: &mut Self, carries)`: each - carried entry's `SpectrumCapture` swaps into the successor's collection at the successor's - index for its key. When the predecessor's selected entry carried, the successor selects that - entry's new index and keeps `selection_epoch`. When it did not carry, the successor has no - selection and its `selection_epoch` is the predecessor's plus one; the host's next `select` - arms an entry as it does today. - - A host whose captures live inside the plan calls them in the swap block from its attachment - hook. The browser does: *Swap and retire browser plans through the Worker's service loop* - (#1381) D3. -- **D4. A warm successor carries by the same move.** A warm successor (*Grow latency during - playback by adopting a primed warm successor*, #1287) renders no block before its adoption at - `S` (D15-8 (round-5 amendment)). D2's section runs in that adoption block, as at any swap, so a - carried observer's `expected_block_sample` is the adoption block's start and no window is - published twice (#1327 D4). This slice adds no other mode. -- **D5. Both hosts.** Everything is in host-core and in #1327's graph table. The browser's capture - lives in its plan's host attachment, and #1381 D3's hook calls D3's pairing in the swap block, - so the browser has no other carry code. The C ABI has nothing to carry. - -## Deliverables - -1. D1 and D2: spectrum observers in #1327's location table and move section. -2. D3 in `crates/host-core/src/spectrum.rs` (the two `pair_carried` calls) and - `crates/host-core/src/prepare.rs` (the key table). The `Spectrum` variant of `ObserverKind` - in `crates/engine/src/realtime/plan.rs`. -3. Gates 1-4 in `crates/host-core/tests/successor_swap.rs`, through #1401's successor entry points. - -## Authorized paths - -- `crates/host-core/src/spectrum.rs` (observer registration, the two pairing calls and their unit - tests only; no kernel change) -- `crates/host-core/src/prepare.rs` (the spectrum key table only) -- `crates/engine/src/realtime/plan.rs` (the `ObserverKind::Spectrum` variant only; outside stream - A's ownership, root sequences it after #1327) -- `crates/graph/src/lib.rs`, `crates/graph/src/runtime.rs` (only if #1327's location table needs a - spectrum owner kind) -- `crates/host-core/tests/successor_swap.rs`, `crates/host-core/tests/support/successor.rs` - -## Non-goals - -- No browser code (#1381's hook, #1290's gate). No change to the spectrum kernels, cadence rules - or read API. -- No meter or observation tap carry (#1327). - -## Objective gates - -1. **Continuous capture carries.** A has a single continuous capture on track `t`'s post-matrix - boundary, both channels, started so that a window is open at the swap block. B adds a muted - track whose ID sorts first. Read through A's `SpectrumCapture` after every block until the swap - block; there, call `pair_carried` on B's and A's captures with B's `observer_carries()`, then - read through B's capture. No read returns `Failed`, `Gap` or `Warming` after the stream first produces; - `stream_epoch` is unchanged; `sequence` has no gap; and every window equals the reference's - field by field (a fresh B with the same capture, fed from frame 0, started at the same block). -2. **Collection and mode.** A has a collection with entries `(t post-matrix, Stereo)` and - `(u post-input, Left)`, the first selected and continuous. B lists them in the other order. - After the swap and `pair_carried`, B's collection reads windows as in gate 1, with the first - entry selected at its new index. `stop_continuous` then stops the rendering observer: no window - is published after the next block. `select` on the second entry arms the successor's observer - for it, and its first window equals the reference's. -3. **A changed key starts fresh.** B changes the channels of `(u post-input, Left)` to `Right` in - its request. That entry's reader starts fresh, and `(t post-matrix, Stereo)` carries as in - gate 1. A single one-shot capture armed before the swap and carried completes with a window - whose `first_sample` precedes the swap block and equals the reference's. -4. **Realtime.** The swap block and the pairing calls make zero allocations and frees, measured with - `bench_support::alloc`'s thread-scoped counters after warm-up. -5. Commands: - - `cargo test --locked -p engine -p graph -p host-core --features graph/test-support,host-core/test-support` - - `cargo build --locked --release -p audit && bash scripts/trace-builtins-graph-audit.sh target/release/audit && bash scripts/trace-graph-audit.sh target/release/audit` - - `bash scripts/check-host-core-policy.sh`, `cargo run --locked --release -q -p realtime-policy`, - `bash scripts/check-workspace-policy.sh` - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` - -## Test value - -- Gate 1 turns red if the carry swaps the state but keeps the successor's producer (the host - reads a dead queue), if the history is lost (a `Failed` epoch), or if the open window is - dropped or relabelled (a sequence gap). -- Gate 2 turns red if the shared mode word stays with the retired observer, so the host's stop or - select no longer reaches the rendering one, or if entries are paired by position (B reverses - the order). -- Gate 3 turns red if the rule ignores the channels and carries a window of the wrong planes, or - if a pending one-shot is lost at the swap. -- Gate 4 turns red if the swap allocates, frees or drops a boxed observer on the render thread. - -## Dependencies - -- *Carry meter and effect observation state across a plan swap* (#1327), for the observer - location table, the move section and the carry record. -- *Prepare every browser preparation branch concurrently, as a successor too, in host-core* (#1401), - for the successor entry points the gates use. - -**Decision 17 amendment (2026-10-13).** D1 and D2 are unchanged. D5's "The C ABI has nothing to carry" ends with #1593: the C ABI gains spectrum captures, and its control plane pairs them with D3's `pair_carried` when it promotes the adopted candidate, on the control thread, from a pairing list built by D1's key at preparation. #1596 moves the browser to the same control-side pairing, so from then on no host pairs spectrum readers in the swap block. - -## Amendment (controls coordinator, batch 10, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Anchors moved.** In `crates/host-core/src/spectrum.rs`: `SPECTRUM_OBSERVER_HANDLE` `:31`, - `SpectrumTarget` `:204`, `SpectrumCapture` `:525`, `SpectrumCaptureCollection` `:967`, `select` - `:1088`, the observer's `impl GraphRuntimeObserver` `:2497` (its `block_end` `:2543`, from - #1609). The successor entries exist since #1401: `prepare.rs:1201` (single) and `:1233` - (collection). -- **Where the readers live now.** Since #1591 and #1592 the captures are held by host-core's - `SpectrumService` (`crates/host-core/src/spectrum_service.rs:585`), on the control side. Since - #1381a the browser moves its capture out of the plan's attachment into it at boot. So D3's "a host - whose captures live inside the plan calls them in the swap block" fits no host, and D5's browser - sentence is void. **D5 now reads:** both hosts pair on the control side when the adopted - candidate is promoted, as #1593 D5 states for the C ABI. This slice adds that pairing to - `SpectrumService` as one method (the successor's prepared captures in, D3's rules applied, the - result installed), built from D3's two functions. The browser's call is #1290 (d); the C ABI's is - #1593. -- **The pairing list.** The swap block writes the carry record into the plan, which the control - side cannot read in Worker mode. The control side builds the same list at preparation from D1's - key, and applies it at promotion when the adoption carried; when the hand-over was refused - (`CarryOutcome::PredecessorMismatch`, counted by `RealtimePlanOwner::carry_mismatch_count`) it - installs the successor's fresh captures. Gates 1-3 call the `SpectrumService` method after the - adoption block instead of calling `pair_carried` in it. -- **One-shot conflict with #1618 (open).** #1618 recommends settling an in-flight one-shot capture - to `Invalid` at a swap (its option A). Gate 3's last sentence carries it instead. Recommendation: - a carried observer keeps its one-shot state (D2 moves the whole observer, so it completes and - gate 3 stands); #1618's settle applies only to a capture that does not carry (its key changed or - it was removed) and to a refused hand-over. #1618 lands after this slice, rescoped so; root - rules. -- **Engine edit.** `ObserverKind::Spectrum` is declared by #1327a if root accepts the fold in - #1327's batch 10 amendment; then this slice edits no engine file. Otherwise the variant is this - slice's one `plan.rs` edit, under its own one-attempt rule. -- **Split (plan v2).** #1395a: the single capture (gates 1, 3's single part, 4); #1395b: the - collection (gate 2, 3's collection part). -- **Dependencies.** #1327a (the table, the record and the section). #1401 is closed. - -## Amendment (controls coordinator, batch 12, 2026-10-15) - -Checked against `ea336c039`. - -- **No engine edit.** #1327a declared `ObserverKind::Spectrum` (`crates/engine/src/realtime/plan.rs:548-555`). - This slice edits nothing under `crates/engine/src/realtime`. -- **The observer table is generic.** `GraphRuntimeObserver::carry_kind` returns - `Option` (default `None`, `crates/graph/src/lib.rs:2887`). `GraphObserverRow` and - `GraphObserverMove` carry their kind (`:3519`, `:3533`), and the move section writes the kind of - each move into the record. So D2 is: `SpectrumCaptureObserver` (`spectrum.rs:2497`) answers - `Some(Spectrum)`, and host-core gives spectrum rows keyed by D1, as `MeterInventoryRow` and - `resolve_meter_carry` do for meters (`crates/host-core/src/prepare.rs:3152`). A graph edit is still - "only if needed". The spectrum observer's binding handle is positional - (`SPECTRUM_OBSERVER_HANDLE - index`, `spectrum.rs:31`). So when a collection is reordered, the - observer swapped in must take its new binding's handle (`adopt_binding_handle`), as the meter - observer does. -- **D3 and D5 as amended in batch 10 still hold.** Pairing happens on the control side when the - adopted candidate is promoted, as one `SpectrumService` method (`spectrum_service.rs:585`) that - uses D3's two functions. No `pair_carried` exists yet. Add `crates/host-core/src/spectrum_service.rs` - to the authorized paths (the batch 10 amendment requires it, but the list was not changed). -- **Where the browser's successor capture is.** The boot plan's capture is moved into - `SpectrumService` (`take_spectrum_capture`, `hosts/host-web/src/lib.rs:5139`, called at `:5932`). - A successor's capture stays in the successor's `RenderCompanions` (`:5055`). #1290 (d) takes it - out at promotion, in the service step after the adoption block, and calls this slice's method. - Between adoption and promotion, a carried entry is served by the capture that the service - already holds, because its observer was swapped. An entry that is not carried reads as pending - until promotion. -- **A refused hand-over.** `RealtimePlanOwner::carry_mismatch_count` - (`crates/engine/src/realtime/plan_exchange.rs:719`) is readable at promotion only when the - control and render halves are one instance (single mode, the only browser mode today). The C ABI's - #1593 and the browser's Worker mode (#1332) must publish it to the control side. Record this as - a rule for those slices. This slice does not change it. -- **Anchors.** The batch 10 anchors in `spectrum.rs` are unchanged. The successor entries are now at - `prepare.rs:1339` (single) and `:1371` (collection). -- **#1618** lands after #1395b, as root ruled. Its settle is part of the same `SpectrumService` - method: an entry that is not carried, or all entries after a refused hand-over. So - `spectrum_service.rs` is edited in this order: #1395a, #1395b, #1618. -- **Order in lane K.** #1395a, then #1395b, after #1584 D6a. Both slices edit - `crates/host-core/src/prepare.rs` and `crates/host-core/tests/successor_swap.rs`. - -## Decision record: #1395a, the single capture (batch 8) - -**Shipped.** A single spectrum capture carries across a plan swap, and the control side keeps -reading it. - -- *Swap block (D1, D2).* `SpectrumCaptureObserver` answers `carry_kind() = Some(Spectrum)` - (`spectrum.rs`). `prepare.rs` gives each single capture an inventory row keyed by - `(target, channels)` (`SpectrumInventoryRow`), and `resolve_spectrum_carry` joins the rows with the - predecessor's by key, never by entry or handle, and only at equal sample rate and quantum (both - are now in `PlanStateInventory`). The moves go into the one observer section with the meters' - (`install_observer_carry`), and a carry program is requested when a capture may carry - (`spectrum_may_carry`). No engine, graph or `latest_cell` edit. The observer keeps no binding - handle, so `adopt_binding_handle` has nothing to do (a collection's reorder in #1395b needs no - handle work either). -- *Pairing at promotion (D3, D5, batch-10 amendment).* The pairing list is built at preparation - from the same join (`PlanStateInventory::spectrum_pairing`, `SpectrumPair`), so it names exactly - the observers the swap block moves. `SpectrumCapture::pair_carried(&mut self, predecessor, - pairs)` swaps the two captures only when the list names entry 0 of both and their target and - channels are equal. `SpectrumService::adopt_successor(successor, pairing, carried)` applies it - when the adoption carried, installs the successor's fresh captures otherwise (a refused - hand-over), and returns the displaced captures so the caller drops them off the audio thread. - The service is allocation- and free-free here. No host pairs in the swap block. -- *Departures.* (1) `pair_carried` takes the control-side pairing list, not the swap block's - `ObserverCarry` slice (the record exists and a test proves it equals the list). (2) The - inventory carries only single captures' rows: a collection has none, so it never moves an - observer; #1395b adds its rows and the collection arm of `adopt_successor`. (3) An in-flight - arm or stream on a capture that does not carry is not settled by `adopt_successor`; that is - #1618's, in this method. (4) `browser_successor.rs` gate 2 read the successor's own capture; that - observer is now carried away, so the test pairs the captures first (a mechanical update). - -**Gates -> tests** (`crates/host-core/tests/successor_swap.rs`, `mod spectrum_carry`): -gate 1 `a_continuous_capture_carries_and_its_reader_keeps_receiving` (status, window fields and -planes equal the reference's block by block, epoch 1, no gap; record == pairing list); -`a_refused_hand_over_installs_the_fresh_capture` (the oracle can fail); -gate 3 `an_armed_one_shot_completes_across_the_swap` (a window that spans the swap, equal to the -reference's) and `a_changed_key_starts_fresh`; gate 4 -`the_swap_block_and_the_promotion_allocate_and_free_nothing` (zero allocator calls in every -block and in `adopt_successor`). Unit tests: `spectrum::tests::pair_carried_swaps_only_a_named_pair_of_one_key`, -`prepare::tests::a_spectrum_capture_carries_by_key_at_one_shape`. Gate 2 (collection) is #1395b. - -## Decision record: #1395b, the collection (batch 8) - -**Shipped.** A collection's entries carry across a plan swap, matched by `(target, channels)`, and -the control side keeps each entry's reader. - -- *Swap block (D1, D2).* `prepare.rs` gives every collection entry an inventory row (`entry` is its - index, its location is looked up by the positional handle `observer_handle(entry)`), and - `resolve_spectrum_carry` joins the rows by key in any order, at equal rate and quantum, into the - same observer section and the same pairing list as the single capture. No engine, graph or - `latest_cell` edit. A collection's reorder needs no handle work: the observer keeps no binding - handle (#1395a). -- *Pairing at promotion (D3).* `SpectrumCaptureCollection::pair_carried(&mut self, predecessor, - pairs) -> bool` swaps each named pair of captures (successor index, predecessor index). It - checks the whole list first (index range, equal key per pair, successor entries strictly - increasing; two pairs on one predecessor entry cannot pass, because a collection admits no - duplicate key); a fault pairs nothing, like an empty list. The selection follows its entry: the - successor selects the entry's new index and keeps `selection_epoch`; when the predecessor's - selected entry did not pair, the successor has no selection and its epoch is the predecessor's - plus one; a predecessor with no selection leaves none and its epoch. It runs for an empty list - too. `SpectrumService::adopt_successor` calls it for a collection with the list when the - adoption carried and with an empty list after a refused hand-over, so a refused hand-over also - advances the epoch. Allocation-free, drops nothing; the caller drops the displaced collection. -- *Departures.* (1) `PlanStateInventory` records whether its captures are a collection - (`spectrum_collection`). A single capture and a collection never carry into each other: the - service holds one kind and pairs inside it, so a plan whose capture kind changed starts fresh. - The spec does not say; no host changes kind across a swap today. (2) As in #1395a, - `pair_carried` takes the control-side `SpectrumPair` list, not the swap block's record (a test - proves the two are equal). (3) The O(n) validation pass uses the key-uniqueness argument above, - not a set. (4) Not settled here, and left to #1618: the stream or arm that the service holds when - the selected entry did not carry. After a lost selection the service still has `stream_active`; - a later `select_stream` then arms a one-shot (a collection restarts a cadence only from a - selected entry), so #1618's settle must reset the service's stream state in the same method. - -**Gates -> tests.** `crates/host-core/tests/successor_swap.rs`, `mod spectrum_carry`: gate 2 -`a_reordered_collection_pairs_by_key_and_its_reader_keeps_receiving` (B lists the entries in the -other order; pairing `[(0,1),(1,0)]` equals the swap block's record; the stream on `t` reads window -for window with no `Failed`/`Gap`/`Warming`, epoch 1, no sequence gap, a window across the swap; -the selection epoch equals the reference's; a `select_stream` to the second entry (which carried -idle) reads windows equal to the reference's, bit for bit; the stop that this switch makes reaches -the rendering observer); -gate 3 collection part `a_collection_entry_with_a_changed_key_starts_fresh` (`u` Left to Right: one -pair, one record, the changed entry reads its own plane like the reference); -`a_refused_hand_over_installs_the_fresh_collection_without_a_selection` (no selection, epoch 2, the -displaced collection keeps its selection); gate 4 -`a_collection_swap_block_and_promotion_allocate_and_free_nothing` (zero allocator calls in every -block and in `adopt_successor`, two entries moved). Unit tests: -`spectrum::tests::collection_pair_carried_follows_keys_and_the_selection` (selection index and -epoch, a stop through the successor's collection leaves the rendering observer's queue empty after -a window was published without it, the other entry arms its own observer, lost selection, three -list faults), `prepare::tests::a_spectrum_collection_carries_by_key_in_any_order`. The harness -now takes either request kind (`Spec`), and the host's script (`Script`) is replaced by the -successor's at the swap. All #1395a tests are unchanged and green. - -**Commands run.** `cargo test --locked -p host-core --all-features`, `-p graph`, `-p control-plane ---features test-support`, `-p host-web --features host-web/test-support`; clippy `-D warnings` on -host-core with no features, default and all features (all targets); the host-core loom step -(`spectrum_handoff_loom`); `realtime-policy`; `check-bench-policy.sh`; `cargo fmt --all --check`; -`check-cross-targets.sh`; `RUSTDOCFLAGS='-D warnings' cargo doc -p host-core --no-deps ---all-features`. All pass. - -## Fold (batch 12a review) - -`PlanStateInventory::spectrum_count` had no caller and no gate that it would defend (the capture -rows are read through `spectrum_carry_pairs`, which the carry gates pin), so it is deleted. diff --git a/.github/ISSUE_SPECS/1396-give-a-plan-a-source-read-clock-that-leads-its-render-clock.md b/.github/ISSUE_SPECS/1396-give-a-plan-a-source-read-clock-that-leads-its-render-clock.md index f12421596..bee72f876 100644 --- a/.github/ISSUE_SPECS/1396-give-a-plan-a-source-read-clock-that-leads-its-render-clock.md +++ b/.github/ISSUE_SPECS/1396-give-a-plan-a-source-read-clock-that-leads-its-render-clock.md @@ -159,3 +159,83 @@ both clocks. - *Anchor every seek on the plan's source-read clock* (#1316). - *Reset latency floors at a host-declared discontinuity* (#1323): `SuccessorBase::discontinuity`. + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `228fca26c`. + +**Moved anchors.** Each fact in Context still holds; only the lines moved. + +- `RenderTime` is `crates/engine/src/realtime/plan.rs:576-579`. The plan's clock field is `:755`, + its getter `:910`, `adopt_absolute_sample` `:918`, `render_contiguous` `:1172-1185`, + `adopt_predecessor_plan` `:1201-1207`. +- `PreparedPlanExecutor` is `plan.rs:273`; its defaulted methods now run to about `:360` and + include #1311's `prime_ready` (`:349`). +- The graph's source-section call is `crates/graph/src/lib.rs:4843-4844`, in the block entry that + #1639 changed (the jump-length read is `:4823-4826`, just before it). `GraphExecutor` is + `:3192`, its `PreparedPlanExecutor` impl `:4588`. +- The source driver's block entry is `crates/source/src/lib.rs:1752-1785`; the + `begin_block_at(first_sample)` call is `:1772`. `begin_block_at` is `:1115`. +- `miso_engine_v1_render_f32_planar` is `crates/capi/src/ffi.rs:818`. +- `RealtimePlanOwner::enter_block` is `plan_exchange.rs:742-795`; the clock adoption is `:779`. + `plan_exchange` is `:367`, `plan_exchange_at_revision` `:410`, and both call + `plan_exchange_inner` (`:436-470`), which is where D4's record is allocated. + `plan_exchange_resource_report` is `:337-361`. +- `SuccessorBase` is `crates/host-core/src/prepare.rs:1070-1097`. It now has `inventory`, + `committed`, `revision` (#1277), `edit_ramps` (#1394), `monitor` (#1559) and `observation` + (#1584). It has no `new` constructor yet; #1323 D3 adds it. `PlanStateInventory` is `:711-757`. +- The control plane's successor construction site is `crates/control-plane/src/control.rs:1670`. + +**Code notes for the implementer.** + +- `GraphExecutorWithoutSplitPairTable` (`crates/graph/src/lib.rs:4438`) mirrors + `GraphExecutor`'s layout for the largest-allocation cap. D1's new offset field goes in both. + This is inside D1's "offset field" scope. +- D4's new record and D1's field can move pinned measurements outside the authorized paths: + `crates/capi/tests/resource_lifecycle.rs` (the capi retained measurement) and + `hosts/host-web/tests/browser-v1/expected.json` (browser boot memory). The control plane + charges the exchange through `crates/control-plane/src/compile.rs:237`. If a pin moves, re-pin + it from the measurement with the reason, as batch 13a and 13b did. Add those two files to the + authorized paths for that purpose only. +- #1637 shipped node time as `s = r + ΣP - a(n)`. No code reads `ΣP` yet, so it is 0 today. + *Build the timeline consumer and producer* (#1640) and *Give every plan a timeline that + carries like a source* (#1641) will pass this slice's source-read sample to the timeline. + Nothing in this slice changes for that. +- Host-core builds a source set for every plan (`crates/host-core/src/prepare.rs:2579`), with no + entries for an empty session (#1682). D1 then offsets a clock that no source reads, and D4 + still publishes both clocks. + +**Dependencies on GitHub.** + +- #1316 and #1323 are open. Both are in the batch-15/16 list. +- #1323 depends on *Supersede an unadopted candidate plan by compare-and-swap* (#1310, open), + and #1310 depends on #1344 (open). Neither is in the batch-15/16 list. So D2's declaration + term and gate 5 cannot run until #1310 and #1323 land. +- Batch order (root, 2026-10-15): batch 15a is the controls lane, #1344, #1398 and #1310. + #1324 left batch 15a (root ruling, 2026-10-15); it lands in a later batch after #1288, #1391, + #1363 and #1325. + The rest of batch 15 follows. #1323 lands after #1310 (B15-R5 in #1323). +- #1314 and #1309 are closed. The control plane is `crates/control-plane/src/`, as the spec says. +- #1354, #1355, #1320, #1358, #1360 and #1361 are open. They are follow-ons or non-goals, not + dependencies. + +**Decisions.** + +- D1-D5 still hold against the code and decision 17. The clocks get no host surface here, and a + later host surface goes through the control plane (decision 17, C1). +- #1323 D1 adds a C export (`miso_engine_v1_declare_discontinuity`). Decision 17 C13 routes + new features through protocol messages. That is #1323's question, not this slice's: gate 5 + calls the control plane, not the export. + +**Gates and commands.** `crates/host-core/tests/source_read_clock.rs` is new. Every named script +and package exists (`realtime-policy` is `tools/realtime-policy`). The features +`engine/realtime-audit`, `host-core/test-support` and `control-plane/test-support` exist. + +**Realtime-path edits.** D1 and D4 edit `crates/engine/src/realtime/plan.rs`, +`plan_exchange.rs` and `mod.rs`. D1 also edits the graph's render block entry. That slice gets +one attempt. + +## Amendment (controls coordinator, batch 15a fold, 2026-10-16) + +- **Batch placement.** "Neither is in the batch-15/16 list" (Dependencies on GitHub, about #1310 + and #1344) is void. #1310 and #1344 are batch 15a; #1323 lands after #1310 (B15-R5). diff --git a/.github/ISSUE_SPECS/1397-duck-swap-the-strips-a-latency-growth-restarts-and-fall-back-to-the-transition-w.md b/.github/ISSUE_SPECS/1397-duck-swap-the-strips-a-latency-growth-restarts-and-fall-back-to-the-transition-w.md index f4c0414a7..3d3e7c811 100644 --- a/.github/ISSUE_SPECS/1397-duck-swap-the-strips-a-latency-growth-restarts-and-fall-back-to-the-transition-w.md +++ b/.github/ISSUE_SPECS/1397-duck-swap-the-strips-a-latency-growth-restarts-and-fall-back-to-the-transition-w.md @@ -317,3 +317,9 @@ warm edit would be `LeadBound`. - *Prepare a successor across a withdrawn candidate plan* (#1344), D1-D3 and D5. - *Size the C ABI's plan capacities and resource admission for a superseding candidate* (#1398), D4. + +## Amendment (controls coordinator, batch 15a, 2026-10-15) + +- `AdmissionPeak::WithReprepare` named above did not ship with #1398 (root ruling: code with no + production caller is a placeholder). #1403 adds the reprepare admission peak together with its + first caller; see its batch 15a amendment. diff --git a/.github/ISSUE_SPECS/1398-size-the-c-abis-plan-capacities-and-resource-admission-for-a-superseding-candida.md b/.github/ISSUE_SPECS/1398-size-the-c-abis-plan-capacities-and-resource-admission-for-a-superseding-candida.md index b23ccffe5..a86ea1c28 100644 --- a/.github/ISSUE_SPECS/1398-size-the-c-abis-plan-capacities-and-resource-admission-for-a-superseding-candida.md +++ b/.github/ISSUE_SPECS/1398-size-the-c-abis-plan-capacities-and-resource-admission-for-a-superseding-candida.md @@ -71,22 +71,9 @@ a withdrawn or displaced candidate and its successor, and three compiled models. (`effect.resource.limit`); `maximum_builtin_retained_bytes` and `maximum_capi_retained_bytes` (`capi.resource.limit`). The largest-allocation check folds every held row. Before #1310 the held set is at most two plans, so no current admission changes. - - **Re-preparation peak.** A candidate that the control thread may later prepare again without - a submit, from the same committed model, is admitted with `AdmissionPeak::WithReprepare`. That - is a warm successor published `Primed`: when it is not ready by its deadline, the control plane - withdraws it and prepares the transition instead, keeping the withdrawn candidate as the donor - of its rings (*Fall back to the transition when a warm successor is not ready by its - deadline*, #1358; *Duck-swap the strips a latency growth restarts, and fall back to the - transition when a warm successor cannot adopt*, #1397). Admission then also checks a second - peak: the running plan plus the prospective row twice (the warm candidate and the transition's - re-preparation, which #1397 keeps no larger than the warm candidate in any D4 term), with the - same model terms. Every other held candidate is dropped before a re-preparation can run, so it - is not in that peak. Both peaks must fit, or the submit is refused before commit with the - cap's diagnostic. `service` then runs no cross-plan admission. The default, - `AdmissionPeak::Single`, is today's check; *Classify a latency-growth edit and publish its - warm successor from the control plane* (#1403) D2 passes `WithReprepare` when it admits a - warm candidate. Releasing the withdrawn plan's row before re-preparing is not an option: the - withdrawn plan is the donor of the rings it holds. + - **Re-preparation peak: dropped.** A second peak for a warm candidate's re-preparation + (`AdmissionPeak::WithReprepare`) is not in this slice; see the root ruling in the decision + record. #1403 adds it with its first caller. - **D5. Sizing rule and reference limits.** The header states it: each cap above must be at least three times one plan's row, and the graph cap also covers three compiled models. The repository's reference limits must admit three plans of their sessions: `audit_limits` @@ -126,10 +113,7 @@ a withdrawn or displaced candidate and its successor, and three compiled models. 2. **Admission sums every held row (new control-plane unit test).** Three held rows and three models, with each cap in D4 one byte below the sum in turn, refuse with that cap's diagnostic; at the sum they pass. With one held row the result equals today's for the same inputs. -3. **Re-preparation peak (new control-plane unit test).** One running row `R` and a prospective - row `C` admitted with `WithReprepare`: with each D4 cap set to `R + 2C - 1` the submit is - refused with that cap's diagnostic, at `R + 2C` it passes; with `Single`, `R + C` passes. With a - withdrawn candidate also held, both peaks are checked and the larger one decides. +3. **Dropped** (root ruling 2026-10-15, decision record): the re-preparation peak moves to #1403. 4. **Reference limits.** `./target/release/audit capi` and the two capi integration tests pass with their limits (raised only if needed, D5). 5. Commands: @@ -154,3 +138,188 @@ a withdrawn or displaced candidate and its successor, and three compiled models. ## Dependencies - *Extract the C ABI control plane into a portable crate both hosts call* (#1309). + +## Amendment (controls coordinator, batch 15 check, 2026-10-15) + +Checked against `e2f6cfb24`. + +**Batch order (root, 2026-10-15).** Batch 15a is the controls lane: #1344, #1398, #1310, in +that order. #1324 left batch 15a (root ruling, 2026-10-15); it lands in a later batch after #1288, +#1391, #1363 and #1325, in that order. The rest of batch 15 follows. + +**Moved anchors.** #1309 has landed; the files are in `crates/control-plane/src/` as the spec +expects. Each fact in Context still holds; only the lines moved. +- The lag refusal and its comment are `control.rs:1579-1586`. `synchronize_plan_epochs` is + `:1401`. +- `plan_alive == false` is `control.rs:1541-1543`. +- The reservation refusals are `control.rs:1751-1765`. Since #1343 and #1311 the error has four + arms: `PublicationFull`, `RetirementFull`, `Incompatible` and `EpochExhausted`. D3 covers the + first two only; the other two stay as they are. +- The full report table or pending list is `control.rs:1781-1788`. The one-candidate refusal + `if !self.pending_providers.is_empty()` is `:1742-1744`; it is #1310's, not this slice's. +- Capacities in `compile.rs`: the exchange resource row's retirement capacity `:237-239`; the + exchange's own `:1121` (`one`, `:1092`); report rows `:1125-1128`; pending providers + `:1168-1171`; retired providers `:1172-1175`. +- `validate_replacement_peak` is `compile.rs:646`; `validate_live_peak` is `:770`; + `CompiledModelAdmission` is `:82`. +- `all_limits_nonzero` is `crates/capi/src/runtime/mod.rs:144-169`. +- The three #1042 tests are `crates/capi/src/runtime/tests.rs:1715`, `:1885` and `:2005`; the + outcome helper is `:1687-1706`. Each still asserts `Backpressure` in the window. +- `audit_limits` is `tools/audit/src/capi.rs:599`. `limits` is `crates/capi/tests/plan_swap_race.rs:56` + and `crates/capi/tests/resource_lifecycle.rs:176`. + +**Code notes for the implementer.** +- D1 holds against #1343. The exchange's `retirement_capacity` now counts retirement credits: + reserved candidates, unadopted candidates and retired plans together + (`crates/engine/src/realtime/plan_exchange.rs:70-71`). D1's three are exactly those. Only the + config value changes; `plan_exchange.rs` does not. +- D4 misses one admission term. #1290a D3 added `ControlLimits::maximum_replacement_peak_bytes` + (`crates/control-plane/src/adapter.rs:81-87`), checked in `validate_replacement_peak` as + `control.resource.replacement_peak` (`compile.rs:686-697`). It sums graph, models and + allocated source bytes. D4's "every held row" rule applies to it too, and gates 2 and 3 cover + it. The C ABI sets it to `u64::MAX` (`crates/capi/src/runtime/mod.rs:211`); the browser sets + its one budget (`hosts/host-web/src/lib.rs:5081`, and `set_replacement_peak_limit` at + `:6347`). +- D4's `maximum_capi_retained_bytes` is `ControlLimits::maximum_control_retained_bytes` in the + control plane (`adapter.rs:59`); the C ABI maps one to the other (`runtime/mod.rs:199`). + +**Decisions.** D1-D4 and D6 hold against the code and decision 17. D5 does not hold as written. +- **ROOT RULING (2026-10-15): this slice covers both hosts.** The finding follows; the ruled + text is after it. The browser now runs this control plane. D5 says "The browser's caps + follow the same rule when it adopts the control plane (#1332)", and the non-goals leave the + browser's caps to #1332. Since #1381a the browser runs `control_plane::SessionState` in its one + instance (`hosts/host-web/src/lib.rs:6-16`). So D1's capacities and D4's sums reach the browser + in this slice, through the same code (decision 17 C1). The browser's limits are its one memory + budget (`control_limits`, `hosts/host-web/src/lib.rs:5053-5083`). D1's larger tables add fixed + bytes there, and browser byte pins can move. Proposed: this slice covers both hosts. D5's + browser sentence becomes: "The browser's caps are its memory budget; its replacement-peak term + must cover the three-plan peak when #1403 admits a warm candidate." Add to Authorized paths, + for moved pins only, with the measured reason: `hosts/host-web/src/tests.rs`, + `hosts/host-web/tests/` and `hosts/host-web/tests/browser-v1/expected.json`. Gate 4 adds + `cargo test --locked -p host-web --features host-web/test-support`. + The ruled text, which supersedes D5's browser sentence and the non-goal "The browser's caps + (#1332)": + - **D5, browser sentence (restated).** The browser's caps are its one memory budget + (`control_limits`, `hosts/host-web/src/lib.rs:5053-5083`). D1's larger tables are charged + in that budget, through the same control-plane code as the C ABI. Its replacement-peak term + (`set_replacement_peak_limit`, `:6347`) must cover the three-plan peak when #1403 admits a + warm candidate. This slice adds no browser cap and no browser default. + - **Host-web test (gate 1b).** A new test in `hosts/host-web/tests/plan_exchange_threads.rs` + is the browser twin of gate 1's first test. Boot, commit a structural edit (add a muted + track), and render the block that adopts it. Before the epoch atomic is published, submit a + second structural edit through the browser's control path. It returns OK (the base returns + `BACKPRESSURE`), its plan is adopted at the next block, and the active epoch then equals the + second plan's epoch. To hold the window, drive the render's two halves through + `control_plane::PlanState::test_owner_mut` and `test_active_epoch` (`crates/control-plane/src/plan.rs:219-229`). + If host-web cannot reach them, add one `#[cfg(feature = "test-support")]` accessor beside + `active_plan_epoch_for_test` (`hosts/host-web/src/lib.rs:2297`). Test value: the lag refusal + kept on the browser path, or a browser budget that does not charge D1's tables and so refuses + the second plan, turns it red; gate 1's C ABI tests do not boot the browser. + - **Authorized paths (added).** `hosts/host-web/tests/plan_exchange_threads.rs` (the new test); + `hosts/host-web/src/lib.rs` (the test-support accessor only); and, for moved pins only, each + with its measured reason in the PR: `hosts/host-web/src/tests.rs`, `hosts/host-web/tests/` + and `hosts/host-web/tests/browser-v1/expected.json`. + - **Gate 4 (added command).** `cargo test --locked -p host-web --features host-web/test-support` + passes, gate 1b included. + +**Dependencies on GitHub.** +- #1309 is closed. #1343 and #1311 are closed. +- Downstream: #1310 (batch 15a). #1403, #1358 and #1397 are open and not in batch 15; #1403 adds the + re-preparation peak with its first caller (root ruling 2026-10-15, decision record). + +**Gates and commands.** Every named command, package, feature and script exists. Gate 4's +`./target/release/audit capi` is built by gate 5's `cargo build` line. + +**Realtime-path edits.** None under `crates/engine/src/realtime`. The capacities are control-side +configuration (`PlanExchangeConfig`), read at boot. + +## Decision record: #1398 (batch 15a) + +**Shipped.** +- D1 in `crates/control-plane/src/compile.rs`: `REPORT_ROWS = 4`, `PENDING_PROVIDERS = 2`, + `RETIRED_PROVIDERS = 2`, `RETIREMENT_CAPACITY = 3` (exchange and its resource report alike). The + capi fixed rows charge the pending and retired lists as two rows (they are two allocations) and + the report table at 4. +- D2: the lag refusal is deleted (`control.rs`), not mapped to another error. +- D3: `PublicationFull` and `RetirementFull` at reservation, and a full report table or pending + list at commit, return `CommandError::Internal` with a `debug_assert!`; `Incompatible`, + `EpochExhausted` and `plan_alive == false` stay `BACKPRESSURE`. +- D4: `validate_replacement_peak` and `validate_live_peak` take every held plan + (`SessionState::held_epochs`: the running plan, then every pending candidate, from the report + table's provider rows, never the atomic's; a fixed array, no allocation) and every compiled model + (`compiled_model_admission_over`). The replacement admission sums every D4 term including + `maximum_replacement_peak_bytes`; the capi term is the newest held row plus every older held + epoch's `epoch_retained` plus the candidate's epoch and prepared-protocol rows (the live + admission's rule). The largest-allocation check folds every held row. (The attempt also added + `AdmissionPeak::{Single, WithReprepare}`; the root ruling below removes it.) +- D5: the header's sizing paragraph sits on `maximum_graph_session_plus_plan_bytes`; + `docs/C_ABI_V1_QUALIFICATION.md` has a #1398 section. No reference limit was raised: the + nine-track EQ session's rows are graph 316,350, source total 12,198, capi 161,258 bytes against + caps of 100,000,000 / 10,000,000 / 10,000,000. +- Gate 1b's accessor is `AudioWorkletEngineHost::render_plan_at_for_test` (`hosts/host-web/src/lib.rs`). + +**Departures.** +- The one-candidate refusal (`if !self.pending_providers.is_empty()`, #1310's) moved from after the + admission to just before it. With D4 summing the pending plan, an edit made while a candidate + waits would otherwise be compile-rejected (permanent) where it was backpressure (retryable), and + `a_valid_edit_inside_the_swap_window_is_admitted_against_the_running_plan` pins that pending + case as `Backpressure`. #1310 replaces that line. +- Gate 1b holds the window with a mistimed render call (the owner adopts the successor, then + refuses the block, so the atomic is not published), through one test-support accessor that uses + only `PlanState::render`. `PlanState::test_owner_mut`/`test_active_epoch` are behind + `control-plane/test-support`, which host-web's `test-support` feature does not enable; enabling + it would turn on the control plane's fault hooks and lifecycle observers in every host-web test + and needs a `Cargo.toml` edit outside the authorized paths. +- `crates/capi/src/runtime/live_peak_tests.rs`: mechanical caller update for the slice signature. +- Gate 1's third test now publishes a fresh candidate after each round (the window's plan is + adopted by the good block, so the next round needs one); its canary assertion is now `false`, + because an admitted command writes its response. + +**Gates.** +- Gate 1: `control_calls_inside_a_plan_swapping_render_call_keep_replacement_live`, + `a_valid_edit_inside_the_swap_window_is_admitted_against_the_running_plan` (renamed), + `a_rejected_render_call_that_swaps_plans_leaves_the_c_session_live` (`crates/capi/src/runtime/tests.rs`): + pass; all three red with the lag refusal restored. +- Gate 1b: `a_structural_edit_inside_the_swap_window_is_admitted_on_the_browser_path` + (`hosts/host-web/tests/plan_exchange_threads.rs`): pass; red (`BACKPRESSURE`) with the lag + refusal restored. +- Gate 2: `compile::admission_tests::the_replacement_admission_sums_every_held_plan_and_model`. +- Gate 3: dropped (root ruling below). +- Gate 4/5: `cargo test --locked -p capi` (incl. `resource_lifecycle`, `plan_swap_race`), + `-p control-plane --all-features`, `-p host-web --all-features`; `audit capi` (pcm_digest + `bf7dcedd1d92621f`, 0 violations; `tools/audit/src/capi.rs` unchanged); `check-capi-abi.sh` and + `--self-test`; `check-workspace-policy.sh`; `realtime_policy`; fmt; clippy on the three crates: + all pass. + +**Measured.** Nine-track EQ reference session `capi_retained_bytes` 159,354 -> 161,258 (+1,904, +#1398 D1's tables). No test pinned that row; no browser pin moved. + +Root ruling (2026-10-15): gate 3 and `AdmissionPeak::WithReprepare` are dropped from this slice: code with no production caller is a placeholder. #1403 adds the reprepare peak together with its first caller. + +**Review fold (batch 15a).** +- *M2 (D4 "a carried ring counted once").* `validate_replacement_peak` subtracted only the new + candidate's carried bytes, while a held candidate's row also holds rings it carried from the + running plan. Each provider epoch now keeps its `CarriedSourceBytes`, `LiveEpochResources` + carries it, and the source-total and source-overhead sums (and so the replacement-peak term) + subtract it for every held epoch after the running one. Gate 2 adds a middle row carrying 13 + source bytes (5 overhead) and a running row carrying 2 (1): each cap passes at the peak and + refuses one byte below; red when every held row's carried bytes are left in. Measured on the + nine-track EQ reference session (T1 adds a source track, T2 a compressor; P0 12,198, A 23,282 + of which 10,640 carried): the smallest admitting `maximum_source_total_bytes` moves 48,122 -> + 37,482, the three-plan sum with each carried ring once (12,198 + 12,642 + 12,642). +- *M1 (with #1310).* The D3 accounting fault could fire after two `Taken` outcomes in a race; a + structural edit now promotes every pending candidate up to a taken newest one before it rebuilds, + so the report table and the pending list never meet more than D1 sizes (see #1310's fold). +- *Item 2.* A refusal at the three-plan peak is `BACKPRESSURE` when the edit fits beside the newest + held plan alone; `COMPILE_REJECTED` only otherwise (see #1310's fold). The header's sizing + paragraph says so, and says the rule is measured from a running successor plan's row. +- *m5.* The `held_epochs` doc no longer claims a displaced plan is reclaimed before any admission: + it is not charged, and between render's swap and the next service step it lives beside the held + plans uncharged. +- *m6.* `capi_retained_bytes` on the reference session is 165,842 at the batch 15a head and 165,922 + after this fold (x86-64; +80, which matches the 16-byte carried field in five provider-epoch + slots; measured with part B's uncommitted host-core and source edits in the tree), within + the 175,296 ceiling of `resource_lifecycle.rs`; the 161,258 above was not the head figure. + `C_ABI_V1_QUALIFICATION.md` states both and no longer says #1310's boot row "does not move" or + that a full plan queue is backpressure (it is `INTERNAL` since D3). The four-lane + `builtin_bank_bytes` comment cites AArch64 CI job 116713280504 (`12969 of 21056`). diff --git a/.github/ISSUE_SPECS/1403-classify-a-latency-growth-edit-and-publish-its-warm-successor-from-the-control-p.md b/.github/ISSUE_SPECS/1403-classify-a-latency-growth-edit-and-publish-its-warm-successor-from-the-control-p.md index 4bebbab7f..3ab8d0074 100644 --- a/.github/ISSUE_SPECS/1403-classify-a-latency-growth-edit-and-publish-its-warm-successor-from-the-control-p.md +++ b/.github/ISSUE_SPECS/1403-classify-a-latency-growth-edit-and-publish-its-warm-successor-from-the-control-p.md @@ -246,3 +246,11 @@ no swap and the same host commands, timed on the source-read clock. - *Duck-swap a strip whose state cannot continue across a plan swap* (#1324): D4's duck set. - *Remove a strip in two phases: ramp out, then a scheduled swap* (#1325): gate 2's path. - *Publish an applied-revision watermark and complete edits asynchronously* (#1314). + +## Amendment (controls coordinator, batch 15a, root ruling 2026-10-15) + +#1403 adds the reprepare admission peak together with its first caller, and carries #1398's former gate 3 (a warm candidate is admitted against its reprepare peak). #1398 no longer ships `AdmissionPeak::WithReprepare`: every reference to it above, and to "#1398 gate 3", now means work this issue does. + +- The peak is the replacement admission counting each copy of a warm candidate's allocated source bytes plus its epoch and prepared-protocol capi rows, as #1398's attempt measured: a second check of the running plan (`held[0]`) beside the candidate twice, with the same model terms, passed alongside the held-plan check; both must fit. +- Code anchors verified in `crates/control-plane/src/compile.rs`: `validate_replacement_peak` (the replacement admission; its parameters are now `held`, `prospective`, `compiled_models`, `limits`, with no `AdmissionPeak`), whose one caller is `control.rs` (the structural command path, at the `#1398 D4` comment); `validate_live_peak` (the live arm, unchanged); `ProspectivePlan`, `CarriedSourceBytes`, `CapiResources` (the candidate's rows, including `capi.epoch_retained` and `capi.prepared_protocol_retained`); `compiled_model_admission` and `compiled_model_admission_over` (the model terms). The unit-test module is `compile::admission_tests` (helpers `epoch`, `candidate`, `peaks`, `assert_each_cap`, `CAPS`). +- Gate 1's "`WithReprepare`" bullet and the Dependencies line on `AdmissionPeak::WithReprepare` read as this amendment: the issue adds the parameter or equivalent shape for the peak, passes it at its first caller, and adds the gate 3 test (`R + 2C - 1` refuses with each cap's diagnostic, `R + 2C` passes, `R + C` passes without the peak; with a withdrawn candidate also held, both peaks are checked and the larger decides). diff --git a/.github/ISSUE_SPECS/1470-cache-design-bounds-across-browser-preparations-in-the-control-worker.md b/.github/ISSUE_SPECS/1470-cache-design-bounds-across-browser-preparations-in-the-control-worker.md deleted file mode 100644 index f126c3c30..000000000 --- a/.github/ISSUE_SPECS/1470-cache-design-bounds-across-browser-preparations-in-the-control-worker.md +++ /dev/null @@ -1,149 +0,0 @@ -# Cache design bounds across browser preparations in the control Worker - -Stream G of decision 15 (`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`, D15-4(b)). -Filed 2026-10-07 by the decision-15 root coordinator's split of *Cache design bounds across -preparations within a stated preparation budget* (#1457, Amendment 1), and aligned on 2026-10-07 -with #1457's Amendment 2 (root rulings: the cache's owners, one entry point, the budget figure, -and the C ABI successor #1471). Ordered after #1457 in stream G and after stream H's *Run the browser control plane in a Worker and keep the AudioWorklet -render-only* (#1332). Verify every code anchor on the branch where both have landed before starting. - -## Product outcome - -A browser rebuild of an unchanged session computes no input-section design bound: the browser's -long-lived control Worker keeps the same design-bound cache that #1457 gives each C ABI engine, so -an audio-only browser boot or rebuild costs no more than the pre-#1329 boot once its designs are -cached. The browser follows the same ownership as the C ABI (#1457 Amendment 2, ruling (b)). - -## Context - -- **What a design bound is.** `builtins::input_section_bounds(rate, strips, cache)` computes, once per - distinct design, the certified tail and exact-rest bound of a strip's input section (trim, - polarity, HPF, LPF), keyed by `InputBoundKey` (the rate, both channels' section words and the trim - magnitudes, `crates/builtins/src/tail.rs`). A strip with a live input lane reports the rate's - live bound instead. The cost is `O(T)` frames walked per design: about 0.2-0.3 ms for typical - designs and 7-8.8 ms for a design near the top of the cutoff domain (x86-64-v3, release). -- **What #1457 ships** (`crates/builtins/src/tail.rs`, `crates/builtins/src/lib.rs`; both hosts - unless noted). - - The per-preparation budget `INPUT_BOUND_BUDGET_FRAMES` = **1,510,000 frames walked**, charged - in strip order, in frame-equivalents of 18.0 ns (the largest median work of five measured - samples per charged frame-equivalent over every calibration batch and gate-2 design, 17.138 ns - by gate 2 against the calibration's 16.909 ns on the figure of record, the stream G batch 3 - verifier's run, times root's measurement margin of 1.05, rounded up to a half nanosecond; - root's second ruling, ruling (c) and margin ruling of 2026-10-08; the margin covers the whole - charged work, the section charge included, so every design's median work is at most its - charge x 18.0 ns / 1.05), so 27.18 ms (#1457 - Amendment 4, restated by #1474 and #1465). The stated worst case is **25.879 ms** of - design-bound work (median of five) measured on the CI-class runner (4,096 cheap two-section - designs, 1 kHz into 1.28 kHz, +24 dB, 88.2 kHz; measured at 18.0 ns and charge 550 by the - stream G batch 3 verifier, the figure of record), 95.2 % of 27.18 ms; medians over the - families and rates: the cheap families 17.81-25.88 ms, typical 11.11-23.00 ms, near-top - 20.65-21.11 ms, the band families 19.91-20.09 ms, the single top designs 3.74-7.36 ms (#1465's - stream G batch 3 follow-up record; the batch verdict states the largest raw sample). Past the - budget a design reports its rate's live bound, which is certified for every history and is - never `Infinite`. A design served from a cache charges the same stored amount it charged when - computed, so every reported bound is a pure function of the session, with or without a cache, - cold or warm. - - The live bound as a table for the four launch rates, checked by a test against - `input_section_live_bound`, so no preparation computes it. - - The cache type `InputBoundCache` (keyed by `InputBoundKey`, which carries the rate), with the - entry cap `INPUT_BOUND_CACHE_ENTRIES` = 8,192 and clear-on-full: the cap holds a whole - preparation's designs, so a rebuild of an unchanged session is served entirely from it while - the cache has not been cleared since that session's designs were inserted (#1457 Amendment 3, - m3: a cache that several sessions fill can clear in the middle of a preparation). A full cache - holds about 2.4 MiB (2,486,520 bytes measured at 8,192 entries). Render never reads it. - - *(#1457 Amendments 3 and 4.)* The budget is in frame-equivalents: each design computed - charges the frames it walks plus `INPUT_BOUND_SECTION_CHARGE` (580 frame-equivalents: - 1.05 x 9.79 us / 18.0 ns = 571.1, rounded up to a ten; the coordinator's whole-work margin fix - of 2026-10-08 on the figure of record, #1465) per section, so a computed design charges at - least 257 + 580 = 837 and one preparation computes at most 1,804 designs. Amendment 4 removed - the per-design charge. - - **One entry point, no parallel API** (Amendment 2, ruling (i)): - `builtins_compiler::prepare_session_builtins_with_live_controls` takes - `bound_cache: Option<&mut InputBoundCache>`. In `host-core` the parameter is internal to - `prepare.rs`'s policy function (`prepare_host_runtime_with_live_controls_policy_and_spectrum`); - every public host-core entry point passes `None`. There is no `_and_bound_cache` variant at - any layer. -- **The C ABI is #1471** (*Wire the engine and session design-bound caches into the C ABI*, after - #1457 and stream B's #1309). Ruling (b): the engine's cache serves - `miso_engine_v1_compile_session`, and each session owns its own cache, seeded at compile with a - copy of the engine's cache after that compile, for its rebuilds. No mutable state is shared - between an engine and its sessions. #1471 adds the parameter once to the two host-core entry - points the C ABI uses (`prepare_host_runtime_with_live_lanes` and - `prepare_host_runtime_with_live_lanes_successor`). -- **Why the browser was split off.** The AudioWorklet creates a new `WebAssembly.Instance` per - processor (`hosts/host-web/web/miso-engine-v1-audio-worklet.js`, instance creation in the - processor constructor), so no browser object lives long enough to own a cache across - preparations today. #1332 moves browser preparation into a long-lived control Worker; that Worker - is the natural owner. -- **The baseline to beat** (#1457, root ruling R2; `scripts/web-mixing-automation-benchmark.mjs - rebuild-round`, Node `--no-liftoff`, `taskset -c 7`, 25 boots per document, p50 ms, two rounds, - audio-only rows with `COMMAND_QUEUE_RECORDS = 0`, the SDK default): pre-#1329 9-track EQ - 2.01 / 2.00, 64-track console 19.13 / 19.13; #1329 attempt 5 9-track 2.24 / 2.24, 64-track - console 34.66 / 34.68. Re-measure the baseline on the branch where #1332 has landed, because - the Worker changes the boot path. - -## Decisions - -Frozen by #1457 Amendment 2 (root, 2026-10-07): - -- **D0. Ownership (ruling (b)).** The Worker's long-lived object that compiles browser sessions owns - one `InputBoundCache`, and a compile reads and fills it. Each browser session owns its own - `InputBoundCache`, seeded at compile with a copy of the compile cache after that compile, and - every rebuild of that session reads and fills it. No mutable cache state is shared between the - compiling object and its sessions, or between sessions. Both caches live on the control Worker - only; render, the AudioWorklet and render-owned memory carry none of it. -- **D0a. One entry point (ruling (i)).** `bound_cache: Option<&mut InputBoundCache>` is added once to - each host-core entry point the Worker's preparation calls (on today's tree the browser host calls - `prepare_host_runtime_with_selected_meters_between_render_calls` and - `prepare_host_runtime_with_live_controls_and_spectrum`; re-read them where #1332 has landed), - passed through to the policy function; every other caller passes `None`. No `_and_bound_cache` - variant. If #1471 has already added the parameter to an entry point this slice needs, it is used - as is. - *(#1457 Amendment 3, attempt-1 verdict n5.)* On #1457's tree the policy function passes - `bound_cache` to `builtins-compiler` only on its default branch: it drops the cache on the - selected-meters branch (`prepare_selected_session_builtins_between_render_calls`) and on the - between-render-calls branch (`prepare_session_builtins_between_render_calls`). The browser - prepares through `prepare_host_runtime_with_selected_meters_between_render_calls`, which takes - one of those branches, so this slice must carry the cache through the branch the Worker's - preparation takes (the two `builtins-compiler` functions of those branches gain the parameter, - or route through `prepare_session_builtins_with_live_controls`), and gate 1 is red until it - does. -- **D0b. Budget and cap.** The budget is #1457's `INPUT_BOUND_BUDGET_FRAMES` (1,510,000 frames) and - the cap and eviction are `INPUT_BOUND_CACHE_ENTRIES` (8,192) with clear-on-full, unchanged. - -To decide in this slice (root approves before implementation): - -- **D1. Which Worker objects own the two caches** under D0, given #1332's Worker shape, and the - compile cache's lifetime across session replacements. -- **D2. A browser-specific cap or eviction** only if a measured reason differs from D0b (for - example the Worker's memory budget); otherwise none. - -## Objective gates - -1. A browser rebuild of an unchanged session in the Worker computes no design bound (counted with - `builtins`' test-support design-bound counter). -2. The audio-only rebuild-cost proxy (`rebuild-round` with `COMMAND_QUEUE_RECORDS = 0`) is recorded - on the 9-track and 64-track documents beside the baseline above; a rebuild of an unchanged - session adds nothing over the pre-#1329 boot. -3. Every reported bound is bit-identical with and without the cache, cold and warm, and the budget - fallback is taken exactly when `INPUT_BOUND_BUDGET_FRAMES` is exhausted, as below the C ABI - (#1457's gates stay green). -4. Render never reads the cache; the worklet chain and the browser memory pins move only where this - slice's retained cache is charged, each re-pin stated with its reason. - -## Non-goals - -- The budget, the live-bound table and the cache type (#1457). The C ABI caches (#1471). A - parallel `_and_bound_cache` entry point. Tightening any bound. Any render-side change. - -## Dependencies - -- #1457 (stream G), #1332 (stream H). Sibling: #1471 (the C ABI caches); if both add the - parameter to the same host-core entry point, the later slice rebases. - -## Attempt record - -- 2026-10-07 (stream G2 batch follow-ups, root order): this local spec was aligned with #1457 - Amendment 2 (ownership ruling (b), one entry point, the 1,510,000-frame budget constant, the - cache cap, the C ABI successor #1471). **The GitHub issue body is a pending sync**: it still - states the Amendment 1 text and was not edited here; root syncs it. diff --git a/.github/ISSUE_SPECS/1482-attribute-each-claim-s-revisions-exactly-in-the-watermark-and-share-one-seqlock.md b/.github/ISSUE_SPECS/1482-attribute-each-claim-s-revisions-exactly-in-the-watermark-and-share-one-seqlock.md index 3132dd57e..928d70ff4 100644 --- a/.github/ISSUE_SPECS/1482-attribute-each-claim-s-revisions-exactly-in-the-watermark-and-share-one-seqlock.md +++ b/.github/ISSUE_SPECS/1482-attribute-each-claim-s-revisions-exactly-in-the-watermark-and-share-one-seqlock.md @@ -120,3 +120,9 @@ seqlock is still guarded by both modules' existing torn-read tests. - A test that greps source or prose is refused. - Attempt budget: three attempts, one adversarial verdict each. - Size: under half a day. + +## Amendment (controls coordinator, root rulings, 2026-10-15) + +- Root ruling B15-R1 (2026-10-15, recorded in #1316): #1316 lands first and creates the shared + `crates/engine/src/realtime/seqlock.rs`. This slice's D3 then moves `observe.rs` onto that module + and does not create a second one; `seqlock.rs` is no longer new here. diff --git a/.github/ISSUE_SPECS/1522-find-and-fix-the-intermittent-sdk-spectrum-gate-failures-in-firefox-and-webkit.md b/.github/ISSUE_SPECS/1522-find-and-fix-the-intermittent-sdk-spectrum-gate-failures-in-firefox-and-webkit.md index b4e315af0..0afa5d95a 100644 --- a/.github/ISSUE_SPECS/1522-find-and-fix-the-intermittent-sdk-spectrum-gate-failures-in-firefox-and-webkit.md +++ b/.github/ISSUE_SPECS/1522-find-and-fix-the-intermittent-sdk-spectrum-gate-failures-in-firefox-and-webkit.md @@ -41,7 +41,7 @@ only after it. - The probes are in `hosts/host-web/qualification/sdk-response-entry.ts`: the collection probe returns `firstCapturedSample` at `:1259`, and `runConfiguredSpectrumHopQualification` starts at `:874`. -- #1106 (`.github/ISSUE_SPECS/1106-spectrum-hop-probe-waits-on-the-clock.md`) covers a different +- #1106 covers a different symptom of the hop probe: `H1024 browser probe published 1 windows`, seen in Chromium, from a poll count that ends before the deadline. It does not explain a failure of `sdk-spectrum-hop` that has no such message, and it does not cover the collection gate. #1248 covers the continuous diff --git a/.github/ISSUE_SPECS/1562-admit-browser-live-edits-in-the-worker-through-the-committed-model.md b/.github/ISSUE_SPECS/1562-admit-browser-live-edits-in-the-worker-through-the-committed-model.md deleted file mode 100644 index 7797406c3..000000000 --- a/.github/ISSUE_SPECS/1562-admit-browser-live-edits-in-the-worker-through-the-committed-model.md +++ /dev/null @@ -1,714 +0,0 @@ -# Admit browser live edits in the Worker through the committed model - -Slice S10a of decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`, design points C1, -C4, C5 and C8). This is #1382, amended in place by decision 17; stream H of decision 15 (D15-1, -D15-2, D15-10, D15-11, D15-17). Code anchors verified on `main` `df585e503`. Feature sprint mode -(owner, 2026-10-13): one implementer, normal tests, a batch review; no per-attempt verifier or -mutation runs. - -Decision 17 splits the browser half of the program in three: this slice (the Worker commits -`SET_CONTROL` and serves the control list through the core), #1563 (the SDK sends every value as -`SET_CONTROL`), and #1565 (host-web's own command path is deleted). They land in that order. - -## Product outcome - -In the browser, a control edit is a `SET_CONTROL` in a `SESSION_TRANSACTION_APPLY` frame. The -Worker hands the frame, unchanged, to the same `control_plane::SessionState::command` that the -C ABI's `miso_engine_v1_submit_command` calls. A live edit advances the committed revision, its -reply carries `{revision, path}`, and render applies it from the plan's live lanes. The control -list and the control state come back through the same entry, as the same bytes as on the C ABI. -Solo is the core's monitor overlay. Nothing in the browser lowers, checks or composes a value. - -## Context - -- **Admission runs on the worklet thread today.** `miso_engine_web_v1_command_submit` - (`hosts/host-web/src/ffi.rs:4015`) calls `AudioWorkletEngineHost::submit_commands` - (`hosts/host-web/src/lib.rs:3477`), then `admit_commands` (`:4798`) and `admit_commands_staged` - (`:5051`): host-web's own 17 kinds (`:837-949`) and its own solo, VCA and follow composition - (after #1548, through the core's check and building blocks, over host-core's mirrors). It writes - plan cells and never updates a session model. The static gate holds that export to the - allocation-only call graph (`scripts/check-web-audioworklet.sh:494-511`). -- **The C ABI's one entry.** `miso_engine_v1_submit_command` (`crates/capi/src/ffi.rs:685`) runs - `SessionState::command` (`crates/control-plane/src/control.rs:1082`): it decodes any command - frame, applies a transaction live or as a rebuild, and serves every query. After #1551, #1552, #1553, - #1554, #1556, #1557, #1558 and #1559 it serves `CONTROL_LIST_GET`, `CONTROL_STATE_GET` and - `SET_CONTROL`, with solo in its monitor overlay. -- **The Worker holds that state.** *Swap and retire browser plans through the Worker's service - loop* (#1381, D1) boots `SessionState` in the Worker through the browser's `WebRuntimePreparer` - (#1400 D5). *Replace the running browser session in the Rust host* (#1290) makes its rebuild arm - work in the browser. *Run the browser control plane in a Worker and keep the AudioWorklet - render-only* (#1332) sets the `worker` and `single` modes and the page-to-control-plane port - (its D7). -- **#1382 as filed** lowered the 48-byte records into a transaction in the Worker (its D2) and - composed solo in the shared commit (its D3). Decision 17 drops the lowering (thin SDK builders, - #1057 section 3.2 option A) and moves the overlay to #1559. -- **#1293 plans a narrower export.** *Export transaction apply and anchored seek from the browser - engine module* (#1293, D2-D3) plans `miso_engine_web_v1_apply` for transaction frames only and - a fixed outcome record. Ruling C4 needs the list and state queries on the same path, so this - slice adds one generic entry (D1). See the open question for root on #1293. -- **Live lanes.** The browser plan's live lanes come from the boot word - `live_control_command_queue_records` (`hosts/host-web/src/lib.rs:1251`). Ruling C6 ("live lanes - on every plan") makes the core choose them; #1555 adds `controls::live_lanes(model)` and keeps the browser on its boot word (#1555 D4). -- **The ramp table.** `SessionRamps` (`hosts/host-web/src/lib.rs:1549`) is built once at boot; - root ruling of 2026-10-12 (#1382 D6 as filed) says the browser must resolve an absent ramp from - the committed model. - -## Decisions frozen for this slice - -- **D1. One entry for every command frame.** - - New exports, all on the control half (the Worker in `worker` mode, the worklet's message - handler in `single` mode): `miso_engine_web_v1_control_request_ptr(handle, len) -> u32` (a - staging buffer for one frame of `len` bytes, at most the boot's - `maximum_control_frame_bytes`; `0` with a typed result otherwise), - `miso_engine_web_v1_control_command(handle, len) -> u32` (runs - `SessionState::command(staged bytes, capacity)` and returns `RESULT_OK` or the mapped - `CommandError`), `miso_engine_web_v1_control_response_ptr(handle) -> u32` and - `miso_engine_web_v1_control_response_bytes(handle) -> u32` (the response frame, valid until - the next call). On an instance with no control half each refuses with - `RESULT_REFUSED_LIFECYCLE` (a pointer export returns 0). - - The page-to-control-plane port (#1332 D7) gains one internal message, `control`, that carries - the frame bytes and a message ID; its reply carries the response frame bytes unchanged and the - export's result word. The message is internal to the SDK (D15-11). - - The browser decodes nothing in the frame. Any frame `SessionState::command` accepts is - accepted: transactions, `CONTROL_LIST_GET`, `CONTROL_STATE_GET`, `SESSION_SNAPSHOT_GET`, the - counter and capability queries. -- **D2. Nothing is lowered** (replaces #1382 D2 as filed). The Worker passes the - `SESSION_TRANSACTION_APPLY` frame to `control-plane`. The ramp of each edit is the edit's own - `ramp_samples` field, resolved by the core (#1394 D5). -- **D3. The overlay is the core's** (replaces #1382 D3 as filed). Solo is a `SET_CONTROL` on strip - own row 2, in #1559's overlay inside the Worker's `SessionState`. Observe subscribe and - unsubscribe (kinds 7 and 8) and the meter lease stay browser monitor operations (ruling C9); the - later browser-feature phase moves them into the core. -- **D4. The acked-batch rule** is the core's (#1559 D8): every fallible check of the whole frame - runs before the first write; a refusal leaves model, revision, overlay and cells unchanged; live - values are latest-target cells, never refused for room. -- **D5. The browser preparer takes the core's live lanes.** `WebRuntimePreparer` attaches - `controls::live_lanes(model)` on every plan, as the C ABI does. The boot word no longer chooses - a lane; until #1565 it only sizes the old command staging. -- **D6. The ramp table follows the model** (keeps #1382 D6 as filed, root 2026-10-12). Every - `SET_CONTROL` resolves its ramp against the committed model inside the core. - `SessionRamps` serves only the old command path, and #1565 deletes it with that path. -- **D7. The old path stays, unchanged, until #1565.** Kinds 1-6 and 9-17 keep working on the - worklet in this slice, because the SDK still sends them until #1563. A value set through them - does not reach the committed model; no SDK path sends both forms (the SDK's transaction API, - #1296, lands after #1563). This order is forced by the SDK dependency; it is not a placeholder. -- **D8. Mirrors.** The new exports join every export list: the ABI layout document and its - generator (`tools/parameter-metadata/src/abi_layout.rs`), `sdk/src/generated/abi.ts` and - `sdk/assets/miso-engine-v1-abi-layout.json` (regenerated, never hand-edited), - `scripts/check-abi-layout-v1.py` and its self-test fixture, - `tools/realtime-policy/render-thread-exports.json` (they are control-half exports, not render - exports), and the shipped host's `.d.ts`. - -## Deliverables - -1. D1 in `hosts/host-web/src/{ffi.rs,lib.rs}`, the control Worker script #1332 adds and - `hosts/host-web/web/miso-engine-v1-audio-worklet.js` (single mode). -2. D5 in host-web's preparer. -3. D8's mirrors. -4. The native tests, the integration test binary `hosts/host-web/tests/worker_admission_realtime.rs` - (gate 5) and the extended parity test (gate 8). - -## Authorized paths - -- `hosts/host-web/src/`, `hosts/host-web/web/`, `hosts/host-web/tests/`, - `hosts/host-web/qualification/` (the gate rows of gate 7). -- `tools/parameter-metadata/src/abi_layout.rs`, `tools/parameter-metadata/tests/abi_layout.rs`, - `scripts/check-abi-layout-v1.py`, `scripts/fixtures/abi-layout-v1-self-test.json`, - `tools/realtime-policy/render-thread-exports.json`. -- Regenerated only: `sdk/assets/miso-engine-v1-abi-layout.json`, `sdk/src/generated/abi.ts`. -- `sdk/src/browser/shipped-host.d.ts` (the new message and exports only). -- `scripts/test-web-audioworklet.mjs` (the fake exports and the `control` message). - -## Non-goals - -- No SDK API change (#1563, #1564). No deletion of the old command path (#1565). -- No change to any classifier row, family or record (the core owns them). -- No observation or meter-lease change (ruling C9). - -## Objective gates - -1. **Every live row commits to the model.** Native host-web test, control half and render half on - two threads. One transaction per row of every live family, through D1: strip fader, mute, pan, - matrix, `matrix_mode`, trim, polarity, HPF, LPF, one block-rate effect parameter, one EQ band - gain, one bypass, send gain, mute and matrix, `follows_mute`, VCA offset and mute, a VCA member - added, on tracks and on a submix. For each: the reply's path is `live`, the revision advances - by 1, the plan epoch is unchanged, the committed document (`SESSION_SNAPSHOT_GET`) holds the - value, and after the ramp every block is bit-identical to a plan prepared from the committed - model with the same overlay, fed the same PCM from frame 0. A `SET_CONTROL` on `delay_samples` - is path `rebuild` and renders as the fresh plan after adoption. -2. **Ramps carry end to end.** For fader, mute, pan and solo: `ramp_samples` 0 steps at the next - block; 480 reaches the target after 480 samples; absent ramps over the session's - `control_smoothing` length (solo: `mute_ms`). -3. **The overlay.** Solo replies `monitor` with the revision unchanged; soloing track 2 mutes the - others and closes a following send; a structural transaction keeps the solo. -4. **All or nothing.** A transaction of a valid fader edit, a solo and an out-of-domain pan is - refused with `control.domain`; model, revision, overlay and cells are unchanged; the next 8 - blocks are bit-identical to a run without the call. -5. **Realtime.** Integration test binary `hosts/host-web/tests/worker_admission_realtime.rs`. It - links `bench_support::alloc` and calls `assert_installed()` first. The render thread's - thread-scoped counters read `allocations == 0 && frees == 0` around every render call while the - Worker thread runs 1,000 `SET_CONTROL` transactions. In the browser, - `miso_engine_web_v1_render_allocation_count` stays 0 on both legs (#1333), and on the - non-isolated leg `singleModeControlAllocations` grows with the control messages. -6. **The ramp table follows the model** (#1382 gate 9 as filed). Leg 1: commit `SET_CONTROL` - session `mute_ms` 5, then a mute with no `ramp_samples`: it ramps over 240 samples at 48 kHz. - Leg 2: commit `SET_CONTROL` on a strip's `pan_window_samples` 100, then a pan with no - `ramp_samples`: it ramps over 100 samples. -7. **Browser qualification.** The live-control rows of the browser qualification pass unchanged in - chromium, firefox and webkit, on both legs (the SDK still uses the old path), and one new row - sends a `control` message with a `CONTROL_LIST_GET` frame and checks the decoded first row. -8. **Cross-host parity, extended** (`hosts/host-web/tests/control_parity.rs`, #1560). The browser - leg sends `SET_CONTROL` through D1 instead of command records. The script gains the edits that - are now expressible on both hosts: `follows_mute` off and on, a VCA member added, `matrix_mode` - to matrix and back. After the script, every page of `CONTROL_LIST_GET` and the - `SESSION_SNAPSHOT_GET` bytes are byte-identical across the two hosts. -9. **The C ABI is unchanged.** `cargo test --locked -p capi` passes, and `target/release/audit capi` - reports the merge base's `pcm_digest` (PR evidence). -10. **Commands:** `cargo test --locked -p host-web --features host-web/test-support` (runs gates - 1-6 and 8), `cargo test --locked -p control-plane --features test-support`, - `cargo test --locked -p capi`, `cargo build --locked --release -p audit -p capi && - target/release/audit capi`, `bash scripts/check-web-audioworklet.sh`, - `bash scripts/test-web-audioworklet.sh`, `python3 -B scripts/check-abi-layout-v1.py`, - `bash scripts/check-sdk-generated.sh target/ci/qualification-artifacts`, - `bash scripts/check-sdk-types.sh`, and `npm run qualify -- --artifacts ... --sdk-root ... - --browser --check-matrix --self-test-mutations` in `hosts/host-web/qualification` for - chromium, firefox and webkit. - -## Test value - -- Gate 1 turns red if a browser edit reaches the cells without reaching the committed model, if a - family lowers differently in the Worker, or if a rebuild-only row is live in the browser. -- Gate 2 turns red if the browser drops a per-edit ramp or reads absent as a length. -- Gate 3 turns red if the browser's solo advances the revision or is lost at a swap. -- Gate 4 turns red if any write precedes a later refusal in the Worker. -- Gate 5 turns red if control work runs on, or allocates on, the render thread, or if single - mode's control allocations go uncounted. -- Gate 6 turns red if any browser path resolves an absent ramp from a table built at boot. -- Gate 7 turns red if the shipped module's `control` message does not reach the core. -- Gate 8 turns red if the two hosts list different controls, hold different committed documents, - or render differently for an edit that only the core path makes expressible. -- Gate 9 turns red if the browser work changes C ABI behaviour. - -## Dependencies - -- #1551, #1552, #1553, #1554, #1555, #1556, #1557, #1558, #1559 (the core serves every row and chooses the - live lanes) and #1560 (the parity test). -- *Swap and retire browser plans through the Worker's service loop* (#1381). -- *Replace the running browser session in the Rust host* (#1290): the rebuild arm in the browser. - Its own dependency on #1562 (formerly #1382) is a question for root (see the report). -- *Run the browser control plane in a Worker and keep the AudioWorklet render-only* (#1332). -- *Report each transaction's edit path in its response* (#1313). -- *Prepare through an adapter-supplied preparer in the control-plane crate* (#1400). - -## Supersedes and shrinks - -- Amends itself (#1382, *Admit browser live edits in the Worker through the committed model*): see - the amendment record. -- Supersedes *Make a send's follows_mute live in the browser* (#1342): gate 1 makes the toggle live - in the browser through the core. - -## Amendment (decision 17, root 2026-10-13) - -**Single instance first.** This slice targets `single` mode on today's one-instance module. -The four `control_*` exports run in the worklet's message handler, never in `process()`. The -`control` message travels on the node's existing port. #1332 later deploys the same exports in the -Worker, with no other logic. Dependencies: #1332 is removed; #1381 means #1381a; #1290 moves to -#1562b. **D7 is withdrawn**: the old command path is deleted in the same batch (#1565A), so it -never runs beside `SessionState`. - -The slice is split. Both pieces stay in this issue. - -- **#1562a**: D1-D6, D8, and D9 below. - - **D9. Rebuild pending (root ruling B).** Until #1562b, the browser preparer's successor entry - that the command path uses refuses a frame that classifies as a rebuild with - `web.apply.rebuild_pending`. The refusal comes before commit: no ack precedes it, nothing is - queued, and the model, revision and plan are unchanged. The browser has no in-place structural - edit today, so this withdraws nothing. Test-support republish (#1381a's gates) keeps working. - - Gates: 1 (the live rows), 2-6, 8 (all but its rebuild part) and 9. Gate 5's browser half and - gate 7 run on the non-isolated leg only. - - Gate 7 reads: "The live-control rows of the browser qualification pass through `SET_CONTROL` - in chromium, firefox and webkit", plus the `CONTROL_LIST_GET` row. - - New gate 11: a `SET_CONTROL` on `delay_samples` is refused with `web.apply.rebuild_pending`; - the model, revision, plan epoch and cells are unchanged, and the next 8 blocks are - bit-identical to a run without the call. It turns red if a rebuild-path frame reaches - `SessionState`'s rebuild arm (a plan swap with no companion pairing) before #1562b. - - Dependencies: as filed, with #1381a for #1381, and without #1332 and #1290. -- **#1562b**: gate 1's `delay_samples` row, the rebuild part of gate 8, and the deletion of D9's - refusal and gate 11. Dependencies: #1562a, #1381b, #1290a. - -**Flip batch 6W (root ruling A1).** #1381a, #1562a, #1570b, #1563B and #1565A land together, in one batch of lane W. Each slice keeps its own commits and tests and is green at its own boundary; the batch runs the full CI-equivalent gate set once. If one slice fails, the batch stops: `main` never gets a partial flip, so it never has two writers of one plan's revision gate and cells. No transitional API lends `SessionState`'s producers or revision to the old command path. - -## Record of #1562a (flip batch, written 2026-10-14) - -### What shipped - -- **D1.** Five exports on the control half, all unwrapped and counted as control allocations: - `miso_engine_web_v1_control_request_ptr(handle, len)`, `_control_command(handle, len)`, - `_control_response_ptr(handle)`, `_control_response_bytes(handle)` and - `_control_revision(handle) -> u64` (the fifth is carry-forward 2). `control_command` runs - `SessionState::command` on the staged bytes and decodes nothing. The one `miso.control.v1` - message (`{requestId, frame}`, reply `{requestId, result, response, revision}`) travels on the - node's port; the worklet stages, runs and copies the response out in `receiveControl` / - `readControlCommand`, in the message handler, and the service step follows as after every message. - The main-realm host has `control(frame)`. The staging is one new 16 KiB buffer - (`PreparedBuffers::control_request`, charged in the bridge's retained rows), not the old command - staging: that one holds 256 records of 48 bytes, 12 KiB, under `maximum_control_frame_bytes`. -- **Result codes.** One new browser result code, `RESULT_COMPILE_REJECTED = 10` - (`compileRejected`), for a typed refusal before commit; the other `CommandError`s map to the - existing codes. After any failure the response bytes are the refusal's diagnostic text. A refused - transaction that the core encodes (`control.domain`) is still `RESULT_OK` with a non-ok frame, as on - the C ABI. The mirrors (`abi_layout.rs` and its test, `check-abi-layout-v1.py`, its fixture, the - regenerated JSON and `abi.ts`, `render-thread-exports.json`, `policy.rs`, both `.d.ts`, - `check-web-audioworklet.sh`) carry the exports and the code. -- **D9.** `RuntimePreparer::refuse_rebuild(&self) -> Option<&'static str>` (default `None`; the - C ABI's preparer takes every rebuild). `SessionState::command` asks it between `commit_live`'s - rebuild verdict and the rebuild arm (`control.rs`), drops the token and returns - `CompileRejected(web.apply.rebuild_pending)`. `WebRuntimePreparer` refuses unless its - `allow_rebuild` switch is set, and only the test-support republish - (`structural_command_for_test`, #1381a gates 2 and 4) sets it, for the one call. Nothing is - stamped, prepared, queued or acknowledged before the refusal. -- **D5, D6.** Nothing to do: #1381a's preparer already attaches `controls::live_lanes`, and the - core resolves every ramp from the committed model. - -### Rules for the next slices - -- **One request-ID allocator.** The frame carries the sender's request ID and the core's replay - cache refuses a reused ID with different bytes; it answers a byte-identical resend - `protocol.failure` (a refused frame consumes its ID, the rebuild-pending refusal included). So one - allocator serves the whole `SessionState::command` endpoint. For #1563B: the SDK's - `ControlTransactionQueue` (`#nextRequestId`) is that allocator, and every catalog or state read - the SDK sends through `host.control()` takes its ID from the same counter; the host method never - rewrites a frame; a retry of a refused frame takes a new ID. -- **Revision seed.** `control_revision()` (and the `revision` of every `miso.control.v1` reply, - and the header of any reply frame) is the committed revision. #1563B constructs the queue with - `initialRevision` from it after a successful boot. -- **edit_path.** It is in the response frame (`TransactionApplied.edit_path`), unchanged: the - parity test and the admission tests decode `live`, `model_only`, `monitor` from it. `rebuild` - never comes back from the browser before #1562b. -- **Reliable event lane (for #1570b).** Every committed transaction queues a `SESSION_COMMITTED` - event on the core's reliable lane, and an undrained lane refuses the next commit with - backpressure. #1562a has no reader: the tests drain through the test-support - `drain_reliable_events_for_test`. Until #1570b drains the lane in the message handler, a page that - sends more transactions than the lane holds is refused; the batch lands whole, so this is - #1570b's to close before the flip is reachable. -- **Memory growth.** `control_command` allocates (the transaction, the classifier's model clone and - JSON, the catalog stage), and on a small session the first live transaction grows the module's - memory (reproduced: 46 to 48 pages). Growth is not a fault (#1332 D5, root ruling A of the batch 7 - CI round, which replaces this note's earlier "a moved `memory.buffer` is sticky - (`reprepareRequired`)"): the worklet re-binds and continues. `rebindMemoryViews` re-cuts every - persistent view over the memory at the end of every message-handler path that can call into the - module (control, service, event drain, observation, track response, spectrum, source submit and - seek, and the feed's attach and prepare-seek handler), and `process()` never creates a view: it - compares the buffers, and on a mismatch renders silence for that quantum and counts it in the - status's `staleViewQuanta`. Growth stays counted on the control side - (`singleModeControlAllocations`); the render-locked allocation count stays exactly 0. A refused - or failed call keeps its previous behaviour (a typed refusal is a reply, a trap or malformed - message is sticky); only the growth stopped being sticky. The host accepts a status whose - `memoryBytes` is the boot size or a larger whole number of pages. - -### Evidence - -- Gate 1 (`tests/control_admission.rs`): 22 rows, each its own boot, one transaction. The reply is - `live`, the revision is +1, the plan epoch is unchanged, the parsed committed document holds the - value on each lane, and the blocks are bit-identical to a plan prepared from the committed - document and fed the same PCM from frame 0 for 18 rows. Deviations from the filed text: - - `hpf`, `lpf`, the effect parameter and the EQ band gain are not bit-identical to a fresh plan at - any block: the running plan smooths the coefficient through filter or effect state, so its - output converges on the fresh plan's (the test asserts the largest difference at the end is - under half that at block 8; measured 7e-4 for `hpf` and 1.3e-2 for `lpf` at block 38 of 40). - Gate 8 holds the same four rows bit-equal to the C ABI at four rates, which is the claim that - can hold. - - `follows_mute` and the VCA member are rows only in a state where they change a gate or a gain, - so each has a setup transaction (mute `t1`; a VCA offset of -3 dB). Otherwise the core answers - `model_only`. - - The VCA member row compares from block 20 (the edit has no length of its own and ramps over the - session's fader length). - - The `delay_samples` part of gate 1 is #1562b's; gate 11 holds its refusal. -- Gate 2: sample-exact on a session with no insert: 0 steps at the first sample of the block - after the commit, 480 reaches the target at the 480th sample of the ramp (sample index 479), an - absent length equals the explicit session length and is not a step, for fader, mute and pan. - Solo is not in this table: its `mute_ms` resolution is the core's (#1559's tests), and the browser - adds no resolution of its own to test. -- Gate 3: solo replies `monitor` with the revision unchanged, the document is untouched, the blocks - equal a fresh plan under the same solo, and a later live fader edit keeps the solo. "A structural - transaction keeps the solo" is #1562b's (a structural transaction is refused here). -- Gate 4: a valid fader edit, a solo and an out-of-domain pan are refused `control.domain`; the - revision, document and the next 8 blocks equal a run without the call. -- Gate 5 (`tests/worker_admission_realtime.rs`, single-thread form; the workspace has no - `bench_support::alloc`, so the binary registers the module's own counting allocator as - `plan_exchange_threads.rs` does): 1,000 `SET_CONTROL` frames between 1,000 renders, the - render-locked count exactly zero across every render, the control count moving across the frames. - The browser halves (the counter in a real page) are gate 7's. The two-thread form is #1332's. -- Gate 6: `mute_ms` 5 then a mute with no length equals the mute with an explicit 240 and differs - from the default; a strip's `pan_window_samples` 100 then a pan with no length equals an explicit - 100 and differs from the default. -- Gate 8 (`tests/control_parity.rs`, un-ignored): the browser leg sends the C ABI leg's frames - through D1; the verdicts (path and revision, or the refusal's codes), every block's bits at 44.1, - 48, 88.2 and 96 kHz, the superseded count, every row of `CONTROL_LIST_GET` and `CONTROL_STATE_GET` - and the `SESSION_SNAPSHOT_GET` bytes are equal. The script gained `follows_mute` off and on, a VCA - member added, and `matrix_mode` of a matrix strip to pan (live) and back to the matrix (`model_only` - on both hosts, as the effective 2x2 is unchanged). The rows are compared one by one because the C ABI leg's 4 KiB frames - page differently. -- Gate 9: `cargo test -p capi` passes; `target/release/audit capi` reports `pcm_digest` - `76525e7ef9e78cf4`. -- Gate 11 (`a_rebuild_frame_is_refused_before_commit_and_changes_nothing`): a `SET_CONTROL` on - `delay_samples` returns `RESULT_COMPILE_REJECTED` with `web.apply.rebuild_pending\t$\n`; the - committed revision, the plan epoch and the document are unchanged, no commit event is queued, the - host takes the next live edit at the unchanged revision, and the next blocks equal a run that never - saw the frame. -- Worklet harness (`scripts/test-web-audioworklet.mjs`): the control message stages the frame, runs - the export, returns the response unchanged with the revision, carries a typed refusal without - going sticky, refuses a frame the staging refuses, goes sticky on a malformed message; a command - that grows the memory is re-bound and continues (every registered view is cut against the new - buffer, the reply is whole, the next `process()` renders the engine's non-silent output through - the new views, a later request is served, and the status reports the grown memory and a zero - `staleViewQuanta`); `process()` on a moved memory with no handler since renders silence, is not - sticky, counts the quantum, and resumes after the next handler; the host sends a copy with the buffer transferred, refuses a bad frame locally, and - fails on a malformed reply. -- `check-sdk-headless.sh`: 25 failing subtests, the number #1381a recorded; all of them send the old - command kinds or read their effects. #1563B (the SDK sends `SET_CONTROL`) restores the edit-driven - ones; #1565A deletes or rewrites the tests of the old kinds (`round_trip.rs` in - `tools/parameter-metadata` is one: it still sends kind 4 and fails on `unsupportedKind`, as it did - at #1381a's head). -- **Classifier cost (carry-forward 4).** `classify_live_delta_with_monitor` on a 64-track session - (one source, route and no insert per track, one fader edit), native release build, x86-64: about - 2.0 ms per call (200 calls); a whole `control_command` with a fader edit about 1.9 ms. It clones - the model and serializes both the committed and the masked model to canonical JSON for each edit, - so it is linear in the session, and the same cost lands in the worklet's message handler, on the - audio thread, in the one-instance mode: in Wasm, several times native, against a 2.7 ms quantum at - 48 kHz and 128 frames. It is slow enough to glitch a large session. A successor is opened: replace - the whole-document comparison of step 1 by a field-wise comparison of the masked model (the same - verdict, no JSON), then re-time. #1562a does not change the classifier (a non-goal). - -### Not done here - -- Browser qualification rows (gate 7) in three browsers: no local browser runs in sprint mode; the - rows and the `CONTROL_LIST_GET` row belong to the batch's qualification run. -- `npm run qualify` and `check-sdk-types.sh` (needs `npm ci`). - -## Amendment (batch 7 review, coordinator 2026-10-14): parity rows owed by #1562b - -The flip's parity test (`hosts/host-web/tests/control_parity.rs`) cannot yet compare edits that the -classifier sends to the rebuild arm, because #1562a refuses them in the browser with -`web.apply.rebuild_pending` while the C ABI rebuilds. #1562b, which deletes that refusal, adds these -rows to the parity script, each compared per step by reply bytes and per block by PCM bits: a route -into the output (gain and mute), the delay and multiband prepared bypass, `delay_samples`, and an -audio-rate effect parameter if a launch effect declares one. Until #1562b lands this is the one -known host difference, and it is a sequenced refusal, not a divergent result. - -## Amendment (controls coordinator, batch 8, 2026-10-14): for #1562b - -- **Meter strip order after a structural edit.** Since #1564C the SDK takes strip order from the - core's control list and re-reads it when the structure epoch moves, but the browser meter feed - keeps the strip IDs of the boot-time shape and `meterProjection` refuses a frame of another - shape. Nothing can move strips before #1562b, because the browser refuses rebuild-path frames. - #1562b's gates include a rebuild that adds or removes a strip, after which meters and observation - follow the new list order. -- **Watermark rebuild leg.** #1349's gate 1 ran with a live edit (rebuild refused). #1562b adds the - rebuild leg: a rebuild, then a live edit before the swap; the browser status words stay at the - initial revision until the swap block and then equal the C ABI's `miso_engine_v1_plan_watermark`. -- **Named effect rows (batch 8 review).** `sdk/src/core/live-controls.ts` (`#effectRowId`) resolves - an effect row's name through the session the SDK built, checked against the booted bytes. After - #1562b a rebuild can change an insert's effect, and a named edit could land on another effect's - row with the same number. #1562b resolves named effect rows from the control list's section - headers (the effect ID) and re-reads them when the structure epoch moves. -- **Meter frames and the structure epoch (batch 8 review).** `pollMeters` (`sdk/src/core/boundary.ts`) - re-reads the strip order only when a frame's `trackCount` differs, so after #1562b a structural - edit that keeps the count would name the wrong strips. #1562b carries the structure epoch in the - meter header (or an equal signal) and re-reads the strip order when it differs. - -## Amendment (controls coordinator, batch 10, 2026-10-14): #1562b - -Checked against `12d4a4f9d`. - -- **#1562b spans both lanes.** Lane W is the only lane that edits `hosts/host-web/**`; lane K owns - the crates and the `sdk/` files outside `hosts/`. Split, in this order: - - **#1562b-W:** delete D9 in the browser (`WebRuntimePreparer::refuse_rebuild`'s override, - `allow_rebuild`, `REBUILD_PENDING` at `hosts/host-web/src/lib.rs:4802`, gate 11 and its test), - gate 1's `delay_samples` row, gate 8's rebuild part with the batch 7 parity rows, the watermark - rebuild leg, and the meter header's structure epoch with its ABI layout rows and the - regenerated `sdk/src/generated` copies. - - **#1562b-K:** delete the then-unused hook `RuntimePreparer::refuse_rebuild` - (`crates/control-plane/src/compile.rs:676`) and its call in `SessionState::command`; the SDK's - named effect rows from the control list's section headers (`sdk/src/core/live-controls.ts`); - and `pollMeters`' re-read of the strip order on the structure epoch (`sdk/src/core/boundary.ts`). - It reads the header field that #1562b-W adds. -- **Dependencies of #1562b:** #1562a, #1381b, #1290 (a) and #1290 (d+e), so the browser's first - reachable rebuild keeps meters, observation taps and the spectrum stream continuous (D15-14); and - #1280a, #1280c, #1281 and #1282, so it does not restart every strip that holds an effect. A strip - in the restart set is not ducked until #1324 (open); the C ABI behaves the same today, so parity - holds. -- **A gap no chain slice closes.** A structural transaction that also changes a strip's live input - value (trim, polarity, a filter) restarts that strip's input section: the input join carries a - section only when every value is equal (`same_input_section`, `crates/host-core/src/prepare.rs:2726`), - and there is no input twin of #1277 D5's retarget. It is not a #1562b blocker (the C ABI does the - same). Proposed successor: *Carry, then retarget a strip input section across a plan swap* - (lane K, `prepare.rs` and `live_delta.rs`), after #1280c. - -## Ruling (root, 2026-10-14): #1562b order - -#1562b-W lands first, then #1562b-K, with the dependencies in the batch-10 amendment (#1290 d+e, -#1280a, #1280c, #1281, #1282). - -## Amendment (controls coordinator, batch 12, 2026-10-15): #1562b - -Checked against `ea336c039`. - -- **Dependencies at this head.** Landed: #1562a, #1381b, #1290 (a), #1280a, #1280c, #1281, #1282, - #1283, #1284, #1327a, #1327b and #1622. Still open: #1290 (d1) and (d2) (see #1290's batch 12 - amendment; (e) needs no code). Added: #1584 D6a and D6b. Without them, the first browser rebuild - that can be reached loses an acknowledged subscription on a restarted effect owner, and ends a - removed owner's subscription silently. #1630 (the input-section carry) is still not a blocker, - because the C ABI behaves the same. -- **Anchors moved.** `REBUILD_PENDING` is at `hosts/host-web/src/lib.rs:5260`. The - `WebRuntimePreparer` override is at `:5301`, `allow_rebuild` at `:5278`, and - `structural_command_for_test` at `:2107`. The core hook is still `compile.rs:676`, and it is - called at `crates/control-plane/src/control.rs:1260`. Gate 11's test is - `hosts/host-web/tests/control_admission.rs:476`. -- **#1562b-W, additions to the batch 10 list:** - - Delete `structural_command_for_test` and the `allow_rebuild` switch with D9. Their tests - (#1290a, #1381b, #1584 D6b) then drive the rebuild through `control_command`. Delete each - test that #1562b-W supersedes in the same slice. - - The structural part of gate 3 ("a structural transaction keeps the solo"), which #1562a's - record deferred. The batch 10 list does not have it. - - Meters and observation follow the new strip order after a rebuild that adds or removes a strip - (batch 8). This includes `meterProjection`'s fixed strip IDs (`sdk/src/browser/measurement.ts:208`, - `:302`), which is lane W's. - - The structure epoch in the meter header. `WebMeterHeader` (`hosts/host-web/src/lib.rs:937`) has - a `reserved_pad` word that is always zero, or the header can grow at its end. The ABI layout rows - and the regenerated `sdk/src/generated` copies change with it. -- **#1562b-K, additions:** - - `sdk/src/core/observation-subscriptions.ts` addresses a tap by strip and effect index. After a - rebuild, it maps its managed subscriptions to the new control list order. It ends a managed - subscription on #1584's `OBSERVATION_ENDED` event and reports this to its subscribers. - - It deletes the core hook only after #1562b-W has deleted the browser's override, so the two - lanes compile at every merge. -- **Frozen decisions.** D1-D6 and the batch 10 split hold. -- **No edit under `crates/engine/src/realtime`.** - -## Decision record: #1562b-W (batch 12) - -Lane W. Checked against the batch 12 head `0c6f35a1d`. - -### What shipped - -- **D9 is gone in the browser.** `REBUILD_PENDING`, `WebRuntimePreparer`'s `refuse_rebuild` - override, the `allow_rebuild` switch and `structural_command_for_test` (with - `native_staging::structural_command`) are deleted. The browser prepares and publishes a - rebuild like the C ABI: a transaction's ack still comes only after the preparer left the plan's - tables (`run_command`). The core hook `RuntimePreparer::refuse_rebuild` and its call in - `SessionState::command` stay for lane K to delete. Gate 11 and its test are deleted. - The #1290a, #1381b and #1584 D6b tests, and the plan-exchange and structural-realtime integration - tests, drive the rebuild through `control_request_mut` and `control_command` (the unit module's - `Rig::entry`, `native_staging::control_command`). -- **Structure epoch in the meter header.** `WebMeterHeader` grew at its end: `structure_epoch: u64` - at offset 80 (88 bytes). `run_command` stamps the newest committed plan's header (the pending - replacement's, else the rendering plan's) with `SessionState::structure_epoch()` (a new read-only - accessor next to `committed_session`; the only edit under `crates/control-plane`), and the boot - stamps the boot plan's. The worklet posts it as `structureEpoch` in every `miso.meter.v1` frame - (two 32-bit words, no BigInt) and in the `miso.observationmap.v1` reply (read from the rendering - plan's header). Mirrors: `abi_layout.rs`, `check-abi-layout-v1.py` and its fixture, the - regenerated `sdk/assets` and `sdk/src/generated/abi.ts`, both `.d.ts`, the host's frame validator. - Lane K reads `meterHeader.structureEpoch` in `pollMeters`. -- **Maps follow the plan.** The worklet read `sources` and the observation bindings once at boot; - both are now read from the module at every `miso.sessionmap.v1` / `miso.observationmap.v1` - request (`readSourceRows`, `readObservationBindings`), so a rebuild that adds a source, a strip or - an effect is seen. The exports those reads call after construction are now `render_locked` and - listed (wrapped) in `render-thread-exports.json`; `check-session-map-shape.py` follows - `readSourceRows`. -- **Browser SDK follows the strip order.** `createBrowserStripOrders` (sdk/src/browser/live-controls.ts) - keeps the last eight strip orders by structure epoch. A meter frame is named by the order of the - epoch it carries (`createMeasurementFeeds(host, orders, available)`, in-order delivery, a frame - whose list the page never held is dropped), and the observation map is paired with the order of - the epoch the reply names. No fixed boot strip IDs remain in `meterProjection`'s callers. -- **Browser qualification.** A new page row, `structural-apply` (`runStructuralApplyQualification`): - a `delay_samples` edit through the SDK is the rebuild path, the applied revision is not early, the - output is the input delayed by exactly the edit (computed, not read back), the meter stream is the - successor's with one structure epoch, and the render-locked count is 0 (11 workloads now). Four - mutations: `structural-apply-path`, `-early`, `-output`, `-meter`. - -### Departures and notes - -- Gate 1's `delay_samples` row is its own test (`a_delay_samples_edit_is_a_rebuild_and_renders_as_a_fresh_plan`), - because a rebuild's reply and swap differ from the live rows' loop. -- The rebuild rows of the batch 7 parity amendment are scripted: output route gain and mute, - `delay_samples`, the multiband and delay prepared bypass. **No launch effect declares an - audio-rate parameter** (`AutomationRate::Sample` has no user in a launch effect), so that row has no - edit. -- `sdk/test` repairs that the browser change forces (the module accepts the rebuild in headless - and browser alike): `control-list-evals.mjs` (rebuild-only row commits as `rebuild`), - `console-evals.mjs` (the delay and multiband lift render the unbypassed session), and the fake - `observationMap` replies carry `structureEpoch`. `sdk/README.md` loses one sentence. -- Not done by lane W: the SDK's named effect rows and `pollMeters`' re-read (lane K), the managed - observation subscription remap (lane K). - -### Gates - -`control_admission.rs`: `a_delay_samples_edit_is_a_rebuild_and_renders_as_a_fresh_plan` (gate 1 row), -`a_structural_transaction_keeps_the_solo` (gate 3, structural part). `control_parity.rs`: -`both_hosts_give_the_same_result_for_the_same_control_edits` (now with the six rebuild steps), -`every_scripted_edit_is_heard`, and `the_status_words_carry_a_rebuilds_watermark_the_c_abi_reports` -(#1349 rebuild leg, four rates, equal to `miso_engine_v1_plan_watermark` after every block). -`tests.rs`: `the_headers_structure_epoch_is_the_control_lists`, the epoch assertions of -`a_strip_count_change_publishes_the_successors_frame_from_the_adoption_block`, and the layout test. -Worklet harness: the epoch in the frame (including past 2^32 and the adoption block), the frame -validator, the re-read session and observation maps. SDK evals: `measurement-evals.mjs` (three new: -frames named by their epoch, in order; an unnameable frame is dropped; the engine follows a rebuild). -Browser: `npm run qualify` with `--sdk-root`, `--check-matrix`, `--self-test-mutations` passes in -chromium 151.0.7922.34, firefox 153.0 and webkit 26.5 (the matrix rows are unchanged). - -## Decision record: #1562b-K (batch 12) - -Lane K. Checked against `2b79cec44` (batch 12a + #1562b-W). - -### What shipped - -- **The core hook is gone.** `RuntimePreparer::refuse_rebuild` (`crates/control-plane/src/compile.rs`) - and its call in `SessionState::command` are deleted. Both hosts take every rebuild: the host-web - rebuild tests (`control_admission`, `structural_apply_realtime`, the unit `structural_apply`) and - the capi tests pass without the hook. -- **A meter frame names its own strips.** `WasmBoundary.pollMeters` reads the structure epoch in the - meter header (offset 80, #1562b-W) and names the frame by the control list of that epoch. The - boundary keeps the last eight orders by epoch; a frame whose list it never read and cannot read now - is dropped (the browser's `createMeasurementFeeds` does the same). `MeterFrame` gains - `structureEpoch`, `trackIds` and `submixIds` (additive), so a frame that a rebuild leaves with the - same strip count is still named right, and a window the old plan completed is named by the old list - even after the SDK read the new one. -- **Named effect rows follow the core's section headers.** `ControlStripOrder` (the one cached read of - the list) now also holds the effect of each console slot and insert, from the list's section - headers (`stripEffectOf`). `LiveControlEdits` resolves a named effect row from it first, from the - booted session second. An edit that names an effect row remembers (effect, row name, strict or - not); `effectRowResolver`, installed as the transaction queue's new `prepare` hook on both engines, - checks it against the list just before the frame is encoded (a probe of the structure epoch, the - list only when it moved; edits with no effect row cost nothing). A named section follows its header; - a builder typed for an effect the insert no longer runs is refused with `MisoUsageError`, and - nothing is sent. The queue draws the transaction's request ID after `prepare`, because the core takes - IDs in ascending order and `prepare` reads the list with IDs of its own. -- **Live controls follow the strips.** `EngineLiveControls.syncStructure()` and - `LiveControlTransport.stripOrder` re-read the order when the epoch moved; an observation record is - re-addressed to its strip's index in the current list just before it is sent, so a manual - `observe()` edit built before a rebuild is also right. The headless `liveControls()` and - `withSession()` drop the booted session from the layout once a rebuild moved the strips (the engine - is then no longer that session); the browser's memoized controls drop it in `update()`. -- **Managed subscriptions follow a rebuild and end on `OBSERVATION_ENDED`.** - `ObservationSubscriptionOwner` takes the event through the new transport member - `observationEnded` (the headless engine and the browser engine decode the reliable lane with - `decodeObservationEnded`, `sdk/src/core/observation-ended.ts`). The event, one per commit with every - key, drops the bindings and tells each subscription once: `onEnd({ revision, ended, remaining })`; - `handle.ended` lists them; a subscription left with no selection is closed. An event that arrives - while an operation is in flight is applied when it ends. Arm and disarm records take their strip - index and effect index from the newest control list and observation map (`syncStructure()`, and - `#remap()` of every held binding), not from the address they were subscribed with. - -### Departures and notes - -- **Files outside `sdk/src/core` and the core hook** (mechanical or forced by the above): - `sdk/src/headless/engine.ts` and `sdk/src/browser/{engine,live-controls}.ts` (the transport wiring), - the three index barrels (`ObservationSubscriptionEnd`), `sdk/README.md`, and the test fakes that - must carry `onControlEvent` (hosts) and `syncStructure` (live controls). The browser's - `createBrowserStripOrders` is now one reader for each host, shared by the queue's check, the live - controls and the feeds. -- **A structural fixture.** The SDK has no builder for a structural edit, so - `hosts/host-web/tests/structural_frames_fixture.rs` writes five frames with the protocol codec - (`fixtures/structural-frames.json`: add a first strip, replace the last strip, swap an insert for - another effect, remove an insert, remove the insert before it), and `structure-evals.mjs` sends - them to a real core. It is a wire format, so the byte comparison is the gate. -- **The queue does not learn a foreign commit's revision.** A structural frame sent past the queue - (as the evals do) leaves the queue one revision behind and the next transaction is refused. The - evals call `queue.reset(controlRevision())`. A transaction API for structural edits (#1296) goes - through the queue and has no such gap. -- The existing `set checks only encodability` eval now asserts that a named insert row resolves from a - document-text boot (it was a refusal before) and that an unknown insert is still refused. - -### Gates - -`sdk/test/structure-evals.mjs`: `a rebuild that adds a strip: frames before name the old strips, -frames after the new`; `a window the old plan completed is named by the old list even after the new -list was read` (red when the frame is named by the newest cached order); `a rebuild that keeps the -strip count and changes the strips is named by the new list`; `headless` and `browser: a row named -before an insert changed effect lands on the new effect's row` (the compressor's `mix` is control 7, -the soft clip's 3; the typed builder and a name the new effect lacks are refused and send nothing); -`the end of a tap is reported once, with the rest of the subscription kept (headless)`; `one commit -that ends every tap of a subscription closes it and says so once`; `removing the insert ends its tap -the same way`; `a tap that survives a rebuild keeps publishing and is disarmed at the strip's new -index`; `an effect that moved to another index is disarmed where it is now` (red without `#remap`); -`through the browser transport, whose live controls were made before the rebuild` (red without -`syncStructure`/the record re-addressing). `check-sdk-headless.sh` 543 pass; `check-sdk-types.sh`, -`check-sdk-generated.sh`, `sdk-package.sh check` pass on `target/ci/b12k-artifacts`; chromium -151.0.7922.34 qualification with `--sdk-root`, `--check-matrix`, `--self-test-mutations` passes (all -render-allocation rows 0). Rust: control-plane (`test-support`), capi and host-web -(`host-web/test-support`) tests, clippy `-D warnings` (host-core with no features, default and all -features, control-plane, capi, host-web), fmt, `realtime-policy` (141 regions) and -`check-bench-policy.sh` pass. - -### Open - -- Between a rebuild's commit and the block that adopts it, `observationMap()` describes the plan that - renders, and a kind 7/8 record is queued on the newest plan. The evals render a block before they - subscribe. Which plan's index space the record addresses in that window is the core's to rule - (successor, with #1589). - -## Fold (batch 12b review) - -- **M1.** The "Open" item above (which plan's index space a record addresses between a rebuild's - commit and its adoption) is closed on the SDK side. The core admits against the newest plan - (`admit_on_newest_plan`), the strip index comes from the newest list, and the effect index came - from `observationMap()`, which describes the plan that renders. `ObservationMap` now carries the - `structureEpoch` of that plan (the browser reply's own; the headless engine reads the meter header - of the rendering plan and names the strips by the list of that epoch). Subscribe, update and close - compare it with the epoch of the list used to address the record (`syncStructure()` returns it). On - a mismatch an engine with a render wait waits for a render and reads again (at most 8 renders and - the engine's request deadline); the headless engine, and a wait that ends, refuse with the typed - retryable `backpressure` error (`sdk.observation.structure_behind`). `submitManagedAt(epoch, ...)` - checks the list epoch again at the send, so a commit between the check and the send is refused, not - misaddressed. No record is sent in the window. -- **Gates** (`sdk/test/structure-evals.mjs`, fixture frame `swap_t3_multi_and_limiter` from - `structural_frames_fixture.rs`: two effects with the same tap change indices): subscribe, close and - update in the window are refused and arm or disarm the right tap after a render; an owner with a - render wait sends the record of the adopted plan; a wait that gives up sends nothing. The browser - engine: `measurement-evals.mjs`, the map's epoch 1 against list epoch 2. All red on the previous SDK. -- **m2.** Meter frames the SDK cannot name by epoch are counted. The browser feed adds their windows to - `lossCount` of the next update (saturating) and sets the loss bit once; the headless `MeterFrame` - carries `unnamedWindows`. Eval: `a meter frame whose list the page cannot name is dropped, counted, - and the stream goes on`. -- **m3.** `BindingState.armRevision` is the committed revision read just before the arm was sent; an - `OBSERVATION_ENDED` event with a revision at or before it is ignored (it ended an earlier arm). - Eval: `an ended event older than the arm is ignored`. - -## Decision record: source staging sized by the launch maxima (batch 12b review B1) - -Since #1562b-W the browser accepts rebuilds, and a rebuild can add a source the boot session did not -have. The browser sized its ID staging (longest ID of the boot session) and its source PCM and plane -staging (`maximum_source_channels` of the boot session) at boot, so an acked rebuild that added a -longer ID tripped a `debug_assert` and then indexed past the buffer inside the render-locked -`miso_engine_web_v1_source_id` export (a wasm trap), and one that added a wider source made the -worklet's `readSourceRows` return `null` (sticky `RESULT_INTERNAL`); either way the acked source -could never be fed (an ack preceded a drop). Root ruling A: size the staging by what the grammar -allows, so nothing depends on the boot session. - -- **Constants.** `session::STABLE_ID_MAXIMUM_BYTES` (127; the one literal `StableId::parse` and - control-plane's observation rows already used) and `session::SOURCE_CHANNELS_MAXIMUM` (255). The - grammar has no source-channel limit narrower than the `u8` of `Source::channels`, so the `u8` domain - is the maximum; no new limit was invented. `project_buffers` no longer takes the shape's two values. -- **Typed error.** `copy_id_into_staging` returns `Result` (`NoStaging`, - `TooLong`) and the export answers zero on either (its existing "no such source"), never a panic. - With the staging at the maximum the second cannot happen for a compiled session. -- **Memory.** Staging grows by 129,536 bytes of source PCM, 4,048 of plane references and 113 of ID - staging (this fixture: `idStagingBytes` 14 to 127, `sourcePcmStagingBytes` 1,024 to 130,560; - `bridge_retained_bytes` 1,631,181 to 1,764,878, `bridge_metadata_bytes` 1,594,186 to 1,598,234; - wasm `memoryBytes` after boot 2,883,584 to 3,014,656, two pages). Re-pinned with the reason in - `retained_ceilings.rs` (`bridge_retained_bytes` ceiling `CAPTURE + 787,968`) and - `browser-v1/expected.json` (`bridgeRetainedBytes` 1,812,352 = 1,048,576 + 763,776, measured beyond - capture 694,314 plus 10 %, rounded up to 64; the two exact staging rows; `memoryBytes`). -- **Boot budget.** The 1 MiB, 512-track worst document's high-water growth - (`scripts/check-web-boot-budget.mjs`) measures 20,447,232 bytes before and after the change, under - the 20 MiB pin plus one page (21,037,056): the parse transient dominates and the staging fits in - the memory the parse already grew. -- **Gates.** `plan_exchange_threads::a_rebuild_adds_the_longest_id_with_more_channels_than_any_boot_source` - (exports: a rebuild adds a 127-byte ID with six channels; `source_id` returns 127 and the staged bytes - match, `source_channels` is 6, a six-plane chunk is accepted and heard; red on a staging sized from - the boot session, both for the ID and the PCM), `control_parity::a_rebuild_adds_the_longest_stereo_source_to_a_session_that_booted_with_a_mono_source_only` - (browser and C ABI render the same bits across the rebuild for the new source at 44.1, 48, 88.2 and - 96 kHz), `host-web` unit `staging_holds_the_grammars_longest_id_and_widest_source_and_refuses_an_overlong_id`. -- **Departure.** A session with no source or track does not prepare (`host.source.graph.prepare`), so - "start empty" is not reachable; the gates boot with one mono source. diff --git a/.github/ISSUE_SPECS/1566-store-automation-targets-as-control-addresses.md b/.github/ISSUE_SPECS/1566-store-automation-targets-as-control-addresses.md deleted file mode 100644 index 831edd2ab..000000000 --- a/.github/ISSUE_SPECS/1566-store-automation-targets-as-control-addresses.md +++ /dev/null @@ -1,356 +0,0 @@ -# Store automation targets as control addresses - -Slice S12a of decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`, design points C2 -and C10). Code anchors verified on `main` `df585e503`. Feature sprint mode (owner, 2026-10-13): one -implementer, normal tests, a batch review; no per-attempt verifier or mutation runs. - -Decision 17's automation slice is split in two: this slice changes the stored target in the -session; #1567 changes `AUTOMATION_ENQUEUE`. - -## Product outcome - -A stored automation's target is a control address, the same address `SET_CONTROL` and the control -list use. A session can store automation for any automatable control, a send's gain or a VCA's -offset included, and the engine refuses a target on a control that is not automatable -(`rebuild_only`, `derived`, `monitor` or a `setting`) with one rule read from the row. Rendering -stored automation stays out of scope (#1315, #1058's successors). - -## Context - -- **The stored target** is `AutomationTarget { entity_id, rack, effect_id, parameter_id, channel }` - (`crates/session/src/model.rs:980-991`), a strip-only shape: `rack` is `inserts`, `builtins` or - `console` (`RackName`, `:999`), and a `builtins` target carries the literal `effect_id: "strip"` - (`BUILTIN_AUTOMATION_EFFECT_ID`, `crates/session/src/validate.rs:844`). -- **Its grammar.** JSON: `parse_target` (`crates/session/src/parse.rs:1584-1605`); keys and - canonical walk: `keys::target` (`crates/session/src/visit.rs:121`) and the `AutomationTarget` - walk in the same file; wire: `schema::session::automation_target` - (`crates/protocol/src/schema.rs:1089-1100`), `tx_automation_target` and - `parse_automation_target` (`crates/protocol/src/session_wire.rs:1137`, `:1903`), carried by - `SetAutomationTarget` (`0x0602`) and `UpsertAutomation` (`0x0600`). The SDK writes the keys in - `sdk/src/internal/session-json.ts:91` and resolves targets in `sdk/src/core/session.ts:1389-1490`. -- **Three validators, one rule each.** - - `validate_automation` (`crates/session/src/validate.rs:935`) requires the owner to be a track - or submix and, for an effect target, a `params` entry with the same `(parameter_id, channel)`. - - `BUILTIN_AUTOMATION_TARGETS` (`validate.rs:868`) is a second spelling of the builtin - descriptors, held to them by `crates/builtins-compiler/tests/builtin_automation_targets.rs`. - - `effect_automation_diagnostics` (`crates/effect-compiler/src/prepare.rs:370`) refuses a - non-block-rate effect target with `effect.automation.rate`; effect preparation calls it - (`:675`), and so does the classifier when the automation table changes - (`crates/host-core/src/live_delta.rs:273-278`). -- **Layering.** `session` sits below `controls`, and `controls` depends on `effect-compiler` (#1549). - So `session` cannot name `controls::ControlAddress`, and `effect-compiler` cannot call - `controls`. #1546 put `ControlAddress`, `Owner` and `Section` in `controls`; their fields are all - `session` types (`StableId`, `RackName` codes, `ParameterChannel`). -- **Documents that carry automation:** `fixtures/session/v1/canonical.json`, - `fixtures/session/v1/builtins-automation.json`, - `fixtures/session-canonical/v1/canonical-writer-corpus.json`, the author-session skill - (`.claude/skills/author-session/{SKILL.md,worked-session.json}`), and inline documents in - `crates/session/tests/*.rs`, `tools/session-validator/tests/validate.rs` and - `hosts/host-web/src/tests.rs`. The schema is described by `docs/SESSION_SCHEMA_V1.md` and - `docs/session-v1.schema.json`. - -## Decisions frozen for this slice - -- **D1. The address moves down.** `ControlAddress`, `Owner` and `Section` move from `controls` to - a new module `session::address` (`crates/session/src/address.rs`), unchanged. `controls` - re-exports them (`pub use session::{ControlAddress, Owner, Section}`), so no call site changes. -- **D2. The model.** `Automation.target` becomes a `ControlAddress`. `AutomationTarget`, - `BUILTIN_AUTOMATION_EFFECT_ID` and `BUILTIN_AUTOMATION_TARGETS` are deleted, with - `crates/builtins-compiler/tests/builtin_automation_targets.rs`. -- **D3. The JSON grammar.** A target is a record tagged by `section`, in the way `sidechain` is - tagged by `kind` (`crates/session/src/parse.rs`, `parse_sidechain`): its keys, in canonical - order, are `owner` (`"strip"`, `"route"` or `"vca"`), `owner_id` (a stable ID), `section` - (`"own"`, `"builtins"`, `"console"` or `"inserts"`), `slot_id` (a stable ID: the console slot or - the insert), `control` (a `u32`) and `lane` (`"left"`, `"right"` or `"both"`). `slot_id` is - required when `section` is `console` or `inserts` and refused (`reject_key`) when it is `own` or - `builtins`; every other key is always required. The old keys are unknown keys and refuse. - `owner: "session"` is not a token: no session row is automatable. -- **D4. One address message on the wire.** The wire form of `automation.target` is #1551 D3's - `control_address` message, the one `SET_CONTROL` carries (`owner_kind` session 1, strip 2, - route 3, VCA 4; `section` own 6, builtins 4, console 5, inserts 2; `slot_id` present exactly for - console and inserts; `control` and `lane` required here). `schema::session::automation_target` - and its two codecs are deleted; the session wire calls the `control_address` codec and refuses - `owner_kind` session. `keys::target` becomes the JSON names of D3 with `control_address`'s field - ids (`OWNER="owner":1`, `OWNER_ID="owner_id":2`, `SECTION="section":3`, `SLOT_ID="slot_id":4`, - `CONTROL="control":5`, `LANE="lane":6`), so the canonical walk and the wire agree. - `UpsertAutomation` and `SetAutomationTarget` keep their opcodes. -- **D5. Session validation is structural only.** `validate_automation` checks that the owner - exists in its namespace (a track or submix for `strip`, a route, a VCA), that a route or VCA owner - uses only section `own`, that a `console` slot is declared in the session console and an - `inserts` slot is one of that strip's inserts. The rule that an - effect target needs a `params` entry is deleted: the row exists whether or not the session states - the value. -- **D6. One row rule.** `controls::automation_diagnostics(model: &SessionModel, sample_rate: u32) - -> Vec` (`ControlDiagnostic { code: &'static str, path: String }`, new in - `controls`) checks every automation against its row, with path - `$.automation[id=].target` or `.segments[]`. It uses the code set of `controls::check` - and #1567 D3, so stored and enqueued automation are refused with the same four codes: - - `control.address.unknown`: no row at the address (an unknown builtin ID, effect parameter - or own row); - - `control.automation.target`: the row is not `automatable()`; - - `control.lane`: `left` or `right` on a shared row; - - `control.domain`: a segment's `unit` is not the row's unit, or a segment value that - `controls::check(descriptor, lane, value, None)` refuses. - - Amended by root's answer to Q13 (2026-10-13): one code set with #1567. -- **D7. Where it runs.** `effect_automation_diagnostics` and its call in effect preparation are - deleted. host-core's preparation calls `controls::automation_diagnostics` where it collects the - effect diagnostics and reports them as preparation diagnostics, so every host refuses alike. - The classifier calls it when the automation table changes (in place of - `effect_automation_diagnostics`). `tools/session-validator`'s stage 5 (prepare-effects) reports - them. -- **D8. The SDK.** `sdk/src/core/session.ts` takes automation targets as `ControlAddress` (#1550's - generated type) and writes D3's keys through `session-json.ts`. Its own target resolution - checks only what D5 checks; the row rule is the engine's. - -## Deliverables - -1. D1-D5 in `crates/session` and `crates/protocol`. -2. D6 and D7 in `crates/controls`, `crates/host-core`, `crates/effect-compiler` (the deletion) and - `tools/session-validator`. -3. D8 in the SDK. -4. Every document of Context migrated; `docs/SESSION_SCHEMA_V1.md` and `docs/session-v1.schema.json` - updated; the conformance corpus re-pinned once. - -## Authorized paths - -- `crates/session/src/{address.rs,model.rs,parse.rs,visit.rs,validate.rs,lib.rs,canonical.rs}`, - `crates/session/tests/**`. -- `crates/protocol/src/{schema.rs,session_wire.rs,model.rs}`, `crates/protocol/src/session_wire/tests.rs`, - `crates/protocol/tests/**`. -- `crates/controls/**`. -- `crates/effect-compiler/src/prepare.rs` (the deletion), `crates/host-core/src/{prepare.rs,live_delta.rs}`. -- `crates/builtins-compiler/tests/builtin_automation_targets.rs` (deleted). -- `crates/capi/src/runtime/live_tests.rs` (its two inline targets), `hosts/host-web/src/tests.rs` - (inline documents only). -- `tools/session-validator/**`, `crates/conformance/src/protocol_corpus.rs`. -- `fixtures/session/v1/{canonical.json,builtins-automation.json}`, - `fixtures/session-canonical/v1/canonical-writer-corpus.json`, and any fixture manifest that pins - their bytes. -- `docs/SESSION_SCHEMA_V1.md`, `docs/session-v1.schema.json`, - `.claude/skills/author-session/{SKILL.md,worked-session.json}`. -- `sdk/src/core/session.ts`, `sdk/src/internal/session-json.ts`, `sdk/src/generated/*` - (regenerated), `sdk/test/**`. - -## Non-goals - -- No automation rendering and no change to `AUTOMATION_ENQUEUE` (#1567). -- No automation on a monitor or setting row, ever (D6 refuses it by the row). -- No new automation shape or unit. - -## Objective gates - -Commands: `cargo test --locked -p session -p protocol -p controls -p effect-compiler`, -`cargo test --locked -p host-core --all-features`, `cargo test --locked -p session-validator`, -`cargo test --locked -p conformance`, `cargo test --locked -p capi`, -`bash scripts/check-sdk-types.sh`, `bash scripts/check-sdk-generated.sh target/ci/qualification-artifacts`. - -1. **The row decides.** A session whose automation targets, one each, a `rebuild_only` row - (`delay_samples`), a derived row (`effective_fader_db`), the monitor row (`solo`) and a setting - row (`pan_window_samples`) is refused with `control.automation.target` at each target; a send's - `gain_db` on a route into a submix, a VCA's `offset_db`, a builtin `fader_db` and a block-rate - effect parameter are accepted. -2. **Lane, unit and domain.** `left` on a route `gain_db` is `control.lane`; a segment - in `hz` on `fader_db` is `control.domain`; a segment value of 30 dB on `fader_db` is - `control.domain`; an automation whose address has no row is `control.address.unknown`. -3. **Structure.** An unknown owner, a route owner with section `builtins`, an undeclared console - slot and a `slot_id` on section `own` are each refused by session - validation with the existing `MissingEntityReference` or `InvalidEnum` codes. -4. **Round trip.** Every migrated fixture parses, writes back byte-identical, and round-trips - through the session wire codec; a target with an old key is refused as an unknown key. -5. **One rule on every path.** The same session is refused with the same codes by host-core - preparation, by a C ABI `SetAutomationTarget` transaction (the rebuild's diagnostic), and by - `session-validator` stage 5. -6. **The SDK writes the grammar.** An SDK-built session with a route and a VCA automation target - serializes to bytes the engine parses and writes back unchanged. - -## Test value - -- Gate 1 turns red if automation can target a value that is not live, or cannot target a live send - or VCA value. -- Gate 2 turns red if a segment can carry a lane, unit or value its row refuses, or if stored - automation is refused with a code that #1567 does not use for the same fault. -- Gate 3 turns red if a target names an entity, section or slot that does not exist. -- Gate 4 turns red if the grammar is not strict or the canonical form is not stable. -- Gate 5 turns red if two hosts or the validator tool judge one target differently. -- Gate 6 turns red if the SDK writes the old shape. - -## Dependencies - -- #1546, #1549 (every row), #1550 (`ControlAddress` in the SDK), #1557 and #1558 (send and VCA rows are live, - so automatable). -- *Refuse commands that would be acknowledged with no effect* (#1315), for the automation-serving - rule it states; not a code dependency. - -## Supersedes and shrinks - -- Shrinks *Refuse commands that would be acknowledged with no effect* (#1315): its stored-target - half (which targets automation may name) is this slice's row rule; its `AUTOMATION_ENQUEUE` - refusal stays until automation renders. - -## Amendment (controls coordinator, batch 9, 2026-10-14) - -Checked against `main` at `7feb27800`. - -- **Anchors moved.** `BUILTIN_AUTOMATION_EFFECT_ID` is `validate.rs:870`, `BUILTIN_AUTOMATION_TARGETS` - `:894`, `validate_automation` `:961`. `schema::session::automation_target` is `schema.rs:1255-1265` - (the session specs start at `:903`). `tx_automation_target` is `session_wire.rs:1011`, - `parse_automation_target` `:1689`. The classifier's call is `live_delta.rs:370`. The SDK resolver is - `session.ts:1389-1502`. `parse.rs:1584`, `visit.rs:121`, `prepare.rs:370`/`:675` and - `session-json.ts:91` held. -- **D1 moves the whole module.** `crates/controls/src/address.rs` now also holds `Lane` (the - `ParameterChannel` alias), `OwnerKind`, `SectionKind` and their `code`/`name`/`kind` impls (#1546 - S7c). An inherent impl cannot stay behind its type, so all of it moves to `session::address` - (it uses only `session` types), and `controls` re-exports all six names, as `controls/src/lib.rs:46` - exports them today. -- **D4's codec exists.** `SET_CONTROL`'s edit already reads and writes the address through - `message_wire::write_set_control_address` and `decode_set_control_address` - (`session_wire.rs:1296-1305`, `control` and `lane` required). The automation target calls the same - two functions and then refuses owner kind session. The spec is the top-level - `crate::schema::control_address` (`schema.rs:344-356`), not a `schema::session` module. -- **#1568 lands first (batch 9).** Its test `session_keys_and_wire_fields_have_equal_ids` pairs the - `target` row with `automation_target::SPEC`. This slice changes that row to - `crate::schema::control_address::SPEC` with no wire-only IDs (both sides `{1..6}`), in - `schema.rs`'s test module (already an authorized path). -- **D3's walk.** Walk the target with `variants!`, one arm per section (the arm token is the section - name), so each arm has one fixed key list. #1569 D2 refuses a record walked with two key lists. - The `own` and `builtins` arms have the same key list; #1569's amendment accepts that. The migrated - corpus walks all four arms. -- **D6 has a gap: `control.inactive`.** `controls::check` returns `control.inactive` for a row that the - owner's shape does not use (a matrix row on a pan strip). Such a row is `automatable()`, so it - reaches `check`, and D6's four codes do not include it. Options: (A) add `control.inactive` as a - fifth code, for stored and enqueued automation alike; (B) report it as `control.automation.target`. - Recommendation: A. It is `check`'s own code for the same fault in `SET_CONTROL`, so one fault has one - code everywhere. `control.read_only` cannot occur, because a derived row fails `automatable()` - first. `ControlError` has no `control.automation.target` variant today; this slice adds one. -- **D7's call sites.** Host-core preparation collects effect diagnostics at `prepare.rs:1908` - (`prepare_native_session_effects(...).map_err(effect_diagnostics)`); the automation diagnostics - join that set. `tools/session-validator` has no `controls` dependency yet: add it in its - `Cargo.toml` (under `tools/session-validator/**`) and update `Cargo.lock`. -- **The browser.** Since #1562a the browser applies session edits through the core. An automation edit - that the row rule refuses classifies as a rebuild (`LiveRebuild::AutomationTarget`), so until - #1562b the browser refuses it with `web.apply.rebuild_pending`, not with the row code. An accepted - edit commits on the live path. Gate 5 keeps its three paths; no browser gate is added. -- **#1315 is closed (batch 8).** Its `AUTOMATION_ENQUEUE` refusal is on `main`; this slice does not - change it. "Supersedes and shrinks" needs no action. -- **More files that name the old target** (authorized): `crates/effect-compiler/tests/native_session.rs`, - `crates/host-core/tests/{launch_registry_once,live_delta,live_edit_classifier}.rs`, - `sdk/src/core/types.ts`, `docs/EFFECT_CONTRACT_V1.md` (its `effect.automation.rate` lines) and - `Cargo.lock`. Closed handoff records under `docs/handoffs/` stay unedited. -- **Edit order.** After #1568 (`visit.rs`, `schema.rs`); after #1341a1/a2 and #1611 - (`effect-compiler/src/prepare.rs`; #1341a1 also `host-core/src/live_delta.rs`); after #1569 - (shared `visit.rs`, `session-json.ts` and the corpus: this slice then regenerates the SDK key - tables instead of editing them by hand, and adds the target arms to #1569 D4's `$defs` table); - after #1602 (`host-core/src/prepare.rs`, `hosts/host-web/src/tests.rs`, different functions). - Before #1567, and before #1615 (`session/src/canonical.rs`). - -## Ruling (root, 2026-10-14): `control.inactive` (the stored path only) - -`controls::check` returns `control.inactive` for a row that is automatable but inactive (for -example a matrix row on a pan strip). Root ruled option A: `control.inactive` is the fifth code a -stored automation target can be refused with, the same code `SET_CONTROL` uses for the same fault. -D6's code set is the four filed codes plus `control.inactive`, and a gate shows one inactive -automatable row refused with it. `ControlError` gains the automation-target variant this slice -needs. - -## Decision record: S12a (batch 9) - -**What shipped.** A stored automation's target is a `session::ControlAddress`. D1-D8 as filed and -amended, with root's ruling: `control.inactive` is the fifth code of the stored-target rule, and -`ControlError` gains `AutomationTarget` (`control.automation.target`). - -- **Model and grammar.** `address.rs` moved whole to `session::address` (`ControlAddress`, `Owner`, - `Section`, `Lane`, `OwnerKind`, `SectionKind`; `controls` re-exports all six). `AutomationTarget`, - `BUILTIN_AUTOMATION_EFFECT_ID` and `BUILTIN_AUTOMATION_TARGETS` are deleted, with - `builtin_automation_targets.rs`. `parse_target` reads D3's six keys (`slot_id` required for - `console` and `inserts`, `reject_key` for `own` and `builtins`; `owner: "session"` is an invalid - enum). `keys::target` has `control_address`'s field IDs, and the walk is `variants!` with one arm - per section (`own`, `builtins`, `console`, `inserts`), so the SDK key tables regenerate to four - `target` arms. -- **Wire.** `schema::session::automation_target` and its two codecs are deleted; `tx_automation_target` - and `parse_automation_target` call `write_set_control_address` and `decode_set_control_address` and - refuse owner kind session. The #1568 pairing row is `target` -> `control_address::SPEC`. -- **Validation.** `validate_automation_target` (session) checks structure only: owner exists in its - namespace (an `Index.routes` set joins the graph map), a route or VCA owner uses `own` - (`schema.invalid_enum`), a console slot is one the strip carries, an insert is one of the strip's. -- **The row rule.** `controls::automation_diagnostics(model, sample_rate)` with `ControlDiagnostic`: - `control.address.unknown`, `control.automation.target`, `control.inactive`, `control.lane`, - `control.domain` (unit differs from the row's, or `check` refuses a segment value). A target is - reported once and hides its segments' faults. -- **Where it runs.** `effect_automation_diagnostics` is deleted from `effect-compiler`. host-core's - session preparation merges the rule's diagnostics with the effect diagnostics (one sorted - `EffectDiagnosticSet`); the classifier calls it in step 2; `session-validator` stage 5 merges it - the same way (new `controls` dependency, `Cargo.lock`). `effect.automation.rate` no longer exists. -- **SDK.** `AutomationTarget` is `ControlAddress | ControlWireAddress`; `AutomationRack` is removed. - The builder checks D5 only, finds the row (by machine name or numeric ID) in `CONTROLS` for the - span's unit, and writes D3's keys; domain checks of span values are gone (the engine's rule). - `enginectl`'s request parser takes the same address. - -**Departures from the text.** - -1. Gate 3, `slot_id` on `own`: it is a parse refusal (`schema.unknown_field` at `target.slot_id`, - D3's `reject_key`), not a validation code; the model cannot hold a slot on `Own`. -2. A target in an effect that has no rows (a `cid` effect, or a native effect the launch registry - lacks) is not reported by the rule: effect preparation refuses that effect with its own code - (`effect.third_party.unavailable_at_launch`, `effect.native.unavailable`), as #1335 D2 did for - the rate code. One fault, one report. -3. `fixtures/session/v1/builtins-automation.json`: the `matrix-ll` automation on a pan strip is - `control.inactive` now, so it became `pan-ride` (builtin 12, left). The fixture also gained a - transparent submix `bus`, a send `send` into it, a VCA `riders` and `route-ride` and `vca-ride` - automations, so one engine-written document holds a route and a VCA target (gate 6). The - canonical-writer corpus's full-surface document gained the VCA, route, builtins and console - targets (`canonical-full-surface` and `canonical-exchanged-strips` changed; they walk all four - arms). `canonical.json` changed only in its target. -4. `COMPLETE_SCHEMA_HASH` is re-pinned once (`0x4f6a_80fb_52c1_2aed`, from `0xcdfd_b054_0f09_3424`): the - transaction fixture's automation edits carry the address message. The retired-rack rows for the - target (`retired.rack_code_{1,3}.set_automation_target`) patch the address `section` and meet - `INVALID_FIELD` (the address decoder's status for an unallocated section), not `MALFORMED_FRAME`. -5. Effect parameters with `AutomationRate::Sample`: the old rule refused them, and the row says - `Live` for any rate but `None` (`effect_rows::parameter_descriptor`), so the new rule accepts a - target on one. No launch effect has such a parameter; the test double that held the case is - deleted with its test. Successor: decide whether the row class must follow `Block` only. -6. Files outside the authorized list, all mechanical: `sdk/src/cli/session-request.ts` (the request - parser's target), `scripts/check-effect-runtime-policy.sh` and - `scripts/test-effect-runtime-policy.sh` (the required-diagnostic list lost - `effect.automation.rate`). - -**Gates and tests.** - -1. `controls/tests/automation.rs::the_row_decides_what_can_be_automated`. -2. `controls/tests/automation.rs::lane_unit_domain_and_unknown_rows_are_refused_with_the_validators_codes` - (includes `control.inactive`), `::every_automation_is_judged_and_a_refused_target_reports_once`, - `::a_target_in_an_unavailable_effect_is_reported_by_the_effect_alone`. -3. `session/tests/automation_target.rs` (`a_target_must_name_things_that_exist`, - `slot_id_exists_exactly_in_a_console_or_inserts_section`) and the migrated `invalid_matrix`, - `diagnostic_parity`, `submix_strip`, `vca` rows. -4. `session/tests/automation_target.rs::{the_retired_target_keys_are_unknown, - every_owner_and_section_round_trips}`; `canonical_writer_corpus_is_rust_generated_and_current`; - `session_schema_lists_exactly_the_walked_keys`; `protocol` session-wire corpus round trip with a - target per owner kind and section; `an_automation_target_cannot_be_owned_by_the_session`; - `session-validator` `canonical_output_reproduces_the_checked_in_canonical_fixtures`. -5. `host-core/tests/stored_automation.rs::preparation_refuses_an_automation_the_row_refuses`; - `capi` `an_automation_on_a_prepared_effect_parameter_is_refused_before_any_ack` (the C ABI - transaction's diagnostic equals preparation's, `control.automation.target`); - `session-validator` `an_automation_on_a_prepared_effect_parameter_fails_effect_preparation`. -6. `sdk/test/builder-evals.mjs` "the builder reproduces a checked-in canonical fixture byte for byte" - (route and VCA targets included) and "the builder resolves a target's row and leaves the row rule - to the engine" (the engine's own codes through `validate()`). - -Measured numbers: none (no performance claim). - -## Fold (batch 9 review) - -Departure 5 above is resolved. A row is `Live` only when `automation_rate == Block` -(`effect_rows::parameter_class`): a `None`-rate parameter keeps its prepared reason, and a -`Sample`-rate parameter is `RebuildOnly(Correctness)` with the reopening condition "sample-accurate -render-path delivery, a later protocol capability" (`EFFECT_CONTRACT_V1.md`). So a `Sample`-rate -parameter is no longer automatable and `automation_diagnostics` refuses a target on it with -`control.automation.target`, as the old `effect.automation.rate` did. The classifier's guard in -`live_delta::parameter_records` now describes a defence that the rows already satisfy, and -`EFFECT_CONTRACT_V1.md` states the rule. - -Test: `controls::effect_rows::tests::a_sample_rate_parameter_is_rebuild_only_and_not_automatable` -holds the `Sample`-rate double. It lives in the crate (a unit test) and not in -`controls/tests/automation.rs` because no launch effect declares a `Sample`-rate parameter and the -integration tests reach only the launch registry; the unit test calls `parameter_descriptor` on the -double. No launch row changed, so the control catalog and the generated SDK files are unchanged. diff --git a/.github/ISSUE_SPECS/1567-address-enqueued-automation-by-control-address.md b/.github/ISSUE_SPECS/1567-address-enqueued-automation-by-control-address.md deleted file mode 100644 index 37a531cde..000000000 --- a/.github/ISSUE_SPECS/1567-address-enqueued-automation-by-control-address.md +++ /dev/null @@ -1,161 +0,0 @@ -# Address enqueued automation by control address - -Slice S12b of decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`, design point -C10: "An `AUTOMATION_ENQUEUE` batch carries a header table of addresses; its records index that -table. No handle crosses a revision."). Code anchors verified on `main` `df585e503`. Feature -sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review; no per-attempt -verifier or mutation runs. - -## Product outcome - -An `AUTOMATION_ENQUEUE` batch names its targets by control address, the same address `SET_CONTROL` -and stored automation use, in one table at the head of the batch. No list handle crosses the wire, -so a client never needs a handle to automate, and a batch can never name a handle of an old -structure. Each record is checked against its address's row. Rendering enqueued automation stays -out of scope; the control plane keeps refusing the command until automation renders (#1315). - -## Context - -- **The wire today.** `AutomationEnqueue { records: &[AutomationRecord] }` - (`crates/protocol/src/message_wire.rs:415`); each record is 32 fixed bytes with a - `handle: ParameterHandle` at bytes 4..8 (`AutomationRecord`, `crates/protocol/src/queue.rs:58`, - decode at `:85`). A batch holds 1 to 256 records (`AUTOMATION_BATCH_RECORDS`), copied into a - preallocated `AutomationBatchSlot` (`queue.rs:199-207`). -- **The check today.** `validate_automation_domains` (`crates/protocol/src/controller.rs:3358`) - reads each record's handle through the provider's descriptor. #1552 D5 serves it from the - control catalog by handle (`control_row(handle)`) and refuses a non-effect row with - `automation.target.unsupported` until this slice. -- **Cancellation.** A structural commit cancels queued batches with `AUTOMATION_CANCELED`, reason - `RevisionChanged` (`message_wire.rs:526-537`). -- **The command is not served.** *Refuse commands that would be acknowledged with no effect* - (#1315 D1-D3) makes the control plane answer `AUTOMATION_ENQUEUE` with `Unavailable` while no - render drain exists. The protocol controller still decodes and validates it when its - `automation` feature is on, which its own tests use. - -## Decisions frozen for this slice - -- **D1. The address table.** `AutomationEnqueue` gains a first field, `addresses`: a repeated - `MESSAGE` of #1551 D3's `control_address` (with `control` and `lane` required), 1 to 256 entries. - The records field follows. A batch whose table is empty, has more than 256 entries, or repeats - an address is refused on decode with `INVALID_FIELD`. -- **D2. The record.** Bytes 4..8 of the 32-byte record become `address_index: u32`, an index into - the batch's table; an index at or past the table's length is refused on decode with - `INVALID_FIELD`. `ParameterHandle` leaves the automation record. The record size and every other - byte do not change. -- **D3. Validation.** Before admission, every address resolves through the provider to a row of - the committed model, and every record is checked against its row, in record order; the first - failure refuses the whole batch with `INVALID_FIELD` and a diagnostic carrying the control code: - `control.address.unknown`, `control.automation.target` (the row is not `automatable()`), - `control.lane` (a `left` or `right` lane on a shared row), or `control.domain` (a start or end - value that `controls::check(descriptor, lane, value, None)` refuses). Nothing is queued before - every record passes (the acked-batch rule). #1552 D5's `automation.target.unsupported` refusal is - deleted: every automatable row is accepted. -- **D4. The slot holds no string.** Admission resolves each address to the committed catalog's - handle and stores it, with the batch's `structure_epoch`, in the preallocated - `AutomationBatchSlot`. This handle never leaves the controller. A commit that advances - `structure_epoch` cancels every queued batch with `AUTOMATION_CANCELED` reason `RevisionChanged`, - as a structural revision does today. -- **D5. Docs.** `docs/CONTROL_BTLV_V1.md` and `docs/CONTROL_PROTOCOL_REGISTRY.md` describe the table - and the record's `address_index`. - -## Deliverables - -1. D1-D4 in `crates/protocol` (`message_wire.rs`, `queue.rs`, `schema.rs`, `controller.rs`) and - the provider's row lookup by address in `crates/host-core/src/control_provider.rs`. -2. The conformance corpus and `tools/audit/src/protocol.rs` re-pinned once; D5. - -## Authorized paths - -- `crates/protocol/src/{message_wire.rs,queue.rs,schema.rs,controller.rs,typed_frame.rs,lib.rs}`, - `crates/protocol/src/controller/tests.rs`, `crates/protocol/tests/**`. -- `crates/host-core/src/control_provider.rs`, `crates/control-plane/src/control.rs` (the provider - call only). -- `crates/conformance/src/protocol_corpus.rs`, `tools/audit/src/protocol.rs`. -- `docs/CONTROL_BTLV_V1.md`, `docs/CONTROL_PROTOCOL_REGISTRY.md`. -- Tests in `crates/capi` that build an `AUTOMATION_ENQUEUE` frame. - -## Non-goals - -- No automation rendering; the control plane's `automation` feature stays off (#1315 D3). -- No change to the stored automation grammar (#1566). -- No SDK encoder for `AUTOMATION_ENQUEUE` (none exists). - -## Objective gates - -Commands: `cargo test --locked -p protocol -p host-core -p control-plane --features test-support`, -`cargo test --locked -p conformance`, `cargo test --locked -p capi`. - -1. **Codec.** A batch of 3 addresses and 5 records round-trips; an empty table, 257 entries, a - repeated address, and a record index equal to the table length are each refused on decode. -2. **Validation by row** (controller test with the `automation` feature on). A batch that targets a - send's `gain_db` and a builtin `fader_db` is admitted; one record on `delay_samples`, on `solo`, - on `effective_fader_db`, with `left` on a route `gain_db`, or with an end value of 30 dB on - `fader_db` refuses the whole batch with the matching code, and the queue is unchanged. -3. **No stale target.** After a commit that adds a VCA (a new `structure_epoch`), a batch queued - before it is canceled with `RevisionChanged`, and a new batch with the same addresses is - admitted. -4. **Still not served.** Through the C ABI, `AUTOMATION_ENQUEUE` still answers `Unavailable` - (#1315). - -## Test value - -- Gate 1 turns red if a record can point outside its table or a table can repeat a target. -- Gate 2 turns red if enqueued automation can target a value that is not live, or the batch is - partly queued before a refusal. -- Gate 3 turns red if a queued batch keeps a target across a change of the row set. -- Gate 4 turns red if this slice turns on a command no render path serves. - -## Dependencies - -- #1551 (the `control_address` message), #1552 (the catalog and `structure_epoch`), #1566 (the row - rule `controls::automation_diagnostics` names). -- *Refuse commands that would be acknowledged with no effect* (#1315). - -## Supersedes and shrinks - -- None beyond #1566's line on #1315. - -## Amendment (controls coordinator, batch 8, 2026-10-14) - -Since #1315 (batch 8), both hosts build their controller without the `automation` provider -feature and refuse `AUTOMATION_ENQUEUE` `UNAVAILABLE` before any check; command `0x0006` is not -advertised. This slice's address table and domain check are therefore reachable only through the -protocol crate's controller with the feature enabled (as `controller::tests` does), not through -either host, until the render-side automation drain (#1058) sets the feature. Gates that drive a -host test the refusal; the address and domain gates run on the protocol controller. - -## Amendment (controls coordinator, batch 9, 2026-10-14) - -Checked against `main` at `7feb27800`. - -- **Anchors moved.** `AutomationEnqueue` is `message_wire.rs:856`; `RevisionChanged` and - `AutomationCanceled` are `:969-982`. `validate_automation_domains` is `controller.rs:3861`. #1552 - D5's `automation.target.unsupported` is `controller.rs:3634-3638`, with the provider's note at - `control_provider.rs:907`. The feature refusal is `execute`, `controller.rs:3156`; the capability - bit is `:3758`. `queue.rs:58`, `:85` and `:199-207` held. -- **D1's codec exists.** Decode and encode each table entry with - `message_wire::decode_set_control_address` and `write_set_control_address` (`SET_CONTROL`'s, - `control` and `lane` required). -- **D3 follows #1566 on `control.inactive`.** `controls::check` returns `control.inactive` for an - automatable row that the owner's shape does not use. #1566's batch 9 amendment proposes it as a - fifth code (recommended); this slice uses the same ruling. -- **D4 in fact.** Today every committed session transaction that is not monitor-only cancels the - queued batches with `RevisionChanged` (`controller.rs:2303` and `:3260`), live ones included. So a - stored handle never outlives a revision. D4's epoch stamp stays as written; gate 3 is unchanged. -- **#1315, more precisely than the batch 8 amendment.** The frame decodes before `execute`'s feature - check. Through a host, a batch that fails D1 or D2 decoding is `INVALID_FIELD`, and a well-formed - batch is `UNAVAILABLE`. Gate 4 sends a well-formed batch in the new shape. Gate 1 stays a codec - test in `protocol`. -- **No drain is filed.** #1058 (research) is closed and no open issue renders stored or enqueued - automation. The refusal holds until such a slice is filed and sets the feature. -- **More files that build an `AUTOMATION_ENQUEUE` frame** (authorized): - `crates/protocol/src/message_wire/tests.rs`, `crates/capi/src/runtime/tests.rs` (`:284-305`, - `:2251`, `:2710`) and `hosts/host-web/tests/protocol_cross_host.rs` (`:704-730`). -- **Edit order.** After #1566. Before #1615 in `protocol/src/controller.rs` (#1615's S2 edits the - structural prepare and commit; this slice edits admission). With #1574 and #1615 in - `host-core/src/control_provider.rs`: this slice first. - - -## Parked (root, 2026-10-14) - -Parked 2026-10-14: #1058 A9 retires AUTOMATION_ENQUEUE; close as superseded when #1627 lands. diff --git a/.github/ISSUE_SPECS/1568-keep-the-session-json-keys-and-the-session-wire-field-ids-equal.md b/.github/ISSUE_SPECS/1568-keep-the-session-json-keys-and-the-session-wire-field-ids-equal.md deleted file mode 100644 index 1b126910e..000000000 --- a/.github/ISSUE_SPECS/1568-keep-the-session-json-keys-and-the-session-wire-field-ids-equal.md +++ /dev/null @@ -1,136 +0,0 @@ -# Keep the session JSON keys and the session wire field IDs equal - -Slice S13a of decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`, "How to add a -control", step 2.1; evidence in `17-s13-assessment.md`). Code anchors verified on `main` -`df585e503`. Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review; -no per-attempt verifier or mutation runs. - -## Product outcome - -A new session value field is declared with one field ID in the session key registry, and a test -turns red if the protocol's session wire schema gives that record a different set of field IDs. Today -the two lists are separate copies that nothing compares, so a new key can reach the canonical JSON -with no wire field, or with a wire field under another ID, and the two forms of one session drift. - -## Context - -- **The key registry.** `session::keys` (`crates/session/src/visit.rs:84-125`): one `key_module!` - per record, each field a `FieldKey { name, id }` (`:7-12`). The canonical JSON walk and the - estimator read it. Its doc says the IDs are "transcribed from `protocol/src/schema.rs`" (`:83`). - The macros are `fields!` and `key_module!` (`:80-81`). -- **The wire schema.** `protocol::schema::session` (`crates/protocol/src/schema.rs:737` onward): - one `pub(crate) mod` per record with `FieldSpec { id, wire, mandatory, repeated, nested }` - (`:162-168`) and a `MessageSpec { name, fields }` (`:209-212`). The wire encoder and decoder - (`crates/protocol/src/session_wire.rs`) read only this. Its existing structural test is - `migrated_specs_are_sorted_unique_and_structurally_consistent` (`schema.rs:1716` onward), which - never looks at `session::keys`. -- **The record pairs** (keys module → wire spec): `console`, `console_slot`, `console_entry`, - `render_profile`, `output_profile`, `source`, `track` (`track::SPEC`; the retired fields 6 and 8 - are in `track::RETIRED`, `:1003-1009`, and have no key), `builtins`, `channel_builtins`, `rack`, - `effect`, `identity` → `effect_identity`, `param`, `sidechain` → `sidechain::KNOWN`, `fader`, - `matrix_or_pan` → `matrix_or_pan::KNOWN`, `submix`, `vca`, `control_smoothing`, `output`, - `route`, `route_source` → `route_source::KNOWN`, `route_destination`, `channel_matrix`, - `automation`, `target` → `automation_target`, `segment` → `automation_segment`. The root module - `session` has no wire message (the root is not sent whole). -- **Wire-only fields.** `matrix_or_pan`'s field 1 is a wire-only tag: the walk emits it with - `wire_tag`, not with a key (`visit.rs:282-287`). Every other tagged record names its tag as a - `kind` key with ID 1. -- **No shared ID test exists** (assessment, row 7). - -## Decisions frozen for this slice - -- **D1. The registry lists itself.** Replace the per-module `key_module!` invocations with one - macro invocation, `key_modules! { (name, "doc"; FIELD = "json" : id, ...) ... }`, that emits the - same modules and constants as today plus, in each module, `pub const ALL: &[FieldKey]` (its - fields in declaration order) and, in `keys`, `pub const MODULES: &[(&str, &[FieldKey])]` (every - module's name and `ALL`, in declaration order). Every existing constant keeps its name and value; - the canonical bytes and the walk do not change. -- **D2. The test.** A unit test in `crates/protocol/src/schema.rs`'s test module, - `session_keys_and_wire_fields_have_equal_ids`: - - a fixed table pairs each keys module name with its wire spec (Context) and with its wire-only - IDs (`{1}` for `matrix_or_pan`, empty for every other); - - for each pair: the set of `ALL` IDs, plus the wire-only IDs, equals the set of the spec's field - IDs; - - every name in `keys::MODULES` except `session` has a row in the table, and every row names a - module in `keys::MODULES`. So a new keys module without a wire pairing, or a removed module, - turns the test red. - It compares ID sets, not wire types: a type mismatch is caught by the existing codec round-trip - tests. -- **D3. The doc line.** `visit.rs:83`'s "transcribed from" text says instead: the IDs are the wire - schema's, and `session_keys_and_wire_fields_have_equal_ids` holds them equal. -- **D4. Order with #1566.** #1566 replaces the `automation_target` spec with #1551's - `control_address` message and gives `keys::target` that message's IDs. Whichever of the two - slices lands second updates the `target` row of D2's table. - -## Deliverables - -1. D1 in `crates/session/src/visit.rs`. -2. D2's test and D3's doc line. - -## Authorized paths - -- `crates/session/src/visit.rs`. -- `crates/protocol/src/schema.rs` (its test module only). -- `crates/session/tests/visit_model.rs` (only if a test there names a macro that D1 renames). - -## Non-goals - -- No change to any field ID, key name, canonical byte or wire byte. -- No generation of the wire schema from the keys (the assessment explains why). -- No check of wire types or of edit payload specs. - -## Objective gates - -Commands: `cargo test --locked -p session -p protocol`, `cargo test --locked -p conformance`. - -1. `session_keys_and_wire_fields_have_equal_ids` passes on `main`'s IDs. -2. **It catches drift** (PR evidence, not a committed test): in a scratch commit, change - `keys::route::FOLLOWS_MUTE` to ID 8 and record that the test is red; add a key to - `keys::fader` with no wire field and record that it is red; restore. -3. Every existing session, protocol and conformance test passes, and `COMPLETE_SCHEMA_HASH` does - not move. - -## Test value - -- Gate 1 turns red if a session record's JSON keys and its wire fields name different IDs, or if a - new record is added to one registry and not the other; no existing test compares them. -- Gate 2 records that the test sees both kinds of drift. -- Gate 3 turns red if D1's macro rewrite changes a key or a byte. - -## Dependencies - -- None. It can land at any point of decision 17; see D4 for its order with #1566. - -## Supersedes and shrinks - -- None. - -## Decision record: S13a keys and wire IDs (batch 9) - -Shipped D1, D2 and D3 as written. - -- **D1.** `crates/session/src/visit.rs`: `key_module!` and `fields!` became one `key_modules!` - invocation. Every module and constant keeps its name and value; each module gains - `ALL: &[FieldKey]` and `keys::MODULES: &[(&str, &[FieldKey])]` (declaration order). The macro - adds no field the old one lacked, so canonical bytes and the walk do not change. -- **D2.** `schema::tests::session_keys_and_wire_fields_have_equal_ids` in `crates/protocol/src/schema.rs`. - A 27-row table pairs each keys module with its wire spec and its wire-only IDs (`{1}` for - `matrix_or_pan`). It also checks that no retired track ID (`track::RETIRED`: 6, 8) has a key. -- **D3.** The `keys` doc line now says the IDs are the wire schema's and names the test. -- **Anchor moves.** The session specs are at `schema.rs:903` onward (not `:737`); the tests module is - at `:1719`. `visit.rs` anchors held. -- **Departure: `control_smoothing`.** The spec lists it as a record pair, but the protocol has no - session wire message for it (it is written only by the `SET_CONTROL` row). Its table row is - `None`: it must still have a row (a new keys module with no row turns the test red), but no ID - comparison is possible. -- **D4.** #1566 has not landed: the `target` row pairs with today's `automation_target::SPEC` and - carries a comment. #1566 updates that row when it lands. -- **Gate 1.** Passes on `main`'s IDs. -- **Gate 2 (scratch change, reverted).** `keys::route::FOLLOWS_MUTE` set to ID 8: red with - `keys::route field IDs differ from wire message Route left: {1,2,3,4,5,6,8} right: {1,2,3,4,5,6,7}`. - `EXTRA="extra":5` added to `keys::fader`: red with `keys::fader field IDs differ from wire - message DualMonoFader left: {1,2,3,4,5} right: {1,2,3,4}`. -- **Gate 3.** `cargo test --locked -p session -p protocol` and `cargo test --locked -p conformance` - pass with no failure (`COMPLETE_SCHEMA_HASH` unmoved; no wire or schema file changed beyond the - test). `cargo clippy --locked -p session -p protocol --all-targets --all-features -- -D warnings` - and `cargo fmt --all --check` are clean. diff --git a/.github/ISSUE_SPECS/1569-generate-the-sdks-session-key-tables-from-the-engines-walk.md b/.github/ISSUE_SPECS/1569-generate-the-sdks-session-key-tables-from-the-engines-walk.md deleted file mode 100644 index 229dd974f..000000000 --- a/.github/ISSUE_SPECS/1569-generate-the-sdks-session-key-tables-from-the-engines-walk.md +++ /dev/null @@ -1,205 +0,0 @@ -# Generate the SDK's session key tables from the engine's walk - -Slice S13b of decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`, "How to add a -control", step 2.1; evidence in `17-s13-assessment.md`, rows 11 and 12). Code anchors verified on -`main` `df585e503`. Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch -review; no per-attempt verifier or mutation runs. - -## Product outcome - -When a session value field is added in Rust, the SDK's canonical session writer learns the key, its -position and its number spelling by regeneration, with no hand edit. The checked JSON Schema -document can no longer miss a key: a test turns red until the document lists every key the engine -writes, in its record, and no other. - -## Context - -- **The SDK's hand copy.** `sdk/src/internal/session-json.ts` holds `ROOT_KEYS` (`:13-30`), - `INTEGER_KEYS` and `FLOAT_KEYS` (`:39-70`, "transcribed from the Rust field walk"), - `OBJECT_KEY_ORDERS` (`:72-93`) and the hand-written variant and parent rules `taggedOrder` and - `objectOrder` (`:95-144`). The canonical writer corpus - (`fixtures/session-canonical/v1/canonical-writer-corpus.json`) holds SDK and Rust output equal - (`sdk/test/builder-evals.mjs`). -- **The engine's walk.** `session::VisitModel` walks a model and calls a `ModelVisitor` - (`crates/session/src/visit.rs:33-79`) with typed calls (`u8`, `u32`, `u64`, `f32`, `bool`, - `id`, `text`, `token`, `source_bit_depth`) and record and array events. The records and their - key modules are bound in the `records!` and `variants!` macros (`visit.rs:142-160`, invoked at - `:220` and `:273`), which know - each record's key module and, for a variant, its arm token. -- **The generator chain.** `tools/parameter-metadata` emits JSON documents (`--print`, - `--print-abi-layout`; `--write DIR` and `--check DIR` cover all of them, - `tools/parameter-metadata/src/lib.rs:851-905`); `sdk/codegen/assets.mjs` copies them into - `sdk/assets/` (its `documents` list); `sdk/codegen/generate.mjs` renders - `sdk/src/generated/*.ts` (`expectedModules`, `:150-165`); `scripts/check-sdk-generated.sh` checks - both arrows byte for byte. -- **The JSON Schema document.** `docs/session-v1.schema.json` (`$defs` per record, every object - closed) is hand-authored: it carries types, patterns, domains and cross-field rules (for example - a route into the output must have `follows_mute: false`) that no key list holds. It is checked by - `crates/session/tests/json_contract_artifacts.rs` (closed objects, the `u64` pattern, the - revision ceiling). Nothing checks its key lists against the engine. -- **#1568** adds `keys::MODULES` and each module's `ALL`. - -## Decisions frozen for this slice - -- **D1. The walk names its records.** `ModelVisitor` gains one defaulted method, - `fn record_schema(&mut self, module: &'static str, arm: Option<&'static str>) -> - Result<(), Self::Error> { Ok(()) }`. The `records!` macro calls it right after `record_begin` - with the key module's name and `None`; the `variants!` macro with the module's name and the - arm's token text (`"pan"`, `"matrix"`, `"native"`, `"routed"`, `"default"` and so on). Existing - visitors keep the default, so no canonical byte, wire byte or estimate changes. -- **D2. A third generated document.** `parameter_metadata --print-session-keys` emits - `miso-engine-v1-session-keys.json`, and `--write`/`--check` cover it. It walks every document of - the canonical writer corpus with `WalkOrder::Canonical` and a recording visitor and writes: - - `records`: per `(module, arm)`, the keys in emit order, each with its kind: `integer` (`u8`, - `u32`, `u64`, `source_bit_depth`), `float` (`f32`), `boolean`, `id`, `text`, `token`, `record` - or `array`; - - `parents`: per parent JSON key (the key of the field or array that holds the record; `""` for - the root), the `(module, arm)` entries found under it. - - The generator fails, naming what is missing, when a key of some `keys::MODULES` entry is never - walked, when one `(module, arm)` is walked with two different key lists, or when one leaf name is - walked as both `integer` and `float`. If the corpus does not reach a key, the fix is a new - corpus document written by the Rust canonical writer. -- **D3. The SDK reads it.** `sdk/codegen/assets.mjs` adds the document; `generate.mjs` renders - `sdk/src/generated/session-keys.ts` from it. `session-json.ts` deletes `ROOT_KEYS`, - `INTEGER_KEYS`, `FLOAT_KEYS`, `OBJECT_KEY_ORDERS`, `taggedOrder` and `objectOrder`. Its one rule: - for a record under parent key `k`, take the candidates `parents[k]` and use the one whose key set - equals the record's key set exactly; none or more than one throws `MisoUsageError`. A leaf - number is written as an integer or a float by its walked kind. The canonical output of every - corpus document does not change. `generate.mjs`'s provenance artifact set gains the new - document if it lists the asset documents. -- **D4. The JSON Schema document's key lists are checked, not generated.** Its types, patterns, - domains and conditionals are real authored content, so the document stays hand-written. A new - test in `crates/session/tests/json_contract_artifacts.rs`, - `session_schema_lists_exactly_the_walked_keys`, holds its keys to the walk: a fixed table maps - each `$defs` name to its `(module, arm)` entries (for example `route` → `(route, None)`; `pan` - → `(matrix_or_pan, "pan")`; `effect_identity` → `(identity, "native")` and `(identity, "cid")`); - for each entry, the union of every `required` array in the def (top level and inside `oneOf`, - `anyOf` and `allOf` arms) equals the union of the walked key lists of its entries, and so do the - def's `properties` keys. Every walked `(module, arm)` must appear in the table. A new key - therefore turns the test red until its property schema, with its type and domain, is written. - -## Deliverables - -1. D1 in `crates/session/src/visit.rs`. -2. D2 in `tools/parameter-metadata` (a new module beside `abi_layout.rs`). -3. D3 in `sdk/codegen/{assets.mjs,generate.mjs}`, `sdk/src/internal/session-json.ts`, the new - asset and generated module. -4. D4's test. - -## Authorized paths - -- `crates/session/src/visit.rs`, `crates/session/tests/json_contract_artifacts.rs`. -- `tools/parameter-metadata/src/**`, `tools/parameter-metadata/tests/**`. -- `sdk/codegen/assets.mjs`, `sdk/codegen/generate.mjs`, `sdk/src/internal/session-json.ts`, - `sdk/src/generated/session-keys.ts` (new, generated), `sdk/src/generated/provenance.ts` - (regenerated), `sdk/assets/miso-engine-v1-session-keys.json` (new, generated), `sdk/test/**`. -- `scripts/build-web-audioworklet.sh` and `scripts/sdk-package.sh`, only if they list the asset - documents by name. -- `fixtures/session-canonical/v1/canonical-writer-corpus.json`, only to add a document D2 needs. -- `docs/session-v1.schema.json`, only to fix a key-list difference that D4's test finds on `main`. - -## Non-goals - -- No generation of the JSON Schema document's types, domains or rules, and none of - `docs/SESSION_SCHEMA_V1.md` (prose). -- No change to the SDK's session builder API or its authoring checks in `sdk/src/core/session.ts`. -- No change to a canonical byte, a wire byte or a field ID. - -## Objective gates - -Commands: `cargo test --locked -p session -p parameter-metadata`, -`bash scripts/check-sdk-generated.sh`, `bash scripts/check-sdk-types.sh`, -`bash scripts/check-sdk-headless.sh target/ci/qualification-artifacts`, -`bash scripts/sdk-package.sh check target/ci/qualification-artifacts`. - -1. **The SDK writer is generated.** `check-sdk-generated.sh` passes with the new document and - module, and `sdk/test/builder-evals.mjs`'s canonical writer corpus eval passes unchanged (every - document written byte for byte as the Rust writer writes it). -2. **The generator refuses gaps.** Unit tests in `tools/parameter-metadata`: a corpus with no - `matrix` arm fails naming `matrix_or_pan`/`matrix`; a visitor stub that walks `trim_db` once as - `f32` and once as `u32` fails naming the key. -3. **The schema document is held.** `session_schema_lists_exactly_the_walked_keys` passes. In a - scratch commit, removing `follows_mute` from the `route` def's `required` makes it red (PR - evidence). -4. **A new key reaches the SDK mechanically** (PR evidence): in a scratch commit, add a key to - `keys::route` and to the `Route` walk; regenerating changes only the generated document and - module, and the SDK writes the key in the right place with no hand edit. - -## Test value - -- Gate 1 turns red if the generated table writes a document in a different key order or number - spelling from the Rust writer. -- Gate 2 turns red if the generator silently omits a record arm or picks one spelling for an - ambiguous leaf. -- Gate 3 turns red if the JSON Schema document misses a walked key or lists one the engine does not - write; no test compares them today. -- Gate 4 records that a new key needs no SDK hand edit. - -## Dependencies - -- #1568 (`keys::MODULES` and each module's `ALL`). - -## Supersedes and shrinks - -- None. - -## Amendment (controls coordinator, batch 9, 2026-10-14) - -Checked against `main` at `7feb27800`. - -- **Anchors moved.** `parameter_metadata`'s `run` is `tools/parameter-metadata/src/lib.rs:1107-1160` - (the `--write`/`--check` document list at `:1138-1141`); `expectedModules` is - `sdk/codegen/generate.mjs:888`; the asset list is `sdk/codegen/assets.mjs:42-46`. The `visit.rs` and - `session-json.ts` anchors held. -- **#1568** is in batch 9 (`codex/controls-batch-9s`). After it merges, `keys::MODULES` and each - `ALL` exist. Its table keeps a `control_smoothing` row with no wire message; that does not affect - this slice. -- **D3 needs a tie rule.** #1566 walks the automation target as one variant arm per section. The `own` - and `builtins` arms have the same key set and the same order, so D3's "more than one candidate - throws" would throw for them. Amended rule: when more than one candidate has the record's key set, - all of them must have the same key order and the same leaf kinds, and the writer uses that order. - D2's generator checks this and fails, naming the candidates, when they differ. So the SDK never - meets the tie at run time. -- **The generator needs `session`.** `tools/parameter-metadata/Cargo.toml` has no direct `session` - dependency today. Add it (and update `Cargo.lock`); both are authorized paths for this slice. -- **Edit order.** After #1568. Before #1566 (shared `visit.rs`, `session-json.ts` and the canonical - writer corpus): #1566 then regenerates the key tables and adds its target arms to D4's table. - -## Decision record: S13b session key tables (batch 9) - -**Shipped.** D1: `ModelVisitor::record_schema(module, arm)` (defaulted), called by `records!` and -`rec!` right after `record_begin`; `session::VARIANT_ARMS` lists every variant record's arm. D2: -`tools/parameter-metadata/src/session_keys.rs` (`--print-session-keys`, `Recorder`, `build`, -`load_corpus`). D3: `sdk/assets/miso-engine-v1-session-keys.json`, `sdk/src/generated/session-keys.ts`, -and `session-json.ts` now holds no key list. D4: `session_schema_lists_exactly_the_walked_keys`. -`docs/session-v1.schema.json` needed no edit: the test is green on the current document. - -**Departures from the text, and why.** - -- **Track and submix are variant records.** Their key list differs by `pan` or `matrix`, so the walk - names them with arms `pan` and `matrix` (`records!` takes an optional arm expression). Without it - the generator's "one shape, two key lists" rule fails on any corpus that has both. -- **A new corpus document.** No corpus document walked a pan track or a matrix submix. The Rust test - that writes the corpus gained `canonical-exchanged-strips` (`crates/session/src/canonical.rs`, a - mechanical addition to an unlisted path; the corpus JSON is regenerated from it). -- **`u64` is its own kind.** Kinds are `integer` (`u8`, `u32`, `source_bit_depth`), `u64`, `float`, - `boolean`, `id`, `text`, `token`, `record`, `array`. A `u64` is a JSON string, so the SDK keeps - refusing a bare number there, as the hand copy did. -- **The key tables are not in `--write`/`--check DIR`.** The release artifact set is exactly six - files (`check-web-audioworklet.sh`, the package manifest) and the key tables are an SDK source, not - a shipped artifact. `assets.mjs` renders the document with cargo and leaves it out of a check - against an artifact directory (that check runs no cargo); `session_keys_asset_is_current` holds - the asset to the generator in `cargo test`. -- **Variant-arm coverage.** The generator also fails when an arm in `VARIANT_ARMS` is never walked - (message `matrix_or_pan/matrix`), before the key-level check. -- `serde_json` moved from a dev dependency to a dependency of `parameter-metadata` (the corpus - manifest is read as data); `session` was added; `Cargo.lock` updated. - -**Gates.** (1) `check-sdk-generated.sh` with and without the artifact directory, and the SDK -headless suite (`check-sdk-headless.sh`, 524 pass, 0 fail, including the canonical writer corpus -eval), `check-sdk-types.sh` and `sdk-package.sh check`. (2) `a_corpus_without_the_matrix_arm_is_refused_by_name`, -`a_numeric_key_walked_as_two_kinds_is_refused_by_name`, `one_shape_walked_with_two_key_lists_is_refused`, -`a_tie_must_agree_on_key_order_and_kinds`. (3) `session_schema_lists_exactly_the_walked_keys`; -renaming `follows_mute` in the `route` def's `required` makes it red (`$defs/route: required -differs`). (4) A scratch swap of `mute` and `follows_mute` in the `Route` walk changed only the -generated document (and, by regeneration, its module); reverted. diff --git a/.github/ISSUE_SPECS/1574-preview-a-response-in-the-browser-with-the-same-protocol-frame-as-the-c-abi.md b/.github/ISSUE_SPECS/1574-preview-a-response-in-the-browser-with-the-same-protocol-frame-as-the-c-abi.md deleted file mode 100644 index 42e60949a..000000000 --- a/.github/ISSUE_SPECS/1574-preview-a-response-in-the-browser-with-the-same-protocol-frame-as-the-c-abi.md +++ /dev/null @@ -1,257 +0,0 @@ -# Preview a response in the browser with the same protocol frame as the C ABI - -Slice S17 of decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`, design points C1 -and C13). Code anchors verified on `main` `df585e503`. Feature sprint mode (owner, 2026-10-13): one -implementer, normal tests, a batch review; no per-attempt verifier or mutation runs. - -## Product outcome - -The SDK's response preview (`createResponsePreview`, browser and headless) sends the same -`RESPONSE_GET` frame that a C ABI host sends, and decodes the same reply bytes. The preview -instance runs the core's code for it, and the browser's own request and result records, their -eleven exports and the SDK's layout code are deleted. The public SDK preview API does not change. - -## Context - -- **The browser path today.** The SDK fills `WebResponseRequest` (`hosts/host-web/src/lib.rs:517`) - and `WebResponseParameter` (`:551`) rows in Wasm memory (`ResponsePreviewModule.#query`, - `sdk/src/core/response.ts:337-418`), calls `miso_engine_web_v1_response_query` - (`hosts/host-web/src/ffi.rs:2467`) and reads `WebResponseResult` (`lib.rs:562`) back - (`response.ts:419-472`). The eleven exports are `ffi.rs:2405-2508`. After #1572, - `run_response_query` (`ffi.rs:2184`) only decodes, calls the core and lays out. -- **The preview instance has no session.** The browser runs it in the SDK's response Worker - (`sdk/src/browser/response-worker.ts:69-72`, a standalone `WebAssembly.Instance` with no - imports); headless runs it in its own instance (`sdk/src/headless/response.ts:30`). The public - entry is `createResponsePreview` (`sdk/src/browser/index.ts:19`, `sdk/src/browser/response.ts:199`; - `sdk/src/headless/response.ts:45`). With no session there is no `SessionState`, so #1562's - generic command entry cannot serve it. -- **SDK logic beyond transport.** `responseParameterRows` (`response.ts:267-284`) splits a - per-lane `both` into `left` and `right` rows, because the core refused `both` on a per-lane - parameter before #1573 D4. `normalizedConfiguration` (`:239-265`) fills defaults for the - `requestedConfiguration` echo. Names map to IDs through the generated catalog (`parameterRows`). -- **Mirrors of the old exports.** `tools/parameter-metadata/src/abi_layout.rs` (export names - `:195-205`; `response_request_fields` `:695`, `response_parameter_fields` `:803`, - `response_result_fields` `:820`; structure rows `"responseRequest"` `:1890` and its two - neighbours; constant tables `"responseTargets"` `:2091`, `"responseFields"` `:2116`), - `scripts/check-abi-layout-v1.py:177-187`, `scripts/check-web-audioworklet.sh:263-273`, - `sdk/src/generated/abi.ts` and `sdk/assets/miso-engine-v1-abi-layout.json` (generated). -- **The staging.** `ResponseStaging` (`ffi.rs:109`) holds the preview fields and the track-response - fields together; `live_response_staging_retained_bytes` (`ffi.rs:381-389`) charges both. -- **The wire and the codec.** #1573 freezes `RESPONSE_GET`; #1563's - `sdk/src/core/control-frame.ts` is the SDK's frame codec; `ProtocolCodec::decode_typed_command` - (`crates/protocol/src/typed_frame.rs:368`), `encode_success_response_frame_into` (`:278`) and - `encode_non_ok_response_frame_into` (`:317`) are the Rust side. - -## Decisions frozen for this slice - -- **D1. The session-less core entry.** `host_core::serve_response_frame(request: &[u8], reply: - &mut Vec) -> Result<(), ResponseFrameError>` (feature `control-provider`). It decodes one - command frame with the frame limit `1 << 20`; a frame whose header does not decode is - `ResponseFrameError::Malformed` and writes no reply. Any other message is answered with a - non-OK `UNSUPPORTED_MESSAGE` frame. For `RESPONSE_GET` it requires a `configuration` target - (else `UNAVAILABLE` `response.target.unavailable`) and both `sample_rate_hz` and - `quantum_frames` (else `INVALID_FIELD` `response.rate`), then runs the same private function as - #1573's provider (one function for both: build the query, prepare, check the reply length, - compute, encode). The reply echoes the request ID with revision 0. -- **D2. One export family, justified.** The frozen per-feature decision gives the session-less - preview one bytes-in/bytes-out export family, in `hosts/host-web/src/ffi.rs`, on no render path - and never called by a worklet: - `miso_engine_web_v1_response_frame_ptr(bytes: u32) -> u32` (staging for a request of `bytes` - bytes, `1..=1 << 20`; `0` otherwise), `miso_engine_web_v1_response_frame_submit(bytes: u32) -> - u32` (`RESULT_OK` when a reply frame exists, OK or non-OK; `RESULT_INVALID_ARGUMENT` for - `Malformed`), `miso_engine_web_v1_response_frame_reply_ptr() -> u32`, - `miso_engine_web_v1_response_frame_reply_bytes() -> u32` and - `miso_engine_web_v1_response_frame_close() -> u32` (frees both buffers). -- **D3. Deleted.** The eleven `miso_engine_web_v1_response_*` exports; `run_response_query`, - `response_buffer_budget`, `response_failure`, `response_header_bytes`, `append_f32_values`, - `response_error_code`, `response_grid`, `response_quality`, `response_link_mode` and - `response_channel`; `WebResponseRequest`, `WebResponseParameter`, `WebResponseResult`, - `RESPONSE_TARGET_*`, `RESPONSE_FIELD_*`, `RESPONSE_MAXIMUM_EFFECT_ID_BYTES`, - `RESPONSE_MAXIMUM_PARAMETER_OVERRIDES`, `RESPONSE_MAXIMUM_RESULT_BYTES` and the three - `RESPONSE_*_BYTES` sizes; the preview fields of `ResponseStaging` and their terms in - `live_response_staging_retained_bytes`; every mirror row listed in Context. The grid and channel - constants that the track response still reads stay until #1580. -- **D4. The SDK.** A new `sdk/src/core/response-frame.ts` encodes a `RESPONSE_GET` request and - decodes the `response_result` reply and a non-OK reply's diagnostics, with #1563's TLV - primitives. `ResponsePreviewModule` (`response.ts:287`) calls D2's exports with it: - - a parameter name maps to its ID through the generated catalog as today; `both` is sent as - `both` (`responseParameterRows`' split is deleted); bypass is control 0; input-filter cutoffs - are controls 3 and 4; quality and link mode keep their codes; - - `maximumPreparedBytes`, `maximumTotalStateBytes`, `maximumScratchBytes`, - `maximumAutomationSpansPerBlock` and `maximumResultBytes` become fields 14-18; - - `ResponsePreviewResult` keeps every field; `resultBytes` is the reply frame length; - `requestedConfiguration` keeps the `normalizedConfiguration` echo (presentation only); - - a non-OK reply throws `MisoEngineError` whose `code` is the diagnostic code and whose `result` - is `invalidArgument` (`INVALID_FIELD`, `VALIDATION_FAILED`, `NOT_FOUND`), `unsupported` - (`UNAVAILABLE`), `refusedBudget` (`LIMIT_EXCEEDED`) or `internal` (`INTERNAL`). -- **D5. One wire.** The preview's reply payload equals the C ABI's for the same request. -- **D6. One generated bound.** The ABI layout generator emits a new constant - `maximumResponsePoints` (`host_core::RESPONSE_MAXIMUM_POINTS`, 4096) into - `sdk/src/generated/abi.ts`; #1579 reads the SDK's track-response bounds from it. The SDK does not - check points itself: the engine refuses with `response.points`. - -## Deliverables - -1. D1 in `crates/host-core/src/response.rs` (sharing #1573's serving function) and its export. -2. D2 and D3 in `hosts/host-web` and the mirrors. -3. D4 in `sdk/src/core/` and the SDK evals. - -## Authorized paths - -- `crates/host-core/src/{response.rs,control_provider.rs,lib.rs}`, `crates/host-core/tests/`. -- `hosts/host-web/src/{ffi.rs,lib.rs}`, `hosts/host-web/Cargo.toml` (the `control-provider` - feature of `host-core`, if #1562 has not enabled it), `hosts/host-web/src/tests.rs`. -- `tools/parameter-metadata/src/abi_layout.rs`, `scripts/check-abi-layout-v1.py` and its - self-test fixture, `scripts/check-web-audioworklet.sh`, `sdk/src/generated/abi.ts` and - `sdk/assets/miso-engine-v1-abi-layout.json` (regenerated, never hand-edited). -- `sdk/src/core/{response.ts,response-frame.ts}`, `sdk/test/response-evals.mjs`, - `hosts/host-web/qualification/sdk-response-entry.ts` (the preview rows only). - -## Non-goals - -- No change to the track response or its exports (#1579, #1580). -- No preview through a booted engine's command entry (the preview needs no session). -- No change to the SDK's public preview types. - -## Objective gates - -Commands: `cargo test --locked -p host-core --all-features`, `cargo test --locked -p host-web`, -`bash scripts/check-web-audioworklet.sh`, `python3 -B scripts/check-abi-layout-v1.py`, -`bash scripts/check-sdk-types.sh`, -`bash scripts/check-sdk-headless.sh target/ci/qualification-artifacts`, -`bash scripts/check-sdk-generated.sh target/ci/qualification-artifacts`, -`python3 -B scripts/check-sdk-deletions.py`, -`bash scripts/sdk-package.sh check target/ci/qualification-artifacts`, and the browser -qualification (`npm run qualify -- --artifacts ... --sdk-root ... --browser --check-matrix ---self-test-mutations` in `hosts/host-web/qualification`) for chromium, firefox and webkit. - -1. **One wire.** A host-core test sends one configuration request (EQ, sections, both lanes) to - `serve_response_frame` at 48 kHz and the same request without fields 12-13 to a 48 kHz C ABI - session (#1573's test session); the two reply payloads (after the 48-byte header) are - byte-identical. This is a wire format, so a byte comparison is the right gate. -2. **Session-less refusals.** `serve_response_frame` answers an `address` target with - `response.target.unavailable`, a missing rate with `response.rate`, a `CAPABILITIES_GET` frame - with `UNSUPPORTED_MESSAGE`, and returns `Malformed` for a 10-byte input. -3. **SDK parity.** `sdk/test/response-evals.mjs` passes on the headless and browser legs with the - eval inputs unchanged. One new eval: a per-lane gain given as `both` gives curves bit-equal to - the same value given as `left` and `right`. One new eval: 4097 points throws `MisoEngineError` - with `code` `response.points` and `result` `invalidArgument`. -4. **The old surface is gone.** The ABI layout check and `check-web-audioworklet.sh` list the five - D2 exports and none of the eleven old ones; `check-sdk-deletions.py` refuses the deleted SDK - names. - -## Test value - -- Gate 1 turns red if the browser and the C ABI encode a different reply for one request. -- Gate 2 turns red if the session-less entry serves a target it cannot resolve or invents a rate. -- Gate 3 turns red if the SDK sends a different request than before, or loses the error code. -- Gate 4 turns red if an old export survives in one list while another drops it. - -## Dependencies - -- #1572 and #1573 (the core function and the wire). -- #1563 (the SDK frame codec, `sdk/src/core/control-frame.ts`). -- #1562 (host-web links `control-plane`, so `host-core`'s `control-provider` code is in the - artifact). -- Stream H owns `hosts/host-web` and `sdk/`; `tools/parameter-metadata/src/abi_layout.rs` follows - `docs/handoffs/decision-15-2026-10-05/STREAMS.md` row 105: this slice rebases after those. -- #1610 (session-less response core function), root 2026-10-14. - -## Supersedes and shrinks - -- None beyond #1573's amendment of #763. - -## Amendment (controls coordinator, batch 9, 2026-10-14) - -Checked against `main` at `7feb27800`. - -- **#1610 landed (batch 8) with a different shape from D1's text.** The core function is - `pub(crate) fn serve_response_at(sample_rate_hz: u32, quantum_frames: u32, maximum_reply_bytes: - usize, request: &ResponseGetRequest, session: bool) -> Result` in `crates/host-core/src/control_provider.rs:672` (typed request in, typed - result out). It runs the preflight (target, then the core's checks) and, when `session` is true, - the `response.rate.session` rule. D1's `serve_response_frame` therefore lives in - `control_provider.rs`, not in `response.rs`: it decodes the frame, checks the target and then that - `sample_rate_hz` and `quantum_frames` are present (D1's order), calls - `serve_response_at(rate, quantum, bound, &request, false)`, and encodes the reply. -- **D1 does not name the reply bound.** Proposal: a host-core constant equal to today's browser bound, - 16 MiB (`RESPONSE_MAXIMUM_RESULT_BYTES = 16 << 20`, which D3 deletes from host-web), narrowed by the - request's `maximum_reply_bytes` (field 18), as `serve_checked` already does. Then no request that - the browser serves today is refused for its size. -- **Anchors moved.** `hosts/host-web/src/lib.rs`: `RESPONSE_*` constants `:178-204`, - `WebResponseRequest` `:535`, `WebResponseParameter` `:569`, `WebResponseResult` `:580`, sizes - `:607-611`. `hosts/host-web/src/ffi.rs`: `ResponseStaging` `:123`, - `live_response_staging_retained_bytes` `:400`, the helpers `:838-910`, `response_buffer_budget` - `:2078`, `response_failure` `:2139`, `run_response_query` `:2150`, the eleven exports `:2339-2432`. - D3 also deletes `mod response_budget_tests` (`ffi.rs:4531`), which tests only the deleted budget. - `abi_layout.rs`: export names `:170-180`, field lists `:722`, `:830`, `:847`, structure rows - `:1653-1669`. `typed_frame.rs`: `encode_success_response_frame_into` `:284`, - `encode_non_ok_response_frame_into` `:323`, `decode_typed_command` `:374`. The SDK anchors held. -- **Dependencies.** #1572, #1573, #1563 and #1610 are closed. #1562 is open, but the part this slice - needs is on `main`: host-web links `control-plane`, which builds `host-core` with - `control-provider`. Host-web's `Cargo.toml` names that feature itself rather than rely on feature - unification (an authorized path). -- **The browser's rebuild refusal does not apply.** The preview has no session and sends no - transaction. -- **Edit order.** In `host-core/src/control_provider.rs`: after #1341a1 (batch 9) and #1567; before - #1615. In `hosts/host-web/src/lib.rs`: after #1602 and #1611 (each changes one line there). - -## Decision record: preview over the C ABI's frame (batch 9) - -**Shipped.** - -- `host_core::serve_response_frame(request, reply)` and `ResponseFrameError::Malformed` in - `control_provider.rs` (feature `control-provider`), calling `serve_response_at(.., false)`. It - decodes with a 1 MiB frame limit, writes no reply when the command header does not decode, answers - any other message with a non-OK `UNSUPPORTED_MESSAGE`, a payload that does not decode with its - decode error's status (`protocol.failure`, as the session endpoint does), then checks the target - (`response.target.unavailable`) and the rate and quantum (`response.rate`). The reply echoes the - request ID with revision 0. The reply payload bound is the 16 MiB constant - `SESSIONLESS_REPLY_BYTES` (today's browser bound), lowered by the request's field 18. The reply - frame is encoded with its own codec, whose frame limit is 16 MiB plus the header. -- Five exports in `hosts/host-web/src/ffi.rs`: `response_frame_ptr`, `_submit`, `_reply_ptr`, - `_reply_bytes`, `_close`, over a leaked, destructor-free `ResponseFrameStaging` (request and reply - `Vec`s) that no boot reserves. The eleven old exports, `WebResponse*`, the old constants, the - preview fields of `ResponseStaging`, the helpers and `mod response_budget_tests` are deleted. -- SDK: `sdk/src/core/response-frame.ts` (request encoder, reply decoder, status mapping) and - `ResponsePreviewModule` over it. `both` is sent as `both`. A refusal is a `MisoEngineError` whose - `code` is the first diagnostic's code, whose `diagnostics` hold the codes and whose `result` is the - ABI code of `invalidArgument`, `unsupported`, `refusedBudget` or `internal` (D4). -- Layout generator, `check-abi-layout-v1.py`, its fixture, `check-web-audioworklet.sh` and the SDK - mirrors list the five exports; `maximumResponsePoints` (4096) is a new constant (D6); the three - old `maximumResponse*` constants and the `responseTargets` and `responseFields` tables are gone. - -**Departures.** - -- `RESPONSE_MAXIMUM_EFFECT_ID_BYTES` also sized the observation ID staging. It is now a private - `OBSERVATION_MAXIMUM_ID_BYTES` (127) in `ffi.rs`; the observation export is unchanged. -- Gate 1 lives in `crates/capi/tests/response_get.rs`, not in `host-core/tests/`: it needs the C ABI, - which `host-core` cannot depend on. Gate 2 is `crates/host-core/tests/response_frame.rs`. -- `tools/parameter-metadata` takes `host-core` as a normal dependency (for `RESPONSE_MAXIMUM_POINTS`). - `SdkResponseCode` is added to `MisoErrorCode` in `sdk/src/core/errors.ts`. -- The old eval assertions on `code === "refusedBudget"` follow the engine's own codes now - (`response.reply_too_large`, `response.points`); the eval inputs are unchanged. -- An input-filter request sends all four cutoffs (controls 3 and 4, both lanes), `0` for an absent - one, as the old request did. A bypass row (control 0, lane `both`, value 1) is sent only when set. - -**Gates.** - -- Gate 1: `the_sessionless_frame_entry_answers_the_payload_of_a_session` (capi). -- Gate 2: `a_request_with_a_rate_and_a_quantum_is_answered`, - `a_request_the_entry_cannot_resolve_is_refused_with_its_code`, - `another_message_is_unsupported_and_a_broken_header_has_no_reply` (host-core). -- Export lifecycle: `ffi::response_frame_tests::the_response_frame_exports_stage_answer_and_close`. -- Gate 3: `sdk/test/response-evals.mjs` (the existing evals, plus "a per-lane value given as both - equals the same value on each lane" and "the engine refuses 4097 points with its own code"). -- Gate 4: `check-abi-layout-v1.py` (and `--self-test`), `check-web-audioworklet.sh`, - `check-sdk-deletions.py`. -- Ran green: `cargo test -p host-core --all-features`, `cargo test -p host-web`, - `cargo test -p capi --test response_get`, `cargo clippy -D warnings` on the touched crates, - `check-web-audioworklet.sh`, `test-web-audioworklet.sh`, `check-sdk-types.sh`, - `check-sdk-headless.sh` (526 pass), `check-sdk-generated.sh`, the browser qualification with - `--check-matrix --self-test-mutations` on chromium. Firefox and webkit were not run here. - -## Fold (batch 9 gates) - -- host-web does not name `host-core`'s `control-provider` feature: `scripts/check-host-core-policy.sh` allows only `control-plane` to enable it, and host-web reaches `serve_response_frame` through `control-plane`'s enablement (the batch-9 amendment's line on this is superseded). -- The `serve_response_frame` doc no longer links the private `serve_response_at` (`cargo doc -D warnings`). diff --git a/.github/ISSUE_SPECS/1602-retire-the-input-design-bound-cache-that-no-host-reaches.md b/.github/ISSUE_SPECS/1602-retire-the-input-design-bound-cache-that-no-host-reaches.md deleted file mode 100644 index 5b631c6e5..000000000 --- a/.github/ISSUE_SPECS/1602-retire-the-input-design-bound-cache-that-no-host-reaches.md +++ /dev/null @@ -1,289 +0,0 @@ -# Retire the input-design bound cache that no host reaches - -Successor of #1556, decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`). Code anchors verified on `82b1e05e1` (branch codex/controls-batch-3). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -The engine keeps no cache of input-section design bounds. `InputBoundCache`, its journal, its seed -record and every parameter, field, resource row, test and document line that serves them are -gone. A C ABI engine owns no control-thread memory outside its compile limits, and its header says -so. Every prepared value, report and rendered bit stays the same, because no host passes a design -to the cache today. - -## Context - -- **Why the cache is dead.** Since #1556 every C ABI plan carries the live input lane: - `controls::live_lanes` sets `strip_input` when any strip has a live input-family row - (`crates/controls/src/live_lanes.rs:26-41`, `:50-56`), and the core attaches it on every compile - and rebuild (`crates/control-plane/src/compile.rs:597`). A strip with the live lane reports the - rate's live bound and designs none. `crates/capi/tests/design_bound_cache.rs:305` - (`a_c_abi_plan_bounds_no_input_design`) asserts 0 designs and 0 frames through the exported C - ABI, and `crates/capi/tests/resource_lifecycle.rs:1799-1806` asserts a seed of 0 bytes. - `docs/C_ABI_V1_QUALIFICATION.md:639-644` already calls the machinery dead. -- **The browser never used the cache.** `hosts/host-web/` names no `InputBoundCache` and no - `bound_cache`. Its three preparation calls (`hosts/host-web/src/lib.rs:7193`, `:7210`, `:7226`) - go through host-core entries that pass `None`. Today the browser still designs input bounds - without a cache: `live_lanes` (`hosts/host-web/src/lib.rs:7850-7856`) picks - `HostLiveLanes::FADER_AND_MATRIX` unless the boot word `live_control_command_queue_records` - enables commands. #1562 D5 makes the browser attach `controls::live_lanes(model)` on every plan, - and #1565 D4 deletes the boot word. After both, no host designs an input bound in production. -- **The cache in `builtins`.** - - Type, cap, journal and seed: `crates/builtins/src/tail.rs:396-590` - (`INPUT_BOUND_CACHE_ENTRIES` `:408`, `InputBoundCache` `:427-435`, `CachedDesign` `:439-445`, - `InputBoundRecord` `:453-483`, `begin_journal` `:543`, `take_journal` `:553`, `lookup` `:563`, - `insert` `:575`, `Default` `:586-590`). Doc mentions at `tail.rs:6`, `:221`. - - Re-exports: `crates/builtins/src/lib.rs:47` (`CachedDesign`), `:49-50`. - - The read and fill: `input_section_bounds` and `input_section_bounds_within` - (`crates/builtins/src/lib.rs:3763-3840`) take `cache: Option<&mut InputBoundCache>`; - `test_support::input_section_bounds_within` (`:6034-6041`) passes it on. - - `InputBoundKey` and `input_bound_key` (`tail.rs:161-177`) also key the per-preparation - `designs` map (`lib.rs:3793`). They stay. -- **The cache in `builtins-compiler`.** Import `crates/builtins-compiler/src/lib.rs:28`; the - parameter of `prepare_session_builtins_with_live_controls` (`:4432-4453`) and of its inner - function (`:4548`); the pass to `input_section_bounds` (`:4681-4703`); the test - `a_preparation_walks_at_most_the_budget_and_a_warm_cache_walks_nothing` (`:13713`, cache legs - at `:13746`, `:13776`). -- **The cache in `host-core`.** Re-export `crates/host-core/src/lib.rs:238-241`; import - `crates/host-core/src/prepare.rs:11`; the `bound_cache` parameter of - `prepare_host_runtime_with_live_lanes` (`:1020-1050`) and `_successor` (`:1056-1078`), of the - policy function (`:1494`) and its pass (`:1965`); the test - `a_design_bound_cache_changes_no_prepared_value` (`:3466-3594`). Callers in - `crates/host-core/tests/{effect_observation,live_lanes,live_transaction,prepare}.rs` pass `None`. -- **The cache in `control-plane`.** Re-export `crates/control-plane/src/lib.rs:66-68`. - `crates/control-plane/src/compile.rs`: `PreparedRuntime::bound_seed` (`:29`); - `capi_resources`'s `bound_seed_bytes` parameter (`:164`), its epoch row (`:227-229`) and its - largest-allocation term (`:286`); `prepared_capi_resources`'s parameter (`:311`, `:350`); - `prepare_runtime`'s `bound_cache` with `begin_journal` and `take_journal` (`:574-618`, `:642`, - `:715`); `compile_children`'s `engine_cache` (`:723-761`, `:801`, `:877`). - `crates/control-plane/src/control.rs`: the epoch's `bound_seed` (`:45-49`), `test_bound_seed` - (`:89-92`), the constructors (`:101-111`, `:125-135`) and the rebuild's - `InputBoundCache::with_seed` (`:1146-1183`). -- **The cache in `capi`.** `Engine::bound_cache` (`crates/capi/src/abi.rs:422-436`); the compile - call (`crates/capi/src/ffi.rs:383-387`); `engine_cache` (`crates/capi/src/runtime/mod.rs:97-107`). - Test helpers that call the live-lanes entries: `crates/capi/src/runtime/live_tests.rs:2566`, - `crates/capi/src/runtime/tests.rs:3008`. `crates/capi/Cargo.toml:24` names the cache test. -- **Resource rows and tests in `capi`.** `crates/capi/tests/resource_lifecycle.rs`: - `three_cutoff_session` (`:523-532`), `HostHalf::bound_seed_bytes` (`:682-703`), its term in - `capi_epoch_terms` (`:712-713`, `:743`), the "three input cutoffs" case (`:1001-1003`) and the - zero-seed assertion (`:1799-1806`). `crates/capi/tests/design_bound_cache.rs` (whole file). -- **Header and documents.** The "Engine memory (issue 1471)" paragraph of - `crates/capi/include/miso_engine_v1.h:120-134` states a 4,300,720-byte cache and a 947,248-byte - open journal outside every compile limit. `docs/C_ABI_V1_QUALIFICATION.md:637-690` ("Design-bound - caches (#1471)"). `docs/REALTIME_DEPENDENCY_POLICY.md:206`, `:352`. - `tools/realtime-policy/src/policy.rs:172` lists the test file. -- **Other users.** `crates/builtins/examples/input_bound_budget.rs` (`:19`, `:90`, `:289`, `:324`, - `:560-565`) measures a warm rebuild. `crates/builtins/tests/tail_contract.rs` uses the cache at - `:437-466` and in four cache tests (`:578`, `:600`, `:634`, `:680`). -- **History.** #1457 shipped the cache; #1471 (spec removed at `1d71fe7d0`; read it with - `git show 1d71fe7d0^:.github/ISSUE_SPECS/1471-wire-the-engine-and-session-design-bound-caches-into-the-c-abi.md`) - wired the engine cache and the per-epoch seed into the C ABI. The open #1470 (*Cache design - bounds across browser preparations in the control Worker*) plans the same cache for the browser. - -## Decisions frozen for this slice - -- **D1. The cache is retired, not kept for later.** No host reaches it after #1562 and #1565, and - a live input lane on every strip is decision 17's model (C6). A host that needs fast bounded - designs again re-earns a cache in a new issue. -- **D2. Removed from `builtins`.** `InputBoundCache`, `InputBoundRecord`, `CachedDesign`, - `INPUT_BOUND_CACHE_ENTRIES`, the journal and the seed. `input_section_bounds(rate, strips)` and - `test_support::input_section_bounds_within(rate, strips, budget)` lose the `cache` argument. - The per-preparation dedupe by `InputBoundKey`, the budget, the stopped walk and the live-bound - fallback stay unchanged. -- **D3. Removed parameters.** `prepare_session_builtins_with_live_controls`, - `prepare_host_runtime_with_live_lanes`, `prepare_host_runtime_with_live_lanes_successor`, the - host-core policy function, `prepare_runtime` and `compile_children` lose their cache parameter. - No replacement variant is added. Every caller drops the argument. -- **D4. Removed state and rows.** `Engine::bound_cache`, `PreparedRuntime::bound_seed`, the epoch's - `bound_seed` and `test_bound_seed`, and the `bound_seed_bytes` epoch row and largest-allocation - term in `capi_resources`. The rebuild prepares with no seed. -- **D5. The C ABI guard stays.** `crates/capi/tests/design_bound_cache.rs` becomes - `crates/capi/tests/input_design_bounds.rs`. It keeps `a_c_abi_plan_bounds_no_input_design` - (compile and rebuild bound 0 designs and walk 0 frames) and drops every cache word. -- **D6. Header.** The "Engine memory" paragraph says the engine owns no control-thread memory - outside the compile limits beyond its own handle. The header's ABI, layout and symbols do not - change; `ABI_VERSION` does not change. -- **D7. #1470 is superseded.** The browser has no designs left to cache after #1562 D5. The - coordinator closes #1470 as superseded when this slice's evidence is upstream. - -## Deliverables - -1. D2 in `crates/builtins` (source, `tests/tail_contract.rs`, `examples/input_bound_budget.rs`). -2. D3 in `crates/builtins-compiler`, `crates/host-core` and every caller. -3. D4 and D5 in `crates/control-plane` and `crates/capi`. -4. D6 in `crates/capi/include/miso_engine_v1.h`; the documents and the allow-list follow. -5. The rewritten tests of the gates below. - -## Authorized paths - -- `crates/builtins/src/{tail.rs,lib.rs}`, `crates/builtins/tests/tail_contract.rs`, - `crates/builtins/examples/input_bound_budget.rs`. -- `crates/builtins-compiler/src/lib.rs`. -- `crates/host-core/src/{prepare.rs,lib.rs}`, - `crates/host-core/tests/{effect_observation,live_lanes,live_transaction,prepare}.rs`. -- `crates/control-plane/src/{compile.rs,control.rs,lib.rs}`. -- `crates/capi/src/{abi.rs,ffi.rs,runtime/mod.rs,runtime/live_tests.rs,runtime/tests.rs}`, - `crates/capi/tests/{design_bound_cache.rs,input_design_bounds.rs,resource_lifecycle.rs}`, - `crates/capi/Cargo.toml` (comment only), `crates/capi/include/miso_engine_v1.h` (comments only). -- `tools/realtime-policy/src/policy.rs` (the renamed path only). -- `docs/C_ABI_V1_QUALIFICATION.md`, `docs/REALTIME_DEPENDENCY_POLICY.md`. - -## Non-goals - -- Removing the design-bound computation itself (`input_section_bounds`, `fixed_input_bound`, the - budget, `INPUT_BOUND_SECTION_CHARGE`, `INPUT_BOUND_BUDGET_FRAMES`). Host-core's entries with - `HostLiveLanes::FADER_AND_MATRIX` and the no-lane entries still reach it. Whether those entries - and the computation retire is a separate successor, after #1565. -- Any change to host-web. It never held the cache. -- Any change to the live bound, the tail law or a prepared value. -- Editing closed handoff records under `docs/handoffs/`. - -## Objective gates - -Commands: `cargo test --locked -p builtins --features test-support`, -`cargo test --locked -p builtins-compiler`, `cargo test --locked -p host-core --all-features`, -`cargo test --locked -p control-plane --features test-support`, `cargo test --locked -p capi`, -`cargo build --locked --release -p builtins --features test-support --example input_bound_budget`, -`scripts/check-workspace-policy.sh`, `cargo test --locked -p realtime-policy`, -`cargo build --locked --release -p audit -p capi -p realtime-policy` then -`./target/release/realtime_policy` and `./target/release/audit capi`. - -1. **No cache left.** `git grep -n -E 'InputBoundCache|InputBoundRecord|CachedDesign|INPUT_BOUND_CACHE_ENTRIES|begin_journal|take_journal|bound_seed|bound_cache|engine_cache'` - over `crates/`, `hosts/`, `tools/` and `docs/*.md` returns nothing. This is a review check, not - a committed test. -2. **C ABI guard.** `input_design_bounds.rs`'s `a_c_abi_plan_bounds_no_input_design` passes: a - compile and a structural rebuild of the 64-track fixture bound 0 designs and walk 0 frames. -3. **Budget law unchanged.** `tail_contract.rs`'s - `the_live_bound_is_taken_exactly_when_the_budget_is_exhausted` and - `a_design_walks_at_most_the_budget_it_is_given` pass with the cache legs removed and every - expected value unchanged. The four cache-only tests (`:578`, `:600`, `:634`, `:680`), the - builtins-compiler warm-cache test and host-core's `a_design_bound_cache_changes_no_prepared_value` - are deleted; the builtins-compiler test keeps its budget leg under a new name without "cache". -4. **Resource accounting complete, ceilings lowered.** `resource_lifecycle.rs`'s complete-charge - checks (`assert_capi_retained_bytes_are_complete`, `assert_host_owners_are_charged`) pass for - every fixture with the seed row gone from both the engine's sum and the test's - `capi_epoch_terms`. The "three input cutoffs" case and `bound_seed_bytes` are deleted. The - header and `C_ABI_V1_QUALIFICATION.md` no longer state the 4,300,720-byte cache ceiling or the - 947,248-byte journal: the engine's memory outside the compile limits drops from those figures to - its handle alone. -5. **Realtime.** `audit capi` reports 0 allocations, frees, locks and syscalls on render. -6. **No bit moved.** Before merge, the implementer runs `cargo test --locked -p capi` and - `cargo test --locked -p host-core --all-features` on the base and on the change; every render - comparison test passes on both. This is PR evidence, not a committed test. - -## Test value - -- Gate 2 turns red if a C ABI plan stops attaching the live input lane and designs bounds again - with no cache to absorb the cost. -- Gate 3 turns red if removing the cache argument changes the dedupe, the budget charge or the - live-bound fallback. -- Gate 4 turns red if the seed row is removed from the engine's sum but a retained owner is left - uncharged, or the reverse. -- Deleted tests defend only the cache; their claims leave with it. - -## Dependencies - -- #1556 (every C ABI strip carries the live input lane), on `codex/controls-batch-3`. -- #1562 D5 (the browser attaches the core's live lanes) and #1565 (deletes the boot word). The - cache retirement does not need them to be correct, because the browser never held the cache. - The order follows the brief and the shared file `crates/host-core/src/lib.rs` (#1565 D5 edits - its re-exports). D7's closure of #1470 needs #1562 D5 on `main`. -- Check open specs that edit `crates/host-core/src/prepare.rs` or the live-lanes entries (#1593, - #1401) and declare an edit order before the slice starts. - -## Size - -Half a day to one day. Most of the work is deletion and dropping one argument at each caller. - -## Amendment (controls coordinator, batch 9, 2026-10-14) - -Checked against `main` at `7feb27800`. - -- **Dependencies are met.** #1556 and #1565 are closed. The browser's preparer - (`WebRuntimePreparer`, `hosts/host-web/src/lib.rs:4900-4960`) attaches the lanes that the control - plane chose (#1381a, #1400), so no host designs an input bound in production. The browser - preparation calls of Context (`lib.rs:7193`, `:7210`, `:7226`, `live_lanes` `:7850`) are gone. D7's - closure of #1470 (still open) needs only this slice upstream. -- **#1400 added a trait parameter.** Every preparation now goes through - `control_plane::RuntimePreparer::prepare(..., bound_cache: &mut InputBoundCache<'_>, ...)` - (`crates/control-plane/src/compile.rs:659-672`). Its three implementations: capi - (`crates/capi/src/runtime/mod.rs:92-114`), host-web (`WebRuntimePreparer::prepare`, `lib.rs:4951`, - which ignores `_bound_cache`) and the test preparer in - `crates/control-plane/tests/runtime_preparer.rs:13-75`. D3 also removes this parameter. The - non-goal "Any change to host-web" is replaced: host-web drops the parameter from its preparer, the - doc lines at `lib.rs:4906-4907` and the two test caches in `src/tests.rs:5758`, `:5816`, and changes - nothing else. -- **The cache type now has a lifetime** (`InputBoundCache<'seed>`, `with_seed`). D2 deletes it whole. -- **Anchors moved.** `host-core`: re-export `lib.rs:258`; `prepare_host_runtime_with_live_lanes` - `prepare.rs:1067-1100` and `_successor` `:1103-1130`; the policy function is now - `prepare_host_runtime_with_live_controls_policy_and_spectrum` (`:1667-1677`), its pass `:2159`; the - test `a_design_bound_cache_changes_no_prepared_value` `:4005`. `builtins-compiler`: `:4437-4443`, - `:4540-4548`, test `:13719`. `control-plane`: re-export `lib.rs:69`; `compile.rs` `bound_seed` is a - field of `CompiledChildren` (`:29`), `capi_resources` `:213-294`, `prepared_capi_resources` - `:375-421`, `prepare_runtime` `:698-832`, `compile_children` `:843-923` and `:1008`; `control.rs` - `:54`, `:96`, `:106-116`, `:130-140`, rebuild `:1330-1353`. `capi`: `abi.rs:426-436`, `ffi.rs:385`, - `runtime/mod.rs:92-149`; `tests/design_bound_cache.rs:301`; `tests/resource_lifecycle.rs` - `three_cutoff_session` `:527`, `bound_seed_bytes` `:692`, `capi_epoch_terms` `:722-751`, "three input - cutoffs" `:1031-1033`, the seed assertion `:1848`. Header `miso_engine_v1.h:167-179`. - `docs/C_ABI_V1_QUALIFICATION.md:783-829`; `docs/REALTIME_DEPENDENCY_POLICY.md:211`, `:377`; - `tools/realtime-policy/src/policy.rs:197`. The `builtins` anchors held. -- **Added authorized paths:** `crates/control-plane/tests/runtime_preparer.rs`, - `hosts/host-web/src/{lib.rs,tests.rs}` (the parameter and its doc lines only). -- **Edit order.** After #1341a1/a2 (`control-plane/src/control.rs`). Before #1566 in - `host-core/src/prepare.rs` and before #1574 and #1611 in `hosts/host-web/src/lib.rs`. The open - #1593 also changes `RuntimePreparer` and `host-core/src/prepare.rs`: this slice lands first (it - only deletes), and #1593 rebases. - -## Decision record: #1602 (batch 9) - -**Shipped.** The input-design bound cache is gone, whole. `builtins`: `InputBoundCache`, -`InputBoundRecord`, `CachedDesign`, `INPUT_BOUND_CACHE_ENTRIES`, the journal and the seed are -deleted; `input_section_bounds(rate, strips)` and -`test_support::input_section_bounds_within(rate, strips, budget)` lose the `cache` argument. The -per-preparation dedupe by `InputBoundKey`, the budget, the stopped walk and the live-bound fallback -are unchanged (a spent budget reports the live bound without a call, as the `ChargeAbove(0)` leg did). -`builtins-compiler`, `host-core` (the two live-lanes entries and the policy function), the -`RuntimePreparer::prepare` trait method and `compile_children` in `control-plane`, and -`Engine::bound_cache` in `capi` lose the parameter. `PreparedRuntime`/`CompiledChildren`'s -`bound_seed`, the epoch's `bound_seed` and `test_bound_seed`, and the `bound_seed_bytes` epoch row and -largest-allocation term of `capi_resources` are deleted; the rebuild prepares with no seed. The header's -"Engine memory" paragraph now says the engine owns no control-thread memory outside the compile limits -beyond its own handle; ABI, layout, symbols and `ABI_VERSION` are unchanged. - -**Departures.** -- `hosts/host-web/src/lib.rs` also drops the trailing `None` of its `compile_children` call (one - argument of the same removal); `tests.rs` drops the two test caches. Nothing else in host-web moved. -- Mechanical callers outside the listed paths: `crates/builtins-compiler/tests/allocation_tracker.rs`, - `crates/graph-compiler/src/lib.rs` (a test call), `crates/host-core/tests/monitor_tick.rs`, - `crates/capi/src/runtime/live_peak_tests.rs`. -- `examples/input_bound_budget.rs` measured "design work" as the no-cache preparation less the warm - rebuild. With no cache there is no warm rebuild, so the subtracted leg is now a keying-only - preparation under a budget of 0 (it prepares and keys every strip and walks nothing), which is the - same non-design work. The example builds; it was not run (benchmark campaigns are out of scope), so - `FRAME_EQUIVALENT_NS` and the charge constants are untouched. -- `a_preparation_walks_at_most_the_budget_and_a_warm_cache_walks_nothing` became - `a_preparation_walks_at_most_the_budget` (budget leg unchanged, every expected value unchanged). - -**Deleted tests (they defended only the cache).** `tail_contract`: `a_full_bound_cache_is_cleared_and_reports_the_same_values`, -`a_journal_holds_every_design_a_preparation_used_through_a_clear`, -`a_journal_holds_the_hits_and_the_stopped_walks_a_preparation_used`, -`a_cache_over_a_record_finds_every_design_and_journals_only_what_it_used`; host-core -`a_design_bound_cache_changes_no_prepared_value`; `resource_lifecycle` "three input cutoffs" case, -`three_cutoff_session`, `HostHalf::bound_seed_bytes` and its zero-seed assertion. `design_bound_cache.rs` -became `input_design_bounds.rs` (git rename), keeping `a_c_abi_plan_bounds_no_input_design`. -`tail_contract`'s `the_live_bound_is_taken_exactly_when_the_budget_is_exhausted` keeps its budget -legs (the cold/warm/other-budget cache legs are removed). - -**Ceilings.** No test pins a ceiling this slice moves: `resource_lifecycle`'s complete-charge checks -compare the engine's rows with the allocator and with `capi_epoch_terms`, and the seed row left both -sides (it was 0 bytes on every fixture since #1556, so no measured row changed). The figures that left -the header and `docs/C_ABI_V1_QUALIFICATION.md` were derived worst cases, not measurements, and -nothing reached them: the engine's memory outside the compile limits drops from "at most 4,300,720 -bytes of cache plus 947,248 bytes of open journal" to the engine handle alone. No re-pin. - -**Gates.** (1) the review grep over `crates/ hosts/ tools/ docs/*.md` returns nothing (only closed -`docs/handoffs/` records match). (2) `input_design_bounds::a_c_abi_plan_bounds_no_input_design`. (3) -`tail_contract::the_live_bound_is_taken_exactly_when_the_budget_is_exhausted` and -`::a_design_walks_at_most_the_budget_it_is_given`, `builtins-compiler` -`a_preparation_walks_at_most_the_budget`. (4) `resource_lifecycle` complete-charge tests -(`capi_retained_bytes_charge_every_byte_the_compile_retains` and the replacement oracle). (5) -`audit capi`. Results are in the implementer's report. diff --git a/.github/ISSUE_SPECS/1611-shrink-the-always-attached-per-effect-live-lanes.md b/.github/ISSUE_SPECS/1611-shrink-the-always-attached-per-effect-live-lanes.md deleted file mode 100644 index bcc615325..000000000 --- a/.github/ISSUE_SPECS/1611-shrink-the-always-attached-per-effect-live-lanes.md +++ /dev/null @@ -1,251 +0,0 @@ -# Shrink the always-attached per-effect live lanes - -Successor of #1381a and #1555, decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`, design point C6: the core attaches live lanes on every plan). Code anchors verified on `846b7d240` (the flip branch, #1381a). -Issue number: assigned at filing (written here as #1611). Feature sprint mode (owner, -2026-10-13): one implementer, normal tests, a batch review. The realtime gates below are mandatory -all the same. - -## Product outcome - -The live lanes that C6 attaches on every plan cost much less memory, so the browser's boot budget -returns to its 20 MiB pin (multiplier 20, 20,971,520 bytes for the 1 MiB worst document) and the -C ABI retains less. Every plan still carries its lanes, every live value still works, and render -stays allocation-free with no higher per-block cost. - -## Context - -- **The measurement (flip branch, root, 2026-10-14).** On the 1 MiB worst browser document - (`hosts/host-web/examples/worst_boot_document.rs`: 192 tracks, 768 effects), attaching the C6 - lanes in the browser raised the wasm high-water growth from 16,121,856 to 25,427,968 bytes and - native retained memory from 6,450,366 to 13,449,640 bytes. The growth is: - - per-effect live lanes from `attach_effect_live_controls`, about 4.0 MB: 768 effects x about - 5.2 KB, in blocks of 2,304 + 720 + 720 + 672 + 384 + 192 + 96 + 96 + 60 bytes; - - render-side bank live lanes, about 2.4 MB: 14,080, 1,760 or 1,280 bytes per bank effect x 96, - plus 4,096 bytes per strip x 192, twice; - - the control plane's fixed rows, about 0.4 MB; per-strip input lanes, about 0.17 MB. -- **The pin.** `scripts/check-web-boot-budget.mjs` pins growth to the document length times - `PARSE_TRANSIENT_MULTIPLIER` plus one page (`:9`, `:53-58`). Root ruled the multiplier from 20 - to 25 for now. -- **The per-effect lane.** `attach_effect_live_controls` - (`crates/effect-compiler/src/prepare.rs:1727-1800`) builds one lane per prepared effect with - `effect_control_lane` (`crates/effect-contract/src/live.rs:623-692`). The lane's shape is - `EffectLaneShape` (`live.rs:164-268`: cells, groups and a `first` table as long as the last live - parameter index). The shared `EffectCells` (`live.rs:349-373`) hold one `LatestCell<1>` per - parameter cell, one `LatestCell` per target cell and the dirty - words. The writer holds one `CellWriter` per cell; the lane (`EffectControlLane`, `live.rs:695`) - holds one `CellReader` per cell, one `DirtyReader` and one `pending` word per 64 cells, and one - `PreparedEffectTarget` per target cell. `retained_resources` (`live.rs:568`, `:818`) charges them. -- **The bank lanes.** `LiveControlEffectBankStage` (`crates/rack/src/lib.rs:1151-1185`) holds a - lane per bank lane, a `staging` window of the bank's `automation_capacity` spans and a `packed` - array, all sized at bind. -- **The C ABI ceiling.** `capi_retained_bytes` is 216,832 at both widths - (`crates/capi/tests/resource_lifecycle.rs:1317-1322`). - -## Decisions frozen for this slice - -- **D1. C6 stands.** Every plan carries a lane for every live value, as today. No lane becomes - lazy at commit time and no lane is built on the render thread. -- **D2. The 2,304-byte block first.** The implementer names the owner of each per-effect block - from a measurement (a per-field `retained_resources` breakdown on one EQ, one compressor and - one delay) before changing anything, and records it. Then the per-effect layout is sized from - the effect's own descriptor: its live parameter count and policy, its target slots and its - observation taps, with no array sized by a crate-wide maximum or by the last parameter index - where a dense map serves. Arrays that the writer and the reader each hold per cell may be - merged or made dense when the cell protocol is unchanged. -- **D3. Bank lanes sized by need.** A bank stage's `staging` and `packed` arrays and the per-strip - arrays are sized at bind from what the bank's lanes can actually stage (their live cells and - targets), not from the bank effect's full `automation_capacity`, where the two differ. Sizing - stays at bind (control thread); render reads fixed lengths. -- **D4. Same behaviour.** Every live value, refusal, superseded count, observation and rendered - bit is unchanged. Lane memory may change only in size and layout. -- **D5. The pins follow the measurement.** If the growth reaches 20 x the document length plus - one page, the multiplier returns to 20. Otherwise it is set to the new measured growth divided - by the document length, rounded up to an integer, and the shortfall is recorded with its named - cause. `capi_retained_bytes` moves to the new measured value plus 10 %, rounded up to 64, with - the file's before-and-after doc paragraph. - -## Deliverables - -1. The D2 measurement and D2's layout change in `crates/effect-contract/src/live.rs` and - `crates/effect-compiler/src/prepare.rs`. -2. D3 in `crates/rack/src/lib.rs`. -3. The new pins (D5) and their records. -4. The before and after figures and one descriptive render measurement in this spec's decision - record. - -## Authorized paths - -- `crates/effect-contract/src/live.rs`, `crates/effect-compiler/src/prepare.rs` - (`attach_effect_live_controls` and its helpers only), `crates/rack/src/lib.rs` (the live bank - stage only). -- `scripts/check-web-boot-budget.mjs` (the multiplier only). -- `crates/capi/tests/resource_lifecycle.rs` and `hosts/host-web/tests/retained_ceilings.rs` (the - affected ceilings and their records only). -- This spec's decision record. - -## Non-goals - -- No change to C6, to which plans carry lanes or to the control interface. -- No change to the cell protocol (`LatestCell`, the dirty words, the superseded count). -- No change to the control catalog or the control plane's fixed rows (a separate successor if - they need one). -- No optimisation of render arithmetic. - -## Objective gates - -Commands: `cargo test --locked -p effect-contract`, `cargo test --locked -p effect-compiler`, -`cargo test --locked -p rack`, `cargo test --locked -p capi`, `cargo test --locked -p host-web`, -`node scripts/check-web-boot-budget.mjs `, -`cargo build --locked --release -p realtime-policy && ./target/release/realtime_policy`. - -1. **Boot budget.** `check-web-boot-budget.mjs` passes on the worst document at the D5 - multiplier. Record the growth before and after. -2. **C ABI retained.** `capi_retained_bytes` passes at its new ceiling; every other row of - `resource_lifecycle.rs` stays within its ceiling, and the charge-equals-freed tests stay exact. - Record native retained bytes for the worst document before and after. -3. **No realtime regression.** The existing render allocation counters stay at exactly zero, and - `realtime_policy` passes. -4. **Same behaviour.** Every existing live-control, automation, observation and bank-parity test - stays green unchanged. -5. **Per-block cost (descriptive).** One invocation, one warmup and two measured rounds of the - existing reference render benchmark before and after; record both. Not a gate unless a named - release budget is missed. - -## Test value - -- Gate 1 turns red if the lanes do not shrink, or grow back, on the document the browser pins. -- Gate 2 turns red if a lane array is charged twice or not at all after the layout change. -- Gate 4 turns red if a dense map loses a parameter or a bank's smaller staging window drops a - span. - -## Dependencies - -- The flip batch: #1381a, #1562a, #1570b, #1563B and #1565A. - -## Size - -One implementer-day, one sonnet-high implementer. - -## Amendment (controls coordinator, batch 9, 2026-10-14) - -Checked against `main` at `7feb27800`. - -- **Anchors held** (`prepare.rs:1727`; `live.rs:164`, `:349`, `:568`, `:623`, `:695`, `:818`; - `rack/src/lib.rs:1151`; `resource_lifecycle.rs:1317-1322`, 216,832). -- **The multiplier is in three places, and they move together.** `scripts/check-web-boot-budget.mjs:9` - (25); `PARSE_TRANSIENT_MULTIPLIER` in `hosts/host-web/src/lib.rs:94` (25), the host's typed - pre-parse refusal `host.budget.parse_projection`, which the script's second block checks at the pin - minus one; and `hosts/host-web/src/tests.rs:195`, which pins it. The two host-web lines are added - authorized paths. -- **The figure has not moved since the flip.** A release browser module built after batch 7 - (`codex/controls-batch-8w`, 2026-10-14 12:56 UTC) grows 25,427,968 bytes (388 pages, 24.25 times - the document) on the 1 MiB worst document. That equals the flip-branch figure: #1599, #1600, #1612 - and #1613 did not change it. The 20 pin plus one page is 21,037,056 bytes (321 pages), so this slice - must remove at least 4,390,912 bytes (67 pages) of high-water growth to reach D5's first branch. -- **#1341a1 and #1341a2 change this slice's files** (batch 9): `effect-contract/src/live.rs` - (`BypassSwitch` in a `LatestCell<2>`, the crossfade state on `EffectControlLane`) and the bank stage - in `rack/src/lib.rs` (`BankCrossfade`, inline arrays, no heap). They are a dependency. Measure every - "before" figure on `main` after both merge, not on the flip-branch figures: the D2 per-field - breakdown (one EQ, one compressor, one delay); the boot growth with the release module - (`scripts/build-web-audioworklet.sh`, then `check-web-boot-budget.mjs`); the native retained bytes - of the worst document; `capi_retained_bytes`; and the one descriptive render run. -- **Edit order.** After #1341a1/a2 (`live.rs`, `rack/src/lib.rs`, `effect-compiler/src/prepare.rs`) - and before #1566 in `effect-compiler/src/prepare.rs` (different functions). After #1602 and - before #1574 in `hosts/host-web/src/lib.rs`. - -## Decision record: lane shrink and the boot pin (batch 9, implementer) - -Measured on the branch head (main `7feb27800` + #1341a1/a2 + #1602 + batch-9 docs), x86-64 eight -lanes, release module, 1 MiB worst document (192 tracks, 768 parametric EQs). - -**The cause the spec did not name.** The lane layout is not what drives the wasm high-water. A -counting allocator on the native boot (every request of 200 KB or more, with its stack) found -`controls::live_lanes::live_lanes`, called by `control_plane::compile::prepare_runtime`, collecting -the whole control list (`controls::list`: 53,763 rows of 232 bytes) into one `Vec` grown by -doubling to 15,204,352 bytes, only to read three booleans (`LiveLanes::of_rows`). On wasm, dlmalloc -does not reuse a freed block above its mmap threshold, so the whole doubling chain (118,784 up to -15,204,352 bytes) counts toward the growth. Evidence: D3 alone removed 1.5 MB of retained bytes and -moved the growth by 0 pages (25,427,968 before and after); streaming `live_lanes` one owner at a -time (the way `ControlCatalog::build` already walks the list) moved it to 20,447,232. - -**Boot growth (`check-web-boot-budget.mjs`, shipped simd128 module).** - -| Step | Growth | Pages | -|---|---|---| -| Before (branch head) | 25,427,968 | 388 | -| D3 (bank windows) + shape (D2) only | 25,427,968 | 388 | -| + `live_lanes` streamed (final) | 20,447,232 | 312 | - -The 20 pin plus one page is 21,037,056 bytes (321 pages); the final growth is 589,824 bytes (9 -pages) under it. The multiplier is 20 in the three places (`scripts/check-web-boot-budget.mjs`, -`PARSE_TRANSIENT_MULTIPLIER` in `hosts/host-web/src/lib.rs` with its doc paragraph, -`hosts/host-web/src/tests.rs`). - -**D2 measurement (per-field `retained_resources` and allocation sites).** One parametric EQ lane -(target owner, 6 slots = 12 target cells): shared allocation 192; target cells 12 x 192 = 2,304 -(`LatestCell<12>`, three slots of 64 bytes); dirty word 8; readers 12 x 8 = 96; dirty reader 8; -pending 8; staged targets 12 x 56 = 672 (lane total 3,288); writers 12 x 32 = 384; the control owner -(`effect-compiler/src/control.rs`) holds two 720-byte parameter copies and a 60-byte dirty array -per EQ. The 2,304 + 384 + 96 + 672 are fixed by the frozen cell protocol (one cell per -`(slot, channel)`), so the EQ document gains nothing from a denser per-cell layout. One compressor: -1,900 (cells 14 x 72 = 1,008, records 14 x 8 + 7 x 8 + 7 x 4 = 196). One delay: 1,068. The shape -(`EffectLaneShape`) no longer holds a per-cell record array or a table as long as the last live -index: one 12-byte group per live parameter, found by binary search on the control thread; a -compressor lane loses 112 bytes, a delay lane 72, an EQ lane none. - -**D3.** `LiveControlEffectBankStage::new` sizes `staging` and `packed` by the widest lane's -`parameter_cells()` (refused with `AutomationWindow` when above `automation_capacity`), not by the -bank's capacity. A prepared-target owner holds no span storage: per EQ bank 15,840 bytes -(staging 1,760 + packed 14,080) removed, 96 banks = 1.52 MB. The cap estimate -(`graph-compiler/src/estimate.rs`) charges the same window. Native retained after boot for the worst -document: 13,446,520 -> 11,907,448 bytes (-1,539,072); native peak 19,869,264 -> 19,002,476. - -**Ceilings.** `capi_retained_bytes` is 197,521 before and after: no re-pin (the spec's D5 bullet -assumed it moves). The three graph rows of the reference session moved at eight lanes, so -`resource_lifecycle.rs` re-pins them to measured + 10 % rounded up to 64: `graph_session_plus_plan_bytes` -407,294 -> 314,918 (ceiling 421,248 -> 346,432), `graph_incremental_plan_bytes` 407,222 -> 314,846 -(421,248 -> 346,368), `graph_metadata_bytes` 199,049 -> 106,673 (203,648 -> 117,376). The four-lane -ceilings stand until CI prints the AArch64 values. `retained_ceilings.rs` (browser fixture) did not -move; no edit. - -**Gates.** `cargo test --locked` (all features) on effect-contract, effect-compiler, rack, -graph-compiler (including `bypass_shunt_identity`, `bypass_cohorts`, `bypass_resources`), capi, -controls, host-core, control-plane, graph and host-web: green. New test -`a_live_slot_sizes_its_spans_by_its_lanes_cells_not_the_capacity` (rack, red on a capacity-sized -window: a multi-gigabyte allocation). `effect_control_resource_uses_independent_queue_and_owner_arithmetic` -(graph-compiler) now computes the bank window from the lanes' cells, and takes its -"largest allocation outgrows the plain plan" witness from the per-node owner (`scalar-target`, -window of the 4,096-span capacity), because a bank's window is no longer capacity-sized. -`check-web-audioworklet.sh`, `check-effect-runtime-fixtures.sh`, `check-console-fixtures.sh`, -`check-cross-targets.sh`, `realtime-policy`, clippy `-D warnings` on the touched crates and -`check-sdk-headless.sh` (527 tests): pass. The #1341a2 bank-equals-node tests are unchanged and green. - -**Per-block cost (descriptive, one warmup of 300 blocks, two rounds of 2,000).** The worst document -natively, `render_next`: 103.71 and 103.72 us/block before, 105.49 and 105.81 after (+1.8 %, no -render code changed; the stage reads a shorter window array). Not a gate. - -**Departures and edits outside the authorized paths (all listed for root).** -- `crates/controls/src/live_lanes.rs` (the cause above; behaviour identical, booleans only). -- `crates/graph-compiler/src/estimate.rs` and the arithmetic test in `crates/graph-compiler/src/lib.rs` - (mirror of D3, required by the charge-equals-freed tests). -- `hosts/host-web/src/lib.rs` and `tests.rs` (the multiplier, as the amendment authorizes). - -**Successors found.** -- `EffectBankProcessBlock::check_automation_window` has no production caller now (the rack was its - last); delete it with its row in `tools/console-workload/tests/paired_spans.rs`. -- `graph`'s per-node `LiveControlEffect` still sizes its window by the capacity (`runtime.rs` - ~1273) and the estimate charges it so; the same sizing by cells applies to an unbanked EQ. -- `EffectControlOwner` keeps `committed` and `candidate` copies and a dirty array per EQ (1,500 - bytes, 1.15 MB on the worst document); `candidate` and `dirty` could be built at `begin`. -- `ControlRow` is 232 bytes and `controls::list` still collects the whole list for its callers - (tests); the production callers now walk per owner. - -## Fold (batch 9 review) - -`EffectBankProcessBlock::check_automation_window` and its rows in `paired_spans.rs` are deleted (the -rack was its last caller). `ProcessBlockError::AutomationWindow` stays: the per-node -`EffectProcessBlock::check_automation_window` (`graph`) still raises it, and its doc now says so. -The rack's own `RackError::AutomationWindow` states the cells rule. The size test -`a_live_slot_sizes_its_spans_by_its_lanes_cells_not_the_capacity` now asserts on -`LiveControlEffectBankStage::span_storage_len`, so a regression fails as an assertion and not as an -abort on a huge allocation. diff --git a/.github/ISSUE_SPECS/1615-shrink-the-peak-heap-of-a-live-only-commit.md b/.github/ISSUE_SPECS/1615-shrink-the-peak-heap-of-a-live-only-commit.md deleted file mode 100644 index 7c722602f..000000000 --- a/.github/ISSUE_SPECS/1615-shrink-the-peak-heap-of-a-live-only-commit.md +++ /dev/null @@ -1,250 +0,0 @@ -# Shrink the peak heap of a live-only commit - -Successor of #1613 and #1562, decision 17. Code anchors verified on the flip branch after -`be71c6097`. -Issue number: assigned at filing (written here as #1615). Feature sprint mode (owner, 2026-10-13): -one implementer, normal tests, a batch review. - -## Product outcome - -A live-only transaction on a small session needs a smaller peak of heap at once, so the browser's -first live transaction stops growing the wasm memory in the AudioWorklet's message handler (a -growth that costs a `memory.grow` on the audio thread). The growth is no fault today: the worklet -re-binds its views and the control allocation count reports it (#1562, root ruling A). This slice -removes the cause, it does not change what a transaction does. - -## Context - -- **The measurement** (flip CI fold, native release, `native_staging::control_command`, a counting - allocator, the qualification's nine-edit live transaction on `observation-session.json`; recorded - in #1613's evidence): 754 allocations, 282,694 requested bytes and 167,303 bytes live at the peak, - steady state; 846, 281,634 and 172,790 on the first call. The browser grew 46 to 48 pages on it. -- **The sites** (steady state): - 1. the strip's catalog rows in one scratch `Vec` (232 bytes a row) that grows by - doubling, 7,424 to 59,392 bytes (`crates/controls/src/registry.rs:39` `push_control`, called from - `effect_rows.rs:336` and `strip.rs:269`, `:414`; the scratch is `fresh` in - `ControlCatalog::stage_values`, `crates/host-core/src/control_provider.rs:1413`), about 111 KB - requested and 89 KB live while the last doubling copies; - 2. the replay-cache clone for the prospective response, 65,536 bytes of storage - (`crates/protocol/src/controller.rs:262`, `try_clone_eager`, from `:1869`), and the response - frame staging of `max_response_bytes`, 16,384 bytes (`controller.rs:1851`); - 3. the derived-row scratch of the staging comparison, 4,656 and 9,296 bytes - (`control_provider.rs:1590`); - 4. the canonical text of the one edited owner, old and new, for the estimate arithmetic: 4,096 and - 8,192 bytes twice (`crates/session/src/canonical.rs:194`, `:54`). - -## Decisions to take (the implementer proposes, root rules) - -- **S1. The row scratch.** Reserve the owner's row count before pushing (the catalog already holds - the owner's run, `run`), or keep one scratch for the commit. Either removes the doubling. -- **S2. The replay clone.** The prospective replay cache is cloned eagerly (storage and entries) to - complete one entry. A copy of the one entry's bytes, or completing the entry in the committed cache - at the commit after the preflight, would drop 80 KB. This is the protocol's constant and changes - the "acked-batch" argument of the cache, so it needs its own review; it is the decision to bring to - root before any code. -- **S3. The text of an edited owner.** Writing the old and new text of an owner twice for the - length arithmetic could use the length alone (a counting writer), with no buffer. - -## Objective gates - -- Thread-scoped allocator gate (as #1612 and #1613): the live-only commit's peak live bytes and - requested bytes are ceilings, equal at 64 and 256 tracks, and the nine-edit transaction's peak is - at most one wasm page above what the boot leaves free; the number is measured on this head and the - ceiling set with the slack the repo's allowance asks for. -- The differential of #1613 (full path as oracle) stays green; the catalog, events, replay entry and - revision are unchanged for every transaction. -- Render-locked allocations stay exactly 0. - -## Non-goals - -No change to a classification, a commit, the response bytes or the replay semantics. Growth stays -legal: the worklet re-binds its views after it. - -## Amendment (controls coordinator, batch 9, 2026-10-14) - -Checked against `main` at `7feb27800`. - -- **Anchors moved.** `push_control` is `crates/controls/src/registry.rs:30`; it is called from - `effect_rows.rs:327`, `strip.rs:269` and `:311`, `route.rs:244`, `vca.rs:79` and - `session_rows.rs:83`. `ControlCatalog::stage_values` is `control_provider.rs:1412-1495`: the scratch - `fresh` (`:1465`) is already one `Vec` for the whole commit, cleared per owner and reused for the - derived rows (`:1488-1490`); it grows by doubling inside one owner. Site 3 is not a separate - derived-row scratch: it is the `memo` map (`:1466`), which grows by `try_reserve(1)` in `stage_run` - (`:1642`). `try_clone_eager` is `controller.rs:281`, called in the structural prepare at - `:1874-1876`; the frame staging is `:1858`; the commit installs the clone at `:2318-2340`. Site 4 is - `owner_bytes` (`session/src/canonical.rs:205`), called through `session::canonical_owner_bytes` - (`in_place.rs:100`) from `protocol/src/model.rs:1448-1449`, once for the old owners and once for the - new. -- **Proposed decisions (root rules; the implementer can start on S1 and S3 at once).** - - **S1, the row scratch.** (A) Before the loops, reserve `fresh` once with `try_reserve_exact` to - the largest of the changed owners' run lengths and the derived-row counts (both are known before - any row is made), and reserve `memo` to the largest run. (B) Keep a scratch in the catalog across - commits (needs `&mut self` or a cell). (C) Compare rows as `owner_rows_in` makes them, with no - scratch (a new visitor API in `controls`). **Recommendation: A.** One exact allocation, no - doubling copy, no change outside host-core; it also removes site 3's growth. - - **S2, the replay clone.** (A) Keep it. (B) At prepare, call the read-only - `ReplayCache::plan_preflight` on the committed cache and keep its `ReplayPreflightPlan` - (`evicted`, `removed_bytes`) in the token instead of a cloned cache; at commit, call - `apply_preflight_plan`, `complete` and the final-frame write on the committed cache. While a token - is outstanding the structural generation is odd and every other entry point refuses - (`controller.rs:1545`, `:1706`, `:1743`, `:1775`), so the cache cannot change between the plan - and the commit, and the commit cannot fail (it asserts, as `overwrite_completed_response` does - today). A dropped token has nothing to undo. (C) Reserve in the committed cache at prepare and - release on drop: refused, because an eviction cannot be undone on abort. **Recommendation: B**, - with the frame staging sized to the measured frame length (`plan.response_measure`) instead of - `max_response_bytes`, if the measure gives the exact length (the final frame has the same length; - the commit asserts it). The acked-batch answer does not change: the reply exists only after the - replay entry is written, in the same commit call. The token's layout changes; keep - `size_of::()` with `_resource_layout_reservation`, or re-pin the C ABI - resource row with its record. - - **S3, the owner text.** (A) A counting sink: `JsonWriter` writes through a small trait with a - `String` sink and a byte-count sink, and `owner_bytes` counts. (B) Reuse one `String` for the old - and the new owner (half the bytes). (C) Keep per-owner byte counts in the compiled estimate (more - state to keep exact). **Recommendation: A.** No buffer; the canonical writer corpus test holds - the bytes unchanged. -- **Edit order.** In `protocol/src/controller.rs`: after #1567. In `host-core/src/control_provider.rs`: - after #1341a1, #1567 and #1574. In `session/src/canonical.rs`: after #1566. - -## Ruling (root, 2026-10-14) - -- **S1 (row scratch): option A.** Reserve `fresh` and `memo` once, exactly, to the largest run - before the loops; no `controls` change. -- **S2 (replay clone): option B.** At prepare, run the read-only `plan_preflight` and keep its plan - in the token; at commit, apply the plan on the committed cache, `complete` the entry and write the - final frame; size the frame staging to the measured length. While a token is outstanding every - other entry point refuses (odd generation), so the commit cannot fail. Prepare and commit of one - transaction run in one synchronous submit, so no real second request meets the refusal; the - decision record still states what a concurrent entry gets (a typed refusal, nothing queued, no - ack) and one test shows it. The token keeps its size; any re-pin of the C ABI token row comes - with measured numbers. -- **S3 (owner text): option A.** A counting writer for `owner_bytes`; no buffer. - -## Decision record: #1615 (batch 9) - -Root's rulings held: S1 option A, S2 option B, S3 option A. Measured natively with -`hosts/host-web/tests/control_command_heap.rs` (new, committed as the gate): a thread-scoped counting -allocator around one `native_staging::control_command`, the qualification's nine-edit live -transaction (seven parametric-EQ edits on console slot `eq-simd1`, two left-lane input-filter -cutoffs) on `observation-session.json`, at 1, 64 and 256 tracks. The path is `live`, the response 80 -bytes. "Requested" counts a `realloc` as its whole new size. - -| steady call, one track | allocations | requested bytes | peak live bytes | -|---|---|---|---| -| before (`7feb27800`) | 786 | 293,477 | 168,031 | -| after S1 | 776 | 215,809 | 153,284 | -| after S1 and S3 | 754 | 183,057 | 153,284 | -| after S1, S3 and S2 | 753 | 101,025 | 71,252 | - -The first call (one track) went 844 / 280,733 / 172,790 to 833 / 121,033 / 76,011. The steady call is -the same number at 1, 64 and 256 tracks. Debug and release builds give the same numbers once the -debug-only stage check (`verify_stages`, which builds a full catalog as an oracle) is off for the -test's thread. The browser's pages grown (46 to 48 before) were not measured: this box has no -browser. The peak fell by 96,779 bytes, 1.48 wasm pages, so the first transaction needs about 1.1 -pages instead of 2.6. - -**What remains at the peak (71,252).** The row scratch `fresh`, 39,672 bytes (171 rows of 232 -bytes, S1's one exact allocation: the strip's rows must all exist to be compared), the `memo` map -18,576 bytes, and about 13 KB of the rest (writes, frame, request copy, events). The scratch is the -S1 option A; removing it needs option C (compare rows as they are made), which root did not rule. - -**S1, the row scratch** (`host-core/src/control_provider.rs`, `stage_values`). Before the loops the -stage finds the longest changed-owner run and the largest derived-row count (both known before any -row is made) and reserves `fresh` once with `try_reserve_exact`, and `memo` to -`min(longest, descriptors.len())` (distinct descriptors cannot exceed the catalog's). The try-reserve -in `stage_run` stays and no longer grows. No `controls` change. - -**S2, the replay clone** (`protocol/src/controller.rs`). `prepare_structural_plan` no longer clones the -replay cache. `plan_structural_command` already ran the read-only `plan_preflight` on the committed -cache and put the plan in `StructuralCommandPlan`; the token now keeps that plan (`replay_plan`, -`evicted` and `removed_bytes`) and a copy of the request bytes (`request`, `Box<[u8]>`, request -length). The commit applies the plan to the committed cache (`apply_preflight_plan`), `complete`s the -entry with the final frame and returns the reply, in one call. The frame staging is sized to -`OUTER_HEADER_BYTES + response_measure.length` (the measured length; prepare refuses with the same -`OutputTooSmall` the old max-sized buffer gave when that exceeds `max_response_bytes`) instead of -`max_response_bytes`; the final frame has the same length and the commit asserts it. -`overwrite_completed_response` and the prepare-time `preflight`/`complete` on the clone are gone. - -*Why the commit cannot fail.* The plan is made against the committed cache in -`plan_structural_command`; between it and the commit nothing writes that cache: the token sets the -controller's structural generation odd and `process`, `process_command_frame_into`, -`prepare_command_frame` and `prepare_command_frame_in_place` refuse while it is odd. So at the commit -the cache is the one the plan saw, the eviction is the planned prefix, and `complete` finds its -reservation, an entry slot and the bytes (the plan checked `request + max_response_bytes` fits). The -commit `expect`s those, as `overwrite_completed_response` did. - -*Acked-batch rule, and the cache's guarantees.* A reply exists only after its replay entry is -written: `CommittedCommandFrame` is built after `complete` returns, in the same call that applies the -session commit and the event. Nothing is written to the committed cache before the commit, so a -token dropped or refused (`WrongController`, `StaleGeneration`) leaves the cache as it was, which is -what dropping the clone did. The entry holds the request bytes and the committed frame (the path the -host reported), so an exact replay returns the same bytes, a changed request under the same ID is -`RequestIdReuse`, an evicted one is `ReplayExpired`, and the eviction happens once, at the commit. -New test `a_structural_commit_writes_its_replay_entry_and_evicts_at_the_commit` (one-entry cache: -nothing before the commit, the dropped token changes nothing, replay is exact, eviction at the -commit) and the existing `structural_token_cancel_owner_generation_and_serial_rules_are_affine`. - -*What a concurrent entry gets.* While a token is outstanding every entry point refuses: the frame -entries return `Err(CommandFrameProcessError::PreparedCommandOutstanding)`, the compatibility -`process` answers a `Backpressure` frame, event egress returns `ReliableQueueFull`, and -`queues_mut` panics by design. Nothing is queued, no replay entry is made and no ack is sent; a -replay of a cached entry is refused the same way and answered once the token is committed or dropped. -In the hosts, prepare and commit of one transaction run in one synchronous submit, so no real second -request meets it. The new test shows the replay refusal; the existing one shows the rest. - -*Token size.* The cache handle (88 bytes) became the plan (16) and the request box (16); -`_resource_layout_reservation` grew from 24 to 80, so `size_of::()` is -960 before and after (new test `the_prepared_token_keeps_the_size_the_resource_row_counts`). - -*Departure: the resource row.* The C ABI and browser resource report charged the prepared-protocol -term as response frame + token + the replay cache (the clone) + catalog -(`control-plane/src/compile.rs`, mirrored in `capi/tests/resource_lifecycle.rs`). With no clone that -term overcharged by the cache. It now charges frame + token + a control-frame-sized request copy + -catalog (the request is bounded by the control frame). For the browser (16,384-byte frame, 64 KiB / -32 entries cache) the term fell from 16,384 + 960 + 67,328 + catalog to 16,384 + 960 + 16,384 + -catalog, 50,944 bytes less. The largest single allocation is unchanged (the cache's 65,536-byte -arena is still the cache's own row). The live-peak gate takes this term; the `capi` and `control-plane` -tests are green at the new number. - -**S3, the owner text** (`session/src/canonical.rs`, `value.rs`). `JsonWriter` is generic over a -`Sink` (`String`, or `ByteCount`, which keeps a length), and `owner_bytes` counts. A number is -spelled in a fixed 128-byte stack buffer (`Spelling`) by the same `spell_f32` the `String` path -uses, so the length is the document writer's by construction; the unit test -`the_counting_sink_counts_the_bytes_of_the_string_sink` walks every exponent and the extremes. The -canonical writer corpus test holds the document bytes unchanged. The change saves 22 allocations and -32,752 requested bytes per call (776 to 754, 215,809 to 183,057); the peak number did not move, so -the owner text was not live at the peak. - -**Gates.** `cargo test`: `-p protocol --features test-support`, `-p control-plane --features -test-support`, `-p host-core --all-features`, `-p session`, `-p capi`, `-p host-web --features -host-web/test-support` pass. The heap gate is -`a_live_only_nine_edit_transaction_has_a_small_peak_at_every_size`: steady peak 71,252 against a -ceiling of 73,000, requested 101,025 against 104,000, first call (one track) 76,011 against 78,000, -the steady call equal at 64 and 256 tracks, at most 256 bytes left live. The #1613 differential -(`verify_stages`, the full path as oracle) stays on in every other test and is green. `cargo clippy --D warnings` on the six touched crates, `scripts/run-protocol-allocation-audit.sh`, `audit protocol` -and the four policy scripts pass. Render-locked allocations: no render-path file changed. - -**Successor found.** The first live commit on a larger session still grows with the session (64 -tracks: peak 196,593, 256 tracks: 663,179, against 76,011 on one track), because it builds what the -later calls keep. It is a one-time cost, outside this slice's gate (the steady call), and worth its -own look if a browser session of hundreds of tracks must not grow the heap on its first edit. - -**Browser fixture pin (batch 9 fold).** The browser identity fixture -(`hosts/host-web/tests/browser-v1/expected.json`, `directOracle.setControlTimeline.beforeDisposeStatus.memoryBytes`) -is the one exact field this slice moves: 3,080,192 -> 2,949,120 bytes (47 pages -> 45 pages, 2 pages -fewer), measured with the built `simd128` module through `direct-oracle.mjs`. Bisected by building and -printing the oracle at each first-parent commit of batch 9: `1d6f91246` (#1341a, #1602), `d7fe0470d` -(#1611) and `47676de09` (#1566, #1568, #1569) print 3,080,192; `af8bcad9b` (this slice) prints -2,949,120, and the later merges (#1574, docs, folds) keep it. The other two `memoryBytes` fields (the `simd128` initial and before-dispose statuses) stay 2,883,584, -every PCM digest, status -snapshot and result transcript is unchanged, and only the `resourceCeilings`-budgeted rows move (all -down). This is the live-transaction peak of the SET_CONTROL timeline's nine-edit commits falling by -about 97 kB, as this record measures natively. - -## Ruling (root, 2026-10-14): the token size - -Root reversed "the token keeps its size" (batch 9 review): the token's layout reservation and the exact `size_of == 960` test are deleted. The C ABI prepared-protocol resource row computes `size_of::()` itself, so it follows the smaller token with no re-pin; an exact byte count is kept only for wire, ABI or on-disk formats (AGENTS.md). - -## Fold (batch 9 CI): the heap gate measures with the audited allocator - -`hosts/host-web/tests/control_command_heap.rs` no longer registers its own `unsafe` counting allocator (the realtime policy allows `unsafe` only in audited files). It measures with `bench_support::alloc`: the thread-scoped counters for allocations and requested bytes, and a new per-thread live-heap window (`current_thread_peak_reset`, `current_thread_peak`: live bytes saturating at zero, and their peak). Same numbers: steady peak 71,204 bytes, requested 100,977, at 1, 64 and 256 tracks; first call 75,963 at one track. diff --git a/.github/ISSUE_SPECS/1616-keep-a-one-shot-spectrum-window-out-of-the-continuous-stream.md b/.github/ISSUE_SPECS/1616-keep-a-one-shot-spectrum-window-out-of-the-continuous-stream.md deleted file mode 100644 index 70aa8a7c8..000000000 --- a/.github/ISSUE_SPECS/1616-keep-a-one-shot-spectrum-window-out-of-the-continuous-stream.md +++ /dev/null @@ -1,102 +0,0 @@ -# Keep a one-shot spectrum window out of the continuous stream - -Successor of #1609, decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`). Code -anchors verified on the batch 8 integration branch `codex/controls-batch-8`. Issue number: -assigned at filing (written here as #1616). Feature sprint mode (owner, 2026-10-13): one -implementer, normal tests, a batch review. - -## Product outcome - -A continuous spectrum stream returns only windows of its own stream epoch. A one-shot window that -render completes while control switches the capture to continuous mode is never read as a -continuous window. - -## Context - -- `SpectrumCapture::commit_continuous` (`crates/host-core/src/spectrum.rs`, after #1609) drains the - result queue, parks the one-shot word idle, stores the new epoch and the phase, and stores - `CONTINUOUS_MODE` last. -- Render can publish a one-shot record (its `stream_epoch` is `0`) after that drain and before - the mode store, if the window was in flight. `try_read_continuous_record` pops records with no - epoch check except the invalidation rule, so it returns that record as a continuous window. -- #1609 found this and left it (its D7: continuous mode is not in that slice). -- Continuous epochs start at `1`, so a one-shot record is always distinguishable by epoch. - -## Decisions to take (the implementer proposes, the coordinator rules) - -- **Option A (reader).** `try_read_continuous_record` discards a record whose `stream_epoch` is - not the current `shared.epoch` and continues the bounded pop loop. No render change. -- **Option B (render).** The one-shot completion in `block_end` publishes nothing while the mode - word reads continuous. This adds a load to render and needs a loom case. -- Recommendation: A. It keeps render unchanged and also covers any older-epoch record left by a - restart. - -## Objective gates - -1. A test drives the race deterministically (a one-shot record pushed between the drain and the - mode store, in the #1609 loom model or a unit test that splits the switch at that point) and shows the continuous read never returns - it; the read answers `Warming` or the first window of the new epoch. -2. The #1609 loom model gains one mode-switch case, or the decision record states why option A - needs none. -3. `cargo test --locked -p host-core --all-features`, the host-core loom step, and the host-web - spectrum tests stay green. - -## Test value - -- Gate 1 turns red if a record of another epoch reaches a continuous reader. - -## Authorized paths - -- `crates/host-core/src/spectrum.rs` and its tests; this spec's decision record. - -## Dependencies - -- #1609. Land before #1593, #1595 and #1596 read continuous windows on new paths. - -## Size - -Half a day, one sonnet-high implementer. - -## Decision record: one-shot window out of the continuous stream (batch 8) - -**Decision: option A.** `try_read_continuous_record` (`crates/host-core/src/spectrum.rs`) loads -the stream epoch once before its bounded pop loop and discards every popped record whose -`stream_epoch` is not that epoch. Render is unchanged on the publish side. The pop loop stays -inside the existing `REALTIME_POLICY_BEGIN/END` region and keeps its bound (`available`, frozen at -entry and capped at `SPECTRUM_RESULT_SLOTS`); a discard spends one pop of it, so a read that -meets only stale records answers `Warming` or `Pending`, never a refill chase. - -**What the check covers.** A one-shot record (epoch 0, below every continuous epoch) that render -pushes after the switch's drain and before its mode store; a record of an epoch a restart -(`restart_continuous`, control) ended while render was publishing; and a record of an epoch a -render failure ended. Only control changes the epoch outside a failure, and a failure moves it -forward before render publishes anything of the new epoch, so one load covers the loop. - -**Departure from the text (code is newer than the spec).** #1609 left an invalidation rule in the -same loop: `invalidated`/`invalidated_epoch` discarded one record with an epoch at or below the -failed one. Every failure bumps the epoch, so the new epoch check subsumes that rule and discards -all such records, not one. Keeping both left a dead flag, so the two fields, their stores in -`continuous_fail` and `commit_continuous`, and the guard arm are removed. Render does two fewer -atomic stores on a failure; nothing else on the render side moved. - -**Gates.** -1. `a_one_shot_record_that_lands_after_the_mode_switch_is_never_a_continuous_window` arms a - capture, completes a one-shot window, switches through the collection's replacement path - (`begin_continuous(.., true)`), pushes the epoch-0 record as render's block end would after the - drain, and shows the read answers `Warming` and the next window is epoch 1, sequence 0. - `a_record_of_an_older_epoch_is_discarded_by_the_continuous_read` does the same for an epoch-1 - record after a restart to epoch 2. Both are red with the epoch arm disabled (as are five - failure tests that formerly relied on the invalidation rule). -2. The loom model gains `spectrum_mode_switch_loom` (same module, so the CI step filter - `spectrum_handoff_loom` runs it): render completes and publishes an armed one-shot window while - control replaces the capture with a continuous stream; every interleaving of the push against - the drain and the mode store must return only windows of epoch 1 (a continuous window of the - same samples is a correct result when render already saw the new mode). `mode` is a plain - `std` atomic, but render loads it before its first modelled operation and control stores it - after its last, so loom still places the push before, between and after the drain; the model is - red with the epoch arm disabled. -3. `cargo test --locked -p host-core --all-features`, the loom step, `host-web` with - `test-support`, `realtime-policy`, `check-bench-policy.sh`, clippy `-D warnings` (no features, - all features) and rustdoc `-D warnings` are green. - -No measured numbers: no benchmark was run (feature sprint mode). diff --git a/.github/ISSUE_SPECS/1617-size-the-browser-id-staging-for-source-ids-only.md b/.github/ISSUE_SPECS/1617-size-the-browser-id-staging-for-source-ids-only.md deleted file mode 100644 index 783bfb9d9..000000000 --- a/.github/ISSUE_SPECS/1617-size-the-browser-id-staging-for-source-ids-only.md +++ /dev/null @@ -1,54 +0,0 @@ -# Size the browser ID staging for source IDs only - -Successor of #1565B, decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`). Code -anchors verified on the batch 8 integration branch `codex/controls-batch-8`. Issue number: -assigned at filing (written here as #1617). Feature sprint mode (owner, 2026-10-13): one -implementer, normal tests, a batch review. - -## Product outcome - -The browser host reserves ID staging only for the IDs that still cross it. Since #1565B, only -source IDs do: track, submix, route and VCA IDs reach the SDK through the core's control list. - -## Context - -- `project_buffers` (`hosts/host-web/src/lib.rs`, the boot path near `:1820`) sizes the ID staging - for the longest source, track, submix, route and VCA ID. The test mirrors in - `hosts/host-web/src/tests.rs` (near `:85` and `:827`) repeat that sizing. -- #1565B deleted `live_control_{track,submix}_{count,id}`, the last readers of the non-source IDs. -- Shrinking the staging moves `id_staging_bytes` in the resource report and the `memoryBytes` - rows of `hosts/host-web/tests/browser-v1/expected.json`. - -## Decisions frozen for this slice - -- **D1.** The ID staging is sized by the longest source ID alone. Nothing else changes in the - boot path. -- **D2.** Re-pin `expected.json`'s `memoryBytes` rows from a measured browser run, with the before - and after numbers and the reason in the decision record. The boot-budget multiplier does not - rise. - -## Objective gates - -1. A host-web test boots a document whose longest track, submix, route and VCA IDs are longer than - every source ID and shows `id_staging_bytes` equals the source-ID bound. -2. `cargo test --locked -p host-web --features host-web/test-support`, - `bash scripts/check-web-audioworklet.sh` (boot budget), and the browser qualification for - chromium (firefox and webkit where installed) with the re-pinned rows. - -## Test value - -- Gate 1 turns red if the staging still reserves room for an ID kind that no longer crosses it. - -## Authorized paths - -- `hosts/host-web/src/lib.rs` (the staging sizing only), `hosts/host-web/src/tests.rs`, - `hosts/host-web/tests/browser-v1/expected.json` (re-pinned rows only), this spec's decision - record. - -## Dependencies - -- #1565B. Lane W. - -## Size - -Half a day, one sonnet-high implementer. diff --git a/.github/ISSUE_SPECS/1618-settle-a-one-shot-spectrum-capture-across-a-plan-swap.md b/.github/ISSUE_SPECS/1618-settle-a-one-shot-spectrum-capture-across-a-plan-swap.md deleted file mode 100644 index 7efc82fa6..000000000 --- a/.github/ISSUE_SPECS/1618-settle-a-one-shot-spectrum-capture-across-a-plan-swap.md +++ /dev/null @@ -1,110 +0,0 @@ -# Settle a one-shot spectrum capture across a plan swap - -Successor of #1609 (found by the batch 8 review), decision 17 -(`docs/rulings/one-control-abstraction-2026-10-13.md`). Code anchors verified on the batch 8 -integration branch `codex/controls-batch-8`. Issue number: assigned at filing (written here as -#1618). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. -The defect predates batch 8. - -## Product outcome - -A one-shot spectrum capture that is in flight or held when the plan swaps ends in a defined state: -the caller reads its window, or reads a typed refusal and can arm again. It never waits forever. - -## Context - -- After #1609, render moves a one-shot capture `CAPTURING -> COMPLETE` by a compare-and-swap at - block end, and holds a completed window when a stale record fills the queue - (`crates/host-core/src/spectrum.rs`, the one-shot state word and `one_shot_block_end`). -- At a plan swap the successor plan's spectrum observer starts from its own prepared state. A - window the predecessor held is lost, and the shared state word can stay `CAPTURING` with no - observer that will complete it. The caller's read then answers pending until it cancels. -- The carry chain (#1280 and later) carries effect lanes across a swap, not the spectrum capture. - -## Decisions to take (the implementer proposes, the coordinator rules) - -- **Option A (settle).** At adoption of the successor plan, off the render thread, the control side - moves a predecessor capture that is `CAPTURING` or held to `INVALID` with its generation, so the - reader gets the existing `Invalid` answer and can arm again. -- **Option B (carry).** The successor's observer adopts the predecessor's armed capture and its - held window. This needs a carry slot and a loom case. -- Recommendation: A. A plan swap is a structural edit; a refused window is honest and cheap. - -## Objective gates - -1. A host-core test arms a one-shot capture, swaps the plan before the window completes, and - shows the read answers `Invalid` (option A) and that a new arm completes on the successor. -2. The same with a window held behind a stale record at the swap. -3. The #1609 loom model and race test stay green; render stays allocation-, lock- and - syscall-free. - -## Test value - -- Gates 1 and 2 turn red if a capture can stay pending forever across a swap. - -## Authorized paths - -- `crates/host-core/src/spectrum.rs`, `crates/host-core/src/spectrum_service.rs` and their tests; - the plan-adoption call site in `crates/control-plane` if the settle needs one; this spec's - decision record. - -## Dependencies - -- #1609. Before #1593 (spectrum taps on every plan) makes swaps with taps common. - -## Size - -Half a day, one sonnet-high implementer. - -## Ruling (root, 2026-10-14): rescoped after #1395 - -A carried spectrum observer keeps its one-shot state across a swap (#1395 gate 3). This issue is -rescoped to the captures that do not carry (no successor observer at the same tap, or a refused -hand-over): such a capture settles to `Invalid` with its generation (option A), and a new arm -completes on the successor. It lands after #1395. - -## Decision record: #1618, the settle (batch 8) - -**Shipped (option A, as ruled).** `SpectrumService::adopt_successor(successor, pairing, carried, -analysis)` settles what the caller had on a capture that did not carry; a carried capture is -untouched (#1395a gate 3 stays green). - -- *One-shot.* `SpectrumCapture::settle_unpaired(&mut self, predecessor)` (`spectrum.rs`): when the - predecessor's word is in any phase but `IDLE` (armed, capturing, held with or without a queued - result, complete, invalid) and its mode is one-shot, the successor's fresh capture takes - `(predecessor generation, INVALID)` by one atomic exchange. The reader's next `read` answers - `CaptureInvalid` once (the existing `INVALID` arm of `try_read_record`, which also drains a queued - stale record and parks the word `IDLE`), and a new arm is generation `g + 1`, so no record of the - old arm can pass for it. The arm stays the caller's until that read (`caller_armed` is kept). - The successor's observer moves the word only from `ARMED` or `CAPTURING`, so the exchange is safe - while that plan renders; no render, loom-model or `latest_cell` edit (the #1609 loom model and - race test are unchanged and green). -- *Stream.* A new private `end_stream_state` (shared with `stop_stream`) runs when the stream did - not carry: `stream_active` false, cadence, smoothing and window gone, metadata `Stopped`, and the - caller's `analysis` ends with it. This is #1395b's note: after a lost selection `select_stream` - now answers `StreamInactive` instead of arming a one-shot, and the host selects and starts again. -- *Collection.* When the selected entry did not pair, the successor has no selection (#1395b), so - the caller's arm is released (`caller_armed` false) and a read answers `NotArmed`; the selection - epoch tells the caller to select again. A single capture and a collection never pair, so a kind - change also drops the arm and the stream. -- *Departures.* (1) `adopt_successor` takes `analysis: Option<&mut SpectrumAnalysis>` (the owner of - the stream's history ends with the stream); its only caller is the test harness, updated. (2) A - lost collection selection reads `NotArmed`, not `CaptureInvalid`: no capture exists to hold a - word. (3) The gate sentence "a later `select_stream` restarts the cadence" is met as - `select` + `start_stream` restarting it: with the stream ended, `select_stream` is a typed - `StreamInactive` (it can only restart a running stream). - -**Gates -> tests.** `crates/host-core/tests/successor_swap.rs`, `mod spectrum_carry`: gate 1 -`a_one_shot_that_does_not_carry_settles_invalid_and_a_new_arm_completes` (changed channel mask: one -`CaptureInvalid` at the promotion, none before or after; the new arm completes at a later block, -window equal to a fresh arm on the same plan bit for bit) and -`a_one_shot_after_a_refused_hand_over_settles_invalid`; no false settle -`an_idle_capture_that_does_not_carry_settles_nothing`; stream -`a_lost_selection_ends_the_stream_and_a_new_start_restarts_the_cadence`, -`a_refused_hand_over_installs_the_fresh_capture` (now also `!stream_active`), -`a_one_shot_on_a_lost_selection_is_released`. Gate 2 `spectrum::tests:: -a_window_held_behind_a_stale_record_settles_invalid_when_the_plan_swaps` (held window behind a -stale record, through the service: `CaptureInvalid`, then `NotArmed`, then a new arm completes with -the new observer's window). Gate 3: `spectrum_handoff_loom`, the #1609 race test and gate 4 of -#1395 (`..._allocate_and_free_nothing`, zero calls in `adopt_successor`) stay green. With the -settle disabled the gate 1 tests and the gate 2 test go red. diff --git a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md deleted file mode 100644 index f0704c408..000000000 --- a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md +++ /dev/null @@ -1,177 +0,0 @@ -# Bound the first live commit of a large session - -Successor of #1615, decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`). Code -anchors verified on the batch 9 branch `codex/controls-batch-9k2`. Issue number: assigned at filing -(written here as #1620). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a -batch review. - -## Product outcome - -The first live-only transaction after boot on a large session needs about the same peak of heap as -a later one, so a browser page's first fader move does not grow the wasm memory in the -AudioWorklet's message handler on a 64- or 256-track session. - -## Context - -- #1615 cut the steady live commit of the nine-edit transaction to a 71,252-byte peak, the same at - 1, 64 and 256 tracks (`hosts/host-web/tests/control_command_heap.rs`). -- The first call is not: its peak is 76,011 bytes at 1 track, 196,593 at 64 and 663,179 at 256 - (#1615's decision record). It builds state that later calls reuse. -- On a non-isolated page this runs in the worklet's message handler, where a growth is a - `memory.grow` on the audio thread (counted and reported; #1562, root ruling A). - -## Decisions to take (the implementer proposes, the coordinator rules) - -- **Option A (boot).** Build the reused state during boot or preparation, off the audio-thread - handler's first edit, and charge it in the boot resource report. -- **Option B (lazy, bounded).** Keep it lazy but size it from the session at preparation so the - first call makes one exact allocation. -- First measure which structures make up the first-call excess at 64 and 256 tracks. - -## Objective gates - -1. The heap test gains a first-call row at 64 and 256 tracks with a ceiling near the steady peak - (measured numbers and the reason in the decision record). -2. The boot budget gate (`scripts/check-web-boot-budget.mjs`) still passes at its current pin. -3. `cargo test --locked -p host-web --features host-web/test-support`, `-p control-plane - --features test-support`, `-p host-core --all-features`. - -## Test value - -- Gate 1 turns red if the first live commit grows with the session again. - -## Authorized paths - -- `crates/host-core/src/control_provider.rs`, `crates/control-plane/src/*` (the first-commit state - only), `hosts/host-web/tests/control_command_heap.rs`, this spec's decision record. - -## Dependencies - -- #1615. - -## Size - -Half a day, one sonnet-high implementer. - -## Decision record: #1620 (batch 14b) -- measurement, stopped before coding - -Anchor correction: the heap gate lives at `tools/parameter-metadata/tests/control_command_heap.rs` -(the batch 9 CI fold moved it to use `bench_support::alloc`), not under `hosts/host-web/tests/`. - -Measured on `8b3ae13ec` (debug, `verify_stages(false)`), the nine-edit transaction of that test: - -| tracks | first call: allocations / requested / peak / left | steady call peak | first-call excess | session canonical text | -|---|---|---|---|---| -| 1 | 834 / 129,249 / 77,347 / 1,078 | 72,588 | 4,759 | 5,474 | -| 64 | 1,848 / 549,392 / 197,929 / 1,078 | 72,588 | 125,341 | 96,887 | -| 256 | 4,922 / 1,866,704 / 663,179 / 1,078 | 72,588 | 590,591 | 375,479 | - -**What the excess is.** Not lazily built state that later calls reuse. Traced with a temporary -backtrace hook on every allocation of 4 KiB or more (reverted): the first call leaves the in-place -path (#1613) and takes the full preparation. `SessionStore::prepare_in_place` -(`crates/protocol/src/model.rs`) returns `None` because `session::owners_counts_equal` is false: the -fixture's track holds a sparse `eq-simd1` parameter list (IDs 1 to 6, ID 4 per lane), and the edits -write IDs 65, 66, 67, 81, 82, 83 and ID 4 on `both`, which adds parameter entries. A count change is -not a change of values to the in-place path, so the store clones the whole model, runs -`compile_session` (a second model clone, the validation maps, the stable sorts) and writes the -whole canonical text. At 256 tracks the peak is that text's `String` doubling to 524,288 bytes -(final 376,405) plus about 66 KB of model clones and sort buffers; at 64 tracks the text reaches -131,072. The 1,078 bytes left are the committed model's added parameter entries. From the second -call on, the entries exist, the counts are equal and every call is in place (72,588 at every size). - -**Why neither option fits.** Option A (build the reused state at boot) and Option B (lazy, one exact -allocation) both assume reusable state; there is none. The cost is the full compile path, taken by -any live edit that first writes a parameter the sparse model does not hold, on any revision, not -only the first. The candidate fixes are outside this spec's authorized paths and need a ruling: -(C1) let the in-place path accept a `SetControl` that adds a parameter entry (adjust -`owners_counts_equal` and the in-place resource estimate in `protocol`/`session` so an added entry -is charged exactly); or (C2) densify every effect's parameter list at preparation (changes the -committed model and its canonical snapshot, a contract change). Per the coordinator's ruling the -work stopped here; no code changed. - -## Root ruling (2026-10-15): C1 - -- **Design.** The in-place path admits a `SetControl` that adds a parameter entry, and charges the - added entry exactly. The committed model and the canonical snapshot equal the full path's for - the same transaction. The rejected alternative (C2, fill every parameter list at preparation) - would change the committed model and its canonical snapshot. -- **Authorized paths** gain `crates/session/src/in_place.rs` and `crates/protocol/src/model.rs` - (the in-place admission and its resource estimate), and the heap gate's real home, - `tools/parameter-metadata/tests/control_command_heap.rs`. -- **Gates.** (1) A differential gate: for edits that add parameter entries, the in-place commit's - model and canonical snapshot equal the full path's, bit for bit. (2) The first-call peak at 64 - and 256 tracks is measured and recorded, with a ceiling near the steady peak. (3) The first - call's allocation count does not depend on the track count. - -### Implementation (batch 14b) - -**What shipped (C1).** `session::owners_counts_equal` is replaced by `session::owners_growth` -(`crates/session/src/in_place.rs`): `Some(OwnersGrowth)` when the edited owners hold as many -console entries, inserts and VCA members as the committed ones and no effect's parameter list is -shorter, with the entries added and the longest grown list; `None` for any other change of a count -(a shorter list included, so every item the estimate takes a maximum over only grows). The new -`ResourceEstimate::after_owners_edit` (`crates/session/src/estimate.rs`) charges the growth exactly -as `estimate_session_resources` and `with_canonical_bytes` do: per added entry one -`size_of::()` in the model and one structural item (1,024 bytes, now the named -constant `CANONICAL_BYTES_PER_ITEM`) in the canonical bound; the single largest allocation is the -largest of the committed one, the new canonical bound and the grown lists. It runs the compile's -own `check_caps` (now `pub(crate)`, its unused model argument dropped) and the platform and -canonical-bound checks on the result. `SessionStore::prepare_in_place` -(`crates/protocol/src/model.rs`) uses both and passes the store's caps: a growth that does not fit -returns `None` and the full path gives its typed refusal before any commit (no ack before a drop). -The entries land where the full path puts them: `controls`' parameter write already keeps the list -in canonical `(ID, channel)` order, the order `compile_session` sorts into. - -Outside the listed paths (mechanical): the in-place admission comment in -`crates/control-plane/src/control.rs` (the in-place transaction may now add parameter entries, never -a count `check_counts` reads); `crates/session/src/estimate.rs` (`ResourceEstimate::after_owners_edit` -and the named `CANONICAL_BYTES_PER_ITEM`), `crates/session/src/compile.rs` (`check_caps` made -`pub(crate)`, its unused model argument dropped) and `crates/session/src/lib.rs` (the exports); -`crates/session/tests/in_place.rs` moved to the new function; the differential gate lives in -`crates/host-core/tests/in_place_commit.rs`, the home of the #1613 `Pair` harness that compares the -two paths. - -**Gates.** - -1. Differential: `in_place_commit::edits_that_add_parameter_entries_commit_in_place_as_the_full_path_does` - (console EQ above IDs 1 to 6 on one lane and both; an empty insert EQ list gaining an entry, then - one below and one between; a split of a `both` entry; one transaction over seven owners of tracks - and the submix, console and inserts; an abandoned one). Every transaction is prepared and - classified in place and the stores (model, canonical snapshot, estimate, revision) equal the - full path's after each. Refusal: `an_added_entry_over_the_caps_is_refused_before_any_commit` - (a store whose compiled-model or single-allocation cap is the committed figure: a fader commits - in place, an added entry is refused by both entries with the same `Validation` error and the - store is unchanged). Session unit: `owners_with_values_the_compiler_refuses_are_not_valid_and_counts_are_compared`. -2. `tools/parameter-metadata/tests/control_command_heap.rs` (debug): the first call is gated at - every size, `FIRST_PEAK_CEILING` 74,000 and `FIRST_REQUESTED_CEILING` 105,000 (was 78,000 peak - at one track only). Re-pinned for #1620 from these measurements, on this head before and after - the change: - - | tracks | first call before: allocations / requested / peak | first call after | steady (unchanged) | - |---|---|---|---| - | 1 | 839 / 130,294 / 77,600 | 757 / 103,033 / 72,865 | 758 / 103,918 / 72,841 | - | 64 | 1,853 / 550,437 / 198,182 | 757 / 103,033 / 72,865 | the same | - | 256 | 4,927 / 1,867,749 / 663,179 | 757 / 103,033 / 72,865 | the same | - - The steady ceilings (73,000 peak, 104,000 requested) were kept here; the batch 14b review fold - re-pinned all four (below). -3. The same test asserts the first call's allocation count equal at 1, 64 and 256 tracks. - -**Review fold (batch 14b).** (1) The comments that still said the in-place form takes only existing -values now say it also adds parameter entries: `protocol/src/model.rs` (`TransactionForm::InPlace`, -`prepare_transaction_in_place`), `protocol/src/controller.rs`, `control-plane/src/control.rs` -(`commit_live`) and `host-core/tests/in_place_commit.rs`. (2) `control_command_heap.rs` drops the -`first.left <= 256` check: it had no unique catch (the first in-place commit frees the boot's -canonical text, and the live-bytes window stops at zero), so the left-bytes claim is removed from -gate 2 above; `steady.left <= 256` stays. (3) `OwnersGrowth`'s fields are private with `const` -getters (`parameters()`, `largest_grown_list()`) and it no longer derives `Default`: only -`owners_growth` builds one, so no caller outside `session` can hand the public -`ResourceEstimate::after_owners_edit` a growth smaller than the owners' real one; the session tests -compare the getters' pair. (4) The heap ceilings follow one slack rule (measured plus 2,000 bytes, -rounded up to the next 1,000): first call 75,000 peak / 106,000 requested, steady 75,000 / 106,000 -(were 74,000 / 105,000 and 73,000 / 104,000); measured, debug, every size: first 757 / 103,033 / -72,865, steady 758 / 103,918 / 72,841 (the #1624 re-pin, recorded there). (5) Original gate 2 -(`scripts/check-web-boot-budget.mjs`) was not run in attempt 1; run in this fold on a fresh -`build-web-audioworklet.sh --module-only` simd128 module (`765e340fa583b48c`, 4,096,024 B, built from this fold's working tree): `node -scripts/check-web-boot-budget.mjs` passes at its current pin ("web boot budget high-water gate -passed; mismatches: 0"). diff --git a/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md b/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md deleted file mode 100644 index edff35760..000000000 --- a/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md +++ /dev/null @@ -1,103 +0,0 @@ -# Size the per-node live effect window by its live cells - -Successor of #1611 (batch 9 review), decision 17. Code anchors verified on the batch 9 branch `codex/controls-batch-9`. Issue number: assigned at filing (written here as #1621). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -An effect that renders per node reserves span storage for its live parameter cells only, as a -banked effect does since #1611, so an unbanked EQ (no parameter cells) holds no span window. - -## Context - -- #1611 sized the live bank stage's window by the widest lane's `parameter_cells()` - (`crates/rack/src/lib.rs`, `LiveControlEffectBankStage::new`) and the bank charge with it - (`crates/graph-compiler/src/estimate.rs`). -- The per-node `LiveControlEffect` window in `crates/graph/src/runtime.rs` (near `:1225-1273`) and - its estimate still reserve `PreparedEffectMetadata::automation_capacity` spans for any lane with - a live channel. - -## Decisions frozen for this slice - -- **D1.** The per-node window holds the lane's parameter cells, the same rule as the bank stage; - a lane with no parameter cells holds none. The estimate charges exactly that. -- **D2.** Per-lane arithmetic does not change; bank equals node stays bit-exact (#1341a2 tests). - -## Objective gates - -1. A graph test shows an unbanked EQ's per-node window is empty and a compressor's equals its - cells; it fails if the window returns to the capacity (an assertion, not an allocation abort). -2. The charge-equals-freed tests and `bypass_shunt_identity`/`bypass_cohorts` stay green; - affected resource ceilings are re-pinned only from measured numbers, recorded. -3. `cargo test --locked -p graph -p graph-compiler -p rack -p capi`; clippy `-D warnings`. - -## Authorized paths - -- `crates/graph/src/runtime.rs` (the per-node window), `crates/graph-compiler/src/estimate.rs`, their - tests, `crates/capi/tests/resource_lifecycle.rs` (affected ceilings), this spec's record. - -## Dependencies - -- #1611. - -## Size - -Half a day. - -## Decision record: #1621 (batch 14b) - -**Shipped.** `LiveControlEffect::new` (`crates/graph/src/runtime.rs`) sizes its staging window by -`control.parameter_cells()`: an unbanked EQ (prepared-target owner, no parameter cell) and a -channel-less lane hold no span storage; a compressor-shaped lane holds exactly its cells. The estimate -(`effect_control_resource`, `crates/graph-compiler/src/estimate.rs`) charges the same -`parameter_cells * size_of::()`. Per-lane arithmetic is untouched. - -**Departure (one, mechanical).** The bind-time check `EffectProcessBlock::check_automation_window` -demanded `window == automation_capacity`, which the cell-sized window cannot meet. The banked stage -already replaced that equality with "the lanes' cells never exceed the capacity" (#1611 D3). The same -rule now holds per node: `check_automation_window` (in `crates/effect-contract/src/lib.rs`) accepts a -window `<= automation_capacity`; the other half of the contract, that the window holds every cell the -lane reads (no drop, #1345 D4), stays `EffectControlLane::check_window` and still runs at bind. The -pairing-rule argument is unchanged: the hazard (#1012) was a window larger than the capacity, which -could stage a twin pair across the effect's `span_index < automation_capacity` cut-off; a window no -longer than the capacity cannot. Doc comments in `effect-contract/src/lib.rs` and `live.rs` say so. -Ack-before-drop: no. A cell-sized window holds one span per cell, `check_window` refuses a shorter -one at bind, and preparation raises `automation_capacity` to at least the descriptor's cells. - -**Gates and tests.** -- Gate 1: `graph` `tests::a_per_node_window_is_its_lanes_cells_not_the_capacity` (EQ-shaped lane: 0; - compressor-shaped lane: 4 = its cells; gain fixture lane: 2; channel-less lane: 0; capacity 4,096). - `graph-compiler` `bypass_resources::a_per_node_live_window_does_not_grow_with_the_automation_capacity` - (real EQ, compressor and limiter inserts; allocator-measured retained bytes and the estimate are equal - at 128 and 4,096 spans). Both are red when the window returns to the capacity (checked by mutation). -- Gate 2: `graph-compiler` `effect_control_resource_uses_independent_queue_and_owner_arithmetic` follows - the new window formula (scalar-target window 0). Its `expected.largest > plain` assertion for the - scalar case was removed: it held only because the capacity-sized window was the largest owner - allocation; no per-node owner allocation outgrows the plain plan's largest any more, and the - exact-cap accept/refuse legs and the `largest` equality stay. The charge-equals-freed test - `a_bypass_or_live_controls_are_charged_at_least_what_they_retain`, `bypass_shunt_identity` and - `bypass_cohorts` stay green. -- Gate 3: `cargo test --locked` for graph, graph-compiler, rack, effect-contract, effect-compiler, - control-plane, controls, host-core (all features), host-web (all features), capi (all, incl. - `resource_lifecycle`) green; clippy `-D warnings` and fmt clean; `realtime-policy` ok; - `audit capi` `pcm_digest` `bf7dcedd1d92621f`; `check-browser-expected-resources.py` green. - -**Measured numbers.** No pinned row or ceiling moved, so none was re-pinned. Before/after on the same -tree: `resource_lifecycle` nine-track EQ, soft-clip and browser-identity rows identical (EQ plan 218,401 -observed, ceilings unchanged); the browser fixture's eight budget rows identical (for example -`graphIncrementalPlanBytes` 30,154). The one printed row that moved is the unpinned "routed submix" -plan row printed by `capi_retained_bytes_charge_every_byte_the_compile_retains`: observed 104,589 -> -98,781 and its estimate 196,180 -> 190,372 (-5,808 each, slack 91,591 unchanged; the per-node EQ/ -compressor windows of that session, 128-span capacity). - -**Review fold (batch 14b).** B1: `console-workload` `paired_spans` -`every_launch_effect_refuses_a_staging_window_that_is_not_its_capacity` still refused a window of -`capacity - 1`, which this issue made legal; it is now -`every_launch_effect_refuses_a_staging_window_longer_than_its_capacity`: it raises the direct -factory request to the live cells as preparation does (D4), accepts a window of the capacity and of -`EffectLaneShape::live_parameter_cells`, and refuses `capacity + 1`. The lower bound is the lane's -(`effect-contract` `live_window` `a_window_shorter_than_the_live_cells_is_refused_with_the_count_it_needs`). -Red when `check_window` returns `Ok(())` unconditionally (checked by mutation). Stale prose fixed in -`effect-contract/src/lib.rs` (`check_window` is the per-node bound, not "shared by both block types") -and `effect-compiler/tests/native_session.rs` (the window is the lane's cells, which the capacity -equals there). Gate 3 adds `cargo test --locked --release -p console-workload` (green; `paired_spans` -13 pass). diff --git a/.github/ISSUE_SPECS/1622-carry-the-bypass-ramp-in-the-bypass-record.md b/.github/ISSUE_SPECS/1622-carry-the-bypass-ramp-in-the-bypass-record.md deleted file mode 100644 index ce0a02421..000000000 --- a/.github/ISSUE_SPECS/1622-carry-the-bypass-ramp-in-the-bypass-record.md +++ /dev/null @@ -1,71 +0,0 @@ -# Carry the bypass ramp in the bypass record - -Successor of #1341a1 (batch 9 review), decision 17. Code anchors verified on the batch 9 branch `codex/controls-batch-9`. Issue number: assigned at filing (written here as #1622). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -A bypass has one spelling from the classifier to the cell: the record carries its ramp, so no -side field travels beside it. - -## Context - -- `EffectControlRecord::Bypass(bool)` (`crates/effect-contract/src/live.rs:144`, mirrored in - `symmetry.rs:144`) is the step spelling; the ramp-carrying unit is `BypassSwitch`, written by - `write_bypass`/`preflight_bypass` (`live.rs:542`). The classifier carries the session ramp in a - side field, `LiveEffectRecords::bypass_ramp_samples` (`crates/host-core/src/live_delta.rs:134`). -- #1341a1 kept the record's shape because the browser host matched it; since #1565A the browser has - no bypass command, and #1393 closes with no work, so the reason is gone. - -## Decisions frozen for this slice - -- **D1.** `EffectControlRecord::Bypass` carries `BypassSwitch { bypassed, ramp_samples }`; - `bypass_ramp_samples` is deleted; one write path remains. -- **D2.** No rendered bit moves (the #1341 crossfade tests and `audit capi` pcm digest unchanged). - -## Objective gates - -1. Every #1341a1/a2 test passes unchanged in meaning; `git grep bypass_ramp_samples` is empty in - code. -2. `cargo test --locked -p effect-contract -p host-core --all-features -p control-plane - --features test-support -p capi -p host-web --features host-web/test-support`; clippy. - -## Authorized paths - -- `crates/effect-contract/src/{live.rs,symmetry.rs}`, `crates/host-core/src/live_delta.rs`, - `crates/control-plane/src/control.rs`, their tests, this spec's record. - -## Dependencies - -- #1341a1, #1341a2. - -## Size - -Half a day. - -## Decision record: #1622 (batch 10) - -Shipped. `EffectControlRecord::Bypass` now carries `BypassSwitch { bypassed, ramp_samples }`. -`LiveEffectRecords::bypass_ramp_samples` is deleted; the classifier builds the record with the -session's `BypassCrossfade` ramp, and the control plane writes the record as it is. - -- **One write path.** `EffectCellWriter::write_bypass`, `EffectControlProducer::write_bypass` and - `EffectControlProducer::preflight_bypass` are deleted. `EffectCellWriter::address` runs - `BypassSwitch::check`, so `check`/`preflight`/`write` refuse a ramp over - `INDEXED_RAMP_LENGTH_MAXIMUM` as `EffectWriteRefusal::Ramp` with nothing written, as - `write_bypass` did. A step is `BypassSwitch::step(bool)`. -- **Departure from the filed text.** Anchors had moved (#1280a added carry code): the carry code - reads the cell words, not the record, so it needed no change. The mirror in `symmetry.rs` is only - `SymmetryEvent::Bypass(bool)` (a separate type, kept); `EffectControlRecord::symmetry_event` now - reads `switch.bypassed`. Callers outside the authorized paths were updated mechanically (test - and workload call sites that spelled `Bypass(bool)` or `write_bypass`): `capi` live_tests, - `graph`, `graph-compiler`, `rack`, `host-core` tests, `effect-contract` tests/support, - `host-core/src/limiter_linked_session.rs`, `effect-compiler/src/prepare.rs` and - `tools/console-workload`. #1280c's `write_bypass` call in host-core `prepare.rs` becomes - `write(stamp, EffectControlRecord::Bypass(switch))` at merge. -- **Gates.** `a_bypass_flip_carries_the_sessions_mute_ramp_and_nothing_else` now reads the ramp - from the record. effect-contract, rack, host-core (all features), control-plane (test-support), - capi, host-web (test-support), and graph-compiler `bypass_shunt_identity` + `bypass_cohorts` - pass; clippy `-D warnings` and fmt clean. `git grep bypass_ramp_samples` is empty in code. -- **D2.** `audit capi` `pcm_digest` is `9bc5ec3d8a4c223c` both before (stash of this change, same - tree) and after; the digest `76525e7ef9e78cf4` named in the brief does not hold on this base. - No rendered bit moved by this change. diff --git a/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md b/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md deleted file mode 100644 index b559cbc48..000000000 --- a/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md +++ /dev/null @@ -1,144 +0,0 @@ -# Generate the SDK's RESPONSE_GET codes and drop its own checks - -Successor of #1574 (batch 9 review), decision 17. Code anchors verified on the batch 9 branch `codex/controls-batch-9`. Issue number: assigned at filing (written here as #1623). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -The browser SDK's response preview is a thin client: its frame field IDs and enum codes come from -the engine's protocol schema, and every refusal (grid, quality, addressing) is the core's, with the -same code the C ABI gives. - -## Context - -- `sdk/src/core/response-frame.ts` copies the `RESPONSE_GET` field IDs and the grid, lane, quality - and link codes by hand. -- `sdk/src/core/response.ts` refuses a logarithmic grid with `minimumHz <= 0` (`:311`), a - non-finite endpoint (`:310`), any quality but normal (`:193`), and checks shared or per-lane - addressing; a browser caller gets `MisoUsageError` where a C ABI caller gets the core's code. -- #1569 generated the session key tables from the engine's walk; the same generator path - (`tools/parameter-metadata`, `sdk/codegen`) can emit the response message table from - `crates/protocol/src/schema.rs`. - -## Decisions frozen for this slice - -- **D1.** A generated SDK module holds the `RESPONSE_GET` request and reply field IDs and codes, - from `protocol::schema`; the hand copies are deleted. -- **D2.** The SDK's grid, quality and addressing checks are deleted; it encodes only what it - cannot encode as a usage error (a value with no wire spelling). The core refuses the rest. - -## Objective gates - -1. An SDK eval sends a logarithmic grid with `minimumHz = 0`, a non-normal quality and a bad lane, - and gets the core's codes, equal to the C ABI's for the same frames. -2. `check-sdk-generated.sh` fails if the generated table drifts from the schema. -3. SDK headless, types and package checks; `cargo test -p parameter-metadata`. - -## Authorized paths - -- `sdk/src/core/{response.ts,response-frame.ts}`, `sdk/codegen/*`, `sdk/src/generated/*`, - `sdk/assets/*` (generated), `sdk/test/response-evals.mjs`, `tools/parameter-metadata/**`. - -## Dependencies - -- #1574, #1569. - -## Size - -Half a day to a day. - -## Decision record: #1623 (batch 14b) - -Shipped. Anchors held except as noted (`response.ts` had moved: the checks were in `#query` and the -`gridName`/`qualityName`/`channelName`/`asF32` helpers). - -**D1.** The protocol crate gained `protocol::response_frame_table()` (`crates/protocol/src/frame_table.rs`). -It reads the field IDs, wire types and mandatory flags from the `schema.rs` specs (a name is paired -with its spec constant and the function refuses a message whose names do not cover its spec in -order) and reads each code table (configuration kind, grid, lane, quality, link mode) from the -strict reader that refuses codes outside it. `parameter-metadata --print-response-frame` writes it -as `sdk/assets/miso-engine-v1-response-frame.json`, a source-only SDK asset like the session keys; -`sdk/codegen/generate.mjs` emits `sdk/src/generated/response-frame.ts`. `response-frame.ts` takes -every ID, wire type, mandatory flag and code from it; its field helpers take a table row of one -literal wire type, so a schema change of a field's type is a `tsc` error. The hand copies -(`GRID_CODES`, `LANE_CODES`, `QUALITY_CODES`, `LINK_CODES`, `KIND_*`, every field number and -`MESSAGE_RESPONSE_GET`) are deleted. - -**D2.** Deleted from the SDK: the grid-kind, finite-endpoint, logarithmic-minimum, point-count, -rate and quantum, quality, channel-name-default-as-check, link-mode-fallback, request-ID-zero, -shared-lane and parameter-domain checks of the response path, the unknown-effect and -no-response-capability throws, the `1 << 20` request-frame bound and the `16 << 20` and `>= 1` -bounds on `maximumResultBytes`/`maximumAutomationSpansPerBlock`. What stays is a value with no wire -spelling, as `MisoUsageError`: a name the code tables lack, a number its field cannot carry -(`requireU32`/`requireU64`), a `fields` selector, a parameter name or label the catalog lacks, a -non-number parameter value, and the SDK's own `requestDeadlineMs` timer bound. - -**Departures and mechanical edits outside the authorized paths.** -- `crates/protocol`: `Wire::name`, `ResponseConfigurationKind` (a `control_code_table!` replacing the - literal `1`/`2` of the configuration kind in the encoder and decoder; same wire, same refusal), - `frame_table.rs`, two re-exports. Needed because `schema.rs` is crate-private. -- `tools/parameter-metadata/Cargo.toml`: `protocol` moved from dev-dependencies to dependencies - (already in the lockfile through `host-web`). -- `sdk/src/core/session.ts`: `parameterWireRows`, the encode-only half of `parameterRows` - (`parameterRows` itself is unchanged in behavior); the response path no longer calls the - authoring checks. -- A response limit the caller leaves out is now a field left out of the frame, so the core's own - default applies (the SDK defaults `1 << 20`, `1 << 30`, `1 << 30`, `4096`, `16 << 20` equalled the - core's and are gone). The frame of a default query is therefore shorter; the replies are equal. -- Rust rows added so the codes are held on both sides: `crates/capi/tests/response_get.rs` - (draft and high quality) and `crates/host-core/tests/response_frame.rs` - (`the_core_refuses_what_a_thin_client_sends_as_given`, `a_non_finite_endpoint_sent_raw_is_refused_by_the_decoder`). -- `effect()` still refuses draft/high quality, a shared parameter on one lane and an out-of-domain - value at declaration: that is session authoring (`session.ts`), not the response path, and a - response preview of a hand-built configuration now reaches the core. - -**Gates.** -1. `sdk/test/response-evals.mjs`: "a grid, a rate, a quality and a value the SDK used to refuse are - the core's refusals" (14 rows: logarithmic axis from zero `response.grid`, NaN and infinite - endpoints `protocol.failure`, draft and high quality `effect.quality.unsupported`, ...), - "a lane given twice is refused by the core with the C ABI's code" (`response.override.duplicate`, - raw frames), "a name or a number with no wire spelling stays a usage error". The codes equal the - rows of `capi` `every_reachable_refusal_is_typed_and_changes_nothing` and host-core - `the_core_refuses_what_a_thin_client_sends_as_given`. The session-less web entry and a C ABI - session cannot take one literal frame (the session endpoint refuses a rate and a quantum in the - request), so equality is by request content and the shared `ResponseQueryError::refusal` table. -2. `bash scripts/check-sdk-generated.sh` fails on a hand edit of the asset or of the generated - module (both tried red); `parameter-metadata` `response_frame_asset_is_current` holds the asset - to the generator. Protocol test `frame_table::tests::every_code_is_the_one_the_encoder_writes` - holds each name to the code its encoder writes. -3. `check-sdk-headless.sh` (563 tests pass), `check-sdk-types.sh`, `sdk-package.sh check`, - `check-sdk-deletions.py` (and `--self-test`), `check-workspace-policy.sh`, - `cargo test --locked -p parameter-metadata -p protocol`, host-core `response_frame`, capi - `response_get`; clippy `-D warnings` and fmt clean on protocol, parameter-metadata, host-core, - capi. - -**Successor found.** `control-frame.ts` still holds the `WIRE_*` type codes, `FLAG_REVISION_ANY`, -the header magic and the status names by hand; the same frame table could carry them. - -**Review fold (batch 14b).** M1: `check-sdk-generated.sh` with an artifact directory skips the -SOURCE_ONLY assets (session keys, response frame), and an sdk-only change routes to the `sdk` job, so -that job now also runs `cargo test --locked -p parameter-metadata --test response_frame --test -session_keys` (a step inside the existing job: no expectation-table change; `check-ci-path-routing`, -`check-test-support-ci` and `check-script-reachability` pass). M2 (root ruling, thick core): the -codec's decoder no longer refuses a request's value rules (an empty effect ID, a NaN or infinite -endpoint or override value, a bypass row off `both` or not 0/1); the encoder still does, since the -core refuses every such request too, and the wire format is unchanged. host-core -`check_response_request` refuses a non-finite endpoint with `response.grid` and a non-finite override -value or input-filter cutoff with the new `response.override.value` (`ResponseQueryError::OverrideValue`, -`INVALID_FIELD`), after the axis and before any shaping; an empty effect ID is `response.effect.unknown`, -a bypass off `both` `response.override.lane`, a bypass between 0 and 1 `response.override.bypass`. -Registered in `docs/CONTROL_PROTOCOL_REGISTRY.md` (the SDK lists no codes). Tests: protocol -`response_get_request_value_rules_are_refused_on_encode_only`; host-core -`a_non_finite_or_bad_value_sent_raw_is_refused_with_the_core_code`, `bounds_pass_at_the_limit_and_refuse_one_past_it` -(override value and cutoff legs), `response_get_asks_the_core_first_and_names_a_bad_bypass`; host-web -`control_parity` `a_response_request_with_a_bad_value_is_refused_alike_on_both_hosts` (one frame, -both hosts: same result, reply bytes, code, unchanged control rows and document); SDK -`response-evals.mjs` gate-1 rows now `response.grid`/`response.override.value`, plus a raw-frame test -for the empty ID and the bypass rows. All red with the decoder check restored or the core check -removed (checked by mutation). Minors: `putF32` refuses a non-number with `MisoUsageError`, and -`response.ts` no longer coerces (`Math.fround` on the grid removed; `cutoff()` passes a non-number -on); a reply with more sections than the generated capability declares is an `internal` error; the -duplicate eval row "a shared parameter on one lane" is deleted and the left-lane comment no longer -overclaims; capi `every_reachable_refusal_is_typed_and_changes_nothing` pins `effect.parameter.initial`. -Successor (with `control-frame.ts`): `sdk/src/core/live-response.ts:227-234` and -`sdk/src/browser/engine.ts:271-286` (the strip-target and live response paths) still hand-code -grid/lane codes and keep their own grid checks (finite endpoints, logarithmic minimum). diff --git a/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md b/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md deleted file mode 100644 index af9a23dd8..000000000 --- a/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md +++ /dev/null @@ -1,125 +0,0 @@ -# Build the effect control owner's candidate at begin - -Successor of #1611 (its decision record), decision 17. Code anchors verified on the batch 9 branch `codex/controls-batch-9`. Issue number: assigned at filing (written here as #1624). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -An effect control owner (the EQ's target owner) keeps no candidate copy and dirty array between -transactions, so the browser's boot and retained memory shrink by about 1.15 MB on the 1 MiB worst -document. - -## Context - -- `EffectControlOwner` keeps a `candidate` copy and a dirty array per EQ, about 780 bytes each, - about 600 KB to 1.15 MB on the worst document (#1611's decision record). -- Both are used only while a transaction stages a value. - -## Decisions to take (the implementer proposes, the coordinator rules) - -- **Option A.** Build the candidate and the dirty array in `begin` from one control-thread scratch - the control plane owns (one per session, sized to the largest owner), and drop them at commit. -- **Option B.** Keep them per owner but allocate on first use and charge them. -- Recommendation: A. Render is not affected; the owner is control-thread state. - -## Objective gates - -1. Boot growth (`scripts/check-web-boot-budget.mjs`) and native retained bytes fall by the - measured amount, recorded; the multiplier 20 holds. -2. The EQ live-edit tests (C ABI and browser parity) pass unchanged. -3. `cargo test --locked -p effect-contract -p control-plane --features test-support -p capi -p - host-web --features host-web/test-support`. - -## Authorized paths - -- `crates/effect-contract/src/live.rs` (the owner), `crates/control-plane/src/control.rs`, their - tests, resource ceilings with measured numbers, this spec's record. - -## Dependencies - -- #1611. - -## Size - -Half a day. - -## Decision record: #1624 (batch 14b) - -**Stopped once before coding.** The spec's Option A (one scratch sized to the largest owner) cannot -hold: `commit_live` step 3 begins every touched EQ owner before any commits, so the open candidates -overlap (up to every EQ, 768 on the worst document). The owner is in -`crates/effect-compiler/src/control.rs`, not in the spec's paths. The coordinator ruled option B' -and approved the added paths (`effect-compiler/src/control.rs`, `prepare.rs`, -`host-core/tests/prepare.rs`, the snapshot doc in `control-plane/src/control.rs`). - -**Shipped (B').** `EffectControlOwner` holds `candidate: Option` (rows + touched mask). -`begin` builds it from the committed rows; `commit`, `discard` and a publication fault drop it, and -`OpenOwners::drop` already discards every begun owner on a refusal. `candidate()` and `dirty()` -return an empty slice while no candidate is open. `resource_facts` drops the two terms (an idle -owner retains the box and its committed rows only). `new` keeps a transient all-false mask for its -validation call. - -**Departure.** The coordinator asked for the rows and mask in one allocation. They are two exact -`Box<[T]>` (720 and 60 bytes per EQ): `EffectTargetRequest` takes `values: &[InitialParameterValue]` -and `changed: &[bool]` as separate slices, and one allocation would need `unsafe` or a contract -change. Cost: two allocations per begun owner per transaction. - -**Measured (x86-64, eight lanes).** - -| Figure | Before | After | -|---|---|---| -| Wasm boot growth, 1 MiB worst document (`check-web-boot-budget.mjs`, shipped simd128 module) | 20,512,768 (313 pages) | 19,857,408 (303 pages), -655,360; the multiplier 20 holds (pin plus one page is 21,037,056) | -| `capi_retained_bytes`, 9-EQ reference session | 166,374 | 159,354 (-7,020 = 9 x 780) | -| effect-control resources (producer table + owner payload) | 19,377 | 12,357 | -| routed-submix `capi` observed (one EQ) | 136,634 | 135,854 | -| Native retained, worst document | not run | 768 EQs x 780 = 599,040 bytes by arithmetic | -| `control_command_heap` steady, 1 track (seven EQ edits on one EQ): allocations / requested / peak | 756 / 103,138 / 72,841 | 758 / 103,918 / 72,841 | -| `control_command_heap` first call, 1 track: allocations / requested / peak | 837 / 129,514 / 77,600 | 839 / 130,294 / 77,600 | - -The peak is unmoved (it is reached elsewhere in the call than the open owner); the steady requested -bytes are 82 under their 104,000 ceiling, so a second EQ in that transaction would break it (the -ceiling is the test's own and was not raised). - -**Re-pins ("#1624", ceilings only fall).** `resource_lifecycle.rs`: `capi_retained_bytes` ceiling -182,400 -> 175,296 at both widths (159,354 + 10 %, up to 64); `REFERENCE_EFFECT_CONTROL_BUDGET` -21,376 -> 13,632 (12,357 + 10 %, up to 64). The four-lane value is the same (lane-independent -rows; CI prints confirm). - -**Gates.** `cargo test --locked` green for effect-compiler, effect-contract, control-plane -(`--all-features`), host-core (`--all-features`), host-web (`--all-features`; includes -`render_locked_*` and `retained_ceilings`, unchanged), capi (including `reference_session_retained_rows_stay_within_their_budgets` -and `capi_retained_bytes_charge_every_byte_the_compile_retains`), and -`parameter-metadata --test control_command_heap`. The EQ live-edit tests (C ABI and browser parity) -pass unchanged. New test (effect-compiler): -`the_candidate_lives_only_between_begin_and_commit_or_discard`, red if `begin` leaves nothing or -`discard` / `commit` keep the candidate (it asserts empty slices when idle and a fresh committed -copy after a discard). `revision_overflow_is_refused_before_candidate_mutation` now asserts no -candidate is built on a refused begin. Updated arithmetic: `prepare.rs` resource tests and -`host-core/tests/prepare.rs` drop the candidate and mask terms. `realtime-policy` ok; clippy -`-D warnings` and fmt clean on the touched crates. - -**Review fold (batch 14b).** (a) `begin` allocates fallibly: the rows and the touched mask are -`try_reserve_exact`'d, then filled within that capacity and boxed in place, and a failed -reservation refuses with the new `EffectControlOwnerError::Allocation`, leaving the owner idle and -unchanged. `commit_live` (`crates/control-plane/src/control.rs`, step 3) maps it, as every failed -`try_reserve_exact` of that commit, to `CommandError::Internal` before the stamp: nothing is written -or acknowledged, and `OpenOwners` discards the owners begun so far. (b) The candidate lives inside -the phase: a private `OwnerState` enum (`Idle`, `Open(candidate)`, `Poisoned(candidate)`, -`Faulted`, `Published { candidate, next_revision }`) replaces the `candidate`, `pending_revision` -and `phase` fields, so a phase and a candidate cannot disagree and the two "impossible state" -error paths are gone; `phase()` still returns the public `EffectControlOwnerPhase`. (c) -`the_candidate_lives_only_between_begin_and_commit_or_discard` moved to `prepare.rs`'s -`owner_tests` (it needs the `OwnerRig` to publish): it now publishes and commits, and asserts the -candidate is held while published and gone after the commit, with the committed rows equal to the -edited candidate. The always-true `resource_facts` comparison is dropped. Red evidence: a `commit` -that leaves the published state (candidate kept) fails it at the post-commit phase check. (d) -`tools/parameter-metadata/tests/control_command_heap.rs` (debug, `verify_stages(false)`), measured -at 1, 64 and 256 tracks: steady 758 allocations / 103,918 requested / 72,841 peak, first call 757 / -103,033 / 72,865. Re-pinned under the file's one slack rule, now written down (measured plus 2,000 -bytes, rounded up to the next 1,000), reason "#1624 (B' adds 780 B and 2 allocations per begun EQ -owner)": steady peak 73,000 -> 75,000 and requested 104,000 -> 106,000 (room 159 and 82 bytes -before, 2,159 and 2,082 after); first call 74,000 -> 75,000 and 105,000 -> 106,000. The doc's old -#1615 figures (71,252 / 101,025 / 753) are kept as history beside the current measurement. (e) The -655,360-byte saving above is boot and idle retention only. A transaction that begins every EQ of -the worst document still allocates about 780 bytes per EQ for its candidates, transiently, so the -wasm high-water mark of such a transaction gains about 600 KB (768 EQs x 780 = 599,040 bytes) -over the idle figure. diff --git a/.github/ISSUE_SPECS/1626-validate-automation-lanes-in-the-session.md b/.github/ISSUE_SPECS/1626-validate-automation-lanes-in-the-session.md deleted file mode 100644 index f29816dc0..000000000 --- a/.github/ISSUE_SPECS/1626-validate-automation-lanes-in-the-session.md +++ /dev/null @@ -1,283 +0,0 @@ -# Validate stored automation lanes in the session crate and state the hold rule - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answer A3, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. - -## Product outcome - -Every session the engine accepts gives each automated lane exactly one renderable curve. Session -validation, which every path runs (both host boots, every C ABI transaction, `session-validator` -stage `typed-model`), refuses three shapes it accepts today: - -- two automation entries that drive one lane (a `both` fader ride and a `left` fader ride on one - track); -- a pan target on a strip that declares a matrix, or a matrix target on a strip that declares a - pan; -- two discontinuities of one entry closer than 64 samples. - -`docs/SESSION_SCHEMA_V1.md` states what the table means: the hold rule of A3, and these rules. -This slice reads no descriptor; unit, domain and shape rules are #1645 and -#1566's row rule (`controls::automation_diagnostics`). Nothing renders -the table yet. - -## Context - -- **What session validation checks today.** `validate_automation` - (`crates/session/src/validate.rs:890-1030`) checks that the target exists, that segments are - ordered and do not overlap (`:972-1000`), the unit-local value rules (`validate_unit_value`, - `:1032-1062`) and that exponential values are positive (`:1013-1027`). Across entries it checks - only that automation IDs are unique (`:134-145`). Nothing compares two entries' targets, so two - entries on one lane validate. -- **The builtin target check** (`validate_builtin_automation_target`, `:854-888`) checks the - parameter ID against `BUILTIN_AUTOMATION_TARGETS` (`:823-840`; pan is 12, the matrix rows are - 7-10) and that a matrix row is addressed `both`. It does not look at the strip. -- **A strip is either a pan or a matrix.** `MatrixOrPan::{Pan, Matrix}` - (`crates/session/src/model.rs:655-680`) is one tagged field of a track or a submix; both drive - the strip's one 2x2 matrix stage. -- **Validation runs on every path, before normalization.** `compile_session` calls - `validate_session` before it sorts anything (`crates/session/src/compile.rs:123-129`; the table - is sorted by `id` at `:155-157`), so diagnostic indices are declared positions. The protocol - compiles every transaction candidate (`crates/protocol/src/model.rs:944`), so the C ABI live path - runs these rules too. -- **Session diagnostic vocabulary.** Codes are variants of `DiagnosticCode` - (`crates/session/src/diagnostic.rs:17-76`) with a stable dotted spelling in `as_str` - (`:78-115`); the automation codes are `automation.out_of_order`, `automation.segment_overlap` - and `automation.invalid_range` (`:104-106`). No separate frozen list exists: - `docs/SESSION_SCHEMA_V1.md` names each code where it states the rule (for example `vca.cycle`, - `:74`), and the `author-session` skill keeps refusal tables by code - (`.claude/skills/author-session/SKILL.md:129-136`, `:189-199`) and an automation paragraph - (`:107-116`). An effect parameter absent from its instance is `reference.missing_entity` at - `.target.parameter_id` (`validate.rs:944-956`). -- **The grid.** Render evaluates a moving curve on a fixed 64-sample grid and takes a jump at each - discontinuity's exact sample (note A1.4). 64 is the input filters' and the EQ's fixed - coefficient ramp (`crates/builtins/src/filter_control.rs:9`; - `crates/parametric-eq/src/lib.rs:116-119`) and the smoothing length of 55 of the 56 launch - `Block` parameters. -- **Checked-in documents with automation.** `fixtures/session/v1/builtins-automation.json` - automates track `vocal`, which declares a `pan`, with a fader ride, a **`matrix_ll` (7)** ride, - a polarity step and a trim ride. Its SDK twin `builtinsAutomationFixture` - (`sdk/test/builder-evals.mjs:1104-1140`) is compared with it byte for byte (`:1041-1048`). - `tools/session-validator/tests/validate.rs:269-305` mutates the fixture's target rows, the - matrix entry among them. `fixtures/session/v1/canonical.json` and the writer corpus's - full-surface document automate an insert; `.claude/skills/author-session/worked-session.json` a - console EQ gain. None of them has two entries on a lane or a discontinuity pair. -- **The hold rule has no home in the schema doc.** `docs/SESSION_SCHEMA_V1.md:220-225` says only - that the table renders nothing. The protocol's record rule ("holds its end value until - replaced", `docs/CONTROL_PROTOCOL_REGISTRY.md:45`) leaves the registry with *Delete the - protocol automation queue, its records and counters* (#1629). - -## Decisions frozen for this slice - -- **D1. One entry per lane.** - - Two entries share a lane when their `entity_id`, `rack`, `effect_id` and `parameter_id` are - equal and their channels overlap: `both` overlaps every channel, `left` overlaps `left`, - `right` overlaps `right`. A `left` and a `right` entry on one parameter are two lanes. - - Of each overlapping pair, the entry whose `id` is greater (later in canonical order) is - refused, once however many entries it overlaps, with a new - `DiagnosticCode::AutomationDuplicateLane`, spelled `automation.duplicate_lane`, at - `$.automation[].target.channel`, `` its declared position. So the refused entry does not - depend on declared order. - - The pass is one sort of `(entity_id, rack, effect_id, parameter_id, id)` keys, `O(n log n)`. - It skips an entry whose target is already refused and an entry whose `id` repeats an earlier - one (already `id.duplicate`), so no entry is reported twice. -- **D2. Pan only on a pan strip, matrix only on a matrix strip.** A `builtins` target with - `parameter_id` 12 is valid only when its strip's `matrix_or_pan` is `Pan`; 7-10 only when it is - `Matrix`. Else `reference.missing_entity` at `$.automation[].target.parameter_id`, the code - and path an effect parameter absent from its instance gets: the strip has no such parameter. - With D1, a strip's 2x2 stage is driven by at most one entry per lane of one family. -- **D3. Discontinuities at least 64 samples apart.** - - Under the hold rule (D4), sample `t` of an entry is a **discontinuity** when the value at `t` - differs, by `f32` bits, from the value just before `t`. Only segment boundaries can be one: a - segment's `start_sample` whose `start_value` differs from the value held just before it (the - previous segment's `end_value`; never the first segment, whose value is held before it), and - a `step` segment's `end_sample` when its `end_value` differs from its `start_value`. Two - boundaries at one sample are one discontinuity. - - Two discontinuities of one entry less than 64 samples apart are refused with a new - `DiagnosticCode::AutomationDiscontinuitySpacing`, `automation.discontinuity_spacing`, at the - later one's field (`$.automation[].segments[].start_sample` or `.end_sample`), once per - offending pair. - - 64 is the least grid period `G` of A1.4 (`G` is 64, or the cell's ramp length when that is - larger), a fixed engine constant, not a session value. The rule allows at most one jump per - lane in any 64-sample window, so no jump starts inside a fixed 64-sample coefficient ramp that - an earlier jump of the same lane began, and the jump count of A1.7 is at most `⌈q/64⌉ + 1` per - cell and block. -- **D4. The schema doc states the meaning.** In `docs/SESSION_SCHEMA_V1.md`, after `:225`: - - **The hold rule.** Before the first segment a lane holds the first `start_value`. Inside a - segment it follows the segment: `linear` is `v0 + (v1 - v0)·x`, `exponential` is - `v0·(v1/v0)^x`, with `x = (t - t0)/(t1 - t0)`. After a segment, and in a gap, it holds that - segment's `end_value`. A `step` segment holds `start_value` on `[start, end)`, and the lane - takes `end_value` at `end`. - - D1-D3 with their codes, and a pointer to the preparation rules (unit, domain, shape and filter - order against the target's row: #1645 and #1566's row rule). - - The sentence that the table renders nothing stays; #1647 changes it. - - The skill's automation paragraph gains the hold rule, and a new "Automation refusals, by - code" table, in the form of the VCA table, lists D1-D3. -- **D5. Fixture migration.** `builtins-automation.json` gains a second track, `keys`, that - declares a `matrix`, reads the same source and routes to `main-out`. The `matrix-ll` entry moves - to `keys`; `vocal` keeps its pan. The engine's canonical writer writes the file, and - `builtinsAutomationFixture` gains the same track, so the SDK twin stays byte-identical. The - validator rows at `validate.rs:269-305` keep their find strings, stages and codes. -- **D6. The acked-batch question** does not arise: every refusal is a validation diagnostic before - anything is committed. No queue changes. - -## Deliverables - -1. D1-D3 in `crates/session/src/{validate,diagnostic}.rs`. -2. D4 in `docs/SESSION_SCHEMA_V1.md` and `.claude/skills/author-session/SKILL.md`. -3. D5's fixture and its SDK twin. -4. The tests below. - -## Authorized paths - -- `crates/session/src/validate.rs`, `crates/session/src/diagnostic.rs`, `crates/session/tests/` -- `fixtures/session/v1/builtins-automation.json`; in `sdk/test/builder-evals.mjs`, - `builtinsAutomationFixture` only -- `tools/session-validator/tests/validate.rs` -- `docs/SESSION_SCHEMA_V1.md`, `.claude/skills/author-session/SKILL.md` (automation paragraph and - an automation refusal table only) - -## Non-goals - -- Any descriptor check: builtin rows are #1645, effect parameters - #1566's row rule (`controls::automation_diagnostics`) and #1335. -- The SDK builder's mirror of these rules (obsolete draft 03b, not filed). -- Rendering any automation (#1637-#1639 and #1646-#1667). The live classifier. -- New shapes, fields or per-segment ramp lengths (A3 keeps the table as it is). - -## Hazards - -- **The fixture migration moves session-document bytes.** Any digest of - `builtins-automation.json`'s bytes moves with D5; list each with the reason "D5 moves the matrix - ride to a matrix strip" and re-pin it on its own. No render digest reads this fixture's - automation, because nothing renders it. -- **`-0.0` against `0.0`.** D3 compares bits, so a hold at `0.0` followed by a segment that starts - at `-0.0` is a discontinuity. That is deliberate: render's change test compares bits too - (`crates/host-core/src/solo.rs:58-67`). -- **Order of checks.** D3 reads segment order. It runs only on an entry whose segments passed the - order and overlap checks, so it never reports on a malformed sequence. - -## Objective gates - -1. **Refusals with parity** (`crates/session/tests/diagnostic_parity.rs`, new rows in the case - table, so the text and typed entry points agree): - - the canonical fixture's automation replaced by two `builtins` fader entries (5) on its track, - the second with the greater `id`: `both`/`both`, `both`/`left` and `left`/`left` each give one - `automation.duplicate_lane` at `$.automation[1].target.channel`; the fixture's own insert - entry plus a copy with a greater `id` gives the same; - - a `matrix_ll` (7) entry on the fixture's pan track, and a pan (12) entry after its - `matrix_or_pan` is set to a matrix, each give `reference.missing_entity` at - `$.automation[0].target.parameter_id`; - - a `step` segment `[0, 960)` from `0` to `1`, then a `linear` segment from `1023` that starts - at `0.5`, give `automation.discontinuity_spacing` at - `$.automation[0].segments[1].start_sample` (discontinuities at 960 and 1023). -2. **Acceptances and order** (`crates/session/tests/automation_lanes.rs`, new). A `left` and a - `right` entry on one parameter compile. Two overlapping entries declared in either order refuse - the one with the greater `id`; three give two diagnostics. Discontinuities exactly 64 samples - apart compile; a segment that starts at the held value is not a discontinuity; a `step` whose - two values are equal has none. -3. **Validator stage** (`tools/session-validator/tests/validate.rs`, new rows in `MUTATIONS`). In - the migrated `builtins-automation.json`, a duplicate fader lane and a pan entry on `keys` each - fail at stage index 1 (`typed-model`) with their codes. -4. **Every checked-in document still validates.** - `fixtures_distinguish_schema_examples_from_launch_effects` - (`tools/session-validator/tests/validate.rs:81`), the skill test - (`tools/session-validator/tests/skill.rs`) and the SDK's byte comparison of the twin - (`sdk/test/builder-evals.mjs:1041-1048`) pass. -5. **No rendered bit moves.** No render code changes. `cargo build --locked --release -p audit -p - bench -p capi -p session-validator`, then `./target/release/audit capi`, shows the same - `pcm_digest` at base and head (PR evidence); the browser legs of the `browser` job in - `.github/workflows/qualification.yml` pass with unchanged digests. -6. **Commands:** - - `cargo test --locked -p session`, `cargo test --locked -p session-validator`, - `cargo test --locked -p protocol --features test-support` - - the workspace debug leg (`test-debug-a` in `.github/workflows/qualification.yml`) - - `mkdir -p target/ci/qualification-artifacts target/ci/qualification-named-twin && bash scripts/build-web-audioworklet.sh --named-twin target/ci/qualification-named-twin target/ci/qualification-artifacts`, - then `bash scripts/check-sdk-headless.sh target/ci/qualification-artifacts` - - `bash scripts/check-session-policy.sh`, `bash scripts/check-workspace-policy.sh` - - `bash scripts/check-cross-targets.sh` (README F19, the iOS memset rule: no new - `memset_pattern16` call; fix one in code, never by a ceiling) - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` - -## Test value - -- Gate 1 turns red if overlap is not refused, if `both` is not treated as overlapping one channel, - if a pan or matrix target is accepted on the wrong strip kind, if a step's end or a segment that - starts away from the held value is not counted as a discontinuity, or if a code or path differs - between the text and typed entry points. No test compares two entries' targets, a target with - its strip kind, or two segment boundaries today. -- Gate 2 turns red if a rule is too wide: `left` plus `right` refused, a 64-sample spacing - refused, a continuous segment start or an equal-valued step counted, or the refused entry - depending on declared order. -- Gate 3 turns red if a rule runs anywhere other than session validation. - -## Dependencies - -- None. -- Batch: P1, first in it. #1627-#1629 (the hold rule's new home before #1629 deletes the old one), - #1645 and #1637 (the evaluator crate) build on it. It is a product outcome alone: - it refuses automation lanes that no rendering could honour. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Target shape.** A target is a `ControlAddress` `{owner, owner_id, section, slot_id, control, - lane}` (#1566; `crates/session/src/model.rs:969-977`). D1's lane key becomes `(owner, owner_id, - section, slot_id, control)` with lane overlap as written (`both` overlaps `left` and `right`); the - diagnostic path is `$.automation[].target.lane`. Routes and VCAs are owners now, so the rule - covers them too. -- **D2 is done.** The pan or matrix rule is #1566's row rule: `controls::automation_diagnostics` - refuses a row the strip's shape does not use with `control.inactive` - (`crates/controls/src/automation.rs:96-100`). Drop D2, its gate 1 case and its validator row; the - `reference.missing_entity` code is not used for it. -- **D5 is done.** #1566 migrated `fixtures/session/v1/builtins-automation.json`: it now holds a - `pan` ride on `vocal` and no matrix ride, and adds `route-ride` and `vca-ride`. Drop D5 and its - fixture paths. Gate 3's mutations use the fixture as it is. -- **Where the rules live.** Session validation is structural (`validate_automation`, - `crates/session/src/validate.rs:963-1042`); D1 and D3 are structural (they read no row), so they - stay in the session crate as written. -- **Anchors gone.** `crates/host-core/src/solo.rs` is deleted (its change test is - `controls::changed_controls`, which compares value bits). The `-0.0` hazard stands. -- **Docs.** `docs/SESSION_SCHEMA_V1.md`'s inert sentence is now near `:278-279`; the registry's - record rule ("holds its end value until replaced") is `docs/CONTROL_PROTOCOL_REGISTRY.md:93`. -- **Gates.** `bash scripts/check-session-policy.sh` and the other commands stand; there is no - `test-debug-a` change. Batch P1, first, as written (section "Batch P1 readiness" of the README). - -## Decision record: #1626 (batch 10) - -**Shipped.** D1 (`automation.duplicate_lane`) and D3 (`automation.discontinuity_spacing`) in -`crates/session/src/{validate,diagnostic}.rs`; D4 in `docs/SESSION_SCHEMA_V1.md` (after the -"consumed by nothing" paragraph, which stays) and the `author-session` skill (the hold rule in the -semantic cautions and an "Automation refusals, by code" table). D2 and D5 were already done by -#1566 and were not touched, as the amendment says. - -**Where.** `validate_automation` collects the entries that reach the lane check (target accepted, -`id` the first of its value) and `validate_lane_uniqueness` sorts them once by the lane key -`(owner kind, owner id, section kind, slot id, control)` then `id`, and in one pass per key refuses -an entry whose lane an earlier (lesser `id`) entry covers (`both` covers both channels). The -discontinuity check runs per entry, only when its segments passed the order, overlap and range -checks (`ordered`), and compares `f32` bits. - -**Departures and choices.** -- A `step` whose end a segment meets at once is one boundary at that sample, decided at the next - segment's start (before it, the lane held the step's `start_value`). The filed text said the - previous segment's `end_value` for every start; the two differ only for a step met at its end, - where the true value just before the sample is `start_value`. A step end followed by a gap is a - jump when its values differ, named at the step's `end_sample`. -- "Once per offending pair" is read as consecutive jumps; a non-consecutive close pair implies a - close consecutive one, so nothing is lost. -- The "target already refused" skip compares the diagnostic count around the target check. - -**Gates.** `cargo test --locked -p session` (new: `tests/automation_lanes.rs`, seven tests; five -new rows in `tests/diagnostic_parity.rs`: `lane-both-both`, `lane-both-left`, `lane-left-left`, -`lane-insert-copy`, `discontinuity-spacing`), `-p session-validator` (two new `MUTATIONS` rows at -stage 1 on `builtins-automation.json`: a trim ride moved to the fader control, and a second segment -40 samples after the polarity step's end), `-p conformance`, `-p controls -p protocol -p host-core --p control-plane` pass; clippy `-D warnings` on session and session-validator, `fmt --check`, -`check-session-policy.sh`, `check-workspace-policy.sh` pass. The gate-1 pan/matrix case and the -validator pan row are dropped with D2. No fixture or SDK eval changed, so the SDK twin comparison -and wasm build were not rerun. Not run here: the audit `pcm_digest` comparison, the workspace -debug leg, `check-cross-targets.sh` (no render code changed). diff --git a/.github/ISSUE_SPECS/1627-retire-the-automation-enqueue-command.md b/.github/ISSUE_SPECS/1627-retire-the-automation-enqueue-command.md deleted file mode 100644 index ee8ba9edc..000000000 --- a/.github/ISSUE_SPECS/1627-retire-the-automation-enqueue-command.md +++ /dev/null @@ -1,363 +0,0 @@ -# Retire the AUTOMATION_ENQUEUE command from the protocol wire and dispatch - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answers A8 and A9, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. Batch P1. - -## Product outcome - -The control protocol has one way to make a sample-timed change: a stored automation edit, which -the engine renders (A1). A peer that sends `AUTOMATION_ENQUEUE` (`0x0006`) is refused at decode as -an unsupported message, before anything changes, on every endpoint. No endpoint advertises -`0x0006`, flag bit 5 or the capability fields that describe the command's admission. Statuses 15 -and 16 are retired. Every retired code is refused, never reallocated and never renumbered. - -This is the first of three slices that retire the command, all in batch P1 (one push): this slice -(the command), #1628 *Retire the AUTOMATION_CANCELED event and the cancellation path* and #1629 *Delete the protocol automation queue, its records and counters*. Together they replace D1-D3 -and gates 1, 2 and 4 of *Refuse commands that would be acknowledged with no effect* (#1315): -instead of a feature switch that is never set true, the command is retired. This slice lands no -later than #1315's C ABI half. #1315 keeps D4 (the browser bypass lift), D5 (docs, reworded: the -refusal is permanent) and D6. - -## Context - -- **Why.** The owner ruled that sample-timed `AutomationEnqueue` delivery has no product consumer - (`docs/rulings/engine-footprint-2026-09-28.md:58-59`) and that code nothing uses is removed - (`:20-21`). Every sample-timed outcome is a stored automation edit (opcodes `0x0600`-`0x0603`, - which stay). Today the C ABI acks a batch that reaches no PCM: the controller enqueues it - (`crates/protocol/src/controller.rs:2961-2999`), and its only dequeue is the cancellation - (`crates/protocol/src/queue.rs:799-806`; #1315 Context item 1; - `docs/CONTROL_PROTOCOL_SEMANTICS.md:15`). -- **The in-place amendment rule.** Before launch an owner ruling may amend v1 in place: nothing is - renumbered, a retired code is refused and never reallocated, a retired field ID is refused in - either flag form (`docs/CONTROL_BTLV_V1.md:61`; precedents at - `docs/CONTROL_PROTOCOL_REGISTRY.md:83`, `:89`). The machinery exists: `schema_spec_retiring` - (`crates/protocol/src/btlv.rs:240-248`) with a retired-field spec such as the track's - (`crates/protocol/src/schema.rs:1000-1010`), and `retired_code_rows` - (`crates/conformance/src/protocol_corpus.rs:359-372`), run by the controller tests - (`crates/protocol/src/controller/tests.rs:1546`). -- **Wire identity.** `MessageId::AutomationEnqueue = 0x0006` (`crates/protocol/src/wire.rs:109-110`), - parsed by `MessageId::from_raw` (`:142-165`, the arm `:150`), which returns - `DecodeError::UnsupportedMessage` for an unassigned ID (`:163`). Status `TIME_IN_PAST = 15` and - `AUTOMATION_ORDER = 16` (`:210-213`, parsed at `:238-239`). -- **The capabilities rule.** `check_capabilities_invariants` - (`crates/protocol/src/message_wire.rs:3493-3530`) ties flag bit 5 to command `0x0006` (`:3512`), - requires `maximum_automation_records == 256` (`:3503`) and `maximum_tlvs >= 27`, the field count - (`:3501`); `allocated_id` (`:3532`) lists the ID. `CapabilityFlags::{B4_BASE, KNOWN}` include bit - 5 (`:57`, `:60`). `capability_registry` (`crates/protocol/src/controller.rs:3219-3289`) always - lists `0x0006` and sets bit 5 (`flags = (1 << 7) - 1`, `:3257`); `capabilities` fills the queue - fields (`:3299-3329`, the automation ones at `:3309-3320`). - -**Inventory for this slice** (the command; the event and the cancellation are #1628's, the queue -#1629's): - -| Item | Where | -|---|---| -| Message ID `0x0006` | `crates/protocol/src/wire.rs:109-110`, `:150` | -| Status codes 15, 16 | `wire.rs:210-213`, `:238-239`; `status_for_automation`, `controller.rs:3596-3607` | -| Command and success payloads, decoded view | `AutomationEnqueue`, `DecodedAutomationEnqueue`, `AutomationEnqueued` (`message_wire.rs:413-475`); codec `:1236-1305` and `write_automation_enqueue`/`_enqueued` (`:2397` and siblings) | -| Schemas | `schema::automation_enqueued`, `automation_enqueue` (`crates/protocol/src/schema.rs:417-438`); their entries in the registry list `:1634-1640` | -| Typed frames | `typed_frame.rs:48`, `:72`, `:124`, `:147`, `:194`, `:212`, `:433-435`, `:494-496`, `:562`, `:663-664`, `:705-706` | -| Capability fields | `maximum_automation_records` (field 9), `automation_batch_slots` (12), `per_block_automation_density` (19), `admission_quantum_frames` (20): `message_wire.rs:80-92`, `:119-130`, codec `:3218`, `:3349`; `schema.rs:684`, `:687`, `:694-695`; `controller.rs:3309-3320` | -| Flag bit 5 | `message_wire.rs:57`, `:60`, `:3512`; `controller.rs:3257` | -| Controller dispatch | `ControlCommand::AutomationEnqueue` (`controller.rs:885-902`, `:924`, `:937`), `Body::AutomationEnqueued` (`:993`), decode routes (`:1499-1504`, `:2142-2152`), encode (`:2246-2247`), execute arm (`:2961-2999`), `validate_automation_domains` (`:3358-3390`) | -| Provider hook | `ControlProvider::parameter_descriptor` (only caller `controller.rs:3365`); `MockProvider`'s `automation_parameter` and its implementation (`:623-650`, `:731-744`); host-core's (`crates/host-core/src/control_provider.rs:334-343`) | -| Exports | `crates/protocol/src/lib.rs:17-38` (the command's payload names) | -| Conformance corpus | `complete_schema_corpus` rows `command.automation_enqueue` (`protocol_corpus.rs:820-822`), `response.automation` (`:926-927`), statuses 15 and 16 (`:984-985`, `:1180-1181`), capability literals (`:857-875`) | -| Protocol audit, wire leg | `tools/audit/src/protocol.rs:41-42`, `:93-134`, `:194`, `:434-456` (10,000 records encoded, decoded and enqueued in 40 batches); its success line `:510`, matched by `scripts/run-protocol-allocation-audit.sh:37-39` | -| Docs | `docs/CONTROL_PROTOCOL_REGISTRY.md:9`, `:11`, `:25`, `:30`; `docs/CONTROL_PROTOCOL_SEMANTICS.md:13`, `:15`; `docs/CONTROL_PROTOCOL_SIZING.md:9`; `docs/CONTROL_PROTOCOL_CONFORMANCE.md:3`, `:7`, `:13`; `docs/CONTROL_PROVIDER_BOUNDARY.md:9`; `docs/C_ABI_V1_QUALIFICATION.md:300` | -| Fuzz | `fuzz/corpus/complete-schema-manifest.md:3-10` (frame counts and the hash); no seed file carries the command | - -**Never retired here:** `ParameterHandle`, the type `ParameterAutomationRate` -(`message_wire.rs:239-247`; #1636 retires its value 1), `ControlProvider::current_sample`, -`reserve_reliable_events`, the stored automation edits `0x0600`-`0x0603`. - -**Tests that send or pin the command today:** - -- protocol: `controller/tests.rs` `public_b1b_automation_replay_uses_the_ordinary_queue` (`:341`), - `backpressure_is_cached_without_second_enqueue` (`:1862`), - `decoded_invalid_automation_rolls_back_after_final_compilation` (`:2115`), - `endpoint_owned_current_sample_rejects_past_automation_without_client_time` (`:2153`), - `b2b_btlv_automation_uses_header_identity_typed_domains_and_backpressure` (`:2166`) exist only for - the command; `full_frame_ingress_dispatches_every_registered_command_through_typed_frames` - (`:570`), `full_frame_ingress_replays_exact_bytes_and_rejects_changed_request_reuse` (`:723`), - `structural_token_cancel_owner_generation_and_serial_rules_are_affine` (`:1213`), - `every_non_ok_status_has_a_canonical_common_response_payload` (`:1697`), - `provider_feature_matrix_derives_capabilities_and_refuses_disabled_dispatch` (`:2789`) touch it; - `message_wire/tests.rs` `b2b_automation_goldens_truncation_and_malformed_cases_are_functional` - (`:943`) exists only for it, and `:250`, `:316`, `:532`, `:721`, `:1554` touch it; - `typed_frame.rs` tests `:923`, `:1072`, `:1236`, `:1319` touch it. -- C ABI (#1315 gate 4 names the dispatch test): `capi_controller_dispatches_every_advertised_command_family` - (`crates/capi/src/runtime/tests.rs:2250`, pinned `ALL_COMMAND_RESPONSE_VECTORS` at `:89`, entry 0 - the capabilities response, the automation response among them) and - `exported_c_replay_revision_event_and_publication_pressure_statuses_are_exact` (`:2485`, reads the - capabilities vector at `:2501`). - -A test that sends the command only to make the event or a cancellation happen is #1628's. - -**Pins this slice moves:** `COMPLETE_SCHEMA_HASH` (`crates/conformance/src/protocol_corpus.rs:737`, -its re-pin history `:706-736`), its self-test strings in `scripts/check-protocol-wasm-parity.sh:172-173`, -the manifest `fuzz/corpus/complete-schema-manifest.md:10`, the record -`docs/CONTROL_PROTOCOL_CONFORMANCE.md:3`, and the C ABI vectors above. - -## Decisions frozen for this slice - -- **D1. Retired identities.** `MessageId` loses `AutomationEnqueue`; `from_raw` returns - `UnsupportedMessage` for `0x0006`, as for any unallocated ID, and a comment names it retired and - never reallocated. `StatusCode::parse` refuses 15 and 16 as `InvalidStatus`. -- **D2. Capabilities.** Fields 9, 12, 19 and 20 leave `Capabilities`, `DecodedCapabilities` and the - schema, and join a retired-field spec read through `schema_spec_retiring`, so a peer that sends - one is refused in either flag form. The record has 23 fields; `maximum_tlvs >= 23` replaces - `>= 27`. Bit 5 leaves `KNOWN` and `B4_BASE`, so a set bit 5 is an unknown flag and refused; - `capability_registry` starts from `((1 << 7) - 1) & !(1 << 5)` and its command loop skips 6; - `allocated_id` drops `0x0006`. -- **D3. Dispatch and hook leave.** The command's dispatch, its domain check and its responses - leave; the provider trait loses `parameter_descriptor`, and host-core and `MockProvider` lose - their implementations. A test that exists only for the command is deleted; a test that touches - it is rewritten without it. The audit's wire leg leaves and its success line drops "10,000 - automation records in 40 batches". -- **D4. Re-pins**, each listed with the reason "#1627 retires `AUTOMATION_ENQUEUE`, statuses 15 - and 16 and capability fields 9, 12, 19 and 20": `COMPLETE_SCHEMA_HASH` and its history line (the - corpus loses the command row, its success row and two status rows), the wasm-parity self-test - strings, the fuzz manifest, the conformance record, and the C ABI vectors. `retired_code_rows` - gains one row per retired message, status, capability field and flag bit. -- **D5. Docs.** The registry lists each retired code with this slice as the reason; the semantics - doc states that sample-timed changes are stored automation edits and that the command is retired - (it replaces #1315 D5's wording); the sizing, conformance, provider-boundary and C ABI - qualification lines above lose the command. -- **D6. The acked-batch question: can an ack ever precede a drop?** No. `0x0006` is refused at - frame decode, before the revision check and before any state changes. Nothing can enqueue after - this slice; #1628 and #1629 remove the event and the queue in the same push. - -## Deliverables - -1. D1-D3 in `crates/protocol` and host-core's provider. -2. D4's corpus rows, re-pins and retired rows; D5's docs; the audit change. -3. The rewritten and deleted tests (D3) and the gates below. - -## Authorized paths - -- `crates/protocol/**` (not `queue.rs`) -- `crates/host-core/src/control_provider.rs` (the `parameter_descriptor` hook and its test) -- `crates/capi/src/runtime/tests.rs` (#1309 D8 keeps the C ABI tests in capi) -- `crates/conformance/src/protocol_corpus.rs`, `crates/conformance/tests/conformance_corpus.rs` -- `tools/audit/src/protocol.rs` (the wire leg and success line), `scripts/run-protocol-allocation-audit.sh` - (the success line), `scripts/check-protocol-wasm-parity.sh` (the self-test hash strings) -- `fuzz/corpus/complete-schema-manifest.md` -- `docs/CONTROL_PROTOCOL_REGISTRY.md`, `docs/CONTROL_PROTOCOL_SEMANTICS.md`, - `docs/CONTROL_PROTOCOL_SIZING.md`, `docs/CONTROL_PROTOCOL_CONFORMANCE.md`, - `docs/CONTROL_PROVIDER_BOUNDARY.md`, `docs/C_ABI_V1_QUALIFICATION.md` -- `crates/capi/src/runtime/live_tests.rs` (only the `AutomationEnqueue` block of - `a_live_edit_emits_the_commit_events_of_a_rebuild`, `:1379-1407`, which builds the deleted - command and payload) - -## Non-goals - -- The event `0x8002`, the cancellation path and `record_canceled_automation` (#1628). -- The queue, its records, config, report, queue kind 2 and the counters (#1629). -- The C limit `maximum_automation_spans_per_block` (#1668). The browser bypass lift (#1315 D4). -- The stored automation edits `0x0600`-`0x0603`. The transport position (#1635). The span kinds - and the sample rate value (#1636). - -## Hazards - -- **Batch P1.** After this slice the cancellation path and the queue have no producer; #1628 - and #1629 remove them in the same push, so `main` never holds them. -- **Corpus re-pin order.** #1313, #1365, #1369 and #1394, #1628, and #1635 and #1636 also re-pin - `COMPLETE_SCHEMA_HASH`. Whichever merges later recomputes from the earlier value and adds its own - history line; never merge two hashes by hand. -- **#1315's C ABI half** must not land before this slice: its feature switch would be code this - slice deletes. Dependencies order it after this slice, and root amends #1315 before either - lands. - -## Objective gates - -1. **Refused at decode** (`crates/protocol/src/controller/tests.rs`, new). A complete command - frame with message ID `0x0006` and a valid former payload is answered `UNSUPPORTED_MESSAGE` (3); - the revision, the reliable event lane, the counters and the replay cache are unchanged. Each - retired status, capability field (in both flag forms) and flag bit is refused by its decoder - (rows in `retired_code_rows`). -2. **Capabilities** (`crates/protocol/src/message_wire/tests.rs`, and the C ABI dispatch test). The - record does not list `0x0006`, clears bit 5, carries 23 fields, and passes - `check_capabilities_invariants`; a record that adds the ID or bit 5 back is refused. -3. **C ABI** (`crates/capi/src/runtime/tests.rs`, rewritten). The dispatch test sends `0x0006` and - gets `UNSUPPORTED_MESSAGE` with nothing changed. Every re-pinned vector is listed in the PR with - D4's reason. -4. **Corpus and audits.** `cargo test --locked -p conformance` with the new - `COMPLETE_SCHEMA_HASH`; `bash scripts/check-protocol-wasm-parity.sh`; - `cargo build --locked --release -p audit -p bench -p capi -p session-validator`, then - `bash scripts/run-protocol-allocation-audit.sh target/release/audit` and - `./target/release/audit capi` with zero allocations, locks and syscalls. -5. **No rendered bit moves.** `./target/release/audit capi` shows the same `pcm_digest` at base and - head (PR evidence); nothing that renders reads the command. -6. **Commands:** - - `cargo test --locked -p protocol --features test-support`, `cargo test --locked -p capi`, - `cargo test --locked -p host-core --features test-support`, `cargo test --locked -p conformance` - - the workspace debug leg (`test-debug-a` in `.github/workflows/qualification.yml`) - - `bash scripts/check-protocol-control-policy.sh`, `bash scripts/test-protocol-control-policy.sh`, - `bash scripts/check-capi-abi.sh`, `bash scripts/check-conformance-boundaries.sh`, - `bash scripts/check-workspace-policy.sh` - - `CARGO_TARGET_DIR=target/ci/wasm-simd RUSTFLAGS='-C target-feature=+simd128' cargo check --locked --target wasm32-unknown-unknown -p target-smoke -p protocol` - - `bash scripts/check-cross-targets.sh` (README F19, the iOS memset rule: no new - `memset_pattern16` call; fix one in code, never by a ceiling) - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` - -## Test value - -- Gate 1 turns red if a retired message, status, field or flag still decodes, if a retired - capability field is skipped as an unknown optional field, or if the refusal changes state first - (an ack before a drop). No test refuses `0x0006` today. -- Gate 2 turns red if an endpoint still advertises the retired ID or bit. -- Gate 3 turns red if the C ABI dispatches the command. - -## Dependencies - -Batch P1. Direct dependencies: - -- #1626 *Validate stored automation lanes in the session crate and state the hold rule*: the - hold rule's new home in `docs/SESSION_SCHEMA_V1.md`, which must exist before #1629 deletes - `docs/CONTROL_PROTOCOL_REGISTRY.md:45`. #1626 is in batch P1 and lands first in it. -- *Extract the C ABI control plane into a portable crate both hosts call* (#1309), for the C ABI - paths. -- Ordering: this draft lands no later than the C ABI half of *Refuse commands that would be - acknowledged with no effect* (#1315). #1627-#1629 amend #1315: they replace its D1-D3 and - gates 1, 2 and 4. - -No rendering slice is a dependency. #1628 *Retire the AUTOMATION_CANCELED event and the -cancellation path* depends on this draft. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **#1315 landed as a switch.** `ProviderFeatures::automation` (`crates/protocol/src/controller.rs:108-150`) - is false in `crates/control-plane/src/compile.rs:938-943`, so both hosts refuse the command - `UNAVAILABLE` and the capability record already omits `0x0006` and flag bit 5 - (`controller.rs:3160`, `:3727`, `:3762`). This slice also deletes the switch, its `ALL` and `NONE` - entries and arms, and the compile.rs comment that says to flip it (README F24). The #1315 rows of - the README's amendment table are moot. -- **The provider hook** is `ControlProvider::control_row` (`controller.rs:602-609`), not - `parameter_descriptor`: its one caller is `validate_automation_domains` (`:3865-3872`). It leaves - with `MockProvider`'s implementation (`:840`) and host-core's - (`crates/host-core/src/control_provider.rs:1075-1094`, tests `:3239-3267`). -- **Retired IDs.** `0x0006` joins `RETIRED_MESSAGE_IDS` (`crates/protocol/src/wire.rs:141`, with - `0x0004` and `0x0005`), the mechanism #1551 added. -- **Gate 2** then tests only what changes: capability fields 9, 12, 19 and 20 leave; `0x0006` and - bit 5 are already absent. -- **Both hosts.** The browser runs the same controller (`compile_children` in control-plane), so it - needs no browser gate. -- **#1567** (parked) is obsolete: root closes it as superseded by #1627-#1629 before P1 merges. -- **Anchors.** Every line in the inventory has moved; re-read them at filing. The C ABI tests are - in `crates/capi/src/runtime/tests.rs` and `live_tests.rs` (`a_live_edit_emits_the_commit_events_of_a_rebuild`, - `:2324`). Batch P1, ready (README "Batch P1 readiness"). - -## Decision record: #1627 retire AUTOMATION_ENQUEUE (batch 10) - -**What shipped.** The command, its success response, statuses 15 and 16, capability fields 9, 12, -19 and 20, flag bit 5 and the provider hook are gone from `protocol`, host-core and the control -plane; nothing else changed on the wire. - -- `MessageId::AutomationEnqueue`, `StatusCode::{TimeInPast, AutomationOrder}`, `AutomationEnqueue`, - `DecodedAutomationEnqueue`, `AutomationEnqueued`, their schemas, codecs, typed-frame arms, - `ControlCommand::AutomationEnqueue`, `Body::AutomationEnqueued`, the controller's execute arm, - `validate_automation_domains`, `status_for_automation` and `AutomationRefusal` are deleted. - `0x0006` joins `RETIRED_MESSAGE_IDS` (`[0x0004, 0x0005, 0x0006]`), so `from_raw` and `allocated_id` - refuse it with no new code. `DecodedCommandPayload` and `DecodedTypedCommandFrame` lose a lifetime - that only the borrowed automation view used (clippy `needless_lifetimes` then elided three more). -- `ProviderFeatures::automation`, its `ALL`/`NONE` entries, its three arms and the compile.rs flip - comment are deleted; the control plane passes `ProviderFeatures::ALL`. -- `ControlProvider::control_row` leaves with `MockProvider`'s and host-core's implementations, and - with it `ParameterProviderError::UnsupportedTarget` (its only producer) and the - `automation.target.unsupported` diagnostic. host-core's test - `only_an_effect_parameter_is_an_automation_target` is deleted with the hook. -- `Capabilities`/`DecodedCapabilities` lose `maximum_automation_records`, `automation_batch_slots`, - `per_block_automation_density` and `admission_quantum_frames` (23 fields). Schema - `capabilities::RETIRED` is read through `schema_spec_retiring`, so each of 9, 12, 19 and 20 is - refused in both flag forms (optional `INVALID_TLV`, mandatory `UNKNOWN_REQUIRED_FIELD`). - `maximum_tlvs >= 23`; bit 5 leaves `KNOWN` and `B4_BASE`; `capability_registry` starts from - `((1 << 7) - 1) & !(1 << 5)`; the invariant check no longer ties bit 5 to a command. -- The audit's wire leg and its "10,000 automation records in 40 batches" success text are gone - (script updated to match); `Corpus.queues` left with it (the egress leg keeps its own queues). - -**Tests.** Deleted (existed only for the command): `public_b1b_automation_replay_uses_the_ordinary_queue`, -`backpressure_is_cached_without_second_enqueue`, `endpoint_owned_current_sample_rejects_past_automation_without_client_time`, -`b2b_btlv_automation_uses_header_identity_typed_domains_and_backpressure`, -`a_controller_without_the_automation_feature_refuses_and_does_not_advertise_the_command`, -`automation_keeps_its_domain_rules_over_the_control_row`, -`b2b_automation_goldens_truncation_and_malformed_cases_are_functional`, host-core -`only_an_effect_parameter_is_an_automation_target`. Rewritten without the command: -`full_frame_ingress_dispatches_every_registered_command_through_typed_frames`, -`full_frame_ingress_replays_exact_bytes_and_rejects_changed_request_reuse` (now a `TRANSPORT_SET`, -replay counted on the reliable event queue), `every_non_ok_status_has_a_canonical_common_response_payload`, -`the_capability_registry_lists_the_control_messages_and_not_the_retired_ones`, -`provider_feature_matrix_derives_capabilities_and_refuses_disabled_dispatch` (23-field count), -`capi_controller_dispatches_every_advertised_command_family` (ten vectors; see re-pins), -`all_five_event_families_cross_c_dequeue_with_exact_oracle_bytes` (no automation step; seven -responses), `a_live_edit_emits_the_commit_events_of_a_rebuild`, and typed-frame/schema/message-wire -fixtures. Kept: `decoded_invalid_automation_rolls_back_after_final_compilation` (it exercises the -stored `SetAutomationSegments` edit, not the command) and -`structural_token_cancel_owner_generation_and_serial_rules_are_affine` (its only touch is -`cancel_pending_automation`, which is #1628's). - -**Gates and their tests.** - -1. Refused at decode: protocol `a_frame_under_the_retired_automation_id_is_refused_before_any_state_changes` - (the conformance row `retired.message_id_0x0006`, a frame with the former payload; revision, - snapshot, replay cache and all five queue reports unchanged, request ID unspent, capabilities - list no `0x0006`/bit 5); `retired_and_console_refused_codes_meet_their_conformance_rows` (rows - 16 + 4 + 7 + 13 + 1 + 3 + 1); `retired_message_ids_are_never_reused_or_advertised`; wire - `retired_status_codes_are_invalid_statuses` (15 and 16 `InvalidStatus`; 14, 17, 18 decode); - message-wire `retired_capability_fields_are_refused_in_either_flag_form` (23 fields; each of 9, - 12, 19, 20 in both flag forms; control splice field 28 decodes). -2. Capabilities: `capability_flag_bit_5_is_unknown_and_the_floor_is_the_field_count` and - `capability_id_views_reject_order_allocation_and_session_family_identically` (a record listing 6 - is refused). -3. C ABI: `the_retired_automation_command_is_refused_at_decode_and_not_advertised` (`INVALID_ARGUMENT`, - buffer untouched, `required_bytes` unwritten, revision/queue/events unchanged) and the re-pinned - dispatch test. -4. Corpus and audits: `cargo test --locked -p conformance`; `bash scripts/check-protocol-wasm-parity.sh` - and `--self-test` pass; `run-protocol-allocation-audit.sh target/release/audit` passes; - `audit capi` shows 0 allocations, locks, syscalls. -5. No rendered bit moves: `audit capi` `pcm_digest` is `76525e7ef9e78cf4` at base (`2e6343a99`) and head. - -**Departures.** - -- Retired statuses, capability fields and flag bit are tested by decoder unit tests, not by rows in - `retired_code_rows`: a `RetiredCodeRow` is a command frame an endpoint answers, and these are - response-side. The one row that fits (`0x0006`, with its former payload) is added there. The C ABI - gate asserts `INVALID_ARGUMENT` (the call result for a frame that decodes to no message) rather - than a response frame: an unknown message ID leaves no safe request identity, so no frame is written, - as for `0x0004` and `0x0005`. -- Mechanical edits outside the authorized paths: `crates/conformance/src/main.rs` and - `crates/conformance/tests/mutation_million.rs` (frame count 50 to 46), `hosts/host-web/tests/protocol_cross_host.rs` - (script step 11 sends `0x0006`; both hosts must refuse it identically), `sdk/src/core/control-frame.ts` - (status names 15 and 16 become unregistered holes; the table is index-by-wire-number), and the - `crates/protocol/src/typed_frame.rs` lifetimes above. -- The wasm-parity self-test and the fuzz manifest still carried the pre-#1566 hash `cdfdb0540f093424` - (the self-test's search text matched nothing and would have failed `--self-test`). Both now carry - the new pin. - -**Re-pins** (reason: #1627 retires `AUTOMATION_ENQUEUE`, statuses 15 and 16 and capability -fields 9, 12, 19 and 20): `COMPLETE_SCHEMA_HASH` `0x4f6a_80fb_52c1_2aed` to `0xcfa5_8ab7_b7bb_b7c4` -(corpus 50 to 46 frames: the command row, its success row and the two status rows are gone, and the -capabilities response has 23 fields and lists no `0006`), the parity self-test strings, the fuzz -manifest, the conformance record, and C ABI vectors: capabilities (27 fields to 23, length -`0x1c8` to `0x188`; flags `0x7fdf` unchanged), the automation response vector deleted (11 to 10), -the request IDs of the six later vectors one lower, and the `all_five_event_families` capabilities -response (request 7) rebuilt from the new vector. The C ABI header and `check-capi-abi.sh` are -unaffected; the SDK generated modules are current. - -**Left for the successors.** #1628: `AUTOMATION_CANCELED`, `cancel_pending_automation`, -`cancel_queued_automation_reserved`, `record_canceled_automation`, `CounterId::{CanceledAutomation, -AutomationTimePast, AutomationOrderReject}`, `ReliablePayload::AutomationCanceled`. #1629: the queue, -`AutomationBatchSlot`/`AutomationRecord`/`ParameterHandle`, `QueueKind::Automation`, -`BackpressureQueueKind::Automation`, `ProtocolQueueConfig::{automation_batch_slots, -per_block_automation_density, quantum_frames}`, and the sizing guide's automation queue and -10,000-record paragraphs and the semantics guide's "Queues and telemetry" automation sentences. -`crates/capi/tests/resource_lifecycle.rs:1119` keeps a historical comment naming the command. - -## Fold (batch 10 review) - -The deletion of `backpressure_is_cached_without_second_enqueue` took a general claim with it: an executed `BACKPRESSURE` outcome is cached, and a retry under the same request ID replays the identical frame without executing. Re-homed on `SESSION_TRANSACTION_APPLY` as `a_backpressure_outcome_is_cached_and_its_retry_executes_nothing` (`crates/protocol/src/controller/tests.rs`): a full reliable-event queue refuses the apply with `BACKPRESSURE`; the queue is freed; the same request ID retried returns the identical response, the revision and the canonical snapshot are unchanged, and no event is enqueued. Red when `process` stops caching a `BACKPRESSURE` outcome (the retry's frame differs), run once and reverted. diff --git a/.github/ISSUE_SPECS/1628-retire-the-automation-canceled-event.md b/.github/ISSUE_SPECS/1628-retire-the-automation-canceled-event.md deleted file mode 100644 index 0a7065556..000000000 --- a/.github/ISSUE_SPECS/1628-retire-the-automation-canceled-event.md +++ /dev/null @@ -1,234 +0,0 @@ -# Retire the AUTOMATION_CANCELED event and the cancellation path - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answers A8 and A9, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. Batch P1. - -## Product outcome - -After #1627 *Retire the AUTOMATION_ENQUEUE command from the protocol wire and dispatch*, no -batch can enter the automation queue, so nothing can be canceled. This slice removes the -cancellation: the event `AUTOMATION_CANCELED` (`0x8002`) and its reason enum are retired, a -revision change or a transport locate cancels nothing, and a transaction commit reserves only its -`SESSION_COMMITTED` event. No endpoint advertises `0x8002`. The retired ID is refused and never -reallocated. - -This is the second of three slices that retire the command, all in batch P1 (one push). #1629 -*Delete the protocol automation queue, its records and counters* follows it. - -## Context - -- **Wire identity.** `MessageId::AutomationCanceled = 0x8002` (`crates/protocol/src/wire.rs:123-124`, - parsed at `:157`). -- **The capabilities rule.** `check_capabilities_invariants` - (`crates/protocol/src/message_wire.rs:3493-3530`) ties `0x8002` to `0x8001` (`:3496`, `:3522`); - `allocated_id` (`:3532`) lists `0x8002`. `capability_registry` - (`crates/protocol/src/controller.rs:3219-3289`) lists it among its event candidates. -- **The cancellation path.** `cancel_pending_automation` and `cancel_queued_automation_reserved` - (`controller.rs:3423-3490`) cancel on a revision change (`:1877`, `:2005`, `:2931`) and on a - locate (`:3051`). Event reservations and token staleness count queue occupancy (`:1733`, - `:1953-1963`, `:2881`, `:3010`); the event is emitted at `:2667-2685`. The provider hook - `ControlProvider::record_canceled_automation` (`:578-589`) has `MockProvider`'s implementation - (`:769-778`) and host-core's (`crates/host-core/src/control_provider.rs:369-380`, test `:923`). - -**Inventory for this slice:** - -| Item | Where | -|---|---| -| Message ID `0x8002` | `crates/protocol/src/wire.rs:123-124`, `:157` | -| Event payload and its reason enum | `AutomationCancellationReason`, `AutomationCanceled` (`message_wire.rs:523-554`); codec `:1475-1535`; `parse_automation_cancellation_reason` (`:1961-1970`); `write_automation_canceled` (`:2526`); `ReliablePayload::AutomationCanceled` and `ReliableSlot::automation_canceled` (`queue.rs:336-350`, `:419-444`) | -| Schema | `schema::automation_canceled` (`crates/protocol/src/schema.rs:488-507`); its entry in the registry list `:1634-1640` | -| Typed frames | `typed_frame.rs:102`, `:172`, `:226`, `:533-535`, `:739-740` | -| Capabilities | `message_wire.rs:3496`, `:3522`, `:3532`; the event candidates of `capability_registry` | -| Cancellation | `controller.rs:3423-3490`, `:1877`, `:2005`, `:2931`, `:3051`; occupancy terms `:1733`, `:1953-1963`, `:2881`, `:3010`; emission `:2667-2685` | -| Provider hook | `record_canceled_automation` (`controller.rs:578-589`, `:769-778`; `crates/host-core/src/control_provider.rs:369-380`, test `:923`) | -| Exports | `crates/protocol/src/lib.rs:17-38` (the event and reason names) | -| Conformance corpus | `event.automation_canceled` (`protocol_corpus.rs:1054-1059`) | -| Docs | `docs/CONTROL_PROTOCOL_REGISTRY.md:9`, `:41-43`; `docs/CONTROL_PROTOCOL_SEMANTICS.md:19` | - -**Tests that raise or expect the event:** - -- protocol: `controller/tests.rs` - `transport_state_only_preserves_automation_and_locate_starts_new_ordering_epoch` (`:2433`) - exists only for the cancellation; `c2b2_event_egress_emits_all_six_schema_closed_full_frames` - (`:3185`) touches it; the `typed_frame.rs` tests that #1627 rewrote lose their event rows. -- C ABI (#1315 gate 4 names the first): `all_six_event_families_cross_c_dequeue_with_exact_oracle_bytes` - (`crates/capi/src/runtime/tests.rs:1857`, pinned `RESPONSES` at `:1858-1873`) and - `a_live_edit_emits_the_commit_events_of_a_rebuild` - (`crates/capi/src/runtime/live_tests.rs:1371`, which queues a batch and expects - `AUTOMATION_CANCELED`). - -## Decisions frozen for this slice - -- **D1. Retired identity.** `MessageId` loses `AutomationCanceled`; `from_raw` returns - `UnsupportedMessage` for `0x8002`, and a comment names it retired and never reallocated. -- **D2. The session family** is `0x0003` with `0x8001`: `check_capabilities_invariants` drops - `has_event(0x8002)` from `session_family` and the `0x8001`/`0x8002` tie; `allocated_id` drops - the ID; `capability_registry`'s event candidates drop it. -- **D3. Nothing cancels.** A revision change and a transport locate cancel nothing; the - cancellation path, the occupancy terms in reservations and token staleness, and the event - emission leave. A transaction commit reserves only its `SESSION_COMMITTED`. The provider trait - loses `record_canceled_automation`, with both implementations. -- **D4. Tests.** A test that exists only for the cancellation is deleted. The C ABI event test - becomes `all_five_event_families_cross_c_dequeue_with_exact_oracle_bytes` (no path raises - `0x8002`); `a_live_edit_emits_the_commit_events_of_a_rebuild` expects `SESSION_COMMITTED` alone. -- **D5. Re-pins**, each with the reason "#1628 retires `AUTOMATION_CANCELED`": - `COMPLETE_SCHEMA_HASH` and its history line (the corpus loses the event row), the wasm-parity - self-test strings, the fuzz manifest, the conformance record, and the C ABI vectors. - `retired_code_rows` gains a row for `0x8002`. -- **D6. Docs.** The registry lists `0x8002` as retired with this slice as the reason; the - semantics doc loses the cancellation paragraph (`:19`). -- **D7. The acked-batch question: can an ack ever precede a drop?** No. No batch can be admitted - (#1627), so there is nothing to cancel, and #1629 removes the queue. - -## Deliverables - -1. D1-D3 in `crates/protocol` and host-core's provider. -2. D4's tests, D5's re-pins and retired row, D6's docs. - -## Authorized paths - -- `crates/protocol/**` (in `queue.rs`, only the event's reliable payload and constructor) -- `crates/host-core/src/control_provider.rs` (the `record_canceled_automation` hook and its test) -- `crates/capi/src/runtime/{tests,live_tests}.rs` (#1309 D8 keeps the C ABI tests in capi) -- `crates/conformance/src/protocol_corpus.rs`, `crates/conformance/tests/conformance_corpus.rs` -- `scripts/check-protocol-wasm-parity.sh` (the self-test hash strings), - `fuzz/corpus/complete-schema-manifest.md` -- `docs/CONTROL_PROTOCOL_REGISTRY.md`, `docs/CONTROL_PROTOCOL_SEMANTICS.md`, - `docs/CONTROL_PROTOCOL_CONFORMANCE.md` - -## Non-goals - -- The command (#1627). The queue, its records, config and counters (#1629). The transport - position (#1635). - -## Hazards - -- **Batch P1.** After this slice the queue has no producer and no consumer; #1629 deletes it - in the same push. -- **Corpus re-pin order** as in #1627. - -## Objective gates - -1. **Refused** (`crates/protocol/src/controller/tests.rs`, through `retired_code_rows`). An event - frame with `0x8002` fails typed event decode. -2. **Capabilities** (`crates/protocol/src/message_wire/tests.rs`). The record does not list - `0x8002` and passes `check_capabilities_invariants`; a record that adds it back is refused. -3. **Nothing cancels** (`crates/protocol/src/controller/tests.rs`, new). A transaction that changes - the revision and a `TRANSPORT_SET` reserve and emit exactly the events they emit without an - automation family: `SESSION_COMMITTED` and `TRANSPORT_STATE`, and no other. -4. **C ABI** (`crates/capi/src/runtime/tests.rs`, `live_tests.rs`, rewritten): D4's two tests. - Every re-pinned vector is listed in the PR with D5's reason. -5. **Corpus and audits.** `cargo test --locked -p conformance` with the new - `COMPLETE_SCHEMA_HASH`; `bash scripts/check-protocol-wasm-parity.sh`; - `cargo build --locked --release -p audit -p bench -p capi -p session-validator`, then - `bash scripts/run-protocol-allocation-audit.sh target/release/audit` and - `./target/release/audit capi` with zero allocations, locks and syscalls, and the same - `pcm_digest` as base (PR evidence). -6. **Commands:** as #1627's gate 6, which includes `bash scripts/check-cross-targets.sh` - (README F19, the iOS memset rule: no new `memset_pattern16` call; fix one in code, never by a - ceiling). - -## Test value - -- Gate 1 turns red if the retired event still decodes. -- Gate 2 turns red if the invariant check still demands `0x8002` beside `0x8001`. -- Gate 3 turns red if a commit or a locate still reserves or emits a cancellation, which would cost - a reliable slot for nothing. -- Gate 4 turns red if the C ABI still raises or expects `AUTOMATION_CANCELED`. - -## Dependencies - -Batch P1. Direct dependency: - -- #1627 *Retire the AUTOMATION_ENQUEUE command from the protocol wire and dispatch*. - -#1629 *Delete the protocol automation queue, its records and counters* depends on this draft. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **The event test** is already `all_five_event_families_cross_c_dequeue_with_exact_oracle_bytes` - (`crates/capi/src/runtime/tests.rs:2194`, renamed by #1315). D4's rename is done; its doc comment - (`:2187-2191`, "the sixth family returns with stored automation") is rewritten: the family is - retired (README F24). -- **`a_live_edit_emits_the_commit_events_of_a_rebuild`** is `crates/capi/src/runtime/live_tests.rs:2324`; - it still decodes `AUTOMATION_CANCELED` (`:2308`). -- **Anchors** in the inventory have moved; re-read at filing. Batch P1, ready. - -## Decision record: #1628 retire AUTOMATION_CANCELED (batch 10) - -**What shipped.** The event `AUTOMATION_CANCELED` (`0x8002`) and the whole cancellation path are -gone from `protocol`, host-core and the conformance corpus; nothing else changed on the wire. - -- `MessageId::AutomationCanceled`, `AutomationCancellationReason`, `AutomationCanceled`, its schema, - codec (`encode`/`decode`/`write`, `parse_automation_cancellation_reason`), the `EventPayload` and - `DecodedEventPayload` arms, `ReliablePayload::AutomationCanceled`, `ReliableSlot::automation_canceled` - and the controller's event encoding are deleted. `0x8002` joins `RETIRED_MESSAGE_IDS` - (`[0x0004, 0x0005, 0x0006, 0x8002]`), so `from_raw` and `allocated_id` refuse it with no new code. -- `cancel_pending_automation`, `cancel_queued_automation_reserved`, - `ProtocolQueues::reset_automation_ordering_after_cancellation` and the provider hook - `record_canceled_automation` (trait default, `MockProvider`, host-core) are deleted. A transaction - commit (planned, prepared and legacy paths) reserves only its `SESSION_COMMITTED` (`required_events` - is 0 for a monitor-only transaction, else 1); a `TRANSPORT_SET` reserves only its `TRANSPORT_STATE`. - `PreparedStructuralCommand` and `StructuralCommandPlan` lose `cancellation_batches`, and a token no - longer goes stale when the automation queue's occupancy changes. -- `check_capabilities_invariants` drops `has_event(0x8002)` from the session family and the - `0x8001`/`0x8002` tie; `allocated_id` refuses `0x8002`; `capability_registry` lists five events. -- Docs: registry (table, retired paragraph, event payload sentence), semantics (cancellation - paragraph), conformance manifest, `C_ABI_V1_QUALIFICATION.md` ("Same events"). - -**Tests.** Deleted (existed only for the cancellation): `transport_state_only_preserves_automation_and_locate_starts_new_ordering_epoch`, -the `cancel_pending_automation` assertion of `structural_token_cancel_owner_generation_and_serial_rules_are_affine`, -the cancellation codec block of `b4_event_payloads_are_typed_canonical_and_truncation_safe`. New: -`a_revision_change_and_a_locate_reserve_and_emit_only_their_own_events` (gate 3: with a batch queued and -a two-slot reliable event queue, a revision change emits `SESSION_COMMITTED` alone and a locate -`TRANSPORT_STATE` alone) and `an_event_frame_under_the_retired_cancellation_id_is_refused` (gate 1, -event side: a valid event frame with its ID patched to `0x8002` fails typed decode with -`UnsupportedMessage`). Rewritten: `c2b2_event_egress_emits_all_five_schema_closed_full_frames`, -`retired_message_ids_are_never_reused_or_advertised` (4 IDs; `0x8002` not an allocated event), -`capability_id_views_reject_order_allocation_and_session_family_identically` (a record that adds -`0x8002` back is refused: gate 2), the `[0x8001]` event lists of the capabilities tests, -`retired_and_console_refused_codes_meet_their_conformance_rows` (`retired_code_rows` gains -`retired.message_id_0x8002`, a command frame under the retired event ID: 16 + 4 + 7 + 13 + 1 + 4 + 1), -C ABI `all_five_event_families_cross_c_dequeue_with_exact_oracle_bytes` (doc comment: the family is -retired, README F24; the capabilities response vector now lists five events) and -`a_live_edit_emits_the_commit_events_of_a_rebuild` (no `AUTOMATION_CANCELED` arm). - -**Re-pins** (reason: #1628 retires `AUTOMATION_CANCELED`): `COMPLETE_SCHEMA_HASH` -`0xcfa5_8ab7_b7bb_b7c4` to `0x4b21_d9e6_f520_afd9` (corpus 46 to 45 frames: the -`event.automation_canceled` row is gone and the capabilities response lists no `8002`), the parity -self-test strings (digest, and the length-assertion mutation 45 to 44), the fuzz manifest, the -conformance manifest, `main.rs`/`conformance_corpus.rs`/`mutation_million.rs` frame counts, and C ABI -vectors: the capabilities response (events TLV `0c` bytes to `0a`, five IDs, same padded length) in -the dispatch test and in the five-family event test. No other C ABI vector moved. - -**Gates.** `cargo test --locked` for protocol (`--features test-support`), control-plane -(`--features test-support`), host-core (`--all-features`), capi, conformance, host-web -(`--features host-web/test-support`): all green. Wasm parity and `--self-test`, both vocabulary -scripts, `check-protocol-control-policy.sh`, `run-protocol-allocation-audit.sh`, fuzz `cargo check`, -clippy `--workspace --all-targets --all-features -D warnings`, `cargo fmt --check`, -`check-sdk-generated.sh` and `check-cross-targets.sh` (no new `memset_pattern16`): pass. `audit capi`: -0 allocations, locks and syscalls; `pcm_digest` `76525e7ef9e78cf4`, equal to base. - -**Departures.** - -- The counters `CANCELED_AUTOMATION` (12), `AUTOMATION_TIME_PAST` (13) and `AUTOMATION_ORDER_REJECT` - (14) stay in `CounterId`, `parse_counter_id` and host-core's served set: #1629's spec retires - counters 2 and 11-14 with a gate of its own. After this slice counter 12 always reads 0 (host-core's - test asserts that); #1629 removes it. #1627's record listed them under #1628, but this spec's - inventory and #1629's spec say otherwise, and a counter retirement needs the `retired_code_rows` - rows that #1629 owns. -- `retired_code_rows()` rows are command frames, so the `0x8002` row is a command-header frame under - the retired event ID (refused `UnsupportedMessage`); the event-side refusal is the separate - `an_event_frame_under_the_retired_cancellation_id_is_refused` test. -- Mechanical edits outside the authorized paths: `crates/conformance/src/main.rs` and - `tests/mutation_million.rs` (frame count), `docs/C_ABI_V1_QUALIFICATION.md` (one stale sentence), - `crates/protocol/src/typed_frame.rs` fixture rows. - -**Left for #1629.** `ProtocolQueues::try_dequeue_automation` now has no production caller (its doc -says so); the gate-3 test enqueues a batch to prove nothing cancels it and loses that line when the -queue goes; `tools/realtime-policy/src/policy.rs:63` still names `try_dequeue_automation`; -`queue.rs:202` ("identity for cancellation") describes the batch slot. diff --git a/.github/ISSUE_SPECS/1629-delete-the-protocol-automation-queue.md b/.github/ISSUE_SPECS/1629-delete-the-protocol-automation-queue.md deleted file mode 100644 index 1d0a9577f..000000000 --- a/.github/ISSUE_SPECS/1629-delete-the-protocol-automation-queue.md +++ /dev/null @@ -1,296 +0,0 @@ -# Delete the protocol automation queue, its records and counters - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answers A8 and A9, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. Batch P1. - -## Product outcome - -After #1627 *Retire the AUTOMATION_ENQUEUE command from the protocol wire and dispatch* and -#1628 *Retire the AUTOMATION_CANCELED event and the cancellation path*, nothing enqueues into the -protocol's automation queue. This slice deletes it: the -32-byte record codec, the batch slot, the queue and its admission state, its configuration and -report, queue kind 2 and the five automation counters. Every C ABI session stops allocating the -queue, so `capi_retained_bytes` falls. Queue kind 2 and counters 2 and 11-14 are retired: refused, -never reallocated, never renumbered. - -This is the third of three slices that retire the command, all in batch P1 (one push), so `main` -never holds a queue that nothing calls. - -## Context - -- **The queue and its record** (`crates/protocol/src/queue.rs`): `AutomationKind`, - `AutomationRecord` and its manual 32-byte codec (`:30-154`), `AutomationBatchError` - (`:157-195`), `AutomationBatchSlot` (`:197-283`), `AutomationEnqueueError` (`:619-636`), the - automation pair, frontier, density and interval rows of `ProtocolQueues` (`:643`, `:651-653`, - `:657-679`), sized in `resource_report_for_config` and `prepare` (`:683-735`, `:738-765`), - `try_enqueue_automation`, `try_dequeue_automation` and - `reset_automation_ordering_after_cancellation` (`:771-813`), `validate_automation_admission` and - its helpers (`:1062-1194`). `AUTOMATION_RECORD_BYTES` is `crates/protocol/src/lib.rs:64-65`. -- **Configuration.** `ProtocolQueueConfig::{automation_batch_slots, per_block_automation_density, - quantum_frames}` (`queue.rs:527-528`, `:535-538`); `quantum_frames` is "used only for - control-side density admission". After #1627 no capability field reads them. -- **Queue kind 2.** `QueueKind::Automation` (`queue.rs:576-577`), mapped to - `BackpressureQueueKind::Automation = 2` (`crates/protocol/src/message_wire.rs:769-770`, decode - `:785`) at `crates/protocol/src/controller.rs:3169`; `docs/CONTROL_PROTOCOL_REGISTRY.md:13`. -- **Counters 2, 11-14.** `CounterId::{AutomationBackpressure = 2, LateAutomation = 11, - CanceledAutomation = 12, AutomationTimePast = 13, AutomationOrderReject = 14}` - (`message_wire.rs:628`, `:637-640`; `parse_counter_id`, `:2035`, `:2044-2047`). After #1627 and #1628 nothing writes any of them (#1628 removes `record_canceled_automation`). No corpus row - or test configures - them (`tools/audit/src/protocol.rs:305` and `crates/capi/src/runtime/tests.rs:2030` use counter - 1). -- **The page bound.** `AUTOMATION_BATCH_RECORDS = 256` (`crates/protocol/src/lib.rs:62-63`) sizes - the batch and also the meter and counter pages (`controller.rs:1232`, `:1238`, `:2741`, `:2788`). -- **The C ABI.** `protocol_queue_config` builds the queue from the limits - (`crates/capi/src/runtime/compile.rs:114-138`: one batch slot, the density from S, the - quantum); `capi_resources` charges its rows (`:151`); the test-support report carries an - `automation` row (`crates/capi/src/runtime/control.rs:156`, `:665-668`). #1309 moves both files - to `crates/control-plane/src/`. The exact-charge oracle is - `capi_retained_bytes_charge_every_byte_the_compile_retains` - (`crates/capi/tests/resource_lifecycle.rs:926`). -- **The protocol audit** configures the queue (`tools/audit/src/protocol.rs:115-119`, `:284-288`). -- **Tests that exist only for the queue:** `queue.rs` tests - `queue_resource_projection_is_nonzero_and_shares_prepare_overflow_rules` (`:1257`), - `transient_record_has_exact_manual_le_codec` (`:1286`), - `ten_thousand_events_fit_as_exactly_forty_atomic_batches` (`:1329`), - `past_order_overlap_and_density_reject_wholly` (`:1353`), - `overlap_is_rejected_across_separately_queued_batches` (`:1386`), - `invalid_and_out_of_order_slots_return_whole_unchanged_batches` (`:1418`), - `zero_record_and_aggregate_frontier_density_reject_wholly` (`:1456`); and - `full_empty_wrap_and_generation_preserve_original_items` (`:1307`), which also covers the other - queues. -- **Docs.** `docs/CONTROL_PROTOCOL_REGISTRY.md:13` (queue kind), `:45` (the record); - `docs/CONTROL_PROTOCOL_SEMANTICS.md:23`; `docs/CONTROL_PROTOCOL_SIZING.md:19`, `:25`; - `docs/REALTIME_MEMORY.md:13`. -- **#1351** (*Report each configured counter's own value in the C ABI counter snapshot*) D1 serves - `CANCELED_AUTOMATION` from construction. - -## Decisions frozen for this slice - -- **D1. The queue leaves.** Every item in Context's queue and record list leaves, with - `AUTOMATION_RECORD_BYTES` and the exports at `crates/protocol/src/lib.rs:34-38` that name them. - `ProtocolQueueConfig` loses its three automation fields. -- **D2. Retired codes.** `QueueKind::Automation` leaves; `BackpressureQueueKind` decode refuses 2. - The five counters leave `CounterId`; `parse_counter_id` refuses 2 and 11-14. None is - reallocated: the next additions take new numbers (#1312's 16, #1351's next free value). - `retired_code_rows` gains a row for queue kind 2 in a backpressure payload and for each counter - ID in a `COUNTERS_GET` and a `TELEMETRY_CONFIGURE` request. -- **D3. The page bound** is renamed to a name of its own for the meter and counter pages (for - example `MAXIMUM_TELEMETRY_PAGE_RECORDS`), still 256, at every use. -- **D4. The C ABI** builds no automation queue and reads neither S nor the quantum for it; the - test-support report loses its row. `capi_retained_bytes` falls by the removed rows; the - exact-charge oracle stays exact, and the reference budgets are ceilings that still hold. The - compile limit S keeps its validation until #1668 reserves it. -- **D5. Tests.** The queue-only tests above are deleted; the wrap test keeps its other queues. -- **D6. Docs.** The registry lists queue kind 2 and the five counters as retired with this slice - as the reason, and removes the record line (`:45`); the semantics, sizing and realtime-memory - lines lose the queue. -- **D7. The acked-batch question** no longer has a subject on the protocol: no queue remains to - hold a batch, so no ack can precede a drop. - -## Deliverables - -1. D1-D3 and D5 in `crates/protocol`; D4 in the C ABI control plane; the audit configuration. -2. D2's retired rows; D6's docs. -3. The gates below. - -## Authorized paths - -- `crates/protocol/**` -- `crates/control-plane/src/`, the files that hold capi's compile and control code after #1309 - (which lands before batch P1; `crates/capi/src/runtime/{compile,control}.rs` on `6ee64f484`) -- `crates/conformance/src/protocol_corpus.rs` (retired rows only) -- `tools/audit/src/protocol.rs` (the queue configuration) -- `docs/CONTROL_PROTOCOL_REGISTRY.md`, `docs/CONTROL_PROTOCOL_SEMANTICS.md`, - `docs/CONTROL_PROTOCOL_SIZING.md`, `docs/REALTIME_MEMORY.md` -- `docs/C_ABI_V1_QUALIFICATION.md` (the retained-byte rows that included the queue; Hazards) -- `crates/capi/src/runtime/tests.rs` (only a use of `protocol::AutomationRecord` that #1627 - and #1628 leave, such as `:1915` and `:2370` on `6ee64f484`) - -## Non-goals - -- Wire identities, dispatch, capabilities and the corpus hash (#1627 and #1628). -- The C limit `maximum_automation_spans_per_block` (#1668). -- The other queues and their configuration. - -## Hazards - -- **Batch P1.** This slice deletes what #1627 and #1628 leave unreachable; it cannot merge - before them (#1627's dispatch enqueues, #1628's cancellation dequeues), and they must not reach - `main` without it. -- **#1351.** *Report each configured counter's own value in the C ABI counter snapshot* (#1351) is - amended (README amendment row) never to serve `CANCELED_AUTOMATION`, so the merge order of #1351 - and this slice does not matter and no served counter is removed later. -- **Retained-byte rows.** Any exact C ABI resource figure in docs that included the queue - (`docs/C_ABI_V1_QUALIFICATION.md` tables) moves; list each with the reason "#1629 deletes the - automation queue". A ceiling that still holds is not lowered here. - -## Objective gates - -1. **Retired codes refused** (`crates/protocol/src/controller/tests.rs`, through - `retired_code_rows`). A backpressure payload with queue kind 2 fails decode; a - `COUNTERS_GET` or `TELEMETRY_CONFIGURE` naming counter 2, 11, 12, 13 or 14 is refused as an - invalid field, with no state change. -2. **Queues still work** (`crates/protocol/src/queue.rs` tests). The rewritten wrap test passes for - the control, response, event and telemetry queues; `resource_report_for_config` and `prepare` - agree on every remaining row. -3. **C ABI charges stay exact.** `capi_retained_bytes_charge_every_byte_the_compile_retains` - (`crates/capi/tests/resource_lifecycle.rs:926`) and - `reference_session_retained_rows_stay_within_their_budgets` pass; the PR records - `capi_retained_bytes` on the reference session before and after. -4. **No rendered bit moves.** `cargo build --locked --release -p audit -p bench -p capi -p - session-validator`, then `./target/release/audit capi` (zero allocations, locks and syscalls, - the same `pcm_digest` as base, PR evidence) and - `bash scripts/run-protocol-allocation-audit.sh target/release/audit`. -5. **Commands:** - - `cargo test --locked -p protocol --features test-support`, `cargo test --locked -p capi`, - `cargo test --locked -p conformance` - - the workspace debug leg (`test-debug-a` in `.github/workflows/qualification.yml`) - - `bash scripts/check-protocol-control-policy.sh`, `bash scripts/test-protocol-control-policy.sh`, - `bash scripts/check-workspace-policy.sh` - - `CARGO_TARGET_DIR=target/ci/wasm-simd RUSTFLAGS='-C target-feature=+simd128' cargo check --locked --target wasm32-unknown-unknown -p target-smoke -p protocol` - - `bash scripts/check-cross-targets.sh` (README F19, the iOS memset rule: no new - `memset_pattern16` call; fix one in code, never by a ceiling) - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` - -## Test value - -- Gate 1 turns red if a retired queue kind or counter still decodes, or is reused. No test refuses - them today. -- Gate 2 turns red if removing the automation rows breaks another queue's sizing or wrap. -- Gate 3 turns red if the C ABI still allocates a row it no longer charges, or charges a row it no - longer allocates. - -## Dependencies - -Batch P1. Direct dependency: - -- #1628 *Retire the AUTOMATION_CANCELED event and the cancellation path*, which brings #1627. - -*Extract the C ABI control plane into a portable crate both hosts call* (#1309) arrives through -#1627. #1668 *Reserve the C ABI's automation span limit* follows this draft. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Moved files.** The queue configuration is `protocol_queue_config` - (`crates/control-plane/src/compile.rs:185-205`), which both hosts build: the browser passes its S - through `control_limits` (`hosts/host-web/src/lib.rs:4494`). The report row is - `crates/control-plane/src/control.rs:252-253`, `:929`. D4 applies to both hosts. -- **#1351 has landed** and serves `CANCELED_AUTOMATION`; this slice retires counter 12 as written. -- **The record line** is `docs/CONTROL_PROTOCOL_REGISTRY.md:93` (not `:45`); #1626 gives the hold - rule its new home first. -- Batch P1, ready. - -## Decision record: #1629 delete the protocol automation queue (batch 10) - -**What shipped.** The automation queue and everything that fed it are gone from `protocol`, the -control plane, host-core, the audit and the policy tool; no wire identity other than the retired -codes below changed. - -- `crates/protocol/src/queue.rs`: `ParameterHandle`, `AutomationKind`, `AutomationRecord` (and its - 32-byte codec), `AutomationBatchError`, `AutomationBatchSlot`, `AutomationEnqueueError`, the - automation `QueuePair`, frontier, density and interval rows, `try_enqueue_automation`, - `try_dequeue_automation`, `validate_automation_admission` and its helpers, - `QueueKind::Automation`, and the config fields `automation_batch_slots`, - `per_block_automation_density` and `quantum_frames`. `resource_report_for_config` and `prepare` no - longer size or allocate them. The remaining queues keep their generation numbers (1, 3, 4, 5, 6): - a gap at 2 is the retired queue's. -- `crates/protocol/src/lib.rs`: `AUTOMATION_RECORD_BYTES` and the exports leave. - `AUTOMATION_BATCH_RECORDS` is renamed `MAXIMUM_TELEMETRY_PAGE_RECORDS` (still 256) at its four - uses, the meter and counter pages (D3). -- `message_wire.rs`: `CounterId::{AutomationBackpressure = 2, LateAutomation = 11, - CanceledAutomation = 12, AutomationTimePast = 13, AutomationOrderReject = 14}` and - `BackpressureQueueKind::Automation = 2` leave; `parse_counter_id` refuses 2 and 11-14 (and every - other unallocated ID) as `InvalidField`, the backpressure decode refuses 2 as `InvalidTlv`. - `controller.rs` drops the `QueueKind::Automation` arm and a stale `large_enum_variant` allow. -- Control plane (`compile.rs`, `control.rs`): `protocol_queue_config` takes only the limits and - builds no automation row; the quantum is no longer read for the queue and `capi_resources` loses - its `quantum_frames` argument; the test-support report loses its `automation` row. Both hosts - build the queue here (the browser through `control_limits`), so D4 applies to both. The compile - limit S (`maximum_automation_spans_per_block`) keeps its validation, untouched (#1668). -- host-core: `SERVED_COUNTERS` is four IDs (`TELEMETRY_COALESCED`, `TELEMETRY_DROPPED`, - `LIVE_VALUES_SUPERSEDED`, `TRANSITION_REPREPARE_REFUSALS`); `CANCELED_AUTOMATION` is no longer - served. -- `tools/realtime-policy/src/policy.rs`: the `try_dequeue_automation` allowlist row leaves - (13 rows to 12). `tools/audit/src/protocol.rs`: the queue configuration loses the three fields. -- Docs: registry (queue kind line, the retired paragraph for kind 2 and the five counters, the - record line, counter ID range), semantics (queue and late-counter sentences), sizing (the - automation and density rows and the 10,000-record paragraph), realtime memory. - -**Tests.** Deleted (existed only for the queue): `transient_record_has_exact_manual_le_codec`, -`ten_thousand_events_fit_as_exactly_forty_atomic_batches`, `past_order_overlap_and_density_reject_wholly`, -`overlap_is_rejected_across_separately_queued_batches`, -`invalid_and_out_of_order_slots_return_whole_unchanged_batches`, -`zero_record_and_aggregate_frontier_density_reject_wholly`, and the controller test helpers `batch` -and `controller_at_sample`. Rewritten: `queue_resource_projection_is_nonzero_and_shares_prepare_overflow_rules` -(the overflow now comes from the event queue's capacity), `full_empty_wrap_and_generation_preserve_original_items` -(gate 2: response, event and telemetry queues, each full outcome returning the original item with its -report; the control queue has no enqueue method, so it is checked by report only), -`every_counter_id_round_trips_and_the_next_is_refused` (the allocated set is pinned by value; 2, 11-14 -and 18 are refused one ID per request, so the sorted-unique rule cannot be the cause), -`a_revision_change_and_a_locate_reserve_and_emit_only_their_own_events` (loses its enqueue line), -`retired_and_console_refused_codes_meet_their_conformance_rows` (16 + 4 + 7 + 13 + 1 + 4 + 10 + 1), -and the C ABI counters tests and the pinned counters response vector (request 8: length 0x100 to -0xd0, TLV count 6 to 5, one 40-byte value message fewer). - -**Gates.** - -1. Retired codes refused. `retired_code_rows` gains ten rows: `retired.counter_{2,11,12,13,14}.telemetry_configure` - (`INVALID_FIELD`, `protocol.failure`) and `retired.counter_{...}.counters_get` (`NOT_FOUND`, - `protocol.failure`), judged by `retired_and_console_refused_codes_meet_their_conformance_rows`. - Queue kind 2 in a backpressure payload: `a_backpressure_payload_naming_the_retired_automation_queue_is_refused` - (message_wire tests; 1, 3, 4, 5 decode, 0, 2 and 7 fail). -2. Queues still work: `full_empty_wrap_and_generation_preserve_original_items` and - `queue_resource_projection_is_nonzero_and_shares_prepare_overflow_rules`. -3. C ABI charges stay exact: `capi_retained_bytes_charge_every_byte_the_compile_retains` and - `reference_session_retained_rows_stay_within_their_budgets` pass. `capi_retained_bytes` on the - reference session: 197,473 before, 164,249 after (33,224 bytes: the batch ring, the density and - interval rows); the ceiling 216,832 stands, not lowered. `largest_named_allocation_bytes` is 48,915 - both ways. -4. No rendered bit moves: `audit capi` `pcm_digest` `76525e7ef9e78cf4`, equal to base, with 0 - allocations, locks and syscalls; `run-protocol-allocation-audit.sh target/release/audit` passes. -5. Commands: protocol (`--features test-support`), control-plane (`--features test-support`), - host-core (`--all-features`), capi, conformance, host-web (`--features host-web/test-support`) and - parameter-metadata tests; clippy `--workspace --all-targets --all-features -D warnings`; fmt; - wasm parity and `--self-test`; both vocabulary scripts; `check-protocol-control-policy.sh` and its - test; `check-workspace-policy.sh`; `check-sdk-generated.sh`; `realtime-policy` (131 regions); - fuzz `cargo check`; `check-cross-targets.sh` (no new `memset_pattern16`). - -**Departures.** - -- **A `COUNTERS_GET` naming a retired counter is `NOT_FOUND`, not `INVALID_FIELD`.** Its IDs are raw - `u32`s that the provider looks up (`CountersRequest`: "unknown values return typed provider - NOT_FOUND"); refusing five numbers at decode while 99 stays `NOT_FOUND` would make the registry - inconsistent. A `TELEMETRY_CONFIGURE` is a decode refusal and is `INVALID_FIELD`: `parse_counter_id` - now returns `InvalidField` (was `InvalidTlv`) for every unallocated ID, as the control tables do. - Both leave no state change. -- **Response-side retirements are not conformance rows** (as in #1627): queue kind 2 in a backpressure - payload is a response field, so it is a message_wire decoder test, not a `RetiredCodeRow`. -- **The wrap test covers response, event and telemetry, not the control queue's enqueue**, which has - no method (removed unused in #1034). -- **No re-pin.** No corpus frame carried the queue, the record or a retired counter, so - `COMPLETE_SCHEMA_HASH` (`0x4b21_d9e6_f520_afd9`), the parity self-test strings, the fuzz manifest and - the conformance manifest stand. Only the C ABI counters-response vector moved (above). The - conformance record's frame count is unchanged (45). -- **The hold rule has no new home yet.** The spec deletes the registry's record line; its one rule - not stated elsewhere, "a segment applies on `[start,end)` and holds its end value until replaced", - was waiting for #1626 to give it a home in `docs/SESSION_SCHEMA_V1.md`, which has not - landed on this base. The registry keeps that sentence alone, pointing at #1626; #1626 moves it. -- Mechanical edits outside the authorized paths: `crates/host-core/src/control_provider.rs` - (the served counter set and its test), `tools/realtime-policy/src/policy.rs` (the allowlist row), - `crates/conformance/src/protocol_corpus.rs` (the former-payload row no longer builds an - `AutomationRecord`; its 32 bytes are written out literally, so the row's bytes are unchanged), - `crates/protocol/src/controller.rs` and `crates/protocol/src/message_wire.rs` (named in the spec), - `docs/CONTROL_PROTOCOL_SIZING.md` line 34 (the word "density"). -- `docs/C_ABI_V1_QUALIFICATION.md` states no exact retained-byte figure that included the queue, so - it is unchanged (Hazards: nothing moved there). - -## Fold (batch 10 gates): the set-control timeline memory re-pin - -`hosts/host-web/tests/browser-v1/expected.json` `memoryBytes` for the set-control timeline row -re-pinned 2,949,120 -> 2,883,584 (one page, 65,536 bytes: the removed protocol automation queue's -heap), measured with `scripts/check-browser-expected-resources.py` on the batch 10 build. diff --git a/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md b/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md deleted file mode 100644 index f85bcc66f..000000000 --- a/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md +++ /dev/null @@ -1,138 +0,0 @@ -# Carry, then retarget a strip input section across a plan swap - -Successor found by the batch-10 carry-chain check, decision 17 -(`docs/rulings/one-control-abstraction-2026-10-13.md`). Code anchors verified on `12d4a4f9d`. -Issue number: assigned at filing (written here as #1630). Feature sprint mode (owner, 2026-10-13): -one implementer, normal tests, a batch review. - -## Product outcome - -A structural edit that also changes a strip's live input value (trim, polarity, an input filter) -keeps the strip's input section running: the successor carries the predecessor's input state and -then ramps to the new value, on both hosts, instead of restarting the section. - -## Context - -- The input join requires every input value to be equal: `same_input_section` - (`crates/host-core/src/prepare.rs:2726`, used at `:1803`). Any live input difference in the same - transaction as a structural edit restarts the input section (its filter state and ramps). -- The effect carry follows the same pattern: #1280a carries, #1280c retargets a live difference - after the carry (D4). -- The C ABI behaves the same today, so this does not block #1562b. - -## Decisions frozen for this slice - -- **D1.** The input join keys on the section's structure (which builtins exist, their designs' - shape), not on live values; a live difference carries the state and retargets. -- **D2.** The retarget writes through the successor's input producer, stamped on its gate, in the - block that writes the strip retargets, with the session's ramps; filter retargets use the - existing live input-filter path (#1556). -- **D3.** The carry is allocation-, lock- and syscall-free on the swap block. - -## Objective gates - -1. A host-core successor-swap test: a structural edit plus a trim change on a surviving strip; the - input section's state carries (no restart transient) and the trim ramps to the new value, - bit-equal to a plan that made the same edit live. -2. The same for an input filter cutoff change and a polarity change. -3. Zero allocations and frees on the swap block; `cargo test --locked -p host-core --all-features - -p capi -p host-web --features host-web/test-support`; clippy. - -## Authorized paths - -- `crates/host-core/src/prepare.rs` (the input join and retarget), `crates/host-core/tests/successor_swap.rs`, - the builtins input stage's carry entry if it lacks one, this spec's decision record. - -## Dependencies - -- #1280c (the retarget pattern). Lane K. - -## Size - -Half a day to a day. - -## Decision record: #1630 (batch 14b) - -**Shipped.** The input join in `crates/host-core/src/prepare.rs` (the `carried_inputs` block of -the successor's preparation) now keys on the section's structure. Every strip's input section has -the same stages (trim, polarity, two filter sections per channel), so the structure is the control -kind (`row.live == live_input`, as before) plus the rate the designs are for. On a plan with input -cells, a surviving strip of the same kind always carries, and the records the live path gives the -same edit (`input_records`, #1556) become its retarget (`InputJoin::retarget`). They are written -through the successor's own `InputCells`, stamped on its held gate in the same block as the strip, -effect and route retargets, with the session's ramps or the transaction's edit ramp for -`(Strip, Input)` (#1394 D6). A plan without input cells cannot retarget, so it still carries only -bit-equal sections (`same_input_section`, unchanged #1276 behaviour). - -**Departures and code truth.** - -- `input_records` and `filter_edits` moved from `live_delta.rs` (built only with - `control-provider`) to `prepare.rs` (every build), as the strip-retarget helpers did for #1277 - D5; the classifier imports them and maps `OutOfDomain` to `LiveRebuild::Domain`. One function, - so a retarget and a live edit cannot disagree. This edits `crates/host-core/src/live_delta.rs`, - outside the authorized paths, mechanically (move, imports, one doc link). -- The join adds a sample-rate check (`base.inventory.sample_rate_hz == compiled rate`): a carried - lane's coefficients and filter state are designed for the predecessor's rate, so a rate change - is a structural difference (D1, "their designs' shape"). The check is defensive: the control - plane and `adopt_predecessor_plan` already refuse a successor at another rate, so it cannot - change any output today. -- A value the live path refuses (out of domain) restarts the section instead of carrying; a - compiled model does not reach it, and a debug build asserts so (review fold). -- No builtins or `crates/engine/src/realtime/` edit was needed: the input stage's carry entry - (#1276 `export_lane`/`import_lane`) and the successor's cell drain already order carry before - retarget on the swap block. No new realtime region. - -**Superseded tests.** `a_restarting_input_lane_does_not_count_its_replaced_writes` and -`a_pending_trim_record_in_a_later_bank_survives_the_swap` restarted one strip by a trim difference -between committed and B, which now carries. They now restart it with a committed model that lacks -the strip (`without_track`), so they keep their discrimination. Gate 4 of #1276 -(`a_strip_whose_input_section_changed_starts_at_rest`) runs on plans without input cells and stays. - -**Gates** (`crates/host-core/tests/successor_swap.rs`, `control-provider`, both widths each): - -1. `a_trim_change_carries_and_retargets_the_input_section` -2. `a_filter_cutoff_change_carries_and_retargets_the_input_section` (HPF on both lanes, LPF one - lane), `a_polarity_change_carries_and_retargets_the_input_section`. - -Each: the carry-then-retarget run equals, in every block, a run that wrote the classifier's records -for the same edit to the predecessor before the swap; it differs at the swap block from a run whose -section restarts and from a run without the edit. Red on the old join (first difference at block 6) -and red when the retarget write is dropped. -3. The same runs: swap block 0 allocations, reallocations and frees (`bench_support::alloc` thread - counters after `audit::warm_up`) and 0 audited allocations. `cargo test --locked -p host-core - --all-features`, `-p capi`, `-p host-web --features host-web/test-support`: pass. Clippy - `-p host-core --all-targets` with and without `--all-features`: clean. `audit capi`: - `pcm_digest` `bf7dcedd1d92621f`, 0 violations. `realtime_policy`: ok, 149 regions (unchanged). - -**Review fold (batch 14b).** (f) Root ruling: D15-2's `live_values_superseded` is exact, so a -carried value the successor's retarget replaces in the adopting block is counted, as an effect -lane's carried write is. The input and strip carry drains apply a predecessor's unread value to lane -state at the swap; until now, when the successor's own cell (the retarget) applied in the adopting -block, before any sample, that carried value was replaced unheard and not counted. Each render-side -cell reader (`InputReader`, `FaderReader`, `MatrixReader`, `crates/builtins-compiler/src/lib.rs`) -now keeps `CarryMarks`: the cells its last drain applied, and the cells a swap carried in. The -adopting lane takes the predecessor reader's applied cells (`adopt_input_lane`, the fader and matrix -banks' `adopt_lane`, which the fused bank shares); the successor's first drain counts one -superseded value per carried cell it applies (`LiveValuesSuperseded::add_carried`) and clears the -marks, so from its second block a carried value counts as heard. No `crates/engine/src/realtime` -or `graph` edit; the new code sits in the existing realtime-policy regions plus one new region (the -`CarryMarks` methods and `adopt_carry_marks`): `realtime_policy` ok at 150 regions, and its -floor (`tools/realtime-policy/src/policy.rs`) rises 149 -> 150 in this change. The marks add four bytes per -reader: `crates/capi/tests/resource_lifecycle.rs`'s eight-lane `builtin_bank_bytes` measured 15,321 --> 15,705 (+384), over its 15,680 ceiling, re-pinned by that file's rule (plus 10 %, up to 64) to -17,280; the four-lane ceiling was not re-measured here. Tests (`successor_swap.rs`, both widths, both strip -forms): `a_carried_input_value_the_retarget_replaces_is_counted_on_every_width` and -`a_carried_strip_value_the_retarget_replaces_is_counted_on_every_width_and_form`: a one-lane write -pending at the swap and committed in the base, then moved again by B, counts exactly 1 on the -successor (0 on the predecessor), is bit-identical in every block to a run whose base never held -the write, and counts 0 when B leaves the value (no retarget). Red with the count removed: -`(0, 0)` against `(0, 1)` in both. The harness reads the successor's own count -(`successor_superseded`), since the test seam gives each plan its own counter. (g) -`an_input_difference_carries_and_retargets` also runs every case with a transaction that gave the -strip's `(Strip, Input)` row a 37-sample ramp, through the successor path (`direct_run_ramped`, -`SuccessorBase::edit_ramps`): it equals the run that writes the classifier's records for the -ramped edit, and for trim and polarity (whose records the row changes) differs at the swap block -from the session-ramp retarget. Red when the join ignores the edit ramps (`StripEditRamps::default()` -at the `input_records` call): trim and polarity fail, first difference at block 6. (h) The -`input_records(..)` call in the join is now `debug_assert!`ed `Ok` before `.ok()?`: an internal -refusal would silently restart the section. diff --git a/.github/ISSUE_SPECS/1631-run-the-automation-row-rule-when-a-rows-activity-changes.md b/.github/ISSUE_SPECS/1631-run-the-automation-row-rule-when-a-rows-activity-changes.md deleted file mode 100644 index f88098c07..000000000 --- a/.github/ISSUE_SPECS/1631-run-the-automation-row-rule-when-a-rows-activity-changes.md +++ /dev/null @@ -1,101 +0,0 @@ -# Run the automation row rule when a row's activity changes - -Defect found by the stored-automation check (#1058 README, finding F21), decision 17 -(`docs/rulings/one-control-abstraction-2026-10-13.md`, C1 and C12). Code anchors verified on -`2e6343a99`. Issue number: assigned at filing (written here as #1631). Feature sprint mode (owner, -2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -A transaction that switches a strip's `matrix_mode` while a stored automation entry targets that -strip's pan or matrix rows never commits a model that its own next rebuild refuses. The switch is -refused with the row rule's `control.inactive` at the entry's path, on both hosts, unless the same -transaction also retargets or removes the entry. Found by reading, not reproduced. - -## Context - -- `matrix_mode` is live (C12). Its write swaps `MatrixOrPan`, which flips which of the strip's pan - and matrix rows are active (`write_matrix_mode`, `crates/controls/src/strip.rs:200-227`). -- The row rule refuses an entry on an inactive row with `control.inactive` - (`controls::automation_diagnostics`, `crates/controls/src/automation.rs:72`, the `active` check). -- The classifier runs the row rule only when the automation table changes: step 2, - `current.automation != next.automation && !automation_diagnostics(next, ..).is_empty()` - (`crates/host-core/src/live_delta.rs:371-376`). A switch with an unchanged table passes step 2, - step 3 lowers it live, and the committed model holds a pan entry on a matrix strip (or the - reverse). The next rebuild's preparation calls the row rule (`crates/host-core/src/prepare.rs`) - and refuses that model with `control.inactive`. A document replacement that changes only the - mode reaches the same path. -- *Make a send's follows_mute and a strip's matrix_mode non-automatable* (#1632) removes the - automated switch. That narrows the fault class but does not remove it: a live `SET_CONTROL` or a - document replacement still flips the rows' activity under a pan or matrix entry. - -## Decisions frozen for this slice - -- **D1. Run the row rule on any commit that changes a row's activity.** Step 2 runs - `automation_diagnostics(next, ..)` when the automation table changes **or** when any row's - `active` differs between `current` and `next`. `controls` gives the one predicate - (`controls::activity_changed(current, next)`): today it is true exactly when a strip's - `matrix_or_pan` variant differs; a later row whose activity depends on a value joins it there, so - the classifier keeps no row list. -- **D2. The existing route.** A refused model gives `LiveRebuild::AutomationTarget`; the rebuild's - preparation refuses it with the row's code at the entry's path, so nothing commits and no ack - precedes the refusal (the acked-batch question: no). A transaction that switches the mode and - retargets or removes the entry passes. -- **D3. Rejected: refuse the switch in the write.** `write_matrix_mode` sees one row, not the - automation table; the row rule is the one author (C1). - -## Objective gates - -1. `crates/host-core/tests/live_delta.rs`: a strip with a pan entry takes `SET_CONTROL matrix_mode - = matrix`; the classifier returns `Err(LiveRebuild::AutomationTarget)`. The same transaction on - the C ABI (`crates/capi/src/runtime/live_tests.rs`) and in the browser - (`hosts/host-web/tests/control_parity.rs`) is refused `control.inactive` at - `$.automation[id=].target`, and the committed revision is unchanged. -2. The switch together with `SetAutomationTarget` to the matching matrix row, in one transaction, - commits. -3. A switch on a strip with no pan or matrix entry stays `live`, with no rebuild. -4. `cargo test --locked -p controls -p host-core --all-features -p capi -p host-web --features - host-web/test-support`; clippy `-D warnings`; `cargo fmt --all -- --check`. - -## Authorized paths - -- `crates/host-core/src/live_delta.rs` (step 2), `crates/controls/src/{lib.rs,diff.rs}` (the - predicate), `crates/host-core/tests/live_delta.rs`, `crates/capi/src/runtime/live_tests.rs`, - `hosts/host-web/tests/control_parity.rs`, this spec's decision record. - -## Dependencies - -- None. Independent of the stored-automation slices and of #1632. - -## Size - -Half a day. - -## Decision record: #1631 (batch 8) - -Shipped. Anchors moved: step 2 sits at `crates/host-core/src/live_delta.rs` ~344 (the spec said -371-376); the rule is otherwise as filed. - -- `controls::activity_changed(current, next)` (`crates/controls/src/diff.rs`, re-exported from - `lib.rs`): true when the two models do not list the same strips in the same order, or when a strip's - `matrix_or_pan` variant differs. A later value-dependent row joins it there. -- Classifier step 2 runs `automation_diagnostics(next, ..)` when the automation table changes or - `activity_changed` holds. A refusal is `LiveRebuild::AutomationTarget`; the rebuild's preparation - refuses it `control.inactive` at `$.automation[id=].target` before any commit. -- Departure (test shape): the browser parity gate is a separate test, not a script row. A rebuild - route refusal is a non-ok entry result with the diagnostic in the last-error buffer on both hosts - (browser 10, C ABI 5: each host numbers its results), not a typed verdict frame, and the script - harness asserts an ok result. `CAbi::send_status` and `CAbi::last_error` were added to - `control_parity.rs` for it (the diagnostic is read before the reliable-lane drain, which clears it). - -Gates and tests: -- Gate 1 and 3 (classifier): `host-core/tests/live_delta.rs::a_mode_switch_that_strands_an_automation_needs_a_rebuild` - (pan entry then matrix switch, and the reverse, give `AutomationTarget`; another strip's entry and a - strip with only a fader entry stay live). -- Gate 1 and 2 (C ABI): `capi/src/runtime/live_tests.rs::set_control::a_mode_switch_under_a_stored_pan_entry_is_refused_before_any_commit` - (refused, diagnostic names `control.inactive` and the path, state unchanged; switch plus - `SetAutomationTarget` commits; an entry-free strip switches live). -- Gate 1 and 2 (parity): `hosts/host-web/tests/control_parity.rs::a_mode_switch_under_a_stored_pan_entry_is_refused_alike_on_both_hosts` - at all four rates (same diagnostic bytes; the same expected revision then commits the switch with - the retarget, `model_only` because the switch keeps the effective 2x2). -- Red on revert: all three fail with step 2 restored to the table-only condition. diff --git a/.github/ISSUE_SPECS/1632-make-send-follows-mute-and-matrix-mode-non-automatable.md b/.github/ISSUE_SPECS/1632-make-send-follows-mute-and-matrix-mode-non-automatable.md deleted file mode 100644 index c57d34ff0..000000000 --- a/.github/ISSUE_SPECS/1632-make-send-follows-mute-and-matrix-mode-non-automatable.md +++ /dev/null @@ -1,103 +0,0 @@ -# Make a send's follows_mute and a strip's matrix_mode non-automatable - -Root's ruling on finding F20 of the stored-automation check (#1058 README, "Root's rulings -(2026-10-14)"), decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`, C3). Code anchors -verified on `2e6343a99`. Issue number: assigned at filing (written here as #1632). Feature sprint -mode (owner, 2026-10-13): one implementer, normal tests, a batch review. Root reports the ruling to -the owner as revisitable. - -## Product outcome - -A route's `follows_mute` and a strip's `matrix_mode` stay live controls but are not automation -targets: the control list says `automatable = false` for both, and a stored automation entry on -either is refused `control.automation.target` at preparation, in the classifier, in -`session-validator` and in the SDK's generated tables, on both hosts. They are a routing setting -and a mode, not performance values. - -## Context - -- `automatable` is computed, never declared: live, model scope and not a `Setting` - (`ControlDescriptor::automatable`, `crates/controls/src/descriptor.rs:124-132`). Both rows pass it - today: `matrix_mode` (`crates/controls/src/strip.rs:80-96`, family `StripMatrix`) and - `follows_mute` on a route into a submix (`crates/controls/src/route.rs:170-190`, family `Route`). -- The row rule refuses a target that is not automatable with `control.automation.target` - (`controls::automation_diagnostics`, `crates/controls/src/automation.rs:72`). Preparation, the - classifier and `session-validator` call it. -- The wire check is an implication (an automatable row must be live, model scope, not a setting: - `crates/protocol/src/message_wire.rs:3294`), so a live row that is not automatable is already a - valid control row. `docs/CONTROL_PROTOCOL_REGISTRY.md:54` states the rule. -- No checked-in fixture automates either row (`fixtures/session/v1/builtins-automation.json` holds - builtins rows of `vocal`, `route-ride` on route control 1 and `vca-ride`). -- The stored-automation drafts assume this ruling: #1654 (amended) renders no `follows_mute` - entry, and a `matrix_mode` entry would make rows active and inactive over time, which no draft - covers. - -## Decisions frozen for this slice - -- **D1. One declared exclusion.** `ControlDescriptor` gains one field that defaults to "computed" - in `BASE` and that a row may set to an exclusion with its reason. `automatable()` becomes the - computed rule **and** not excluded. `matrix_mode` declares "a mode, not a performance value"; - `follows_mute` declares "a routing setting, not a performance value". Both reasons name the - reopening condition: an owner request. Every other row keeps its value. -- **D2. The row rule needs no new code.** Its `!descriptor.automatable()` check refuses both rows - with `control.automation.target`. -- **D3. Docs.** `docs/CONTROL_PROTOCOL_REGISTRY.md:54` and the automation-target text of - `docs/SESSION_SCHEMA_V1.md` (near `:252-267`) name the two exclusions; decision 17's record gains - one line under C3 (root's ruling, revisitable). -- **D4. Fixtures, corpus and SDK follow.** The SDK's generated control tables are regenerated - (`automatable` of the two rows); every pinned digest that moves (a control-list corpus row in - `crates/conformance/src/protocol_corpus.rs`, the qualification artifacts) is re-pinned with the - reason "#1632 makes follows_mute and matrix_mode non-automatable". - -## Objective gates - -1. `crates/controls/tests/automation.rs`: an entry on a send's `follows_mute` and an entry on a - strip's `matrix_mode` are refused `control.automation.target` at `.target`; preparation and - `session-validator` report the same code. -2. `crates/controls/tests/list.rs`: both rows list `automatable = false`; every other row's - `automatable` is unchanged (the full table compared with the previous one, except the two rows). -3. `bash scripts/check-sdk-generated.sh target/ci/qualification-artifacts` and - `bash scripts/check-sdk-types.sh` pass with the regenerated tables. -4. `cargo test --locked -p controls -p protocol -p conformance -p host-core --all-features - -p session-validator`; clippy `-D warnings`; `cargo fmt --all -- --check`. - -## Authorized paths - -- `crates/controls/src/{descriptor.rs,strip.rs,route.rs}`, `crates/controls/tests/{automation.rs,list.rs}` -- `crates/conformance/src/protocol_corpus.rs` (re-pins only), `sdk/src/generated/*` (regenerated) -- `docs/CONTROL_PROTOCOL_REGISTRY.md`, `docs/SESSION_SCHEMA_V1.md`, - `docs/rulings/one-control-abstraction-2026-10-13.md` (one line), this spec's decision record - -## Dependencies - -- None. #1654 of the stored-automation program (README) assumes it (ordering only). - -## Size - -Half a day. - -## Decision record: #1632 (batch 13a) - -Shipped as written (D1 to D4). `ControlDescriptor` gains `automation: AutomationRule` (`BASE` and every -other table: `Computed`); `automatable()` is the computed rule and `automation == Computed`. -`matrix_mode` ("a mode, not a performance value") and `follows_mute` ("a routing setting, not a -performance value") declare `AutomationRule::Excluded(&AutomationExclusion { reason, reopen: "an -owner request" })`. The row rule (`controls::automation_diagnostics`), preparation, the classifier -and `session-validator` need no new code: all four already refuse `!automatable()` as -`control.automation.target`. The wire check (an implication) is unchanged. - -Departure from the text: the exclusion is held by reference (`Excluded(&'static AutomationExclusion)`), -not inline. A listing copies one descriptor per row, and an inline pair of strings measurably grew -the live-only `control_command` heap (#1615 gate: first peak 81,451 against a ceiling of 78,000); -the reference costs one word per row (first peak 77,347, steady 72,588; ceilings 78,000 and 73,000 -hold). - -Gates and tests: `crates/controls/tests/automation.rs` -`the_row_decides_what_can_be_automated` (both rows refused at `.target`), -`crates/controls/tests/list.rs` `automatable_is_exactly_the_live_model_value_rows` (previous list -less the two names) and `follows_mute_and_matrix_mode_are_live_and_not_automatable`. Re-pins: the -host-web `tests/fixtures/control-list.json` (`automatable` of the five listed rows; reason -"#1632 makes follows_mute and matrix_mode non-automatable"), `sdk/assets/miso-engine-v1-parameter-metadata.json` -and `sdk/src/generated/controls.ts` (regenerated). No conformance corpus digest moved. No checked-in -automation fixture used either row. #1631 interplay: it still covers a pan entry on a strip -switched to matrix; a `matrix_mode` entry is now refused at the target before activity is judged. diff --git a/.github/ISSUE_SPECS/1633-allocate-an-effect-lanes-carried-slots-only-when-a-successor-carries.md b/.github/ISSUE_SPECS/1633-allocate-an-effect-lanes-carried-slots-only-when-a-successor-carries.md deleted file mode 100644 index dce351306..000000000 --- a/.github/ISSUE_SPECS/1633-allocate-an-effect-lanes-carried-slots-only-when-a-successor-carries.md +++ /dev/null @@ -1,116 +0,0 @@ -# Allocate an effect lane's carried slots only when a successor carries - -Successor of #1280a, decision 17 (`docs/rulings/one-control-abstraction-2026-10-13.md`). Code -anchors verified on the batch 10 branch `codex/controls-batch-10`. Issue number: assigned at filing -(written here as #1633). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a -batch review. Root, 2026-10-14: the first lane-K slice of batch 11, before #1282. - -## Product outcome - -An effect lane holds no carry storage until a plan swap carries it, so a first boot pays nothing for -the carry: the browser's boot growth regains its margin under the 20 MiB pin, and the C ABI retains -less. - -## Context - -- #1280a gave every effect lane preallocated carried slots (`carried_targets: - Box<[CarriedCell]>` and `carried_parameters`, built in - `effect_control_lane`, `crates/effect-contract/src/live.rs`). On the 1 MiB worst browser document - that is 672 B x 768 EQ lanes = 516 KB, plus lane-struct growth; boot growth went from 20,447,232 - to 20,971,520 bytes (exactly at the pin after batch 10's #1281 fix). -- The slots serve only a successor that carries: `EffectControlLane::carry_from` fills them and - `stage` resolves them in the successor's first block. -- `graph::install_effect_carry` already allocates the carry program's storage (`carry_scratch`) on - the control thread for the lanes in `moves`, charged through `effect_carry_retained_bytes`. - -## Decisions frozen for this slice - -- **D1.** The carried slots are allocated in `install_effect_carry`, on the control thread, for the - successor lanes in `moves` only, and charged through `effect_carry_retained_bytes`. A lane that no - successor carries holds none. -- **D2.** `EffectControlLane::carry_from` reads a missing predecessor slot as empty, and - `can_carry_from` refuses a successor lane whose slots were not installed, so a carry can never - write into storage that does not exist. -- **D3.** The swap block stays allocation-, lock- and syscall-free; render reads no new state. - -## Objective gates - -1. Boot budget: `node scripts/check-web-boot-budget.mjs` growth measured below the 20 MiB pin, with - the margin recorded in the decision record (batch 10: 20,971,520, zero margin). The multiplier - does not change. -2. `capi_retained_bytes` is tightened to the new measurement (measured + 10 %, rounded up to 64), at - both lane widths from the CI print. -3. Every carry gate of #1280a (and #1280b/c, #1281) passes unchanged, including zero allocations and - frees on the swap block; the charge-equals-freed resource tests hold. -4. `cargo test --locked -p effect-contract -p graph -p graph-compiler -p rack -p host-core - --all-features -p capi -p host-web --features host-web/test-support`; clippy; realtime-policy. - -## Authorized paths - -- `crates/effect-contract/src/live.rs`, `crates/graph/src/{lib.rs,runtime.rs}`, - `crates/graph-compiler/src/estimate.rs`, their tests, `crates/capi/tests/resource_lifecycle.rs` - (the ceilings), `hosts/host-web/tests/browser-v1/expected.json` (measured memory rows only), this - spec's decision record. - -## Dependencies - -- #1280a. Before #1282. - -## Size - -Half a day to a day. - -## Decision record: carried slots on demand (batch 11) - -Shipped D1-D3 as frozen. - -- `effect_control_lane` builds no carried slots. `EffectControlLane::install_carry_slots` (control - thread, idempotent, none for a lane without cells) allocates them; `carry_slot_bytes` is the one - spelling of their size; `carry_slots_installed` reports them. -- `graph::install_effect_carry` installs the slots of the successor sites in `moves` only (bank - lanes of a live stage, and per-node instances), after every move validated. `effect_carry_slot_bytes` - (new) reads the charge from the bound plan before the install; `effect_carry_retained_bytes` takes - it as a third argument; `carry_program_retained_bytes` sums the same function over the installed - successor sites, so admission and the retained row cannot disagree (the existing - `debug_assert_eq!` in `prepare.rs` ties them). -- D2: `can_carry_from` is the old shape check plus "this lane's slots are installed"; - `carry_from` reads a predecessor that holds no slots as empty (`.get(i)` with an empty default). - `Reads::read` takes `Option<&mut CarriedCell>`, so a lane without slots reads none in `stage`. - The shape check alone is now `same_cell_shape`. -- D3: the swap block allocates and frees nothing and reads no new state beyond the slot slices' - lengths; `stage` does one bounds-checked `get_mut` per cell, which `Box<[T]>` indexing did already. - -Departure (mechanical, outside the authorized list): `rack::LiveControlEffectBankStage::can_carry_instance` -was used in both directions of a bank-lane/per-node move, but the new rule needs the *successor's* -slots installed, so the into-instance direction got its own `can_carry_lane_into_instance` (the -instance is then the successor); `carry_slot_bytes(lane)` and `install_carry_slots(lane)` were added -on the stage; `crates/graph/src/runtime.rs` calls them; `crates/host-core/src/prepare.rs` passes the -slot charge into the admission. No edit under `crates/engine/src/realtime/`. - -Gates. - -1. Boot budget (`target/ci/b11k-artifacts`, scripts/check-web-boot-budget.mjs with the growth - printed): growth 20,971,520 -> 20,447,232 bytes; the pin is 20,971,520, so the margin is - 524,288 bytes (eight Wasm pages), the same growth as before #1280a. The multiplier is unchanged. - `check-browser-expected-resources.py` passes with no re-pin: no expected.json memory row moved - (a one-document first boot carries nothing). -2. `capi_retained_bytes` is 165,166 before and after (it charges capi's own allocations, not the - plan's lanes). Its eight-lane ceiling is tightened 216,832 -> 181,696 (165,166 + 10 %, rounded up - to 64). The four-lane value stays 216,832 until the next `aarch64-debug` print. The plan rows - the slots were in fell at eight lanes: `graph_session_plus_plan_bytes` 321,846 -> 315,798, - `graph_incremental_plan_bytes` 321,774 -> 315,726, `graph_metadata_bytes` 113,601 -> 107,553 - (-6,048 each); their ceilings (346,432, 346,368, 117,376 and the four-lane ones) still bound - them and are left for the same print. -3. All carry gates pass unchanged: `cargo test --locked -p host-core --all-features` (the whole - `successor_swap` suite, including zero allocations on the swap block and the charge tests), - `-p effect-contract -p graph -p graph-compiler -p rack -p capi`, `-p host-web --features - host-web/test-support`. -4. New tests, both in `crates/effect-contract/tests/live_control.rs`: - `carried_slots_are_installed_for_a_successor_only_and_charged_exactly` (a built lane holds no - slots and refuses a carry; the install grows `retained_resources` by exactly `carry_slot_bytes`; - a second install changes nothing) and - `a_predecessor_without_slots_hands_its_unread_write_to_an_installed_successor` (D2's - missing-predecessor-slot read). - Clippy `-D warnings`, fmt, rustdoc `-D warnings`, `check-builtins-fixtures.sh`, - `check-cross-targets.sh`, `check-web-audioworklet.sh` with the artifacts, `realtime-policy` and - `check-bench-policy.sh` pass. diff --git a/.github/ISSUE_SPECS/1635-retire-the-protocol-transport-position.md b/.github/ISSUE_SPECS/1635-retire-the-protocol-transport-position.md deleted file mode 100644 index 2fa26781c..000000000 --- a/.github/ISSUE_SPECS/1635-retire-the-protocol-transport-position.md +++ /dev/null @@ -1,206 +0,0 @@ -# Retire the protocol's stored transport position - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), README finding F4 as decided, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. Batch P1. - -## Product outcome - -The control protocol no longer stores a playhead that nothing plays. `TRANSPORT_SET` takes only the -transport state; its position field is retired and refused in either flag form. The transport -snapshot and the `TRANSPORT_STATE` event carry the state and the effective sample, with no position. -The engine has one playhead: the host moves it with source seeks today and with the session seek -of #1642 and #1643 later. Every retired field is refused, never reallocated -and never renumbered. - -## Context - -- **The stored position.** `TRANSPORT_SET` (`0x0008`) takes state `1:U8 R` and an optional - absolute position `2:U64 O` (`docs/CONTROL_PROTOCOL_REGISTRY.md:32`; - `crates/protocol/src/schema.rs:448-456`). The snapshot carries state, position and effective - sample (`schema.rs:458-467`), and so does the `TRANSPORT_STATE` event, with sequence and origin - (`schema.rs:469-486`). The typed forms are `TransportSetRequest`, `TransportSnapshot` and - `TransportStateEvent` (`crates/protocol/src/message_wire.rs:488-521`). -- **Nothing reads it.** The host-core provider stores the position and echoes it - (`crates/host-core/src/control_provider.rs:424-438`, field `:59`, seed `:161`), as the protocol's - `MockProvider` does (`crates/protocol/src/controller.rs:638-639`, `:848-860`). No render path and - no seek reads it; the boundary doc says seek execution stays outside the provider - (`docs/CONTROL_PROVIDER_BOUNDARY.md:5`). *Refuse commands that would be acknowledged with no - effect* (#1315) kept `TRANSPORT_SET` because its readback reaches the stored state. -- **Why it goes.** A1.2 of the #1058 note gives the engine a timeline that sources and automation - follow. A second, stored position that a peer can set and read back would contradict it, and it - cannot move the timeline: a seek needs the host to refill every ring at a new generation, which - only the host can do. The owner ruled that code nothing uses is removed - (`docs/rulings/engine-footprint-2026-09-28.md:20-21`). -- **The locate cancellation** that read the position (`controller.rs:3050-3056`) is gone after - #1628. -- **Retirement machinery** as in #1627: `schema_spec_retiring` - (`crates/protocol/src/btlv.rs:240-248`) and `retired_code_rows` - (`crates/conformance/src/protocol_corpus.rs:359-372`). -- **Corpus rows:** `command.transport_set` (`protocol_corpus.rs:827`), `response.transport_get` - (`:935`), `response.transport_set` (`:943`), `event.transport_state` (`:1065`). -- **Tests that set a position:** `crates/protocol/src/controller/tests.rs:2421`, - `crates/host-core/src/control_provider.rs:910`, `crates/capi/src/runtime/tests.rs:1900`, `:2403`, - `:2557`. -- **The audit corpus** builds `TransportSetRequest { state, position: None }` - (`tools/audit/src/protocol.rs:206`), so D2 changes that literal too. -- **The reliable event slot.** The `TRANSPORT_STATE` event's queued payload carries the position: - `ReliablePayload::TransportState { event_sequence, state, position, effective_sample, - origin_request_id }` (`crates/protocol/src/queue.rs:323-335`) and its constructor - `ReliableSlot::transport_state(revision, event_sequence, state, position, effective_sample, - origin_request_id)` (`:395-417`). `crates/protocol/src/controller.rs` builds it (`:3040`) and - reads it (`:2688`, `:3398`); `crates/protocol/src/controller/tests.rs:3206` and `:3416` call the - constructor. -- **Every user.** `git grep` on `423b9d4a1` (code equal to `6ee64f484`) for `TransportSetRequest`, - `TransportSnapshot`, `TransportStateEvent`, `ReliablePayload::TransportState`, - `transport_state(`, `transport_position`, `TRANSPORT_SET` and `TRANSPORT_STATE` (outside - `docs/handoffs/` and `.github/`) finds them only in `crates/protocol/`, `crates/host-core/src/control_provider.rs`, - `crates/capi/src/runtime/tests.rs`, `crates/conformance/src/protocol_corpus.rs` and - `tools/audit/src/protocol.rs`; the fuzz manifest names the position in prose - (`fuzz/corpus/complete-schema-manifest.md:7`). - -## Decisions frozen for this slice - -- **D1. Retired fields.** `TRANSPORT_SET` field 2, snapshot field 2 and event field 3 join - retired-field specs read through `schema_spec_retiring`, so a peer that sends one is refused in - either flag form. The remaining field IDs keep their numbers. -- **D2. Types.** `TransportSetRequest`, `TransportSnapshot` and `TransportStateEvent` lose - `position`; `ReliablePayload::TransportState` and `ReliableSlot::transport_state` lose it too - (`crates/protocol/src/queue.rs:323-335`, `:395-417`), and the controller stops passing - `snapshot.position`; the provider trait's `transport_get` and `transport_set` keep their shape without it; - host-core's provider and `MockProvider` lose `transport_position`. -- **D3. Re-pins**, each with the reason "#1635 retires the stored transport position": - `COMPLETE_SCHEMA_HASH` and its history line, the wasm-parity self-test strings, the fuzz manifest, - the conformance record and the C ABI vectors that carry a transport frame. `retired_code_rows` - gains one row per retired field. -- **D4. Docs.** The registry rows for `0008`, the snapshot and the event lose the position and list - the retired fields with this slice as the reason. `docs/CONTROL_PROVIDER_BOUNDARY.md:5` says - transport is an absolute state and effective sample, and that the playhead is moved by host - seeks, not by the transport. -- **D5. The acked-batch question: can an ack ever precede a drop?** No. A set with a position is - refused at decode, before any change. - -## Deliverables - -1. D1 and D2 in `crates/protocol` and host-core's provider. -2. D3's re-pins and retired rows, D4's docs, the rewritten tests. - -## Authorized paths - -- `crates/protocol/**`; in `crates/protocol/src/queue.rs` only the `TransportState` reliable - payload and its `transport_state` constructor (`:323-335`, `:395-417`) -- `crates/host-core/src/control_provider.rs` -- `crates/capi/src/runtime/tests.rs` (#1309 D8 keeps the C ABI tests in capi) -- `crates/conformance/src/protocol_corpus.rs`, `crates/conformance/tests/conformance_corpus.rs` -- `tools/audit/src/protocol.rs` (the `TransportSetRequest` literal only) -- `scripts/check-protocol-wasm-parity.sh` (the self-test hash strings), - `fuzz/corpus/complete-schema-manifest.md` -- `docs/CONTROL_PROTOCOL_REGISTRY.md`, `docs/CONTROL_PROVIDER_BOUNDARY.md`, - `docs/CONTROL_PROTOCOL_CONFORMANCE.md` - -## Non-goals - -- The transport state (stopped or playing), which stays endpoint-local and readable. -- The timeline and the session seek (#1640-#1644). A protocol command that seeks. - -## Hazards - -- **Corpus re-pin order** as in #1627. - -## Objective gates - -1. **Refused** (`crates/protocol/src/controller/tests.rs`, through `retired_code_rows`). A - `TRANSPORT_SET` with field 2, in either flag form, is refused as an invalid field, and the - transport state, the revision and the reliable lane are unchanged. A snapshot or event frame - that carries the retired field fails typed decode. -2. **State still works** (same file). `TRANSPORT_SET` with state playing returns a snapshot with - that state and the effective sample, and emits one `TRANSPORT_STATE` event with the same two - values. -3. **Corpus and audits.** `cargo test --locked -p conformance` with the new - `COMPLETE_SCHEMA_HASH`; `bash scripts/check-protocol-wasm-parity.sh`; - `cargo build --locked --release -p audit -p bench -p capi -p session-validator`, then - `bash scripts/run-protocol-allocation-audit.sh target/release/audit` and - `./target/release/audit capi` with zero allocations, locks and syscalls, and the same - `pcm_digest` as base (PR evidence). -4. **Commands:** as #1627's gate 6, which includes `bash scripts/check-cross-targets.sh` - (README F19, the iOS memset rule: no new `memset_pattern16` call; fix one in code, never by a - ceiling). - -## Test value - -- Gate 1 turns red if a peer can still store a position, or if the refusal changes state first. -- Gate 2 turns red if removing the position breaks the state round trip. - -## Dependencies - -Batch P1. Direct dependency: - -- #1628 *Retire the AUTOMATION_CANCELED event and the cancellation path*: it removes the locate - cancellation, the last reader of a set position. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Anchors moved.** The stored position is `transport_position` (`crates/host-core/src/control_provider.rs:339`, - seed `:405`, set `:1186-1187`, snapshot `:1179`). #1315 has landed and kept `TRANSPORT_SET`. - Decision 17 does not touch the transport. Batch P1, ready. - -## Decision record: #1635 (batch 13b) - -**Shipped.** `TRANSPORT_SET` field 2, `TransportSnapshot` field 2 and `TransportStateEvent` field 3 -(the stored position) are retired. Each message has a `RETIRED` spec in `crates/protocol/src/schema.rs` -(`transport_set`, `transport_snapshot`, `transport_state_event`) and its decoder reads through -`schema_spec_retiring`, so a retired field is refused in either flag form: optional as -`InvalidTlv` (`MALFORMED_FRAME`), mandatory as `UnknownRequiredField` -(`UNKNOWN_REQUIRED_FIELD`). The remaining field IDs keep their numbers (snapshot 1 and 3; event 1, 2, -4 and 5). `TransportSetRequest`, `TransportSnapshot`, `TransportStateEvent`, -`ReliablePayload::TransportState` and `ReliableSlot::transport_state` lose `position`; the encoded -lengths are 16 (set), 48 and 64 (event without and with origin; were 64 and 80). The provider trait -keeps its shape; host-core's provider and `MockProvider` lose `transport_position`. D5: a set with a -position is refused at decode, before any change and before any event reservation, so no ack -precedes a drop. - -**Re-pins** (reason "#1635 retires the stored transport position"): `COMPLETE_SCHEMA_HASH` -`0xcc9a_7bcc_502c_4936` to `0x3316_54b3_24d0_76e8` with its history line in -`crates/conformance/src/protocol_corpus.rs`; the two parity strings in -`scripts/check-protocol-wasm-parity.sh`; `fuzz/corpus/complete-schema-manifest.md` (hash, history -line, and the prose that named an optional transport position); `docs/CONTROL_PROTOCOL_CONFORMANCE.md`; -the C ABI vectors with a transport frame in `crates/capi/src/runtime/tests.rs` (the two snapshot -vectors, `TRANSPORT_20`/`TRANSPORT_21`, the `RESPONSES` snapshot, and the three event vectors), -re-derived mechanically by deleting the position field and fixing the payload length and field count. -The corpus frame count stays 46. - -Integrated in batch 13b: one re-pin from `0xcc9a_7bcc_502c_4936` to `0x0c71_96f8_6abe_6d57` for #1635 and #1678 together. - -**Departures.** - -- D3 says `retired_code_rows` gains one row per retired field. A `RetiredCodeRow` is a command frame - an endpoint answers, so only `TRANSPORT_SET` field 2 can be a row. It has two rows, one per flag - form (`retired.transport_set_field_2.mandatory`, `.optional`). The snapshot and event fields are - refused by `b3a_transport_goldens_truncation_and_direct_codec_are_strict` in both flag forms. -- Gate 1's status: the refusals answer `UNKNOWN_REQUIRED_FIELD` (mandatory) and `MALFORMED_FRAME` - (optional), the statuses the retiring machinery maps to for the track fields, not `INVALID_FIELD`. -- Corpus `counter_command_frame` now gives `TRANSPORT_SET` an exact revision 7 (a set requires one). -- Anchors had moved (queue.rs `:61`/`:138`, host-core `:337`); followed the code. Mechanical edits - outside the listed paths: `hosts/host-web/tests/protocol_cross_host.rs` (the `TransportSetRequest` - literal and the unused `SampleTime` import). The SDK mirrors no transport frame; no SDK input changed. - -**Gates.** - -1. Refused: `a_transport_set_with_the_retired_position_changes_nothing` (both flag forms: status, revision, - session, reliable lane and `TRANSPORT_GET` state unchanged) and - `retired_and_console_refused_codes_meet_their_conformance_rows` (rows 16+4+7+13+1+4+10+2+1). - Decoder: `b3a_transport_goldens_truncation_and_direct_codec_are_strict`. -2. State still works: `a_transport_set_answers_the_state_and_queues_one_event_with_it`, - `b3a_transport_btlv_is_typed_idempotent_and_emits_reliable_state` (event golden re-pinned, 64 bytes). -3. `cargo test --locked -p protocol` (163), `-p conformance` (`frozen_corpus_bytes_and_typed_decoders_are_unchanged`), - `-p host-core --lib control_provider`, `-p capi --lib runtime` (114), all green; - `bash scripts/check-protocol-wasm-parity.sh` ok (simd128); `scripts/run-protocol-allocation-audit.sh` - ok; `audit capi`: zero allocations, locks and syscalls, `pcm_digest` `bf7dcedd1d92621f`, equal to base. -4. `scripts/check-cross-targets.sh` was not run here (coordinator's batch gate); the change adds no - memset-style code. - -**Successor.** `docs/CONTROL_PROTOCOL_SEMANTICS.md:23` still says "locating transport cancels -nothing"; reword when that file is next in scope (outside this slice's authorized paths). diff --git a/.github/ISSUE_SPECS/1636-retire-the-span-kinds-and-sample-rate-no-producer-uses.md b/.github/ISSUE_SPECS/1636-retire-the-span-kinds-and-sample-rate-no-producer-uses.md deleted file mode 100644 index 80b0b8724..000000000 --- a/.github/ISSUE_SPECS/1636-retire-the-span-kinds-and-sample-rate-no-producer-uses.md +++ /dev/null @@ -1,203 +0,0 @@ -# Retire the automation span kinds and the sample rate that no producer uses - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answers A1.6 and A9 and README finding F6 as decided, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. Batch P1. - -## Product outcome - -The effect contract states only what the engine delivers. An effect receives `Point` spans and -nothing else, and a parameter is automated at block rate or not at all. The span kinds `Step`, -`Linear` and `Exponential`, the function `automation_segment_value`, the descriptor value -`AutomationRate::Sample` and the protocol's `ParameterAutomationRate::Sample` (wire value 1) are -retired: each raw value is refused, never reallocated and never renumbered. No rendered bit moves. - -## Context - -- **The vocabulary.** `AutomationRate {Sample=1, Block=2, None=3}` and - `AutomationSpanKind {Point=1, Step=2, Linear=3, Exponential=4}` - (`crates/effect-contract/src/lib.rs:139`, `:145`). `valid_runtime_span` checks each kind - (`:1420-1455`); `automation_segment_value` evaluates the three moving kinds (`:1632-1656`); the - descriptor check accepts `Sample` like `Block` (`:198-206`). The contract doc calls the three - kinds and `Sample` "descriptor and protocol vocabulary" whose render-path delivery "is a later - protocol capability" (`docs/EFFECT_CONTRACT_V1.md:149-155`). -- **No producer.** Live records stage only `Point` spans (`crates/effect-contract/src/live.rs:21`, - `:456`), and every launch effect accepts only a `Point` (`crates/compressor/src/lib.rs:434`, - `crates/gate-expander/src/lib.rs:563`, `crates/soft-clip/src/lib.rs:622`, - `crates/true-peak-limiter/src/lib.rs:3847`; the graph's check `crates/graph/src/lib.rs:6641`). - Stored automation stages only `Point` spans (#1058 A1.6). The "later protocol capability" was - `AUTOMATION_ENQUEUE`, which #1627-#1629 retire (#1058 A9). No launch descriptor declares - `Sample`: its uses are test fixtures (`crates/effect-contract/src/lib.rs:2608-2624`, `:2711`; - `crates/conformance/src/effect.rs:51`) and the mappings in - `crates/host-core/src/control_provider.rs:656` and `tools/parameter-metadata/src/lib.rs:830`. - `sdk/assets/miso-engine-v1-parameter-metadata.json` has no `"sample"` value. -- **The reference mock and the generator.** The conformance reference delay evaluates the moving - kinds (`crates/conformance/src/effect.rs:345-362`), the randomized generator draws them - (`crates/conformance/src/randomized.rs:1770-1780`), and - `crates/conformance/tests/effect_contract.rs:25`, `:161-163` tests `automation_segment_value`. - `crates/delay/src/lib.rs:2605-2625` builds a `Linear` span in a malformed-span test. -- **Other test fixtures that declare `Sample`:** `crates/effect-contract/src/step.rs:927` (in its - `#[cfg(test)]` module, `:908`), `crates/effect-compiler/tests/native_session.rs:34`, - `crates/protocol/src/controller/tests.rs:490` and - `crates/conformance/tests/conformance_corpus.rs:81`. -- **A policy pin.** `scripts/check-effect-runtime-policy.sh:61` pins `fn automation_segment_value(` - at count 1 in its duplicated-helper manifest. -- **Every user.** `git grep` on `c63f5f37d` for `AutomationRate::Sample`, - `ParameterAutomationRate::Sample`, `AutomationSpanKind::{Step, Linear, Exponential}` and - `automation_segment_value` (outside `docs/handoffs/` and `.github/`) finds them only in the files - this Context names, in `crates/effect-contract/tests/response_analysis.rs`, - `crates/protocol/src/message_wire/tests.rs` and in `docs/EFFECT_CONTRACT_V1.md:151-152`. -- **The protocol value.** `ParameterAutomationRate::Sample = 1` - (`crates/protocol/src/message_wire.rs:244`, parsed at `:3039`); the registry lists "automation - sample/block/none `1..3`" (`docs/CONTROL_PROTOCOL_REGISTRY.md:13`). -- **The removal ruling.** Code nothing uses is removed - (`docs/rulings/engine-footprint-2026-09-28.md:20-21`). - -## Decisions frozen for this slice - -- **D1. Retired values.** `AutomationSpanKind` keeps `Point = 1`; `from_raw` returns `None` for 2, - 3 and 4, with a comment that names them retired and never reallocated. `AutomationRate` keeps - `Block = 2` and `None = 3`; `from_raw` returns `None` for 1. The protocol's - `ParameterAutomationRate` keeps 2 and 3, and its parse refuses 1 as an invalid field value. -- **D2. Code leaves.** `automation_segment_value` leaves; `valid_runtime_span` checks only a - `Point`; the descriptor check accepts only `Block` for an automatable parameter; host-core's and - the metadata tool's mappings lose `Sample`. The conformance reference mock and the generator - keep only `Point` spans; a test that exists only for a retired kind is deleted, and the delay's - malformed-span test uses a `Point` with a malformed field instead. Each test fixture that - declares `Sample` (Context) declares `Block`. The policy manifest's `fn automation_segment_value(` - row leaves with the function. The independent metadata gate's rate table - (`scripts/check-parameter-metadata-v1.py:30`, `RATES = {1: "sample", 2: "block", 3: "none"}`, - read at `:477`) loses its `1: "sample"` entry, so that gate also refuses the retired rate. -- **D3. Re-pins**, with the reason "#1636 retires the span kinds and the sample rate": any - conformance corpus row or digest that carries a retired value, and `COMPLETE_SCHEMA_HASH` only if - a corpus frame carries `ParameterAutomationRate` value 1. `retired_code_rows` gains a row for - value 1 in a parameter descriptor. -- **D4. Docs.** `docs/EFFECT_CONTRACT_V1.md:149-155` says: runtime automation is `Point` spans; the - other kinds and the sample rate are retired by this slice. The registry line lists rate 1 as - retired. -- **D5. The acked-batch question** has no subject here: nothing is queued or acknowledged. - -## Deliverables - -1. D1 and D2 in `crates/effect-contract`, `crates/protocol`, host-core's provider mapping, the - metadata tool and `crates/conformance`. -2. D3 and D4. - -## Authorized paths - -- `crates/effect-contract/src/lib.rs`, `crates/effect-contract/src/step.rs` (its test module's - fixture only), `crates/effect-contract/tests/` -- `crates/effect-compiler/tests/native_session.rs` (the fixture's rate only) -- `crates/protocol/src/controller/tests.rs` (the fixture's rate only) -- `crates/conformance/tests/conformance_corpus.rs` -- `scripts/check-effect-runtime-policy.sh` (the `automation_segment_value` manifest row only) -- `crates/protocol/src/message_wire.rs`, `crates/protocol/src/message_wire/tests.rs` -- `crates/host-core/src/control_provider.rs` (the rate mapping only) -- `tools/parameter-metadata/src/lib.rs` (the rate name only), - `scripts/check-parameter-metadata-v1.py` (the `RATES` entry at `:30` only) -- `crates/conformance/src/{effect.rs,randomized.rs,protocol_corpus.rs}`, - `crates/conformance/tests/effect_contract.rs` -- `crates/delay/src/lib.rs` (the malformed-span test only), and each effect test support module - that builds a retired kind (`crates/*/tests/support/mod.rs`, `crates/*/tests/common/mod.rs`) -- `docs/EFFECT_CONTRACT_V1.md`, `docs/CONTROL_PROTOCOL_REGISTRY.md` - -## Non-goals - -- `Point` spans and every effect's span handling, which stay. Stored automation (A1). -- The protocol command (#1627-#1629). - -## Hazards - -- **Generated SDK files.** If the build regenerates a file that spells the rate names, regenerate - it with the build, never by hand (`bash scripts/check-sdk-generated.sh`). - -## Objective gates - -1. **Retired values refused** (`crates/effect-contract` unit tests and `retired_code_rows`). - `AutomationSpanKind::from_raw(2..=4)` and `AutomationRate::from_raw(1)` are `None`; a parameter - descriptor frame with rate 1 fails decode; a descriptor that declares an automatable parameter - with a non-`Block` rate fails the descriptor check. -2. **Effects unchanged.** Every effect crate's tests and the conformance suite pass, the - randomized conformance run included; its generator now reaches only `Point` spans. -3. **No rendered bit moves.** `cargo build --locked --release -p audit -p bench -p capi -p - session-validator`, then `./target/release/audit capi` shows the same `pcm_digest` as base (PR - evidence) and zero allocations, locks and syscalls. -4. **Commands:** - - `cargo test --locked -p effect-contract`, `cargo test --locked -p effect-compiler`, - `cargo test --locked -p conformance`, - `cargo test --locked -p protocol --features test-support`, - `cargo test --locked -p host-core --features test-support`, `cargo test --locked -p delay` - - the workspace debug leg (`test-debug-a`) and the DSP leg (`test-debug-b`) in - `.github/workflows/qualification.yml` - - `bash scripts/check-sdk-generated.sh target/ci/qualification-artifacts` after - `bash scripts/build-web-audioworklet.sh`, `bash scripts/check-workspace-policy.sh` - - `bash scripts/check-effect-runtime-policy.sh`, `bash scripts/test-effect-runtime-policy.sh` - - `python3 -B scripts/check-parameter-metadata-v1.py --self-test` (also run by - `scripts/test-web-audioworklet.sh:205`) - - `bash scripts/check-cross-targets.sh` (README F19, the iOS memset rule: no new - `memset_pattern16` call; fix one in code, never by a ceiling) - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` - -## Test value - -- Gate 1 turns red if a retired kind or rate still parses, or if a descriptor can still declare a - rate the engine never delivers. -- Gate 2 turns red if a launch effect depended on a retired kind. - -## Dependencies - -Batch P1. Direct dependency: - -- #1629 *Delete the protocol automation queue, its records and counters*: its record codec - carries the span kind (`AutomationKind`, `crates/protocol/src/queue.rs:30-44`), so the protocol's last producer of the moving kinds leaves first. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **The protocol half is done.** The protocol's `ParameterAutomationRate` left with - `PARAMETER_METADATA_GET` (decision 17 C4, #1551); drop its wire value from D-items, gates and paths. -- **One more user:** `controls`' `parameter_class` maps `AutomationRate::Sample` to rebuild-only - with the reopen text "sample-accurate render-path delivery, a later protocol capability" - (`crates/controls/src/effect_rows.rs:146-160`). Retiring `Sample` deletes that arm and its text; - add `crates/controls/src/effect_rows.rs` to the authorized paths. -- **Anchors.** `AutomationRate` is now `crates/effect-contract/src/lib.rs:415`; re-read the rest at - filing. Batch P1, ready. - -## Decision record: #1636 (batch 13b) - -**Shipped.** `AutomationSpanKind` is `{Point=1}` and `AutomationRate` is `{Block=2, None=3}`; -`from_raw` refuses 2 to 4 and 1 (comments name the values retired, never reallocated). -`automation_segment_value` is deleted; `valid_runtime_span` checks only a `Point`; the descriptor -check (`parameter_automation_smoothing_valid`) accepts only `Block` for an automatable parameter; -`validate_automation_block` no longer re-checks the kind. The metadata tool's rate name and the -independent gate's `RATES` lose `sample`; the gate's self-test gains the mutation "the retired -sample automation rate returns". `controls`' `parameter_class` loses its `Sample` arm and the test -that existed only for it. The conformance reference mock keeps only `Point` spans: its per-lane -`active` span state is gone, so its state payload is 20 bytes per lane (was 56). The randomized -generator draws only `Point` spans at the block's first sample. The delay's malformed-span test -uses a `Point` whose end sample differs from its start. Fixtures that declared `Sample` declare -`Block` (`live_window.rs` loses its `Sample` row: `NOT_LIVE` is 2). The policy manifest row, the -docs (`EFFECT_CONTRACT_V1.md`, `CONTROL_PROTOCOL_SEMANTICS.md`) and two comments are updated. - -**Departures.** -- The protocol half was already gone (amendment): no `ParameterAutomationRate`, no parameter - descriptor frame, so D3 has nothing to re-pin. `COMPLETE_SCHEMA_HASH` and the wasm-parity strings - do not move (`check-protocol-wasm-parity.sh` passes unchanged), and `retired_code_rows` gains no - row: there is no frame that can carry rate 1. Gate 1 is held by `retired_automation_values_are_refused` - and the truth table in `crates/effect-contract/src/lib.rs`. -- Mechanical edits outside the listed paths: `crates/effect-contract/tests/live_window.rs`, - `crates/host-core/src/prepare.rs` (a comment), `docs/CONTROL_PROTOCOL_SEMANTICS.md`. -- The registry has no rate line any more, so `CONTROL_PROTOCOL_REGISTRY.md` is unchanged. - -**Gates.** `cargo test --locked` for effect-contract, effect-compiler, conformance (randomized -included), controls, delay, parameter-metadata, compressor, gate-expander, soft-clip, -true-peak-limiter, parametric-eq, multiband-compressor, transient-shaper, protocol and host-core -(`--features test-support`) and `capi --lib runtime`: all pass. `check-effect-runtime-policy.sh`, -`test-effect-runtime-policy.sh`, `check-protocol-wasm-parity.sh`, `check-sdk-generated.sh`, -`check-workspace-policy.sh`, `check-parameter-metadata-v1.py --self-test`: pass. Clippy -`-D warnings` (all-targets, all-features) on the touched crates and `cargo fmt --all`: clean. -`audit capi`: `pcm_digest` `bf7dcedd1d92621f` (same as base), zero allocations, locks and -syscalls; protocol allocation audit ok. diff --git a/.github/ISSUE_SPECS/1637-compile-stored-automation-into-per-cell-events.md b/.github/ISSUE_SPECS/1637-compile-stored-automation-into-per-cell-events.md deleted file mode 100644 index 2fe1933ab..000000000 --- a/.github/ISSUE_SPECS/1637-compile-stored-automation-into-per-cell-events.md +++ /dev/null @@ -1,421 +0,0 @@ -# Compile stored automation into per-cell events in node time - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answers A1 (A1.1, A1.4, A1.7), A4 and A5, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. - -## Product outcome - -A new leaf crate, `automation`, turns an automated cell's segment table into the events render -applies, and nothing else. Given one block's node-time mapping, it yields the cell's events in -offset order: grid retargets on a fixed node-time grid, jumps at discontinuities, and an exact set -at a seek. Every value is computed in scalar `f64` through `crates/math` and rounded once to `f32`, -so the same table and the same mapping give the same events, bit for bit, on x86-64, AArch64 and -wasm, at any quantum, wherever a seek lands. It allocates nothing after its builder. No stage uses -it yet; #1645's order check (`value_at`) and #1646 and #1647 are its first users. - -## Context - -- **The table.** `Automation { id, target, segments }` with `AutomationSegment { shape, - start_sample: u64, end_sample: u64, start_value: f32, end_value: f32, unit }` and the shapes - `step`, `linear`, `exponential` (`crates/session/src/model.rs:874-955`). Session validation - orders segments and refuses overlap (`crates/session/src/validate.rs:972-1000`); #1626 adds - one entry per lane, the hold rule and discontinuities at least 64 samples apart. -- **The evaluation law** (README A1.4): `linear` `v0 + (v1 - v0)·x`, `exponential` - `v0·(v1/v0)^x`, `step` `v0`, with `x = (t - t0)/(t1 - t0)`; these are the Web Audio ramp formulas. - The hold rule holds the first `start_value` before the first segment and a segment's `end_value` - after it and in a gap; a `step` holds `start_value` on `[start, end)`. -- **Deterministic math.** `crates/math` is vendored libm with no target-conditional path - (`crates/math/src/lib.rs:1-30`; `pow` at `:105-109`), `no_std`, and `clippy.toml` bans platform - transcendentals (`clippy.toml:1-40`). Rust does not contract scalar `f64` arithmetic. -- **Where ramps complete.** The fader kernel assigns the target exactly on the ramp's last frame - (`crates/lane/src/kernels/builtins.rs:165-170`, `:184-202`), so a ramp of `L` samples started at - sample `τ` equals the target at `τ + L - 1`. The EQ and input-filter coefficient ramps reach the - target at `A + 64` (`docs/EFFECT_CONTRACT_V1.md:165-166`), one sample later. -- **Node time** (README A1.3): a node at render sample `r` reads source-read sample - `s = r + ΣP - a(n)`, and its node time is `τ(r) = timeline(s)`. #1640's - `timeline_block` gives one node block's mapping: a start value and at most one step (a seek). - This crate cannot depend on `source` (D1), so it takes the mapping as its own plain type, and - #1647 fills it from #1640's reader. -- **The cross-target corpus.** Gate G5 digests every case on x86-64, AArch64 and wasm and compares - with one set of pins (`tools/wasm-gate-corpus/src/lib.rs:1-45`). Crates delegate scalar cases - through their own `corpus` module and pins (`math::corpus`, `:55`, `:118`); new blocks are - appended at the end (`:34-45`) and the case decoder fixes the order (`case_of`, `:444-511`); a - scalar case is not width dependent (`is_width_dependent`, `:513-529`). The native owner is - `tools/wasm-gates/tests/g5_native_corpus.rs`; the wasm leg is `scripts/run-wasm-gates.sh`; the - AArch64 release job runs G5 unfiltered (`scripts/run-aarch64-tests.sh:12-17`). -- **Realtime marking.** Render-reachable code sits between `REALTIME_POLICY_BEGIN` and - `REALTIME_POLICY_END`; `scripts/check-realtime-policy.sh` scans every marked file and holds floors - of 25 files and 89 regions, which the change that adds a marker raises (`:66-74`); the forbidden - forms inside a region include `.expect(` and `.unwrap(` (`:76-77`). -- **Crate naming.** The package, the lib and the directory are `automation` - (AGENTS.md; `scripts/check-workspace-policy.sh:158-161`). - -## Decisions frozen for this slice - -- **D1. The crate.** `crates/automation`, package and lib `automation`, `#![no_std]` with `alloc` - for the builder only, dependent on `math` alone. A workspace member and a workspace dependency. -- **D2. Types.** - - `Shape { Step, Linear, Exponential }`. - - `Segment { start: u64, end: u64, start_value: f32, end_value: f32, shape: Shape }`, 32 bytes - (a `const` assertion). - - `Completion { End, After }`: a ramp of `n` samples started at node time `τ` completes at - `τ + n - 1` (`End`, the fader, matrix, trim and effect ramps) or `τ + n` (`After`, the EQ and - input-filter coefficient ramps). For `n = 0` both complete at `τ`. - - `CellProgram`: its segment slice, grid period `G`, grid ramp `L`, a jump-length key, - `Completion`, and `no_restart` (a target whose ramp cannot restart while one runs: the delay - time). `G` is 64, or the cell's ramp length `L` when that is larger (128 for the delay time); it - is always a multiple of 64. - - `JumpLength { Word(JumpWord), Fixed(u32) }`, the jump-length key, with - `JumpWord { Fader, Mute, Pan }`. A builtin strip row stores which word of #1639's plan cell - it reads (fader and trim: `Fader`; mute and polarity: `Mute`; pan and matrix: `Pan`). A filter - target stores `Fixed(64)` and an effect parameter `Fixed(smoothing_samples)`. The program never - stores a copied jump length `J` for a word key: `J` is read from the plan cell, so a - `control_smoothing` edit reaches the next jump with no rebuild (#1639). - - `CellState`: cursor, current target bits, the node time at which the ramp in flight on the - lane completes, a held flag, and the last seek boundary. At most 48 bytes (a `const` assertion). -- **D3. `value_at(segments: &[Segment], t: i64) -> f32`.** A free function of the crate that takes - only a segment table, never a `CellProgram`: render calls it with a program's segment slice, and - #1645 calls it with a table it builds from one session entry (`Segment` per - `AutomationSegment`, the same fields), so preparation's check and render read one law. The hold - rule over the segment containing `t`. Inside a - `linear` or `exponential` segment, `x = (t - t0) as f64 / (t1 - t0) as f64` (both differences are - exact below `2^53`); `linear` is `v0 + (v1 - v0)·x` in `f64`; `exponential` is - `v0·math::pow(v1/v0, x)` in `f64`. One rounding to `f32` at the end. No lane math, no platform - math. -- **D4. Events of one block, pulled.** `CellEvents::new(program, &mut state, mapping, mode, - jump_lengths)` returns an iterator of `Event { offset: u32, value: f32, ramp: u32, - kind: Grid | Jump | Set, held: bool }` in increasing offset, with no buffer. `mapping` is a - `NodeSpan { start: i64, step: Option<(u32, i64)>, frames: u32 }`: the node time at offset 0, the offset and node time of - a seek step if the block has one, and the block length. `mode` is `SeekMode::Exact` (builtin - cells) or `SeekMode::Ramp` (effect cells). Rules, at most one event per offset, priority - `Set > Jump > Grid`. `jump_lengths: [u32; 3]` is the block's `fader`, `mute` and `pan` words of - #1639's plan cell; `J` below is the key resolved against it (a `Word` key) or the `Fixed` - length: - - **Grid.** At each offset whose node time `τ` is a multiple of `G`, a retarget over `L` to - `value_at(completion(τ, L))`. - - **Jump.** At each offset whose node time is a discontinuity of the table, a retarget over `J` - to `value_at(completion(τ, J))`. A `no_restart` cell whose ramp is in flight takes the jump at - the ramp's last sample plus one instead. - - **Held.** A grid event whose completion is before the end of the ramp in flight on the lane is - emitted with `held = true`: the stage remembers the value and does not retarget, and the cell's - current target does not change. The caller reports ramps it starts itself (a mute ramp, a live - record's ramp) through `CellState::hold_until(node_time)`. - - **Set.** At a seek step, `Exact` emits `Set` (no ramp) to `value_at(τ)` and `Ramp` emits a - `Jump` over `J` to `value_at(completion(τ, J))`. The step may be at offset 0: #1640's reader - reports `step = (0, t)` when the seek's source-read block starts the node block, and the cell - then emits its `Set` (or ramped `Jump`) at offset 0. So every seek that reaches a block yields - its event, whatever its offset. The cursor is repositioned first by binary - search over the segment table (`⌈log2(n + 1)⌉` compares). The same repositioning serves an - adoption (`CellState::reposition(t)`). - - **Change only.** A `Grid` or `Jump` whose value has the current target's bits emits nothing. - `Set` always emits. - - The cursor advances one segment per boundary crossed and never moves backwards except at a seek - or a reposition. - - Bound: at most `⌈q/G⌉ + 1` grid events, `⌈q/64⌉ + 1` jumps (#1626's spacing rule) and one - seek: `2⌈q/64⌉ + 3` per cell and block, plus one for a `no_restart` cell (a jump deferred from - the previous block); `CellProgram::max_events(q)` returns it. Amended at attempt 2, which has - the proof (decision record, "Attempt 2", finding 1). -- **D5. Builder.** `ProgramBuilder` takes neutral cell descriptions (segments and D2's parameters; - no session type) and reports the exact bytes it will allocate (`bytes(&cells) -> u64`) before it - allocates them with fallible reservations. It refuses, typed, a description that breaks D2's - invariants (segments out of order or overlapping; a `G` that is not a multiple of 64 or that - differs from the larger of 64 and `L`; a non-positive exponential value). Preparation ( - #1646) charges the report. -- **D6. G5 corpus.** `automation::corpus` holds a fixed curve set: a linear dB fade over 48,000 - samples, an exponential 20 Hz to 20 kHz sweep, steps, gaps, one entry with a seek, and a cell - with `no_restart`. Each case runs D4 over a fixed sequence of node-time mappings and digests every - event's offset, kind, ramp and value bits. The case block is appended at the end of - `tools/wasm-gate-corpus`'s case list, scalar (not width dependent), with its pins in the - `automation` crate, and runs on the x86-64, wasm and AArch64 legs. -- **D7. Realtime marking.** Event generation and `value_at` sit inside realtime policy regions; - the floors in `scripts/check-realtime-policy.sh` rise by the file and regions this adds. The - builder is outside them. -- **D8. The acked-batch question.** The crate has no queue. It reads a table and writes a fixed - state. Nothing can be acknowledged and dropped. - -## Deliverables - -1. The crate (`Cargo.toml`, `src/lib.rs`, `src/corpus.rs`) and the workspace entries. -2. Unit tests and the allocation test in `crates/automation/tests/`. -3. D6's delegation in `tools/wasm-gate-corpus/src/lib.rs` and its manifest, and the G5 test's - layout check. -4. D7's floor change. - -## Authorized paths - -- `crates/automation/` (new) -- `Cargo.toml` (workspace member and dependency), `Cargo.lock` -- `tools/wasm-gate-corpus/src/lib.rs`, `tools/wasm-gate-corpus/Cargo.toml` (one appended case - block), `tools/wasm-gates/tests/g5_native_corpus.rs` (the layout check for the appended block), - `tools/wasm-gates/tests/g6_full_corpus_ftz.rs` (its `expected_rows`, `:188-192`, counts only math - and delay cases as width-independent, `tools/wasm-gate-corpus/src/lib.rs:527-529`; the new - scalar block joins that count) -- `scripts/check-realtime-policy.sh` (the two floors only) - -## Non-goals - -- Reading a session, a descriptor or a plan; mapping targets to cells (#1646, #1647 and later). -- Any stage operation or render wiring (#1638, #1646 and #1647). -- Vector evaluation. Events are sparse control arithmetic; vector lane math diverges on AArch64 - release builds (LANE-3, `docs/TARGET_MATRIX.md:163-167`), so evaluation stays scalar `f64` - (README A1.7). This is the reason the no-scalar rule asks for. - -## Hazards - -- **Corpus order is part of the pin.** The block goes on the end; no existing index moves. The G5 - layout test checks it. -- **Exponential near the edges.** `v1/v0` is positive by validation (#1645 and #1566's row rule, - `controls::automation_diagnostics`); the builder still refuses a non-positive value so the evaluator has no NaN path. -- **The grid in node time, not render time.** A grid on render samples would make the event values - depend on where a seek landed when `q` is not a multiple of 64 (the README's blocker B1). The - quantum-100 gate defends it. - -## Objective gates - -1. **Evaluation** (`crates/automation/tests/evaluate.rs`, new). The hold rule before, inside, after - and between segments; a step's end; `linear` at `x = 0`, `0.5`, `1`; `exponential` equal, bit for - bit, to `(v0 as f64 * math::pow(v1 as f64 / v0 as f64, x)) as f32` with `x` computed as D3 says, - and different from the same formula in `f32` on at least one fixed input. -2. **Event rules** (`crates/automation/tests/events.rs`, new). A grid event at every node-time - multiple of 64 and none elsewhere; a jump at an off-grid discontinuity at its exact offset; `End` - and `After` completions; change only (a flat curve emits nothing after its first event); a held - event inside a reported ramp, and the grid resuming after it; a `no_restart` jump deferred to the - end of its ramp; `Set` at a seek step for `Exact` and a ramped `Jump` for `Ramp`, at a mid-block - offset and at offset 0; a `Word` key's jump ramps over the word passed for the block, so two - blocks with different words give two jump lengths. -3. **Bound.** An adversarial table (a discontinuity every 64 samples, every segment moving) and a - seek in the block give at most `2⌈q/64⌉ + 3` events per block at `q` in {64, 100, 128, 1024}, - plus one for a `no_restart` cell (D4, amended at attempt 2). -4. **Quantum and seek invariance.** The same table rendered through mappings of quantum 128 and - quantum 100, and with one seek to the same timeline sample landing at two different render - samples, gives the same sequence of `(node time, kind, ramp, value bits)` events from the first - grid sample after the seek on. -5. **No allocation.** Generating events for 10,000 blocks allocates and frees nothing - (`bench_support::alloc` thread counters after warming). The builder's `bytes` report equals the - bytes its build leaves live (`requested_bytes - released_bytes` over the build). -6. **G5.** The appended cases' digests match their pins natively and on the wasm and AArch64 legs. -7. **Policy.** `bash scripts/check-realtime-policy.sh` and `bash scripts/check-workspace-policy.sh` - accept the crate. -8. **Commands:** - - `cargo test --locked -p automation` - - `cargo test --locked --release -p lane -p math -p wasm-gates --features math/lane` - - `cargo build --locked --release -p wasm-gates && bash scripts/run-wasm-gates.sh --without-v8-spill --without-native` - - `bash scripts/run-aarch64-tests.sh release` (the `aarch64-release` job) - - `bash scripts/check-realtime-policy.sh`, `bash scripts/check-workspace-policy.sh`, - `bash scripts/check-cross-targets.sh` (README F19, the iOS memset rule) - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` - -## Test value - -- Gate 1 turns red if a segment is evaluated in `f32`, with a platform `powf`, or with the hold rule - on the wrong side of a boundary. -- Gate 2 turns red if the grid follows render samples, a jump snaps to the grid, a completion is - off by one, a redundant retarget is emitted, a held event retargets the stage, a seek at offset 0 - emits nothing, or a jump uses a stored length instead of the block's word. -- Gate 3 turns red if a pathological table makes the per-block work grow past the bound A1.7 - states. -- Gate 4 turns red if event values depend on the quantum or on the render sample a seek landed at - (blocker B1). -- Gate 5 turns red if event generation allocates, or if the builder under-reports what preparation - charges. -- Gate 6 turns red if any target computes a different event bit, the only cross-target owner of - this claim. - -## Dependencies - -- #1626 *Validate stored automation lanes in the session crate and state the hold rule*. -- Batch: R1, with #1645 *Validate builtin automation targets against their rows at - preparation* and #1646 *Prepare stored fader automation and render it flat* and #1647 *Render - moving stored fader automation, seeks and latency*, its first users. The key names the words of - #1639's plan cell, but this crate reads no cell: the caller passes the words, so #1639 is - not a dependency. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Jump lengths follow the ramp rows.** Decision 17 gives every row a `LiveRampRow` - (`crates/effect-contract/src/control.rs:70-102`); polarity ramps over twice `mute_ms` - (`PolarityInvert`), and sends and VCAs have rows of their own. Replace D2's `JumpWord { Fader, - Mute, Pan }` with the row's `LiveRampRow`, resolved per block against #1639's three key - lengths by `controls::LiveRamps::for_row` (`crates/controls/src/live_ramps.rs`), so no row's rule - is written twice. `Fixed(n)` stays for filter, EQ and effect cells. -- **Realtime policy.** D7 and gate 7: `scripts/check-realtime-policy.sh` is deleted; raise - `marked_files_floor` and `marked_regions_floor` in `tools/realtime-policy/src/policy.rs` and run - `./target/release/realtime_policy`. Authorized paths change to that file. -- **The table.** `Automation.target` is a `ControlAddress` (#1566); the crate reads no session type, - so nothing else changes. -- **OQ1 = C** adds no evaluator work: the caller adds the offset after `value_at` (#1650 D6, - #1658). - -## Decision record: #1637 (batch 13b) - -**Shipped.** `crates/automation` (package, lib and directory `automation`; `#![no_std]` + `alloc` -for the builder; depends on `math` only): `value_at`, `ProgramBuilder` (neutral `CellDescription`, -typed `BuildError`, `bytes()` report), `CellProgram`, `CellState`, `NodeSpan`, `CellEvents`, and -`automation::corpus` (six scalar G5 cases and their pins). The G5 block is appended after the -builtins tail in `tools/wasm-gate-corpus`, scalar (not width dependent); `g5_native_corpus.rs` -checks its layout and `g6_full_corpus_ftz.rs` counts it as width independent. Realtime regions -mark `value_at` (`evaluate.rs`) and the event generator (`events.rs`). - -**Departures from the filed text (each a code-truth or correctness call).** - -1. **The jump-length key.** The amendment replaces `JumpWord` with `LiveRampRow` resolved by - `controls::LiveRamps::for_row`. `effect-contract` is `std` and pulls `engine` and `lane` - (the AVX2-pinned crate), and `controls` pulls `session`; either would break D1 ("`math` alone"). - `JumpLength` is `Block | Fixed(u32)`: a `Block` cell takes one resolved `u32` per block - (`CellEvents::new(..., jump)`). The caller resolves the cell's row through `for_row` (which owns - the polarity doubling and every other row rule), so no row's rule is written here and the program - still stores no copy of `J`. `jump_lengths: [u32; 3]` became that single `jump: u32`. -2. **A grid or jump ramp never aims past the next discontinuity.** D4 says a grid event aims at - `value_at(completion(τ, L))`. Taken literally, a grid event up to 63 samples before an off-grid - step aims at the post-step value, the step's own `Jump` then has the same target and change-only - drops it, so the step lands on the grid's ramp and not at its exact offset (gate 2 cannot hold). - The aim time is `min(completion, d - 1)` for the first discontinuity `d` in `(τ, completion]`; - the `Jump` at `d` retargets from there. The same clamp applies to a `Jump` and a ramped seek - landing. Test `jump_at_off_grid_discontinuity_at_its_exact_offset` is red without it. -3. **The discontinuity list is built and stored.** The cursor indexes a per-cell list of - discontinuity times (value just before the time differs from the value at it, bit for bit; built - by the builder, charged in `bytes`), not the raw segment boundaries. Scanning raw boundaries for - "the next discontinuity" is unbounded when continuous segments are dense (one-sample segments); - the list makes every lookup `O(1)` and the reposition one binary search. -4. **`no_restart` also holds grid events.** A jump off the grid starts a ramp that a later grid - event (not completing before it) would restart; for a `no_restart` cell every grid event inside - a ramp in flight is `held`. The jump itself is deferred to `completion + 1` (`End`) or - `completion` (`After`), carried in `CellState` across blocks. -5. **Held bookkeeping.** `CellState::held` means "a held value is outstanding at the stage": the - next retarget or held event emits even when its value equals the target, once, so the stage's - remembered value is refreshed (otherwise a curve that leaves the target and returns inside a - long ramp leaves a stale value). `CellState` is 32 bytes (limit 48). The spec's "last seek - boundary" field has no user and is not stored. -6. **`NodeSpan::new` is fallible** (`StepOutsideBlock` when the step offset is not below `frames`), - so a seek cannot be silently dropped in render. A fresh `CellState` has no target, so its first - retarget always emits; `reposition(&program, t)` places the cursor at the first discontinuity at - or after `t`, keeps the target, and drops the in-flight ramp, any deferred jump and an outstanding - held event (batch 13b review: the earlier "cursor only" wording left out the held event), so a - retarget to the value the cell already targets then emits nothing. -7. **Builder limits** beyond D5: `start < end`, every time below `2^53` (the evaluator's exactness - bound), finite values, at least one segment per cell. `value_at` on an empty table returns `0.0` - and never panics. -8. **Realtime floors.** `scripts/check-realtime-policy.sh` is gone; `marked_files_floor` and - `marked_regions_floor` in `tools/realtime-policy/src/policy.rs` were 28 and 129 but the tree - held 29 and 141 before this change (the floors had fallen behind). They are now 31 and 143, the - tree's counts with this crate's two files and two regions. - -**Observation for successors.** A seek `Set` in `Exact` mode clears the in-flight ramp in the -state, so the stage must treat `Set` as an immediate assignment. `CellState::hold_until` takes the completion node time in the cell's own `Completion` -convention; on a `no_restart` cell it must not be earlier than the ramp in flight (the caller -does not restart a busy lane), or a deferred jump lands late (attempt-3 review, MINOR). -A seek landing in `Ramp` mode restarts a busy `no_restart` lane, as D4's `Set` rule says, with no -`no_restart` exception. #1646 and #1647 use `no_restart = false`. The first issue that automates -a `no_restart` target (delay time) decides whether a seek landing waits for the lane. - -**Gates and tests.** - -| Gate | Test (`cargo test --locked -p automation`) | Result | -|---|---|---| -| 1 Evaluation | `evaluate`: `hold_rule_before_inside_after_and_between`, `step_end_value_applies_only_from_its_end`, `linear_at_zero_half_and_one`, `exponential_is_scalar_f64_through_math_pow` (bit equal to the `f64` formula, differs from the `f32` one), `empty_table_is_zero` | pass | -| 2 Event rules | `events`: `grid_events_sit_on_node_time_multiples_of_64` (quantum 100, start -37), `jump_at_off_grid_discontinuity_at_its_exact_offset`, `end_and_after_completions`, `change_only`, `held_inside_a_reported_ramp_and_the_grid_resumes`, `a_held_value_back_at_the_target_still_emits_once`, `no_restart_jump_waits_for_the_ramp_to_end`, `set_at_a_seek_step_for_exact_and_a_ramped_jump_for_ramp` (offsets 0 and 37), `set_always_emits`, `a_block_keyed_jump_ramps_over_the_blocks_length`, `a_fixed_key_ignores_the_blocks_length`, `a_step_outside_the_block_is_refused` | pass | -| 3 Bound | `events_per_block_are_bounded` (a discontinuity every 64 samples, every segment moving, seeks mid-block, q in {64, 100, 128, 1024}, at most `2⌈q/64⌉ + 3`) | pass | -| 4 Invariance | `quantum_invariance` (128 vs 100, both modes), `seek_invariance` (one timeline sample reached at three render positions after three histories, both modes) | pass | -| 5 No allocation | `allocation::ten_thousand_blocks_allocate_nothing` (0 allocations, 0 frees, seeks every 97th block); `build::bytes_report_equals_the_live_bytes` (`requested - released == bytes()`, 3 reservations, 0 reallocations); `build::refusals_are_typed`, `build::discontinuities_are_the_steps_of_the_hold_rule` | pass | -| 6 G5 | `cargo test --release -p wasm-gates --test g5_native_corpus --features math/lane` (7 tests incl. `g5_delegated_cases_use_the_owning_crates_pins` for the block, `g5_native_digests_match_pins`); wasm leg `scripts/run-wasm-gates.sh --without-v8-spill --without-native`; `automation::corpus` self-check `corpus::cases_produce_events_and_match_their_pins` | native pass; wasm leg pass (`wasm gates: ok`, no mismatch) | -| 7 Policy | `./target/release/realtime_policy` (143 regions in 31 files), `scripts/check-workspace-policy.sh` | ok | - -**AArch64** cannot run on this host. The pins are scalar `f64` through `math::pow` (no fused -operations, no lane type), generated on x86-64, and must match on the `aarch64-release` job. - -**Measured.** `Segment` 32 bytes, `CellState` 32 bytes (assertion at most 48). The six corpus cases -yield 20+ events each (case 5, `no_restart`, 42). - -### Attempt 2 - -The adversarial verdict on attempt 1 (`63741fb3c`) failed it on two MEDIUM, two LOW and one INFO -finding. Each is fixed below with the test that is red without the fix. - -1. **MEDIUM: the bound `2⌈q/64⌉ + 3` was false for a `no_restart` cell.** A jump deferred from - the previous block (its ramp, or a ramp the caller reported through `hold_until`, ran past the - discontinuity) fires in the run before a seek step, on top of that run's own discontinuities. - A tighter rule is not possible without changing what the cell renders: the deferred jump and - the next discontinuity are both real value changes (in the verifier's case value 2 holds for - three samples, 131 to 133, before the step to 3 at 134). Proof of the new bound - (`CellProgram::max_events` documents it): a block is at most two runs, `a + b = q - 1`; each run - holds at most `⌈run/64⌉` grid points (`G >= 64`) and `⌈run/64⌉` discontinuities (spacing at - least 64), and `⌈a/64⌉ + ⌈b/64⌉ <= ⌈q/64⌉ + 1`; the seek adds one; only a `no_restart` cell - defers, at most one deferral is outstanding (later covered discontinuities fold into it), a seek - clears it, so it adds one jump to the run before the step only. The bound is `2⌈q/64⌉ + 3`, and - `2⌈q/64⌉ + 4` for a `no_restart` cell; the verifier's case reaches `+ 4` exactly, so it is tight. - The per-cell capacity follows it: new `CellProgram::max_events(frames)`. The verifier's fuzzer, - with its bound read from `max_events`, over 173,438 spaced tables (seeks, caller ramps, forward - node-time moves) finds no excess, and its quantum-invariance and jump-oracle checks still pass. - - **The spacing dependency.** The 64-sample spacing is a session contract: #1626 D3 refuses - closer discontinuities (`automation.discontinuity_spacing`, `crates/session/src/validate.rs`, - `docs/SESSION_SCHEMA_V1.md`) with the same hold-rule definition the builder uses. So the - builder refuses them again, typed: `BuildError::Spacing { cell, time }` (the later - discontinuity of the first close pair), with `automation::DISCONTINUITY_SPACING = 64`. A valid - session never meets it; a caller that bypasses the session cannot break the bound silently. - The crate docs state the bound and the rule. Corpus case 2 had discontinuities at 3190 and - 3250 (60 apart), a table the session refuses; its step now starts at 3180, so **case 2's G5 - pin moved** (regenerated with `print_pins`). With attempt 1's `events.rs` and the new table the - regenerated digest is the same, so the move comes from the table alone; no event-rule change - below moves any of the six pins. - - Tests: `events::events_per_block_are_bounded` now asserts the stated bound for each cell - (`max_events(q)` equals `2⌈q/64⌉ + 3 + no_restart`, every block at most that, every offset in - the block), over a plain cell and a `no_restart` cell with caller ramps, seeks and - q in {1, 63, 64, 100, 128, 1024}; `events::a_carried_deferral_reaches_the_no_restart_bound` is - the verifier's case (six events, equal to `max_events(64)`), red against attempt 1's `+ 3`; - `build::closer_discontinuities_are_refused` (163 refused, 164 built, a step's own end counts), - red on attempt 1 (no refusal). -2. **MEDIUM: an offset outside the block.** A deferral below the span start (the caller moved node - time forward with no seek step) gave a negative distance that `as u32` wrapped. A deferred jump - is a jump already owed, so it is now **due at the run's first sample**, aimed at the curve - there (`target_at`), not dropped: dropping it would leave the old target until the next grid - event. Inside a run a deferral is always ahead, so only that case changes. The distance is now - converted with `u32::try_from`. Discontinuities behind the run are still dropped (the next grid - event retargets along the curve), now with one `partition_point` over the rest of the list - (`skip_behind`) instead of a linear loop. Test - `events::a_deferral_behind_a_forward_span_is_due_at_its_start` (one `Jump` at offset 0 to 3.0), - red on attempt 1 (`offset: 4294966927`). -3. **LOW: a second owner of the G5 pins.** `tests/corpus.rs` no longer compares digests; - `g5_native_digests_match_pins` is the single owner. The non-vacuity check stays as - `corpus::cases_produce_events`. -4. **LOW: clamp plus change only dropped a real jump.** When `value(d - 1) == value(d)` but the - left limit at `d` differs, the grid ramp aimed at `d - 1` has the jump's target, and change only - dropped the jump while that ramp was still short of it. `jump_event` now emits whenever the ramp - in flight completes after `completion(d, J)`, even with an unchanged value. Test - `events::a_jump_to_the_in_flight_target_still_lands_at_its_time` (the verifier's table: a `Jump` - at 32 to 32.0, ramp 0), red on attempt 1. -5. **LOW: render-reachable code outside the regions.** The `events.rs` region now begins at - `NodeSpan` and covers `NodeSpan::new`, `CellState`, `to_cursor` and `completion_of`; a new - region in `program.rs` covers the `CellProgram` accessors, `max_events`, `Program::{len, - is_empty, cell}` and `range`. The tool reports 144 regions in 32 files; the floors in - `tools/realtime-policy/src/policy.rs` are 32 and 144 (departure 8's 31 and 143 are superseded). -6. **INFO.** The `tools/wasm-gate-corpus` header names the automation block at the end of the - layout. `quantum_invariance` runs once: its blocks hold no seek, so the mode loop added nothing. - -### Attempt 3 - -The adversarial verdict on attempt 2 (`40514b1fc`) failed it on one MEDIUM finding. - -1. **MEDIUM: a deferral fired inside a caller ramp reported after it.** `next_in_run` checked the - lane (`next < free_at()`) only for a fresh discontinuity. A deferral fired at its stored time - even when the caller had called `hold_until(later)` between blocks; with a jump ramp of 0, - `commit` then set the lane idle, so the next discontinuity also fired inside the caller's ramp. - That broke D4: a `no_restart` cell whose ramp is in flight takes the jump at the ramp's last - sample plus one. Fix: a deferral that is due (`is_deferred`) meets the same lane check as a - discontinuity; while the lane is busy it is deferred again to `free_at()` (the cursor advances - only for a discontinuity). The check also covers a deferral behind a forward span, which is due - at the run's first sample (attempt 2, finding 2). A deferral only moves forward, so the loop - ends. The bound is unchanged: at most one deferral is outstanding. - - Test `events::a_deferral_waits_for_a_caller_ramp_reported_after_it` (the verifier's case on - `deferral_table`): `hold_until(130)`, block at 64, `hold_until(200)`, block at 128 gives only - held grid events (attempt 2 gave a `Jump` at 131 and at 134), and the next block gives one - `Jump` at 201 to 3.0. Its second half carries the deferral behind a forward span to 500 with - the caller's ramp to 600: no jump before 601, one `Jump` at 601 to 3.0. Each half is red on - attempt 2's `events.rs` and green after the fix. - - `events::events_per_block_are_bounded` again proves the tables stress the bound: for - `q >= 64` the most events in one block exceeds `⌈q/64⌉`. - - D4's bound bullet and gate 3 now state the `no_restart` term. diff --git a/.github/ISSUE_SPECS/1638-apply-timed-operations-inside-a-block-on-the-fader-stage.md b/.github/ISSUE_SPECS/1638-apply-timed-operations-inside-a-block-on-the-fader-stage.md deleted file mode 100644 index 48bbc27f1..000000000 --- a/.github/ISSUE_SPECS/1638-apply-timed-operations-inside-a-block-on-the-fader-stage.md +++ /dev/null @@ -1,284 +0,0 @@ -# Apply timed operations inside a block on the strip fader stage - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answer A1 (A1.5), under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. - -## Product outcome - -The strip fader and mute stage can apply a lane's fader and mute changes at any sample inside a -block, not only at block entry. A block with an operation at offset `o` renders, bit for bit, what -the same block split at `o` renders with the operation applied at the second part's entry, at -every width. A new "remember only" operation updates a lane's remembered fader gain without a -retarget while the lane is muted or a mute ramp is in flight. Nothing feeds operations yet; -#1647 drives them from stored fader automation. The live path is unchanged. - -## Context - -- **The stage.** `FaderRampStage` (`crates/builtins/src/lib.rs:2653-2663`) holds per channel a - `GainMuteRamp` (current gain, target, step, an `f32` countdown, the mute mask), each lane's - remembered `fader_gain`, its `muted` flag and a `u32` countdown `remaining`. - - `retarget` (`:2730-2750`) sets one lane's target and computes one step per event; a window of - 0 assigns at once. - - `set_fader_gain` (`:2752-2771`) remembers the gain and retargets; on a muted lane it retargets - toward `0.0` again (`:2768`), which restarts a mute ramp in flight (README finding F12). - - `set_mute` (`:2773-2794`) retargets to `0.0` or back to `fader_gain` and updates the mask. - - `process_plane` (`:2813-2865`) runs the settled kernel when no lane ramps, else the ramp kernel - over `min(max remaining, frames)` frames, then bookkeeping, then the settled kernel over the - rest. The `f32` countdown is recomputed from the `u32` one at each ramping call (`:2829-2834`). -- **The kernels.** `gain_mute_block` (`crates/lane/src/kernels/builtins.rs:127-140`) and - `gain_mute_ramp_block` (`:159-202`). The ramp kernel advances each lane by its own additions and - assigns the target exactly on the last frame, so partition and cohort invariance hold by - construction (`:180-183`). A settled lane in the ramp kernel multiplies by its exact target, as - the settled kernel does (`crates/builtins/src/lib.rs:2649-2652`). *Keep every trim, fader and - matrix ramp inside its endpoints* (#1408, its site 1) changes `gain_mute_ramp_block`'s last-step - law; it is a per-sample rule inside the kernel, so each lane still advances by its own additions - and a split still moves no bit for the same events. -- **Three users of the stage.** `BuiltinFaderBank` at Simd4 and Simd8 - (`crates/builtins/src/lib.rs:3777-3783`, `:3808-3968`), and `FaderMuteRampBuiltins`, the same - stage at `f32` (`:4163-4233`), which is the scalar track and the scalar oracle. -- **Bank processors.** The fader bank processor drains its lanes' live records at block entry and - processes the bank (`crates/builtins-compiler/src/lib.rs:744-752`; the drain `:1063-1105`). The - fused fader-and-matrix processor drains both and takes the fused settled path only when no fader - or matrix lane ramps (`:1198-1219`; `try_process_settled_with_matrix`, - `crates/builtins/src/lib.rs:3926-3956`). -- **Order at one entry.** Live records apply fader before mute, left before right - (`crates/host-core/src/live_delta.rs:66-68`). - -## Decisions frozen for this slice - -- **D1. Operations.** `FaderOperation` (in `crates/builtins`), per lane and channel selector: - - `RetargetGain { gain, samples }`: as `set_fader_gain` when the lane's gain is free, as - `RememberGain` when it is held (D2); - - `SetGain { gain }`: the same with a window of 0 (an exact set); - - `SetMute { muted, samples }`: as `set_mute`; - - `RememberGain { gain }`: update `fader_gain` only; no retarget. - Gains are linear (the caller converts dB with `checked_fader_gain`, `crates/builtins/src/lib.rs:4292-4297`). -- **D2. Held gain.** A lane's gain is held while it is muted, or while a mute ramp is in flight on - it. The stage tracks the second with one flag per lane and channel: `SetMute` with a nonzero - window sets it; the lane's countdown reaching 0, or a later gain retarget on that lane through - the live path, clears it. `gain_held(lane, channel)` and `remaining(lane, channel)` are readable, - so a caller (#1637's `hold_until`) knows when the lane's ramp ends. The live path's - `set_fader_gain` keeps its own rule, which *Session controlSmoothing: configurable ramp lengths for - live mute, fader and pan changes* (#1054) D10 sets as amended by the #1058 note (finding F12): - while the lane is muted (settled, or ramping toward 0) a live gain change only remembers the gain; - during an unmute ramp it retargets as today. This slice does not change the live path. -- **D3. In-block application.** `FaderRampStage::process_with` takes the block and a pull source of - operations: `next_offset() -> Option` and `apply(offset, &mut FaderOperationSink)`, which - applies every operation at that offset. The stage runs `process_plane`'s existing logic over each - piece between two offsets, then applies the operations at the next offset, and so on. The - countdown words are recomputed at each piece, as at each block today. No buffer is sized - `q × lanes`; nothing is allocated. A source with no operation gives today's single call. -- **D4. Fixed order at one offset.** Gain operations before mute operations, left before right, as - the live drain (`crates/host-core/src/live_delta.rs:66-68`). So an unmute and a fader move at one - sample unmute toward the new gain, and a mute and a fader move at one sample end muted. -- **D5. Exposure.** `BuiltinFaderBank::process_with` and `FaderMuteRampBuiltins::process_with` - forward to D3. `try_process_settled_with_matrix` declines a block with any operation (it already - declines a ramping lane). The bank processors keep calling `process`; #1647 switches the fader - bank processor to `process_with`. -- **D6. Class A.** No new arithmetic. The pieces run the existing kernels in the existing order, so - a split moves no bit. -- **D7. The acked-batch question.** No queue is added or changed. - -## Deliverables - -1. D1-D5 in `crates/builtins/src/lib.rs`. -2. Tests in `crates/builtins/tests/fader_ramp.rs` and the in-crate tests module. - -## Authorized paths - -- `crates/builtins/src/lib.rs`, `crates/builtins/src/tests.rs` (the in-crate tests module, `lib.rs:5627`), - `crates/builtins/tests/fader_ramp.rs` -- `crates/builtins-compiler/src/lib.rs`: only if a bank processor must name the new type to compile; - no wiring -- `crates/lane/src/kernels/builtins.rs`: only if a range entry to an existing kernel is needed; - no new arithmetic - -## Non-goals - -- Producing operations from stored automation (#1646 and #1647); the matrix, input and route - stages (#1651-#1653, #1656-#1657, #1658 and #1659-#1660). -- Changing the live path's records, drains or `set_fader_gain` (#1054 D10 as amended, finding - F12). -- The mute-cell composition with VCA and solo terms (#1651-#1653). - -## Hazards - -- **The settled tail.** `process_plane` runs the settled kernel after the last ramp ends. A piece - boundary inside that tail must not run the ramp kernel again with a stale countdown; D3's - recomputation at each piece covers it, and gate 1 checks offsets past a ramp's end. -- **Padding lanes** take no operation and stay inert. -- **The flag of D2** is new state. It must be cleared exactly when the lane's countdown reaches 0, - in the same bookkeeping step that assigns the target (`crates/builtins/src/lib.rs:2840-2854`). - -## Objective gates - -1. **Partition identity** (`crates/builtins/tests/fader_ramp.rs`, new). For a bank at Simd4 and - Simd8 and for `FaderMuteRampBuiltins` (scalar), at quantum 128 with an operation at offset `o` in - {1, 63, 64, 127}, and at quantum 100 with `o` in {1, 50, 99}: one `process_with` call equals, bit - for bit, `process` over `[0, o)`, the operation applied through the existing setters, then - `process` over `[o, q)`. Run for each D1 kind, on a settled lane, a ramping lane, a lane whose - ramp ends before `o`, and two lanes with operations at different offsets in one block. -2. **Remember only.** A lane muted over 480 samples, with `RetargetGain` operations every 64 - samples inside the ramp, renders the bits of the mute ramp alone; after unmute, the lane ramps to - the last remembered gain. The same with `RememberGain`. -3. **Order at one offset.** Unmute and a gain move at one offset end at the new gain; mute and a - gain move end muted. -4. **Existing behaviour.** Every existing test in `crates/builtins/tests/fader_ramp.rs` and - `crates/builtins-compiler` passes unchanged; `process` with no operations is the existing code - path. -5. **No allocation.** 1,000 blocks of `process_with` with operations allocate and free nothing - (`bench_support::alloc` thread counters after warming). -6. **No rendered bit moves.** No production path calls `process_with` yet; every render digest is - unchanged (`cargo test --locked -p builtins-compiler --features test-support`, the browser legs of - the `browser` job in `.github/workflows/qualification.yml`). -7. **Commands:** - - `cargo test --locked -p builtins --features builtins/test-support` - - `cargo test --locked -p builtins-compiler --features test-support` - - `bash scripts/run-aarch64-tests.sh debug` (the `aarch64-debug` job: Simd4 on NEON) - - `bash scripts/check-realtime-policy.sh`, `bash scripts/check-unfused-seal.sh`, - `bash scripts/check-workspace-policy.sh` - - `bash scripts/check-cross-targets.sh` (README F19, the iOS memset rule: no new - `memset_pattern16` call; fix one in code, never by a ceiling) - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` - -## Test value - -- Gate 1 turns red if a piece reuses a stale countdown, applies an operation one sample early or - late, or if a width diverges from the scalar oracle. The existing partition test - (`a_cross_block_ramp_is_partition_invariant`, `crates/builtins/tests/fader_ramp.rs:284`) has no - operation inside a block. -- Gate 2 turns red if an automation event during a mute ramp restarts the ramp toward 0 (finding - F12's artefact) or loses the remembered gain. -- Gate 3 turns red if the order at one offset differs from the live drain's. - -## Dependencies - -- None. -- Batch: R1, with #1646 *Prepare stored fader automation and render it flat* (which uses - `SetGain`) and #1647 *Render moving stored fader automation, seeks and latency*. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **F12 never landed.** #1054 is closed and its D10 was not amended: `set_fader_gain` still - retargets a muted lane toward 0 (`crates/builtins/src/lib.rs:3083-3098`). This slice takes the - live rule as well (README F22): while a lane is muted or a mute ramp toward 0 is in flight, a live - gain change only remembers the gain; during an unmute ramp it retargets as today. D2's sentence - "This slice does not change the live path" is replaced by this rule, and gate 2 gains the live - case. Authorized paths are unchanged (`crates/builtins/src/lib.rs`). -- **Anchors moved.** `FaderRampStage`'s setters are now near `crates/builtins/src/lib.rs:3083-3140`, - `BuiltinFaderBank` `:4314`, `FaderMuteRampBuiltins` `:4732`; re-read at filing. #1408 has landed. -- **Realtime policy** as in #1637's amendment. - -## Decision record: #1638 (batch 13b) - -**Shipped** (`crates/builtins/src/lib.rs`). `FaderOperation` (D1), `FaderOperationError` -(`Lane`, `Gain`, `Duplicate`), `FaderOperationSink` (fixed size, on the render stack; one gain and -one mute slot per lane and channel; `push` refuses a padding lane, a gain that is not finite or is -negative, and a second operation of one kind at one offset, and a refusal queues nothing on either -channel), and the pull trait `FaderOperationSource { next_offset, apply }`. `FaderRampStage` gains -one flag per lane and channel, `mute_ramp` (D2): set by a mute change with a nonzero window, cleared -by a zero-window mute change, in the bookkeeping step that ends the countdown, by a live gain -retarget, and by `reset`; it is carried by `export_lane`/`import_lane` (a new private field of -`FaderChannelState`), so a plan swap during a mute ramp keeps the hold. `process_with` cuts the -block at each offset and runs `process_plane` over each piece, so a split moves no bit (D6); the -sink applies gains before mutes, left before right, whatever the push order (D4). Exposure (D5): -`BuiltinFaderBank::{process_with, apply_fader_operation, gain_held, ramp_remaining, -try_process_settled_with_matrix_and_operations}` and `FaderMuteRampBuiltins::{process_with, -apply_fader_operation, gain_held, ramp_remaining}`. No production path calls `process_with`. - -**Live rule (amendment, README F22).** `set_fader_gain` on a muted lane (settled or ramping toward -`0`) now only remembers the gain; on an unmuted lane, including one in an unmute ramp, it retargets -and clears `mute_ramp`. Timed `RetargetGain`/`SetGain` remember only while the gain is held (muted -or `mute_ramp`), so they also remember during an unmute ramp, as D2 says. - -**Departures.** -- Offsets: an offset at or past the block is not pulled and stays with the source; offsets must - rise strictly, and a repeated or lower offset stops pulling for the block (debug assertion), - which bounds the loop at `frames` pieces. -- D5's decline is a new method, `try_process_settled_with_matrix_and_operations`, which declines - when the source has an operation inside the block and otherwise forwards; the existing method and - its callers are unchanged. -- Gate 4 "every existing test passes unchanged" cannot hold with the amended live rule: in - `crates/builtins-compiler/src/lib.rs`, - `the_drain_applies_the_fader_before_the_mute_whatever_the_write_order` relied on a fader move - after a *mute* being observable; under F22 both orders are now alike. The test now starts the lane - muted and writes an *unmute*, where the order is still observable (fader-then-unmute ramps to the - new gain; unmute-then-fader cuts the unmute ramp short). Test-only edit; no wiring. -- `tools/realtime-policy/src/policy.rs`: `marked_regions_floor` 144 -> 147 (the sink, the bank's and - the scalar track's `process_with`); files stay 32. - -**Gates.** -1. `an_operation_inside_a_block_renders_the_split_blocks_bits` (Simd4, Simd8, scalar; q 128 at - o in {1, 63, 64, 127} and q 100 at o in {1, 50, 99}; six operations; settled, ramping, ramp - ending before o, muted, mute ramp, unmute ramp; one lane and two lanes at different offsets; - the next block compared too), `an_operation_at_offset_zero_is_block_entry_and_one_past_the_block_waits`. -2. `gain_moves_during_a_mute_ramp_are_remembered_and_never_restart_it` (timed `RetargetGain`, - `RememberGain` and the live setter, every 64 samples in a 480-sample mute ramp). -3. `at_one_offset_the_gain_applies_before_the_mute`. -4. Existing `fader_ramp.rs` tests unchanged and green; builtins-compiler as above. -5. `process_with_operations_never_allocates` (1,000 blocks per subject, thread counters: 0 and 0). -- D2 flag: `the_held_flag_follows_the_mute_ramp`; sink refusals: - `the_fader_operation_sink_refuses_without_queuing` (in-crate). -- Red on revert (copied aside, not stashed): live muted retarget restored -> gate 2 red; timed held - retarget -> gate 2 red; mutes applied before gains -> gate 3 red; operations one sample late -> - gate 1 red; flag not cleared at the countdown end -> gate 2 and the flag test red. - -**Successor obligation (#1647).** D2 holds a lane's gain during an *unmute* ramp too: a -`SetMute` with a nonzero window sets `mute_ramp`, and a timed `RetargetGain` or `SetGain` on a held -lane only remembers the gain (`FaderRampStage::timed_gain`, `crates/builtins/src/lib.rs`), while a -live gain change during an unmute ramp retargets. #1637 holds only `Grid` events: a `Jump` (on a -cell that is not `no_restart`) and a seek `Set` always emit and commit the cell's target with -`held = false`. So when #1647 converts a `Jump` or a `Set` that lands inside an unmute ramp to -`RetargetGain` or `SetGain`, the stage only remembers it; the unmute ramp settles at the -pre-unmute gain while `fader_gain` and the cell hold the new value, and on a flat curve the cell -emits nothing more (its target equals the curve and `held` is false), so the lane stays wrong with -no end. #1647 D1.2 (`hold_until`) covers only `Grid` events. #1647's amendment from this review -records the obligation and its gate. - -**Attempt 2 (review fold).** -- Docs: the successor obligation above; #1647 gains an amendment (the reissue at the end of a held - ramp, and its gate). `CellState::hold_until` only sets the in-flight completion, and `held` is set - only by a held `Grid` event whose value differs from the cell's target, so #1637 cannot re-emit a - committed `Jump` or `Set`; the amendment therefore makes #1647 reissue the cell's target itself. -- `FaderOperationSource` rustdoc states the contract: strictly rising offsets, and a source keeps - every operation it did not apply. `BuiltinProcessReport` gains `fader_operations_stopped` (`1` - when a `process_with` block stopped pulling on a repeated or lower offset, else `0`), set by both - `process_with` methods from a `bool` the stage now returns; the debug assertion stays. Not - tested: a debug test build fails the assertion first. No rendered bit moves. -- Test: `the_fused_settled_path_declines_only_an_operation_inside_the_block` - (`crates/builtins/tests/fader_ramp.rs`; Simd4, and Simd8 with AVX2): an operation inside the - block (declines, the block and every fader and matrix word unchanged, the source keeps the - operation, then `process_with` equals a twin), at `offset == frames` (forwards: equals a twin on - `try_process_settled_with_matrix`, the source keeps it), and no operation (forwards). Red, file - copied aside and restored with `cp`: `<=` for `<` (the block-end case), no decline (the inside - case), a decline that first runs the fused path (block untouched fails), a decline that applies - the operation (stage words fail). - -**Attempt 3 (record fold, coordinator).** The attempt-2 review measured that #1647's D1.6 as first -written was not quantum-invariant: refreshing `hold_until` at every block entry inside a held ramp -overwrote a cell `Jump`'s own ramp end. #1647's amendment now says when D1.2 reports the hold (once, -at the block entry that starts or restarts the ramp, and after an adoption that carries it), holds -the grid for the reissue's own ramp, composes the reissued gain as D1.4, and adds the moving-curve -cases to gate 8 (cases (c) and (e), whose samples `[E, E + J - 1]` equal one direct `RetargetGain` -at `E` over `J`). No code change. The counter `fader_operations_stopped` is to be summed by #1647 -when it wires `process_with` into production. - -**Attempt 3 review and rescope (coordinator).** The attempt-3 review confirmed the hold rule is -quantum-invariant and found two more design gaps in #1647's D1.6 (a grid event that is not held -inside the hold's last 64 samples; stopping the cell exactly at the ramp end). Three reviews found -this slice's code correct; every failure was in the successor's mechanism. The mechanism is -therefore #1647's design: #1647's amendment now states the requirement, the measured constraints and -gate 8, and #1648's amendment the carry. This slice's obligation is that record. - -**Gates 6 and 7 on the integrated tree (batch 13b, cddff37d9).** Gate 6: `audit capi` gives -`pcm_digest` `bf7dcedd1d92621f`, as on main; the browser fixtures' digests and exact rows are -unchanged (`check-browser-expected-resources`), and chromium 151, firefox 153 and webkit 26.5 -qualification pass with 0 render allocations. No fixture moves a fader on a muted lane, so the F22 -live rule moves no pinned digest; its behaviour is held by gate 2. Gate 7: the batch gate set -(workspace tests, clippy, `check-cross-targets.sh`, the policies) passes. The AArch64 legs -(`aarch64-debug`, Simd4 on NEON) run only in CI: this host has no AArch64 toolchain. diff --git a/.github/ISSUE_SPECS/1639-hold-the-automation-jump-lengths-in-a-cell.md b/.github/ISSUE_SPECS/1639-hold-the-automation-jump-lengths-in-a-cell.md deleted file mode 100644 index 3d0fd0349..000000000 --- a/.github/ISSUE_SPECS/1639-hold-the-automation-jump-lengths-in-a-cell.md +++ /dev/null @@ -1,278 +0,0 @@ -# Hold the automation jump lengths in a plan cell - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answers A3 and A10, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. - -## Product outcome - -A host that changes a playing session's `control_smoothing` hears the new ramp lengths on the next -automation jump, with no plan rebuild. On a session with stored automation the edit's response -reports `live`; on a session without it, `model_only`, as today's rule for that edit says. A jump -already in flight keeps the length it started with. - -## Context - -- **Jump lengths** (README A3 and A1.4): a jump of a fader or trim cell ramps over the - session's `fader_ms`, of a mute or polarity cell over `mute_ms`, of a pan or matrix cell over - `pan_ms`, each converted to samples by `SessionModel::control_smoothing_samples()` (*Session - `controlSmoothing`: configurable ramp lengths for live mute, fader and pan changes*, #1054 D2-D3). - HPF, LPF and EQ targets use their fixed 64-sample ramp; other effect parameters their descriptor's - `smoothing_samples`. A session `explicit` 0 makes a jump a step. -- **No such field at HEAD.** `SessionModel` has no `control_smoothing` yet, and - `LiveRamps::for_session` returns 0 for every row (`crates/host-core/src/live_delta.rs:33-58`). - #1054 adds both. This slice is written against #1054. -- **The edit.** *Edit control_smoothing by a session transaction, model-only* (#1365) adds opcode - `0x0008` and masks `control_smoothing` in the classifier (#1365 D4), so its delta is empty and its - path `model_only` (#1313 D2). #1365 lands before this slice and is not amended: this slice - makes the path `live` when stored automation reads the lengths (D3). -- **The cell primitive.** #1312 D1: a triple-buffered latest-target cell with one writer (control) - and one reader (render); a write that completes before a block's drain is applied in that block, - and render never skips the newest completed write. -- **The block entry.** The graph's render begins each block with the source set's `begin_block` - (`crates/graph/src/lib.rs:3254-3256`). -- **#1637 D2** gives each cell program a jump-length key, never a copied `J`: a builtin row's key - names a word of this slice's cell (`Fader`, `Mute`, `Pan`); filter and effect cells keep a fixed - length. This slice lands before #1646, the first program that reads the cell. - -## Decisions frozen for this slice - -- **D1. One plan-level cell.** Every host-core plan owns one #1312 latest-target cell of three - words: `mute`, `fader`, `pan`, in samples, the value of `control_smoothing_samples()`. - Preparation writes it on both hosts. A builtin cell program reads the word its row names (fader - and trim: `fader`; mute and polarity: `mute`; pan and matrix: `pan`; #1637 D2's key). Filter, - EQ and effect cells keep their fixed length. The cell's bytes are fixed and do not depend on the - session. -- **D2. One reader.** The cell has one reader, so render reads it once per block: the graph reads it - at block entry, beside the source set's `begin_block`, and holds the three values for that block. - #1647 passes them, with the node-time mapping, in the block context it gives every strip - stage. A jump reads its row's value when it starts. A ramp in flight keeps its length. No stage - reads the cell itself. -- **D3. The edit path.** In `classify_live_delta`: - - when `control_smoothing_samples()` differs between `current` and `next`, and `next` has at least - one automation entry on a row whose jumps read a word of D1 (builtin rows 1, 2, 5, 6, 7-10 and - 12), the delta carries one `jump_lengths: Option<[u32; 3]>` value. `LiveDelta::is_empty` reads - it, so the path is `live`. The test reads the session only, not which rows render yet: the cell - is in every plan, so the write always lands, and a row whose rendering slice is still to come - reads the new value from its first jump; - - otherwise nothing changes: the delta stays empty and the path `model_only` (#1365 D4); - - equal sample counts give no value, even when the milliseconds differ (never a redundant write). - The commit writes the cell after every fallible check. A write to the newest candidate follows - #1053 D7, as every live write does. -- **D4. Swaps.** A successor is prepared from the committed model, which holds the edit, so its cell - starts with the new lengths. Nothing carries. -- **D5. The browser.** Preparation fills the cell, through the same host-core preparation on both - hosts. D3 lives in the shared classifier, which the browser's Worker commit calls after *Admit - browser live edits in the Worker through the committed model* (#1382). So a browser transaction - that edits `control_smoothing` meets the same rule as the C ABI's, from when the browser's apply - export (*Export transaction apply and anchored seek from the browser engine module*, #1293, before - #1643 in batch R1) can carry one. This slice adds no browser path and no browser gate. -- **D6. #1365.** D3 is the `live` classification of #1365's `control_smoothing` edit. This slice - depends on #1365, which lands first, and changes the classifier code #1365 adds; #1365 is not - amended. -- **D7. Docs.** The `controlSmoothing` paragraph that #1054 adds to `docs/SESSION_SCHEMA_V1.md` - gains one sentence: the same row lengths are the jump ramps of stored automation (README A3). - #1054 is not amended. -- **D8. The acked-batch question: can an ack ever precede a drop? No.** The cell write cannot fail, - every check precedes it, and a replaced value is in the committed model. - -## Deliverables - -1. D1 and D2: the cell, preparation's seed and the graph's block-entry read. The strip stages' reads - come with their rendering slices (#1647 for the fader; #1651, #1657 and #1658 for their - words). -2. D3 in the classifier and the commit path; D7. - -## Authorized paths - -- `crates/host-core/src/prepare.rs` (the seed only), `crates/host-core/src/live_delta.rs`, - `crates/host-core/tests/live_delta.rs` -- `crates/graph/src/lib.rs` (the block-entry read and its `test-support` reader only; stream A's - file, sequenced by root) -- `crates/control-plane/src/` (the commit path), `crates/capi/src/runtime/live_tests.rs` -- `docs/SESSION_SCHEMA_V1.md` (D7's sentence only) -- Tests that pin a plan byte row the cell moves (re-pin each with the reason "the jump-length cell, - #1639 D1", never in bulk) -- `crates/capi/Cargo.toml` (the `graph` dev-dependency's `test-support` feature, as #1642 adds - it; only if #1642 has not landed), `Cargo.lock` - -## Non-goals - -- The `control_smoothing` grammar, defaults and rounding (#1054). The edit's opcode (#1365). A - per-edit ramp length (#1394): stored automation has no per-edit length. -- Changing a ramp already in flight. - -## Hazards - -- **One reader only.** A second reader of a #1312 cell breaks its triple buffer. The reader half is - one owned value, moved into the graph at preparation and not `Clone`, so a second reader cannot be - written. -- **A transaction that changes `control_smoothing` and a rendered automated row together** is a - rebuild once #1648 classifies that row (its D1); the successor is prepared with the new - lengths (D4), so D3's write is not needed there. - -## Objective gates - -1. **Block-entry read** (`crates/graph` test, new, through a `test-support` reader of the block's - held values). A fresh plan holds the prepared lengths at its first block. A write of new lengths - before a block entry is held from that block on, not before; two writes before one block give - the last. The audible effect on a jump is #1647's gate 2. -2. **Classifier** (`crates/host-core/tests/live_delta.rs`, new). With a fader entry, and with a mute - entry: a `control_smoothing` change gives a live delta with `jump_lengths` and no other record. - Without stored automation, or with only an effect entry: an empty delta. A change of `fader_ms` - from 10.0 to 10.001 at 48 kHz (both 480 samples): an empty delta. -3. **C ABI** (`crates/capi/src/runtime/live_tests.rs`, new). A playing session with a fader entry takes a `SetControlSmoothing` - edit: the response path is `live`, the provider epoch is unchanged, and the plan's cell holds the - new lengths from the next block (gate 1's reader). The same edit on a session without automation - reports `model_only`. -4. **Realtime.** The render block that reads a written cell allocates and frees nothing - (`bench_support::alloc` thread counters, statics warmed), and - `cargo build --locked --release -p audit -p capi && ./target/release/audit capi` reports all - violation counts 0. -5. **Commands:** - - `cargo test --locked -p host-core --features host-core/test-support --test live_delta`, - `cargo test --locked -p builtins-compiler --features test-support`, - `cargo test --locked -p graph --features test-support`, `cargo test --locked -p capi` - - the workspace debug leg (`test-debug-a` in `.github/workflows/qualification.yml`) - - `bash scripts/check-realtime-policy.sh`, `bash scripts/check-graph-policy.sh`, - `bash scripts/check-host-core-policy.sh`, `bash scripts/check-workspace-policy.sh` - - `bash scripts/check-cross-targets.sh` (README F19, the iOS memset rule: no new - `memset_pattern16` call; fix one in code, never by a ceiling) - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` -6. **No rendered bit moves** for a session with no stored automation: `audit capi` gives the same - `pcm_digest` at base and head (PR evidence), and the browser legs of the `browser` job pass with - unchanged digests. - -## Test value - -- Gate 1: red if the graph reads the cell only at preparation, under no snapshot, or reads an - older write, or if the cell's allocations drift from its retained row. -- Gate 2: red if the edit is `live` with no entry that reads a word (a needless write), `model_only` - with one (the new lengths never reach render), or if equal sample counts still write. -- Gate 3: red if the commit does not write the cell, or reports the path from anything but the - classifier's value; with a structural edit beside it (D4), red if the successor's cell is not - seeded from the committed model. -- Gate 4: red if the block-entry read or the hand-off to the stages allocates on render. - -## Dependencies - -- *Session `controlSmoothing`: configurable ramp lengths for live mute, fader and pan changes* - (#1054). -- *Hold live values in latest-target cells on both hosts* (#1312). -- *Edit control_smoothing by a session transaction, model-only* (#1365), for D3. It brings the - edit's `model_only` path from *Report each transaction's edit path in its response* (#1313). -- *Admit browser live edits in the Worker through the committed model* (#1382), for D5. -- Batch: R1. #1646 builds on it; no dependency on #1648. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **The keys are setting rows.** `control_smoothing`'s keys are the session rows `fader_ms`, - `mute_ms` and `pan_ms`, family `Setting` (`crates/controls/src/session_rows.rs:38-75`), set by - `SET_CONTROL`; `SetControlSmoothing` (`0x0008`) is retired (#1561; - `crates/protocol/src/model.rs:2281-2285`). A setting change gives no record and sets - `LiveDelta::settings_changed`. D3's "edit path" is that row change; the `live` classification - rule of D3 stands. -- **The cell holds the three key lengths**, the values of `controls::LiveRamps` - (`crates/controls/src/live_ramps.rs:14-24`); a stage resolves a row's length with - `LiveRamps::for_row` (polarity is twice `mute_ms`; sends and VCAs have their own rows), as #1637 - (amended) says. D1's row-to-word list is replaced by that function. -- **Context.** #1054 and #1365 have landed; `LiveRamps` moved from host-core to `controls`. -- **The browser** is the same `SessionState` (#1562a); D5 holds by construction. - -## Decision record: #1639 (batch 13b) - -**Shipped.** `crates/graph`: `GraphJumpLengths { fader_samples, mute_samples, pan_samples }` (the -three `LiveRamps` key values), one `LatestCell<3>` per plan with its dirty word in one `Arc`, the -boxed one reader in the executor (`JumpLengthReader`, not `Clone`, built only by -`graph::attach_jump_lengths`), the writer `GraphJumpLengthProducer::write(stamp, lengths)`, and -`jump_length_retained_bytes` / `jump_length_largest_allocation_bytes`. `render` reads the cell once -per block right after it takes the block's live snapshot, beside the source set's `begin_block` -(`read_counted`, so a replaced write joins `live_values_superseded`), and holds the three values for -the block. The test-support reader is `graph::test_only_block_jump_lengths()` (a thread-local the -block entry sets). `crates/host-core`: `session_jump_lengths(model)` (from -`LiveRamps::for_session`), preparation attaches and seeds the cell on every plan after bind and -charges it in the graph row, `HostLiveControlHandles::jump_lengths`; the classifier's -`LiveDelta::jump_lengths: Option` (read by `is_empty`) and -`automation_jump_row(target)`. `crates/control-plane`: the epoch's `StripLanes` holds the writer (preparation refuses a plan -without one, review fold 1); `commit_live_once` writes the value with the records, under the stamp, -before the revision's publication. D7's sentence is in `docs/SESSION_SCHEMA_V1.md`. - -**Departures and choices (code truth).** - -1. **Attach after bind, not in `PreparedGraphPlan`.** The cell is attached to the bound plan by - `graph::attach_jump_lengths(&mut PreparedRenderPlan, ...)` (a downcast, as the carry installers - do), so no builtins-compiler pass-through and no change to `bind` were needed. The seed is the - reader's initial held value, not a cell write: a fresh plan holds the prepared lengths at its - first block with no stamp at preparation. -2. **D3's row set** is `automation_jump_row`: a builtin, strip-own, route or VCA row whose live - motion is `Motion::Ramp(row)` (`controls::row_motion`), plus an effect's bypass row - (`BYPASS_CONTROL` -> `LiveRampRow::BypassCrossfade`, the `mute_ms` crossfade; its descriptor's - motion is `Fixed` because it takes no per-edit length). No hard-coded row-number list. Filters, - the delay, effect parameters and the (rebuild-only) link mode read no length. -3. **The count is charged on every plan that made its own.** The cell's reader holds a clone of - `live_values_superseded` in every plan, so the graph row now charges the preparation's own count - whenever the host passed none (before: only with strip controls). -4. **The fast classifier** (`classify_rows`) never sees a `control_smoothing` change - (`changed_rows` refuses session rows, so the full classifier runs); it leaves `jump_lengths` - `None`, with a comment. -5. **Known gap, by the spec's design:** a `control_smoothing` change committed while the session has - no reading entry is not written; an entry added later (model-only until #1648 classifies - automation edits) reads the cell's older lengths. The hazard section's rebuild rule (#1648 D1) - closes it for rendered rows. -6. **D3's field type.** D3 names `jump_lengths: Option<[u32; 3]>`; the shipped field is - `LiveDelta::jump_lengths: Option`, the same three lengths with a name for each - key (`fader_samples`, `mute_samples`, `pan_samples`), so no caller depends on the word order. - The cell still holds three `u32` words (`GraphJumpLengths::to_words`). - -**Re-pinned rows** (each "the jump-length cell, #1639 D1"; measured: the cell's two allocations are -168 bytes, a count 24 bytes, the revision gate 72 bytes; since the review fold gate 1 measures the -cell's bytes and its two allocations against `jump_length_retained_bytes()` with the thread -counters): - -| Test | Row | Before | After | -|---|---|---|---| -| `host-core` `tests/prepare.rs` `the_graph_row_counts_the_plans_gate_and_a_count_it_made_and_holds` | plain preparation (no strip controls), graph row less incremental | 72 (gate) | 264 (gate + cell + its own count) | -| same | strip controls, the preparation's own count | 96 | 264 | -| same | strip controls, the session's count | 72 | 240 | -| `host-core` `tests/live_routes.rs` `live_send_lanes_are_charged_against_the_graph_cap` | live total less idle total | route lanes + 24 (count) | route lanes (both plans hold the cell and their count); the test pins no total; live total 118304 at head (read from a failing run), 168 bytes (one cell) above the base it derives from | -| `host-web` `tests/retained_ceilings.rs` `the_graph_row_charges_the_plans_gate_and_not_the_sessions_superseded_count` | browser graph row less incremental | 72 | 240 | - -`capi` `model_only_edits_commit_without_a_plan_rebuild`: the `control_smoothing` edit now comes -before the automation upsert (on a session with a fader entry it is `live` by D3); not a byte row. - -**Gates.** - -| Gate | Test | Result | -|---|---|---| -| 1 Block-entry read | `graph` `tests/jump_lengths.rs` `the_block_entry_holds_the_newest_published_lengths` (prepared lengths at block 0; an unpublished write not held; held from the first block after publication; two writes give the last and count one superseded; a second attach refused; the attach leaves exactly `jump_length_retained_bytes()` live in 2 allocations, 0 frees). The probe is set right after the block's snapshot, so the test does not show where in the block the read is | pass | -| 2 Classifier | `host-core` `tests/live_delta.rs` `a_control_smoothing_change_writes_the_jump_lengths_only_for_automation_that_reads_them` (fader, mute, pan entries live; none, EQ parameter entry and 10.0 -> 10.001 ms empty; effect bypass entry live), `the_rows_that_read_a_jump_length_are_the_ramped_rows` | pass | -| 3 C ABI | `capi` `runtime::live_tests::a_control_smoothing_edit_on_an_automated_session_writes_the_jump_lengths_live` (SET_CONTROL of the three smoothing rows: `live`, same epoch, no candidate, next block holds 96/48/144; without automation `model_only`, prepared 960/480/960 held). Red with the commit's write removed. No candidate is pending, so it does not show which plan a write with a candidate reaches. D4: `a_successor_prepared_with_a_control_smoothing_edit_holds_the_new_lengths` (an added track and the three rows in one transaction: `rebuild`, one candidate; after adoption, a new epoch, the next block holds 96/48/144). Red with the seed made a constant 960/480/960: the successor held 960/480/960. | pass | -| 4 Realtime | gates 1 and 3 count 0 allocations and 0 frees in the block that applies the write; `audit capi`: every violation count 0 | pass | -| 5 Commands | `graph --features test-support`, `host-core` (all features), `capi`, `control-plane`, `host-web` suites; the `test-debug-a` workspace leg (2228 passed, 0 failed); `realtime_policy` (149 regions in 32 files; floor 147 -> 149 for this issue's two regions), graph, host-core and workspace policy scripts; `check-cross-targets.sh` (PASS, iOS memset only the known #1018 builtins/lane calls); workspace clippy `-D warnings`; `fmt --check` | pass | -| 6 No bit moves | `audit capi` `pcm_digest` `bf7dcedd1d92621f` at base and head. Browser legs (batch 13b integration, cddff37d9): `check-browser-expected-resources` digests and exact rows unchanged; chromium 151, firefox 153 and webkit 26.5 qualification pass | pass | - -**Review fold (batch 13b).** Five minor findings of the adversarial review, folded: - -1. **The invariant is typed.** The epoch's `StripLanes::jump_lengths` holds the writer, not an - `Option`: `prepare_runtime` refuses a handle set without it with the cell's own attach failure - (`host.jump_lengths.attach`, stage `Graph`), and `commit_live_once` lost its unreachable - `Internal` refusal. `HostLiveControlHandles::jump_lengths` stays an `Option` (handle sets built - without a preparation) and converts at the control-plane boundary. -2. **The cell's bytes are measured.** Gate 1 counts the attach with the thread counters (statics - warmed on a throwaway plan): requested less released bytes equal - `jump_length_retained_bytes()`, in 2 allocations and 0 frees. The counters carry no largest - allocation, so `jump_length_largest_allocation_bytes()` is not measured. -3. **Two claims removed.** Gate 1 no longer claims "a block late": its probe is set right after the - block's snapshot. Gate 3 no longer claims "a plan that is not the newest": that test has no - candidate. -4. **D4 has a C ABI test** (gate 3's row): `a_successor_prepared_with_a_control_smoothing_edit_holds_the_new_lengths`, - red with the seed made a constant. -5. **`session_jump_lengths`** moved out of the `use` block in `crates/host-core/src/prepare.rs`. - `prepare.rs` also moves the graph row's charge (the superseded count charged when the - preparation made its own), the largest-allocation row and a `HostLiveControlHandles` field, - past the authorized "the seed only": all three are needed for D1 and are covered by the root - ruling on byte rows. diff --git a/.github/ISSUE_SPECS/1643-seek-the-session-from-the-headless-engine.md b/.github/ISSUE_SPECS/1643-seek-the-session-from-the-headless-engine.md index 1c8c9ab8e..a8a3ce14c 100644 --- a/.github/ISSUE_SPECS/1643-seek-the-session-from-the-headless-engine.md +++ b/.github/ISSUE_SPECS/1643-seek-the-session-from-the-headless-engine.md @@ -176,3 +176,16 @@ Checked against `12d4a4f9d`. exports of #1562a are `:3308-3383`. - **Dependencies.** #1293 (open) for the anchored export's rules; #1381a has landed, #1381 and #1387 are open. Decision 17 C13 applies as in #1642's amendment. + +## Amendment (controls coordinator, root rulings, 2026-10-15) + +- **Gate 3a uses a dropped export.** Root ruling B15-R8 (2026-10-15, recorded in #1293) drops #1293's + D1 staging, and decision 17 dropped `miso_engine_web_v1_apply`. Gate 3a commits its structural + transaction as a `SESSION_TRANSACTION_APPLY` frame through #1562's generic entry + (`miso_engine_web_v1_control_request_ptr` and `miso_engine_web_v1_control_command`), and reads + the outcome from the response frame. The rest of the gate is unchanged. +- **Discontinuity first.** Root ruling B15-R7 (2026-10-15, recorded in #1323): the headless `seek` + (D2) calls `miso_engine_web_v1_declare_discontinuity` (#1323) before it calls the session seek + export, as a C ABI host declares before its session seek (#1642). So browser latency floors reset + as they do on the C ABI. `WasmBoundary` gets the call. #1323 adds the export and its mirrors and + is a dependency. Gate 2 asserts the declaration comes first. diff --git a/.github/ISSUE_SPECS/1644-seek-the-session-from-the-browser-sdk.md b/.github/ISSUE_SPECS/1644-seek-the-session-from-the-browser-sdk.md index 2ebf8cc0c..a0d939103 100644 --- a/.github/ISSUE_SPECS/1644-seek-the-session-from-the-browser-sdk.md +++ b/.github/ISSUE_SPECS/1644-seek-the-session-from-the-browser-sdk.md @@ -225,3 +225,16 @@ Checked against `12d4a4f9d`. - **The browser's own command path is deleted** (#1565); the session seek message is one more control message beside #1562a's. - **Dependencies.** #1643; #1294 and #1387 are open; #1332 only for the Worker half. + +## Amendment (controls coordinator, batch 15a fold, root ruling 2026-10-16) + +- **Discontinuity first (B15-R7 reach).** The browser SDK's seek declares a discontinuity before + its session seek, as #1643's headless `seek` does and as a C ABI host does before its session + seek (#1642). D1's handler calls `miso_engine_web_v1_declare_discontinuity` (#1323) once, then + `miso_engine_web_v1_session_seek`, in the same control turn. If the declaration does not return + OK, the handler replies with that result and does not call the seek. So browser latency floors + reset at a seek as they do on the C ABI. +- **Gate 2** adds: the handler calls the declaration before the seek export, once per message; a + refused declaration calls no seek export. +- **Dependencies** gain #1323 (the export and its mirrors). Authorized paths do not change: the + call is in D1's handler. diff --git a/.github/ISSUE_SPECS/1645-validate-builtin-automation-targets-at-preparation.md b/.github/ISSUE_SPECS/1645-validate-builtin-automation-targets-at-preparation.md deleted file mode 100644 index 7a981fbf3..000000000 --- a/.github/ISSUE_SPECS/1645-validate-builtin-automation-targets-at-preparation.md +++ /dev/null @@ -1,353 +0,0 @@ -# Validate builtin automation targets against their rows at preparation - -Proposed by *Research: render stored session automation in the engine, identically on every -platform* (#1058), answer A3, under decision 15 D15-16 -(`docs/rulings/live-updates-seamless-swaps-and-one-control-plane-2026-10-05.md`). A draft for root -to file. Code anchors verified on `6ee64f484`. - -## Product outcome - -A session whose builtin-target automation does not fit its row is refused, typed, on every path -that prepares a session: browser boot, C ABI boot and every C ABI rebuild, and `session-validator` -stage `prepare-builtins`. Refused today and accepted silently: a fader ride written in `linear`, a -mute that ramps, a fader ride to +30 dB, an HPF that glides out of "off", and an HPF curve that -crosses the LPF on the same lane. The order check is one function of a session model, so the slice -that classifies input-filter edits on an automated strip (#1660) can refuse a later static edit -that would cross a curve with the same code. Nothing renders the table yet. - -## Context - -- **The session crate cannot see descriptors.** It depends on `engine` and `json-syntax` only - (`crates/session/Cargo.toml:15-17`; `scripts/check-session-policy.sh`), so its target check - (`crates/session/src/validate.rs:854-888`) reads only the parameter ID and the channel form. - #1626 adds the lane, strip-kind and discontinuity rules there. -- **The builtin descriptors.** `BUILTIN_PARAMETER_DESCRIPTORS` (`crates/builtins/src/lib.rs:453-673`); - `builtin_parameter_unit` (`:254-269`) gives `db` for trim and fader, `hz` for the filters and - `linear` for polarity, mute, matrix and pan. `BuiltinParameterDomain` (`:288-296`) and its - `contains(value, sample_rate)` (`:298-316`): `BooleanExact` accepts the bits of `0.0` or `1.0`; - `FiniteInclusive` is `[-144, 24]` dB for trim and fader and `[-1, 1]` for matrix and pan; - `DisabledOrRateKeyedHertz` accepts `0` ("off") or `[10, max(rate)]` Hz through - `validate_builtin_filter_cutoff` (`:336-352`) and `builtin_filter_cutoff_maximum_hz` (`:322-330`). -- **The static filter order rule.** `validate_input_filter_pair` - (`crates/builtins/src/filter_control.rs:36-50`) and the chain's preflight - (`crates/builtins/src/lib.rs:3290-3294`) refuse `hpf_hz > 0 && lpf_hz > 0 && hpf_hz >= lpf_hz` - as `BuiltinParameterError::FilterOrder`, reported as `builtin.filter.order` - (`crates/builtins-compiler/src/lib.rs:5012`) at the strip's `lpf_hz` (`filter_order_path`, - `:5062-5071`). Nothing compares curves. -- **Builtin diagnostic vocabulary.** `BuiltinDiagnostic { code: &'static str, path: String }` - (`crates/builtins-compiler/src/lib.rs:273-276`), built by `diag` (`:4992-4997`) with literal - dotted codes (`parameter_diagnostic`, `:5004-5030`). No document freezes the builtin codes as a - list; `docs/SESSION_SCHEMA_V1.md:85` names `builtin.gain.domain` where it states its rule. -- **The one builtin preparation.** Every public entry (`prepare_session_builtins`, `:3298`, and its - four siblings, `:3316-3391`) runs `prepare_session_builtins_with_live_controls_and_policy` - (`:3405`). Its domain preflight loops over the strips (`:3484-3494`) and returns every diagnostic - before it allocates (`:3515-3517`). host-core calls the entries for the browser and the C ABI - (`crates/host-core/src/prepare.rs:1503-1530`); `session-validator` runs `prepare_session_builtins` - as stage `prepare-builtins` (`tools/session-validator/src/lib.rs:25-27`, `:87-93`). -- **The C ABI automation-only edit skips preparation.** `classify_live_delta` masks the table - (`crates/host-core/src/live_delta.rs:234-236`), so `commit_live` commits such an edit with no - preparation (`crates/capi/src/runtime/control.rs:1065-1080`). *Refuse automation on effect - parameters that are not block-rate* (#1335 D4) has landed on `main` since `6ee64f484` (commit - `0c19119d0`) and added the route that closes this for its own rule: the classifier runs - `effect_automation_diagnostics` on `next` whenever the automation differs, and a non-empty result - returns `LiveRebuild::AutomationTarget`, so the commit takes the rebuild path whose preparation - refuses. The anchors in this draft stay those of `6ee64f484`. Note A10 removes the mask one row at - a time, from #1648 to #1667, so until #1667 some automation edits still skip - preparation. -- **Checked-in builtin automation** (`fixtures/session/v1/builtins-automation.json`, after #1626's - migration): fader `both` linear `db` 0 to -3, matrix `both` linear `linear` 1 to 0.5 on a matrix - strip, polarity `right` step 0 to 1, trim `left` linear `db` 0 to -6. Each fits D1. No checked-in - document automates a filter. - -## Decisions frozen for this slice - -- **D1. Unit, domain and shape per row.** - - One function in `builtins-compiler`, `builtin_automation_diagnostics(model: &SessionModel) -> - Vec`, reads each `rack = "builtins"` entry's descriptor by `parameter_id`. - Each rule is checked per segment `k`, at `$.automation[id=].segments[].` (the - #1335 D2 path form): - - **Unit.** The segment's `unit` equals `builtin_parameter_unit(row)` (session and - effect-contract units map one to one, as `same_unit` does, - `crates/effect-compiler/src/prepare.rs:1679-1689`). Else `builtin.automation.unit` at - `.unit`, and no other rule is checked for that segment. - - **Domain.** `row.domain.contains(value, model.sample_rate_hz)` for `start_value` and for - `end_value`. Else `builtin.automation.domain` at that field. - - **Shape.** A `BooleanExact` row (polarity, mute) takes only `step`. `exponential` is allowed - only on a row whose unit is not `db` and whose enabled values are strictly positive: of the - launch rows, only `hpf_hz` and `lpf_hz`. On a `DisabledOrRateKeyedHertz` row a `linear` or - `exponential` segment has neither value equal to the disabled value, so "off" is reached only - by a `step`. Else `builtin.automation.shape` at `.shape`. - - Targets the session already refuses (an unknown ID, `delay_samples`, a pan or matrix target on - the wrong strip kind) never reach preparation. -- **D2. Input filter order on every lane, at all times.** - - A second function, `input_filter_order_diagnostics(model: &SessionModel) -> - Vec`, runs per strip and lane (`left`, `right`) where at least one of HPF - (3) and LPF (4) is automated. A `both` entry drives both lanes. Each side's curve is its entry - under the hold rule (#1626 D4), or, when that side is not automated, the constant curve of - its static value. A lane with neither side automated is the static rule's. - - **Split.** The time axis `[0, ∞)` is cut at every `start_sample` and `end_sample` of either - curve. On each sub-interval `[a, b]` each side is one monotone piece: a hold (constant), a - `step` piece (constant), a `linear` piece or an `exponential` piece (the last sub-interval is - a hold on both sides). A side that is a constant `0` ("off") on the sub-interval leaves it - unchecked; D1 keeps every `linear` and `exponential` piece away from `0`. - - **Order.** On each sub-interval where both sides are enabled, with values evaluated at `a` and - at `b` by #1637's `automation::value_at(&segments, t)` over a `Segment` table built from - the entry's segments (D3 there, A1.4's law; a constant is its `f32` value): - - both constant: `H < L` in `f32`, the static rule; - - both `linear`, or both `exponential`: `L > H·(1 + 2^-22)` at `a` and at `b`; - - otherwise: `min(L(a), L(b)) > max(H(a), H(b))·(1 + 2^-22)`. - - **Why no rendered pair crosses.** Each piece is monotone, so its extremes lie at `a` and `b`. - Two `linear` pieces have a linear difference `L - H·(1 + 2^-22)`, and two `exponential` pieces - a monotone ratio `L/H`, so the order at both ends holds inside. For mixed pieces the extremes - rule is sufficient, not necessary: it refuses some pairs that never cross, by design. The - relative margin `2^-22` is four times the `f32` rounding bound `2^-24`: after render rounds - each value once to `f32`, `f32(L) > f32(H)` still holds at every sample, so no rendered pair - ties or crosses. Constants need no margin because render does not round them. - - **Refusal.** `builtin.filter.order`, the static rule's code, once per lane at the first failing - sub-interval, at `$.automation[id=].segments[]`: `` the LPF entry when LPF is - automated, else the HPF entry; `` the segment that defines that entry's value on the - sub-interval (the first segment for the leading hold, the held segment in a gap or after the - last segment). - - **For a later live edit.** The function reads only the model, so a classifier can run it on a - candidate model to refuse a static HPF or LPF edit that would cross an automated curve. Wiring - it into `classify_live_delta` is #1660's. -- **D3. One call site.** `prepare_session_builtins_with_live_controls_and_policy` appends both - functions' results before its check at `:3515`, after the per-strip preflight, so no host and no - validator stage changes its own code. Both functions allocate only off render, at preparation. -- **D4. Docs.** `docs/SESSION_SCHEMA_V1.md` (the automation section #1626 writes) names the three - `builtin.automation.*` codes with the rows they admit, and the curve form of the filter order - rule. The `author-session` skill's automation refusal table gains the four rows at stage - `prepare-builtins`. -- **D5. The live path routes a refusable edit to preparation.** #1335's route is an existing - route on `main`: `classify_live_delta` runs `effect_automation_diagnostics` on `next` whenever - `current.automation != next.automation`. This slice adds D1's and D2's functions to that same - check. A non-empty result returns #1335's `LiveRebuild::AutomationTarget`, so the - commit takes the rebuild path, and that path's preparation refuses with the D1 or D2 diagnostic. - Nothing is pushed or committed first. The classifier only routes; preparation words the refusal. - Once #1667 removes the last mask row, every automation edit is a rebuild and #1667 deletes - this route with #1335's. -- **D6. The acked-batch question** does not arise: both refusals happen before preparation - allocates or anything is committed. No queue changes. - -## Deliverables - -1. D1-D3 in `crates/builtins-compiler/src/lib.rs`: two functions, exported, and their one call. - D5 in `crates/host-core/src/live_delta.rs`: the two calls beside #1335's existing call. -2. D4 in `docs/SESSION_SCHEMA_V1.md` and `.claude/skills/author-session/SKILL.md`. -3. The tests below. - -## Authorized paths - -- `crates/builtins-compiler/src/lib.rs` (the two functions, their exports and their call only), - `crates/builtins-compiler/Cargo.toml` (the `automation` dependency only), `Cargo.lock`, - `crates/builtins-compiler/tests/builtin_automation_targets.rs` -- `scripts/check-builtins-policy.sh` (only `expected_compiler`, `:17`, which pins builtins-compiler's - exact dependency list, sorted by `gate_toml_dependencies`, `scripts/lib/gate.sh:162-196`: - `automation` joins it first) -- `crates/host-core/src/live_delta.rs` (D5 only; stream B owns it, root sequences the merge), - `crates/host-core/tests/live_delta.rs`, `crates/capi/src/runtime/live_tests.rs` (tests only) -- `tools/session-validator/tests/validate.rs` -- `docs/SESSION_SCHEMA_V1.md`, `.claude/skills/author-session/SKILL.md` (the automation section and - refusal table only) - -## Non-goals - -- Effect-target rules: #1566's row rule (`controls::automation_diagnostics`) and #1335. Session-level rules: #1626. -- The SDK builder's mirror (obsolete draft 03b, not filed). The classifier route for static filter edits (#1660). -- Rendering any automation. - -## Hazards - -- **The C ABI automation-only path.** Without D5, a C ABI automation-only edit could commit a - builtin segment its own rebuild refuses. Gate 7 holds it. -- **Hand-built test automation.** `crates/capi/src/runtime/live_tests.rs:1446-1467` and - `crates/host-core/tests/live_delta.rs:509-527` build a fader ride in `db` within the domain; they - stay valid. Any other hand-built builtin segment that now fails is a test defect to fix in its - value, never a reason to relax D1. -- **`-0.0`.** `BooleanExact` refuses the bits of `-0.0` (`contains`, `crates/builtins/src/lib.rs:303-305`); - a mute segment authored with `-0.0` is refused, as a static mute of `-0.0` already is. -- **Evaluation code.** D2 needs A1.4's value at two samples. It calls the evaluator crate of - #1637 (`automation::value_at`), which lands first, so one law exists from the start. The check - needs only ordering, and its `2^-22` margin dwarfs any `f64` evaluation error. - -## Objective gates - -1. **Rows** (`crates/builtins-compiler/tests/builtin_automation_targets.rs`, new tests). From the - migrated `fixtures/session/v1/builtins-automation.json`, `prepare_session_builtins` returns - exactly one diagnostic, with the stated code and path, for each of: a fader segment in `linear` - (`builtin.automation.unit`); a fader value of `24.001` (`builtin.automation.domain`); a matrix - value of `1.5`; a polarity value of `0.5`; a mute `linear` segment - (`builtin.automation.shape`); an `exponential` fader segment; an HPF `linear` segment from `0` - to `100`; an HPF value above the 44.1 kHz maximum in a 44.1 kHz session. It accepts the fixture - as written, an HPF `exponential` segment from `20` to `200`, and an HPF `step` from `0` to `80`. -2. **Filter order** (same file, new tests), static LPF 1,000 Hz unless stated: - - refused: an HPF `linear` ride from 200 to 1,000 Hz (it reaches the LPF at its end); an HPF - `step` from 100 to 1,200 Hz (refused on the hold after it); an HPF `exponential` from 100 to - 900 Hz against an LPF `linear` from 500 to 2,000 Hz over the same samples (both ends ordered; - the mixed rule refuses, by design); - - accepted: an HPF `linear` from 100 to 900 Hz; an HPF `linear` from 100 to 900 Hz against an - LPF `linear` from 500 to 2,000 Hz over the same samples (both linear, ordered at both ends); - an HPF `step` to `0` while an LPF ride passes below the HPF's former value; - - the function, run on a model whose automation is unchanged and whose static LPF moves from - 1,000 to 500 Hz under an HPF ride to 900 Hz, returns `builtin.filter.order`. -3. **Validator stage** (`tools/session-validator/tests/validate.rs`, new rows in `MUTATIONS`). A - fader segment in `linear` fails at stage index 3 (`prepare-builtins`) with - `builtin.automation.unit`, and stages 0-2 pass. -4. **Every checked-in document still prepares.** - `fixtures_distinguish_schema_examples_from_launch_effects` - (`tools/session-validator/tests/validate.rs:81`) and the skill test pass unchanged. -5. **No rendered bit moves.** No render code changes. `cargo build --locked --release -p audit -p - bench -p capi -p session-validator`, then `./target/release/audit capi`, shows the same - `pcm_digest` at base and head (PR evidence); the browser legs of the `browser` job in - `.github/workflows/qualification.yml` pass with unchanged digests. -6. **Commands:** - - `cargo test --locked -p builtins-compiler --features test-support`, - `cargo test --locked -p session-validator`, - `cargo test --locked -p host-core --features test-support --test live_delta`, - `cargo test --locked -p capi` - - the workspace debug leg (`test-debug-a` in `.github/workflows/qualification.yml`) - - `bash scripts/check-builtins-policy.sh`, `bash scripts/check-workspace-policy.sh` - - `bash scripts/check-cross-targets.sh` (README F19, the iOS memset rule: no new - `memset_pattern16` call; fix one in code, never by a ceiling) - - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, - `cargo fmt --all -- --check` - -7. **The live path** (`crates/host-core/tests/live_delta.rs` and - `crates/capi/src/runtime/live_tests.rs`, new). A delta that only adds a fader segment in - `linear` units gives `Err(LiveRebuild::AutomationTarget)`; a delta that adds a valid fader - segment stays as A10's rows say. On a playing C ABI engine, a transaction that only adds the - invalid segment is refused with the typed compile rejection; the revision does not advance, no - plan is replaced, and the next blocks are bit-identical to an engine that never received it. - -## Test value - -- Gate 1 turns red for each row rule left out, for a domain check that ignores the session rate - (the 44.1 kHz case), for a shape rule that refuses the filters' legal `exponential`, or for a - filter rule that refuses "off" by `step`. No test reads a builtin segment's unit or values today. -- Gate 2 turns red if the check samples only segment boundaries of one curve (the hold case), uses - the end-point rule on mixed pieces (the `exponential` against `linear` case), checks a lane whose - HPF is off, or reads the static value of an automated side; its last case turns red if the - function reads anything but the model it is given. -- Gate 3 turns red if D1 runs anywhere `session-validator` does not run. -- Gate 7 turns red if a C ABI automation-only edit can commit a builtin segment that preparation - refuses: the acked-but-unrenderable gap #1335 closes for effect rates. - -## Dependencies - -- #1626 *Validate stored automation lanes in the session crate and state the hold rule* (the - hold rule, the migrated fixture, the strip-kind rule). -- #1637 *Compile stored automation into per-cell events in node time*: `value_at`, the one - evaluation law D2 calls. -- *Refuse automation on effect parameters that are not block-rate* (#1335): D5 uses its - `LiveRebuild::AutomationTarget` route, which is on `main` at `0c19119d0`. -- Batch: R1. - -## Amendment (controls coordinator, 2026-10-14) - -Checked against `12d4a4f9d`. - -- **Most of D1 is done by #1566.** `controls::automation_diagnostics` refuses a segment whose unit - is not the row's unit or whose value the row refuses (`control.domain`, through `controls::check`, - which applies the rate-keyed filter bound), for every row: builtin, effect, route and VCA. The - `builtin.automation.unit` and `.domain` codes are not added. -- **What remains, in `controls`.** The shape rule (D1's third bullet) and the curve form of the - filter order rule (D2) move into `controls::automation_diagnostics`, the one row rule (#1566 D6), - not into `builtins-compiler`. The shape rule is one rule for every row, so it also takes obsolete - draft 03a's (not filed): a `Toggle` or `Enumeration` row takes only `step`; `exponential` only on a continuous row - whose minimum is strictly positive and whose unit is not dB; on a row with a `disabled_value` - (HPF, LPF) a `linear` or `exponential` segment never touches it. New codes of the control set: - `control.automation.shape` and `control.automation.order` (at the LPF entry's segment, as D2 - says). `controls` gains a normal dependency on #1637's `automation` crate for `value_at`. -- **D3 and D5 are done.** Preparation (`crates/host-core/src/prepare.rs:1890`), the classifier - (`crates/host-core/src/live_delta.rs:372-376`, `LiveRebuild::AutomationTarget`) and - `session-validator` (`tools/session-validator/src/lib.rs:396`) already call the row rule, so the - new checks reach every path with no new call site. -- **Authorized paths** become `crates/controls/**`, `crates/host-core/tests/live_delta.rs`, - `crates/capi/src/runtime/live_tests.rs` (tests), `tools/session-validator/tests/validate.rs`, the - docs, `Cargo.lock`. Not `builtins-compiler` and not `scripts/check-builtins-policy.sh`. - `crates/builtins-compiler/tests/builtin_automation_targets.rs` is deleted (#1566); gates 1-2 go to - `crates/controls/tests/automation.rs`. -- **Context anchors gone.** `validate_builtin_automation_target`, `BUILTIN_AUTOMATION_TARGETS`, - `effect_automation_diagnostics` and `crates/capi/src/runtime/control.rs` (now - `crates/control-plane/src/control.rs`). -- **Dependencies.** #1626, #1637, #1566 (in place of #1335's route). Obsolete draft 03a is not filed; this spec - carries its shape rule. Batch R1. - -## Decision record: #1645 (batch 14a) - -- **Shipped, per the amendment.** `controls::automation_diagnostics` (`crates/controls/src/automation.rs`) - gains the shape rule and the curve form of the input filter order rule; `ControlError` gains - `AutomationShape` (`control.automation.shape`) and `AutomationOrder` (`control.automation.order`). - `controls` gains a normal dependency on `automation` and calls `automation::value_at` for every - evaluated value. No new call site: preparation, the classifier's step 2 and `session-validator` - stage `prepare-effects` already call the row rule, so browser and C ABI refuse alike. -- **Shape rule, every row.** A segment is reported once at `.segments[]`: `control.domain` - first, else `control.automation.shape`. Toggle or enumeration row: `step` only. `exponential`: - continuous, `minimum > 0`, unit not dB. A row with a `disabled_value`: a `linear` or - `exponential` segment has neither value equal to it. -- **Order rule.** As D2, per strip and lane (a `both` entry drives both lanes). Departures, each - for the code's truth: (1) the value at a sub-interval's right end is the left limit, evaluated on - the defining segment alone (`value_at(&[segment], b)`), because `value_at` on the whole table at - a segment's `end` returns the next segment's `start_value` when one begins there; (2) the rule - skips a lane whose HPF or LPF entry has its own fault (target, domain or shape), so it never reads - a curve that ramps through "off", and a fault is reported once; (3) a `both` entry failing both - lanes at one segment is reported once (identical code and path). Stage index: the row rule runs - at stage index 4 (`prepare-effects`), not 3 as gate 3 says. -- **Gates.** - - 1 and 2: `crates/controls/tests/automation.rs` - `builtin_rows_refuse_a_segment_by_unit_domain_and_shape`, - `input_filter_curves_keep_the_hpf_below_the_lpf` (adds one case: the LPF ride against an HPF - that stays on, refused in the gap's hold at the LPF entry's segment 0). PASS. - - 3: `tools/session-validator/tests/validate.rs` two `MUTATIONS` rows (fader ride in `linear` - units -> `control.domain`, polarity flip as `linear` -> `control.automation.shape`, both stage - index 4). PASS; `fixtures_distinguish_schema_examples_from_launch_effects` and `skill` PASS. - - 7: `crates/host-core/tests/live_delta.rs` `an_automation_the_builtin_rows_refuse_needs_a_rebuild`; - `crates/capi/src/runtime/live_tests.rs` - `a_builtin_automation_the_rows_refuse_is_refused_before_any_ack` (shape and order refusals on a - playing engine: `RESULT_COMPILE_REJECTED`, preparation's diagnostic, no revision, candidate or - event, next three blocks bit-identical). PASS. - - 6: `cargo test` for controls, session-validator, host-core (`test-support,control-provider`; - the `live_delta` target is `cfg(feature = "control-provider")`), protocol, builtins-compiler, - capi: PASS. `check-workspace-policy.sh`, `check-builtins-policy.sh`, `check-cross-targets.sh` - (no new `memset_pattern16`; the #1018 expected failures unchanged), `realtime-policy`, clippy on - the touched crates with `-D warnings`, `cargo fmt --check`: PASS. - - 5: no render code changed; the `audit capi` digest comparison is left to the coordinator's PR - evidence. -- **Docs.** `docs/SESSION_SCHEMA_V1.md` names both codes, the shape rule and the curve order rule; - the `author-session` skill's automation refusal table gains three rows at `prepare-effects`. -- **Fold (batch 14a).** The gap the first pass found is closed here: a static HPF or LPF edit that - crosses an automated curve with the automation table unchanged classified live, so the committed - model could break the order rule. `controls` gains `filter_statics_changed_under_automation` - (a strip's static HPF or LPF bits differ and `next` automates either filter of that strip) and - `filter_under_automation` (an address is such a strip's HPF or LPF). The classifier's step 2 - (`crates/host-core/src/live_delta.rs`) also runs `automation_diagnostics` on `next` when the - first holds, and refuses with `LiveRebuild::AutomationTarget`, as the #1631 activity path does; - the fast path (`classify_rows`, used by the C ABI and the browser through `control-plane`) - answers `None` for a row the second names, so the full classifier judges it. Tests: - `a_static_filter_edit_that_crosses_an_automated_curve_needs_a_rebuild` (host-core `live_delta`; - red without the step-2 change: `Ok([("eq0", [], None)])` against `Err(AutomationTarget)`) and - `a_static_lpf_edit_that_crosses_an_automated_hpf_is_refused_before_any_ack` (capi; a - `SET_CONTROL` of the LPF to 500 Hz under an HPF ride to 900 Hz: red without the fix, and red - again with the step-2 change but without the fast-path fallback, both times `RESULT_OK` (0) - against `RESULT_COMPILE_REJECTED` (5); a move to 2 kHz commits live). #1660 keeps only the - classification of input-filter edits on an automated strip, not this refusal. - -**Review fold (batch 14a).** - -- **Root ruling (2026-10-15): the order rule has a reach.** The input-filter order rule is checked - on each sub-interval widened by the cell's reach, 64 samples. That is #1637's grid lookahead - (`crates/automation/src/events.rs`, `target_at`) and the filter's jump ramp. So stored automation - is never accepted and then refused in render. The reviewer's counterexample is now a refused - case: the HPF holds 900 Hz on [0,1000) and then steps to 100; the LPF is linear from 1000 to 300 - on [1000,1010). Before the widening, each piece was ordered, but the LPF target at sample 950 is - its value at 1014, 300 Hz, below the HPF's 900 Hz. `docs/SESSION_SCHEMA_V1.md` states the rule - with the reach. -- **#1660 must keep its reach at or below 64 samples**, or widen this rule in the same change. -- **The reach is 128 samples (coordinator, within the ruling).** Widening each curve by its - 64-sample reach means a rendered pair can read stored values up to 128 samples apart: render - leads a curve by up to 64 (grid lookahead) and lags it by up to 64 (jump ramp), on either filter. - `ORDER_REACH` is 128; with 64 the test `input_filter_curves_keep_their_order_within_renders_reach` - is red (an LPF fall 100 samples after the HPF's step is accepted). #1660 must keep each of its two - reaches at or below 64 samples, or widen this rule in the same change. -- **Path.** The fold edits `crates/host-core/src/live_delta.rs`. That file is outside the original - authorized paths. The coordinator approved the edit. diff --git a/.github/ISSUE_SPECS/1654-render-stored-send-gain-mute-and-matrix-automation.md b/.github/ISSUE_SPECS/1654-render-stored-send-gain-mute-and-matrix-automation.md index a93f30772..80a369dde 100644 --- a/.github/ISSUE_SPECS/1654-render-stored-send-gain-mute-and-matrix-automation.md +++ b/.github/ISSUE_SPECS/1654-render-stored-send-gain-mute-and-matrix-automation.md @@ -178,8 +178,7 @@ hand offset; a hand send mute ORs (D5 confirmed); on the send matrix automation stands: the edit stores the fallback and commits `model_only`. - **D5 is confirmed** by root: a hand send mute ORs with the automated send mute. D5's "a reading for root to confirm" and its alternative are void. -- **D7 is replaced.** `follows_mute` is not automatable (placeholder - `.github/ISSUE_SPECS/1632-make-send-follows-mute-and-matrix-mode-non-automatable.md`): the row +- **D7 is replaced.** `follows_mute` is not automatable (#1632): the row rule refuses an entry on route row 7 with `control.automation.target`, so this slice renders none and its flags word gains no automated `follows_mute` bit. - **Gate 3** becomes: a send automation edit gives path `rebuild` and completes `exact`; a diff --git a/.github/ISSUE_SPECS/1676-cap-the-observation-bindings-by-what-one-ended-event-holds.md b/.github/ISSUE_SPECS/1676-cap-the-observation-bindings-by-what-one-ended-event-holds.md deleted file mode 100644 index ca384cd2c..000000000 --- a/.github/ISSUE_SPECS/1676-cap-the-observation-bindings-by-what-one-ended-event-holds.md +++ /dev/null @@ -1,241 +0,0 @@ -# Cap the observation bindings by what one ended event holds - -Successor of #1584 D6a (batch 12a, root 2026-10-15), decision 17 -(`docs/rulings/one-control-abstraction-2026-10-13.md`). Code anchors verified on `45a2ab9df` -(PR #1675). Issue number: assigned at filing (written here as #1676). Feature sprint mode (owner, -2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -A subscription the engine acknowledges can always be ended honestly: no commit that removes a -subscribed target is refused because its `OBSERVATION_ENDED` event would not fit one control -frame, and the memory reserved for that event follows the bindings the session can hold, not the -frame size. - -## Context - -- #1584 D6a ends every binding whose target is gone with one `OBSERVATION_ENDED` per commit - (root ruling A). The event must fit `maximum_control_frame_bytes`; a commit whose ended keys do - not fit is refused as a resource limit (`crates/control-plane/tests/observation_bindings.rs`, - `an_end_event_over_one_control_frame_refuses_the_removal_and_changes_nothing`). So a valid - removal can be refused, and the app must disarm first. -- The batch 12a review fold charges each `OBSERVATION_ENDED` cell a flat ceiling bounded by the - frame size (`ended_event_ceiling`, `crates/control-plane/src/compile.rs:572`; - `observation_reserve_bytes`, `:553`). The reserve grows with `maximum_control_frame_bytes` - (about 4x at the browser's 16 KiB frame against the 4 KiB reference); `capi_retained_bytes`' - ceiling rose from 181,696 to 205,312 with it. - -## Decisions frozen for this slice - -- **D1. Cap at admission.** A subscribe (an Observe that would add a binding) is refused with a - typed refusal before any ack when the bindings table already holds as many keys as one ended - event can carry in one control frame (computed once at preparation from the frame limit and the - largest encoded key the session's IDs allow). Nothing is queued and no state moves. -- **D2. Reserve by capacity.** The ended-event reserve per cell is the smaller of the frame bound - and the encoded size of the bindings capacity (the slots the plan has), so a session with few - observation slots reserves little whatever the frame size. -- **D3. Same on both hosts**, with a parity row. - -## Objective gates - -1. With a small `maximum_control_frame_bytes`, the subscribe that would exceed the cap is refused - with the typed code, and a removal of every subscribed target afterwards commits with one event - holding every key (the over-frame refusal can no longer be reached by a valid removal). -2. `capi_retained_bytes` on the reference session falls back toward its pre-fold figure at the - 4 KiB frame, and the reserve no longer scales with the frame for a plan with few slots; the - ceiling is re-pinned to the measured value (plus 10 %, rounded up to 64) at both lane widths. -3. The charge-equals-retained capi tests stay exact; `cargo test --locked -p control-plane - --features test-support -p capi -p host-web --features host-web/test-support`; clippy. - -## Authorized paths - -- `crates/control-plane/src/{compile.rs,control.rs}`, `crates/controls/src/observation.rs`, - `crates/protocol` (the refusal code, if a new one is needed), their tests, - `crates/capi/tests/resource_lifecycle.rs` (the ceiling), docs, this spec's decision record. - -## Dependencies - -- #1584 D6a (PR #1675). - -## Size - -Half a day. - -## Decision record: #1676 (batch 13b) - -**What shipped.** - -- **D1, the cap.** `control_plane::compile::observation_binding_cap(frame_bytes)` measures, with the - real codec, the `OBSERVATION_ENDED` frame of one key and what a second adds, with both IDs at - `STABLE_ID_MAXIMUM_BYTES` (127), and returns how many keys one control frame holds (1 KiB frame: - 3, 4 KiB: 12, the browser's 16 KiB: 51). `SessionState` keeps it (`observation_cap()`), with an - exact `observation_armed()` count. Each observation slot now carries `armed` (the table's state - with every push folded in) beside its pending push, so a subscription is judged without a key - lookup or an allocation inside `process()`. A rebuild rebuilds the slots from the bindings it - keeps (`observation_slots(producers, bindings)`), so the count survives a swap and drops the - taps a transaction ended. `SessionState::admit_observation_changes(changes)` admits a batch by - the taps it leaves armed (arms of unarmed taps add, disarms of armed taps remove, a window change - adds nothing) and returns `ObservationLimit { index, cap }` for the first arm that crosses. It - moves nothing, so a refusal is whole and precedes every ack. -- **Browser mapping** (attempt 1; superseded by attempt 2 below: reason 15 `observationLimit`, - `RESULT_UNSUPPORTED`). `admit_commands` calls it after pass one and before the room pass; a - refusal is `RESULT_BACKPRESSURE` / `COMMAND_REASON_BACKPRESSURE` at the record that crossed it - (doc comments on the Rust constant and both `.d.ts` copies say so). Nothing else in the adapter. -- **D2, the reserve.** The ended-event heap per cell is now `min(slots, cap)` keys times - (`size_of::()` + two maximum IDs), not the frame size. The binding table and - its transient copies use the same `min(slots, cap)`. -- **Charge by high water.** The slots in that charge are the most any committed plan of the session - has had (`observation_slots_high_water`, passed to `prepare_runtime`), not only the plan being - prepared. A cell holds an event until its frame is delivered, so an event a larger earlier plan - emitted can outlive that plan; charging the shrunk plan by its own slots would let a cell hold - more than its row says. The first plan charges its own slots, so the reference figures are exact. -- **Docs.** `docs/CONTROL_PROTOCOL_SEMANTICS.md` (observation paragraph) states the cap and its - figures. - -**Departures.** - -1. **Refusal code** (attempt 1; superseded by attempt 2 below: reason 15 `observationLimit`, - `RESULT_UNSUPPORTED`). The browser's `COMMAND_REASON_*` vocabulary is gate-pinned as contiguous from - 0 across six files, and 12-14 are retired and never reused; a new reason would need a vocabulary - decision. The typed refusal is `ObservationLimit` in the core; the browser maps it to the - existing backpressure reason (unsubscribe a tap, then retry succeeds). No `crates/protocol` - change and no re-pin. A dedicated reason is successor work if the owner wants one. -2. **Parity row.** Only the browser subscribes today (the C ABI's `Observe` edit is #1584 D1-D5 and - `observation_taps` stays 0 there), so there is no C ABI path to put in - `hosts/host-web/tests/control_parity.rs`. Both hosts share the one core cap, the one reserve - and the one slot state; the C ABI edit must call `admit_observation_changes` (or its own edit - check against `observation_cap()`). (Since attempt 2 that function is private; the C ABI edit - calls `admit_and_push_observation_batch`, as #1584's batch 13b amendment says.) -3. (Attempt 1; superseded by attempt 2 below, which removes the uncapped public push.) - `push_effect_observe` does not itself refuse past the cap: a batch can push its arms before its - disarms, so the cap is judged on the batch, by `admit_observation_changes`, which every caller - must call first. The over-frame refusal in the commit stays as a defence; its existing test now - says it pushes past the cap on purpose. - -**Gates and tests.** - -- Gate 1: `control-plane/tests/observation_bindings.rs` - `a_subscription_past_the_cap_is_refused_and_the_removal_of_all_still_commits` (small frame; the - cap+1 arm is refused with index and cap, a disarm in the batch makes room, re-arming an armed tap - adds nothing, admission moves nothing, and removing the strip commits with one event holding all - `cap` keys), `the_cap_is_what_one_frame_carries_with_the_longest_ids` (cap longest keys fit the - frame, one more does not, at 1/4/16 KiB), `the_armed_count_follows_the_bindings_across_a_rebuild`; - browser: `host-web` `a_subscription_past_the_cap_is_refused_and_an_unsubscribe_makes_room`. -- Gate 2: `the_reserve_follows_the_slots_and_a_shrunk_plan_keeps_the_larger_reserve` (the reserve - at 4 KiB equals the reserve at 16 KiB for a few slots; a plan that shrank from 14 to 8 slots - charges at least the larger reserve). `capi_retained_bytes` on the reference session measured - 165,782 (x86-64, 4 KiB frame; the row does not depend on the lane width; it was 186,614 after the - 12a fold and 165,758 before it), so the ceiling is re-pinned to 165,782 plus 10 %, rounded up to - 64: 182,400, at both widths (was 205,312). The four-lane figure is not measured here (no AArch64). -- Gate 3: `capi` `resource_lifecycle` (charge equals retained, exact) and the full crate suites - pass; clippy `-D warnings` on control-plane, controls, capi and host-web, all targets and - features. - -**Successor candidates.** A dedicated browser command reason for the cap (needs a vocabulary -decision; attempt 1, superseded: attempt 2 adds reason 15 `observationLimit`). The C ABI `Observe` edit must admit against `observation_cap()`. 51 simultaneous taps at -the browser frame is the product limit; a larger frame raises it linearly. - -### Attempt 2 (batch 13b) - -**Root ruling (2026-10-15).** A cap refusal is never backpressure. The browser vocabulary gains -command reason 15, `observationLimit`; reasons 12-14 stay retired and are never reallocated, and the -declared values are exactly `range(0, max + 1)` minus `{12, 13, 14}`. This supersedes attempt 1's -departures 1 (the cap reported as `backpressure`) and 3 (an uncapped public push). - -- **The reason, in every spelling.** `COMMAND_REASON_OBSERVATION_LIMIT = 15` in - `hosts/host-web/src/lib.rs` (result `RESULT_UNSUPPORTED`, the same result as - `observationUnbound`: the session's configuration cannot take one more tap, and only an - unsubscribe changes that); the host JS table (`null` at 12-14; `validCommandReason` refuses a - `null` entry); the `.d.ts` `MisoCommandReason.ObservationLimit = 15` and its SDK mirror; the - metadata generator and the ABI-layout generator rows (`tools/parameter-metadata`); the schema - gate's own list; the ABI-layout gate's list (now explicit `(value, name)` pairs). The SDK assets - and `sdk/src/generated/{abi,catalog}.ts` were regenerated with `node codegen/assets.mjs` and - `node codegen/generate.mjs`. The C ABI header declares no command reasons, so nothing changes - there. No schema hash or ABI digest is pinned for these files, so nothing was re-pinned - (`abiVersion` stays `65536`: a new reason value is additive). -- **Gates for the gap.** `check-command-reason-vocabulary.py`, `check-parameter-metadata-v1.py` - and `check-abi-layout-v1.py` require "dense except the retired set". The vocabulary gate reads - the JS table's `null` holes (a hole at a non-retired value is red), takes the generator's values - from the Rust constants, maps the schema gate's hole-free list onto the non-retired values, and - requires the schema gate's `RETIRED_COMMAND_REASONS` to be its own. New red mutations: the JS - bound accepting a retired `null`; the JS table losing a hole or holing a live value; - `observationLimit` moved to 12 or to 16 in the Rust constants; dropped from the `.d.ts`, the - generator or the schema list; on 12 in the `.d.ts`; the schema retired set losing 14; plus eight - direct cases of the dense rule (31 mutations in all). The metadata gate's self-test gains four - (reallocated 12, gap at 16, missing 15, missing 11) and the ABI-layout gate's two. The on-disk - drift mutation in `test-web-audioworklet.sh` now adds a reason at 16. The hermetic worklet test - (`scripts/test-web-audioworklet.mjs`) settles a reason-15 acknowledgement as a typed refusal with - result 7 and a healthy host afterwards. -- **SDK.** `#submit` already resent only on `backpressure`; a comment now names - `observationLimit` among the reasons thrown at once, and `commandFailure` names the reason and the - record in the message, with a hint for the cap ("unsubscribe another tap first"). - -**Review findings.** - -1. *Short IDs let the removal half pass with no cap.* The core test - `a_subscription_past_the_cap_is_refused_and_the_removal_of_all_still_commits` now renames - `comp1` to a 127-byte strip ID and adds 127-byte insert IDs at the 1 KiB frame, so the `cap` - keys of the removal fill the frame. Its second half pushes `cap + 1` taps past admission (the - test-support hook `test_push_observe_unadmitted`) and requires the same removal to be refused - by the over-frame defence (`ResourceFault::Limit`). **Red without the fix:** raising the cap by - one (`observation_binding_cap(..)? + 1` in `compile.rs`) turns it red at the removal (`cap + 1` - longest keys are refused); run and observed red, then restored. -2. *`push_effect_observe` was public and uncapped.* It is now `pub(crate)`. The one public entry - is `SessionState::admit_and_push_observation_batch(changes)` (`ObservationChange` items): it - runs the cap check, then the queue-room check (moved from the browser into the core, since the - queues are the core's), then queues every change, and returns `ObservationRefusal::{Limit, - Full, NoProducer}` with nothing written on a refusal. A `debug_assert!` checks armed <= cap - after the push, in the core and again in the browser after its admission. The browser's - `admit_commands` calls it once after pass one; the room-refusal order is unchanged (cap, then - room, both before any write) and its own host-side state (armed masks, arm samples, in-flight - counts) is written only after the core admits. One departure in order: the browser's in-flight - counter overflow check (unreachable: the queue room bounds it far below `u32::MAX`) now runs - before the core call, because nothing may be refused after a push. No token was needed. Tests - that push without admission use the test-support hook only. -3. *A refusal at record index > 0.* The host-web test submits `[unsubscribe 0, subscribe cap, - subscribe cap + 1]` with the cap full and requires `observationLimit` at rejected index 2, with - nothing kept. The core test also requires `ObservationLimit { index: 1 }` for a batch whose - first change re-arms an armed tap. **Red without the fix:** mapping the crossing change to index - 0 in `lib.rs` turns the host-web test red (`rejected_index` 0, expected 2); run and observed. -4. *The host-web test expected backpressure.* `a_subscription_past_the_cap_is_refused_and_an_unsubscribe_makes_room` - now expects `(RESULT_UNSUPPORTED, COMMAND_REASON_OBSERVATION_LIMIT, 0, 0)`. **Red without the - fix:** mapping the cap back to `COMMAND_REASON_BACKPRESSURE` turns it red (6, 8 against 7, 15); - run and observed. -5. *The SDK must not resend the cap.* New eval in `sdk/test/observation-subscription-evals.mjs`: - a scripted host refuses one subscription with result 7, reason 15; the call rejects at once with - a `MisoEngineError` whose message names `observationLimit at record 0`, and exactly one command - was sent although a render happens 30 ms later. **Red without the fix:** letting `#submit` retry - on `observationLimit` turns it red; run and observed. - -**Docs.** `docs/CONTROL_PROTOCOL_SEMANTICS.md` (observation paragraph) now says the browser reports -the cap as `RESULT_UNSUPPORTED` with reason `observationLimit` and that the core admits and queues a -batch in one call. `hosts/host-web/MUTATIONS.md` lists the new mutations. - -**Review fold (batch 13b).** Three MINOR findings on attempt 2 (`fba4eeab8`, PASS). - -1. *The core only documented "each tap named once".* `admit_and_push_observation_batch` now - refuses a second naming of a (producer position, tap index) in one batch with the new - `ObservationRefusal::Duplicate { index }`, at the second naming and before anything is pushed; - the precondition sentence is gone from the doc. The browser maps it to `MALFORMED` at that - record, the answer its own pass one already gives a duplicate, so browser behaviour does not - change. Test: `a_batch_that_names_one_tap_twice_is_refused_at_the_second_naming` - (`crates/control-plane/tests/observation_bindings.rs`), at the 1 KiB frame with the cap full: - `[disarm A, arm A, arm B]` and `[arm B, arm B]` are each `Duplicate { index: 1 }`, move nothing - and queue nothing, and a later batch naming each tap once is admitted. Every shipped effect - declares one tap, so its queue has one slot and the cap+1 outcome itself is not reachable from a - test: the batch is reached through the public core call, and the room check masks the - undercount. **Red without the fix:** with the check disabled, the first batch is `Full - { index: 0 }` and the second `Limit { index: 0 }`; both run and observed red, then restored. -2. *The room check was O(n^2).* Admission is now three linear passes over the batch with no - allocation: pass 1 marks each change's slot and counts each producer's records, pass 2 is the - cap, pass 3 the room, reading one count per producer. The scratch is a per-slot `mark` in - `ObservationSlot` and a per-producer `ObservationBatchCount { mark, needed }` box built with the - plan's slots in `observation_slots` (and charged as a retained row beside the offsets); a batch - stamp makes stale marks invalid without clearing, and on the stamp's wrap every mark is reset - (bounded, allocation-free). Refusal order: **duplicate** (the first second-naming in batch - order, over the whole batch), then **cap** (the arm that crosses), then **no producer or full** - (one check, the first offending record in order). A duplicate therefore wins over a cap or room - refusal at an earlier record, because the cap count is only sound once duplicates are out. -3. *`hosts/host-web/MUTATIONS.md` was stale.* The vocabulary self-test row now says thirty-one - in-memory mutations (the script reports 31 plus 8 dense-rule cases), and the metadata self-test - row says reason 11 is renumbered to 16, and lists #1676's four new mutations. diff --git a/.github/ISSUE_SPECS/1677-address-observation-records-by-stable-key-in-the-core.md b/.github/ISSUE_SPECS/1677-address-observation-records-by-stable-key-in-the-core.md deleted file mode 100644 index e116d4a3d..000000000 --- a/.github/ISSUE_SPECS/1677-address-observation-records-by-stable-key-in-the-core.md +++ /dev/null @@ -1,117 +0,0 @@ -# Address observation records by stable key in the core - -Successor of the batch 12b review (M1), decision 17. Code anchors verified on the batch 12b branch `codex/controls-batch-12b`. Issue number: assigned at filing (written here as #1677). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -An observation subscribe or unsubscribe names its tap by stable key (strip, section, slot or -insert ID, tap ID), the same key `OBSERVATION_ENDED` uses, and the core resolves it on the plan it -admits against. No client keeps an index space, so a structural edit can never make a record arm -another effect's tap. - -## Context - -- Today a browser observation record (kinds 7 and 8) carries a strip index and an effect index. - The SDK takes the strip index from the newest control list and the effect index from - `observationMap()`, and re-addresses records after a rebuild (`#remap`, `OBSERVED_STRIP` in - `sdk/src/core/observation-subscriptions.ts`); the batch 12b fold makes it wait when the map's - structure epoch differs from the list's. -- The core admits a record against the newest plan (`admit_on_newest_plan`). -- #1589 (keep only handles and callbacks in the SDK) is the related SDK slice. - -## Decisions to take (the implementer proposes, the coordinator rules) - -- The record shape: a stable key in place of the two indices, on both hosts, with the parity row. -- The core resolves the key on the plan it admits against and refuses an unknown key with a typed - code before any ack. -- The SDK's re-addressing and its epoch wait are deleted. - -## Objective gates - -1. A subscribe sent between a rebuild's commit and its adoption arms the named tap, never another. -2. An unknown key is refused with the typed code and nothing changes. -3. SDK headless and browser evals; host-web and control-plane tests; parity row. - -## Dependencies - -- #1584 D6a/D6b; #1589. - -## Size - -One day. - -## Decision record: observation records addressed by stable key (batch 13b, lane W) - -**What shipped.** A browser observation record (kinds 7 and 8) names its tap by stable key, the -same key `OBSERVATION_ENDED` uses: the strip ID, the rack (`1` inserts, `3` console), the console -slot's or insert's ID and the effect-local tap ID. It carries no index of any plan. - -- **Record.** 272 bytes, little endian: `kind` u8 @0, `rack` u8 @1, `stripIdBytes` u8 @2, - `effectIdBytes` u8 @3, `tapId` u32 @4, `windowBlocks` u32 @8, a required-zero u32 @12, `stripId` - 128 bytes @16, `effectId` 128 bytes @144, each ID zero padded (an ID is 1 to 127 bytes, so the - last byte of a field is always zero). `COMMAND_RECORD_BYTES` 48 to 272, new constant - `RECORD_ID_FIELD_BYTES` 128; `ABI_VERSION` did not move (prelaunch, V1). The command staging grows - from 12,288 to 69,632 bytes, allocated at boot as before (`MAXIMUM_COMMAND_RECORDS` is 256). -- **Core.** One function in control-plane resolves the key: - `SessionState::resolve_observation_key(&ObservationKeyRef { strip_id, rack, effect_id, tap_id })` - returns `ResolvedObservationTap { producer, tap_index }` or an `ObservationKeyError` - (`UnknownStrip`, `UnknownEffect`, `UnknownTap`, `ComputedTap`), writing nothing. It resolves on the - newest committed plan: the strip in the committed model (so a strip with no such effect is an - unknown effect, not an unknown strip), the effect among `effect_producers()` (the pending - candidate's while a rebuild waits for its block), the tap in the descriptor's menu - (`resolve_observation_tap`, also exported). Byte compares, no allocation, so it runs inside - `process()`'s message handler; the C ABI's future observe edit (#1584 D1-D5) calls the same - function. host-web only decodes the staged bytes into the borrowed key (a rack code outside 1 and - 3 is `unknownRack` there) and maps the refusal to the wire's reason (`unknownTrack`, - `unknownEffect`, `unknownTap`, `unsupportedKind` for a computed tap), all in pass one before the - room pass and any write, so nothing is queued and no ack precedes a drop. It then reaches its - per-slot tables from the producer position through the new `PlanShape::producer_slots` (the - inverse of `EffectInstance::producer`, built with it) and refuses `observationUnbound` for an - effect with no reader. A malformed record (a nonzero reserved word, a length that fills its field, - a byte after the end of an ID) is `malformed`. The codes keep their numbers and names. The - selected read still resolves its IDs on the rendering plan through - `ReadyOwnership::effect_by_id`, which is now only the read's. -- **Edits to control-plane.** `control.rs` (the function, its key and error types and - `resolve_observation_tap`) and its re-exports in `lib.rs`; no edit to `prepare.rs`, host-core or - capi. -- **Worklet.** `command()` takes `{ kind, rack, stripId, effectId, tapId, windowBlocks }`; - `observe()` takes `{ stripId, rack, effectId, tapId, windowBlocks, armed }` and its binding map is - `{ stripId, rack, effectId, tapId, windowBlocks }`, sorted by those. `frameSlot`, `trackIndex` and - `effectIndex` leave the subscription map (a strip's gain-reduction slot is its position in the - control list's strip order). An ID over 127 UTF-8 bytes or a non-string rejects locally. The - worklet's `receiveCommand` is unchanged apart from the record size; `process()` is untouched. -- **SDK (thin client).** `LaneEdit` is `{ kind, rack, stripId, effectId, tapId, windowBlocks }`; - `observationEdit()` builds it from a key; `effect(...).observe()` needs a stable ID - (`console(slot, ...)`, `insert(id, ...)`, or an index resolved by `withSession`). The managed - owner builds each record from the selection and sends it. Deleted: `#remap`, `OBSERVED_STRIP`, the - strip-index re-addressing in `submitObservation`, `#settle`'s epoch wait and - `MAXIMUM_STRUCTURE_WAITS`, `submitManagedAt`, `structureBehindError` and - `sdk.observation.structure_behind`, `syncStructure`, `ObservationMap.structureEpoch`, the - `stripOrder` of `LiveControlTransport`, `LiveControlEdits.update`, and the strip and effect index - plumbing of the generated builders (`StripAddressing`, `EffectInstance`). -- **C ABI.** It does not admit observation records (no subscribe export), so there is no parity row. - -**Gates and tests.** - -1. A subscribe between a rebuild's commit and its adoption arms the named tap, never another: - control-plane `a_stable_key_resolves_on_the_newest_plan_and_an_unknown_key_is_refused` (the key - follows the effect when a committed removal moves its producer, and an unknown strip, insert, - rack, tap or reserved tap zero is refused with its own error); host-web `structural_apply::the_catalogs_describe_the_successor_from_the_adoption_block_on` (the - record for `track`'s compressor, which is index 1 in the successor, arms exactly it and not - `a-track`'s) and `a_tap_command_admitted_at_sample_zero_survives_the_hand_over`; SDK - `structure-evals.mjs` "#1677 -- an observation record names its tap by stable key" (subscribe, - close and update in the window, headless and through the browser transport over the real core, - on `swap_t3_multi_and_limiter` where two effects with the same tap swap indices); - `measurement-evals.mjs` (the browser engine sends the key with a stale map, without a wait). -2. An unknown key is refused with its typed code and nothing changes: control-plane (above); host-web - `observation_misuse_is_typed_and_all_or_nothing` (malformed forms, unknown rack, effect, strip, - tap), the in-window refusals above, and `structure-evals.mjs` (a valid record in front of an - unknown strip, effect, tap or rack is not armed either; a tap the replacement no longer has is - refused against the newest plan). -3. SDK headless and browser evals (554 pass), `test-web-audioworklet.mjs`, `direct-oracle.mjs`, - the writer evals for the record encoding (`writer-evals.mjs`), the layout checks - (`check-abi-layout-v1.py` and its self-test, `check-parameter-metadata-v1.py`, - `check-command-kind-vocabulary.py`, `check-command-reason-vocabulary.py`). - -**Departures.** The test fake of `test-web-audioworklet.mjs` now has three pages of memory (the -command staging no longer fits one). The `observe()` map lost `frameSlot` (above). diff --git a/.github/ISSUE_SPECS/1678-refuse-an-effect-row-edit-whose-effect-changed-in-the-core.md b/.github/ISSUE_SPECS/1678-refuse-an-effect-row-edit-whose-effect-changed-in-the-core.md deleted file mode 100644 index c022b11fc..000000000 --- a/.github/ISSUE_SPECS/1678-refuse-an-effect-row-edit-whose-effect-changed-in-the-core.md +++ /dev/null @@ -1,58 +0,0 @@ -# Refuse an effect-row edit whose effect changed, in the core - -Successor of the batch 12b review (m4), decision 17. Code anchors verified on the batch 12b branch `codex/controls-batch-12b`. Issue number: assigned at filing (written here as #1678). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -A `SET_CONTROL` edit on an effect section's row names the native effect it was written for, and -the core refuses it with a typed code when that slot or insert now runs another effect. Every -client (SDK, C ABI app) gets the same protection, and the SDK stops probing the control list -before each named-row edit. - -## Context - -- #1562b-K resolves named effect rows from the control list's section headers and checks each - effect-row edit just before encoding (`effectRowResolver`, `sdk/src/core/live-controls.ts:122`), - at the cost of a list probe per transaction; C ABI clients have no such check, so after a rebuild - a numeric effect row can land on another effect's parameter. - -## Decisions frozen for this slice - -- **D1.** An effect-section `SET_CONTROL` address may carry the expected native effect ID; the - core refuses a mismatch with a typed code before commit, nothing changes. -- **D2.** The SDK sends the expected ID from its builders and deletes the resolver hook. - -## Objective gates - -1. After a rebuild that changes an insert's effect, an edit written for the old effect is refused - with the typed code on both hosts (parity row); the same edit without the expected ID keeps - today's behaviour. -2. Wire re-pins with the repository tooling; SDK evals; protocol, control-plane, capi, host-web - tests. - -## Dependencies - -- #1562b-K. - -## Size - -Half a day to a day. - -## Decision record: #1678 (batch 13b) - -**What shipped.** - -- Wire: `SET_CONTROL`'s address is its own message, `SetControlAddress` (`schema::set_control_address`): the list's six address fields and an optional `7:expected-effect UTF8`. The control list's address message and a stored automation target keep `ControlAddress` (the session JSON keys pair with it, so field 7 would break that pairing). Encoder and decoder: `write_set_control_edit_address` / `decode_set_control_edit_address` in `message_wire`; the field is refused as `INVALID_FIELD` unless the section is `inserts` or `console` and the value is a stable ID. -- Model: `SessionEdit::SetControl` gains `expected_effect: Option`; `SessionEdit::set_control` keeps its signature (no expected effect) and `with_expected_effect` adds one. -- Core check: `controls::check_expected_effect` (in `effect_rows.rs`) returns the new `ControlError::EffectChanged`, code `control.effect.changed`. It runs in `apply_session_edit` (full path, against the candidate at the edit's place in the transaction, so a transaction that replaces the effect first is judged by the new one) and in `prepare_in_place` against the committed model; the in-place path declines on a mismatch and the full path answers with the typed refusal. An address that names no insert or slot is still `control.address.unknown`. Both hosts reach this through `SessionStore`, so there is no host code. -- SDK: the builders send the expected effect (typed effect of an effect builder, the effect of the section for a named row, `expectedEffect` of a numeric `ControlWireAddress`); `effectRowResolver`, its provenance map and the control queue's `prepare` hook (its only user) are deleted, so a named-row edit no longer probes the control list. A named row follows no header any more: an edit built from the old view is sent for the old effect and the core refuses it. -- Re-pins: `COMPLETE_SCHEMA_HASH` `0xcc9a_7bcc_502c_4936` to `0x63ff_2e60_976e_0e21` (the corpus transaction appends a `SetControl` on an insert row with expected effect `miso.parametric-eq`: 36 edits; the controller test's edit-limit boundary is now 35); parity self-test strings, `fuzz/corpus/complete-schema-manifest.md`, `docs/CONTROL_PROTOCOL_CONFORMANCE.md` history lines; `crates/protocol/tests/fixtures/set-control-frames.json` blessed (new `expectedEffect` key and an `expected_effect` command). `docs/CONTROL_PROTOCOL_REGISTRY.md` documents field 7 and the code. No C ABI vector carries the field. No rendered bit moved (audit capi `pcm_digest` `bf7dcedd1d92621f`). -- Integrated in batch 13b: one re-pin from `0xcc9a_7bcc_502c_4936` to `0x0c71_96f8_6abe_6d57` for #1635 and #1678 together. - -**Departure.** The spec says the codec "gains an optional field"; the field is on a new message that shares fields 1 to 6 with the list's address, not on the shared message (reason above). - -**Gates -> tests.** - -1. `hosts/host-web/tests/control_parity.rs::a_stale_effect_row_edit_is_refused_alike_on_both_hosts` (rebuild replaces `t2`'s transient shaper by a soft clip; the stale edit gets one verdict, one reply, `control.effect.changed`, unchanged control rows and document, equal blocks; the same edit without the expected effect, and with the new one, commit live on both hosts, at four rates). Store level, both preparation paths: `crates/protocol/tests/expected_effect.rs` (red with the check removed). Wire: `session_wire::tests::set_control_expected_effect_is_held_to_its_section`, `crates/protocol/tests/set_control_frames.rs` fixture, `sdk/test/control-frame-evals.mjs`. -2. `sdk/test/structure-evals.mjs` ("#1678", headless and browser transport), `sdk/test/control-builders-evals.mjs` (every effect row carries `expectedEffect`). -3. Run: protocol (`test-support`), control-plane (`test-support`), controls, capi, conformance, host-core, host-web (`host-web/test-support`), parameter-metadata; clippy `-D warnings` on host-core (no features, default, all features) and the touched crates; fmt; `check-protocol-wasm-parity.sh` and `--self-test`; command kind and reason vocabulary checks; fuzz `cargo check`; protocol allocation audit; SDK types, generated, headless (557 evals), package; realtime-policy; bench-policy; chromium qualification. diff --git a/.github/ISSUE_SPECS/1679-expose-the-armed-tap-mask-from-host-core.md b/.github/ISSUE_SPECS/1679-expose-the-armed-tap-mask-from-host-core.md deleted file mode 100644 index 6cad70b0a..000000000 --- a/.github/ISSUE_SPECS/1679-expose-the-armed-tap-mask-from-host-core.md +++ /dev/null @@ -1,39 +0,0 @@ -# Expose the armed tap mask from host-core - -Successor of the batch 12b review (m5), decision 17. Code anchors verified on the batch 12b branch `codex/controls-batch-12b`. Issue number: assigned at filing (written here as #1679). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -The browser copies a successor's armed taps from host-core instead of re-deriving host-core's -match, so one function decides which tap a binding arms. - -## Context - -- #1584 D6b's `build_plan_shape` (`hosts/host-web/src/lib.rs`, near `:5768-5790`) re-derives the - match host-core makes between bindings and taps (`crates/host-core/src/prepare.rs`, near - `:2271-2283`, `ObservationArm`). - -## Decisions frozen for this slice - -- **D1.** host-core's `EffectObservationHandle` exposes the armed tap mask it prepared; the - browser's table copies it; the duplicate match is deleted. - -## Objective gates - -1. The #1584 D6b browser tests pass unchanged; a test shows the browser's arm bits equal host-core's - mask for a session with several bindings, including an insert ID reused for another effect. -2. host-core and host-web tests; clippy. - -## Dependencies - -- #1584 D6b. - -## Size - -A few hours. - -## Decision record: expose the armed tap mask (batch 13a) - -- `EffectObservationHandle` is defined in `crates/effect-compiler/src/prepare.rs` (host-core re-exports it), so the field `armed_taps: u32` (bit `i` = `descriptor.observations[i]`) was added there. `attach_effect_observation` sets a bit exactly when `ObservationLane::arm_prepared` armed that tap, so the mask is the lane's own arming, not a second match. -- host-web's `build_plan_shape` copies `handle.armed_taps` into `observation_armed`; the duplicate binding match and its `bindings` parameter (and the caller's lookup) are deleted. An arm for a tap past bit 31 has no bit, as before. -- Gate: `tests::structural_apply::the_browsers_armed_bits_are_the_mask_host_core_prepared` (a console slot `comp` and insert `comp` on two tracks, one insert and one console tap armed, restart rebuild; every slot's word equals the handle's mask; 4 observed, 2 armed). It goes red when the copy is replaced by 0. The #1584 D6b browser tests pass unchanged. diff --git a/.github/ISSUE_SPECS/1681-refuse-a-source-with-more-than-two-channels.md b/.github/ISSUE_SPECS/1681-refuse-a-source-with-more-than-two-channels.md deleted file mode 100644 index 8d05064fb..000000000 --- a/.github/ISSUE_SPECS/1681-refuse-a-source-with-more-than-two-channels.md +++ /dev/null @@ -1,116 +0,0 @@ -# Refuse a source with more than two channels - -Owner answer (2026-10-15): sources are mono or stereo only. Decision 17 programme. Code anchors verified on `4be10cd67` (PR #1680). Issue number: assigned at filing (written here as #1681). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -A session whose source declares more than 2 channels is refused at session load with a typed -error, the same on both hosts, and the browser's source staging is sized for 2 channels (about -1 KB instead of the 130 KB the grammar's 255-channel domain needs). - -## Context - -- `session::SOURCE_CHANNELS_MAXIMUM` is `u8::MAX` (`crates/session/src/model.rs:217`), the - domain of `Source::channels`; nothing narrower is validated. -- #1562b's review fold (B1) sized the browser's source PCM and plane staging by that maximum - (`hosts/host-web/src/lib.rs`, `project_buffers`), so a rebuild can add any valid source. -- Product rule (dual mono by source file): only mono source files get dual-mono processing; stereo - files stay stereo. No source has more than 2 channels. - -## Decisions frozen for this slice - -- **D1.** One rule in the session validator: `channels` is 1 or 2; any other value is refused with a - typed code (for example `session.source.channels`), at load and in a structural edit's - validation, so both hosts refuse it the same way. -- **D2.** A launch constant (for example `SOURCE_CHANNELS_LAUNCH_MAXIMUM = 2`) in the session crate - is the one source of the limit; the browser staging, the C ABI's source checks and the SDK - builder read it (the SDK through its generated constants). -- **D3.** The browser's staging is sized by D2. - -## Objective gates - -1. A session with a 3-channel source is refused at load on both hosts with the typed code; a - structural edit that adds one is refused before commit; a parity row. -2. Mono and stereo sources load, feed and render as today (existing tests green). -3. Browser `sourcePcmStagingBytes` falls back to the 2-channel size; `expected.json` and - `retained_ceilings.rs` re-pinned from measurements; the boot budget passes. -4. `cargo test --locked -p session -p session-validator -p capi -p host-web --features - host-web/test-support`; SDK checks; the author-session skill and `SESSION_SCHEMA_V1.md` state - the limit. - -## Authorized paths - -- `crates/session/src/{model.rs,validate.rs,diagnostic.rs}`, `crates/session/tests/`, - `hosts/host-web/src/lib.rs` (the staging size), host-web tests and `expected.json` rows, - `crates/capi` tests, `sdk/` generated constants and evals, `docs/SESSION_SCHEMA_V1.md`, - `.claude/skills/author-session/SKILL.md`, this spec's record. - -## Dependencies - -- #1562 (B1 staging). - -## Size - -Half a day. - -## Decision record: refuse a source with more than two channels (batch 8) - -**Shipped.** - -- D1. One rule in `session::validate`: `Source::channels` is 1 or 2. Zero and more than two are one - rule with one new typed code, `DiagnosticCode::SourceChannelsUnsupported` - (`source.channels.unsupported`, path `$.sources[i].channels`). It replaces the old - `capacity.zero` for `channels == 0` (frames keep `capacity.zero`). Every load and every - structural edit already validates the candidate model through this function, so both hosts - refuse alike. -- D2. `session::SOURCE_CHANNELS_LAUNCH_MAXIMUM = 2` replaces `SOURCE_CHANNELS_MAXIMUM` (255, - deleted: nothing else read it). Readers: the validator, the browser's staging - (`hosts/host-web/src/lib.rs` `project_buffers`), the C ABI's source submission - (`crates/capi/src/ffi.rs`: the plane-count bound and the stack plane array, 255 to 2, through - `host_core::SOURCE_CHANNELS_LAUNCH_MAXIMUM`, a re-export beside `LAUNCH_SAMPLE_RATES`) and the - SDK (new ABI-layout constant `maximumSourceChannels`, written by `parameter-metadata`, read by - the session builder). -- D3. The browser's source PCM and plane staging are sized by the constant: `sourcePcmStagingBytes` - 130,560 to 1,024. The ID staging keeps its 127 bytes (#1562b B1). - -**Departure.** `HostPrepareCaps::maximum_source_channels` stays `None` everywhere: the validator is -the one rule, a second per-host cap would be a second place to drift. - -**C ABI.** A chunk with three or more planes is now `RESULT_INVALID_ARGUMENT` at the boundary -(it was `source.channels.mismatch` from the facade); the row "three planes for a stereo source" in -`source_rejections_reach_the_c_host_as_their_own_diagnostic` became "one plane for a stereo source" -and a separate assertion covers three planes. - -**Fixtures and corpora.** No fixture under `fixtures/` declares a source of more than two channels. -Tests that did: `host-web` `three_source_session` (alpha, 4 channels, now 1) and -`plan_exchange_threads` (a six-channel source, now a stereo one; renamed -`a_rebuild_adds_the_longest_id_and_feeds_the_new_source`; #1562's record still names the old test -and is history). `hosts/host-web/tests/stem-pump-v1.mjs` pumps a 3-channel PCM file through the -stem pump: it is the pump's own channel-generic test and builds no session, so it stays. - -**Gates.** - -1. `control_parity::both_hosts_refuse_a_source_of_more_than_two_channels_alike`: a three-channel - source is refused at load by the C ABI compile and the browser boot with identical diagnostics - containing `source.channels.unsupported`; mono and stereo load; a structural `UpsertSource` of - three channels is refused on both with identical reply bytes, the code - `source.channels.unsupported` and the revision unmoved (four rates). Session crate: - `diagnostic_parity` (`source-channels-zero`, `source-channels-three`) and `invalid_matrix` - (`source_identity_and_shape_category_has_22_distinct_cases`). -2. Mono and stereo unchanged: the existing host-web, capi, control-plane and conformance suites. -3. Browser `expected.json`: `sourcePcmStagingBytes` 1,024, `bridgeRetainedBytes` ceiling 1,812,352 - to 1,667,648 (measured 1,611,330 on the simd128 module; 562,754 beyond the capture, plus 10 %, - rounded up to 64), the three `memoryBytes` 3,014,656 to 2,883,584. `retained_ceilings.rs` - `bridge_retained_bytes` 787,968 to 641,024 (measured 1,631,294 x86-64, 582,718 beyond the - capture). Boot high-water growth (`check-web-boot-budget.mjs`): 20,447,232 bytes (before: - 20,447,232). -4. SDK: `builder-evals` "a source of other than one or two channels is refused at its own path"; - `sdk/src/generated/abi.ts` and `sdk/assets/miso-engine-v1-abi-layout.json` regenerated; - `scripts/check-abi-layout-v1.py` and its self-test fixture know `maximumSourceChannels`. - `docs/SESSION_SCHEMA_V1.md` and the author-session skill state the limit. - -**Found in review of callers.** `crates/host-core/src/limiter_linked_session.rs` (a `#[cfg(test)]` -unit test, the 64-track linked-limiter corpus) widened its one source to 128 channels. It now -declares 64 stereo sources (`stream-NN`), track `k` reads source `k`'s two channels and the same -noise streams `2k` and `2k + 1` feed them; `maximum_sources` 16 to 64. The `PIN` digest is -unchanged and the three width tests pass, so no rendered bit moved. diff --git a/.github/ISSUE_SPECS/1682-boot-an-empty-session-on-both-hosts.md b/.github/ISSUE_SPECS/1682-boot-an-empty-session-on-both-hosts.md deleted file mode 100644 index ac3d1624d..000000000 --- a/.github/ISSUE_SPECS/1682-boot-an-empty-session-on-both-hosts.md +++ /dev/null @@ -1,136 +0,0 @@ -# Boot an empty session on both hosts - -Owner answer (2026-10-15): an app opens an empty mixer and adds stems through structural edits. -Decision 17 programme. Code anchors verified on `4be10cd67` (PR #1680). Issue number: assigned at filing (written here as #1682). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -A session with no source and no track boots on both hosts (it renders silence at the output), and -a structural edit can then add a stereo stem with its track; both hosts render it identically. - -## Context - -- A session with no source or no track fails to boot today: host-core's source graph preparation - refuses it (`graph_failure("host.source.graph.prepare")`, `crates/host-core/src/prepare.rs:2570`), - at least on the browser; #1562b's gates had to boot with one mono source. -- Since #1562b both hosts accept rebuilds through the one core path, so the first stem can be added - by a structural transaction. - -## Decisions frozen for this slice - -- **D1.** An empty session (no source, no track; submixes, VCAs and console slots may be declared) - is valid and prepares a plan that renders silence at the output, on both hosts, with no special - path in a host adapter. -- **D2.** Every boot-time size that depends on the session (strip counts, meter frames, staging) - handles zero, and grows by the existing successor paths when the first strip arrives. - -## Objective gates - -1. Boot empty on both hosts: the plan renders silence, status, meters (an empty frame) and the - control list are consistent; zero render allocations. -2. **Parity row:** boot empty, add a stereo stem and a track by one structural transaction, feed the - stem and hear it, bit-identical on the browser and the C ABI at the four launch rates. -3. The browser qualification gains an empty-boot leg (chromium, firefox, webkit). -4. `cargo test --locked -p host-core --all-features -p control-plane --features test-support -p - capi -p host-web --features host-web/test-support`; boot budget. - -## Authorized paths - -- `crates/host-core/src/prepare.rs` (and `shape.rs`), `crates/graph-compiler` if an empty graph - needs it, `crates/control-plane`, `crates/capi` tests, `hosts/host-web/**` (empty-boot sizes), - the worklet JS and SDK only where an empty shape is refused today, `hosts/host-web/qualification`, - this spec's record. - -## Dependencies - -- #1562 (rebuilds on the browser). - -## Size - -Half a day to a day. - -## Decision record: core and C ABI part (batch 8) - -What shipped: - -- The one refusal on the path was `prepare_graph_source_set` (`crates/source/src/lib.rs`), which - returned `SourceGraphSourceSetError::EmptySources` for a session with no source; the variant and - its check are deleted. The session validator, the graph compiler, the shape report, the meter - and control-list builders, the plan swap and the C ABI staging already handled zero strips: the - empty boot needed no other change, and no host adapter has a special path (D1). Every - boot-time size that depends on the strip count was exercised at zero by the gates below and - needed no fix (D2); the successor path grows from empty to one strip and shrinks back. -- From #1681: `HostPrepareCaps::maximum_source_channels` and its `host.source.channels` check are - deleted (the validator rule is the one limit), with its test `source_channel_cap_is_optional`, - its `MUTATIONS.md` row M-06 and the stale doc line; callers dropped the field. - -Gates: - -- `crates/host-core/tests/empty_boot.rs`: `an_empty_session_prepares_and_renders_silence` (the - console-slot-only mixer and `canonical-minimal.json`), `declared_submixes_and_vcas_do_not_make_an_empty_session_noisy` - (an unfed bus and a VCA over it). Each renders four blocks of `+0.0` with zero render - allocations. -- `crates/capi/tests/empty_boot.rs` (its own binary, `bench_support`'s allocator): - `an_empty_session_boots_and_renders_silence_without_allocating` at 44.1, 48, 88.2 and 96 kHz: - resource row (0 sources, 0 tracks), boot watermark, silence with zero render allocations, control - list (the session header only, no strip), counters, snapshot, and a render-peak meter that reads - `0.0`; `a_structural_transaction_adds_the_first_stem_to_an_empty_session` at the same rates: one - `SESSION_TRANSACTION_APPLY` (source, track, route) takes the rebuild path, the swap block - allocates nothing, the structure epoch and the control list grow, the fed stem is heard, and a - second transaction removes it again (silence, counts back to zero). The browser parity row - mirrors this file. - -Departures: `crates/capi/tests/empty_boot.rs` joins the realtime policy's unsafe allowlist -(`tools/realtime-policy/src/policy.rs`, `docs/REALTIME_DEPENDENCY_POLICY.md`) as every capi test -binary that calls the C entries does. Otherwise none. Browser part (empty-boot sizes, parity row, qualification leg) stays with lane W; -the stale remark "A session with no source or track does not prepare" in -`hosts/host-web/tests/control_parity.rs` (`mono_model`) is theirs to drop. - -## Decision record: browser part (batch 13, lane W) - -What shipped: - -- **Two browser boot-time sizes were fixed at zero** (D2); everything else the spec lists (strip - order, meter frame and header, source map, observation tables, staging) already handled zero and - needed no change: - 1. `AudioWorkletEngineHost::meters_attached` (the meter lease's gate) read the plan's strip - meters, so an empty plan refused the lease with `RESULT_UNSUPPORTED` while the worklet and the - SDK already derive `metersAttached` from the boot option (`liveControlMeterBlocks != 0`): - the SDK's preparation rehearsal (`prepareBrowserSessionInWorker`) would have thrown "Preparation - meter lease refused" on an empty session. It now reads the configured window. - 2. The master-window ring was sized from the first strip meter's queue (`master_capacity`), so a - session that booted empty kept a zero-length ring for its whole life and the first strip's - master peaks were all counted lost. It is `METER_QUEUE_DEPTH` when a window is configured and - no strip meter exists. -- `poll_meters` with no strip meter publishes the master's own windows (a frame of the master's - two peaks, header `trackCount` 0), so a page that leases the meters on an empty mixer gets a - continuous stream; the first strip's adoption restarts delivery as any non-carried swap does. - No allocation: the branch reads the preallocated master ring and writes the preallocated frame. -- A boot with a designated master strip (`liveControlMasterTrackPlusOne != 0`) over an empty - session is still refused (`host.observation.master_track`: the strip does not exist). That is the - honest answer for a strip index; a live master designation is successor work (see the report). - -Gates: - -- Parity row: `hosts/host-web/tests/control_parity.rs` - `both_hosts_boot_an_empty_session_and_add_the_first_stem_alike` (four launch rates): empty boot - (a console slot declared), identical silence, identical control list / state / snapshot bytes, - one structural transaction (source, track, route) with the same verdict on both legs, the stem - fed after the swap and heard bit for bit alike. The stale "does not prepare" remark at - `mono_model` is dropped. -- Browser module: `hosts/host-web/tests/empty_boot.rs` - `an_empty_session_boots_and_grows_by_a_structural_transaction` (own binary, the render-locked - allocator, through the exports): status, boot watermark, control list (session rows only), the - master-only meter frame, zero render-locked allocations on the empty plan, across the first - strip's swap and while the stem plays; the meter frame grows to the strip and the control list's - structure epoch moves. -- Fixtures: `hosts/host-web/tests/empty_boot_fixture.rs` writes `qualification/empty-session.json` - (the live-control session with no source, track or route) and - `qualification/empty-boot-frame.json` (the codec's frame that adds them back), byte-compared. -- Qualification leg (`runEmptyBootQualification`; gates `empty-boot-silence`, `empty-boot-shape`, - `empty-boot-meter`, `empty-boot-first-stem`; the render-allocation list gains `empty-boot` and - `empty-boot-first-stem`): chromium, firefox, webkit with `--check-matrix --self-test-mutations`. - The SDK has no structural-edit builder, so the page sends the codec's frame through the SDK's - endpoint (`sendStructuralFrameThroughSdk`, which stamps the allocator's next request ID and the - committed revision). -- Wasm build: boot high-water growth 20,447,232 bytes (before: 20,447,232). diff --git a/.github/ISSUE_SPECS/1683-designate-the-browser-master-strip-after-boot.md b/.github/ISSUE_SPECS/1683-designate-the-browser-master-strip-after-boot.md deleted file mode 100644 index e6c0a2382..000000000 --- a/.github/ISSUE_SPECS/1683-designate-the-browser-master-strip-after-boot.md +++ /dev/null @@ -1,132 +0,0 @@ -# Designate the browser master strip after boot - -Successor of #1682 (an empty session boots), decision 17. Code anchors verified on the batch 13a -branch `codex/controls-batch-13aw`. Issue number: assigned at filing (written here as #1683). -Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. - -## Product outcome - -An app that opens an empty mixer can still have a designated master strip: the designation is a -control the app sets (or changes) after boot, through the one core path, instead of a boot option -that must name a strip that exists at boot. - -## Context - -- The browser boot option `liveControlMasterTrackPlusOne` (with observation taps) designates a - master strip at boot; over an empty session the boot is refused `host.observation.master_track`, - because the strip does not exist yet (#1682's decision record). -- Since #1682 an empty session boots on both hosts and grows by structural transactions. - -## Decisions to take (the implementer proposes, the coordinator rules) - -- Where the designation lives: a session-scoped control row (a setting) the core owns and both - hosts read, or a host option that may name a strip added later (resolved at each plan's - preparation). -- What a designation naming a strip that does not exist yet does (refused before commit, or held - until the strip exists). - -## Objective gates - -1. Boot empty, add a track, designate it master after boot; the master's observation behaves as a - boot-time designation does today, on the browser (and on the C ABI if the row is core-owned, - with a parity row). -2. host-web and control-plane tests; the browser qualification. - -## Dependencies - -- #1682. - -## Size - -Half a day. - -## Decision record: #1683 (batch 14a) - -**Decisions taken (implementer's proposal, for the coordinator's ruling).** - -- **Where it lives: a core-owned monitor row.** Every strip (track and submix) has an own row - `master` (control ID 4, toggle, `scope: monitor`, family `setting`, fixed motion, not - automatable). Like `solo` it is host-local: held in the control plane's monitor overlay - (`controls::MonitorOverlay::master`), never saved, never in a snapshot, and a write is a - monitor-only `SET_CONTROL` that spends no revision and replies path `monitor`. It is held by - strip ID, so a strip added or removed before the master never moves it. At most one strip is - the master: designating one ends the other's designation; clearing a strip that is not the - master changes nothing. Both hosts reach it through the same `SESSION_TRANSACTION_APPLY`; the - SDK's generated builders gain `strip.master(enabled)`. -- **A strip that does not exist yet: refused before commit.** The write is checked by - `controls::check_monitor` like any address; an unknown strip is `AddressUnknown`. A - structural transaction that removes the master strip ends the designation - (`MonitorOverlay::retain_strips_of`), as a removed strip keeps no solo. - -**What shipped.** - -- `controls`: `MonitorOverlay` gains `master`, `is_master`, `set_master`, `same_solo`; - `MonitorWrite` gains `row: MonitorRow { Solo, Master }` and its `soloed` field is now `on`; - `MonitorOverlay::after` applies each write to its row, and only solo writes choose the ramp. -- `protocol`: a later monitor write replaces an earlier one of the same strip *and row*. -- `host-core`: the classifiers compare solos only for the effective values (a master change - moves none) and mark the owners whose master row changed, so the catalog refreshes them. The - preparation resolves the master once, before allocating: a first plan from - `HostLiveControlRequest::master_track` (the browser's boot word, by index, refused - `host.observation.master_track` when it names no strip, as before); a successor from - `SuccessorBase::monitor.next` by ID. A successor request that names an index is refused with - the same code. The boot catalog lists the boot word's designation. -- `control-plane`: the boot plan's `master_track` seeds the session's overlay - (`PreparedRuntime::master_track`); `SessionState::master_strip()` reads it. -- `host-web`: the boot word designates the first plan only (successor requests carry `None`); - after every successful control command `ReadyOwnership::sync_master` resolves the core's - master by ID in the rendering plan's tables and in each replacement's that waits for adoption, - and writes `PlanShape::master_track` and the meter header's `master_track_plus_one`. The meter - fold is unchanged. - -**Departures.** None from frozen text. The boot word `liveControlMasterTrackPlusOne` stays: it -seeds the core's overlay, so a boot designation and a later `SET_CONTROL` are one state. It also -fixes a latent defect: a successor plan no longer reuses the boot index (after a strip was added -before the master, it named another strip, or refused the rebuild). - -**Gates.** - -1. Browser: `structural_apply::a_master_designated_after_an_empty_boot_reads_as_a_boot_designation` - (`hosts/host-web/src/tests.rs`): empty boot refuses the boot word; a rebuild adds `t0` - (compressor); `SET_CONTROL master` on `t0` applies on path `monitor` at the same revision; - the header names strip 1 and the master word is `t0`'s own, bit for bit; a later rebuild adds - `a0` before it and the master follows `t0` to strip 2; an unknown strip is refused; clearing - reads absent. C ABI parity row: - `both_hosts_designate_the_master_strip_after_an_empty_boot_alike` - (`hosts/host-web/tests/control_parity.rs`): same verdicts, control list and state pages and - snapshots on both hosts for designate, move, refuse and clear. - Core: `the_master_designation_is_one_monitor_row_held_by_strip_id` - (`crates/controls/tests/monitor.rs`), `a_successor_takes_its_master_from_the_overlay_by_strip_id` - (`crates/host-core/tests/strip_meters.rs`); `list_yields_every_expected_address_once_in_ruled_order` - updated for the new row. -2. Suites (all green): `cargo test -p controls`, `-p protocol -p control-plane --all-features`, - `-p host-core --all-features`, `-p host-web --all-features`, `-p capi --all-features`, - `-p conformance`; `check-sdk-generated.sh`, `check-sdk-types.sh`; realtime policy ok (149 - regions); clippy `-D warnings` and fmt on the touched crates. The real-browser qualification - was not run here (no browser matrix in this worktree); the boot word it uses is unchanged. - -**Moved pins (measured).** - -- `hosts/host-web/tests/browser-v1/expected.json`, simd128 `initialStatus` and - `beforeDisposeStatus` `memoryBytes`: 2,818,048 -> 2,883,584 (one 64 KiB page), reason #1683 - (one more own row per strip in the catalog and the overlay's master). Measured with - `check-browser-expected-resources.py` on this tree; the base `616106095` measures 2,818,048. -- `hosts/host-web/tests/fixtures/control-list.json`: blessed (`MISO_ENGINE_BLESS_FIXTURES=1`), four - `master` rows added, one per strip of the parity session. -- SDK regenerated: `sdk/assets/miso-engine-v1-parameter-metadata.json`, - `sdk/src/generated/controls.ts`, `sdk/src/generated/control-builders.ts`. No wire change, so no - schema hash moved. - -**Departures (batch 14a, appended).** "Departures: none" above needs one line. The C ABI lists, -accepts and reads back the `master` row, but nothing on the C ABI consumes it yet. Under decision 17 -C1 this is correct: the designation is core state, readable on both hosts. Its reading on the C ABI -is the successor #1690 ("Read the master strip's meter on the C ABI"). - -**Review fold (batch 14a).** - -- **The master takes effect at adoption.** `sync_master` updates only the pending shapes once a - replacement is published, so the rendering plan keeps its master until the swap. -- **A solo and a master write on one strip in one transaction both apply.** The replace rule of - `protocol` is per strip and row. -- **The boot word with no observation taps is refused; the row is accepted with a null reading.** - The two are documented side by side. diff --git a/.github/ISSUE_SPECS/1685-admit-observation-bindings-by-the-exact-encoded-bytes-of-the-ended-event.md b/.github/ISSUE_SPECS/1685-admit-observation-bindings-by-the-exact-encoded-bytes-of-the-ended-event.md deleted file mode 100644 index 71e03ce0f..000000000 --- a/.github/ISSUE_SPECS/1685-admit-observation-bindings-by-the-exact-encoded-bytes-of-the-ended-event.md +++ /dev/null @@ -1,180 +0,0 @@ -# Admit observation bindings by the exact encoded bytes of the ended event - -Successor of #1676 (cap the observation bindings by what one ended event holds) and #1584 D6a -(root ruling A: one `OBSERVATION_ENDED` per commit). Code anchors verified on the batch 13b fold -branch `codex/controls-b13b-fold`. Issue number: assigned at filing (written here as #1685). -Batch 14, the first K (core) slice. Feature sprint mode (owner, 2026-10-13): one implementer, -normal tests, a batch review. - -## Product outcome - -A mixer can watch a meter on every insert of a large session. The host refuses a subscription only -when the one `OBSERVATION_ENDED` event that a commit could emit would not fit in one control frame, -measured with the real IDs of the armed taps, not with the longest IDs the grammar allows. - -## Context - -- **The cap today.** `observation_binding_cap(frame_bytes)` (`crates/control-plane/src/compile.rs`, - near `:483-523`) encodes an `ObservationEnded` event with keys whose strip ID and effect ID are - both `session::STABLE_ID_MAXIMUM_BYTES` (127) bytes, and returns how many such keys one frame - holds: 3 at 1 KiB, 12 at 4 KiB, 51 at the browser's 16 KiB (`CONTROL_FRAME_BYTES`, - `hosts/host-web/src/lib.rs`). `SessionState::observation_cap()` (`crates/control-plane/src/control.rs`, - near `:2677-2683`) keeps that count, and `admit_and_push_observation_batch` (near `:2718`) refuses - the arm that would leave more taps armed (`ObservationRefusal::Limit`, near `:2835-2845`). -- **What it costs.** Real IDs are short. A 64-track session with a compressor insert on every - track already needs 64 taps, more than the browser's 51, and the C ABI at 4 KiB allows 12. -- **The event.** A commit that ends bindings emits one `OBSERVATION_ENDED` with every ended key, in - key order, after its `SESSION_COMMITTED` (#1584 D6a, root ruling A). The room is reserved with - the commit before anything is prepared (`attach_observation_ended`, `control.rs` near - `:1616-1660`); an event over one frame is refused as a resource fault, a defence only. -- **The reserve.** `ended_event_ceiling(bindings)` (`compile.rs` near `:590-603`) charges each key - as `size_of::()` plus two maximum IDs, and the binding table rows - (`observation_binding_rows`) the same. - -## Decisions frozen for this slice - -- **D1. Admit by bytes.** The session keeps the exact encoded size of the `OBSERVATION_ENDED` - frame that would carry every armed key (the frame with no key, plus each armed key's own encoded - bytes, with the real strip and effect IDs). A batch is admitted when that size, after every - removal and arm of the batch, is at most the control frame limit. The arm that crosses is refused - with `ObservationRefusal::Limit` (its index; the reason names bytes, not a key count), with - nothing queued and no state moved. One naming per tap and the room check stay as they are. -- **D2. No allocation.** The per-key size is computed from the key's lengths with the codec's own - size rules (no encode into a buffer), so admission stays allocation-free in the browser - worklet's message handler. -- **D3. One event per commit.** Ruling A stands: a commit emits at most one `OBSERVATION_ENDED`, - and every binding a commit can end is armed, so its keys are a subset of the admitted set and - the event always fits. The room is still reserved at commit, before anything is prepared. -- **D4. The reserve follows the bytes.** The ended-event heap and the binding table rows charge - what the admitted keys can own, as ceilings. `observation_cap()` and its key-count figures are - removed with the docs that state them. - -## Deliverables - -1. D1 to D4 in `crates/control-plane`. -2. The browser's `observationLimit` reason (15) and its message keep their meaning; the SDK hint - text says the frame is full, not that a count is reached. -3. Docs: `docs/CONTROL_PROTOCOL_SEMANTICS.md` (the observation paragraph's 3/12/51 figures). - -## Authorized paths - -- `crates/control-plane/src/{control.rs,compile.rs}`, `crates/control-plane/tests/observation_bindings.rs` -- `hosts/host-web/src/lib.rs` (comments only), `hosts/host-web/src/tests.rs`, - `hosts/host-web/tests/control_parity.rs` -- `sdk/src/core/observation-subscriptions.ts` (the hint text only), its eval -- `docs/CONTROL_PROTOCOL_SEMANTICS.md` - -## Non-goals - -- A second `OBSERVATION_ENDED` per commit, or splitting the event over frames. -- A larger control frame. -- The C ABI observe edit itself (#1584 D1-D5); it calls the same core admission (#1584's batch 13b - amendment). - -## Objective gates - -1. **Large session.** A 64-track session with one compressor insert per track subscribes the - compressor's meter tap on every insert at the browser's 16 KiB frame: all 64 admitted; on the C - ABI (no default frame: the host sets `maximum_control_frame_bytes`; the capi tests use 4 KiB), - the same, through #1584's observe edit when it is on `main`, else at the core through - `SessionState` with a 4 KiB limit. -2. **The edge.** Keys sized so that the frame holds exactly `k` of them: `k` arms are admitted, - arm `k + 1` is refused `Limit` at its index, a disarm in the same batch makes room, and removing - the strips commits with one `OBSERVATION_ENDED` that holds all `k` keys and fits the frame. -3. **Realtime.** Admission allocates nothing (`bench_support::alloc` thread counters, statics - warmed). -4. Commands: `cargo test --locked -p control-plane --features test-support`, - `cargo test --locked -p host-web --features host-web/test-support`, the browser qualification, - clippy `-D warnings`, `fmt --check`. - -## Test value - -- Gate 1: red if admission still assumes 127-byte IDs (64 taps are refused). -- Gate 2: red if the byte count is off by one key or by the frame header, or if the commit can - emit an event over one frame. -- Gate 3: red if admission encodes into an allocated buffer. - -## Dependencies - -- #1676 and #1677 (the cap, the stable key; batch 13b). -- #1584 D6a (one event per commit). - -## Decision record: #1685 (batch 14a) - -**What shipped.** - -- **D1.** `SessionState` keeps `observation_armed_bytes` (the armed keys' encoded bytes, pushes not - yet folded included) and an `ObservationWire` (the frame with no key, a key's fixed part, the - control frame limit). `admit_and_push_observation_batch` judges the batch by - `base + armed bytes after every removal and arm <= frame`; the arm that crosses is - `ObservationRefusal::Limit(ObservationLimit { index, event_bytes, frame_bytes })` (bytes, not a - count), with nothing queued and no state moved. The naming and room checks are unchanged. - `observation_cap()` is removed; `observation_ended_bytes()` and `observation_frame_bytes()` - replace it. -- **D2.** A key's size is `key_fixed + pad8(strip_len) + pad8(effect_len)` (the BTLV value - alignment). `observation_wire` measures `base` and `key_fixed` with the real codec once per - session compile and checks the model against the codec for padded and longest IDs; a mismatch - fails the compile. Each observation slot stores its key's bytes as a `u16` (in the slot's - padding: `ObservationSlot` stays 16 bytes), set when the plan's slots are built, so admission - reads a slot field and allocates nothing. -- **D3.** Unchanged: one event per commit, room reserved at commit; a rebuild recomputes the armed - bytes from the kept bindings. -- **D4.** The ended-event heap and the binding rows charge, as ceilings, the peak over the key count - `n <= min(reserve slots, room / shortest key)` of `n` struct sizes plus the ID text, where the - text is `min(n * 254, room - n * key_fixed)` (an ID's text is at most its padded encoded length, - and every admitted key fits one frame together). The table no longer charges text per spare - capacity entry. - -**Measured** (`observation_reserve_bytes(frame, slots)` as `(active, token)`, before -> after): -1 KiB, >= 16 slots: (7306, 930) -> (6630, 930). 4 KiB, 1 slot: (4174, 310) -> (2396, 310); 4 KiB, ->= 51 slots: (26616, 3720) -> (27880, 3928). 16 KiB, 51 slots: (113118, 15810) -> (100164, 15810); -16 KiB, 64 slots: (113118, 15810) -> (101792, 15904); 16 KiB, 1000 slots: (113118, 15810) -> -(112992, 15904). The 4 KiB and 16 KiB tokens rise by 208 and 94 bytes because exact-byte admission -lets 13 keys of 120-byte IDs in at 4 KiB where the cap allowed 12 longest keys; that is the measured -rise (#1685 exact-byte admission). No retained byte row was pinned to these values: the capi -allocator oracle reads `observation_reserve_bytes`, the browser identity fixture has no observation -capacity (its rows are unchanged and within budget), and `retained_ceilings.rs` passed. No re-pin -was needed. - -**Departure (gate 1, 4 KiB).** A key takes at least 80 encoded bytes (nested TLV 8, message header -8, strip 16, section 16, effect 16, tap 16), so a 4 KiB frame holds about 50 short keys and 64 -cannot fit one event at 4 KiB by any exact count. D1 is frozen, so gate 1 is met at the browser's -16 KiB on both the browser and the core; at 4 KiB the core test proves the exact edge for the -64-track session (the arm that crosses is refused at its index with the codec's bytes, and exactly -the taps that fit are admitted). A C ABI host that wants 64 insert meters configures a frame of at -least about 5.2 KiB. The C ABI has no observe edit yet (#1584), so the C ABI side is the core -admission through `SessionState`. - -**Outside the authorized paths (mechanical).** `crates/control-plane/Cargo.toml` (+ `Cargo.lock`): -`bench-support` as a dev dependency, for gate 3's thread counters (allowed for `crates/` by -`check-bench-policy.sh`). `hosts/host-web/src/lib.rs`: the `debug_assert` that called -`observation_cap()` now compares `observation_ended_bytes()` with `observation_frame_bytes()`. - -**Gates.** - -1. Large session: `every_insert_of_a_sixty_four_track_session_subscribes_at_the_browser_frame` - (core, 16 KiB and 4 KiB), `tests::every_insert_of_a_sixty_four_track_session_subscribes` - (browser, 16 KiB, one submission of 64 records): pass. -2. Edge: `a_subscription_past_the_frame_is_refused_and_the_removal_of_all_still_commits` (core, - 1 KiB, longest IDs, `k` from the codec), `the_ended_bytes_are_the_encoded_event_of_the_armed_keys` - (IDs on each side of the alignment), browser - `tests::a_subscription_past_the_frame_is_refused_and_an_unsubscribe_makes_room`: pass. -3. Realtime: `admission_allocates_nothing` (`bench_support::alloc` thread counters after a warm-up; - arms, a `Limit` refusal and disarms): 0 allocations. -4. `cargo test --locked -p control-plane --features test-support`, `-p host-web --all-features`, - `-p host-core --all-features`, `-p capi`: pass. `scripts/check-web-audioworklet.sh`, - `scripts/check-browser-expected-resources.py`, `scripts/check-sdk-headless.sh` (559 evals), - `scripts/check-sdk-generated.sh`, `check-command-reason-vocabulary.py`, `realtime-policy`, - clippy `-D warnings`, `fmt --check`: pass. - -**Correction (batch 14a).** The departure above says a 64-tap C ABI frame is "about 5.2 KiB". The -measured figure is 5,184 bytes (about 5.06 KiB) for strip and effect IDs of 8 bytes or less, and -6,208 bytes with 16-byte IDs. The comment on `maximum_control_frame_bytes` in -`crates/capi/include/miso_engine_v1.h` gives the rule: 64 bytes plus, per armed tap, 64 bytes plus -each ID padded to a multiple of 8. - -**Review fold (batch 14a).** - -- **The binding-table reserve charges two peaks.** It charges the capacity peak and the text peak - separately. The `Vec` keeps its high-water capacity, while the text belongs to a later armed set, - so the two peaks do not come from one armed set. diff --git a/.github/ISSUE_SPECS/1686-remove-the-redundant-span-kind-and-end-from-prepared-automation-spans.md b/.github/ISSUE_SPECS/1686-remove-the-redundant-span-kind-and-end-from-prepared-automation-spans.md deleted file mode 100644 index 7248febe0..000000000 --- a/.github/ISSUE_SPECS/1686-remove-the-redundant-span-kind-and-end-from-prepared-automation-spans.md +++ /dev/null @@ -1,136 +0,0 @@ -# Remove the redundant span kind and end from prepared automation spans - -Successor of #1636 (retire the span kinds and sample rate no producer uses). Code anchors verified -on the batch 13b fold branch `codex/controls-b13b-fold`. Issue number: assigned at filing (written -here as #1686). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch -review. - -## Product outcome - -An effect's automation input says only what the engine delivers: one value for one parameter -channel at one sample. No field can describe a span shape that no producer makes, so no effect and -no test has to check one. - -## Context - -- **The struct.** `PreparedAutomationSpan` (`crates/effect-contract/src/lib.rs`, near - `:1650-1658`) has `kind: AutomationSpanKind`, `channel`, `parameter_index`, `start_sample`, - `end_sample`, `start_value` and `end_value`. It is a plain Rust struct (no `repr(C)`), not on a - wire or the C ABI. -- **Only `Point` is left.** After #1636, `AutomationSpanKind` is `{Point=1}` (`lib.rs` near `:426`; - `from_raw` refuses 2 to 4). `valid_runtime_span` (near `:1806-1834`) checks that a `Point` has - `start_sample == end_sample` and equal value bits, so `kind`, `end_sample` and `end_value` carry no - information: a valid span always repeats its start in its end. -- **Who builds and reads spans.** Live records stage only `Point` spans - (`crates/effect-contract/src/live.rs`, near `:1315-1325`), and stored automation stages only - `Point` spans (#1058 A1.6). Construction and read sites are in `crates/effect-contract`, - `effect-compiler`, `rack`, `graph`, `graph-compiler`, `host-core`, `conformance`, each native - effect crate (`compressor`, `delay`, `gate-expander`, `multiband-compressor`, `parametric-eq`, - `soft-clip`, `transient-shaper`, `true-peak-limiter`), and `tools/{audit,console-workload,parameter-metadata}`. - A test that exists only to check a mismatched end (the delay's malformed-span test as #1636 left - it: a `Point` whose end sample differs from its start) has nothing left to check. - -## Decisions frozen for this slice - -- **D1. Remove the fields.** `PreparedAutomationSpan` loses `kind`, `end_sample` and `end_value`. - `valid_runtime_span` loses the checks that read them. `validate_automation_block`'s ordering - rule (near `:1869-1872`, which compares a span's start with the previous span's `end_sample` and - `end_value`) reads the previous span's sample and value instead, with the same verdict for every - valid `Point`. -- **D2. Rename what is left.** `start_sample` becomes `sample` and `start_value` becomes `value` - (a start names nothing without an end). The rename is mechanical. -- **D3. The kind enum.** `AutomationSpanKind` (and its alias `AutomationKind`) is deleted when no - other type reads it; its retired raw values stay documented as retired where #1636 put that note. -- **D4. Tests.** A test that exists only for a removed field is deleted in the same PR; a fixture - that sets the removed fields drops them. - -## Deliverables - -1. D1 to D4 across the construction and read sites named in Context. -2. `docs/EFFECT_CONTRACT_V1.md`: the span description names one sample and one value. - -## Authorized paths - -- `crates/effect-contract/src/`, `crates/effect-contract/tests/` -- The construction and read sites only, in `crates/{effect-compiler,rack,graph,graph-compiler,host-core,conformance}` - and the eight native effect crates (sources and tests) -- `tools/{audit,console-workload,parameter-metadata}` (span construction only) -- `docs/EFFECT_CONTRACT_V1.md` - -## Non-goals - -- Any change to how an effect applies a `Point` (smoothing, ramps, counters). -- New span kinds. - -## Hazards - -- **Retained bytes.** The struct shrinks, so a staging window's bytes shrink. Ceilings stay; a test - that pins an exact byte count of a span window is re-pinned with the reason - "#1686 removes the span kind and end". - -## Objective gates - -1. **No rendered bit moves.** `cargo build --locked --release -p audit -p capi && - ./target/release/audit capi` reports the same `pcm_digest` at base and head (PR evidence), and - the browser legs pass with unchanged digests. -2. **Contract.** The effect-contract truth table for `valid_runtime_span` keeps every case that - still has a field to read (non-finite value, sample outside the block, value outside the - domain, channel against the parameter's channel policy), and `validate_automation_block` - gives the same verdicts on its ordering cases (spans out of order; two spans at one sample with - different values). -3. Commands: `cargo test --locked --workspace` (or the `test-debug-a` and `test-debug-b` legs), - the randomized conformance run, `bash scripts/check-effect-runtime-policy.sh`, - `bash scripts/check-workspace-policy.sh`, clippy `-D warnings`, `fmt --check`. - -## Test value - -- Gate 1: red if removing the fields changed what an effect applies. -- Gate 2: red if a check on a remaining field was dropped with the removed ones, or if the ordering - rule changed its verdict when it lost the end fields. - -## Dependencies - -- #1636 (batch 13b). - -## Decision record: #1686 (batch 14a) - -**Shipped.** `PreparedAutomationSpan` is `{channel, parameter_index, sample, value}`. `AutomationSpanKind` -and its alias `AutomationKind` are deleted (nothing else read them; `protocol`'s own `AutomationKind` was -already gone with #1629). The retired-values note for kinds 2 to 4 now sits in the doc comment of the -struct, beside the `AutomationRate` note that stays. `valid_runtime_span` is the conjunction of the -remaining checks (finite value, sample inside the block, value in the domain, channel against the -channel policy). `validate_automation_block`'s ordering rule compares a span with the previous span's -`sample` and `value` (same verdict for every valid point). Every effect's span accept rule lost the -kind/end terms (they were constant true for a representable span). `docs/EFFECT_CONTRACT_V1.md` names -one sample and one value. - -**Departures.** None from the frozen decisions. Tests that only checked a removed field: -- `delay`: `malformed_automation_is_counted_and_never_applied` lost its "end differs from start" row - (count 5 to 4). -- `soft-clip` `automation_rejects_everything_but_canonical_ordered_points` lost its `ranged` row (3 invalid). -- `parametric-eq` `malformed_automation_rejects_each_span_without_losing_valid_targets` lost two rows - (8 to 6; the EQ counts every raw span). -- `conformance/tests/effect_contract.rs` span-validation fuzz: modulus 5 to 4; the end-only cases are - replaced by one non-finite value case. -- `effect-contract` `retired_automation_values_are_refused` lost the `AutomationSpanKind` asserts; - `live_control` and `rack` lost their kind/end asserts. -No retained byte row or fixture byte count moved (the estimate reads `size_of::()`). - -**Gates.** -1. `./target/release/audit capi`: base `pcm_digest` `bf7dcedd1d92621f`, head `bf7dcedd1d92621f` - (allocations 0, total_violations 0). -2. Contract truth table: `crates/effect-contract` unit tests and `crates/conformance/tests/effect_contract.rs` - keep non-finite value, sample outside the block, value outside the domain and channel-policy rows; - ordering rows unchanged. Randomized conformance run passes. -3. `cargo test --locked` for effect-contract, effect-compiler, conformance, rack, graph, graph-compiler, - the eight native effect crates, capi, host-core (--all-features), host-web (--all-features), - console-workload, parameter-metadata, audit: all pass. `check-effect-runtime-policy.sh`, - `check-workspace-policy.sh`, `check-effect-runtime-fixtures.sh`, `realtime-policy`: ok. - `cargo clippy --workspace --all-targets --all-features -- -D warnings` and `cargo fmt --all --check`: clean. - -**Review fold (batch 14a).** The departure above lists "modulus 5 to 4" for the span validation -fuzz. That change stands for the span loop, which has four cases. The descriptor loop in the same -test (`ten_thousand_descriptor_and_span_mutations_reject_without_panic`) also took the modulus 4 -by mistake, and its fifth case, the duplicate parameter ID, became unreachable. Its modulus is -restored to 5. `docs/EFFECT_CONTRACT_V1.md` near `:358` is rewrapped to the file's width, with no -word changed. diff --git a/.github/ISSUE_SPECS/1688-report-the-plan-latency-on-the-browser.md b/.github/ISSUE_SPECS/1688-report-the-plan-latency-on-the-browser.md new file mode 100644 index 000000000..6aae3f7ff --- /dev/null +++ b/.github/ISSUE_SPECS/1688-report-the-plan-latency-on-the-browser.md @@ -0,0 +1,146 @@ +# Report the plan latency on the browser + +One of the one-host differences of decision 17 C13 (`docs/rulings/one-control-abstraction-2026-10-13.md`). +Code anchors verified on `main` at `e2f6cfb24`. Issue number: assigned at filing (written here +as #1688). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. + +## Product outcome + +A browser app reads the running plan's output latency and output tail, as a C ABI app does. After +a rebuild is adopted, the app reads the new plan's values. Both hosts read them from one core +function, so they always agree. An app that aligns a meter, a playhead or a recording to the +output can do it on the browser. + +## Context + +- **The core row.** Host-core reports a plan's latency as `HostPrepareReport::latency_samples` + (`crates/host-core/src/prepare.rs:273`, filled at `:3134` from the output arrival). The control + plane copies it into `PlanResources::latency_samples` (`crates/control-plane/src/compile.rs:1003`; + the field is `crates/control-plane/src/adapter.rs:104`, beside `output_tail: TailSamples` at + `:106`). The control plane keeps one row per plan epoch. +- **The core read.** `PlanQueries::resources()` (`crates/control-plane/src/plan.rs:97`) copies the + row of the active plan epoch (`active_resources`, `:49`). It runs on any thread, beside render. +- **The C ABI.** `miso_engine_v1_plan_resources` (`crates/capi/src/ffi.rs:930`) writes that row + as `PlanResourceReport`: `latency_samples`, `tail_kind`, `tail_samples` + (`crates/capi/include/miso_engine_v1.h:366-368`, `crates/capi/src/abi.rs:283`). The tail + encoding is in the capi adapter: `From for PlanResourceReport` + (`crates/capi/src/runtime/mod.rs:217-222`), with `TAIL_FINITE` 0 and `TAIL_INFINITE` 1 + (`crates/capi/src/abi.rs:37-39`). +- **The browser today.** The browser runs the same control plane (#1381a, + `hosts/host-web/src/lib.rs:6-16`), and its adapter's row is `PlanResources` itself + (`WebAdapter`, `lib.rs:5243-5244`). It holds `queries: PlanQueries` (`lib.rs:1514`). + But it exports no latency and no tail. Its `WebResourceReport` (`lib.rs:1216`, exported by + `miso_engine_web_v1_resource_ptr`, `hosts/host-web/src/ffi.rs:3450`) is a byte projection made + once at boot (`lib.rs:2220`) and is never refreshed. It has no latency field. +- **The pattern to follow.** The watermark is a core read the browser copies into a fixed + record: `read_watermark` (`lib.rs:2522-2544`) fills `WebPlanWatermark` (`lib.rs:1189`) from + `ready.queries.watermark()`. The exports are `miso_engine_web_v1_watermark_read` and + `miso_engine_web_v1_watermark_ptr` (`ffi.rs:3434`, `:3442`; #1381a D5). +- **Parity.** `hosts/host-web/tests/control_parity.rs` plays one script through both hosts. The + script already holds a rebuild that moves the plan's latency: a strip's `delay_samples` + (`control_parity.rs:226`, `:727`). +- **Floors.** *Keep every node's latency from dropping during playback* (#1285) makes the + reported latency the floored output arrival (its D4). Its batch-15 amendment notes that the + browser reports no latency, so its gate 3 covers the C ABI only. + +## Decisions frozen for this slice + +- **D1. One core function.** Both hosts read the latency and tail through + `PlanQueries::resources()`. The tail's two-word encoding moves into the control plane as one + function, `control_plane::tail_words(TailSamples) -> (u64, u64)`, with the two kind constants + beside it. The capi adapter's `From` impl calls it, and the C ABI's `TAIL_FINITE` and + `TAIL_INFINITE` keep their values. No new latency logic. +- **D2. The browser form.** `miso_engine_web_v1_plan_latency_read(handle) -> u32` copies the + active row's `latency_samples` and the D1 tail words into a fixed record, `WebPlanLatency`: + `struct_size: u32`, `reserved0: u32`, `latency_samples: u64`, `tail_kind: u64`, + `tail_samples: u64`, `reserved: [u64; 4]` (56 bytes). `miso_engine_web_v1_plan_latency_ptr(handle) + -> u32` returns the record's stable address. A host that is not ready returns + `RESULT_WRONG_STATE` and leaves the record alone, as `read_watermark` does. The read runs where + the plan queries are (the worklet's message handler in `single` mode; #1332 moves it with the + rest of the control half). +- **D3. Mirrors.** Both exports and the record's offsets join every export list and layout + mirror, as the watermark's did. +- **D4. Parity row.** `control_parity.rs` compares, after every scripted edit and at the end, the + browser record's three words with the C ABI report's `latency_samples`, `tail_kind` and + `tail_samples` for the same plan. +- **D5. The acked-batch question.** This is a read. It acks nothing and drops nothing. + +## Deliverables + +1. D1 in `crates/control-plane/src/` and `crates/capi/src/runtime/mod.rs`. +2. D2 in `hosts/host-web/src/lib.rs` and `hosts/host-web/src/ffi.rs`, with native tests. +3. D3 in the checkers, the generator and the generated SDK files. +4. D4 in `hosts/host-web/tests/control_parity.rs`. + +## Authorized paths + +- `crates/control-plane/src/plan.rs` or `crates/control-plane/src/adapter.rs` (D1), + `crates/control-plane/src/lib.rs` (re-export only), `crates/capi/src/runtime/mod.rs` (the `From` + impl only), `crates/capi/src/abi.rs` (the constants' source only, values unchanged) +- `hosts/host-web/src/lib.rs`, `hosts/host-web/src/ffi.rs`, `hosts/host-web/src/tests.rs`, + `hosts/host-web/tests/control_parity.rs` +- `scripts/check-web-audioworklet.sh` (export list only), `scripts/check-abi-layout-v1.py` and + `scripts/fixtures/abi-layout-v1-self-test.json`, `tools/parameter-metadata/src/abi_layout.rs`, + `sdk/assets/miso-engine-v1-abi-layout.json`, `sdk/src/generated/abi.ts`, + `scripts/check-sdk-generated.sh`, `scripts/test-web-audioworklet.mjs` (the mock's export list + only) + +## Non-goals + +- No SDK API. A later issue gives the SDK a `planLatency()` call on this export. +- No change to how latency is computed (#1285 owns the floors) or to the C ABI's report. +- No refresh of `WebResourceReport`. It stays a boot projection. +- The other one-host differences of C13. + +## Objective gates + +1. **Parity (D4).** In `control_parity.rs`, at each launch rate the test already runs, the + browser record equals the C ABI report's three words after every scripted edit and at the + end. The `delay_samples` edit changes `latency_samples` on both hosts; the test asserts that + the value moved, so the row is not vacuous. +2. **The read (native, `hosts/host-web/src/tests.rs`).** Before boot completes, the read returns + `RESULT_WRONG_STATE` and the record keeps `struct_size` 0. After boot, it returns OK and the + record holds the boot plan's latency and tail. After a rebuild that adds a true-peak limiter + insert, the record keeps the old values until the adoption block, and holds the new plan's + latency from the first read after it. +3. **Mirrors.** `python3 -B scripts/check-abi-layout-v1.py` and its self-test pass, and fail if a + list omits an export or a record offset. +4. **Commands:** + - `cargo test --locked -p host-web --features host-web/test-support` + - `cargo test --locked -p capi`, `cargo test --locked -p control-plane --features control-plane/test-support` + - `mkdir -p target/ci/qualification-artifacts target/ci/qualification-named-twin && bash scripts/build-web-audioworklet.sh --named-twin target/ci/qualification-named-twin target/ci/qualification-artifacts` + - `bash scripts/check-web-audioworklet.sh --without-metadata-regeneration target/ci/qualification-artifacts target/ci/qualification-named-twin/miso-engine-v1-audio-worklet.simd128.named.wasm` + - `bash scripts/test-web-audioworklet.sh`, `bash scripts/check-sdk-generated.sh target/ci/qualification-artifacts` + - `bash scripts/check-capi-abi.sh` + - `cargo clippy --locked --workspace --all-targets --all-features -- -D warnings`, + `cargo fmt --all -- --check`, `bash scripts/check-workspace-policy.sh` + +## Test value + +- Gate 1 turns red if the browser computes latency or the tail by logic of its own, reads a + stale plan's row, or encodes the tail differently from the C ABI. +- Gate 2 turns red if the read reports the candidate's latency before render adopts it, or keeps + the boot value after a rebuild (a boot-time copy like `WebResourceReport`). +- Gate 3 turns red if a mirror lets the SDK call a name the module lacks. + +## Dependencies + +- Decision 17 C13 lists "plan latency and tail" as a one-host difference for root to file. + This is that issue. +- *Keep every node's latency from dropping during playback* (#1285), ordering: land this after + #1285, so the parity row reads floored latencies. #1285 then needs no browser gate of its own; + this row covers the browser. +- #1381a has landed (the browser runs the control plane). #1332 (open) moves the read with the + control half; it is not a dependency. + +## Size + +Half a day. + +## Amendment (controls coordinator, batch 15a fold, 2026-10-16) + +- **D2's record size.** `WebPlanLatency` is 64 bytes, not 56: `struct_size` and `reserved0` (4 + 4), + `latency_samples`, `tail_kind` and `tail_samples` (3 x 8), `reserved: [u64; 4]` (32), with no + padding. D2's "(56 bytes)" reads "(64 bytes)". The ABI layout mirrors (D3) record 64. +- **Decision 17 C13.** The two exports are listed there (root, 2026-10-16) as a transport read of + the core's `PlanQueries::resources()` row, so they are not a C13 exception. diff --git a/.github/ISSUE_SPECS/1689-retire-the-browser-master-boot-word.md b/.github/ISSUE_SPECS/1689-retire-the-browser-master-boot-word.md new file mode 100644 index 000000000..afba3e31b --- /dev/null +++ b/.github/ISSUE_SPECS/1689-retire-the-browser-master-boot-word.md @@ -0,0 +1,199 @@ +# Retire the browser master boot word + +Successor of #1683 (the master strip is a core `master` monitor row), decision 17 C1 +(`docs/rulings/one-control-abstraction-2026-10-13.md`: a behaviour that exists on one host only is +a defect). Code anchors verified on branch `codex/controls-batch-14aw` at `5ef2c81d8` (the #1683 +tree). Issue number: assigned at filing (written here as #1689). Feature sprint mode (owner, +2026-10-13): one implementer, normal tests, a batch review. + +## Product outcome + +A browser app designates the master strip one way only: it writes the strip's `master` row +(`strip.master(enabled)`, `SET_CONTROL`), as a C ABI app does. The boot options carry no master +word. The master reading (`masterGrDb`) and the meter header's `masterTrackPlusOne` are the +values they are today; only the second route to the designation is gone. + +## Context + +- **Two routes today.** #1683 made the master a core monitor row held by strip ID + (`controls::MonitorOverlay::master`). The browser boot word still designates the first plan's + master by strip index. It seeds the same overlay, so the two routes are one state, but the word + is a browser-only feature, it needs a strip that exists at boot, and it ties the master reading + to observation capacity at boot (`live_control_master_track_plus_one != 0` needs + `live_control_observation_taps != 0`). +- **The core path of the word.** + - `HostLiveControlRequest::master_track` (`crates/host-core/src/prepare.rs:401`, default + `None` at `:411`); the preparation resolves it for a first plan and refuses a successor + request that names one, both with `host.observation.master_track` (`:1942-1955`); the boot + overlay is built from it (`:1957-1968`). + - `HostLiveControlHandles::master_track` echoes it (`prepare.rs:536`, filled at `:3289`). + - `PreparedRuntime::master_track` (`crates/control-plane/src/compile.rs:30`, set at `:937` and + `:1039`, ignored at `:1130`), which seeds the session overlay at boot (`compile.rs:1083-1095`). + `crates/control-plane/src/control.rs:1699` ignores it in a pattern. +- **The browser path of the word.** + - `WebBootOptions::live_control_master_track_plus_one` at offset 56, the last word of the 64-byte + block (`hosts/host-web/src/lib.rs:1103`, default `:1121`; `hosts/host-web/src/ffi.rs:519`). + - Its validation, `web.options.live_controls` (`lib.rs:4926-4930`); its translation into the + request (`live_control_request`, `lib.rs:6459-6462`); the successor reset to `None` + (`lib.rs:5578`); the plan shape (`PlanShape::master_track`, `lib.rs:1398`; built at + `:5841` and `:5891`; resolved after every control command by `ReadyOwnership::sync_master`, + `:1655-1675`). + - The precedent for a retired word: offset 32 was the live-command queue word. #1565 reserved it + as `reserved1`, kept it in the layout, and refuses a writer that sets it with + `RESULT_REFUSED_OPTIONS` and `web.options.reserved1` (`lib.rs:4917-4924`), by the worklet's `writeBootOptions` + (`miso-engine-v1-audio-worklet.js:939-957`) and by `validBootOptions` + (`miso-engine-v1-audio-worklet-host.js:630-645`). +- **Every place the word is laid out or written** (all verified by search on the #1683 tree): + - Rust host: `hosts/host-web/src/lib.rs`, `hosts/host-web/src/ffi.rs`, the layout tests in + `hosts/host-web/src/tests.rs` (`:315` and the boot helpers at `:3085`, `:4415`, `:4459`, + `:5234`, `:8097`, `:8242`, `:8271`). + - Worklet JS: `hosts/host-web/web/miso-engine-v1-audio-worklet.js` (the field list `:81`, + `writeBootOptions` and its observation check `:939-957`), `miso-engine-v1-audio-worklet-host.js` + (the field list `:73`, the descriptor copy `:520`, `validBootOptions` `:640-642`), and the declaration `miso-engine-v1-audio-worklet-host.d.ts:718` with its + byte-identical twin `sdk/src/browser/shipped-host.d.ts:718`. + - SDK boot policy: `sdk/src/browser/host-mirror.ts:45-58,81` (the `masterTrackPlusOne` word and + its "requires liveControlObservationTaps" check), `sdk/src/core/abi.ts:118-124` (the + `liveControls.masterTrackPlusOne` option), `:145` (`POLICY_WORDS`), `:190` (the write), + `sdk/src/core/boundary.ts:204-209` (the option copy), and the tests that set it + (`sdk/test/host-mirror.ts:41`, `sdk/test/browser-evals.mjs:38,141`, + `sdk/test/observation-subscription-evals.mjs:564,592`, `sdk/test/capability-evals.mjs:227,290,342`). + - ABI layout generator and checkers: `tools/parameter-metadata/src/abi_layout.rs:275-279`, + `tools/parameter-metadata/tests/abi_layout.rs:300`, `scripts/check-abi-layout-v1.py` + (`BOOT_OPTION_FIELDS` `:252`, `POLICY_WORDS` `:259`), `scripts/fixtures/abi-layout-v1-self-test.json:124`; + generated mirrors `sdk/assets/miso-engine-v1-abi-layout.json:124` and + `sdk/src/generated/abi.ts:192`. + - Qualification harness and scripts: `hosts/host-web/qualification/qualification.js` + (`:21` `OBSERVATION_MASTER_TRACK_PLUS_ONE`, `:58`, `:71`, `:663`, `:886`), + `hosts/host-web/qualification/suspended-host.mjs` (`:46`, `:427`, `:579`, `:739`), + `hosts/host-web/qualification/rebuild-cost.mjs` (`:72`, `:109`), + `hosts/host-web/tests/browser-v1/browser-correctness.js:21`, + `scripts/test-web-audioworklet.mjs` (`:65`, `:1043`, `:3268-3287`), + `scripts/web-mixing-automation-benchmark.mjs` (`:290`, `:776`), + `scripts/check-browser-expected-resources.py` (`:308`, `:508`, `:519`, `:597`: it edits the + word to build its mutations). + - Docs: `docs/EFFECT_OBSERVATION_V1.md`. +- **Not this word.** `WebMeterHeader::master_track_plus_one` (offset 40, `lib.rs:975`) is the + frame's report of the resolved master. It stays. + +## Decisions frozen for this slice + +- **D1. The field stays, reserved, never reused.** The word at offset 56 becomes `reserved2` + (`u64`). The block stays 64 bytes and every other offset is unmoved. No later word takes the + offset. This is the #1565 pattern for `reserved1`. +- **D2. A boot that sets it is refused.** `validate_options` refuses a nonzero `reserved2` + with `RESULT_REFUSED_OPTIONS` and the typed reason `web.options.reserved2`, as `reserved1` + does. The worklet's `writeBootOptions` and the host's `validBootOptions` refuse it the same way + before the module sees it. The `web.options.live_controls` rule loses its master clauses. +- **D3. The core request loses the field.** `HostLiveControlRequest::master_track`, + `HostLiveControlHandles::master_track` and `PreparedRuntime::master_track` are deleted, with + the first-plan overlay seeding and the `host.observation.master_track` refusals that existed + only for them. The master of a plan is the overlay's, by strip ID, for a first plan and a + successor alike. A boot catalog lists no master; the app writes the row after boot. +- **D4. The SDK drops the option.** `liveControls.masterTrackPlusOne` leaves the SDK's boot + options and `POLICY_WORDS`; the SDK always writes `reserved2` as `0n`. `strip.master(enabled)` + (the generated builder from #1683) is the SDK's one way. The ABI-layout JSON, the generated + TypeScript and the self-test fixture are regenerated, not edited. +- **D5. The master needs no boot capacity.** A strip's `master` row is a monitor write that spends + no revision. Its `masterGrDb` reading still needs an armed observation tap on the master strip; + it no longer needs `liveControlObservationTaps` at boot to be nonzero for the word. (An app + that wants taps still asks for capacity, as before.) +- **D6. The qualification harness designates through the row.** Every harness boot that set the + word (`OBSERVATION_MASTER_TRACK_PLUS_ONE`, the mock options of `test-web-audioworklet.mjs`, the + benchmark scripts) boots with `0n` and sends `SET_CONTROL master` for the strip after the + boot, or for a harness with no command path, boots a session the row can already address. + +## Deliverables + +1. D1-D3 in `hosts/host-web/src/{lib.rs,ffi.rs}`, `crates/host-core/src/prepare.rs`, + `crates/control-plane/src/{compile.rs,control.rs}` and the Rust tests that name the word. +2. D1-D2 in the worklet JS and its two declaration twins. +3. D4 in `sdk/src/**`, `sdk/test/**` and the regenerated mirrors. +4. D6 in the qualification harness and the scripts listed above. + +## Authorized paths + +- `hosts/host-web/src/{lib.rs,ffi.rs,tests.rs}`, `hosts/host-web/tests/control_parity.rs` (only + where it boots with the word), `hosts/host-web/tests/browser-v1/**`, + `hosts/host-web/qualification/**`, + `hosts/host-web/web/miso-engine-v1-audio-worklet{,-host}.js`, + `hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts`, `sdk/src/browser/shipped-host.d.ts` + (byte-identical twins). +- `crates/host-core/src/prepare.rs`, `crates/host-core/tests/**` (callers of the request field), + `crates/control-plane/src/{compile.rs,control.rs}`. +- `sdk/src/{core,browser,generated}/**`, `sdk/test/**`, `sdk/assets/miso-engine-v1-abi-layout.json`, + `sdk/assets/miso-engine-v1-parameter-metadata.json` only if the generator moves it. +- `tools/parameter-metadata/src/abi_layout.rs`, `tools/parameter-metadata/tests/abi_layout.rs`, + `scripts/check-abi-layout-v1.py`, `scripts/fixtures/abi-layout-v1-self-test.json`, + `scripts/test-web-audioworklet.mjs`, `scripts/web-mixing-automation-benchmark.mjs`, + `scripts/check-browser-expected-resources.py`, `docs/EFFECT_OBSERVATION_V1.md`. + +## Non-goals + +- No new master behaviour: the `master` row, its monitor overlay, `sync_master` and the meter + header's `masterTrackPlusOne` keep their #1683 meaning. +- No change to the 64-byte block size or to any other boot word. +- No master reading on the C ABI (#1690). +- No change to the memory pins in `expected.json` unless the measurement moves; if it does, the + PR records the measured numbers. + +## Objective gates + +Commands: `cargo test --locked -p host-core --all-features`, +`cargo test --locked -p control-plane --features test-support`, `cargo test --locked -p capi`, +`cargo test --locked -p host-web --all-features`, +`cargo test --locked -p parameter-metadata`, `python3 -B scripts/check-abi-layout-v1.py` and its +self-test, `bash scripts/check-sdk-generated.sh `, `bash scripts/check-sdk-types.sh`, +`cargo run --locked --release -q -p realtime-policy`, and the worklet gates of #1534 gate 3 +(`build-web-audioworklet.sh --named-twin`, `check-web-audioworklet.sh`, +`test-web-audioworklet.sh`). + +1. **Boot with the word at zero; the row sets the master; the reading is today's.** In + `hosts/host-web/src/tests.rs`, a session with a compressor track and observation capacity boots + with `reserved2 == 0`, the app writes `SET_CONTROL master` for the track, arms the tap, and the + header's `masterTrackPlusOne` and the master gain word equal the values the + boot-word designation gives on the base (`structural_apply::a_master_designated_after_an_empty_boot_reads_as_a_boot_designation` + already asserts the bit-for-bit equality with the track's own word; this gate keeps it and + removes the boot-word half). In the browser qualification, `masterMatchesTrack` and + `masterPresent` (`qualification.js:733-736`) hold, and `armedMasterEqualsTrack` in + `expected.json` stays `true`. +2. **The reserved field refuses.** A native boot with `reserved2 != 0` returns + `RESULT_REFUSED_OPTIONS` with the reason `web.options.reserved2`; the worklet's + `writeBootOptions` and the host's `validBootOptions` refuse it in `test-web-audioworklet.mjs`; both + refuse for the values `1n` and `0xffff_ffff_ffff_ffffn`. Boot with `0n` is accepted. +3. **Mirrors current.** `check-abi-layout-v1.py` and its self-test pass with the field named + `reserved2` at offset 56; `check-sdk-generated.sh` and `check-sdk-types.sh` pass; the two + declaration twins are byte-identical; the SDK exposes no `masterTrackPlusOne` boot option and + writes `reserved2` as `0n`. +4. **Nothing else moved.** The memory pins, the meter frame bytes and `control-list.json` are + unchanged (the PR records a base run). + +## Test value + +- Gate 1 turns red if the master designation or its reading depends on the boot word, or if the + row path gives another value than the word did. +- Gate 2 turns red if a writer that still sets the retired offset is accepted and ignored, so a + later word at offset 56 would silently inherit an old writer's value. +- Gate 3 turns red if the generated SDK, the layout JSON or a declaration twin still names the + word, so the SDK writes a field the module refuses. +- Gate 4 turns red if the retirement moves a published browser byte. + +## Dependencies + +- #1683 (the `master` row and `sync_master`). It must land first; this slice rebases after it. +- *Boot an empty session on both hosts* (#1682) landed before #1683. +- `hosts/host-web/src/lib.rs` and the worklet files are shared with the open browser streams + (#1332, #1381, #1588); this slice rebases after any of them that edits the same lines. + +## Size + +Half a day. + +## Amendment (controls coordinator, batch 15a fold, 2026-10-16) + +- **Pin moves: the non-goal governs.** The non-goal allows a memory pin in `expected.json` to move + when the measurement moves, with the measured numbers in the PR. Gate 4 now reads: the meter + frame bytes and `control-list.json` are unchanged; each memory pin in `expected.json` either + equals the base run or moves to the measured value, with the base and measured numbers and the + reason in the PR. No other published browser byte moves. Gate 4's test value line reads the same. +- **Dependencies.** #1381 is closed, so it is not an open stream to rebase after. The open browser streams are #1332 and #1588. #1683 is closed: the `master` row + and `sync_master` are on `main`. diff --git a/.github/ISSUE_SPECS/1690-read-the-master-strip-meter-on-the-c-abi.md b/.github/ISSUE_SPECS/1690-read-the-master-strip-meter-on-the-c-abi.md new file mode 100644 index 000000000..b9c833af6 --- /dev/null +++ b/.github/ISSUE_SPECS/1690-read-the-master-strip-meter-on-the-c-abi.md @@ -0,0 +1,190 @@ +# Read the master strip's meter on the C ABI + +Successor of #1683 (the master strip is a core `master` monitor row), decision 17 C1 and C13 +(`docs/rulings/one-control-abstraction-2026-10-13.md`: a behaviour that exists on one host only is +a defect). Code anchors verified on branch `codex/controls-batch-14aw` at `5ef2c81d8` (the #1683 +tree). The specs of #1581, #1582, #1583 and #1587 are read as frozen text: none of their code is +on that tree yet. Issue number: assigned at filing (written here as #1690). Feature sprint mode +(owner, 2026-10-13): one implementer, normal tests, a batch review. + +## Product outcome + +A mobile app that designates a master strip with the `master` row reads that strip's gain +reduction, the reading the browser reports as `masterGrDb`, without naming the strip again. The +app gets it from the same meter batch as its other meters. Both hosts compute the reading with one +core function, from the same windows, so the same session gives the same bits. When no strip is +the master, or the master has no fresh window, the reading is absent on both hosts (`null` in the +browser, an invalid record on the C ABI), never `0`. + +## Context + +- **Where the browser computes it.** In `AudioWorkletEngineHost::poll_meters` + (`hosts/host-web/src/lib.rs:3601`), from the gain-reduction fold that runs after the track + fold: + - each armed observation tap's latest window since its arm (`window_since_arm`, `:4465`) + becomes a decibel magnitude with `observed_decibels` (`:4559`) and folds, as a maximum, into + the frame's per-strip gain word at `gain_base + strip` (`gain_base = 2 * strips + 2`); + - the master slot at `gain_base + strips` copies the designated strip's word, but only when that + strip is `observation_present` (`:3969-3995`); then `master_gr_present` is set in the header + (`WebMeterHeader::master_gr_present`, `:977`; read as the header's word at byte 44 by + `direct-oracle.mjs:443` and `sdk/src/core/boundary.ts:1272`), so absence is `null`, not `0`. + - The designated strip is `PlanShape::master_track` (`:1398`), resolved after every control + command by `ReadyOwnership::sync_master` (`:1655-1675`) from the core's master + (`SessionState::master_strip`, `crates/control-plane/src/control.rs:2966`). +- **The C ABI has no master reading.** It has no meter-batch strip or gain-reduction entry yet + (see the dependencies), and no way to name the master in a meter table. +- **What the planned slices already move.** + - #1582 moves the whole fold above into `host_core::MeterDelivery` in + `crates/host-core/src/meters.rs`, including `observed_decibels` (as `reduction_db`), + `window_since_arm`'s predicate and the master slot: its `MeterFrame` carries + `strip_gain_reduction_db`, `strip_gain_reduction_present` and + `designated_gain_reduction_db`. So the computation itself is **not** moved by this slice. + - #1583 gives the C ABI a meter table of `MeterEntry { target: Output | Strip(id), metric: + Peak | GainReduction }` and refuses a gain-reduction entry `UNAVAILABLE` until #1587. + - #1587 delivers `{strip, gain reduction}` records on the C ABI from `MeterDelivery` + (`{h, Aggregate, VALID, db}` when the strip's reading is present, else + `{h, Aggregate, 0, 0.0}`). +- **#1581 is not the home.** #1581 gives every plan the strip `SAMPLE_PEAK` meters and the output + peak (`meter_selection`, `OutputPeakReader`). Those are peaks. `masterGrDb` is gain reduction + from the observation taps, which #1584 and #1585 arm and publish. The C ABI delivery of both + is #1587's, so #1587 is the home, and this slice builds on it. +- **What is still missing after those slices.** + - `MeterDelivery::new(.., designated_strip: Option)` (#1582 D1) fixes the designated strip + by index at construction. #1683 made the designation a row the app writes at any time, held + by strip ID, so a constructor argument cannot follow it. The browser works around that with + `sync_master`; the C ABI has no equivalent. + - A C ABI table entry can name a strip by ID, but an app that wants "the master" would have to + read the control list, find the strip whose `master` row is on, configure `{strip, gain + reduction}`, and reconfigure on every move. The entry does not follow the row. + +## Decisions frozen for this slice + +- **D1. One core function.** `host_core::MeterFrame::master_gain_reduction(&self, master: + Option) -> Option` in `crates/host-core/src/meters.rs`: `Some(db)` exactly when + `master` names a strip of the frame whose `strip_gain_reduction_present` is true, with that + strip's `strip_gain_reduction_db`; else `None`. `MeterFrame::designated_gain_reduction_db` and + the constructor's `designated_strip` argument (#1582 D1-D2) are deleted. The browser's master + slot and the C ABI's master record both come from this function. No decibel logic is written + twice. +- **D2. The master is read from the core row, by strip ID, at each plan.** The control half + resolves the master's strip index against the adopted plan's strip list at every adoption and + after every `SET_CONTROL master` write, from `SessionState::master_strip()`, as #1683's + `sync_master` does for the browser. The browser keeps `PlanShape::master_track` as the resolved + index it passes to D1; it stops owning any gain-reduction logic (its block at `lib.rs:3969-3995` + is deleted by #1582 and replaced by one call). The C ABI keeps the resolved index with its + table. A removed master strip ends the designation (#1683), so the reading is absent. +- **D3. A master table entry on the C ABI.** The meter table (#1583 D1) gains the target + `Master`: `MeterTarget::{Output, Strip(Box), Master}`, wire `target` code 3 with no + `strip_id`. It is valid with the metric `GainReduction` only; `{master, peak}` and + `{output, gain reduction}` are refused `INVALID_FIELD`. A `{master, gain reduction}` entry's + handle is its 1-based table index, like any entry, and never moves. Its record follows #1587 + D3: `{h, Aggregate, VALID, db}` when D1 returns `Some`, else `{h, Aggregate, 0, 0.0}`; in a + fold of `k` frames the value is the last frame's reading that is present, as for a strip. An + entry stays in the table when no strip is the master. +- **D4. The browser meter frame is unchanged.** Its master word, `master_gr_present`, + `masterGrDb` and the SDK's `null` stay bit for bit what #1683 leaves. +- **D5. The wire and the docs.** The schema `meter_entry` field `target` gains code 3; the + conformance corpus and `COMPLETE_SCHEMA_HASH` move with it. `docs/CONTROL_PROTOCOL_REGISTRY.md` + (the meter-enum rows), `docs/C_ABI_V1_QUALIFICATION.md` and the header prose + (`crates/capi/include/miso_engine_v1.h`) state D3. No C symbol or struct changes. +- **D6. The acked-batch question.** A configure is checked whole before it is stored (#1583 D8). + A master record is lossy telemetry like any other: a window that never reaches a batch is + counted by `MeterWindowsLost`. + +## Deliverables + +1. D1 in `crates/host-core/src/meters.rs` (and its re-export), D2 in + `crates/control-plane/src/control.rs` and `hosts/host-web/src/lib.rs` (the call and the + resolved index only). +2. D3 and D5 in `crates/protocol` (schema, codec checks, corpus, hash), the meter staging of + `crates/control-plane`, and the docs. +3. The tests below. + +## Authorized paths + +- `crates/host-core/src/meters.rs`, `crates/host-core/src/lib.rs` (re-export), + `crates/host-core/src/control_provider.rs`, `crates/host-core/tests/meters.rs`. +- `crates/control-plane/src/{control.rs,compile.rs}`. +- `crates/protocol/**` (schema, message wire, controller checks, conformance corpus, schema hash). +- `crates/capi/src/runtime/{tests.rs,live_tests.rs}`, `crates/capi/tests/resource_lifecycle.rs`, + `crates/capi/include/miso_engine_v1.h` (prose), `docs/C_ABI_V1_QUALIFICATION.md`, + `docs/CONTROL_PROTOCOL_REGISTRY.md`. +- `hosts/host-web/src/{lib.rs,tests.rs}` (the master call and the resolved index), + `hosts/host-web/tests/control_parity.rs`. +- The regenerated SDK files, only as `scripts/check-sdk-generated.sh` requires. + +## Non-goals + +- No change to the observation taps, their arm rule (#1534, #1584, #1585) or the reduction + formula (`reduction_db`). +- No master peak. The master has no structural bus; its peak is the strip's own entry. +- No change to #1683's row, the boot word (#1689) or the browser's lease frame bytes. +- No SDK API beyond the generated table type; a later issue gives the SDK a `masterGainReduction` + call on the batch. + +## Objective gates + +Commands: `cargo test --locked -p host-core --all-features`, +`cargo test --locked -p control-plane --features test-support`, `cargo test --locked -p protocol`, +`cargo test --locked -p capi`, `cargo test --locked -p host-web --all-features`, +`cargo run --locked --release -q -p realtime-policy`, `bash scripts/check-sdk-generated.sh `, +`bash scripts/check-capi-abi.sh`. + +1. **Parity row.** In `hosts/host-web/tests/control_parity.rs`, one session with a compressor + insert on `t1` and a submix `bus`, played through both hosts: write `master` on `t1` with + `SET_CONTROL`, arm the compressor's reduction tap, feed the same PCM, and take the same tap + windows. The browser leg reads `masterGrDb` from its lease frame; the C ABI leg configures + `[{master, gain reduction}]` with a period equal to the meter window and reads the record. Per + window the two values are equal bit for bit, and present together. The row then moves the + master to `bus` (a strip with no armed tap) and both read absent in the same window, then + clears it and both read absent. The row asserts the first value is nonzero, so it is not + vacuous. +2. **Absent is not zero.** Core unit test: `master_gain_reduction` returns `None` for `master: + None`, for a master strip with no fresh window (before its first window since its arm, and + after a disarm), and `Some(0.0)` for a master whose tap is armed and reducing nothing. The C + ABI record flags are `0` for the first group and `VALID` with `0.0` for the last. +3. **The entry follows the row.** A capi test configures the table once, then writes `master` on + `t1`, on `t2` and on `t1` again, adds a track before the master (a rebuild), and removes the + master strip. The handle never moves; the records read the master's own reduction at each + step, and read absent from the removal on. +4. **Refusals.** `{master, peak}` and `{output, gain reduction}` refuse `INVALID_FIELD` and keep + the stored configuration; a `{master, gain reduction}` entry on a session with no master is + accepted. +5. **Realtime.** `bench_support::alloc` shows `allocations == 0 && frees == 0` around render and + the delivery's poll on both hosts with a master designated and an armed tap, after warm-up + (the capi race test of `crates/capi/tests/resource_lifecycle.rs`, extended with this table). +6. **Browser bytes unchanged.** The existing master tests pass unchanged, among them + `structural_apply::a_master_designated_after_an_empty_boot_reads_as_a_boot_designation` and + the `masterGrDb` cases of `scripts/test-web-audioworklet.mjs`; the browser qualification's + `masterMatchesTrack` holds. +7. **Schema.** The protocol corpus and `COMPLETE_SCHEMA_HASH` are regenerated and the conformance + suite passes. + +## Test value + +- Gate 1 turns red if either host keeps a master computation of its own, reads another window, or + reports `0` where the other reports absent. +- Gate 2 turns red if absence collapses to `0`, so an app cannot tell "not reducing" from "no + reading". +- Gate 3 turns red if the entry is bound to an index or a strip that was master once, so it + reports a stale strip after a move, an add or a removal. +- Gate 4 turns red if a meaningless entry is stored and then silently reports nothing. +- Gate 5 turns red if the master read allocates on the render thread. +- Gate 6 turns red if moving the master slot into the core changes a published browser byte. + +## Dependencies + +- #1587 (*Report strip peaks and gain reduction to C ABI apps from the core's meter fold*): the + home of the C ABI's gain-reduction record and of the `MeterDelivery` adoption step. This slice + lands after it. It brings its own dependencies: #1583 (the meter table and batch), #1582 + (`MeterDelivery` and the browser's move onto it), #1584 and #1585 (observation capacity on C ABI + plans and the arm state in the slot). +- #1581 is a dependency of the above, not of this slice: it supplies the strip and output peaks + but no gain reduction, so it does not host this reading. +- #1683 (the `master` row and `master_strip()`). +- #1588 serves the same batches in the browser; this slice does not need it (gate 1 reads the + browser's lease frame), and after #1588 the browser reads the same batch record as well. + +## Size + +Half a day. diff --git a/.github/ISSUE_SPECS/1694-diagnose-simd4-on-avx2-refusing-the-nine-track-console-fixture.md b/.github/ISSUE_SPECS/1694-diagnose-simd4-on-avx2-refusing-the-nine-track-console-fixture.md new file mode 100644 index 000000000..e7f6f1ec3 --- /dev/null +++ b/.github/ISSUE_SPECS/1694-diagnose-simd4-on-avx2-refusing-the-nine-track-console-fixture.md @@ -0,0 +1,220 @@ +# Diagnose Simd4 on AVX2 refusing the nine-track console fixture + +Found by #1344's implementer (batch 15a). Code anchors verified on the batch 15a branch +`codex/controls-batch-15a` at `f74f39951`, on an AVX2 x86-64 host (`avx2` set in `/proc/cpuinfo`; +the workspace pins `+avx2,+fma` in `.cargo/config.toml`). Issue number: assigned at filing +(written here as #1694). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a +batch review. + +## Product outcome + +The statement "a console slot always banks, on every target and for every track count" (decision +12; AGENTS.md) and the doc comment that says `console.slot.unbanked` is "unreachable" for a valid +session are either both true of everything the engine runs, or one of them is corrected. A reader of +`crates/graph-compiler/src/banks.rs` learns the exact condition under which a valid session is +refused, and the refusal that a caller sees names that condition. + +## Context + +**Reproduction (done for this spec, then removed).** A throwaway test +`crates/graph-compiler/tests/probe_tmp.rs` ran the production pipeline (`parse_session_json` -> +`compile_session` -> `prepare_native_session_effects` with `launch_native_effect_registry` -> +`prepare_session_builtins` -> `GraphCompiler::compile_with_builtins`) over the unmodified +`fixtures/session/v1/parametric-eq-nine-track.json` once per backend. It copied the caps and the +steps of `compile_bind_render` (`crates/graph-compiler/tests/bank_levels.rs:247`). Command: + +``` +cargo test -p graph-compiler --test probe_tmp -- --nocapture +``` + +Output (the probe file is deleted; `git status` is clean apart from this spec): + +``` +PROBE Scalar: ok, 0 bound slots +PROBE Simd4: refused [("console.slot.unbanked", "$.console.pre_insert[slot=eq].bank[pool=stereo,level=2]")] +PROBE Simd8: ok, 2 bound slots +``` + +The same probe over `fixtures/session/v1/console-sixty-four-track-intended.json` gives the same +refusal at `Simd4` (`$.console.pre_insert[slot=eq].bank[pool=stereo,level=2]`) and 24 bound slots +at `Simd8`. So nine tracks are not special: no track count passes at `Simd4` on this build. Nine is +only the first fixture someone ran at the foreign width. (Nine tracks do pad: eight lanes bind eight +tracks and one padded bank of eight takes the ninth, which is why `Simd8` reports 2 bound slots for +the one `eq` slot.) + +**Where `Backend::Simd4` can be selected on an AVX2 build.** + +- `lane::Backend` (`crates/lane/src/backend.rs:26`) always has `Simd4`; `Simd8` exists only with + `avx2`. `Backend::current()` (`:49`) is a compile-time constant: `Simd8` where `avx2` is set. The + doc on `Backend::VECTOR` (`:74`, the constant at `:80`) says tests iterate it so each build runs every vector width it + has. +- `GraphCompiler::compile_with_builtins` takes `dispatch` in its public request + (`GraphBuiltinsCompileRequest`), so any caller may pass `Simd4`. Every production caller passes + `Backend::current()`: `crates/host-core/src/prepare.rs` (the calls at `:1246`-`:1476`; the + `*_with_backend` variants at `:1673` and `:1714` are `pub(crate)`), and no file under `hosts/`, + `crates/capi/src` or `crates/control-plane/src` names a `Backend` variant (grep: no hit). +- The only non-test-only selections of a foreign width are bench and test code: `tools/ + console-workload/src/lib.rs:1284-1330` (`build_with_dispatch`; it folds the console into every + track's inserts first when `dispatch != Backend::current()`, its doc says why: "a console slot + never renders per node on a vector backend, so the console as written is refused there"), + `crates/host-core/src/limiter_linked_session.rs:300` (`folded(backend)`, same fold, tests at + `:686`), and `crates/graph-compiler/tests/bank_levels.rs:1077-1087` (`FOREIGN_CONSOLE_REFUSAL`, + `compiled_at`: a foreign width may be refused with exactly `graph console.slot.unbanked`). + +**The planner path that emits the code.** + +1. `bind_rack_banks_indexed` (`crates/graph-compiler/src/banks.rs:137`) collects every console and + insert chain, forms cohort groups, padded for console racks, and for each slot calls + `entries[0].factory.bind_homogeneous_bank(request)` (`:641`). `request.width` is + `BankWidth::for_backend(dispatch)` (`:208`), so `Four` for `Simd4`. +2. `let Some(...) = ... else { continue; }` (`:642-646`) treats `Ok(None)` as "the factory declined": + the slot stays unbound. +3. `unbanked_console_slot` (`:505-540`) then finds the first console node in a collected chain that + no bound slot carries and returns `console.slot.unbanked` at + `$.console.
[slot=].bank[pool=,level=]`. `uncollected_console_slot` + (`:548-586`) is the twin for a node the walk never collected. +4. The factory answers `Ok(None)` because it binds only the build's own width. For the EQ: + `ParametricEqFactory::bind_homogeneous_bank` (`crates/parametric-eq/src/lib.rs:3567`) calls + `bind_bank(request, Backend::current().width())`, and `bind_bank` (`:3581-3592`) returns + `Ok(None)` when `request.width.lanes() != native_lanes`, with the comment "a width this build + does not execute is a capability gap and a legal `Ok(None)`". The compressor + (`crates/compressor/src/lib.rs:1013`), the gate/expander (`crates/gate-expander/src/lib.rs:967`), + the transient shaper (`crates/transient-shaper/src/lib.rs:924`) and the soft clip + (`crates/soft-clip/src/lib.rs:1133`) test the same `Backend::current().width()` condition. The + comment above `folded` in `limiter_linked_session.rs:290-299` states it as the rule: "a build's + factories bind only the widths it executes (D4)". + +**Why the code comment is wrong as written.** `crates/graph-compiler/src/banks.rs:96-101` says +"for a valid session that is unreachable" and `unbanked_console_slot`'s doc (`:491-504`) says the +same, "it guards a regression in the planner, the binder or a factory". The comment holds only for +`dispatch == Backend::current()`. At a foreign vector width the first factory that declines the +width makes the code fire for every valid session with a console. The doc at `:502-504` names only +the `Scalar` oracle as exempt, and `docs/SESSION_SCHEMA_V1.md:230-237` says "on every vector +width". The guard, the tests that pin it (`crates/graph-compiler/src/tests/console_banking.rs:737` +and `:858`, `crates/graph-compiler/src/lib.rs:5351` and `:5914`) and `bank_levels.rs` all treat the +code as a regression guard or as the accepted foreign-width refusal; none gives the foreign-width +case its own code. + +## Decisions to take (the implementer proposes, the coordinator rules) + +- **Gate 1 verdict.** Which of two cases this is. Preliminary reading of the planner owner, to be + confirmed or overturned by the implementer with evidence: + - **Test-only mode (the likely case).** Nothing in production selects `Simd4` on an AVX2 build: + the compile-time `Backend::current()` is the only dispatch every host path passes, and the + engine's rule is one lane width per build (no runtime SIMD dispatch; lane widths keyed on + target features). Every console-eligible native factory declines a width other than the + build's. The only callers that reach the refusal are the fold-first bench and test helpers + above, which already treat it as expected. The defect is then the diagnostic and the comment: + a foreign-width refusal reports `console.slot.unbanked`, which reads as "the planner lost a + slot", and the comment says it cannot happen. + - **Planner defect (the case to rule out).** It would hold if the engine must run `Simd4` on an + AVX2 build: for example if any product path (the browser build is `simd128`, not AVX2, so not + that one) or an owner decision ("two shipped bank widths", the bench's `Simd4` leg, the + #968 resource fixtures) requires a four-lane console plan on an eight-lane build. The + implementer greps for such a requirement and records the answer. Evidence against: the + `limiter` binds four-lane banks on `x86-64-v3` by design, but it is not a console-eligible + slot in any fixture here, and the console slots (EQ, compressor, ...) decline. +- **If a test-only mode: where the typed refusal lives.** The check belongs before any slot is + tried, once, in `bind_rack_banks_indexed`: a vector `dispatch` whose `BankWidth` is not + `BankWidth::for_backend(Backend::current())` and a session with at least one console slot is + refused with its own code (proposal: `graph.backend.foreign_bank_width`, path `$.dispatch`), + distinct from `console.slot.unbanked`. The `console.slot.unbanked` guard stays for the build's own + width, where it is a true regression guard, and its comment is corrected to say so. A session + with no console slots compiles at a foreign width as today (insert remainders render per node). + The implementer states whether the check keys on `Backend::current()` or on a `bank_widths` + capability the factories publish; the proposal is `Backend::current()`, because that is what the + factories test. +- **If a planner defect: the fix shape.** The factories bind the requested width when the build can + execute it (`Four` on any vector build; `Eight` only with `avx2`), and the tests that expected the + refusal at a foreign width are deleted in the same issue. + +## Deliverables + +Common to both cases: + +1. Gate 1's verdict, with its evidence, in the decision record below (this section is filled in by + the implementer). +2. The probe above recorded as a committed test only where it defends something the existing tests + do not (see Test value); no probe file is left behind. + +If a test-only mode: + +3. A typed refusal with its own code for a foreign vector width with a console, from + `bind_rack_banks_indexed` (or one level up, at the compile entry, if the implementer shows the + compile entry already knows the build's width); and wherever the new code is listed (the schema document's Banking paragraph names the old one). +4. The comments in `crates/graph-compiler/src/banks.rs` (`:97-104`, `:491-504`) corrected to name + the condition: `console.slot.unbanked` is unreachable for a valid session at the build's own + width, and a foreign width is refused earlier with the new code. +5. `docs/SESSION_SCHEMA_V1.md` (the Banking paragraph at `:230-237`) and `bank_levels.rs` + (`FOREIGN_CONSOLE_REFUSAL`, its module doc at `:25-31`) follow the new code. The `console-workload` + and `limiter_linked_session` fold helpers keep their fold. + +If a planner defect: + +3. The fix named in Decisions, and a regression test: the nine-track fixture prepares and banks + every console slot on `Simd4` and on `Simd8`, bit-identical per lane to the scalar reference, as + the existing parity tests (`bank_levels.rs`, "banking moved a rendered bit") require. +4. The comments in `banks.rs` and the schema document updated to say the width is no longer + build-bound. + +## Authorized paths + +- `crates/graph-compiler/src/banks.rs`, `crates/graph-compiler/src/lib.rs` (its tests) and `crates/graph-compiler/src/tests/console_banking.rs`; + `crates/graph-compiler/tests/bank_levels.rs`. +- `docs/SESSION_SCHEMA_V1.md` (Banking paragraph) and any other place the new code must be listed + (`grep -rn console.slot.unbanked docs` finds only that paragraph today). +- If a planner defect only: the console-eligible native factories' `bind_homogeneous_bank` + (`crates/parametric-eq`, `crates/compressor`, `crates/gate-expander`, `crates/transient-shaper`, + `crates/soft-clip`) and their padding tests. +- This spec file's decision record. + +Nothing under `hosts/`, `crates/capi`, `crates/host-core` or `tools/` changes in the test-only case. + +## Non-goals + +- No second console bank width on an AVX2 build as a product feature, and no runtime SIMD dispatch. +- No change to what `Backend::current()` selects or to which widths a build has. +- No per-node fallback for a console slot on a vector backend (decision 12 stands). +- No change to the `Scalar` oracle's exemption. +- No benchmark or optimisation work for a mode production does not run. + +## Objective gates + +1. **Verdict recorded.** The decision record states "test-only mode" or "planner defect", with the + grep evidence for who selects a foreign width in production (none, expected) and the reproduction + output above re-run on the implementer's tree. +2. **If a test-only mode:** at a foreign vector width, a valid session with a console is refused + with the new code and not with `console.slot.unbanked`, at the build's own width every console + slot still binds, and the comments in `banks.rs` match. `cargo test -p graph-compiler`, + `-p host-core --all-features`, `-p console-workload` green; `bank_levels.rs`'s `compiled_at` + expects the new code. +3. **If a planner defect:** the nine-track fixture prepares and banks every console slot at `Simd4` + and at `Simd8`, with no refusal, bit-identical per lane to the scalar reference; the existing + parity tests pass unchanged; the tests that expected the foreign-width refusal are gone. +4. `cargo clippy -D warnings` and `cargo fmt --check` on the touched crates; the realtime policy + check; `scripts/check-workspace-policy.sh`. + +## Test value + +- Test-only case: a test that compiles the nine-track fixture at the foreign vector width and + asserts the new code (and that the build's own width compiles). Plausible defect that turns it + red and no existing test catches: the foreign-width refusal falling back to + `console.slot.unbanked`, or the new check firing at the build's own width. The existing tests + accept `console.slot.unbanked` at a foreign width (`compiled_at`) and so cannot see either. + On a 4-lane (NEON/simd128) build there is no foreign vector width, so the test is a no-op there + and says so. +- Planner-defect case: the regression test is exempt from the new-test rule while its PR records it + red on the fix's revert, and it defends the nine-track fixture banking at both widths. The + implementer deletes the tests that pinned the old refusal. + +## Dependencies + +- None. Independent of #1344, which found it and does not need it. + +## Size + +Half a day. + +## Decision record: #1694 + +To be filled by the implementer. Gate 1's verdict goes first. diff --git a/.github/ISSUE_SPECS/1695-read-the-browsers-armed-observation-taps-from-the-core.md b/.github/ISSUE_SPECS/1695-read-the-browsers-armed-observation-taps-from-the-core.md new file mode 100644 index 000000000..378bfa32a --- /dev/null +++ b/.github/ISSUE_SPECS/1695-read-the-browsers-armed-observation-taps-from-the-core.md @@ -0,0 +1,183 @@ +# Read the browser's armed observation taps from the core + +Successor of #1679 (the successor's armed bits come from the core's mask) and #1677 (observation +records addressed by stable key in the core), decision 17 C1 and C13. Code anchors verified on the +batch 15a branch `codex/controls-batch-15a` at `e8f72edc5`, and read-only on the in-progress #1310 +tree `codex/controls-batch-15aw` (uncommitted). Issue number: assigned at filing (written here as +#1695). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. + +## Product outcome + +The browser holds no copy of which observation taps are armed. The core already knows: the control +plane's `SessionState` keeps one observation slot per declared tap (armed or not, and whether a +record waits in the queue of a plan render has not adopted) and the armed bindings by tap key. The +browser reads that state wherever it needs it (the meter fold, the window read, the swap's tap +hand-over, the adoption of a replacement), so a subscription behaves the same after a rebuild, a +supersession and an adoption because one owner holds it. Decision 17 C1: "A behaviour that exists +on one host only is a defect"; a second copy of the armed state that only the browser keeps and +re-derives at every plan change is host logic. + +## Context + +- **The copy.** `PlanShape` (`hosts/host-web/src/lib.rs`) holds, per plan and per dense effect + slot, `observation_armed: Box<[u32]>` (`:1390`, one bit per declared tap) and + `observation_commanded: Box<[u32]>` (`:1402`, a bit when a tap command was admitted against this + shape's plan), beside `observation_arm_samples` (`:1396`, the admission sample of the current + arm). They are written in three places: + 1. `record_observe` (`:2003`): at admission of a record the control plane has just queued, it + sets or clears the armed bit and sets the commanded bit; + 2. `build_plan_shape` (`:5716`, the loop at `:5846`-`:5906` of that function): a fresh shape takes + `EffectObservationHandle::armed_taps` (`crates/effect-compiler/src/prepare.rs:1804`), which + #1679 made the core's mask, copied "not repeated"; the commanded bit starts clear; + 3. `PlanShape::carry_tap_state` (`:6005`, called at the swap from `:1832`): for each carried + reader, a tap whose commanded bit is clear takes the predecessor's armed bit and arm sample; + a tap commanded against the successor keeps its own. Then the bits are cleared. +- **The readers of the copy.** The meter fold (`:3925`-`:3950`, skips a slot with `armed == 0`), + `read_observation_values` (`:4493`-`:4512`, answers `Unarmed` from the bit), and the test hook + `observation_armed_taps` (`:2700`, summed popcount; the tests call it at `tests.rs:2493`, + `:4333`, `:7554`, `:7581` and others). +- **What the core holds.** `SessionState` (`crates/control-plane/src/control.rs`) keeps + `observation_pending: Box<[ObservationSlot]>` (`:426`; `ObservationSlot` at `:670`: `armed`, + `pending`, `mark`, `key_bytes`), an armed count (`observation_armed()`, same file), and the + armed bindings by key (`observation_bindings()`, `:3019`, `controls::ObservationBindings` with + `window_of(&TapKey)`). Admission (`admit_and_push_observation_batch`) is the one way a host arms + or disarms a tap (#1676), and the browser's `record_observe` runs after it, so the two copies are + written by the same call and held equal by that sequence, not by one owner. +- **#1310 adds a third writer (verified on the batch 15a fold branch `codex/controls-b15a-fold` + at `804fe6939`).** `ObservationSlot` gains `queued` (`control.rs:802`: "a record for this tap + waits in the queue of a plan render has not adopted"), and `observation_forwards` + (`control.rs:869`) re-queues, on the replacement B, the bound state of every tap whose slot is + `queued` on a withdrawn candidate A, because a tap B carries from the running plan P0 takes P0's + state at the swap and would lose the record. The control plane keeps the result as + `SessionState::observation_forwarded()` (`control.rs:3657`; `ObservationForward`, `:843`). The + browser copies it: after a committed rebuild, `run_command` (`hosts/host-web/src/lib.rs:1637`- + `:1648`) calls `PlanShape::note_observe` (`lib.rs:5952`) on B's `PlanShape` for each forward. + That loop is a fourth writer of the copy, and the only reader of `observation_forwarded()`. +- **The rule that decides the carry** is the same in both copies and exists once in the core after + #1310: a tap carried across a swap takes the running plan's state unless a command named it + against the successor. The browser's `observation_commanded` is that "named it" bit; the core's + `queued` is its counterpart for a plan not yet adopted. +- **Tests that pin the copy.** `the_browsers_armed_bits_are_the_mask_host_core_prepared` + (`tests.rs:7890`, #1679), `an_armed_tap_and_its_windows_follow_the_effect_through_a_swap` + (`:7540`), `a_tap_command_admitted_at_sample_zero_survives_the_hand_over` (`:7607`), + `a_restarted_effects_subscription_keeps_publishing_windows_on_the_browser` (`:7801`), + `a_removed_inserts_subscription_ends_with_an_event_and_has_no_slot` (`:7988`); the qualification + rows `observationTimeline` in `hosts/host-web/tests/browser-v1/expected.json` (`:122`) and the + `resourceCeilings`, driven by `browser-correctness.js`. + +## Decisions to take (the implementer proposes, the coordinator rules) + +- **The core accessor.** What the core returns for the browser's questions: for a producer + position and a tap, `armed` and the window (from the slots and bindings), and for the swap and + the adoption, the state the carry rule resolves. The proposal is one read-only function on + `SessionState` that takes the producer position and tap index and allocates nothing, so the + worklet's message handler and `process()` can call it (no allocation in `process()`). +- **What replaces `observation_arm_samples`.** The arm sample is the transport's freshness stamp + (a resident cell keeps its last window, so a read is fresh only if the window starts at or after + the arm sample). Decide whether it stays a browser-side render-transport value (it is not an + "armed" fact; it is a sample of this host's block clock) or moves with the bits. The proposal + keeps it in the shape as transport, and the issue states why it is not a second copy of + state: it holds a sample number, never a bit that could disagree with the core. +- **The adoption read.** At the swap block the browser must know, for a carried tap, whether a + command named it against the successor. Decide whether the core's `queued` flag (or a sibling + the implementer adds) is that fact, and prove with a test that it equals + `observation_commanded` for every sequence the five tests above drive (command before the + commit, after the commit before adoption, at sample zero, across a supersession). +- **Where the read happens.** The meter fold runs in the worklet's message handler on a + non-isolated page and in the worklet on an isolated one; the issue names which instance may + read the control plane's `SessionState` there (decision 15, D15-10), or the core publishes the + armed words on the plan's render-half state for the worklet to read without a lock. + +## Deliverables + +1. The core accessor of the first decision, with unit tests in `control-plane`. +2. `PlanShape::observation_armed` and `PlanShape::observation_commanded` deleted, with + `record_observe`'s mask and commanded writes, `build_plan_shape`'s copy of + `EffectObservationHandle::armed_taps`, and `carry_tap_state`'s armed-bit merge. The meter + fold and `read_observation_values` read the core. `observation_armed_taps` (test hook) reads + the core's count. +3. The superseding commit's forwards reach the browser through the same read; no browser code + copies `observation_forwards`' result (deliverable 5). +4. The #1679 test is deleted (its subject, a browser copy of the mask, no longer exists); the + others are rewritten only where they named a deleted field. +5. The forward copy in `run_command` (`lib.rs:1637`-`:1648`) is deleted. `note_observe` goes with + the fields it writes. `SessionState::observation_forwarded()` and the stored + `observation_forwarded` field (`control.rs:568`) are deleted if no reader remains (the C ABI has + none); `ObservationForward` stays only if the forwards' own code needs it. + +## Authorized paths + +- `hosts/host-web/src/lib.rs` and `hosts/host-web/src/tests.rs`. +- `crates/control-plane/src/control.rs` and `crates/control-plane/src/compile.rs` (the accessor; + the slot tables' sizing only if the accessor needs a new field), and their tests. +- `crates/host-core/src/prepare.rs` and `crates/effect-compiler/src/prepare.rs` only if + `EffectObservationHandle::armed_taps` has no remaining reader and is deleted. +- `hosts/host-web/tests/browser-v1/expected.json` only for a re-pin the gates below measure, with + its reason. +- This spec file's decision record. + +## Non-goals + +- No change to which taps an effect declares, to the window length, to the observation wire, or to + `OBSERVATION_ENDED`. +- No change to the C ABI: it has no copy to delete, and its tap state already comes from the same + core. +- No move of the meter fold or the window cells (decision 17 C13's later slices). +- No allocation, lock or syscall added to `process()`; no new field behind a `cfg` for one target. + +## Objective gates + +1. The browser's observation tests (`structural_apply` and the `selected_observation_*`, + `observation_*` rows of `hosts/host-web/src/tests.rs`) and the qualification's observation + rows pass unchanged, apart from the deleted #1679 test and tests that named a deleted field. +2. A subscription acked across a plan rebuild, across a supersession (#1310: the second structural + commit while the first waits unadopted) and across the adoption block publishes windows. The + supersession leg is the existing + `a_subscription_queued_on_a_superseded_candidate_publishes_on_its_replacement`, which passes + unchanged with the forward copy deleted; a new test drives the rebuild and adoption legs and + reads at least one window after each, with the subscription taken before the rebuild and no + re-arm. +3. The browser source holds no per-shape armed or commanded state, **proved by type**: the + fields `observation_armed` and `observation_commanded` are gone from `PlanShape`, and no type + in `hosts/host-web/src` has a field of the shape `Box<[u32]>` indexed by dense effect slot that + holds tap bits. The proof is that the crate compiles and the tests above run without them, not + a source grep. +4. `allocations == 0` in `process()` after warm-up for the observation path, measured with + `bench_support::alloc` (the existing `a_bus_observation_admits_and_renders_without_allocating` + passes); the realtime policy check passes. +5. `cargo test -p host-web --all-features`, `-p control-plane --all-features`, + `-p host-core --all-features`; `cargo clippy -D warnings` and `cargo fmt --check` on the + touched crates; `scripts/check-workspace-policy.sh`; the browser qualification observation + rows (run on a machine with the browser matrix; record if not run). + +## Test value + +- The sequence test (gate 2): plausible defect that turns it red and no existing test catches: a + subscription acked while a candidate waits, then superseded by a second structural commit, that + stops publishing after the adoption because the replacement's armed state came from a source + that never saw the queued record. `a_subscription_queued_on_a_superseded_candidate_publishes_on_its_replacement` + (`hosts/host-web/src/tests.rs:7139`, #1310) already drives the supersession leg, through the + copy; gate 2's new test covers the other two legs and keeps that test as the supersession leg, + so after the deletion it proves the core read, not the copy. +- The equivalence test for the adoption read (third decision): red if the core's "named against + the successor" fact differs from the deleted `observation_commanded` bit for any drive of the + five existing sequences; the deleted bit is the oracle, so the test is written before the + deletion and kept as PR evidence, not committed (a one-time comparison against a pre-change + base, per the repository rule). +- The #1679 test is deleted, not rewritten: its claim, that the browser's bits equal the core's + mask, holds by construction once there is one owner. + +## Dependencies + +- #1310 (batch 15a), for the supersession sequence and the forwards; this issue starts after it + merges. +- #1679 (the mask the browser copies today) and #1677 (the stable-key addressing the core's slots + and bindings use). + +## Size + +Half a day. + +## Decision record: #1695 + +To be filled by the implementer. The accessor decision goes first. diff --git a/.github/ISSUE_SPECS/1696-charge-the-monitor-overlays-and-edit-ramps-in-the-capi-row.md b/.github/ISSUE_SPECS/1696-charge-the-monitor-overlays-and-edit-ramps-in-the-capi-row.md new file mode 100644 index 000000000..655e37318 --- /dev/null +++ b/.github/ISSUE_SPECS/1696-charge-the-monitor-overlays-and-edit-ramps-in-the-capi-row.md @@ -0,0 +1,176 @@ +# Charge the monitor overlays and edit ramps in the retained rows + +Successor of #1310 (supersede an unadopted candidate; it adds the kept base) and #1683 (the master +row joined the overlay), decision 17 C1 (one core, one accounting on both hosts). Code anchors +verified on the batch 15a branch `codex/controls-batch-15a` at `e8f72edc5`, and read-only on the +in-progress #1310 tree `codex/controls-batch-15aw` (uncommitted; its names are marked below). +Issue number: assigned at filing (written here as #1696). Feature sprint mode (owner, 2026-10-13): +one implementer, normal tests, a batch review. + +## Product outcome + +The retained-bytes row that bounds the control plane (`capi_retained_bytes` on the C ABI, the +control plane's `control_retained_bytes` that the browser's report reads) charges every byte the +control plane keeps for the session's monitor overlay and for the edit-ramp lengths it holds +between transactions, including the copies a pending successor keeps. An agent that raises the +limit to the byte it measured is never surprised by a heap the row did not count, and the two +hosts charge the same, because the row is computed once, in the core. + +## Context + +- **The row.** `capi_resources` (`crates/control-plane/src/compile.rs:210`) sums fixed allocation + rows (the queues, the replay cache, the observation tables, the binding table), the adapter's + rows (`ADAPTER_ALLOCATIONS`), the epoch rows (control tables, inventory, producers) and the + catalog into `CapiResources { active_retained, epoch_retained, prepared_protocol_retained, + largest }`. The C ABI stores it as `capi_retained_bytes` (`crates/capi/src/runtime/mod.rs:250`, + read back at `:290`); the control plane checks it against `maximum_control_retained_bytes` + (`compile.rs:844`, `:917`, `:1093`). The browser reuses it: `hosts/host-web/src/lib.rs:6254` + adds `resources.control_retained_bytes` to its own bridge rows, so a charge added to the core's + row reaches both hosts without a host edit (the implementer confirms this on the final tree and + adds a bridge row only if a host holds a copy; none was found). +- **The overlay is uncharged.** `SessionState::monitor` (`crates/control-plane/src/control.rs:411`) + holds a `controls::MonitorOverlay` (`crates/controls/src/effective.rs:32`): `soloed: + Vec` (sorted, no duplicate) and `master: Option` (#1683). Each `StableId` is + a `String` of at most `STABLE_ID_MAXIMUM_BYTES = 127` bytes (`crates/session/src/id.rs:7`). A + search of `compile.rs` for `monitor` or `overlay` finds only the boot seed (`:1207`) and the + struct's doc (`:29`): no row charges the vector, its spare capacity or the IDs' heap text. Each + commit also builds the next overlay (`MonitorOverlay::after`, `control.rs:1624` and `:1998`), + a transient copy that lives until the commit moves it into `self.monitor` (`:1858`, `:2351`). +- **Edit ramps are transaction metadata that #1310 makes a retained value.** `EditRamps` + (`crates/session/src/edit_ramp.rs:100`) holds three tables (`:101`-`:106`, verified on + `codex/controls-b15a-fold` at `804fe6939`): `entries: BTreeMap` where + `Key { owner, row }` (`:65`); `unset: BTreeSet`, the rows the latest edit wrote with no + length; and `cleared: BTreeSet` (`Owner`, `:56`), the targets an upsert rewrote whole + (`clear_target`, `:142`). #1310 added the two sets for `overlay` (`:172`). A prepared + transaction carries one (`crates/protocol/src/model.rs:931`) and it dies with the + transaction. Rows: a strip has `Fader`, `Mute`, `Matrix`, `Input`, `Link` (five); a route has + `RouteGain`, `RouteMute`, `RouteMatrix` (three); a VCA has `VcaFader`, `VcaMute` (two); the + console has `Link` (one). So `entries` and `unset` each hold at most `5 x strips + 3 x routes + + 2 x VCAs + 1` keys (each key's owner ID text is at most 127 bytes; `entries` adds a `u32`), and + `cleared` holds at most one `Owner` per strip, route, VCA and the console. The charge covers all + three tables. +- **#1310's kept base.** `KeptBase { model, monitor, edit_ramps, + monitor_ramp }` (`control.rs:78` on `codex/controls-b15a-fold` at `804fe6939`; the other anchors + of this bullet are from the b15aw tree and are re-read by the implementer) is set on the epoch a pending successor succeeds + (`:2380`), extended by every later commit (`:2355`, `:2987`), merged by `merged_ramps` (a clone + of the kept lengths plus the new transaction's), and dropped when the epoch stops being the + running one (`:1745`, `:2391`). Only `model` is charged: `held_models` (`:1651`) returns one + `ResourceEstimate` per kept model, and both admissions add them (#1398 D4). The kept `monitor` + and `edit_ramps` are held and not charged; `monitor_ramp` is a fixed `Option>`. + At most one kept base exists per pending epoch, so the count of held bases is bounded by the + pending-provider list (`PENDING_PROVIDERS`, same file). +- **The oracle that exists.** `capi/tests/resource_lifecycle.rs` counts every allocation of the + compile with a thread-scoped allocator and asserts the exact equality of + `capi_retained_bytes` with its observed terms + (`assert_capi_retained_bytes_are_complete`, `:867`; driver `:995`). It observes the compile of + one session; it never populates a solo, a master or a pending candidate, so a term that the + commit path adds later is outside it. The browser's ceilings are + `hosts/host-web/tests/retained_ceilings.rs` and `tests/browser-v1/expected.json` + (`resourceCeilings`, `:69`). + +## Decisions to take (the implementer proposes, the coordinator rules) + +- **The bounds, stated as ceilings.** The overlay never holds more than one entry per strip plus + the master: `soloed.len() <= strips` (a solo names a strip that exists; a removed strip leaves, + `retain_strips_of`) and `master` is one ID. The ramps never hold more than the row count above. + Both are bounded by the session's admitted caps (`maximum_tracks`, `maximum_submixes`, routes, + VCAs in `ControlLimits`), not by the number of edits, so the charge needs no per-edit growth. + The implementer proves each bound at the write site and adds a test that tries to exceed it. +- **Capacity versus length.** A `Vec` that grows by pushes may hold twice its length. Either the + overlay reserves `strips` entries when a plan is prepared and the charge is exact on that + capacity, or the charge is `2 x strips`. The proposal reserves: the charge is then a layout, and + a solo write never allocates after the reserve. For the `BTreeMap` and the two `BTreeSet`s, the + charge is a per-entry ceiling derived from the layouts of `Key`, `u32` and `Owner` and the leaf + node's 11-entry layout, checked against the allocator, because a B-tree's node count is not + exact. +- **Retained versus peak.** The retained charge is one overlay and one ramp table on the committed + state plus, for each pending epoch, a kept overlay and a kept ramp table. The transients (the + next overlay of `after`, the clone in `merged_ramps`) belong to the peak rows + (`validate_replacement_peak`, `validate_live_peak`), not the retained row. The implementer + names which row each term joins and does not charge a transient twice. +- **Where the held copies are counted.** The kept overlay and ramps join `held_models` (so both + admissions count them with the model they belong to) or the control plane's epoch rows. The + proposal extends `held_models` to a held-base charge, since the lifetime is the same. +- **Does the browser need its own row?** Expected no. The browser keeps the master by strip + index in `PlanShape::master_track` and the meter header (a `u32`), a fixed field. The + implementer states it and charges a bridge row only if a host-side copy of the overlay appears. + +## Deliverables + +1. The overlay and ramp charge in `capi_resources` (and `prepared_capi_resources` if the prepared + protocol row holds a copy), as layouts over the bounds, with the reserve of the second decision + if taken. +2. The kept base's overlay and ramps charged where its model is (after #1310 merges), in both + admissions. +3. Re-pins of every moved row, each measured on the final tree with its reason: the capi + `maximum_capi_retained_bytes` fixtures that sit at a measured floor, the browser + `resourceCeilings` in `expected.json` and the `retained_ceilings.rs` budgets + (`scripts/check-browser-expected-resources.py` measures the browser), and the SDK or fixture + files that carry a figure. Owner decision 4 applies: budgets stay ceilings, not exact counts, + except the `capi_retained_bytes` equality, which stays exact. +4. The allocator oracle of gate 1. + +## Authorized paths + +- `crates/control-plane/src/compile.rs` and `crates/control-plane/src/control.rs` (the charge, + the held-base term, the reserve), with their tests. +- `crates/controls/src/effective.rs` (a reserve or a capacity accessor on `MonitorOverlay`) and + `crates/session/src/edit_ramp.rs` (a ceiling function over the row counts), with their tests. +- `crates/capi/tests/resource_lifecycle.rs`, `crates/capi/src/runtime/live_tests.rs` and + `live_peak_tests.rs` where a floor moves; `hosts/host-web/tests/retained_ceilings.rs`, + `hosts/host-web/tests/browser-v1/expected.json`. +- This spec file's decision record. + +## Non-goals + +- No new row in the C ABI report struct: the charge joins `capi_retained_bytes`. +- No change to what the overlay or the ramps hold, to their wire forms, or to the ramp merge rule. +- No release of the kept base earlier or later than #1310 decides. +- No byte-pinned test of a layout (owner decision 4); no benchmark. + +## Objective gates + +1. **Allocator-measured oracle.** In `crates/capi/tests/resource_lifecycle.rs`, with the + thread-scoped allocator: populate the committed overlay to its bound (every strip soloed, a + master set), populate the ramps to the row bound (a transaction that gives every row of every + owner a length, a transaction that writes every row with no length so `unset` is full, and + one that upserts every owner whole so `cleared` is full), and hold a pending candidate so a kept base holds both again (a structural + commit followed by a live edit). Then the bytes the control plane retains for those values are + `<=` the bytes the charge adds. The comparison names the terms observed (overlay vector + capacity and ID text, the map's and both sets' nodes), not a byte literal. +2. The existing exact equality `capi_retained_bytes == observed terms` still holds on every + fixture, with the new term added to the right-hand side as an observed value. +3. The same session on the browser: `WebResourceReport` reads the same core charge; the + `retained_ceilings.rs` budgets and `check-browser-expected-resources.py` pass with the moved + rows re-pinned from measurement, each with its reason. +4. A control plane at the exact `maximum_control_retained_bytes` that the session needs without + the overlay is refused when the overlay is populated, on both hosts, with the typed resource + code, and accepted with the limit raised by the charge. +5. `cargo test -p control-plane --all-features`, `-p capi --all-features`, `-p controls`, + `-p host-web --all-features`; `cargo clippy -D warnings` and `cargo fmt --check` on the touched + crates; realtime policy check; `scripts/check-workspace-policy.sh`. + +## Test value + +- Gate 1: plausible defect that turns it red and no existing test catches: a populated solo set, + a master, or a kept base's ramp table that holds heap the retained row never counted, so the + `maximum_control_retained_bytes` limit under-reports on a large mixer. The existing oracle + observes one compile with an empty overlay and no pending candidate, so it cannot see any of + the three. +- Gate 4: red if the charge is added to the report but not to the admission that compares it with + the limit (the row moves and the refusal does not). +- The re-pins are measurements, not tests; each records the base and the measured value. + +## Dependencies + +- #1310 (batch 15a): the kept base and its ramp merge. The overlay charge on the committed state + does not need it, but the held-base term does. +- #1683: the master joined the overlay, so the bound is `strips + 1` IDs. + +## Size + +Half a day. + +## Decision record: #1696 + +To be filled by the implementer. The bounds decision goes first. diff --git a/.github/ISSUE_SPECS/1697-export-the-browser-session-replace-transport.md b/.github/ISSUE_SPECS/1697-export-the-browser-session-replace-transport.md new file mode 100644 index 000000000..7d2aac691 --- /dev/null +++ b/.github/ISSUE_SPECS/1697-export-the-browser-session-replace-transport.md @@ -0,0 +1,100 @@ +# Export the browser session replace transport + +Split from #1386 (root ruling 2026-10-16), decision 17 C1 and C13. Code anchors verified on the +batch 15a fold branch `codex/controls-b15a-fold` at `804fe6939`. Issue number: assigned at filing +(written here as #1697). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a +batch review. + +## Product outcome + +A browser app replaces the whole running session with a new document while it plays, as a C ABI +app does. The document can be as large as the boot accepts (1 MiB), not the 16 KiB control frame. +The browser only moves the bytes: the core's `SESSION_REPLACE` path (#1386 (a)) checks, parses, +diffs and commits them, so both hosts give the same reply, revision and rendered bits. + +## Context + +- **The core path.** #1386 (a) adds `SESSION_REPLACE` (`0x0013`) and + `SessionState::command_with_document(request, document, output_capacity)`. Until this slice, + the browser passes no document, so a replace through the browser gets + `session.replace.document_length`. +- **The approved export.** #1386 D7's browser half, root-approved on 2026-10-15: + `miso_engine_web_v1_replace_document_ptr(handle: u32, len: u32) -> u32`, the live twin of + `miso_engine_web_v1_document_ptr` (`hosts/host-web/src/ffi.rs:2959`), on the same + `BOOT_STAGING.document` buffer (`ffi.rs:566`). Decision 17 C13's 2026-10-16 line lists it as + transport. #1386 D7 is the frozen text for its checks, its staging, and the free after every + command. +- **The command entry.** `miso_engine_web_v1_control_request_ptr` and + `miso_engine_web_v1_control_command` (`ffi.rs:3347`, `:3366`); the host side is + `control_command` (`hosts/host-web/src/lib.rs:2402`) and `run_command` (`lib.rs:1611`). +- **The worklet message.** `miso.control.v1` takes `CONTROL_FIELDS = ["tag", "requestId", + "frame"]` (`hosts/host-web/web/miso-engine-v1-audio-worklet.js:93`), dispatched at `:1116`. +- **The boot projection.** The boot checks `document_bytes * PARSE_TRANSIENT_MULTIPLIER` (20, + `hosts/host-web/src/lib.rs:98`) against its budget. + +## Decisions to take (the implementer proposes, the coordinator rules) + +- None of substance: #1386 D7 froze the export. The implementer records where the worklet's + `document` field is staged (before the frame, in the message handler) and confirms it in the + `worker` mode note for #1332. + +## Deliverables + +1. The export and its staging consumption in `control_command`, as #1386 D7 states. +2. The optional `document` field (a `Uint8Array`, the same checks as `frame`) on + `miso.control.v1` in the worklet's message handler. `process()` never stages, hashes or parses. +3. Both `.d.ts` twins: `hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts` and + `sdk/src/browser/shipped-host.d.ts` (the field's type). +4. The ABI layout mirrors of the export, regenerated by their own tools. +5. #1386's gate 6 and gate 8's browser half (below). + +## Authorized paths + +- `hosts/host-web/src/{ffi.rs,lib.rs}` (the export and the staging consumption only), + `hosts/host-web/src/tests.rs`, `hosts/host-web/tests/control_parity.rs` +- `hosts/host-web/web/miso-engine-v1-audio-worklet.js` (the `document` field only), + `hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts`, `sdk/src/browser/shipped-host.d.ts` +- `hosts/host-web/web/miso-engine-v1-audio-worklet-artifact.sha256` (re-pinned by + `scripts/build-web-audioworklet.sh`), `scripts/check-web-audioworklet.sh` (the export list), + `scripts/test-web-audioworklet.mjs` (the fake module's export and the field cases) +- `tools/parameter-metadata/src/abi_layout.rs`, `scripts/check-abi-layout-v1.py`, + `scripts/fixtures/abi-layout-v1-self-test.json`, `sdk/assets/miso-engine-v1-abi-layout.json`, + `sdk/src/generated/abi.ts` (regenerated, never hand-edited) +- `hosts/host-web/qualification/` (gate 3's real-browser case only) + +## Non-goals + +- No core change: the diff, the message and the refusals are #1386 (a)'s. +- No SDK `replaceSession` (#1296). +- No change to `document_ptr`, its pre-boot refusal or `CONTROL_FRAME_BYTES`. + +## Objective gates + +1. **Parity (#1386 gate 6).** The same `SESSION_REPLACE` frames and documents through the C ABI + (`miso_engine_v1_submit_document_command`) and the browser (this staging, then + `control_command`): one rebuild, one live, one identical document and one refusal. Same reply + bytes (request ID aside), revision, `SESSION_SNAPSHOT_GET` bytes and bit-identical blocks, at + 44.1 and 48 kHz. +2. **Large documents (#1386 gate 8, browser half, native).** A replace of more than 64 KiB, and one + at exactly `MAXIMUM_DOCUMENT_BYTES`, stage and commit; one byte more gets 0 from + `replace_document_ptr`. After the reply the staging's capacity is 0, also for the boot copy + freed at the first command. +3. **Real browsers (#1386 gate 8, measured in `hosts/host-web/qualification`).** One replace of + the 1 MiB boot-ceiling document: the wasm linear memory growth is recorded against the boot's + projection `document_bytes * PARSE_TRANSIENT_MULTIPLIER`; a growth above it is reported to + root, not pinned. On the page without cross-origin isolation the replace is a counted blocking + rebuild (decision 15, D15-10), and the `process()` allocation count stays 0. +4. **Commands:** `cargo test --locked -p host-web --features host-web/test-support`; the web + artifact gates with the arguments `.github/workflows/qualification.yml` gives them + (`build-web-audioworklet.sh`, `check-web-audioworklet.sh`, `test-web-audioworklet.mjs`); + `bash scripts/check-sdk-generated.sh`, `python3 scripts/check-abi-layout-v1.py`; the browser + qualification; clippy and fmt on the touched crates. + +## Dependencies + +- #1386 (a): the core `SESSION_REPLACE` path and the C ABI export. +- #1296 (open) is downstream: `replaceSession` sends this field. + +## Size + +Half a day. diff --git a/.github/ISSUE_SPECS/1698-carry-an-owner-or-source-a-superseding-candidate-keeps-unchanged-from-p0.md b/.github/ISSUE_SPECS/1698-carry-an-owner-or-source-a-superseding-candidate-keeps-unchanged-from-p0.md new file mode 100644 index 000000000..aefe50fe1 --- /dev/null +++ b/.github/ISSUE_SPECS/1698-carry-an-owner-or-source-a-superseding-candidate-keeps-unchanged-from-p0.md @@ -0,0 +1,140 @@ +# Carry an owner or source a superseding candidate keeps unchanged from P0 from P0's state + +Successor of #1344 (prepare a successor across a withdrawn candidate plan) and #1310 (supersede an +unadopted candidate plan by compare and swap), both batch 15a. Found in the batch 15a review; root +ruling 2026-10-16. Code anchors verified on the batch 15a fold branch `codex/controls-b15a-fold` +at `804fe6939` (committed tree; the branch has uncommitted edits in `control.rs` while this is +written, so the implementer re-reads the line numbers). Issue number: assigned at filing (written +here as #1698). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch +review. + +## Product outcome + +An edit followed quickly by its undo is not audible. When the running plan P0 has a committed +candidate A that render has not adopted, and a later transaction commits B that puts an owner or a +source X back to its P0 form, B carries X from P0, as a direct P0 -> B swap would. Today X starts +fresh in B whenever A restarted or removed it. + +## Context + +- **The defect.** #1344 D2 prepares B with `SuccessorBase { inventory: A.carried_base, committed: + P0's committed model }`. `carried_base` is P0's inventory restricted to the rows A's installed + hand-over reads (`PlanStateInventory::restricted_to`, `crates/host-core/src/prepare.rs:1086`; + `PreparedHost::carried_base`, `:689`). So an owner carries from P0 only if A carried it. + Anything A restarted, added or removed is fresh in B (the doc at `prepare.rs:1188`-`:1196` + states this as the rule). For an X that A restarted or removed and B reverts to its P0 form, + that is wrong: P0 still holds X's state, because A was never adopted, and the combined twin + (P0 -> B with one transaction) carries it. +- **Where B's base is built.** `SessionState`'s structural commit, `let base = if superseding` + (`crates/control-plane/src/control.rs:2044`), reads A's `carried_base` at `:2053` and P0's kept + model. +- **Where producers move.** After the commit, the superseding branch applies the checked donation + (`:2325`-`:2347`, #1344 D3: rings A created fresh that B keeps) and then + `candidate_provider.sources.adopt_persisting(&mut donor.sources)` (`:2349`) from A's set. After + the swap is published, `holder.sources.adopt_persisting(&mut donor.sources)` (`:2461`) returns + to P0's epoch every producer A holds that P0 holds vacant, before A drops. + `SourceControlSet::adopt_persisting` (`crates/host-core/src/source.rs:291`) fills each vacant + entry from the predecessor's entry of the same ID, keyed by ID, allocation-free. +- **Host submits between A's commit and B's swap.** A restarted X with a fresh ring. The host's + chunks for X after A's commit went to A's X ring, not P0's. If B carries X's ring from P0, A's + ring and its acked chunks drop with A, and P0's ring underruns for those frames. That breaks + the acked-batch rule (AGENTS.md: can an ack ever precede a drop?). +- **Tests to extend.** `crates/host-core/tests/withdrawn_successor.rs` (P0, A and B models at + `:147`, `:164`, `:178`; `twin_run`, `:306`; `backends`, `:184`) and + `crates/capi/src/runtime/supersede_tests.rs` + (`a_superseding_replacement_renders_as_one_combined_transaction`, `:157`; + `a_removed_sources_producer_returns_to_the_running_epoch`, `:554`). + +## Decisions frozen for this slice + +- **D1. No ack before a drop (root, 2026-10-16).** Every chunk a host submit acks for X between + A's commit and B's swap is either read by B or refused before its ack. No acked chunk drops + with A. The implementer proposes the mechanism (B reads A's X ring for those frames, or the + submit to a ring that a later supersession can drop is refused with typed backpressure before + its ack, or another shape) and the coordinator rules; the rule itself is not open. + +## Decisions to take (the implementer proposes, the coordinator rules) + +- **Amend #1344 D1 and D2.** The base for an owner or source that B keeps unchanged from P0 is + P0's inventory row, not A's restricted base. Decide whether `PreparedHost::carried_base` and + `PlanStateInventory::restricted_to` stay (for example as the base of what A changed and B keeps) + or are replaced by P0's inventory with a rule for the rows A changed. #1344's doc at + `prepare.rs:1188`-`:1196` is amended with the decision. +- **A P0 producer pass.** Before the donation pass (`control.rs:2325`-`:2347`), add one + `adopt_persisting`-shaped pass from P0's source set, so a reverted X takes P0's producer and + never A's fresh one. State its order against the donation and against the return pass + (`:2461`), and prove it allocation-free. +- **Where submits for X go between A's commit and B's swap**, under D1. + +## Deliverables + +1. The base and the producer pass in `crates/host-core/src/{prepare.rs,source.rs}` and + `crates/control-plane/src/control.rs`. +2. D1's mechanism, wherever the ruling places it. +3. The gates below; #1344's and #1310's spec text amended by appending, not rewritten. + +## Authorized paths + +- `crates/host-core/src/prepare.rs`, `crates/host-core/src/source.rs`, + `crates/host-core/tests/withdrawn_successor.rs` +- `crates/control-plane/src/control.rs` and its tests +- `crates/capi/src/runtime/supersede_tests.rs` +- The source ring's submit path (`crates/source/src/lib.rs`) only if the D1 ruling puts the + refusal there +- This spec file's decision record; appended amendments in #1344's and #1310's specs + +## Non-goals + +- No change to the donation of rings A created fresh that B also creates fresh (#1344 D3). +- No change to what a direct P0 -> successor swap carries. +- No new C export or protocol message. + +## Objective gates + +1. **Revert carries from P0.** A removes X, B reverts X; and A restarts X, B reverts X; for X a + strip owner and for X a source. X carries from P0: B's rendered blocks are bit-identical to the + combined twin's (P0 -> B with one transaction, fed the same chunks), at both lane widths + (`Backend::VECTOR`), in `crates/host-core/tests/withdrawn_successor.rs` and in the capi + supersede tests. Each case is red on the base before the fix (recorded in the PR). +2. **No acked chunk is lost (D1).** A host submits chunks for X between A's commit and B's swap. + Every submit either returns a typed refusal or is acked, and every acked chunk's frames are + read by B: B's output equals the twin fed exactly the acked chunks, and the count of acked + chunks no plan reads is 0. +3. **The swap block allocates nothing.** `allocations == 0` across B's adoption block, measured + with `bench_support::alloc`'s thread-scoped counters after warm-up. +4. **Commands:** `cargo test --locked -p host-core --all-features`, + `cargo test --locked -p control-plane --features control-plane/test-support`, + `cargo test --locked -p capi`, `cargo run --locked --release -q -p realtime-policy`; + clippy `-D warnings` and fmt on the touched crates; `bash scripts/check-workspace-policy.sh`. + +## Test value + +- Gate 1 turns red if a reverted owner or source starts fresh in B, or takes A's fresh producer. +- Gate 2 turns red if a chunk acked into A's ring drops with A. +- Gate 3 turns red if the producer pass or the D1 mechanism allocates at the swap. + +- **Lifts the batch 15a guard, reverted case only.** Batch 15a (fold 2, widened in fold 3 by root + ruling M-A of 2026-10-16) refuses with `BACKPRESSURE` a supersession B that leaves behind a ring + A created (A added or redeclared the source) whose producer has acked anything + (`host_core::drops_acked_fresh_source`, called in `SessionState::rebuild` right after + `check_donation`). #1698 lifts only case (a), where B declares X as P0 does, once D1's mechanism + lets B read A's acked chunks: that case leaves the guard and its rows in + `supersede_tests::a_supersession_that_would_drop_acked_pcm_on_a_fresh_source_is_backpressure` + and `control_parity::a_supersession_that_would_drop_acked_pcm_is_backpressure_alike_on_both_hosts` + are replaced by gate 2. Cases (b) B redeclares X a third way, (c) B removes X, and (d) A added X + and B removes or redeclares it stay refused until an issue gives B a way to read those chunks or + the owner rules that a removed source's chunks may drop. + +## Dependencies + +- #1344 and #1310 (batch 15a): the withdrawn-candidate base, the donation and the supersession. +- Root ruling 2026-10-16: this is the first K slice of batch 15b, ahead of the rest of the source + chain. + +## Size + +Half a day. + +## Decision record: #1698 + +To be filled by the implementer. The base decision goes first. diff --git a/.github/ISSUE_SPECS/1699-count-the-frames-a-callers-own-source-removal-discards.md b/.github/ISSUE_SPECS/1699-count-the-frames-a-callers-own-source-removal-discards.md new file mode 100644 index 000000000..18c8ed28a --- /dev/null +++ b/.github/ISSUE_SPECS/1699-count-the-frames-a-callers-own-source-removal-discards.md @@ -0,0 +1,42 @@ +# Count the frames a caller's own source removal discards + +Successor of #1310 (batch 15a fold 3, root ruling of 2026-10-16), decision 15 D15-2 and decision 17 +C1. Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. + +## Problem + +A source that the caller's own edit removes discards the audio still queued for it: a direct P0 -> B +removal, or a superseding B that removes a source A carried unchanged from P0. Root ruled that this +is not an unintended drop, because the caller asked for the removal, and the C header and +`docs/CONTROL_PROTOCOL_SEMANTICS.md` state the contract. But no counter reports how many accepted +frames were discarded, so a host cannot see it, and D15-2 wants every such loss counted. + +## Decisions + +- **D1. One core counter.** The core counts, per session, the accepted source frames that a removal + discards (frames written to the removed source's ring and not yet read by render), at the swap + that retires the ring, off the render thread. It is a monotone `u64`, never reset. +- **D2. Both hosts read it the same way.** It is a counter in the existing counter snapshot (a new + counter ID from the free range, never a retired one), so the C ABI `COUNTERS_GET` and the browser + read the same value through the same core function. No host-specific code. +- **D3. Render stays clean.** Counting happens where the retired ring is reclaimed (control side); + render adds no work, no allocation, no lock. + +## Gates + +1. A session feeds N frames to source X, then an edit removes X before render reads them: the + counter rises by exactly the unread frame count, on both hosts (parity row), at two rates. +2. A removal of a source with an empty ring adds 0. +3. A superseding B that removes a source A carried unchanged from P0 counts the same as the direct + removal. +4. Render-thread allocations stay 0 across the removal swap. + +## Authorized paths + +`crates/host-core/src/source.rs`, `crates/control-plane/src/`, the counter vocabulary and its +mirrors, `crates/capi/include/miso_engine_v1.h`, `hosts/host-web/src/`, tests next to them, +`docs/CONTROL_PROTOCOL_SEMANTICS.md`, this spec. + +## Dependencies + +#1310 (on main with batch 15a). diff --git a/crates/capi/include/miso_engine_v1.h b/crates/capi/include/miso_engine_v1.h index 2cbaf99a2..8c88b46d5 100644 --- a/crates/capi/include/miso_engine_v1.h +++ b/crates/capi/include/miso_engine_v1.h @@ -201,6 +201,42 @@ extern "C" { * generation 1, frame 0, with the PCM accepted for it before the command discarded with its plan; * the host restarts its feed. * + * A transaction behind an unadopted replacement (issue 1310). A SESSION_TRANSACTION_APPLY that + * needs a new plan while an earlier transaction's replacement plan has not been adopted by render + * (a paused host never adopts one) commits as any other: it is not refused as backpressure because + * a replacement waits, with the one exception below. The engine takes the waiting replacement + * back, prepares one plan that carries both transactions against the plan that renders, and + * publishes it; nothing of the earlier transaction is lost. A source the new plan keeps from the + * waiting replacement or from the plan that renders keeps playing: the PCM accepted for it and a + * seek or seek_at queued on it move into the new plan, so the host neither refeeds nor reseeks it. + * One case is not covered yet: a source or owner that the earlier transaction restarted or removed + * and the new one puts back in its form in the plan that renders starts fresh in the new plan (a + * source at generation 1, frame 0, as an added one, and the host restarts its feed); that gap + * closes with the successor issue "Carry an owner or source a superseding candidate keeps + * unchanged from P0 from P0's state". No accepted chunk on a ring the waiting replacement created + * is dropped across a supersession: when the waiting replacement created a source's ring (the + * earlier transaction added the source, or redeclared it) and the session has accepted PCM, a seek + * or an end of region for that source since, a new transaction that does not keep the source as + * the waiting replacement declares it (it declares it as the plan that renders does, redeclares it + * another way, or removes it) is refused with MISO_ENGINE_V1_BACKPRESSURE before it commits: + * nothing changes, the waiting replacement stays published, no event is queued and the revision + * does not move. Once render adopts the waiting replacement, a retry commits. A host that does not + * render keeps getting MISO_ENGINE_V1_BACKPRESSURE for such a transaction until it renders. A + * source the waiting replacement carries unchanged from the plan that renders is not refused this + * way. A source the caller's own transaction removes discards its queued audio, whether the plan + * that renders or the waiting replacement (carrying it unchanged) holds it: the caller asked for + * the removal, so this is not an unintended drop. No counter reports the discarded frames yet; a + * counter for the discarded frames is a successor issue (placeholder #1699). A resource cap counts + * every plan the session still holds and the new one: the running plan, the waiting replacement, + * and a plan render has displaced until a later call reclaims it. A refusal is + * MISO_ENGINE_V1_BACKPRESSURE when the new plan fit against the newest held plan alone, so a retry + * after a render normally commits; the split is approximate, and such a retry may still be refused + * within a small band (a plan prepared on the adopted replacement differs by a few hundred bytes + * from one prepared across it). A refusal that does not fit even then is + * MISO_ENGINE_V1_COMPILE_REJECTED with the cap's diagnostic. When render adopts the new plan, the + * watermark advances to its revision with MISO_ENGINE_V1_OUTCOME_SUPERSEDED set, and + * superseded_count grows by the revisions the earlier replacements carried. + * * Starting an added stem in time (issue 1275). miso_engine_v1_source_seek_at is * miso_engine_v1_source_seek with an anchor: source_frame enters the graph in the block that starts * at absolute render sample anchor_sample, the clock miso_engine_v1_render_f32_planar takes and @@ -286,6 +322,20 @@ typedef struct miso_engine_v1_compile_limits { uint64_t maximum_sources; uint64_t maximum_routes; uint64_t maximum_effects; + /* Sizing rule for the resource caps from here to maximum_capi_retained_bytes (#1398). A + structural transaction is admitted against every plan the session still holds plus its + own: the running plan, and at most a withdrawn candidate and its successor, so up to three + plans live at once (a plan render has just displaced also counts until a later call + reclaims it; a refusal only it causes is MISO_ENGINE_V1_BACKPRESSURE). Set each of these + caps to at least three times one plan's row (the + matching field of miso_engine_v1_plan_resource_report, for the largest session the host + will compile, read while a successor plan of it runs: such a plan carries its sources and + owns their rings), and maximum_graph_session_plus_plan_bytes to that plus three compiled + models. A cap below that refuses a transaction before it commits, and nothing is + acknowledged: with MISO_ENGINE_V1_BACKPRESSURE when the transaction fit against the + newest held plan alone, so a retry after a render normally commits (the split is + approximate: such a retry may still be refused within a small band), else with + MISO_ENGINE_V1_COMPILE_REJECTED and a resource diagnostic. */ uint64_t maximum_graph_session_plus_plan_bytes; uint64_t maximum_source_total_bytes; uint64_t maximum_source_overhead_bytes; diff --git a/crates/capi/src/runtime/live_peak_tests.rs b/crates/capi/src/runtime/live_peak_tests.rs index c0a6e2fa1..a73505708 100644 --- a/crates/capi/src/runtime/live_peak_tests.rs +++ b/crates/capi/src/runtime/live_peak_tests.rs @@ -28,6 +28,7 @@ fn epoch(base: PlanResourceReport, terms: &Terms) -> LiveEpochResources { prepared_protocol_retained: terms.prepared_protocol, largest: terms.capi_largest, }, + carried: control_plane::CarriedSourceBytes::default(), } } @@ -46,9 +47,10 @@ fn verdict( models: CompiledModelAdmission, limits: CompileLimits, ) -> Result<(), String> { + let held = [Some(current), pending]; + let held = held.iter().flatten().copied().collect::>(); validate_live_peak( - current, - pending, + &held, models, LiveCatalogAdmission::default(), limits.into(), diff --git a/crates/capi/src/runtime/mod.rs b/crates/capi/src/runtime/mod.rs index 02db20659..8687aac0f 100644 --- a/crates/capi/src/runtime/mod.rs +++ b/crates/capi/src/runtime/mod.rs @@ -30,6 +30,8 @@ mod live_peak_tests; #[cfg(test)] mod live_tests; #[cfg(test)] +mod supersede_tests; +#[cfg(test)] mod tests; pub(crate) use error::*; diff --git a/crates/capi/src/runtime/supersede_tests.rs b/crates/capi/src/runtime/supersede_tests.rs new file mode 100644 index 000000000..a4c8b04c2 --- /dev/null +++ b/crates/capi/src/runtime/supersede_tests.rs @@ -0,0 +1,1413 @@ +//! #1310: a structural transaction submitted while an earlier one's candidate plan waits +//! unadopted commits and supersedes it. + +use super::*; +use crate::runtime::tests::{ + command_bytes_at_revision, continuity_block, limits, render_direct, stateless_session, + submit_direct, +}; +use protocol::{ExpectedRevision, SessionEdit, SessionRevision}; +use session::{ + Effect, EffectIdentity, EffectQuality, LinkMode, RackName, RouteSource, SessionModel, + SidechainDeclaration, StableId, +}; + +const FIXTURE: &[u8] = b"fixture-source"; +const ADDED: &[u8] = b"s-source"; + +fn id(value: &str) -> StableId { + StableId::parse(value).expect("stable ID") +} + +/// A native insert `instance` of `effect` with its defaults. +fn insert(instance: &str, effect: &str) -> Effect { + Effect { + id: id(instance), + identity: EffectIdentity::Native { + effect_id: id(effect), + }, + quality: EffectQuality::Normal, + bypass: false, + link_mode: LinkMode::DualMono, + params: Vec::new(), + sidechain: SidechainDeclaration::None, + } +} + +fn compressor() -> Effect { + insert("comp", "miso.compressor") +} + +/// The compressor's replacement under the same insert ID: a prepared change that restarts the +/// insert. No launch effect declares a second quality, so a quality edit does not prepare (#1344). +fn gate() -> Effect { + insert("comp", "miso.gate-expander") +} + +fn put(model: &SessionModel, track: usize, effect: Effect) -> SessionEdit { + SessionEdit::PutTrackEffect { + track_id: model.tracks[track].id.clone(), + rack_name: RackName::Inserts, + final_position: 0, + effect, + } +} + +fn remove(model: &SessionModel, track: usize) -> SessionEdit { + SessionEdit::RemoveTrackEffect { + track_id: model.tracks[track].id.clone(), + rack_name: RackName::Inserts, + effect_id: id("comp"), + } +} + +/// `UpsertSource` of `s-source`, `UpsertTrack` of track `ts` on it (a copy of the first track +/// with no insert) and `UpsertRoute` of its route. +fn add_source_track(model: &SessionModel) -> [SessionEdit; 3] { + let mut source = model.sources[0].clone(); + source.id = id("s-source"); + let mut track = model.tracks[0].clone(); + track.id = id("ts"); + track.source_id = source.id.clone(); + track.inserts.effects.clear(); + let mut route = model.routes[0].clone(); + route.id = id("ts-main"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("track route") + }; + *track_id = track.id.clone(); + [ + SessionEdit::UpsertSource { source }, + SessionEdit::UpsertTrack { track }, + SessionEdit::UpsertRoute { route }, + ] +} + +fn transaction(request_id: u64, revision: u64, edits: &[SessionEdit]) -> Vec { + command_bytes_at_revision( + request_id, + ExpectedRevision::Exact(SessionRevision(revision)), + protocol::CommandPayload::SessionTransactionApply(edits), + ) +} + +/// Commits `edits` at `base` and drains the reliable events, so a later transaction never meets a +/// full event lane. Panics unless the transaction commits as revision `base + 1`. +fn commit(children: &mut CompiledChildren, request_id: u64, base: u64, edits: &[SessionEdit]) { + let length = children + .session + .command(&transaction(request_id, base, edits), 4_096) + .unwrap_or_else(|error| panic!("request {request_id}: {error:?}")); + let response = children.session.command_response(length).to_vec(); + let mut fields = [0_u16; 64]; + let header = match ProtocolCodec::default() + .decode_typed_response(&response, &mut DecodeScratch::new(&mut fields)) + .expect("a response frame") + { + protocol::DecodedTypedResponseFrame::Success { header, .. } => header, + protocol::DecodedTypedResponseFrame::NonOk { header, payload } => { + panic!( + "request {request_id}: {:?} {:?}", + header.status, payload.diagnostics + ) + } + }; + assert_eq!( + header.revision, + SessionRevision(base + 1), + "request {request_id}" + ); + while children + .session + .dequeue_event(EventLane::Reliable, 4_096) + .expect("a reliable event") + .is_some() + {} +} + +fn compile(model: &SessionModel, limits: CompileLimits) -> CompiledChildren { + let document = session::canonical_session_json(model).expect("canonical"); + compile_children(&document, limits).unwrap_or_else(|_| panic!("the session compiles")) +} + +fn bits(block: &[f32]) -> Vec { + block.iter().map(|sample| sample.to_bits()).collect() +} + +fn revision(children: &CompiledChildren) -> u64 { + children.session.test_controller().session().revision().0 +} + +/// Gate 1: three structural transactions submitted without a render between them each commit, +/// and the plan the last one publishes renders exactly what one transaction of all three edits +/// renders; the watermark completes the first two as superseded. +/// +/// P0 has tracks `t1`-`t4` on the fixture source, `t2` and `t3` with a compressor insert. T1 adds +/// source `s` with its track and a compressor on `t1`, and moves `t4`'s fader; T2 replaces `t2`'s +/// compressor; T3 removes `t3`'s. `s` is fed between T1 and T2, so its ring is the withdrawn +/// candidate's and moves by the donation; `t4` carries from P0 with its fader retargeted at T1's +/// own ramp length. +/// +/// Test value: red if any of T1-T3 is refused, if the replacement carries from the withdrawn +/// candidate's identity while P0 runs (the carry finds no predecessor and the fixture source and +/// `t2` restart), if it is prepared against the current model instead of P0's (`t4`'s fader is +/// then no change, and `t4` keeps P0's), if the retarget forgets T1's ramp length, if `s`'s acked +/// PCM is lost with the withdrawn candidate, or if the superseded count is off. +#[test] +fn a_superseding_replacement_renders_as_one_combined_transaction() { + let mut model = stateless_session(48_000); + model.tracks[1].inserts.effects.push(compressor()); + model.tracks[2].inserts.effects.push(compressor()); + // `t4` is untouched but for a fader change T1 carries: it carries from P0 in every plan. + let mut t4 = model.tracks[0].clone(); + t4.id = id("t4"); + let mut route = model.routes[0].clone(); + route.id = id("t4-main"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("track route") + }; + *track_id = t4.id.clone(); + model.tracks.push(t4); + model.routes.push(route); + let fader = session::DualMonoFader { + left_db: -6.0, + right_db: -9.0, + left_mute: false, + right_mute: false, + }; + // The fader edits carry their own ramp length (#1394), which the replacement's base keeps + // across the withdrawn candidate (root ruling B15-Q3). + let mut fader_edits = protocol::control_core::fader_edits(&id("t4"), &fader); + for edit in &mut fader_edits[..2] { + let SessionEdit::SetControl { ramp_samples, .. } = edit else { + panic!("a control edit") + }; + *ramp_samples = Some(1); + } + let first: Vec = add_source_track(&model) + .into_iter() + .chain([put(&model, 0, compressor())]) + .chain(fader_edits) + .collect(); + let second = [put(&model, 1, gate())]; + let third = [remove(&model, 2)]; + let combined: Vec = first.iter().chain(&second).chain(&third).cloned().collect(); + + let run = |superseding: bool| { + let mut children = compile(&model, limits()); + submit_direct(&mut children, &[FIXTURE], 0); + let mut blocks = vec![render_direct(&mut children, 0)]; + let before = children.plan.queries().watermark().expect("watermark"); + if superseding { + for (request, (base, edits)) in [(42, &first[..]), (43, &second[..]), (44, &third[..])] + .into_iter() + .enumerate() + { + commit(&mut children, request as u64 + 1, base, edits); + if request == 0 { + // `s` is fed while T1's candidate waits, into the candidate's own ring. + submit_direct(&mut children, &[ADDED], 0); + submit_direct(&mut children, &[ADDED], 1); + } + } + assert_eq!(revision(&children), 45); + } else { + commit(&mut children, 1, 42, &combined); + submit_direct(&mut children, &[ADDED], 0); + submit_direct(&mut children, &[ADDED], 1); + } + submit_direct(&mut children, &[FIXTURE], 1); + blocks.push(render_direct(&mut children, 1)); + let adopted = children.plan.queries().watermark().expect("watermark"); + for block in 2..9 { + submit_direct(&mut children, &[FIXTURE, ADDED], block); + blocks.push(render_direct(&mut children, block)); + } + (blocks, before, adopted) + }; + let (superseded, before, adopted) = run(true); + let (twin, _, _) = run(false); + for (block, (actual, expected)) in superseded.iter().zip(&twin).enumerate() { + assert!( + expected.iter().any(|sample| *sample != 0.0), + "block {block}: the twin plays" + ); + assert_eq!(bits(actual), bits(expected), "block {block}"); + } + assert_eq!(adopted.revision, 45); + assert_eq!( + adopted.outcome_flags, + engine::realtime::OUTCOME_EXACT | engine::realtime::OUTCOME_SUPERSEDED + ); + assert_eq!(adopted.superseded, before.superseded + 2); + assert_eq!(adopted.exact, before.exact + 1); +} + +/// Gate 2: a source a withdrawn candidate added keeps the PCM and the anchored seek the host was +/// acked for: the replacement plays it from the anchor exactly as when the candidate was adopted +/// before the next transaction. +/// +/// Only `s` is fed, so the mix is `s`'s track alone; T2 replaces another track's insert, which +/// renders silence either way. +/// +/// Test value: red if the replacement allocates a fresh ring for `s` instead of taking the +/// candidate's: the acked chunks and the held seek are lost, and `s` is silent from the anchor on. +#[test] +fn a_donated_source_keeps_its_pcm_and_its_held_seek() { + const ANCHOR_BLOCK: u64 = 5; + let mut model = stateless_session(48_000); + model.tracks[0].inserts.effects.push(compressor()); + let first = add_source_track(&model); + let second = [put(&model, 0, gate())]; + let run = |superseding: bool| { + let mut children = compile(&model, limits()); + let mut blocks = vec![render_direct(&mut children, 0)]; + commit(&mut children, 1, 42, &first); + children + .session + .seek_at(ADDED, 2, 0, ANCHOR_BLOCK * 128) + .unwrap_or_else(|_| panic!("the anchored seek")); + for block in 0..4 { + let (left, right) = continuity_block(block, 1); + children + .session + .submit( + ADDED, + SourceSubmission { + generation: 2, + start_frame: block * 128, + sample_rate_hz: 48_000, + planes: &[&left, &right], + frames: 128, + end_of_region: false, + }, + ) + .unwrap_or_else(|_| panic!("chunk {block}")); + } + if !superseding { + blocks.push(render_direct(&mut children, 1)); + } + commit(&mut children, 2, 43, &second); + for block in blocks.len() as u64..9 { + blocks.push(render_direct(&mut children, block)); + } + blocks + }; + let superseded = run(true); + let twin = run(false); + for (block, (actual, expected)) in superseded.iter().zip(&twin).enumerate() { + assert_eq!(bits(actual), bits(expected), "block {block}"); + assert_eq!( + expected.iter().any(|sample| *sample != 0.0), + block as u64 >= ANCHOR_BLOCK, + "block {block}: `s` plays from its anchor" + ); + } +} + +/// The lifecycle counters every constructed owner and every reservation are balanced in. +fn assert_balanced(label: &str, counters: TestOwnerCounters) { + assert_eq!( + counters.current_provider_constructed + counters.candidate_provider_constructed, + counters.current_provider_disposed + counters.candidate_provider_disposed, + "{label}: providers" + ); + assert_eq!( + counters.current_plan_constructed + counters.candidate_plan_constructed, + counters.current_plan_disposed + counters.candidate_plan_disposed, + "{label}: plans" + ); + assert_eq!( + counters.token_constructed, counters.token_disposed, + "{label}: tokens" + ); + assert_eq!( + counters.reservation_constructed, + counters.reservation_canceled + counters.reservation_committed, + "{label}: reservations" + ); +} + +/// Gate 3: a replacement refused after the candidate was withdrawn restores it. T2 is refused +/// for its effect state (a cap one byte below its own), with the compile rejection; the revision +/// stays T1's, and the next block renders exactly what the T1-only session renders. Then each +/// structural fault phase is armed during T2, the one between the donation check and the commit +/// (`BeforeProtocolCommit`) and the commit's own included: the refusal changes no committed or +/// published state, the candidate is republished each time it was withdrawn, a retried T2 +/// supersedes it and renders as a supersession nothing refused, and every owner and reservation +/// balances once both handles are gone. Last, the withdrawn candidate is itself a replacement +/// with a nonzero `superseded` word (batch 15a review fold m4): the refusal republishes it with +/// that word, and a retry's word counts every revision both candidates folded in. +/// +/// Test value: red if a refusal after the withdrawal drops or leaks the candidate (the next block +/// is P0's, or an owner stays undisposed), if a donation runs before the commit (the fault +/// between the check and the commit would leave the candidate without its rings), or if a +/// republished replacement loses its `superseded` word (T1 then never completes). +#[test] +fn a_refused_replacement_restores_the_withdrawn_candidate() { + use TestStructuralFaultPhase::{ + AfterAdmission, AfterPlanReservation, AfterProtocolPrepare, AfterRuntimePrepare, + BeforeProtocolCommit, BeforeRuntimePrepare, ProtocolCommit, + }; + let model = stateless_session(48_000); + let first = add_source_track(&model); + let second = [put(&model, 1, compressor()), put(&model, 2, compressor())]; + // T1 adds no effect, so the replacement's effect state is T2's two compressors'. + let mut after_second = model.clone(); + for track in &mut after_second.tracks[1..] { + track.inserts.effects.push(compressor()); + } + let need = compile(&after_second, limits()) + .plan + .resources() + .effect_scalar_state_bytes; + // T1 is pending, with block 1 of both sources fed. + let pending = |limits: CompileLimits| { + let mut children = compile(&model, limits); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + commit(&mut children, 1, 42, &first); + submit_direct(&mut children, &[ADDED], 0); + submit_direct(&mut children, &[FIXTURE, ADDED], 1); + children + }; + let twin = render_direct(&mut pending(limits()), 1); + assert!(twin.iter().any(|sample| *sample != 0.0), "the twin plays"); + // T2 superseding T1 with nothing refused, the reference of the retries below. + let superseding = { + let mut children = pending(limits()); + commit(&mut children, 2, 43, &second); + render_direct(&mut children, 1) + }; + + let mut tight = limits(); + tight.maximum_effect_state_bytes = need - 1; + let mut children = pending(tight); + let epoch = children.session.test_pending_providers()[0].test_epoch(); + let counters = children.session.test_owner_counters(); + let refused = children + .session + .command(&transaction(2, 43, &second), 4_096); + let Err(CommandError::CompileRejected(failure)) = &refused else { + panic!("a compile rejection, not {refused:?}") + }; + let diagnostic = String::from_utf8(failure.diagnostic().into_owned()).expect("UTF-8"); + assert!( + diagnostic.contains("effect") && diagnostic.contains("resource.limit"), + "{diagnostic}" + ); + assert_eq!(revision(&children), 43, "the revision is T1's"); + let after = children.session.test_owner_counters(); + assert_eq!(after.candidate_withdrawn, counters.candidate_withdrawn + 1); + assert_eq!( + after.candidate_republished, + counters.candidate_republished + 1 + ); + assert_eq!( + children.session.test_pending_providers()[0].test_epoch(), + epoch, + "the candidate is republished with its own epoch" + ); + let block = render_direct(&mut children, 1); + assert_eq!(bits(&block), bits(&twin), "T1's candidate renders"); + + for phase in [ + AfterProtocolPrepare, + BeforeRuntimePrepare, + AfterRuntimePrepare, + AfterAdmission, + AfterPlanReservation, + BeforeProtocolCommit, + ProtocolCommit, + ] { + control_plane::test_reset_lifecycle_observer(); + let mut children = pending(limits()); + let snapshot = children.session.test_transaction_snapshot(); + children + .session + .test_set_structural_faults([Some(phase), None]); + let refused = children + .session + .command(&transaction(2, 43, &second), 4_096); + assert!( + matches!( + refused, + Err(CommandError::Backpressure | CommandError::Internal) + ), + "{phase:?}: {refused:?}" + ); + assert_eq!( + children.session.test_transaction_snapshot(), + snapshot, + "{phase:?}: the refusal changed nothing" + ); + let counters = children.session.test_owner_counters(); + let withdrawn = !matches!(phase, AfterProtocolPrepare | BeforeRuntimePrepare); + assert_eq!( + (counters.candidate_withdrawn, counters.candidate_republished), + (u64::from(withdrawn), u64::from(withdrawn)), + "{phase:?}" + ); + // The retry supersedes the republished candidate. + commit(&mut children, 3, 43, &second); + assert_eq!( + children.session.test_owner_counters().candidate_superseded, + 1, + "{phase:?}" + ); + let block = render_direct(&mut children, 1); + assert_eq!( + bits(&block), + bits(&superseding), + "{phase:?}: the retry renders as a supersession nothing refused" + ); + children.session.service().expect("service"); + assert_eq!(revision(&children), 44, "{phase:?}"); + assert!(children.session.test_pending_providers().is_empty()); + drop(children); + assert_balanced( + &format!("{phase:?}"), + control_plane::test_lifecycle_counters(), + ); + } + + // Batch 15a review fold m4: the withdrawn candidate is itself a replacement, B, whose + // `superseded` word counts T1. T3 is refused after B was withdrawn; B is republished with its + // word, so the block that adopts it completes T1 as superseded and T2 as exact. Then a retried + // T3 supersedes B, and its word counts T1 and T2. + let third = [put(&model, 0, compressor())]; + for retry in [false, true] { + control_plane::test_reset_lifecycle_observer(); + let mut children = pending(limits()); + let before = children.plan.queries().watermark().expect("watermark"); + commit(&mut children, 2, 43, &second); + children + .session + .test_set_structural_faults([Some(BeforeProtocolCommit), None]); + let refused = children.session.command(&transaction(3, 44, &third), 4_096); + assert!( + matches!(refused, Err(CommandError::Backpressure)), + "retry {retry}: {refused:?}" + ); + let counters = children.session.test_owner_counters(); + assert_eq!( + (counters.candidate_withdrawn, counters.candidate_republished), + (2, 1), + "retry {retry}: B was withdrawn and republished" + ); + if retry { + commit(&mut children, 4, 44, &third); + } + render_direct(&mut children, 1); + let adopted = children.plan.queries().watermark().expect("watermark"); + let folded = if retry { 2 } else { 1 }; + assert_eq!(adopted.revision, 44 + u64::from(retry), "retry {retry}"); + assert_eq!( + adopted.outcome_flags, + engine::realtime::OUTCOME_EXACT | engine::realtime::OUTCOME_SUPERSEDED, + "retry {retry}" + ); + assert_eq!( + (adopted.superseded, adopted.exact), + (before.superseded + folded, before.exact + 1), + "retry {retry}: the republished word completes every revision it folded in" + ); + children.session.service().expect("service"); + drop(children); + assert_balanced( + &format!("m4 retry {retry}"), + control_plane::test_lifecycle_counters(), + ); + } +} + +/// A plan's graph bytes and its compiled model's bytes. +type GraphRow = (u64, u64); + +/// The graph rows of P0, of A (P0 with `first`) and of B (A with `second`), each read while its +/// plan renders (gate 4). +fn graph_rows( + model: &SessionModel, + first: &[SessionEdit], + second: &[SessionEdit], +) -> (GraphRow, GraphRow, GraphRow) { + let graph = |children: &CompiledChildren| { + ( + children.plan.resources().graph_session_plus_plan_bytes, + children + .session + .committed_session() + .resource_estimate() + .compiled_model_bytes, + ) + }; + let mut children = compile(model, limits()); + let p0 = graph(&children); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + commit(&mut children, 1, 42, first); + render_direct(&mut children, 1); + children.session.service().expect("service"); + let a = graph(&children); + commit(&mut children, 2, 43, second); + render_direct(&mut children, 2); + children.session.service().expect("service"); + (p0, a, graph(&children)) +} + +/// Gate 4: the replacement admission holds the running plan, the withdrawn candidate and the +/// replacement at once (#1310 D3, #1398 D4). With a graph cap above every two-plan peak and below +/// that three-plan peak, T2 behind a pending T1 is refused, the revision stays T1's and T1's +/// candidate renders; once it is adopted the same T2 commits. T2 fits beside T1's plan alone, so +/// the refusal is retryable `BACKPRESSURE` (batch 15a review fold, item 2). With a cap that admits +/// T1 beside the running plan but not T2 beside T1's plan, the refusal is the permanent +/// `graph.resource.limit`, before and after T1's plan is adopted. +/// +/// Test value: red if a three-plan peak is admitted beyond the cap, if the refusal loses the +/// withdrawn candidate, if a refusal a render clears is permanent, or if one no render clears is +/// retryable. +#[test] +fn a_replacement_over_the_three_plan_ceiling_is_refused_and_the_candidate_kept() { + let model = stateless_session(48_000); + let first = add_source_track(&model); + let second = [put(&model, 1, compressor())]; + let (p0, a, b) = graph_rows(&model, &first, &second); + let two_plans = (p0.0 + a.0 + p0.1 + a.1).max(a.0 + b.0 + a.1 + b.1); + let three_plans = p0.0 + a.0 + b.0 + p0.1 + a.1 + b.1; + let cap = two_plans + (p0.0 + p0.1) / 2; + assert!(cap < three_plans, "{cap} against {three_plans}"); + let mut limits = limits(); + limits.maximum_graph_session_plus_plan_bytes = cap; + + let twin = { + let mut children = compile(&model, crate::runtime::tests::limits()); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + commit(&mut children, 1, 42, &first); + submit_direct(&mut children, &[ADDED], 0); + submit_direct(&mut children, &[FIXTURE, ADDED], 1); + render_direct(&mut children, 1) + }; + assert!(twin.iter().any(|sample| *sample != 0.0), "the twin plays"); + let mut children = compile(&model, limits); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + commit(&mut children, 1, 42, &first); + let refused = children + .session + .command(&transaction(2, 43, &second), 4_096); + assert!( + matches!(refused, Err(CommandError::Backpressure)), + "a refusal a render clears is retryable, not {refused:?}" + ); + assert_eq!(revision(&children), 43); + submit_direct(&mut children, &[ADDED], 0); + submit_direct(&mut children, &[FIXTURE, ADDED], 1); + let rendered = render_direct(&mut children, 1); + assert_eq!(bits(&rendered), bits(&twin), "A was republished"); + commit(&mut children, 3, 43, &second); + + // A cap that admits A beside P0 but not B beside A: no render clears it. + let a_beside_p0 = p0.0 + a.0 + p0.1 + a.1; + let b_beside_a = a.0 + b.0 + a.1 + b.1; + assert!( + a_beside_p0 < b_beside_a, + "{a_beside_p0} against {b_beside_a}" + ); + let mut limits = crate::runtime::tests::limits(); + limits.maximum_graph_session_plus_plan_bytes = a_beside_p0; + let mut children = compile(&model, limits); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + commit(&mut children, 1, 42, &first); + for (request, label) in [(2, "while A waits"), (3, "once A runs")] { + let refused = children + .session + .command(&transaction(request, 43, &second), 4_096); + let Err(CommandError::CompileRejected(failure)) = &refused else { + panic!("{label}: a compile rejection, not {refused:?}") + }; + assert_eq!( + &*failure.diagnostic(), + b"graph.resource.limit\t$\n", + "{label}" + ); + assert_eq!(revision(&children), 43, "{label}"); + render_direct(&mut children, request - 1); + } +} + +/// Gate 6: P0 has sources `s` and `u`. T1 changes `u`'s track's insert, so its candidate A +/// persists `s` and holds `s`'s producer; T2 removes `s`'s track and `s`. The superseding +/// replacement B holds no `s`, and `s`'s producer returns to the running epoch, which still renders +/// `s` until B is adopted; it retires with that epoch. +/// +/// Test value: red if the supersession drops A with the producers B did not take (D5 step 5 +/// missing, or after A's drop): the running epoch's entry for `s` is then vacant. Gates 1-5 do not +/// see it, because none removes a source A persisted. +#[test] +fn a_removed_sources_producer_returns_to_the_running_epoch() { + let mut model = stateless_session(48_000); + model.tracks.truncate(2); + model.routes.truncate(2); + model.tracks[0].inserts.effects.push(compressor()); + let mut source = model.sources[0].clone(); + source.id = id("s-source"); + model.tracks[1].source_id = source.id.clone(); + model.sources.push(source); + model.sources.sort_by(|left, right| left.id.cmp(&right.id)); + let first = [put(&model, 0, gate())]; + let second = [ + SessionEdit::RemoveRoute { + route_id: model.routes[1].id.clone(), + }, + SessionEdit::RemoveTrack { + track_id: model.tracks[1].id.clone(), + }, + SessionEdit::RemoveSource { + source_id: id("s-source"), + }, + ]; + let mut children = compile(&model, limits()); + submit_direct(&mut children, &[FIXTURE, ADDED], 0); + render_direct(&mut children, 0); + submit_direct(&mut children, &[FIXTURE, ADDED], 1); + commit(&mut children, 1, 42, &first); + let running = children.session.test_providers().test_epoch(); + let vacant = |children: &mut CompiledChildren, epoch: u64, source: &[u8]| { + children + .session + .test_epoch_mut(epoch) + .expect("the epoch") + .test_source_vacant(source) + }; + assert_eq!( + vacant(&mut children, running, ADDED), + Some(true), + "A holds `s`'s producer" + ); + let disposed = children.session.test_owner_counters(); + commit(&mut children, 2, 43, &second); + let counters = children.session.test_owner_counters(); + assert_eq!( + counters.candidate_superseded, + disposed.candidate_superseded + 1 + ); + assert_eq!( + counters.candidate_provider_disposed, + disposed.candidate_provider_disposed + 1, + "A's epoch is dropped" + ); + let replacement = children.session.test_pending_providers()[0].test_epoch(); + assert_eq!( + vacant(&mut children, running, ADDED), + Some(false), + "`s`'s producer is back in the running epoch" + ); + assert_eq!( + vacant(&mut children, running, FIXTURE), + Some(true), + "`u`'s producer moved on to the replacement" + ); + assert_eq!(vacant(&mut children, replacement, ADDED), None); + assert_eq!(vacant(&mut children, replacement, FIXTURE), Some(false)); + // B's adoption retires the running epoch, `s`'s producer with it. + submit_direct(&mut children, &[FIXTURE], 2); + render_direct(&mut children, 1); + children.session.service().expect("service"); + assert_eq!(children.session.test_providers().test_epoch(), replacement); + assert_eq!( + children + .session + .test_owner_counters() + .current_provider_disposed, + counters.current_provider_disposed + 1 + ); +} + +/// `model` with a fourth track `t4` on the fixture source, a copy of the first track with its +/// route: no transaction below changes its chain, so every plan carries it from P0. +fn with_t4(mut model: SessionModel) -> SessionModel { + let mut t4 = model.tracks[0].clone(); + t4.id = id("t4"); + t4.inserts.effects.clear(); + let mut route = model.routes[0].clone(); + route.id = id("t4-main"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("track route") + }; + *track_id = t4.id.clone(); + model.tracks.push(t4); + model.routes.push(route); + model +} + +/// The `SET_CONTROL`s of `t4`'s fader at `left_db`/`right_db`, the two gain rows with `ramp`. +fn t4_fader(left_db: f32, right_db: f32, ramp: Option) -> Vec { + let fader = session::DualMonoFader { + left_db, + right_db, + left_mute: false, + right_mute: false, + }; + let mut edits = protocol::control_core::fader_edits(&id("t4"), &fader); + for edit in &mut edits[..2] { + let SessionEdit::SetControl { ramp_samples, .. } = edit else { + panic!("a control edit") + }; + *ramp_samples = ramp; + } + edits +} + +/// Renders blocks 0-8 of `model` fed on the fixture source, with `steps` committed one after +/// another after block 0 (each `(edits, advances)`: whether it commits a model revision) and +/// then with `combined` committed as one transaction instead. Returns both block lists. +fn superseding_and_combined( + model: &SessionModel, + steps: &[(Vec, bool)], + combined: &[SessionEdit], +) -> (Vec>, Vec>) { + let run = |superseding: bool| { + let mut children = compile(model, limits()); + submit_direct(&mut children, &[FIXTURE], 0); + let mut blocks = vec![render_direct(&mut children, 0)]; + if superseding { + let mut base = 42; + for (request, (edits, advances)) in steps.iter().enumerate() { + if *advances { + commit(&mut children, request as u64 + 1, base, edits); + base += 1; + } else { + // A transaction of solo writes commits no model (#1559 D4). + children + .session + .command(&transaction(request as u64 + 1, base, edits), 4_096) + .unwrap_or_else(|error| panic!("solo: {error:?}")); + assert_eq!(revision(&children), base); + } + } + assert_eq!( + children.session.test_owner_counters().candidate_superseded, + steps.iter().filter(|(_, advances)| *advances).count() as u64 - 1, + "every structural step but the last was superseded" + ); + } else { + commit(&mut children, 1, 42, combined); + } + for block in 1..9 { + submit_direct(&mut children, &[FIXTURE], block); + blocks.push(render_direct(&mut children, block)); + } + blocks + }; + (run(true), run(false)) +} + +fn assert_same_blocks(label: &str, actual: &[Vec], expected: &[Vec]) { + assert_eq!(actual.len(), expected.len()); + for (block, (actual, expected)) in actual.iter().zip(expected).enumerate() { + assert!( + expected.iter().any(|sample| *sample != 0.0), + "{label} block {block}: the twin plays" + ); + assert_eq!(bits(actual), bits(expected), "{label} block {block}"); + } +} + +/// #1310 (D3 with root ruling B15-Q3, "B wins on a shared row"): T1 moves `t4`'s fader with its own +/// ramp length; T2, which supersedes T1, moves it again with no length. The replacement ramps +/// `t4` with the session default, as a transaction holding T1's structural edit and T2 does: B +/// wrote the row last, so T1's length no longer applies to it. +/// +/// Test value: red if the merge keeps T1's length on a row B wrote without one (it lays only B's +/// lengths over T1's), so `t4` steps where the combined transaction ramps. +#[test] +fn a_superseding_edit_without_a_ramp_length_takes_the_default_on_its_row() { + let mut model = with_t4(stateless_session(48_000)); + model.tracks[1].inserts.effects.push(compressor()); + let first: Vec = [put(&model, 0, compressor())] + .into_iter() + .chain(t4_fader(-6.0, -9.0, Some(1))) + .collect(); + let second: Vec = [put(&model, 1, gate())] + .into_iter() + .chain(t4_fader(-12.0, -3.0, None)) + .collect(); + // One transaction may not give a row two lengths (`control.ramp.conflict`), so the reference + // holds the last write of `t4`'s fader only, which is what both edits together leave. + let combined: Vec = [put(&model, 0, compressor())] + .into_iter() + .chain(second.iter().cloned()) + .collect(); + let (superseded, twin) = + superseding_and_combined(&model, &[(first, true), (second, true)], &combined); + assert_same_blocks("ramp", &superseded, &twin); +} + +/// #1310 (D3 with root ruling B15-Q3, the kept monitor overlay): T1 is pending, a solo of `t1` with its +/// own ramp length commits on it, and T2 supersedes T1. The replacement renders the solo from the +/// overlay the running plan's cells hold, with the solo's ramp: exactly what one transaction +/// holding T1, the solo and T2 renders. +/// +/// Test value: red if the replacement takes the session's current overlay as the one the running +/// plan holds (the solo is then no change, and the other tracks stay unmuted), or if it forgets the +/// solo's ramp length (the mutes ramp with `mute_ms` instead). +#[test] +fn a_solo_committed_while_a_candidate_waits_holds_on_its_replacement() { + let mut model = with_t4(stateless_session(48_000)); + model.tracks[1].inserts.effects.push(compressor()); + let first = vec![put(&model, 0, compressor())]; + let solo = vec![SessionEdit::set_control( + protocol::control_core::row_address( + protocol::control_core::Owner::Strip(model.tracks[0].id.clone()), + protocol::control_core::Section::Own, + "solo", + protocol::control_core::Lane::Both, + ) + .expect("the solo row"), + 1.0, + Some(1), + )]; + let second = vec![put(&model, 1, gate())]; + let combined: Vec = first.iter().chain(&solo).chain(&second).cloned().collect(); + let (superseded, twin) = superseding_and_combined( + &model, + &[(first, true), (solo, false), (second, true)], + &combined, + ); + assert_same_blocks("solo", &superseded, &twin); +} + +/// #1310 (D3 with root ruling B15-Q3, the live path's merge): T1 is pending, a live edit moves `t4`'s +/// fader with its own ramp length on T1's lanes, and T2 supersedes T1. The replacement carries +/// `t4` from the running plan and retargets its fader at the live edit's ramp length: exactly +/// what one transaction holding T1, the fader edit and T2 renders. +/// +/// Test value: red if a live edit committed while a candidate waits does not join the kept base's +/// ramp lengths (the live path's merge deleted): the replacement then ramps `t4`'s fader with the +/// session default where the combined transaction ramps it over one sample. +#[test] +fn a_live_ramp_committed_while_a_candidate_waits_holds_on_its_replacement() { + let mut model = with_t4(stateless_session(48_000)); + model.tracks[1].inserts.effects.push(compressor()); + let first = vec![put(&model, 0, compressor())]; + let live = t4_fader(-6.0, -9.0, Some(1)); + let second = vec![put(&model, 1, gate())]; + let combined: Vec = first.iter().chain(&live).chain(&second).cloned().collect(); + let run = |superseding: bool| { + let mut children = compile(&model, limits()); + submit_direct(&mut children, &[FIXTURE], 0); + let mut blocks = vec![render_direct(&mut children, 0)]; + if superseding { + commit(&mut children, 1, 42, &first); + let pending = children.session.test_pending_providers()[0].test_epoch(); + commit(&mut children, 2, 43, &live); + assert_eq!( + children.session.test_pending_providers()[0].test_epoch(), + pending, + "the fader edit rode the candidate's live lanes" + ); + commit(&mut children, 3, 44, &second); + assert_eq!( + children.session.test_owner_counters().candidate_superseded, + 1, + "T2 superseded T1" + ); + } else { + commit(&mut children, 1, 42, &combined); + } + for block in 1..9 { + submit_direct(&mut children, &[FIXTURE], block); + blocks.push(render_direct(&mut children, block)); + } + blocks + }; + assert_same_blocks("live ramp", &run(true), &run(false)); +} + +/// Renders block `block` as the first half of `PlanState::render` does: the owner swaps to a +/// published candidate and retires the plan it displaced, but the atomic the control plane reads +/// is not published (a render call preempted before it returns). +fn render_unpublished(children: &mut CompiledChildren, block: u64) -> Vec { + let mut pcm = vec![f32::NAN; 256]; + children + .plan + .test_owner_mut() + .render_contiguous( + RenderIo { + output: PlanarBufferMut::try_new(&mut pcm, 2, 128, 128).expect("output"), + }, + block * 128, + ) + .expect("block"); + pcm +} + +/// Batch 15a review fold M1: render claims candidate A and is preempted before A's predecessor +/// reaches the control side, so a structural edit C finds A taken but cannot promote it; C +/// publishes B on A. Render then adopts B as well, and edit D finds B taken with A still pending. +/// Each edit that finds the newest candidate taken promotes every pending candidate up to it, so +/// D commits, the blocks equal a run whose renders each published their epoch, and once the +/// retirements reach the control side every plan and owner is accounted for. +/// +/// Test value: red if a taken candidate stays pending until its predecessor's retirement is +/// reclaimed: D then meets two pending candidates and a full pending list, the accounting fault +/// of #1398 D3 (`debug_assert!`, `INTERNAL`). +#[test] +fn edits_after_render_took_two_candidates_unretired_each_commit() { + let model = stateless_session(48_000); + let edits = [ + add_source_track(&model).to_vec(), + vec![put(&model, 1, compressor())], + vec![put(&model, 2, compressor())], + ]; + let run = |race: bool| { + control_plane::test_reset_lifecycle_observer(); + let mut children = compile(&model, limits()); + let mut blocks = Vec::new(); + submit_direct(&mut children, &[FIXTURE], 0); + blocks.push(render_direct(&mut children, 0)); + control_plane::test_hold_reclamation(race); + for (index, edits) in edits.iter().enumerate() { + let block = index as u64 + 1; + commit(&mut children, block, 41 + block, edits); + if index == 0 { + submit_direct(&mut children, &[ADDED], 0); + } else { + submit_direct(&mut children, &[ADDED], block - 1); + } + submit_direct(&mut children, &[FIXTURE], block); + blocks.push(if race { + render_unpublished(&mut children, block) + } else { + render_direct(&mut children, block) + }); + } + if race { + let counters = children.session.test_owner_counters(); + assert_eq!( + (counters.candidate_taken, counters.candidate_withdrawn), + (2, 0), + "both later edits found the newest candidate taken" + ); + } + control_plane::test_hold_reclamation(false); + submit_direct(&mut children, &[FIXTURE], 4); + submit_direct(&mut children, &[ADDED], 3); + blocks.push(render_direct(&mut children, 4)); + children.session.service().expect("service"); + assert_eq!(revision(&children), 45); + assert!(children.session.test_pending_providers().is_empty()); + assert!(children.session.test_retired_providers().is_empty()); + drop(children); + assert_balanced( + if race { "race" } else { "paced" }, + control_plane::test_lifecycle_counters(), + ); + blocks + }; + let paced = run(false); + let raced = run(true); + assert!( + paced[3].iter().any(|sample| *sample != 0.0), + "the compared blocks play" + ); + for (block, (actual, expected)) in raced.iter().zip(&paced).enumerate() { + assert_eq!(bits(actual), bits(expected), "block {block}"); + } +} + +/// Batch 15a review fold 2, B-1: render claims A and is preempted before it retires P0; edit C +/// finds A taken and parks P0; render adopts B and retires A; edit D finds B taken and parks A; +/// render then adopts C and publishes its epoch. The control side now holds two parked epochs, the +/// running B and the pending C, with the atomic naming C. The next service step reclaims P0's, A's +/// and B's plans and promotes C, and the session takes edits again. +/// +/// Test value: red if the parked list holds fewer epochs than the retirement queue has credits +/// (the step that reclaims nothing cannot park a third), and red if, with that list short, the +/// service step promotes from the atomic before it reclaims: every later call then answers +/// `INTERNAL`, permanently. +#[test] +fn two_taken_candidates_and_a_published_adoption_never_wedge_the_session() { + let model = stateless_session(48_000); + let edits = [ + add_source_track(&model).to_vec(), + vec![put(&model, 1, compressor())], + vec![put(&model, 2, compressor())], + ]; + control_plane::test_reset_lifecycle_observer(); + let mut children = compile(&model, limits()); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + control_plane::test_hold_reclamation(true); + for (index, edits) in edits.iter().enumerate() { + let block = index as u64 + 1; + commit(&mut children, block, 41 + block, edits); + submit_direct(&mut children, &[ADDED], block - 1); + submit_direct(&mut children, &[FIXTURE], block); + if index < 2 { + render_unpublished(&mut children, block); + } else { + render_direct(&mut children, block); + } + } + let snapshot = children.session.test_transaction_snapshot(); + assert_eq!( + ( + snapshot.retired_provider_epochs, + snapshot.provider_epoch, + snapshot.pending_provider_epochs, + snapshot.active_plan_epoch, + ), + (vec![0, 1], 2, vec![3], 3), + "two parked epochs, B running on the control side, C adopted by render" + ); + // A step that reclaims nothing (the seam holds every plan) still promotes C from the atomic: + // the parked list holds one epoch per retirement credit, so the third fits. + children + .session + .service() + .expect("a step that reclaims nothing promotes C"); + assert_eq!( + children + .session + .test_transaction_snapshot() + .retired_provider_epochs, + vec![0, 1, 2] + ); + control_plane::test_hold_reclamation(false); + for block in 4..8 { + submit_direct(&mut children, &[ADDED], block - 1); + submit_direct(&mut children, &[FIXTURE], block); + render_direct(&mut children, block); + children + .session + .service() + .unwrap_or_else(|error| panic!("block {block}: {error:?}")); + } + assert_eq!(children.session.test_providers().test_epoch(), 3); + assert!(children.session.test_pending_providers().is_empty()); + assert!(children.session.test_retired_providers().is_empty()); + commit(&mut children, 4, 45, &[remove(&model, 1)]); + render_direct(&mut children, 8); + children.session.service().expect("service"); + drop(children); + assert_balanced("wedge", control_plane::test_lifecycle_counters()); +} + +/// Batch 15a review fold 2, M-1: render claims A and retires P0, and the service step has not yet +/// reclaimed P0's plan. Edit T2 finds A taken and promotes it, but P0's plan is still alive, so +/// the admission charges P0, A and B together. With gate 4's cap (above every two-plan peak, +/// below the three-plan peak) T2 is refused as `BACKPRESSURE`; one service step reclaims P0 and +/// the same T2 commits. +/// +/// With a source-total cap instead, the parked P0 is charged with each carried ring once: T2 is +/// admitted at the counted-once peak and refused one byte below. +/// +/// Test value: red if a parked epoch's plan is left out of the admission after a `Taken` +/// promotion (T2 is then admitted at the three-plan peak while P0's plan still lives, and one +/// byte below the source peak), and red if the parked epoch's place in the held list double counts +/// a ring A carried from it (refused at the counted-once peak). +#[test] +fn a_displaced_plan_not_yet_reclaimed_is_charged_after_a_taken_promotion() { + let model = stateless_session(48_000); + let first = add_source_track(&model); + let second = [put(&model, 1, compressor())]; + let (p0, a, b) = graph_rows(&model, &first, &second); + let two_plans = (p0.0 + a.0 + p0.1 + a.1).max(a.0 + b.0 + a.1 + b.1); + let three_plans = p0.0 + a.0 + b.0 + p0.1 + a.1 + b.1; + let cap = two_plans + (p0.0 + p0.1) / 2; + assert!(cap < three_plans, "{cap} against {three_plans}"); + let mut limits = limits(); + limits.maximum_graph_session_plus_plan_bytes = cap; + + control_plane::test_reset_lifecycle_observer(); + let mut children = compile(&model, limits); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + commit(&mut children, 1, 42, &first); + control_plane::test_hold_reclamation(true); + submit_direct(&mut children, &[ADDED], 0); + submit_direct(&mut children, &[FIXTURE, ADDED], 1); + render_unpublished(&mut children, 1); + let refused = children + .session + .command(&transaction(2, 43, &second), 4_096); + assert!( + matches!(refused, Err(CommandError::Backpressure)), + "P0's plan is alive and charged: {refused:?} (cap {cap}, three plans {three_plans})" + ); + assert_eq!(children.session.test_owner_counters().candidate_taken, 1); + assert_eq!(revision(&children), 43); + assert_eq!( + children + .session + .test_transaction_snapshot() + .retired_provider_epochs, + vec![0], + "P0 is parked until its plan is reclaimed" + ); + control_plane::test_hold_reclamation(false); + children.session.service().expect("service"); + assert!(children.session.test_retired_providers().is_empty()); + commit(&mut children, 3, 43, &second); + submit_direct(&mut children, &[FIXTURE, ADDED], 2); + render_direct(&mut children, 2); + children.session.service().expect("service"); + drop(children); + assert_balanced("parked charge", control_plane::test_lifecycle_counters()); + + // #1398 M2 with the parked epoch: each carried ring is counted once, in the oldest held plan + // that holds it. The source peak is P0's rows, A's less what A carried from P0, and B's less + // what B carried from A: admitted at that cap, `BACKPRESSURE` one byte below. + let source_rows = { + let mut children = compile(&model, crate::runtime::tests::limits()); + let row = |children: &CompiledChildren| { + ( + children.plan.resources().source_total_bytes, + children.session.test_providers().test_carried().0, + ) + }; + let p0 = row(&children); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + commit(&mut children, 1, 42, &first); + render_direct(&mut children, 1); + children.session.service().expect("service"); + let a = row(&children); + commit(&mut children, 2, 43, &second); + render_direct(&mut children, 2); + children.session.service().expect("service"); + [p0, a, row(&children)] + }; + assert!( + source_rows[1].1 > 0 && source_rows[2].1 > 0, + "A and B carry rings: {source_rows:?}" + ); + let counted_once = source_rows[0].0 + source_rows[1].0 - source_rows[1].1 + source_rows[2].0 + - source_rows[2].1; + for (cap, admitted) in [(counted_once, true), (counted_once - 1, false)] { + let mut limits = crate::runtime::tests::limits(); + limits.maximum_source_total_bytes = cap; + let mut children = compile(&model, limits); + submit_direct(&mut children, &[FIXTURE], 0); + render_direct(&mut children, 0); + commit(&mut children, 1, 42, &first); + control_plane::test_hold_reclamation(true); + submit_direct(&mut children, &[ADDED], 0); + submit_direct(&mut children, &[FIXTURE, ADDED], 1); + render_unpublished(&mut children, 1); + let result = children + .session + .command(&transaction(2, 43, &second), 4_096); + assert_eq!( + children + .session + .test_transaction_snapshot() + .retired_provider_epochs, + vec![0], + "cap {cap}: P0 parked" + ); + control_plane::test_hold_reclamation(false); + if admitted { + assert!(result.is_ok(), "cap {cap} = {source_rows:?}: {result:?}"); + } else { + assert!( + matches!(result, Err(CommandError::Backpressure)), + "cap {cap}: {result:?}" + ); + } + } +} + +/// `UpsertSource` of `s-source` as `model` declares it, with its content identity's `7e94` +/// replaced by `content` when given: a declaration change that restarts the source in a fresh +/// ring. +fn upsert_added_source(model: &SessionModel, content: Option<&str>) -> SessionEdit { + let mut source = add_source_track(model) + .into_iter() + .find_map(|edit| match edit { + SessionEdit::UpsertSource { source } => Some(source), + _ => None, + }) + .expect("the added source"); + if let Some(content) = content { + source.content = source.content.replace("7e94", content); + assert!( + source.content != model.sources[0].content, + "the content identity changed" + ); + } + SessionEdit::UpsertSource { source } +} + +/// `RemoveRoute`, `RemoveTrack` and `RemoveSource` of what [`add_source_track`] adds. +fn remove_source_track() -> Vec { + vec![ + SessionEdit::RemoveRoute { + route_id: id("ts-main"), + }, + SessionEdit::RemoveTrack { track_id: id("ts") }, + SessionEdit::RemoveSource { + source_id: id("s-source"), + }, + ] +} + +/// One case of the acked-fresh-source guard (#1310, root ruling M-A of 2026-10-16): what P0, A +/// and B do to `s-source`. +struct FreshSourceCase { + label: &'static str, + /// Whether P0 declares `s-source` (with its track): committed and adopted before A. + in_p0: bool, + /// T(A): creates `s-source`'s ring fresh in A (adds or redeclares it). + first: Vec, + /// T(B): does not keep `s-source` as A declares it. + second: Vec, + /// Whether B declares `s-source`, restarting it fresh: the host refeeds it from frame 0. + in_b: bool, +} + +fn fresh_source_cases(model: &SessionModel) -> Vec { + let redeclared = |content| vec![upsert_added_source(model, Some(content))]; + vec![ + FreshSourceCase { + label: "(a) A redeclares, B reverts to P0", + in_p0: true, + first: redeclared("0e0e"), + second: vec![upsert_added_source(model, None)], + in_b: true, + }, + FreshSourceCase { + label: "(b) A redeclares, B redeclares a third way", + in_p0: true, + first: redeclared("0e0e"), + second: redeclared("1e1e"), + in_b: true, + }, + FreshSourceCase { + label: "(c) A redeclares, B removes", + in_p0: true, + first: redeclared("0e0e"), + second: remove_source_track(), + in_b: false, + }, + FreshSourceCase { + label: "(d) A adds, B removes", + in_p0: false, + first: add_source_track(model).to_vec(), + second: remove_source_track(), + in_b: false, + }, + FreshSourceCase { + label: "(d) A adds, B redeclares", + in_p0: false, + first: add_source_track(model).to_vec(), + second: redeclared("1e1e"), + in_b: true, + }, + ] +} + +/// #1310, root rulings of 2026-10-16 (no ack before a drop; M-A of fold 3). P0 plays the fixture +/// source and, in cases (a)-(c), `s-source`. T(A) creates `s-source`'s ring fresh in A (it +/// redeclares it, or in case (d) adds it), and the host's chunk for it is acked into A's ring. +/// T(B) does not keep `s-source` as A declares it: it reverts it to P0's declaration (a), +/// redeclares it a third way (b, d), or removes it (c, d). B does not take A's ring by donation, +/// so A's acked chunk would drop with A. So T(B) is refused with `BACKPRESSURE` before its commit: +/// the revision stays T(A)'s, A stays the candidate, no event is queued. The next block adopts A +/// and plays the acked chunk; the same T(B) then commits on A. Every block equals the sequential +/// run (T(B) submitted only after A's adoption), and the acked chunk is heard (the block differs +/// from a run whose host never fed A's ring). +/// +/// Test value: red if the guard is missing or narrower than the ruling (the fold-2 guard let +/// (b)-(d) commit at once and supersede A), or if the refusal changes anything (a block, the +/// revision, the candidate or an event differs). +#[test] +fn a_supersession_that_would_drop_acked_pcm_on_a_fresh_source_is_backpressure() { + let model = stateless_session(48_000); + for case in fresh_source_cases(&model) { + let label = case.label; + let run = |attempt_early: bool, feed_a: bool| { + let mut children = compile(&model, limits()); + let mut blocks = Vec::new(); + let mut base = 42; + submit_direct(&mut children, &[FIXTURE], 0); + blocks.push(render_direct(&mut children, 0)); + if case.in_p0 { + commit(&mut children, 1, base, &add_source_track(&model)); + base += 1; + submit_direct(&mut children, &[ADDED], 0); + } + submit_direct(&mut children, &[FIXTURE], 1); + blocks.push(render_direct(&mut children, 1)); + children.session.service().expect("service"); + commit(&mut children, 2, base, &case.first); + base += 1; + let candidate = children.session.test_pending_providers()[0].test_epoch(); + if feed_a { + // A created `s-source` at generation 1, frame 0: this chunk is acked into A's ring. + submit_direct(&mut children, &[ADDED], 0); + } + submit_direct(&mut children, &[FIXTURE], 2); + if attempt_early { + let refused = children + .session + .command(&transaction(3, base, &case.second), 4_096); + assert!( + matches!(refused, Err(CommandError::Backpressure)), + "{label}: the supersession would drop A's acked chunk: {refused:?}" + ); + assert_eq!( + revision(&children), + base, + "{label}: the revision does not move" + ); + let pending: Vec = children + .session + .test_pending_providers() + .iter() + .map(|provider| provider.test_epoch()) + .collect(); + assert_eq!(pending, vec![candidate], "{label}: A stays the candidate"); + assert!( + children + .session + .dequeue_event(EventLane::Reliable, 4_096) + .expect("the event lane") + .is_none(), + "{label}: the refusal queues no event" + ); + } + blocks.push(render_direct(&mut children, 2)); + children.session.service().expect("service"); + assert_eq!( + children.session.test_providers().test_epoch(), + candidate, + "{label}: A is adopted" + ); + commit(&mut children, 4, base, &case.second); + for block in 3..6 { + if case.in_b { + // B restarts `s-source` again: the host refeeds it from frame 0. + submit_direct(&mut children, &[ADDED], block - 3); + } + submit_direct(&mut children, &[FIXTURE], block); + blocks.push(render_direct(&mut children, block)); + } + blocks + }; + let sequential = run(false, true); + let refused = run(true, true); + for (block, (actual, expected)) in refused.iter().zip(&sequential).enumerate() { + assert_eq!(bits(actual), bits(expected), "{label}: block {block}"); + } + let unfed = run(false, false); + assert_ne!( + bits(&sequential[2]), + bits(&unfed[2]), + "{label}: A's adoption block plays the acked chunk" + ); + } +} diff --git a/crates/capi/src/runtime/tests.rs b/crates/capi/src/runtime/tests.rs index 506f9fbf8..b67c795fc 100644 --- a/crates/capi/src/runtime/tests.rs +++ b/crates/capi/src/runtime/tests.rs @@ -976,16 +976,6 @@ fn structural_command_keeps_protocol_plan_provider_and_event_epochs_atomic() { ExpectedRevision::Exact(SessionRevision(43)), protocol::CommandPayload::SessionTransactionApply(core::slice::from_ref(&second_edit)), ); - assert!(matches!( - children.session.command(&second_request, 4_096), - Err(CommandError::Backpressure) - )); - assert_eq!( - children.session.test_controller().session().revision(), - SessionRevision(43) - ); - assert_eq!(children.session.test_controller().replay().len(), 1); - // The swap block's continuity is `a_c_abi_structural_transaction_keeps_the_source_playing`'s. children .plan @@ -1114,7 +1104,7 @@ fn continuity_signal(frame: u64, channel: usize, salt: u64) -> f32 { } /// One block of [`continuity_signal`] at source frame `block * 128`. -fn continuity_block(block: u64, salt: u64) -> (Vec, Vec) { +pub(super) fn continuity_block(block: u64, salt: u64) -> (Vec, Vec) { let start = block * 128; ( (start..start + 128) @@ -1461,7 +1451,7 @@ fn an_added_c_abi_source_starts_at_its_anchored_render_sample() { } /// Submits block `block` of each of `ids`' signals (salted by position) through the session state. -fn submit_direct(children: &mut CompiledChildren, ids: &[&[u8]], block: u64) { +pub(super) fn submit_direct(children: &mut CompiledChildren, ids: &[&[u8]], block: u64) { for (salt, id) in ids.iter().enumerate() { let (left, right) = continuity_block(block, salt as u64); let report = children @@ -1483,7 +1473,7 @@ fn submit_direct(children: &mut CompiledChildren, ids: &[&[u8]], block: u64) { } /// Renders block `block` through the plan state. -fn render_direct(children: &mut CompiledChildren, block: u64) -> Vec { +pub(super) fn render_direct(children: &mut CompiledChildren, block: u64) -> Vec { let mut pcm = vec![f32::NAN; 256]; children .plan @@ -1496,7 +1486,7 @@ fn render_direct(children: &mut CompiledChildren, block: u64) -> Vec { } /// Feeds `ids` one block ahead and renders `blocks` blocks from `first`. -fn feed_and_render_direct( +pub(super) fn feed_and_render_direct( children: &mut CompiledChildren, ids: &[&[u8]], first: u64, @@ -1705,6 +1695,11 @@ fn control_outcome(result: &Result) -> String { /// report row, skipped because it matched the lagging atomic, kept the table full, so every later /// structural command returned `Backpressure`. /// +/// #1398 gate 1: the structural command inside the window is admitted against the running plan +/// and commits there, while the report table holds three rows (the lagging atomic's, the running +/// plan's and the new candidate's); the next round's swapping block adopts its plan. Red if the +/// lag refusal is kept or the report table is a row short (`Backpressure` in the window). +/// /// Issue #1273 gate 4: the trigger adds and removes a muted track (`UpsertTrack`), so every swap /// carries the source's ring, and the source is fed twice a round: after the commit, while the /// candidate is pending, and inside the window. Both submits must land in the carried ring. Test @@ -1827,6 +1822,14 @@ fn control_calls_inside_a_plan_swapping_render_call_keep_replacement_live() { .find_map(|(epoch, report)| (*epoch == old_epoch).then_some(*report)), Some(window_query) ); + assert_eq!( + window_rows + .iter() + .map(|(epoch, _)| *epoch) + .collect::>(), + vec![old_epoch, new_epoch, new_epoch + 1], + "round {round}: the lagging row, the running plan's and the window's candidate" + ); // Second half of `PlanState::render`: publish the epoch that rendered the block. children @@ -1855,10 +1858,10 @@ fn control_calls_inside_a_plan_swapping_render_call_keep_replacement_live() { ( "ok".to_owned(), "ok".to_owned(), - // The window's structural command waits for the lagging row (transient - // backpressure), acks nothing, and commits nothing. - "Backpressure".to_owned(), - revision, + // #1398: the window's structural command is admitted against the running plan + // and commits; the same request after the window is its replay. + "ok".to_owned(), + revision + 1, "ok".to_owned(), new_epoch, Vec::new(), @@ -1867,22 +1870,43 @@ fn control_calls_inside_a_plan_swapping_render_call_keep_replacement_live() { ), "round {round}: a control call inside the swapping render call" ); + assert_eq!( + children.session.test_controller().session().revision().0, + revision + 1, + "round {round}: the replay commits nothing more" + ); revision += 1; } + // The last window's candidate is adopted at the next block. + let last = children.session.test_pending_providers()[0].test_epoch(); + feed(&mut children, "last candidate"); + children + .plan + .render( + 5 * 128, + PlanarBufferMut::try_new(&mut pcm, 2, 128, 128).expect("output"), + ) + .expect("adopting block"); + assert_eq!(children.plan.owner().active_epoch().0, last); } -/// Issue #1042, attempt 2: a valid structural edit inside the swap window is backpressured, never -/// reported as a compile rejection. +/// Issue #1042, attempt 2, and #1398 gate 1: a valid structural edit inside the swap window is +/// admitted against the running plan, never refused. /// -/// The peak check pairs a candidate with the resource row of the epoch the atomic names. Inside -/// the window that is the retired plan, not the one rendering. The limit here admits exactly the -/// pairs that really coexist (nine EQs with eight, either way round) and refuses nine with nine. -/// The test removes one EQ and puts it back twice: while the eight-EQ plan is still pending, and -/// inside the window after the swap. Before the fix, both calls paired nine EQs with nine and -/// returned `CompileRejected(effect.resource.limit)`; the same request was admitted once the -/// render call published its epoch. +/// The limit here admits exactly the pairs that really coexist (nine EQs with eight, either way +/// round) and refuses nine with nine. The test removes one EQ and puts it back twice: while the +/// eight-EQ plan is still pending, and inside the window after the swap. While the eight-EQ plan +/// is pending the edit would supersede it (#1310), so the running nine-EQ plan, the withdrawn +/// eight-EQ plan and the new nine-EQ plan are held at once: the three-plan peak is over the limit +/// (#1310 gate 4). The edit fits beside the eight-EQ plan alone, which is all a retry meets once +/// render adopts it, so the refusal is retryable `Backpressure`, not a permanent compile rejection +/// (batch 15a review fold, item 2). Inside the window the nine-EQ plan is reclaimed, so the edit +/// sits beside the eight-EQ plan alone and is admitted; its plan is adopted at the next block, +/// and the report table holds three rows meanwhile. Red if the lag refusal is kept (the window +/// answers `Backpressure`), if the admission pairs the candidate with the atomic's retired row +/// (`effect.resource.limit`), or if the three-plan refusal is permanent. #[test] -fn a_valid_edit_inside_the_swap_window_is_backpressured_not_compile_rejected() { +fn a_valid_edit_inside_the_swap_window_is_admitted_against_the_running_plan() { // The fixture's EQ is a console slot on all nine tracks, which a track cannot drop (decision // 12), so the edited EQ is a tenth instance: an insert on the first track. Console session // edits are #1094's. @@ -1946,8 +1970,9 @@ fn a_valid_edit_inside_the_swap_window_is_backpressured_not_compile_rejected() { .is_some() ); let request = edit(2, 43, &put_back); - // Before the swap the eight-EQ plan is still pending: the edit waits for it, and it is the - // plan the edit will sit beside, so it is backpressure here too, not a compile rejection. + // Before the swap the eight-EQ plan is still pending: superseding it would hold three plans, + // which the limit refuses until a render adopts it, so the refusal is retryable; the + // eight-EQ plan is republished and adopted below. let pending = children.session.command(&request, 4_096); // First half of `PlanState::render`: swap to the eight-EQ plan. @@ -1962,36 +1987,50 @@ fn a_valid_edit_inside_the_swap_window_is_backpressured_not_compile_rejected() { ) .expect("swapping block"); assert_eq!(report.swap, engine::realtime::SwapOutcome::Applied); - let compiled_before = test_lifecycle_counters().candidate_plan_constructed; let window = children.session.command(&request, 4_096); - // Refused before compiling: the report table is full while the atomic lags. - let window_compiles = test_lifecycle_counters().candidate_plan_constructed - compiled_before; let window_revision = children.session.test_controller().session().revision().0; + let window_rows = children + .session + .test_transaction_snapshot() + .resource_rows + .iter() + .map(|(epoch, _)| *epoch) + .collect::>(); // Second half: publish the epoch that rendered the block. children .plan .test_active_epoch() .store(report.active_epoch.0, Ordering::Release); + // The same request again is the window's replay. let after = children.session.command(&request, 4_096); assert_eq!( ( control_outcome(&pending), control_outcome(&window), - window_compiles, window_revision, + window_rows, control_outcome(&after), children.session.test_controller().session().revision().0, ), ( - "Backpressure".to_owned(), - "Backpressure".to_owned(), - 0, - 43, + control_outcome::<(), _>(&Err(CommandError::Backpressure)), + "ok".to_owned(), + 44, + vec![0, 1, 2], "ok".to_owned(), 44 ), "the eight-EQ plan rendering beside nine EQs fits; only a retired row says it does not" ); + // The window's plan is adopted at the next block. + children + .plan + .render( + 256, + PlanarBufferMut::try_new(&mut pcm, 2, 128, 128).expect("output"), + ) + .expect("adopting block"); + assert_eq!(children.plan.owner().active_epoch().0, 2); } /// Issue #1042 on one thread: a *rejected* render call can open the window too. @@ -2026,8 +2065,16 @@ fn a_rejected_render_call_that_swaps_plans_leaves_the_c_session_live() { protocol::CommandPayload::SessionTransactionApply(core::slice::from_ref(&edit)), ) }; - let mut request = structural(1, 42); - assert_eq!(command_c(c_session, &request).0, crate::RESULT_OK); + // #1398 gate 1: each round publishes a candidate, a rejected render call swaps to it, and a + // structural command inside that window is admitted against the running plan and commits; the + // next good block adopts its plan. The retry of the same request is its replay. + let mut request_id = 1; + let mut revision = 42; + assert_eq!( + command_c(c_session, &structural(request_id, revision)).0, + crate::RESULT_OK + ); + revision += 1; for round in 1..=4_u64 { let next_sample = round * 128; @@ -2037,11 +2084,24 @@ fn a_rejected_render_call_that_swaps_plans_leaves_the_c_session_live() { assert_eq!(pending_before, 1, "round {round}: a candidate is published"); let rejected = crate::ffi::test_render(c_plan, next_sample + 1, &output); let (lossy, _) = event_c(c_session, crate::EVENT_LANE_LOSSY); - request = structural(round + 1, 42 + round); + request_id += 1; + let request = structural(request_id, revision); let (window, canary) = command_c(c_session, &request); - let (revision, _, provider, pending) = crate::ffi::test_session_state_summary(c_session); + let (window_revision, _, provider, pending) = + crate::ffi::test_session_state_summary(c_session); + let window_rows = crate::ffi::test_transaction_snapshot(c_session) + .resource_rows + .len(); let good = crate::ffi::test_render(c_plan, next_sample, &output); let (after, _) = command_c(c_session, &request); + let (after_revision, _, after_provider, after_pending) = + crate::ffi::test_session_state_summary(c_session); + // Drain the window's `SESSION_COMMITTED`, then publish the next round's candidate. + let (window_event, _) = event_c(c_session, crate::EVENT_LANE_RELIABLE); + revision += 1; + request_id += 1; + let (next, _) = command_c(c_session, &structural(request_id, revision)); + revision += 1; assert_eq!( ( reliable, @@ -2049,25 +2109,44 @@ fn a_rejected_render_call_that_swaps_plans_leaves_the_c_session_live() { lossy, window, canary == vec![0xa5; 4_096], - revision, + window_revision, provider, pending, - good, - after, + window_rows, ), ( crate::RESULT_OK, crate::RESULT_RENDER_REJECTED, crate::RESULT_OK, - crate::RESULT_BACKPRESSURE, - true, - revision_before, + crate::RESULT_OK, + false, + revision_before + 1, provider_before + 1, + 1, + 3, + ), + "round {round}: control after a rejected, swapping render call" + ); + assert_eq!( + ( + good, + after, + after_revision, + after_provider, + after_pending, + window_event, + next, + ), + ( + crate::RESULT_OK, + crate::RESULT_OK, + revision_before + 1, + provider_before + 2, 0, crate::RESULT_OK, crate::RESULT_OK, ), - "round {round}: control after a rejected, swapping render call" + "round {round}: the window's plan is adopted at the next block" ); } crate::ffi::test_plan_destroy(c_plan); @@ -2836,30 +2915,41 @@ fn exported_c_replay_revision_event_and_publication_pressure_statuses_are_exact( assert_eq!(commit_event.0, crate::RESULT_OK); assert_eq!(commit_event.1, pinned_hex(COMMIT_EVENT)); + // #1310: a second structural edit while the first candidate waits unadopted supersedes it + // (it was publication backpressure before): it commits at once, the first candidate is + // withdrawn and dropped, and its replacement is the one pending plan. let second_edit = rebuild_edit(SESSION, 0x02); - let publication_full = command_bytes_at_revision( + let superseding = command_bytes_at_revision( 24, ExpectedRevision::Exact(SessionRevision(43)), protocol::CommandPayload::SessionTransactionApply(core::slice::from_ref(&second_edit)), ); let before = crate::ffi::test_session_state_summary(c_session); let before_owners = crate::ffi::test_owner_counters(c_session); - let (result, canary) = command_c(c_session, &publication_full); - assert_eq!(result, crate::RESULT_BACKPRESSURE); - assert_eq!(canary, vec![0xa5; 4_096]); - assert_eq!(crate::ffi::test_session_state_summary(c_session), before); - let mut canceled = before_owners; - canceled.token_constructed += 1; - canceled.token_disposed += 1; - canceled.replay_candidate_constructed += 1; - canceled.replay_candidate_disposed += 1; - canceled.candidate_provider_constructed += 1; - canceled.candidate_provider_disposed += 1; - canceled.candidate_plan_constructed += 1; - canceled.candidate_plan_disposed += 1; - assert_eq!(crate::ffi::test_owner_counters(c_session), canceled); assert_eq!(before.0, 43); assert_eq!(before.3, 1); + let committed = dispatch!(superseding.clone(), StatusCode::Ok, pinned_hex(RETRY_24)); + assert!(!committed.is_empty()); + let mut owners = before_owners; + owners.token_constructed += 1; + owners.token_disposed += 1; + owners.replay_candidate_constructed += 1; + owners.replay_candidate_published += 1; + owners.replay_current_disposed += 1; + owners.candidate_provider_constructed += 1; + owners.candidate_provider_published += 1; + owners.candidate_plan_constructed += 1; + owners.candidate_plan_published += 1; + owners.reservation_constructed += 1; + owners.reservation_committed += 1; + owners.candidate_withdrawn += 1; + owners.candidate_superseded += 1; + // The superseded candidate's plan and provider drop on the control thread. + owners.candidate_plan_disposed += 1; + owners.candidate_provider_disposed += 1; + assert_eq!(crate::ffi::test_owner_counters(c_session), owners); + let superseded = crate::ffi::test_session_state_summary(c_session); + assert_eq!((superseded.0, superseded.2, superseded.3), (44, 0, 1)); let mut c_pcm = [f32::NAN; 256]; let output = crate::PlanarOutput { @@ -2876,26 +2966,16 @@ fn exported_c_replay_revision_event_and_publication_pressure_statuses_are_exact( crate::RESULT_OK ); assert!(c_pcm.iter().all(|sample| sample.to_bits() == 0)); - let retry = dispatch!(publication_full, StatusCode::Ok, pinned_hex(RETRY_24)); - assert!(!retry.is_empty()); - canceled.token_constructed += 1; - canceled.token_disposed += 1; - canceled.replay_candidate_constructed += 1; - canceled.replay_candidate_published += 1; - canceled.replay_current_disposed += 1; - canceled.candidate_provider_constructed += 1; - canceled.candidate_provider_published += 1; - canceled.candidate_plan_constructed += 1; - canceled.candidate_plan_published += 1; - canceled.reservation_constructed += 1; - canceled.reservation_committed += 1; - canceled.current_provider_disposed += 1; - canceled.current_plan_disposed += 1; - assert_eq!(crate::ffi::test_owner_counters(c_session), canceled); - let after_retry = crate::ffi::test_session_state_summary(c_session); - assert_eq!(after_retry.0, 44); - assert_eq!(after_retry.2, 1); - assert_eq!(after_retry.3, 1); + // The same request again is its replay, and the replacement was adopted: the running plan's + // provider and plan retire. + let replay = dispatch!(superseding, StatusCode::Ok, pinned_hex(RETRY_24)); + assert_eq!(replay, committed); + owners.current_provider_disposed += 1; + owners.current_plan_disposed += 1; + assert_eq!(crate::ffi::test_owner_counters(c_session), owners); + let after_adoption = crate::ffi::test_session_state_summary(c_session); + assert_eq!(after_adoption.0, 44); + assert_eq!(after_adoption.3, 0); crate::ffi::test_plan_destroy(c_plan); crate::ffi::test_session_destroy(c_session); diff --git a/crates/capi/tests/plan_swap_race.rs b/crates/capi/tests/plan_swap_race.rs index 9ea019e8a..9e915d315 100644 --- a/crates/capi/tests/plan_swap_race.rs +++ b/crates/capi/tests/plan_swap_race.rs @@ -49,8 +49,9 @@ fn one_race_at_a_time() -> MutexGuard<'static, ()> { RACES.lock().unwrap_or_else(PoisonError::into_inner) } -/// Render blocks after an admission beyond which `RESULT_BACKPRESSURE` is a wedge. A healthy -/// replacement is admitted again once its swapping block has rendered. +/// Render blocks an admission may wait for before the race counts it as wedged. Since #1310 a +/// structural edit is never refused as backpressure for a pending candidate, so this only bounds +/// the blocks the race renders. const WEDGE_BLOCKS: u64 = 256; fn limits() -> CompileLimits { @@ -296,6 +297,14 @@ struct Control<'a> { admission_block: u64, next_frame: u64, fed_blocks: u64, + /// The frames fed so far, for a render thread that does not lock the control state. + fed_frames: &'a AtomicU64, + /// Offer the next structural transaction at once, not two blocks after the last admission + /// (#1310 gate 5): an edit then lands while the last candidate may still be unadopted. + back_to_back: bool, + /// The control plane's withdrawal outcomes on this, the control, thread (#1310 D1): + /// `(withdrawn, taken, superseded)`. + outcomes: (u64, u64, u64), } impl Control<'_> { @@ -318,6 +327,7 @@ impl Control<'_> { RESULT_OK => { self.next_frame += accepted; self.fed_blocks += 1; + self.fed_frames.store(self.next_frame, Ordering::Release); Ok(()) } RESULT_BACKPRESSURE => Ok(()), @@ -336,11 +346,26 @@ impl Control<'_> { /// resource query against the render thread, and, once two blocks have rendered since the /// last admission, offer the next structural transaction. fn step(&mut self) { + // #1310 gate 5: back to back, every other step feeds only after its submit, so no block + // can start while the command runs and the candidate before it stays unadopted + // (`Withdrawn`); the others feed first, so render may adopt it during the command + // (`Taken`). + let starve = self.back_to_back && self.admitted % 2 == 1; + self.step_feeding(!starve); + if starve && !self.is_done() { + let admitted = self.admitted; + if let Err(refusal) = self.feed() { + self.fail(format!("after {admitted} swaps: {refusal}")); + } + } + } + + fn step_feeding(&mut self, feed: bool) { if self.is_done() { return; } let admitted = self.admitted; - if let Err(refusal) = self.feed() { + if feed && let Err(refusal) = self.feed() { return self.fail(format!("after {admitted} swaps: {refusal}")); } let session = self.session_address as *mut Session; @@ -362,7 +387,7 @@ impl Control<'_> { } // The block in flight at admission may predate the publication; the next one swaps. let blocks = self.rendered.load(Ordering::Acquire); - if blocks < self.admission_block + 2 { + if !self.back_to_back && blocks < self.admission_block + 2 { return; } let mut output = BytesOut { @@ -413,11 +438,18 @@ impl Control<'_> { self.admitted.is_multiple_of(2), ); self.admission_block = self.rendered.load(Ordering::Acquire); + let counters = control_plane::test_lifecycle_counters(); + self.outcomes = ( + counters.candidate_withdrawn, + counters.candidate_taken, + counters.candidate_superseded, + ); if self.is_done() { self.done.store(true, Ordering::Release); } } - RESULT_BACKPRESSURE if blocks - self.admission_block <= WEDGE_BLOCKS => {} + // #1310 gate 5: a structural edit behind a pending candidate supersedes it, so any + // `RESULT_BACKPRESSURE` here fails. other => self.fail(format!( "submit after {admitted} swaps and {} blocks: {other}", blocks - self.admission_block @@ -454,7 +486,7 @@ impl Control<'_> { /// block waits until its PCM is queued. A submit may only be accepted or backpressured by a full /// ring: `source.ring.vacated` (a submit routed to an epoch whose producer has moved) or /// `source.frame.noncontiguous` (a ring that lost its write position) fails. -fn race_plan_swaps(swaps: u64, readers: usize) { +fn race_plan_swaps(swaps: u64, readers: usize, back_to_back: bool) { let _race = one_race_at_a_time(); assert_installed(); // Warm the JSON frontend's lazily boxed process statics before anything is observed. @@ -463,6 +495,21 @@ fn race_plan_swaps(swaps: u64, readers: usize) { let mut model = session::parse_session_json(SESSION).expect("fixture"); // A region the race never reaches the end of. model.sources[0].frames = 1 << 40; + if back_to_back { + // #1310 gate 5 compares the settled output with a fresh plan of the final model, so the + // session holds no DSP state a swap could carry: no console slot, no insert, and the + // input filters off. + model.console.pre_insert.clear(); + model.console.post_insert.clear(); + for track in &mut model.tracks { + track.console.clear(); + track.inserts.effects.clear(); + for lane in [&mut track.builtins.left, &mut track.builtins.right] { + lane.hpf_hz = 0.0; + lane.lpf_hz = 0.0; + } + } + } let document = session::canonical_session_json(&model).expect("canonical"); // SAFETY: The returned handles are uniquely owned until the matching destroy calls below. let (session, plan) = unsafe { compile_c(&document, &limits()) }; @@ -471,6 +518,7 @@ fn race_plan_swaps(swaps: u64, readers: usize) { let in_render = AtomicBool::new(false); let done = AtomicBool::new(false); let rendered = AtomicU64::new(0); + let fed_frames = AtomicU64::new(0); let request = track_command(1, 42, &model, true); let mut control = Control { session_address: session as usize, @@ -492,6 +540,9 @@ fn race_plan_swaps(swaps: u64, readers: usize) { admission_block: 0, next_frame: 0, fed_blocks: 0, + fed_frames: &fed_frames, + back_to_back, + outcomes: (0, 0, 0), }; let mut pcm = [f32::NAN; 256]; @@ -526,34 +577,69 @@ fn race_plan_swaps(swaps: u64, readers: usize) { // Stops the reader threads however the render loop leaves their scope, so a failed // assertion cannot hang the join. (`render_while_producing` stops its own producer thread.) let stop_readers = StopOnDrop(&stop); - render_while_producing( - block_bound, - &mut control, - |control| control.step(), - |control| control.ready(), - |_| { + let render_next = || { + if done.load(Ordering::Acquire) { + return; + } + let plan = plan_address as *mut Plan; + let block = rendered.load(Ordering::Acquire); + let mark = current_thread_counters(); + in_render.store(true, Ordering::SeqCst); + // SAFETY: This thread is the plan's only renderer; `output` is owned storage. + let code = unsafe { + miso_engine_v1_render_f32_planar(plan, block * u64::from(QUANTUM), &output) + }; + in_render.store(false, Ordering::SeqCst); + let delta = current_thread_delta_since(mark); + assert_eq!(code, RESULT_OK, "render refused block {block}"); + assert_eq!( + (delta.allocations, delta.reallocations, delta.deallocations), + (0, 0, 0), + "block {block}: the render thread allocated or freed across a plan swap" + ); + rendered.store(block + 1, Ordering::Release); + }; + if back_to_back { + // #1310 gate 5: the control thread runs unlocked, so a block renders whenever its PCM + // is queued, concurrently with a command: render may adopt a candidate before the + // command's service step (an ordinary rebuild), between it and the withdrawal + // (`Taken`), or not before it (`Withdrawn`). + let control = &mut control; + let stop = &stop; + let producer = scope.spawn(move || { + while !stop.load(Ordering::Acquire) && !control.is_done() { + control.step(); + std::thread::yield_now(); + } + }); + let deadline = std::time::Instant::now() + std::time::Duration::from_secs(120); + for _ in 0..block_bound { if done.load(Ordering::Acquire) { - return; + break; } - let plan = plan_address as *mut Plan; let block = rendered.load(Ordering::Acquire); - let mark = current_thread_counters(); - in_render.store(true, Ordering::SeqCst); - // SAFETY: This thread is the plan's only renderer; `output` is owned storage. - let code = unsafe { - miso_engine_v1_render_f32_planar(plan, block * u64::from(QUANTUM), &output) - }; - in_render.store(false, Ordering::SeqCst); - let delta = current_thread_delta_since(mark); - assert_eq!(code, RESULT_OK, "render refused block {block}"); - assert_eq!( - (delta.allocations, delta.reallocations, delta.deallocations), - (0, 0, 0), - "block {block}: the render thread allocated or freed across a plan swap" - ); - rendered.store(block + 1, Ordering::Release); - }, - ); + while fed_frames.load(Ordering::Acquire) < (block + 1) * u64::from(QUANTUM) + && !done.load(Ordering::Acquire) + { + assert!( + std::time::Instant::now() < deadline, + "block {block}: no PCM was queued" + ); + std::thread::yield_now(); + } + render_next(); + } + stop.store(true, Ordering::Release); + producer.join().expect("control thread"); + } else { + render_while_producing( + block_bound, + &mut control, + |control| control.step(), + |control| control.ready(), + |_| render_next(), + ); + } drop(stop_readers); pollers .into_iter() @@ -570,6 +656,68 @@ fn race_plan_swaps(swaps: u64, readers: usize) { plan_carry_counts(plan), ) }; + // #1310 gate 5: after quiescence the raced plan renders what a fresh plan of the final model + // renders. The race toggles a muted track, so an even count of admissions ends on the boot + // model; the feed is constant, so once every ramp has settled the block is the twin's. + let settled = back_to_back.then(|| { + let ramps = control.model.control_smoothing_samples(); + let settle = u64::from(ramps.fader.max(ramps.mute)).div_ceil(u64::from(QUANTUM)) + 2; + let mut raced = Vec::new(); + let mut next_frame = control.next_frame; + for block in blocks + 1..=blocks + settle { + // SAFETY: The race has joined, so this thread is the session's and plan's only caller. + unsafe { + while next_frame < (block + 1) * u64::from(QUANTUM) { + let (code, accepted) = + submit_dc_block::<{ QUANTUM as usize }>(session, next_frame); + if code != RESULT_OK { + break; + } + next_frame += accepted; + } + assert_eq!( + miso_engine_v1_render_f32_planar(plan, block * u64::from(QUANTUM), &output), + RESULT_OK + ); + } + raced = pcm + .iter() + .map(|sample| sample.to_bits()) + .collect::>(); + } + assert!(swaps.is_multiple_of(2), "the race ends on the boot model"); + // SAFETY: These handles are uniquely owned until the matching destroy calls below. + let twin = unsafe { + let (twin_session, twin_plan) = compile_c(&document, &limits()); + let mut twin_pcm = [f32::NAN; 256]; + let twin_output = PlanarOutput { + samples: twin_pcm.as_mut_ptr(), + ..output + }; + for block in 0..=settle { + let (code, _) = submit_dc_block::<{ QUANTUM as usize }>( + twin_session, + block * u64::from(QUANTUM), + ); + assert_eq!(code, RESULT_OK); + assert_eq!( + miso_engine_v1_render_f32_planar( + twin_plan, + block * u64::from(QUANTUM), + &twin_output + ), + RESULT_OK + ); + } + miso_engine_v1_session_destroy(twin_session); + miso_engine_v1_plan_destroy(twin_plan); + twin_pcm + .iter() + .map(|sample| sample.to_bits()) + .collect::>() + }; + (raced, twin) + }); // SAFETY: These are the exact live handles returned by `compile_c` and are destroyed once. unsafe { miso_engine_v1_session_destroy(session); @@ -591,13 +739,39 @@ fn race_plan_swaps(swaps: u64, readers: usize) { control.admitted, swaps, "every swap admitted within the block bound" ); + let (withdrawn, taken, superseded) = control.outcomes; assert_eq!( carry_counts, - (control.admitted, 0), - "every applied swap carried the source's ring" + (control.admitted - superseded, 0), + "every applied swap carried the source's ring; a superseded candidate never swapped" ); + assert_eq!( + withdrawn, superseded, + "every withdrawn candidate was superseded" + ); + if let Some((raced, twin)) = settled { + assert!( + raced == twin, + "the settled raced plan renders as a fresh plan of the final model: {:?} against {:?} \ + ({withdrawn} withdrawn, {taken} taken, carry {carry_counts:?}, blocks {blocks})", + [f32::from_bits(raced[0]), f32::from_bits(raced[128])], + [f32::from_bits(twin[0]), f32::from_bits(twin[128])], + ); + println!("{swaps} back-to-back edits: {withdrawn} withdrawn, {taken} taken"); + assert!( + withdrawn > 0 && taken > 0, + "vacuous: {withdrawn} withdrawn and {taken} taken over {swaps} back-to-back edits" + ); + } else { + assert_eq!( + superseded, 0, + "two blocks between admissions adopt every candidate" + ); + } + // Back to back, one block is fed per admission (the last admission's step ends the race). + let fed_floor = if back_to_back { swaps - 1 } else { swaps + 1 }; assert!( - control.fed_blocks > swaps, + control.fed_blocks >= fed_floor, "vacuous: {} source blocks fed across {swaps} swaps", control.fed_blocks ); @@ -605,7 +779,11 @@ fn race_plan_swaps(swaps: u64, readers: usize) { control.overlapped > 0, "vacuous: no control call overlapped a render call" ); - assert!(blocks > swaps, "each swap rendered its own block"); + // Back to back, a superseded candidate never renders a block of its own. + assert!( + blocks > swaps - superseded, + "each adopted swap rendered its own block" + ); } /// Issue #1042, F1: on the unmodified base, a control call that fell between the swap at the start @@ -613,7 +791,7 @@ fn race_plan_swaps(swaps: u64, readers: usize) { /// every structural command after it returned `RESULT_BACKPRESSURE` for good. #[test] fn control_calls_racing_plan_swapping_renders_never_wedge_replacement() { - race_plan_swaps(24, 0); + race_plan_swaps(24, 0, false); } /// Issue #1042, attempt 2: `miso_engine_v1_plan_resources` may run on any thread, concurrently with @@ -623,7 +801,7 @@ fn control_calls_racing_plan_swapping_renders_never_wedge_replacement() { /// (and a release build, with `panic = "abort"`, aborts). #[test] fn resource_queries_racing_plan_swaps_always_find_the_published_row() { - race_plan_swaps(200, 2); + race_plan_swaps(200, 2, false); } /// One applied-revision watermark read through the any-thread entry, or `None` if it was busy. @@ -1192,3 +1370,18 @@ fn a_two_strip_transaction_lands_in_one_block_and_the_watermark_names_it() { commit's duration of their render call's start, fewer than {THIN_FLOOR}" ); } + +/// #1310 gate 5: a control thread submitting 64 structural edits back to back while a render +/// thread renders. Each edit lands while the last candidate is pending or just adopted: every +/// submit commits (`RESULT_OK`, never `RESULT_BACKPRESSURE`), the run withdraws and supersedes some +/// candidates and finds others taken by render first, and after quiescence the plan renders what a +/// fresh plan of the final model renders. +/// +/// Test value: red if a structural edit behind a pending candidate is refused, if the withdrawal +/// races render's claim into a lost or doubly owned plan (a failed render, a carry count that +/// misses a swap, or output other than the twin's), or if the run never reaches one of the two +/// outcomes it is judged by. +#[test] +fn structural_edits_back_to_back_supersede_or_swap_and_never_backpressure() { + race_plan_swaps(64, 0, true); +} diff --git a/crates/capi/tests/resource_lifecycle.rs b/crates/capi/tests/resource_lifecycle.rs index 41e5e4136..abd49d748 100644 --- a/crates/capi/tests/resource_lifecycle.rs +++ b/crates/capi/tests/resource_lifecycle.rs @@ -311,7 +311,7 @@ unsafe fn submit(session: *mut Session, request: &[u8], response: &mut [u8; 4_09 } } -fn lifecycle(plan_first: bool) -> (Snapshot, Snapshot, Snapshot, Snapshot) { +fn lifecycle(plan_first: bool) -> (Snapshot, Snapshot, Snapshot) { let capability = capability_command(); let first = command(2, 42, 0x01); let second = command(3, 43, 0x02); @@ -361,9 +361,8 @@ fn lifecycle(plan_first: bool) -> (Snapshot, Snapshot, Snapshot, Snapshot) { let cached = submit(session, &capability, &mut response); let cached_delta = snapshot().delta(before_cached); let first_structural = submit(session, &first, &mut response); - let before_full = snapshot(); - let full = submit(session, &second, &mut response); - let full_delta = snapshot().delta(before_full); + // #1310: the second structural edit supersedes the first, which waits unadopted. + let superseding = submit(session, &second, &mut response); let before_render = snapshot(); let render = miso_engine_v1_render_f32_planar(plan, 0, &output); let render_delta = snapshot().delta(before_render); @@ -384,13 +383,12 @@ fn lifecycle(plan_first: bool) -> (Snapshot, Snapshot, Snapshot, Snapshot) { immediate, cached, first_structural, - full, + superseding, render, retry, second_render, ], cached_delta, - full_delta, render_delta, ) }; @@ -398,18 +396,11 @@ fn lifecycle(plan_first: bool) -> (Snapshot, Snapshot, Snapshot, Snapshot) { assert_eq!( codes.0, [ - RESULT_OK, - RESULT_OK, - RESULT_OK, - RESULT_OK, - RESULT_OK, - RESULT_BACKPRESSURE, - RESULT_OK, - RESULT_OK, + RESULT_OK, RESULT_OK, RESULT_OK, RESULT_OK, RESULT_OK, RESULT_OK, RESULT_OK, RESULT_OK, RESULT_OK, ] ); - (total, codes.1, codes.2, codes.3) + (total, codes.1, codes.2) } fn rejected_compile_lifecycle() -> Snapshot { @@ -465,11 +456,8 @@ fn exported_c_candidates_replay_render_and_both_destroy_orders_balance_exactly() .0 .assert_balanced("plan-first complete lifecycle"); plan_first.1.assert_balanced("cached replay"); - plan_first - .2 - .assert_balanced("publication-full canceled candidate"); assert_eq!( - plan_first.3, + plan_first.2, Snapshot { allocations: 0, deallocations: 0, @@ -483,10 +471,7 @@ fn exported_c_candidates_replay_render_and_both_destroy_orders_balance_exactly() .0 .assert_balanced("session-first complete lifecycle"); session_first.1.assert_balanced("cached replay"); - session_first - .2 - .assert_balanced("publication-full canceled candidate"); - assert_eq!(session_first.3, plan_first.3); + assert_eq!(session_first.2, plan_first.2); assert_eq!(session_first.0, plan_first.0, "destroy-order ownership"); } @@ -812,8 +797,10 @@ fn observe_compile(document: &str, compile_limits: &CompileLimits) -> CompileObs sources, report: host_report, inventory, + carried_base, control_catalog, } = prepared; + assert!(carried_base.is_none(), "a boot plan has no carried base"); let owners = HostOwners { routes: freed_by_drop(routes), effects: freed_by_drop(effects), @@ -1234,6 +1221,18 @@ impl Budget { /// up to 64: 17,275.5 -> 17,280 (was 15,680). The four-lane ceiling (21,056) was not re-measured /// (no AArch64 run in this fold); `builtin_processor_payload_bytes` moved 36,171 -> 36,387 inside /// its ceiling. +/// +/// The four-lane row, measured on the AArch64 CI job after #1630 (PR #1693; GitHub Actions job +/// 116713280504, which printed `builtin_bank_bytes: 12969 of 21056`): `builtin_bank_bytes` 12,969 +/// (four lanes), so its ceiling is 12,969 plus 10 %, rounded up to 64: 14,265.9 -> 14,272 (was +/// 21,056, set before the four-lane value was measured; tightened in batch 15a). +/// +/// #1310 batch 15a fold 2: the parked-provider list holds one epoch per retirement credit (three, +/// was two), and each provider epoch keeps the resource row the admissions charge while it is +/// parked (`ProviderEpoch::parked`, 248 bytes). `capi_retained_bytes` measured 165,922 before and +/// 168,810 after (x86-64, +2,888: one more 1,648-byte provider-epoch slot and 248 bytes in each of +/// the five slots; the row does not depend on the lane width), within the 175,296 ceiling, so the +/// ceiling stays. const REFERENCE_BUDGETS: [Budget; 19] = [ Budget { row: "graph_session_plus_plan_bytes", @@ -1281,7 +1280,7 @@ const REFERENCE_BUDGETS: [Budget; 19] = [ row: "builtin_bank_bytes", value: |report| report.builtin_bank_bytes, eight_lanes: 17_280, - four_lanes: 21_056, + four_lanes: 14_272, }, Budget { row: "builtin_bank_scratch_bytes", @@ -1765,43 +1764,6 @@ fn double_live_oracle_drives_exact_and_one_below_c_caps() { (current, prospective, prospective_document) }; assert_ne!(prospective_document, session_document); - // The swap is also proved by admission: with one publication slot, a second replacement is - // admitted only once the render has swapped the first one in. - // SAFETY: These handles are uniquely owned until their matching destroy calls. - unsafe { - let (session, plan) = compile_c(&session_document, &limits()); - let request = command(1, 42, "a-muted"); - let mut response = [0xa5_u8; 4_096]; - assert_eq!(submit(session, &request, &mut response), RESULT_OK); - let again = muted_track_command(&prospective_document, 2, 43, "a-muted-2"); - assert_eq!( - submit(session, &again, &mut response), - RESULT_BACKPRESSURE, - "a second replacement waits for the render" - ); - let mut pcm = [f32::NAN; 256]; - let output = PlanarOutput { - struct_size: PLANAR_OUTPUT_SIZE, - channels: 2, - samples: pcm.as_mut_ptr(), - sample_capacity: pcm.len() as u64, - frames: 128, - plane_stride_samples: 128, - reserved: [0; 2], - }; - assert_eq!( - miso_engine_v1_render_f32_planar(plan, 0, &output), - RESULT_OK - ); - assert_eq!( - submit(session, &again, &mut response), - RESULT_OK, - "the render swapped the prospective plan in" - ); - miso_engine_v1_session_destroy(session); - miso_engine_v1_plan_destroy(plan); - } - let current_host = host_half(&session_document, &limits()); let prospective_host = host_half(&prospective_document, &limits()); let current_model = current_host.model_estimate(); @@ -1864,9 +1826,12 @@ fn double_live_oracle_drives_exact_and_one_below_c_caps() { // strips the swap restarts (#1277 D6, #1279 D1). This session's console slots have live // effect lanes, which the effect carry does not move (#1280), so every strip restarts and the // inventory is that much larger than `capi_epoch_terms` read off the fresh preparation. + // It also keeps the base a replacement would be prepared from were it withdrawn unadopted + // (#1310 D2, #1344 D1), which a fresh preparation has none of. let prospective_epoch = prospective_epoch - prospective_host.prepared.report.inventory_retained_bytes - + successor.inventory_retained_bytes; + + successor.inventory_retained_bytes + + successor.carried_base_retained_bytes; let rows: [(&str, u64); 8] = [ ( "graph", @@ -2098,7 +2063,8 @@ unsafe fn render_and_destroy(session: *mut Session, plan: *mut Plan) -> Vec { /// Halfway through, the render thread parks (as a paused audio session does) while this thread /// sends forty live fader edits to one track, none of which may be refused (a fader value is a /// latest-target cell, #1312), and submits a second structural edit behind a pending candidate, -/// which is typed plan backpressure; it is retried once the render thread resumes. Without the -/// pause the render thread outruns this thread, and no plan retry would ever run. +/// which supersedes it (#1310). Without the pause the render thread outruns this thread, and no +/// supersession would ever run. /// /// The render thread never waits for PCM, so the source falls behind the render clock by however /// many blocks the render thread outran this thread's feed. The source stays on the render clock: @@ -3110,6 +3083,7 @@ impl RaceControl<'_> { /// its own global allocator, and this file's counting allocator is the one the counts come from. fn race_live_edits(run: usize) -> RaceCounts { use core::sync::atomic::Ordering; + let superseded_before = control_plane::test_lifecycle_counters().candidate_superseded; let document = race_session(); // SAFETY: The returned handles are uniquely owned until the matching destroy calls below. @@ -3268,6 +3242,8 @@ fn race_live_edits(run: usize) -> RaceCounts { let final_block = fed_render_c(session, plan, &mut feed, block); let (committed_revision, final_document) = snapshot_c(session, &mut request_id); counts.replacements = plan_replacement_count(plan); + counts.superseded = + control_plane::test_lifecycle_counters().candidate_superseded - superseded_before; miso_engine_v1_session_destroy(session); miso_engine_v1_plan_destroy(plan); (committed_revision, final_document, latency, final_block) @@ -3327,9 +3303,8 @@ const PAUSED_LIVE_EDITS: u32 = 40; /// The paused phase of [`race_live_edits`]: with the render thread parked, [`PAUSED_LIVE_EDITS`] /// live edits on the first track, none of which may be refused for room (a fader value is a -/// latest-target cell), then a structural edit and a second one behind it, which -/// `control.plan.backpressure` refuses. The render thread resumes and the second structural edit -/// is retried through [`RaceControl::commit`]. +/// latest-target cell), then a structural edit and a second one behind it, which supersedes the +/// first while it waits unadopted (#1310): both commit at once, and the render thread resumes. fn paused_bursts( control: &mut RaceControl<'_>, edits: &mut RaceEdits, @@ -3374,14 +3349,12 @@ fn paused_bursts( } if !matches!( control.submit_once(&second, Kind::Structural)?, - Submitted::PlanBackpressure + Submitted::Committed ) { - return Err("a structural edit behind a pending candidate was not refused".to_owned()); + return Err("a structural edit behind a pending candidate did not supersede it".to_owned()); } paused.store(false, Ordering::Release); - // The second structural edit waits for the swap. `retries` counts only the resubmissions - // `commit` finds refused again. - control.commit(&second, Kind::Structural) + Ok(()) } /// #1258 gate 1: live fader, mute and pan edits, several per render period, and a structural edit @@ -3395,8 +3368,8 @@ fn paused_bursts( /// Test value: red if a fader or matrix drain allocates or frees on the render thread, if a live /// edit that races a plan swap reaches the retiring plan or is lost (the final block differs from /// a fresh plan of the final snapshot), if epoch synchronization returns `RESULT_INTERNAL` under -/// live traffic, if a `RESULT_BACKPRESSURE` carries any other diagnostic or never clears, or if a -/// live edit rebuilds the plan or a structural one swaps in other than exactly one plan. +/// live traffic, if any structural edit is refused as plan backpressure (#1310), or if a live edit +/// rebuilds the plan or a structural one neither swaps in exactly one plan nor is superseded. #[test] fn live_edits_racing_a_rendering_plan_and_its_swaps_stay_exact_and_allocation_free() { let mut total = RaceCounts::default(); @@ -3409,12 +3382,18 @@ fn live_edits_racing_a_rendering_plan_and_its_swaps_stay_exact_and_allocation_fr "run {run}" ); assert!(counts.monitor >= RACE_EDITS / 8, "run {run}"); - assert!(counts.plan_backpressure >= 1, "run {run}"); - // Every admitted structural transaction swapped in exactly one plan, and no live edit - // swapped in one of its own: a live edit that silently rebuilt is counted here. + // #1310: a structural edit behind a pending candidate supersedes it; nothing refuses it. + assert_eq!(counts.plan_backpressure, 0, "run {run}"); + assert!( + counts.superseded >= 1, + "run {run}: the paused phase supersedes" + ); + // Every admitted structural transaction published exactly one plan epoch (adopted, or + // superseded unadopted, #1310), and no live edit published one of its own: a live edit + // that silently rebuilt is counted here. assert_eq!( counts.replacements, counts.structural, - "run {run}: plan replacements against admitted structural transactions" + "run {run}: plan epochs against admitted structural transactions" ); total.live += counts.live; total.monitor += counts.monitor; @@ -3423,6 +3402,7 @@ fn live_edits_racing_a_rendering_plan_and_its_swaps_stay_exact_and_allocation_fr total.event_backpressure += counts.event_backpressure; total.retries += counts.retries; total.replacements += counts.replacements; + total.superseded += counts.superseded; total.overlapped += counts.overlapped; total.blocks += counts.blocks; } diff --git a/crates/control-plane/src/adapter.rs b/crates/control-plane/src/adapter.rs index 6f8bfe3a3..552911d2a 100644 --- a/crates/control-plane/src/adapter.rs +++ b/crates/control-plane/src/adapter.rs @@ -43,7 +43,8 @@ pub struct ControlLimits { pub maximum_routes: u64, /// Effect instances of one session. pub maximum_effects: u64, - /// Graph, session and plan bytes, both plans and both compiled models included at a swap. + /// Graph, session and plan bytes, every held plan and compiled model included at a swap + /// (#1398 D4). pub maximum_graph_session_plus_plan_bytes: u64, /// Source ring bytes, payload and overhead. pub maximum_source_total_bytes: u64, @@ -78,9 +79,10 @@ pub struct ControlLimits { /// (Strip, VCA) reach pairs of one session (ruling V-Q3): the bound on one VCA move's work. /// Every adapter sets [`host_core::MAXIMUM_VCA_REACH_PAIRS`]. pub maximum_vca_reach_pairs: u64, - /// The most a replacement may hold at its peak, summed over the categories: the running plan's - /// and the replacement's graph, session and plan bytes, both compiled models, and the source - /// bytes of both plans, a carried ring counted once (#1290a D3). Each category also has its own + /// The most a replacement may hold at its peak, summed over the categories: the graph, session + /// and plan bytes of every plan the control plane holds and of the replacement, every compiled + /// model, and the source bytes of all those plans, a carried ring counted once (#1290a D3, + /// #1398 D4). Each category also has its own /// ceiling above; an adapter that owns one aggregate budget (the browser's) sets this to what /// that budget leaves for them after its own fixed bytes, and one that has none sets /// `u64::MAX`. diff --git a/crates/control-plane/src/compile.rs b/crates/control-plane/src/compile.rs index 33ee5183d..70bdbcdec 100644 --- a/crates/control-plane/src/compile.rs +++ b/crates/control-plane/src/compile.rs @@ -23,6 +23,10 @@ pub(crate) struct PreparedRuntime { pub(crate) carried: CarriedSourceBytes, /// What the plan holds, for preparing its successor (issue #1273 D1). pub(crate) inventory: PlanStateInventory, + /// What the plan carries from the plan it was prepared to displace (#1344 D1), the base its + /// replacement is prepared from should it be withdrawn before render adopts it (#1310 D2). + /// `None` for a plan prepared without a successor base. + pub(crate) carried_base: Option, pub(crate) control_catalog: ControlCatalog, pub(crate) capi: CapiResources, /// The designated master strip the plan was prepared with, as a strip index (#1683): a first @@ -70,47 +74,59 @@ pub struct CapiResources { /// /// The successor's [`PlanResources`] count them in its source rows, because it owns those /// rings once active; until the swap the running plan owns them, so the double-live admission -/// subtracts them from the successor's rows and counts each carried ring once. +/// subtracts them from the successor's rows and counts each carried ring once. A pending +/// candidate keeps its own (`LiveEpochResources::carried`), so a ring two held plans and the +/// candidate share is counted once across all three (#1398 D4, batch 15a review fold M2). +/// +/// Public only to the adapters' tests, under `test-support`; crate-private otherwise. #[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] -pub(crate) struct CarriedSourceBytes { - pub(crate) total: u64, - pub(crate) overhead: u64, +#[cfg_attr(not(feature = "test-support"), allow(unreachable_pub))] +pub struct CarriedSourceBytes { + /// The carried rings' share of `source_total_bytes`. + pub total: u64, + /// The carried rings' share of `source_overhead_bytes`. + pub overhead: u64, } -/// Both compiled models' charge while a prospective session lives beside the committed one. +/// Every compiled model's charge while a prospective session lives beside the ones the control +/// plane holds (#1398 D4): the committed model, the model an epoch keeps while a successor of it +/// is pending (#1310 D3), +/// and the prospective one. /// /// Public only to the adapters' tests, under `test-support`; crate-private otherwise. #[derive(Clone, Copy)] #[cfg_attr(not(feature = "test-support"), allow(unreachable_pub))] pub struct CompiledModelAdmission { - /// Both models' retained bytes. + /// Every model's retained bytes. pub retained_bytes: u64, - /// The larger of the two models' largest single allocation. + /// The largest of the models' largest single allocation. pub largest_allocation_bytes: u64, } -pub(crate) fn compiled_model_admission( - current: &CompiledSession, - prospective: &CompiledSession, -) -> Result { - compiled_model_admission_between(current.resource_estimate(), prospective.resource_estimate()) -} - -/// [`compiled_model_admission`] from the two models' resource estimates, for a prospective model -/// that is not held as a compilation (#1613). -pub(crate) fn compiled_model_admission_between( - current: session::ResourceEstimate, +/// [`CompiledModelAdmission`] over every model the control plane holds, `held`, and the +/// prospective one (#1398 D4): their retained bytes summed, and the largest of their largest +/// allocations. +pub(crate) fn compiled_model_admission_over( + held: &[session::ResourceEstimate], prospective: session::ResourceEstimate, ) -> Result { - Ok(CompiledModelAdmission { - retained_bytes: current - .compiled_model_bytes - .checked_add(prospective.compiled_model_bytes) - .ok_or_else(|| failure(ResourceFault::Arithmetic))?, - largest_allocation_bytes: current - .single_allocation_bytes - .max(prospective.single_allocation_bytes), - }) + held.iter().try_fold( + CompiledModelAdmission { + retained_bytes: prospective.compiled_model_bytes, + largest_allocation_bytes: prospective.single_allocation_bytes, + }, + |admission, model| { + Ok(CompiledModelAdmission { + retained_bytes: admission + .retained_bytes + .checked_add(model.compiled_model_bytes) + .ok_or_else(|| failure(ResourceFault::Arithmetic))?, + largest_allocation_bytes: admission + .largest_allocation_bytes + .max(model.single_allocation_bytes), + }) + }, + ) } /// What the committed model's control catalog adds to a live commit's peak (#1558 D6). @@ -206,6 +222,30 @@ pub(crate) fn protocol_queue_config( /// `OBSERVATION_ENDED` (#1584 D6a). const RELIABLE_EVENT_SLOTS: usize = 2; +/// The plan-resource report table's rows (#1398 D1): a lagging epoch's row (the atomic names a +/// plan render has already displaced), the running plan's, a withdrawn candidate's and its +/// successor's. +pub(crate) const REPORT_ROWS: usize = 4; + +/// The pending provider epochs (#1398 D1): a withdrawn candidate's and its successor's coexist +/// until the former is dropped (#1310 D5). +pub(crate) const PENDING_PROVIDERS: usize = 2; + +/// The provider epochs parked until render's displaced plan is reclaimed (#1398 D1): one per +/// retirement credit, because each parked epoch's plan holds one until the service step reclaims +/// it (batch 15a review fold 2, B-1). With two, a `Taken` withdrawal in each of two commands while +/// render lagged its retirements parked two epochs, and a third promotion found the list full. +pub(crate) const RETIRED_PROVIDERS: usize = RETIREMENT_CAPACITY; + +/// The plan exchange's retirement credits (#1398 D1), the exchange and its resource report alike: +/// a displaced plan not yet reclaimed, a withdrawn candidate's credit (held until the candidate +/// is dropped or republished) and its successor's reservation. +pub(crate) const RETIREMENT_CAPACITY: usize = 3; + +pub(crate) fn retirement_capacity() -> NonZeroUsize { + NonZeroUsize::new(RETIREMENT_CAPACITY).expect("three is nonzero") +} + #[allow(clippy::too_many_arguments)] pub(crate) fn capi_resources( limits: ControlLimits, @@ -214,6 +254,7 @@ pub(crate) fn capi_resources( strip_table_bytes: u64, provider: host_core::SessionControlProviderResources, inventory_bytes: u64, + carried_base_bytes: u64, effect_controls: host_core::EffectControlResources, route_controls: host_core::RouteControlResources, observation: ObservationResources, @@ -238,7 +279,7 @@ pub(crate) fn capi_resources( let replay = ReplayCache::resource_report_for_config(replay_config) .map_err(|_| failure(ResourceFault::Arithmetic))?; let exchange = plan_exchange_resource_report(PlanExchangeConfig { - retirement_capacity: NonZeroUsize::new(1).expect("one is nonzero"), + retirement_capacity: retirement_capacity(), }) .map_err(|_| failure(ResourceFault::Arithmetic))?; // The control-source table and ID arena are the facade's own layout; capi reads the mirror @@ -282,6 +323,9 @@ pub(crate) fn capi_resources( source_id_row, // Issue #1273 D5: each epoch keeps its plan's state inventory for the successor. inventory_bytes, + // #1310 D2 (#1344 D1): and the base a successor is prepared from should this epoch's plan + // be withdrawn before render adopts it; zero for a plan prepared without a base. + carried_base_bytes, strip_table_bytes, effect_row, route_row, @@ -340,8 +384,10 @@ pub(crate) fn capi_resources( provider.fixed_retained_bytes, checked_layout::(maximum_configuration_items)?, checked_layout::(maximum_configuration_items)?, - checked_layout::(2)?, - checked_layout::<(u64, A::Row)>(2)?, + // #1398 D1: the pending and retired provider lists, and the report table. + checked_layout::(PENDING_PROVIDERS)?, + checked_layout::(RETIRED_PROVIDERS)?, + checked_layout::<(u64, A::Row)>(REPORT_ROWS)?, ]; // #1309 D5: the adapter's own per-session handles (capi's `Session` and `Plan`) follow the // report table, where capi's two rows stood. @@ -403,10 +449,12 @@ pub(crate) fn capi_resources( }) } +#[allow(clippy::too_many_arguments)] pub(crate) fn prepared_capi_resources( compiled: &CompiledSession, catalog: &ControlCatalog, inventory: &PlanStateInventory, + carried_base_bytes: u64, effect_controls: host_core::EffectControlResources, route_controls: host_core::RouteControlResources, observation: ObservationResources, @@ -446,6 +494,7 @@ pub(crate) fn prepared_capi_resources( .ok_or_else(|| failure(ResourceFault::Arithmetic))?, provider, inventory.retained_bytes(), + carried_base_bytes, effect_controls, route_controls, observation, @@ -770,88 +819,153 @@ pub(crate) fn controller_retained_capacity( }) } +/// The candidate a structural command prepared, as the replacement admission reads it. +#[derive(Clone, Copy)] +pub(crate) struct ProspectivePlan { + /// The candidate's plan row: what it owns once active, carried rings included. + pub(crate) resources: PlanResources, + /// The part of its source rows it carries from the plan it was prepared to displace (#1273 + /// D5): the newest held plan, or across a withdrawn candidate the running one (#1310 D2). + pub(crate) carried: CarriedSourceBytes, + /// The capi resources it was prepared with. + pub(crate) capi: CapiResources, +} + +/// The replacement admission (#1042, #1398 D4): may the prospective plan live beside every plan +/// the control plane holds unreclaimed until each is reclaimed? +/// +/// `held` is those plans in epoch order, the oldest first and the newest last: every parked epoch +/// whose plan render displaced but the service step has not reclaimed (batch 15a review fold 2, +/// M-1), the running plan, then every pending or withdrawn candidate. It is never empty. `compiled_models` charges +/// every model the control plane holds plus the prospective one. Each term is the sum over the +/// held plans plus the candidate, against its own cap: graph plus models, the source totals and +/// overheads (a carried ring counted once, in the oldest held plan that holds it: the candidate's +/// and every held epoch's but the oldest one's carried bytes leave their rows), the +/// replacement peak (graph, models and allocated source bytes, #1290a D3), effect state and +/// scratch, builtins, and the control plane's retained bytes (the newest plan's row, which holds +/// the session's fixed rows once, every older held epoch's own rows, and the candidate's epoch and +/// prepared-protocol rows). The largest-allocation check folds every held row. +/// +/// Nothing is committed on a refusal. pub(crate) fn validate_replacement_peak( - current: PlanResources, - prospective: PlanResources, - prospective_carried: CarriedSourceBytes, - prospective_capi: CapiResources, + held: &[LiveEpochResources], + prospective: ProspectivePlan, compiled_models: CompiledModelAdmission, limits: ControlLimits, ) -> Result<(), CompileFailure> { - let combined = |left: u64, right: u64| { - left.checked_add(right) - .ok_or_else(|| failure(ResourceFault::Arithmetic)) + let arithmetic = || failure(ResourceFault::Arithmetic); + let newest = held.last().ok_or_else(arithmetic)?; + // Each held row's `field`, summed, plus the candidate's `candidate`. + let peak = |field: fn(&PlanResources) -> u64, candidate: u64| { + held.iter() + .try_fold(0_u64, |total, epoch| { + total.checked_add(field(&epoch.report)) + }) + .and_then(|total| total.checked_add(candidate)) + .ok_or_else(arithmetic) + }; + // Issue #1273 D5: a carried ring is the plan's it was carried from until the swap, so the peak + // counts it once, there: the candidate's carried bytes leave its row, and every held epoch + // after the oldest leaves its own (#1398 D4: a ring a pending candidate carried from the + // running plan is the running plan's, and one the running plan carried from a parked epoch + // is the parked epoch's; each epoch carries only from the one before it in `held` or from + // a plan already reclaimed). + let allocated = |row: u64, carried: u64| row.checked_sub(carried).ok_or_else(arithmetic); + let source_peak = |field: fn(&PlanResources) -> u64, + carried: fn(&CarriedSourceBytes) -> u64, + candidate: u64| { + held.iter() + .enumerate() + .try_fold(0_u64, |total, (index, epoch)| { + let row = field(&epoch.report); + let own = if index == 0 { + Some(row) + } else { + row.checked_sub(carried(&epoch.carried)) + }; + own.and_then(|own| total.checked_add(own)) + }) + .and_then(|total| total.checked_add(candidate)) + .ok_or_else(arithmetic) }; - let graph_peak = combined( - current.graph_session_plus_plan_bytes, - prospective.graph_session_plus_plan_bytes, + let candidate = prospective.resources; + let graph_peak = peak( + |row| row.graph_session_plus_plan_bytes, + candidate.graph_session_plus_plan_bytes, )? .checked_add(compiled_models.retained_bytes) - .ok_or_else(|| failure(ResourceFault::Arithmetic))?; + .ok_or_else(arithmetic)?; if graph_peak > limits.maximum_graph_session_plus_plan_bytes { return Err(diagnostic("graph.resource.limit")); } - // Issue #1273 D5: a carried ring is the current plan's until the swap, so the double-live - // peak counts it once, there. - let allocated = |row: u64, carried: u64| { - row.checked_sub(carried) - .ok_or_else(|| failure(ResourceFault::Arithmetic)) - }; - if combined( - current.source_total_bytes, - allocated(prospective.source_total_bytes, prospective_carried.total)?, - )? > limits.maximum_source_total_bytes - || combined( - current.source_overhead_bytes, + let source_total = source_peak( + |row| row.source_total_bytes, + |carried| carried.total, + allocated(candidate.source_total_bytes, prospective.carried.total)?, + )?; + if source_total > limits.maximum_source_total_bytes + || source_peak( + |row| row.source_overhead_bytes, + |carried| carried.overhead, allocated( - prospective.source_overhead_bytes, - prospective_carried.overhead, + candidate.source_overhead_bytes, + prospective.carried.overhead, )?, )? > limits.maximum_source_overhead_bytes { return Err(diagnostic("source.resource.limit")); } // #1290a D3: the categories above each fit alone; their sum must fit an adapter's one budget. - if combined( - graph_peak, - combined( - current.source_total_bytes, - allocated(prospective.source_total_bytes, prospective_carried.total)?, - )?, - )? > limits.maximum_replacement_peak_bytes + if graph_peak + .checked_add(source_total) + .ok_or_else(arithmetic)? + > limits.maximum_replacement_peak_bytes { return Err(diagnostic("control.resource.replacement_peak")); } - if combined( - current.effect_scalar_state_bytes, - prospective.effect_scalar_state_bytes, + if peak( + |row| row.effect_scalar_state_bytes, + candidate.effect_scalar_state_bytes, )? > limits.maximum_effect_state_bytes - || combined( - current.effect_scalar_scratch_bytes, - prospective.effect_scalar_scratch_bytes, + || peak( + |row| row.effect_scalar_scratch_bytes, + candidate.effect_scalar_scratch_bytes, )? > limits.maximum_effect_scratch_bytes { return Err(diagnostic("effect.resource.limit")); } - if combined( - current.builtin_retained_payload_bytes, - prospective.builtin_retained_payload_bytes, + if peak( + |row| row.builtin_retained_payload_bytes, + candidate.builtin_retained_payload_bytes, )? > limits.maximum_builtin_retained_bytes { return Err(failure(ResourceFault::Limit)); } - let capi_peak = current - .control_retained_bytes - .checked_add(prospective_capi.epoch_retained) - .and_then(|value| value.checked_add(prospective_capi.prepared_protocol_retained)) - .ok_or_else(|| failure(ResourceFault::Arithmetic))?; + // The newest held row holds the session's fixed rows once and its own epoch's; every older + // held epoch adds its own rows, and the candidate its epoch and prepared-protocol rows. + let older_epochs = held[..held.len() - 1] + .iter() + .try_fold(0_u64, |total, epoch| { + total.checked_add(epoch.capi.epoch_retained) + }) + .ok_or_else(arithmetic)?; + let capi_peak = prospective + .capi + .epoch_retained + .checked_add(prospective.capi.prepared_protocol_retained) + .and_then(|candidate| candidate.checked_add(newest.report.control_retained_bytes)) + .and_then(|value| value.checked_add(older_epochs)) + .ok_or_else(arithmetic)?; if capi_peak > limits.maximum_control_retained_bytes { return Err(failure(ResourceFault::Limit)); } - if current - .largest_named_allocation_bytes - .max(prospective.largest_named_allocation_bytes) - .max(prospective_capi.largest) + let largest = held + .iter() + .map(|epoch| epoch.report.largest_named_allocation_bytes) + .fold(0, u64::max); + if largest + .max(prospective.resources.largest_named_allocation_bytes) + .max(prospective.capi.largest) .max(compiled_models.largest_allocation_bytes) > limits.maximum_named_allocation_bytes { @@ -860,8 +974,8 @@ pub(crate) fn validate_replacement_peak( Ok(()) } -/// One provider epoch's resources as the live admission reads them: its plan's report row and the -/// capi resources it was prepared with (#1257 D4). +/// One provider epoch's resources as the live and replacement admissions read them: its plan's +/// report row and the capi resources it was prepared with (#1257 D4, #1398 D4). /// /// Public only to the adapters' tests, under `test-support`; crate-private otherwise. #[derive(Clone, Copy)] @@ -871,61 +985,71 @@ pub struct LiveEpochResources { pub report: PlanResources, /// The capi resources the epoch's plan was prepared with. pub capi: CapiResources, + /// The part of the report's source rows the epoch's plan carries from the plan it was + /// prepared to displace (#1273 D5): zero for the boot plan. The replacement admission + /// subtracts it for every held epoch after the oldest, whose carried rings belong to the + /// plan before it while both are held (#1398 D4, batch 15a review fold M2). + pub carried: CarriedSourceBytes, } -/// The live arm's admission (#1053 D8, #1257 D5): may the prospective compiled model live beside -/// the current one, next to the plans that exist, until the commit? +/// The live arm's admission (#1053 D8, #1257 D5, #1398 D4): may the prospective compiled model +/// live beside the current one, next to the plans that exist, until the commit? /// -/// The live arm prepares no plan, so the plans are the current epoch's and, when a candidate is -/// pending, that candidate's; the newest of them is the pending one if any, else the current one. -/// The terms, each against its own cap: +/// The live arm prepares no plan, so the plans are those the control plane holds unreclaimed, +/// `held`, in epoch order: every parked epoch whose plan is not yet reclaimed, the running plan, +/// then every pending or withdrawn candidate; the newest of them is the last. It is never empty. The terms, each against its own cap: /// -/// - **graph:** both plans' `graph_session_plus_plan_bytes` plus both compiled models' -/// `retained_bytes`; -/// - **capi:** the newest plan's `control_retained_bytes`, plus the current epoch's `epoch_retained` -/// while a candidate is pending, plus the newest epoch's `prepared_protocol_retained`. That last -/// term includes a provider catalog (`catalog_retained_bytes`), which a live commit replaces by -/// the committed model's (#1552 D3), so the term is the same one a rebuild charges; +/// - **graph:** every held plan's `graph_session_plus_plan_bytes` plus every compiled model's +/// `retained_bytes` (the held ones and the prospective one); +/// - **capi:** the newest plan's `control_retained_bytes`, plus every older held epoch's +/// `epoch_retained`, plus the newest epoch's `prepared_protocol_retained`. That last term +/// includes a provider catalog (`catalog_retained_bytes`), which a live commit replaces by the +/// committed model's (#1552 D3), so the term is the same one a rebuild charges; /// - **catalog growth:** the new catalog's retained bytes beyond the current catalog's are added to /// the capi term, and its largest single allocation joins the largest-allocation term, so a VCA /// added live that grows the catalog meets the ceiling a rebuild would (#1558 D6); -/// - **largest allocation:** the largest of both plans' `largest_named_allocation_bytes`, the -/// newest epoch's capi `largest`, the compiled models' largest allocation and the new catalog's. +/// - **largest allocation:** the largest of every held plan's `largest_named_allocation_bytes`, +/// the newest epoch's capi `largest`, the compiled models' largest allocation and the new +/// catalog's. /// /// Public only to the adapters' tests, under `test-support`; crate-private otherwise. +/// +/// # Errors +/// +/// The cap a term exceeds, with its diagnostic; arithmetic overflow, or an empty `held`. #[cfg_attr(not(feature = "test-support"), allow(unreachable_pub))] pub fn validate_live_peak( - current: LiveEpochResources, - pending: Option, + held: &[LiveEpochResources], compiled_models: CompiledModelAdmission, catalog: LiveCatalogAdmission, limits: ControlLimits, ) -> Result<(), CompileFailure> { let arithmetic = || failure(ResourceFault::Arithmetic); - let newest = pending.unwrap_or(current); - let graph = current - .report - .graph_session_plus_plan_bytes - .checked_add(pending.map_or(0, |pending| pending.report.graph_session_plus_plan_bytes)) - .and_then(|value| value.checked_add(compiled_models.retained_bytes)) + let (newest, older) = held.split_last().ok_or_else(arithmetic)?; + let graph = held + .iter() + .try_fold(compiled_models.retained_bytes, |total, epoch| { + total.checked_add(epoch.report.graph_session_plus_plan_bytes) + }) .ok_or_else(arithmetic)?; if graph > limits.maximum_graph_session_plus_plan_bytes { return Err(diagnostic("graph.resource.limit")); } - let capi = newest - .report - .control_retained_bytes - .checked_add(pending.map_or(0, |_| current.capi.epoch_retained)) + let capi = older + .iter() + .try_fold(newest.report.control_retained_bytes, |total, epoch| { + total.checked_add(epoch.capi.epoch_retained) + }) .and_then(|value| value.checked_add(newest.capi.prepared_protocol_retained)) .and_then(|value| value.checked_add(catalog.growth_bytes)) .ok_or_else(arithmetic)?; if capi > limits.maximum_control_retained_bytes { return Err(failure(ResourceFault::Limit)); } - if current - .report - .largest_named_allocation_bytes - .max(pending.map_or(0, |pending| pending.report.largest_named_allocation_bytes)) + if held + .iter() + .map(|epoch| epoch.report.largest_named_allocation_bytes) + .fold(0, u64::max) .max(newest.capi.largest) .max(compiled_models.largest_allocation_bytes) .max(catalog.largest_allocation_bytes) @@ -1085,6 +1209,7 @@ pub(crate) fn prepare_runtime( compiled, &prepared.control_catalog, &prepared.inventory, + prepared.report.carried_base_retained_bytes, prepared.report.effect_control_resources, prepared.report.route_control_resources, observation_resources(&effects, observation_slots_high_water)?, @@ -1158,6 +1283,7 @@ pub(crate) fn prepare_runtime( }, carried, inventory: prepared.inventory, + carried_base: prepared.carried_base, control_catalog: prepared.control_catalog, capi, master_track, @@ -1230,7 +1356,6 @@ where max_string_bytes: control_bytes, max_nesting: 4, }); - let one = NonZeroUsize::new(1).expect("one is nonzero"); let queues = ProtocolQueues::prepare(protocol_queue_config(limits)?) .map_err(|_| failure(ResourceFault::ProtocolQueue))?; let replay = ReplayCache::try_new(ReplayCacheConfig { @@ -1249,6 +1374,8 @@ where resources, carried: _, inventory, + // A boot plan is prepared without a successor base, so it has none. + carried_base: _, control_catalog, capi, master_track: _, @@ -1260,13 +1387,13 @@ where &mut revision_gate, store.revision().0, PlanExchangeConfig { - retirement_capacity: one, + retirement_capacity: retirement_capacity(), }, ) .map_err(|_| failure(ResourceFault::PlanExchange))?; let mut reports = Vec::new(); reports - .try_reserve_exact(2) + .try_reserve_exact(REPORT_ROWS) .map_err(|_| failure(ResourceFault::Allocation))?; reports.push((0, A::Row::from(resources))); let shared = Arc::new(SharedPlanState { @@ -1309,11 +1436,11 @@ where .map_err(|_| failure(ResourceFault::Allocation))?; let mut pending_providers = Vec::new(); pending_providers - .try_reserve_exact(1) + .try_reserve_exact(PENDING_PROVIDERS) .map_err(|_| failure(ResourceFault::Allocation))?; let mut retired_providers = Vec::new(); retired_providers - .try_reserve_exact(1) + .try_reserve_exact(RETIRED_PROVIDERS) .map_err(|_| failure(ResourceFault::Allocation))?; let decode_field_count = control_bytes / size_of::(); let mut decode_fields = Vec::new(); @@ -1350,6 +1477,7 @@ where observation_wire: observation_wire(limits.maximum_control_frame_bytes)?, observation_armed_bytes: 0, observation_slots_high_water: 0, + observation_forwarded: Vec::new(), }; let (offsets, pending, counts, armed, armed_bytes) = crate::control::observation_slots( &session.providers.effects, @@ -1368,3 +1496,273 @@ where plan: PlanState::new(owner, shared), }) } + +#[cfg(test)] +mod admission_tests { + //! #1398 gate 2: the replacement admission sums every held plan and model. + + use super::*; + + /// The caps the replacement admission checks, each with the diagnostic it refuses with. + const CAPS: [(&str, &str); 9] = [ + ("graph", "graph.resource.limit\t$\n"), + ("source total", "source.resource.limit\t$\n"), + ("source overhead", "source.resource.limit\t$\n"), + ("replacement peak", "control.resource.replacement_peak\t$\n"), + ("effect state", "effect.resource.limit\t$\n"), + ("effect scratch", "effect.resource.limit\t$\n"), + ("builtin", "capi.resource.limit\t$\n"), + ("control retained", "capi.resource.limit\t$\n"), + ("largest allocation", "capi.resource.limit\t$\n"), + ]; + + /// One plan whose every term is a multiple of `base`, so a sum that misses or repeats a plan + /// misses or repeats a distinct number. + fn epoch(base: u64) -> LiveEpochResources { + LiveEpochResources { + report: PlanResources { + sample_rate_hz: 48_000, + quantum_frames: 128, + source_count: 1, + track_count: 1, + latency_samples: 0, + output_tail: TailSamples::Finite(0), + graph_session_plus_plan_bytes: base, + graph_incremental_plan_bytes: 0, + graph_metadata_bytes: 0, + graph_delay_bytes: 0, + effect_bank_scratch_bytes: 0, + effect_bank_runtime_buffer_bytes: 0, + effect_bank_metadata_bytes: 0, + builtin_bank_bytes: 0, + builtin_bank_scratch_bytes: 0, + source_pcm_payload_bytes: base, + source_overhead_bytes: 2 * base, + source_total_bytes: 3 * base, + effect_scalar_state_bytes: 4 * base, + effect_scalar_scratch_bytes: 5 * base, + builtin_processor_payload_bytes: 6 * base, + builtin_meter_payload_bytes: 0, + builtin_retained_payload_bytes: 6 * base, + control_retained_bytes: 7 * base, + largest_named_allocation_bytes: base, + }, + capi: CapiResources { + active_retained: 7 * base, + epoch_retained: 8 * base, + prepared_protocol_retained: 9 * base, + largest: base, + }, + carried: CarriedSourceBytes::default(), + } + } + + /// [`epoch`] carrying `total` source bytes, `overhead` of them overhead, from the plan before. + fn carrying(base: u64, total: u64, overhead: u64) -> LiveEpochResources { + LiveEpochResources { + carried: CarriedSourceBytes { total, overhead }, + ..epoch(base) + } + } + + fn model(retained: u64, largest: u64) -> session::ResourceEstimate { + session::ResourceEstimate { + entity_count: 0, + source_count: 0, + track_count: 0, + effect_count: 0, + route_count: 0, + automation_segment_count: 0, + retained_string_bytes: 0, + parameter_count: 0, + queue_items: 0, + queue_bytes: 0, + source_ring_frames: 0, + source_ring_bytes: 0, + canonical_upper_bound_bytes: 0, + canonical_bytes: 0, + compiled_model_bytes: retained, + requested_runtime_bytes: 0, + single_allocation_bytes: largest, + platform_allocation_limit_bytes: 0, + estimated_bytes: 0, + } + } + + fn unlimited() -> ControlLimits { + ControlLimits { + source_ring_frames: 0, + maximum_automation_spans_per_block: 0, + maximum_document_bytes: 0, + maximum_diagnostic_bytes: 0, + maximum_tracks: 0, + maximum_sources: 0, + maximum_routes: 0, + maximum_effects: 0, + maximum_graph_session_plus_plan_bytes: u64::MAX, + maximum_source_total_bytes: u64::MAX, + maximum_source_overhead_bytes: u64::MAX, + maximum_effect_state_bytes: u64::MAX, + maximum_effect_scratch_bytes: u64::MAX, + maximum_builtin_retained_bytes: u64::MAX, + maximum_control_retained_bytes: u64::MAX, + maximum_named_allocation_bytes: u64::MAX, + maximum_meter_streams: 0, + maximum_meter_items: 0, + maximum_meter_bytes: 0, + maximum_control_frame_bytes: 0, + maximum_replay_bytes: 0, + maximum_replay_entries: 0, + maximum_submixes: 0, + maximum_vcas: 0, + maximum_vca_reach_pairs: 0, + maximum_replacement_peak_bytes: u64::MAX, + } + } + + /// `unlimited` with cap `index` of [`CAPS`] set to `value`. + fn with_cap(index: usize, value: u64) -> ControlLimits { + let mut limits = unlimited(); + *match index { + 0 => &mut limits.maximum_graph_session_plus_plan_bytes, + 1 => &mut limits.maximum_source_total_bytes, + 2 => &mut limits.maximum_source_overhead_bytes, + 3 => &mut limits.maximum_replacement_peak_bytes, + 4 => &mut limits.maximum_effect_state_bytes, + 5 => &mut limits.maximum_effect_scratch_bytes, + 6 => &mut limits.maximum_builtin_retained_bytes, + 7 => &mut limits.maximum_control_retained_bytes, + _ => &mut limits.maximum_named_allocation_bytes, + } = value; + limits + } + + fn candidate(base: u64) -> ProspectivePlan { + let epoch = epoch(base); + ProspectivePlan { + resources: epoch.report, + carried: CarriedSourceBytes::default(), + capi: epoch.capi, + } + } + + fn verdict( + held: &[LiveEpochResources], + prospective: ProspectivePlan, + models: CompiledModelAdmission, + limits: ControlLimits, + ) -> Result<(), String> { + validate_replacement_peak(held, prospective, models, limits) + .map_err(|failure| String::from_utf8(failure.diagnostic().into_owned()).expect("UTF-8")) + } + + /// The peak of each cap in [`CAPS`] for `held` (in base units, oldest first) beside a + /// candidate of base `candidate`, with `models` retained model bytes, spelled out term by + /// term. `carried` is the source total and overhead bytes every held epoch but the oldest + /// one carries from the plan before it, summed: each carried ring is counted once. + fn peaks(held: &[u64], candidate: u64, models: u64, carried: (u64, u64)) -> [u64; 9] { + let sum = + |factor: u64| held.iter().map(|base| factor * base).sum::() + factor * candidate; + let graph = sum(1) + models; + let (older, newest) = (&held[..held.len() - 1], held[held.len() - 1]); + let control = + 7 * newest + older.iter().map(|base| 8 * base).sum::() + (8 + 9) * candidate; + [ + graph, + sum(3) - carried.0, + sum(2) - carried.1, + graph + sum(3) - carried.0, + sum(4), + sum(5), + sum(6), + control, + 0, + ] + } + + fn assert_each_cap( + label: &str, + held: &[LiveEpochResources], + prospective: ProspectivePlan, + models: CompiledModelAdmission, + expected: [u64; 9], + ) { + for (index, (cap, code)) in CAPS.iter().enumerate().take(8) { + assert_eq!( + verdict(held, prospective, models, with_cap(index, expected[index])), + Ok(()), + "{label}, {cap}: accepted at the peak {}", + expected[index] + ); + assert_eq!( + verdict( + held, + prospective, + models, + with_cap(index, expected[index] - 1) + ), + Err((*code).to_owned()), + "{label}, {cap}: refused one byte below" + ); + } + } + + /// Gate 2. Three held plans and three models (two held, one prospective): each cap one byte + /// below the sum refuses with its diagnostic, and at the sum passes; the largest allocation + /// folds every held row. A held candidate's carried rings are counted once, in the plan they + /// were carried from: the middle row's carried bytes leave the source sums, the running plan's + /// do not (it owns its rings). With one held plan the peaks are today's: the running plan + /// beside the candidate. Test value: an admission that forgets a held plan or model (a missing + /// third term) passes one byte over the cap; one that subtracts only the candidate's carried + /// bytes (batch 15a review fold M2) refuses the source caps at the peak, and one that also + /// subtracts the running plan's admits one byte below it. + #[test] + fn the_replacement_admission_sums_every_held_plan_and_model() { + let models = compiled_model_admission_over( + &[model(1 << 20, 3), model(1 << 21, 5)], + model(1 << 22, 7), + ) + .expect("models"); + assert_eq!(models.retained_bytes, (1 << 20) + (1 << 21) + (1 << 22)); + assert_eq!(models.largest_allocation_bytes, 7); + let held = [epoch(1), epoch(10), epoch(100)]; + let prospective = candidate(1_000); + assert_each_cap( + "three held", + &held, + prospective, + models, + peaks(&[1, 10, 100], 1_000, models.retained_bytes, (0, 0)), + ); + // The middle row carries 13 source bytes (5 of them overhead) and the running row 2 (1): + // only the middle row's leave the sums. + let carrying_held = [carrying(1, 2, 1), carrying(10, 13, 5), epoch(100)]; + assert_each_cap( + "three held, a carrying candidate", + &carrying_held, + prospective, + models, + peaks(&[1, 10, 100], 1_000, models.retained_bytes, (13, 5)), + ); + // Today's check, for the same candidate beside the running plan alone. + let today = + compiled_model_admission_over(&[model(1 << 20, 3)], model(1 << 22, 7)).expect("models"); + assert_each_cap( + "one held", + &held[..1], + prospective, + today, + peaks(&[1], 1_000, today.retained_bytes, (0, 0)), + ); + // The largest allocation: a middle held plan's is the strict maximum. + let mut middle = epoch(10); + middle.report.largest_named_allocation_bytes = 50_000; + let held = [epoch(1), middle, epoch(100)]; + for (cap, expected) in [(50_000, Ok(())), (49_999, Err(CAPS[8].1.to_owned()))] { + assert_eq!( + verdict(&held, prospective, models, with_cap(8, cap)), + expected + ); + } + } +} diff --git a/crates/control-plane/src/control.rs b/crates/control-plane/src/control.rs index e90e06eb7..8f7ce18b9 100644 --- a/crates/control-plane/src/control.rs +++ b/crates/control-plane/src/control.rs @@ -56,6 +56,57 @@ pub struct ProviderEpoch { /// the epoch's life and moves only with the epoch, which holds no open stamp between /// commands. pub(crate) revision_gate: PlanRevisionGate, + /// What this epoch's plan carries from the plan it was prepared to displace (#1344 D1): the + /// base inventory of a replacement prepared across this plan should it be withdrawn before + /// render adopts it (#1310 D2). `None` for the boot plan. + pub(crate) carried_base: Option, + /// The part of this epoch's plan's source rows it carries from the plan it was prepared to + /// displace (#1273 D5), kept for the replacement admission while the epoch is a held + /// candidate (#1398 D4, batch 15a review fold M2). Zero for the boot plan. + pub(crate) carried: CarriedSourceBytes, + /// This epoch's committed base while a successor prepared from it is pending (#1310 D3 and the + /// batch 15 root ruling): the model its plan was prepared from with every live record pushed + /// to it, the monitor overlay its cells hold, and the ramp lengths of every edit committed + /// after its last revision. A successor of a withdrawn candidate is prepared against it. + /// Set by the structural commit that displaces this epoch's model, extended by every later + /// commit until this epoch stops being the running one, and dropped then. + pub(crate) kept: Option, + /// This epoch's resources as the admissions charge them, captured when the epoch is parked in + /// `retired_providers` (batch 15a review fold 2, M-1): its report row as it stood when it was + /// promoted out, which no later commit changes, its capi resources and its carried bytes. The + /// report table keeps no row for a parked epoch, but its plan lives until the service step + /// reclaims it, so both admissions charge it from here until then. `None` while the epoch is + /// running or pending. + pub(crate) parked: Option, +} + +/// The committed base an epoch keeps while a successor prepared from it is pending (#1310 D3, +/// root ruling of 2026-10-15 on D2 and D3). +pub(crate) struct KeptBase { + /// The compiled session the epoch's plan renders: the model it was prepared from plus every + /// live record pushed to it. The structural commit that displaced it returned it. Both + /// admissions charge it as a held model (#1398 D4). + pub(crate) model: CompiledSession, + /// The monitor overlay the epoch's cells hold: the session's overlay when the displacing + /// commit landed. A later solo is written to the newer epoch's cells, not to these. + pub(crate) monitor: controls::MonitorOverlay, + /// The ramp length of every row an edit committed after the epoch's last revision gave one, + /// the latest edit winning on every row it wrote (#1394 D2's rule across transactions). + pub(crate) edit_ramps: session::EditRamps, + /// The ramp of the latest overlay change committed after the epoch's last revision, or `None` + /// while no commit since wrote a monitor row: a replacement that writes none renders the + /// overlay change between `monitor` and its own with it, as one transaction holding every + /// write would (`MonitorChange::ramp`). + pub(crate) monitor_ramp: Option>, +} + +/// `earlier`'s ramp lengths with `later`'s laid over them: `later` wins on every row it wrote, with +/// or without a length (`EditRamps::overlay`). The +/// merge of the kept base's lengths with a newer transaction's (#1310, root ruling B15-Q3). +fn merged_ramps(earlier: &session::EditRamps, later: &session::EditRamps) -> session::EditRamps { + let mut merged = earlier.clone(); + merged.overlay(later); + merged } impl ProviderEpoch { @@ -65,6 +116,13 @@ impl ProviderEpoch { self.epoch } + /// The source total and overhead bytes this epoch's plan carries from the plan it was + /// prepared to displace (#1273 D5). + #[cfg(feature = "test-support")] + pub fn test_carried(&self) -> (u64, u64) { + (self.carried.total, self.carried.overhead) + } + /// This epoch's source producers. #[cfg(feature = "test-support")] pub fn test_sources(&self) -> &SourceControlSet { @@ -89,6 +147,29 @@ impl ProviderEpoch { &self.effects } + /// Whether this epoch's set holds source `id` vacant (`Some(true)`), holds its producer + /// (`Some(false)`), or does not hold it (`None`) (#1310 gate 6). It probes with a submission + /// at a sample rate of 1, which the set refuses before its ring sees it, so nothing changes. + #[cfg(feature = "test-support")] + pub fn test_source_vacant(&mut self, id: &[u8]) -> Option { + let empty: [f32; 0] = []; + match self.sources.submit( + id, + host_core::SourceSubmission { + generation: 1, + start_frame: 0, + sample_rate_hz: 1, + planes: &[&empty, &empty], + frames: 0, + end_of_region: false, + }, + ) { + Err(SourceControlError::UnknownSource) => None, + Err(SourceControlError::Vacated) => Some(true), + _ => Some(false), + } + } + pub(crate) fn current( sources: SourceControlSet, inventory: PlanStateInventory, @@ -105,15 +186,22 @@ impl ProviderEpoch { effects, capi, revision_gate, + carried_base: None, + carried: CarriedSourceBytes::default(), + kept: None, + parked: None, }; #[cfg(feature = "test-support")] update_test_owners(|owners| owners.current_provider_constructed += 1); owner } + #[allow(clippy::too_many_arguments)] pub(crate) fn candidate( sources: SourceControlSet, inventory: PlanStateInventory, + carried_base: Option, + carried: CarriedSourceBytes, strips: StripLanes, effects: Box<[host_core::EffectControlProducer]>, capi: CapiResources, @@ -127,6 +215,10 @@ impl ProviderEpoch { effects, capi, revision_gate, + carried_base, + carried, + kept: None, + parked: None, }; #[cfg(feature = "test-support")] update_test_owners(|owners| owners.candidate_provider_constructed += 1); @@ -219,6 +311,17 @@ pub struct TestOwnerCounters { pub reservation_canceled: u64, /// Lifecycle count: reservation committed. pub reservation_committed: u64, + /// Withdrawal outcome (#1310 D1): a structural command took a pending candidate back before + /// render adopted it. + pub candidate_withdrawn: u64, + /// Withdrawal outcome (#1310 D1): render had adopted the pending candidate first. + pub candidate_taken: u64, + /// A withdrawn candidate published again, unchanged, because its replacement was refused + /// (#1310 D4). + pub candidate_republished: u64, + /// A withdrawn candidate superseded: its replacement committed and was published, and it was + /// dropped on the control thread (#1310 D5). + pub candidate_superseded: u64, } /// One effect producer's row of `SessionControlSnapshot::effect_cells`: `(epoch, strip ID, @@ -287,6 +390,9 @@ thread_local! { static TEST_NEXT_ADOPTION: core::cell::Cell> = const { core::cell::Cell::new(None) }; static FORCE_FULL_PREPARATION: core::cell::Cell = const { core::cell::Cell::new(false) }; + /// Whether the service step leaves displaced plans in the retirement queue (batch 15a review + /// fold M1): a render preempted between claiming a candidate and retiring the plan it displaced. + static TEST_HOLD_RECLAMATION: core::cell::Cell = const { core::cell::Cell::new(false) }; /// `(commits made from an in-place token, in-place tokens materialized)` on this thread. static IN_PLACE_COUNTS: core::cell::Cell<(usize, usize)> = const { core::cell::Cell::new((0, 0)) }; static TEST_OWNER_STATE: core::cell::Cell = @@ -311,6 +417,10 @@ thread_local! { reservation_constructed: 0, reservation_canceled: 0, reservation_committed: 0, + candidate_withdrawn: 0, + candidate_taken: 0, + candidate_republished: 0, + candidate_superseded: 0, }) }; } @@ -341,6 +451,7 @@ pub fn test_reset_lifecycle_observer() { TEST_FAULT_STATE.with(|state| state.set(([None; 2], 0))); TEST_OWNER_STATE.with(|state| state.set(TestOwnerCounters::default())); TEST_NEXT_ADOPTION.with(|state| state.set(None)); + TEST_HOLD_RECLAMATION.with(|state| state.set(false)); } /// Makes this thread's control commands prepare their transactions by the full path (#1613): a @@ -351,6 +462,15 @@ pub fn test_force_full_preparation(force: bool) { FORCE_FULL_PREPARATION.with(|state| state.set(force)); } +/// Makes this thread's service steps leave every displaced plan in the retirement queue while +/// `hold` is set (batch 15a review fold M1): the control side then sees what it sees while render, +/// having claimed a candidate, has not yet retired the plan it displaced. A test interleaves a +/// structural edit with that window deterministically; clearing it lets the next step reclaim. +#[cfg(feature = "test-support")] +pub fn test_hold_reclamation(hold: bool) { + TEST_HOLD_RECLAMATION.with(|state| state.set(hold)); +} + /// Whether a live commit on this thread checks, in a debug build, that its catalog stage equals a /// full build of the committed model (on by default; [`host_core::verify_stages`]). The check /// builds the whole catalog, so a test that counts the allocations of a command turns it off. @@ -372,6 +492,36 @@ pub fn test_lifecycle_counters() -> TestOwnerCounters { TEST_OWNER_STATE.with(core::cell::Cell::get) } +/// The most plans the control plane can hold unreclaimed (#1398 D4; batch 15a review fold 2, +/// M-1): every parked epoch, the running one and every pending candidate. +const HELD_EPOCHS: usize = RETIRED_PROVIDERS + 1 + PENDING_PROVIDERS; + +/// Every plan the control plane holds unreclaimed, oldest first: the parked epochs, the running +/// plan, the pending candidates (#1398 D4), without an allocation. +pub(crate) struct HeldEpochs { + rows: [LiveEpochResources; HELD_EPOCHS], + len: usize, +} + +impl HeldEpochs { + pub(crate) fn as_slice(&self) -> &[LiveEpochResources] { + &self.rows[..self.len] + } +} + +/// Every compiled model the control plane holds (#1398 D4, #1310 D3): the committed one, then the +/// model each held epoch keeps for a pending successor, without an allocation. +pub(crate) struct HeldModels { + rows: [session::ResourceEstimate; 2 + PENDING_PROVIDERS], + len: usize, +} + +impl HeldModels { + pub(crate) fn as_slice(&self) -> &[session::ResourceEstimate] { + &self.rows[..self.len] + } +} + /// One session's control plane: the control-protocol controller, the provider epochs, the plan /// exchange's control ends and the shared plan projection, generic over the adapter (#1309 D4). pub struct SessionState { @@ -452,6 +602,10 @@ pub struct SessionState { /// The most observation slots any committed plan has had; the ended-event cells and the /// binding table are charged for it (#1676 D2). pub(crate) observation_slots_high_water: usize, + /// The observation records the last rebuild queued again on the plan it published, taken + /// from the candidate it superseded (#1310, root ruling B15-Q4); empty after a rebuild that + /// superseded nothing. + pub(crate) observation_forwarded: Vec, } /// The stable key of one observation tap, borrowed from the bytes that name it (#1677): the strip's @@ -642,6 +796,7 @@ pub(crate) fn observation_slots( pending: PendingArm::None, mark: 0, key_bytes, + queued: false, }); } } @@ -679,6 +834,12 @@ pub(crate) struct ObservationSlot { /// The encoded bytes of the tap's key in an `OBSERVATION_ENDED` event (#1685 D2): what arming /// it adds to the event that would end every armed tap. Set when the plan's slots are built. pub(crate) key_bytes: u16, + /// A record for this tap waits in the queue of a plan render has not adopted (#1310, root + /// ruling B15-Q4): pushed while the newest plan was a pending candidate, or forwarded to it from + /// a candidate it superseded. A replacement of that candidate re-queues the tap's bound state + /// on its own instance, because a tap it carries from the running plan takes that plan's + /// state and would lose the record. + pub(crate) queued: bool, } /// What the host last pushed to one observation slot (#1584 D6a). @@ -712,6 +873,101 @@ fn tap_key_of( }) } +/// One observation record a superseded candidate's queue held, as it is queued again on its +/// replacement's matching instance (#1310, root ruling B15-Q4). +/// +/// A host that keeps its own copy of each plan's armed taps reads them through +/// [`SessionState::observation_forwarded`] and copies them onto the replacement's copy, as it does +/// for a record it admitted itself. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct ObservationForward { + /// The replacement's effect producer, a position in [`SessionState::effect_producers`]. + pub position: usize, + /// The tap's index in that producer's observation menu. + pub tap_index: u32, + /// The tap's bound state now: armed, with its window, or disarmed. + pub armed: bool, + /// The bound window, `0` for the plan's meter window. + pub window_blocks: u32, +} + +/// The records a withdrawn candidate's observation queues hold, as the replacement queues them +/// (#1310, root ruling B15-Q4): one per tap whose slot in `slots` (the candidate's, `offsets` its +/// producers' first slots) is `queued`, on the `successor` instance with the same strip, rack, +/// effect ID and tap ID, carrying the tap's state in `bindings`, which every admitted record has +/// been folded into. An observation queue holds only `Observe` records (arm, with a window, or +/// disarm), so the bound state is what the last record asked for, and draining it at the +/// replacement's first block arms the tap where the candidate's first block would have. A tap the +/// replacement no longer has is skipped: the transaction ended its binding, and the +/// `OBSERVATION_ENDED` event says so. +/// +/// # Errors +/// +/// [`CommandError::Internal`] if a successor queue lacks the room, which cannot happen: its queue +/// is fresh and holds one record per declared tap, and at most one is forwarded per tap. +/// [`CommandError::CompileRejected`] if the list cannot be allocated. +pub(crate) fn observation_forwards( + donor: &[host_core::EffectControlProducer], + offsets: &[u32], + slots: &[ObservationSlot], + successor: &[host_core::EffectControlProducer], + bindings: &controls::ObservationBindings, +) -> Result, CommandError> { + let mut forwards = Vec::new(); + for (position, producer) in donor.iter().enumerate() { + let (Some(&first), Some(&end)) = (offsets.get(position), offsets.get(position + 1)) else { + break; + }; + for slot in first..end { + if !slots.get(slot as usize).is_some_and(|entry| entry.queued) { + continue; + } + let Some(key) = tap_key_of(producer, slot - first) else { + continue; + }; + let target = successor.iter().enumerate().find_map(|(into, candidate)| { + if candidate.track_id != producer.track_id + || candidate.address.rack != producer.address.rack + || candidate.effect_id != producer.effect_id + { + return None; + } + let tap = candidate + .descriptor + .observations + .iter() + .position(|declared| declared.id.0 == key.tap_id) + .filter(|tap| *tap < candidate.observe_capacity())?; + Some((into, u32::try_from(tap).ok()?)) + }); + let Some((position, tap_index)) = target else { + continue; + }; + let window = bindings.window_of(&key); + forwards + .try_reserve(1) + .map_err(|_| CommandError::CompileRejected(failure(ResourceFault::Allocation)))?; + forwards.push(ObservationForward { + position, + tap_index, + armed: window.is_some(), + window_blocks: window.unwrap_or(0), + }); + } + } + for forward in &forwards { + let needed = forwards + .iter() + .filter(|other| other.position == forward.position) + .count(); + if needed > successor[forward.position].observe_available_capacity() { + debug_assert!(false, "a fresh observation queue holds one record per tap"); + return Err(CommandError::Internal); + } + } + Ok(forwards) +} + /// The wire key of an ended binding. fn ended_key(key: &controls::TapKey) -> protocol::ObservationTapKey { protocol::ObservationTapKey { @@ -1363,6 +1619,14 @@ where &self.retired_providers } + /// The provider epoch `epoch`, current or pending, mutably (#1310 gate 6). + #[cfg(feature = "test-support")] + pub fn test_epoch_mut(&mut self, epoch: u64) -> Option<&mut ProviderEpoch> { + core::iter::once(&mut self.providers) + .chain(&mut self.pending_providers) + .find(|provider| provider.epoch == epoch) + } + /// Schedules the next structural publication's adoption (#1348 gate 4): `command` reserves /// that one candidate with `adoption` instead of [`PlanAdoption::Next`]. Taken once, by the /// next structural publication on this thread. @@ -1387,25 +1651,84 @@ where .values } - /// The resource row a structural candidate must fit beside (issue #1042). + /// Every plan the control plane holds unreclaimed, as the admissions read them (#1398 D4), in + /// epoch order, oldest first: every parked epoch whose plan render displaced but the service + /// step has not yet reclaimed, then the running plan, then every pending or withdrawn + /// candidate, the newest last. /// - /// That is the newest plan, the one still live when the candidate would be published: the - /// pending candidate's while it waits for its swap, else the current provider's. With nothing - /// pending and the atomic caught up, this is the rendering plan's row, as before. Pairing a - /// candidate with an older row would report a valid edit as a compile rejection whenever the - /// two plans could never coexist. - pub(crate) fn replacement_base_report(&self) -> Result { - let newest = self - .pending_providers - .last() - .map_or(self.providers.epoch, |provider| provider.epoch); - self.shared + /// The running and pending rows are the provider's own epoch's, read from the report table, + /// never the atomic's. A parked epoch's row is the one it captured when it was promoted out + /// (`ProviderEpoch::parked`; batch 15a review fold 2, M-1): its plan is alive until the + /// service step reclaims it from the retirement queue, so it is charged until then. The + /// service step runs first in every command and reclaims every plan render has already + /// handed back, so what stays parked is a plan render displaced but has not yet retired (a + /// render preempted between its claim and its retirement), or one a `Taken` withdrawal + /// promoted past before render retired it. A plan the atomic names while it lags is the + /// running provider's or a parked one's, so it is charged once (#1042). + pub(crate) fn held_epochs(&self) -> Result { + let reports = self + .shared .reports .lock() - .map_err(|_| CommandError::Internal)? - .iter() - .find_map(|(epoch, report)| (*epoch == newest).then_some(PlanResources::from(*report))) - .ok_or(CommandError::Internal) + .map_err(|_| CommandError::Internal)?; + let row = |provider: &ProviderEpoch| { + reports + .iter() + .find_map(|(epoch, report)| (*epoch == provider.epoch).then_some(*report)) + .map(|report| LiveEpochResources { + report: PlanResources::from(report), + capi: provider.capi, + carried: provider.carried, + }) + .ok_or(CommandError::Internal) + }; + if self.pending_providers.len() > PENDING_PROVIDERS + || self.retired_providers.len() > RETIRED_PROVIDERS + { + return Err(CommandError::Internal); + } + // No allocation: the live arm's admission runs it too. + let running = row(&self.providers)?; + let mut rows = [running; HELD_EPOCHS]; + let mut len = 0; + for provider in &self.retired_providers { + rows[len] = provider.parked.ok_or(CommandError::Internal)?; + len += 1; + } + rows[len] = running; + len += 1; + for provider in &self.pending_providers { + rows[len] = row(provider)?; + len += 1; + } + Ok(HeldEpochs { rows, len }) + } + + /// Every compiled model the control plane holds, as both admissions charge them (#1398 D4, + /// #1310 D3): the committed model, then each held epoch's kept model. A kept model lives only + /// while a successor of its epoch is pending, so while one waits and a live edit is admitted, + /// three models are charged with the prospective one. No allocation: the live arm runs it. + /// + /// Every held epoch has a row, so a kept model never lacks one; were it to, the charge would + /// miss a model, so it is an accounting fault, never a skipped row. + pub(crate) fn held_models(&self) -> Result { + let committed = self.controller.session().compiled().resource_estimate(); + let mut models = HeldModels { + rows: [committed; 2 + PENDING_PROVIDERS], + len: 1, + }; + for kept in core::iter::once(&self.providers) + .chain(&self.pending_providers) + .filter_map(|provider| provider.kept.as_ref()) + { + let Some(slot) = models.rows.get_mut(models.len) else { + debug_assert!(false, "a row for every held epoch's kept model"); + return Err(CommandError::Internal); + }; + *slot = kept.model.resource_estimate(); + models.len += 1; + } + Ok(models) } /// Brings the provider epochs and the resource-report table up to the render thread. @@ -1413,17 +1736,25 @@ where /// A render call swaps plans and retires the outgoing one at its *start*, but publishes /// `active_epoch` only after it returns (issue #1042). A control call in between reads an /// atomic that lags the plan actually rendering, so the atomic alone cannot drive promotion: - /// a reclaimed plan whose epoch is still the current provider's proves that the one pending + /// a reclaimed plan whose epoch is still the current provider's proves that the oldest pending /// candidate has replaced it, and promotes that candidate there. After such a promotion the /// atomic lags the providers until the render call returns, so the atomic promotes only when /// it is ahead of them. + /// + /// **Drain first (batch 15a review fold 2, B-1).** Every reclaimable plan is reclaimed, and + /// its parked epoch released, before the atomic promotes: a promotion parks epochs, and the + /// parked list holds only as many as the retirement queue has credits, so promoting before + /// the drain could find it full of epochs whose plans are already reclaimable and refuse + /// forever. This order is defence in depth, not a tested gate: at the parked list's capacity + /// of three each parked plan still holds a retirement credit, so the list has room for every + /// epoch a promotion parks whichever order runs, and no test is red without the order (batch + /// 15a fold 3, m-B). fn synchronize_plan_epochs(&mut self) -> Result<(), CommandError> { - let active_epoch = self.shared.active_epoch.load(Ordering::Acquire); - if active_epoch > self.providers.epoch { - self.promote_pending_provider(active_epoch)?; - } - - while let Ok((retired_epoch, retired_plan)) = self.retirer.try_reclaim() { + #[cfg(feature = "test-support")] + let hold = TEST_HOLD_RECLAMATION.with(core::cell::Cell::get); + #[cfg(not(feature = "test-support"))] + let hold = false; + while !hold && let Ok((retired_epoch, retired_plan)) = self.retirer.try_reclaim() { drop(ObservedRetiredPlan::new(retired_plan)); if retired_epoch.0 == self.providers.epoch { let replacement = self @@ -1441,9 +1772,14 @@ where self.retired_providers.remove(index); } + let active_epoch = self.shared.active_epoch.load(Ordering::Acquire); + if active_epoch > self.providers.epoch { + self.promote_pending_provider(active_epoch)?; + } + // Keep exactly the rows a reader can still ask for: the atomic's (possibly lagging), the - // current provider's, and a pending candidate's. The atomic is read under the lock, and - // `retain` shrinks in place. + // current provider's, and every pending candidate's. The atomic is read under the lock, + // and `retain` shrinks in place. let current = self.providers.epoch; let pending = &self.pending_providers; let mut reports = self @@ -1462,18 +1798,54 @@ where /// Makes the pending provider of `epoch` current and parks the previous one until its plan /// is reclaimed. + /// + /// Every pending candidate older than `epoch` is parked with it (#1310): render adopts + /// candidates in publication order, so one it adopted before `epoch` was displaced in turn. + /// The service step promotes the candidate the atomic or a reclaimed plan names; a structural + /// edit that finds the newest candidate taken promotes up to it before it rebuilds + /// (`withdraw_pending_candidate`), so after that edit's commit one candidate at most is + /// pending. Each parked epoch captures its report row first (`ProviderEpoch::parked`): the + /// table drops the row, but the admissions charge the plan until it is reclaimed. fn promote_pending_provider(&mut self, epoch: u64) -> Result<(), CommandError> { let index = self .pending_providers .iter() .position(|provider| provider.epoch == epoch) .ok_or(CommandError::Internal)?; - if self.retired_providers.len() == self.retired_providers.capacity() { + if self.retired_providers.len() + index + 1 > self.retired_providers.capacity() { return Err(CommandError::Internal); } - let next = self.pending_providers.remove(index); - let previous = core::mem::replace(&mut self.providers, next); - self.retired_providers.push(previous); + // Fold 2, M-1: each parked epoch keeps the row the admissions charge until its plan is + // reclaimed. Every row is read before anything moves, so a missing one moves nothing. + let mut parked = [None; 1 + PENDING_PROVIDERS]; + { + let reports = self + .shared + .reports + .lock() + .map_err(|_| CommandError::Internal)?; + let rows = core::iter::once(&self.providers).chain(&self.pending_providers[..index]); + for (slot, provider) in parked.iter_mut().zip(rows) { + let report = reports + .iter() + .find_map(|(epoch, report)| (*epoch == provider.epoch).then_some(*report)) + .ok_or(CommandError::Internal)?; + *slot = Some(LiveEpochResources { + report: PlanResources::from(report), + capi: provider.capi, + carried: provider.carried, + }); + } + } + for row in parked.into_iter().take(index + 1) { + let next = self.pending_providers.remove(0); + let mut previous = core::mem::replace(&mut self.providers, next); + // #1310 D3: render no longer runs the previous plan, so no successor is prepared + // against its kept base again; the base drops here, on the control thread. + previous.kept = None; + previous.parked = row; + self.retired_providers.push(previous); + } Ok(()) } @@ -1595,14 +1967,6 @@ where ))); } } - // Issue #1042: while the atomic lags the providers, a render call has swapped plans - // but not yet published the epoch. The retired plan's row stays for any-thread - // readers, so the report table is full and no candidate could be admitted: refuse - // before compiling, with retryable backpressure rather than a wasted compile. The - // live arm above adds no row, so it skips this (#1053 D7). - if self.shared.active_epoch.load(Ordering::Acquire) < self.providers.epoch { - return Err(CommandError::Backpressure); - } #[cfg(feature = "test-support")] if self.take_test_fault(TestStructuralFaultPhase::BeforeRuntimePrepare) { drop(prepared); @@ -1630,8 +1994,6 @@ where .ok_or(CommandError::Internal)? .normalized_model(), ); - // Issue #1273 D1: the candidate succeeds the newest plan, so every source the - // transaction leaves unchanged is prepared to carry that plan's ring. // #1584 D6a: the subscriptions the successor is prepared with, and those the // transaction ends. The room for their one `OBSERVATION_ENDED` event (every ended // key in it) is reserved with the token's `SESSION_COMMITTED`, before anything is @@ -1678,189 +2040,500 @@ where Err(_) => return Err(CommandError::Internal), } } - let prepared_runtime = prepare_runtime::( - &*self.preparer, - prepared - .get() - .prospective_session() - .compiled() - .ok_or(CommandError::Internal)?, - self.limits, - Some(SuccessorBase { - inventory: &self.newest_providers().inventory, - committed: self.controller.session().compiled().normalized_model(), - revision, - // #1394 D6: a carry retarget is the record of the transaction's own - // edit, so it takes that edit's ramp length. - edit_ramps: prepared.get().prospective_session().edit_ramps(), - monitor: host_core::MonitorBase { - committed: &self.monitor, - next: &monitor_change.overlay, - ramp: monitor_change.ramp, - }, - observation: &observation_change.bindings, - }), - &self.live_values_superseded, - self.observation_slots_high_water, - ) - .map_err(CommandError::CompileRejected)?; - let PreparedRuntime { - sources, - strips, - effects, - plan: candidate_plan, - resources, - carried, - inventory, - control_catalog: candidate_catalog, - capi: prospective_capi, - revision_gate, - master_track: _, - } = prepared_runtime; - let mut candidate_provider = ProviderEpoch::candidate( - sources, - inventory, - strips, - effects, - prospective_capi, - revision_gate, + // #1310 D1: a pending candidate is taken back before anything is prepared, so the + // replacement knows the plan render runs and the candidate's host-fed rings can + // move while the control thread owns them. + let mut withdrawn = self.withdraw_pending_candidate()?; + let result = self.rebuild( + prepared, + revision, + monitor_change, + observation_change, + &mut withdrawn, ); - let candidate_plan = ObservedCandidatePlan::new(candidate_plan); - #[cfg(feature = "test-support")] - { - if self.take_test_fault(TestStructuralFaultPhase::AfterRuntimePrepare) { - drop(candidate_plan); - drop(candidate_provider); - drop(prepared); - return Err(CommandError::Backpressure); - } + // #1310 D4: a refusal leaves the withdrawn candidate here, whole; it is published + // again with its own epoch, credit, revision gate and words. Nothing was published + // since it was withdrawn, so this cannot fail. + if let Some(candidate) = withdrawn.take() { + self.publisher.republish(candidate); + #[cfg(feature = "test-support")] + update_test_owners(|owners| owners.candidate_republished += 1); } - validate_replacement_peak( - self.replacement_base_report()?, - resources, - carried, - prospective_capi, - compiled_model_admission( - self.controller.session().compiled(), - prepared - .get() - .prospective_session() - .compiled() - .ok_or(CommandError::Internal)?, - ) - .map_err(CommandError::CompileRejected)?, - self.limits, - ) - .map_err(CommandError::CompileRejected)?; + result + } + } + } + + /// Takes back the newest pending candidate if render has not adopted it (#1310 D1). `None` + /// while nothing is pending, or when render adopted it first (`Taken`), which is the path a + /// rebuild on the newest plan has always taken. + /// + /// **Render took the newest candidate (batch 15a review fold, M1).** `Taken`, or an empty + /// mailbox while a candidate is still pending, means render has claimed the newest one; it + /// adopts candidates in publication order, so it has adopted (or is adopting) every pending + /// one. Each is promoted here, up to the newest, before the rebuild runs: the promotion would + /// otherwise wait for the displaced plan's retirement or the atomic, which lag render (a render + /// preempted between its claim and its retirement), and a second `Taken` in that window would + /// leave two adopted candidates pending beside a third (#1398 D3's accounting fault). So the + /// rebuild always finds at most one pending candidate, the one it may supersede. The epochs + /// it parks keep their resource rows (`ProviderEpoch::parked`), so the rebuild's admission + /// still charges a plan render displaced but has not yet retired (batch 15a review fold 2, + /// M-1). + fn withdraw_pending_candidate( + &mut self, + ) -> Result, CommandError> { + let Some(newest) = self.pending_providers.last().map(|provider| provider.epoch) else { + return Ok(None); + }; + match self.publisher.withdraw() { + engine::realtime::Withdrawal::Withdrawn(candidate) => { #[cfg(feature = "test-support")] - if self.take_test_fault(TestStructuralFaultPhase::AfterAdmission) { - drop(candidate_plan); - drop(candidate_provider); - drop(prepared); - return Err(CommandError::Backpressure); - } - if !self.pending_providers.is_empty() { - return Err(CommandError::Backpressure); - } + update_test_owners(|owners| owners.candidate_withdrawn += 1); + Ok(Some(candidate)) + } + engine::realtime::Withdrawal::Taken => { #[cfg(feature = "test-support")] - let adoption = TEST_NEXT_ADOPTION - .with(core::cell::Cell::take) - .unwrap_or(PlanAdoption::Next); - #[cfg(not(feature = "test-support"))] - let adoption = PlanAdoption::Next; - let reservation = match self - .publisher - .reserve_replacement(candidate_plan.take(), adoption) - { - Ok(reservation) => reservation, - Err(error) => { - let returned = match error { - PlanReplacementReservationError::PublicationFull(plan) - | PlanReplacementReservationError::RetirementFull(plan) - | PlanReplacementReservationError::Incompatible(plan) - | PlanReplacementReservationError::EpochExhausted(plan) => plan, - }; - drop(ObservedCandidatePlan::returned(returned)); - return Err(CommandError::Backpressure); + update_test_owners(|owners| owners.candidate_taken += 1); + self.promote_pending_provider(newest)?; + Ok(None) + } + // An earlier withdrawal found it taken: render runs it. + engine::realtime::Withdrawal::Nothing => { + self.promote_pending_provider(newest)?; + Ok(None) + } + } + } + + /// The rebuild of a structural transaction: prepare, admit, reserve and commit a successor plan + /// and publish it (#1273, #1398, #1310). + /// + /// **Supersession (#1310).** With `withdrawn` holding a candidate A taken back unadopted, the + /// successor B is prepared against the plan A succeeds, P0 (D2): A's carried base, and P0's + /// kept model, monitor overlay and edit ramps merged with B's own (D3, root ruling B15-Q3). Every + /// check runs while A is withdrawn, before the commit (D4): preparation, admission, + /// reservation, the source donation from A and the room for A's queued observation records + /// on B (root ruling B15-Q4), then the protocol's commit predicate. A refusal returns with + /// `withdrawn` untouched, and the caller republishes A. After the commit, infallibly and in + /// order (D5): the donation and the producer moves, B's `superseded` word and revision, B's + /// publication, the return to P0's epoch of every producer A still holds that P0 holds + /// vacant, and A's drop. + /// + /// **Acked-batch question (D8): can an ack ever precede a drop? No.** A's revisions stay + /// committed: their edits are in the model B is compiled from, A's host-fed rings move into + /// B by a donation checked before the commit, the producers of sources P0 still renders go + /// back to P0 before A drops, and every observation record queued on A is queued on B before + /// B is published. B is acked only after every fallible step; A's revisions complete as + /// `superseded` when B is adopted (D6). + #[allow(clippy::too_many_lines)] + fn rebuild( + &mut self, + prepared: ObservedPreparedToken, + revision: u64, + monitor_change: controls::MonitorChange, + observation_change: controls::ObservationChange, + withdrawn: &mut Option, + ) -> Result { + let superseding = withdrawn.is_some(); + if let Some(candidate) = withdrawn.as_ref() + && self.pending_providers.last().map(|newest| newest.epoch) != Some(candidate.epoch().0) + { + // The mailbox's last publication is the newest pending candidate by construction. + return Err(CommandError::Internal); + } + let own_ramps = prepared.get().prospective_session().edit_ramps(); + // Whether B writes a monitor row: then its overlay change's ramp is the latest. + let writes_monitor = !prepared.get().monitor_edits().is_empty(); + // The ramp lengths the epoch B succeeds keeps from now on: B's own, or, across a + // withdrawn candidate, the kept base's with B's laid over them (B wins on a shared row). + let kept_ramps = if superseding { + let kept = self + .kept_holder() + .and_then(|holder| holder.kept.as_ref()) + .ok_or(CommandError::Internal)?; + merged_ramps(&kept.edit_ramps, own_ramps) + } else { + own_ramps.clone() + }; + let prospective = prepared + .get() + .prospective_session() + .compiled() + .ok_or(CommandError::Internal)?; + let base = if superseding { + // #1310 D2: A's carried base (every row a P0 row) and P0's kept base. + let kept = self + .kept_holder() + .and_then(|holder| holder.kept.as_ref()) + .ok_or(CommandError::Internal)?; + let inventory = self + .pending_providers + .last() + .and_then(|candidate| candidate.carried_base.as_ref()) + .ok_or(CommandError::Internal)?; + SuccessorBase { + inventory, + committed: kept.model.normalized_model(), + revision, + edit_ramps: &kept_ramps, + monitor: host_core::MonitorBase { + committed: &kept.monitor, + next: &monitor_change.overlay, + ramp: if writes_monitor { + monitor_change.ramp + } else { + kept.monitor_ramp.unwrap_or(monitor_change.ramp) + }, + }, + observation: &observation_change.bindings, + } + } else { + // Issue #1273 D1: the candidate succeeds the newest plan, so every source the + // transaction leaves unchanged is prepared to carry that plan's ring. + SuccessorBase { + inventory: &self.newest_providers().inventory, + committed: self.controller.session().compiled().normalized_model(), + revision, + // #1394 D6: a carry retarget is the record of the transaction's own edit, so it + // takes that edit's ramp length. + edit_ramps: own_ramps, + monitor: host_core::MonitorBase { + committed: &self.monitor, + next: &monitor_change.overlay, + ramp: monitor_change.ramp, + }, + observation: &observation_change.bindings, + } + }; + let prepared_runtime = prepare_runtime::( + &*self.preparer, + prospective, + self.limits, + Some(base), + &self.live_values_superseded, + self.observation_slots_high_water, + ) + .map_err(CommandError::CompileRejected)?; + let PreparedRuntime { + sources, + strips, + effects, + plan: candidate_plan, + resources, + carried, + inventory, + carried_base, + control_catalog: candidate_catalog, + capi: prospective_capi, + revision_gate, + master_track: _, + } = prepared_runtime; + let mut candidate_provider = ProviderEpoch::candidate( + sources, + inventory, + carried_base, + carried, + strips, + effects, + prospective_capi, + revision_gate, + ); + let candidate_plan = ObservedCandidatePlan::new(candidate_plan); + #[cfg(feature = "test-support")] + { + if self.take_test_fault(TestStructuralFaultPhase::AfterRuntimePrepare) { + drop(candidate_plan); + drop(candidate_provider); + drop(prepared); + return Err(CommandError::Backpressure); + } + } + // #1398 D4: the candidate beside every plan the control plane holds unreclaimed, a + // withdrawn candidate included, and every model it holds, the kept ones included (#1310 + // D3), checked before anything is reserved or committed. + let held = self.held_epochs()?; + let models = self.held_models()?; + let limits = self.limits; + let admit = |held: &[LiveEpochResources], models: &[session::ResourceEstimate]| { + validate_replacement_peak( + held, + ProspectivePlan { + resources, + carried, + capi: prospective_capi, + }, + compiled_model_admission_over(models, prospective.resource_estimate())?, + limits, + ) + }; + if let Err(failure) = admit(held.as_slice(), models.as_slice()) { + // Batch 15a review fold, item 2: a refusal is permanent only if the edit cannot fit + // even beside the newest plan alone and the committed model, which is all a retry + // meets once render has adopted every pending candidate (the plans before it are + // then reclaimed and their kept models dropped). Otherwise the peak is the pending + // plans', which a render frees, and the refusal is retryable backpressure. The + // second check charges the candidate's rows as prepared: a ring it takes from a + // withdrawn candidate by donation is counted in both, so it errs toward the + // permanent refusal by at most those rings' bytes, never toward an admission. + let held = held.as_slice(); + return Err( + match held.split_last().filter(|(_, older)| !older.is_empty()) { + Some((newest, _)) + if admit(core::slice::from_ref(newest), &models.as_slice()[..1]) + .is_ok() => + { + CommandError::Backpressure } - }; - let reservation = ObservedReservation::new(reservation); - #[cfg(feature = "test-support")] - { - if take_test_fault_state(TestStructuralFaultPhase::AfterPlanReservation) { - drop(reservation); - drop(candidate_provider); - drop(prepared); - return Err(CommandError::Backpressure); + _ => CommandError::CompileRejected(failure), + }, + ); + } + #[cfg(feature = "test-support")] + if self.take_test_fault(TestStructuralFaultPhase::AfterAdmission) { + drop(candidate_plan); + drop(candidate_provider); + drop(prepared); + return Err(CommandError::Backpressure); + } + // #1310 D1: B is published with the adoption of its own classification, never A's. + #[cfg(feature = "test-support")] + let adoption = TEST_NEXT_ADOPTION + .with(core::cell::Cell::take) + .unwrap_or(PlanAdoption::Next); + #[cfg(not(feature = "test-support"))] + let adoption = PlanAdoption::Next; + let reservation = match self + .publisher + .reserve_replacement(candidate_plan.take(), adoption) + { + Ok(reservation) => reservation, + Err(error) => { + // #1398 D3: the exchange's publication cells and its retirement credits (#1398 + // D1) hold every plan a candidate can find there, so a full one is an accounting + // fault, not backpressure. An incompatible envelope and an exhausted epoch stay as + // they were. + let (returned, refusal) = match error { + PlanReplacementReservationError::PublicationFull(plan) + | PlanReplacementReservationError::RetirementFull(plan) => { + debug_assert!(false, "#1398 D1 sizes the exchange for every candidate"); + (plan, CommandError::Internal) } - } - let epoch = reservation.epoch(); - candidate_provider.epoch = epoch; - let mut reports = self - .shared - .reports - .lock() - .map_err(|_| CommandError::Internal)?; - if reports.len() == reports.capacity() - || self.pending_providers.len() == self.pending_providers.capacity() - { - return Err(CommandError::Backpressure); - } + PlanReplacementReservationError::Incompatible(plan) + | PlanReplacementReservationError::EpochExhausted(plan) => { + (plan, CommandError::Backpressure) + } + }; + drop(ObservedCandidatePlan::returned(returned)); + return Err(refusal); + } + }; + let mut reservation = ObservedReservation::new(reservation); + #[cfg(feature = "test-support")] + { + if take_test_fault_state(TestStructuralFaultPhase::AfterPlanReservation) { + drop(reservation); + drop(candidate_provider); + drop(prepared); + return Err(CommandError::Backpressure); + } + } + // #1310 D4 (#1344 D3): the rings A created fresh that B keeps, checked while A is + // withdrawn; nothing moves until after the commit. A withdrawn candidate was never + // rendered, so neither refusal can arise: either is an internal fault. + let donation = match withdrawn.as_ref() { + Some(candidate) => { + let donor = self + .pending_providers + .last() + .ok_or(CommandError::Internal)?; + Some( + host_core::check_donation( + host_core::DonationSide { + plan: reservation.plan(), + sources: &candidate_provider.sources, + inventory: &candidate_provider.inventory, + model: prospective.normalized_model(), + }, + host_core::DonationSide { + plan: candidate.plan(), + sources: &donor.sources, + inventory: &donor.inventory, + model: self.controller.session().compiled().normalized_model(), + }, + ) + .map_err(|_| CommandError::Internal)?, + ) + } + None => None, + }; + // #1310, root rulings of 2026-10-16 (no ack before a drop; M-A of fold 3): a ring A + // created (A added or redeclared the source) that the host fed and the donation above does + // not move to B would drop with A, and with it PCM the host was acked for. Until an issue + // lets B read it (#1698 for a source B reverts to its P0 form), such a supersession is + // refused here as backpressure. The guard reads the donation plan itself, so it sits after + // the check, still before the commit: the reservation it follows is cancelled by the + // return (it returns its credit and consumes no epoch), nothing else has changed, the + // caller republishes A, and once render adopts A the same edit commits on it. + if let (Some(plan), Some(candidate)) = (donation.as_ref(), withdrawn.as_ref()) { + let donor = self + .pending_providers + .last() + .ok_or(CommandError::Internal)?; + if host_core::drops_acked_fresh_source( + host_core::DonationSide { + plan: candidate.plan(), + sources: &donor.sources, + inventory: &donor.inventory, + model: self.controller.session().compiled().normalized_model(), + }, + plan, + ) { + return Err(CommandError::Backpressure); + } + } + // #1310 (root ruling B15-Q4): the observation records queued on A, re-queued on B's matching + // instances, with their room checked here. A tap B carries from P0 takes P0's state at + // the swap, which never saw them; arming from the bindings covers only a tap B starts. + let forwards = match self.pending_providers.last() { + Some(donor) if superseding => observation_forwards( + &donor.effects, + &self.observation_offsets, + &self.observation_pending, + &candidate_provider.effects, + &observation_change.bindings, + )?, + _ => Vec::new(), + }; + let epoch = reservation.epoch(); + candidate_provider.epoch = epoch; + let mut reports = self + .shared + .reports + .lock() + .map_err(|_| CommandError::Internal)?; + // #1398 D3: the report table and the pending list hold every row a candidate can meet + // (#1398 D1), so a full one is an accounting fault. A superseded candidate's row and + // entry leave before B's arrive (#1310 D5). + let freed = usize::from(superseding); + if reports.len().saturating_sub(freed) >= reports.capacity() + || self.pending_providers.len().saturating_sub(freed) + >= self.pending_providers.capacity() + { + debug_assert!( + false, + "#1398 D1 sizes the report table and the pending list" + ); + return Err(CommandError::Internal); + } - #[cfg(feature = "test-support")] - if take_test_fault_state(TestStructuralFaultPhase::BeforeProtocolCommit) { - drop(reports); - drop(reservation); - drop(candidate_provider); - drop(prepared); - return Err(CommandError::Backpressure); - } + #[cfg(feature = "test-support")] + if take_test_fault_state(TestStructuralFaultPhase::BeforeProtocolCommit) { + drop(reports); + drop(reservation); + drop(candidate_provider); + drop(prepared); + return Err(CommandError::Backpressure); + } - // #1312 D7: the candidate's gate is stamped with the structural revision, as the - // last fallible step before the protocol commit. A fresh gate takes any revision - // in `1..=GATE_REVISION_MAX`, which #1503 holds for a prepared transaction, so - // this cannot refuse; were it to, the candidate and its handle are dropped here - // with nothing published. Nothing else is stamped on the handle before the - // publication below. The handle inherits the open stamp of the carry retargets - // preparation wrote (#1277 D5), so this is the same revision again, which the pin - // allows. - if candidate_provider.revision_gate.stamp(revision).is_err() { - drop(reports); - drop(reservation); - drop(candidate_provider); - drop(prepared); - return Err(CommandError::Internal); + // #1312 D7: the candidate's gate is stamped with the structural revision, as the last + // fallible step before the protocol commit. A fresh gate takes any revision in + // `1..=GATE_REVISION_MAX`, which #1503 holds for a prepared transaction, so this cannot + // refuse; were it to, the candidate and its handle are dropped here with nothing + // published. Nothing else is stamped on the handle before the publication below. The + // handle inherits the open stamp of the carry retargets preparation wrote (#1277 D5), so + // this is the same revision again, which the pin allows. + if candidate_provider.revision_gate.stamp(revision).is_err() { + drop(reports); + drop(reservation); + drop(candidate_provider); + drop(prepared); + return Err(CommandError::Internal); + } + // #1584 D6a: the newest plan's pending-subscription storage is built here, with the other + // fallible steps, so no allocation follows the commit; the commit then only assigns it. A + // refusal drops what was built and changes nothing. + let Ok(mut slots) = observation_slots( + &candidate_provider.effects, + &observation_change.bindings, + self.observation_wire, + ) else { + drop(reports); + drop(reservation); + drop(candidate_provider); + drop(prepared); + return Err(CommandError::CompileRejected(failure( + ResourceFault::Allocation, + ))); + }; + // #1310 D4: the protocol's own commit predicate, the last check. + if self + .controller + .check_prepared_structural(prepared.get()) + .is_err() + { + return Err(CommandError::Internal); + } + // #1313 D2: a transaction that takes this arm is a rebuild, whatever else it holds. + let mut committed = prepared + .commit(&mut self.controller, protocol::EditPath::Rebuild) + .map_err(|_| CommandError::Internal)?; + // #1310 D3: the model the commit displaced, never one committed in place (#1613). + let displaced = committed.take_displaced(); + debug_assert!(displaced.is_some(), "a rebuild displaces a compiled model"); + // #1559 D8: the overlay is written after the last fallible step, with the commit. + let monitor_ramp = writes_monitor.then_some(monitor_change.ramp); + let previous_monitor = core::mem::replace(&mut self.monitor, monitor_change.overlay); + // #1584 D6a: and the subscriptions the successor was prepared with. + self.observation = observation_change.bindings; + // From here on nothing fails. + let mut superseded = None; + let superseded_revisions = match withdrawn.take() { + Some(candidate) => { + // #1310 D5 step 2: the checked donation, then the producers of the sources B + // carries from P0, which A's commit had moved into A's set. A's own model, which + // B's commit displaced, drops here. + drop(displaced); + let mut donor = self + .pending_providers + .pop() + .expect("the withdrawn candidate is the newest pending epoch"); + let mut donor_plan = candidate.into_plan(); + if let Some(donation) = donation { + host_core::apply_donation( + host_core::DonationSideMut { + plan: reservation.plan_mut(), + sources: &mut candidate_provider.sources, + }, + host_core::DonationSideMut { + plan: &mut donor_plan, + sources: &mut donor.sources, + }, + donation, + ); } - // #1584 D6a: the newest plan's pending-subscription storage is built here, with - // the other fallible steps, so no allocation follows the commit; the commit then - // only assigns it. A refusal drops what was built and changes nothing. - let slots = match observation_slots( - &candidate_provider.effects, - &observation_change.bindings, - self.observation_wire, - ) { - Ok(slots) => slots, - Err(_) => { - drop(reports); - drop(reservation); - drop(candidate_provider); - drop(prepared); - return Err(CommandError::CompileRejected(failure( - ResourceFault::Allocation, - ))); + candidate_provider + .sources + .adopt_persisting(&mut donor.sources); + // P0, the epoch B succeeds, keeps its base with B's edit ramps merged in. + let holder = self + .pending_providers + .last_mut() + .unwrap_or(&mut self.providers); + if let Some(kept) = holder.kept.as_mut() { + kept.edit_ramps = kept_ramps; + if monitor_ramp.is_some() { + kept.monitor_ramp = monitor_ramp; } - }; - // #1313 D2: a transaction that takes this arm is a rebuild, whatever else it holds. - let committed = prepared - .commit(&mut self.controller, protocol::EditPath::Rebuild) - .map_err(|_| CommandError::Internal)?; - // #1559 D8: the overlay is written after the last fallible step, with the commit. - self.monitor = monitor_change.overlay; - // #1584 D6a: and the subscriptions the successor was prepared with. - self.observation = observation_change.bindings; + } + // #1310 D6: every revision after the last one P0's gate published and before B's + // was committed onto A or a candidate A superseded, and its content is in B. + let last_running = holder.revision_gate.last_published(); + superseded = Some((donor_plan, donor)); + revision.saturating_sub(1).saturating_sub(last_running) + } + None => { // Issue #1273 D2: the protocol commit was the last fallible step, so the - // persisting producers move now and never move back. Infallible. - // The newest epoch, spelled by field: the reservation borrows the publisher. + // persisting producers move now and never move back. let newest = self .pending_providers .last_mut() @@ -1868,45 +2541,100 @@ where candidate_provider .sources .adopt_persisting(&mut newest.sources); - self.controller - .provider_mut() - .replace_session_catalog(candidate_catalog, revision); - // #1502 D6: the candidate carries the revision just committed. It goes on the - // candidate's own gate before the candidate is published, so render reports it - // when it adopts the candidate and never earlier, and nothing is stamped on the - // handle in between. - debug_assert_eq!( - revision, - self.controller.session().revision().0, - "the revision stamped is the revision committed (#1312 D7)" - ); - candidate_provider.revision_gate.publish(revision); - self.pending_providers.push(candidate_provider); - // #1584 D6a: the newest plan's slots, built above. The fold ran when this command - // was serviced and nothing has been pushed since (one thread), so none is lost. - ( - self.observation_offsets, - self.observation_pending, - self.observation_batch_counts, - self.observation_armed, - self.observation_armed_bytes, - ) = slots; - self.observation_slots_high_water = self - .observation_slots_high_water - .max(self.observation_pending.len()); - reports.push((epoch, A::Row::from(resources))); - reservation.commit(); - #[cfg(feature = "test-support")] - { - update_test_owners(|owners| { - owners.candidate_provider_published += 1; + // #1310 D3: the epoch B succeeds keeps the model the commit displaced, the + // overlay its cells hold and B's edit ramps, while B is pending. + if let Some(model) = displaced { + newest.kept = Some(KeptBase { + model, + monitor: previous_monitor, + edit_ramps: kept_ramps, + monitor_ramp, }); } - Ok(committed - .write_into(&mut self.response_scratch) - .expect("prepared response capacity was admitted before protocol commit")) + // Nothing was withdrawn, so nothing is pending but B: a taken candidate was + // promoted before the rebuild (`withdraw_pending_candidate`), and promotion drops + // every older epoch's kept base. + debug_assert_eq!( + self.pending_providers.len(), + 0, + "a rebuild that supersedes nothing meets no pending candidate" + ); + 0 } + }; + self.controller + .provider_mut() + .replace_session_catalog(candidate_catalog, revision); + // #1502 D6: the candidate carries the revision just committed. It goes on the candidate's + // own gate before the candidate is published, so render reports it when it adopts the + // candidate and never earlier, and nothing is stamped on the handle in between. + debug_assert_eq!( + revision, + self.controller.session().revision().0, + "the revision stamped is the revision committed (#1312 D7)" + ); + candidate_provider.revision_gate.publish(revision); + // #1310 D5 step 3, D6: B's `superseded` word, written before B is published. + reservation.set_superseded(superseded_revisions); + // #1310 (root ruling B15-Q4): A's queued observation records, on B's queues before B is + // published. Their room was checked before the commit. + for forward in &forwards { + candidate_provider.effects[forward.position] + .try_push_observe(forward.tap_index, forward.armed, forward.window_blocks) + .expect("the room for every forwarded record was checked before the commit"); + let first = slots.0[forward.position]; + if let Some(slot) = slots.1.get_mut((first + forward.tap_index) as usize) { + slot.queued = true; + } + } + // A move: the list was built before the commit. + self.observation_forwarded = forwards; + self.pending_providers.push(candidate_provider); + // #1584 D6a: the newest plan's slots, built above. The fold ran when this command was + // serviced and nothing has been pushed since (one thread), so none is lost. + ( + self.observation_offsets, + self.observation_pending, + self.observation_batch_counts, + self.observation_armed, + self.observation_armed_bytes, + ) = slots; + self.observation_slots_high_water = self + .observation_slots_high_water + .max(self.observation_pending.len()); + if let Some((_, donor)) = superseded.as_ref() { + // #1310 D5 step 6: A's report row leaves; `retain` shrinks in place. + let donor_epoch = donor.epoch; + reports.retain(|(row, _)| *row != donor_epoch); + } + reports.push((epoch, A::Row::from(resources))); + // #1310 D5 step 4: B is published only now, owning every donated ring and record. + reservation.commit(); + drop(reports); + #[cfg(feature = "test-support")] + { + update_test_owners(|owners| { + owners.candidate_provider_published += 1; + }); } + if let Some((donor_plan, mut donor)) = superseded { + // #1310 D5 step 5: P0 still renders until B is adopted, so every producer A still + // holds that P0 holds vacant (a source B removed or redeclared) goes back to P0's + // epoch before A drops. Infallible and allocation-free; keyed by source ID. + let holder = self + .kept_holder_mut() + .expect("B succeeds the epoch A succeeded"); + holder.sources.adopt_persisting(&mut donor.sources); + // Step 6: A's plan, provider epoch (with its gate handle) and credit drop here, on the + // control thread. The credit went back when the plan was taken out. + drop(ObservedCandidatePlan::returned(donor_plan)); + drop(donor); + #[cfg(feature = "test-support")] + update_test_owners(|owners| owners.candidate_superseded += 1); + } + Ok(committed + .write_into(&mut self.response_scratch) + .expect("prepared response capacity was admitted before protocol commit")) } /// The live arm of a structural command (#1257 D2, #1053 D6). @@ -2045,6 +2773,30 @@ where // with no record is a revision render has nothing to do for), and a transaction of // monitor writes does when the overlay change produced a record (#1559 D4, #1603 D7). let publishes = !monitor_only || !delta.is_empty(); + // #1310 (root ruling B15-Q3): while a successor is pending, the ramp lengths this edit gives + // its rows join the kept base of the epoch it succeeds, so a replacement prepared across a + // withdrawn candidate ramps them as this edit does. Built before any check, so the commit + // below only moves it; `None` when nothing is pending or the edit wrote no live row. + let kept_ramps = self + .kept_holder() + .and_then(|holder| holder.kept.as_ref()) + .filter(|_| { + !prepared + .get() + .prospective_session() + .edit_ramps() + .is_untouched() + }) + .map(|kept| { + merged_ramps( + &kept.edit_ramps, + prepared.get().prospective_session().edit_ramps(), + ) + }); + // And the ramp of the overlay change, when the edit writes a monitor row: a replacement + // that writes none renders the solo with it (#1310, kept monitor overlay). + let kept_monitor_ramp = + (!prepared.get().monitor_edits().is_empty()).then_some(monitor_change.ramp); // 1. Admission (#1053 D8), with the catalog of the committed model built first, so that // its growth is charged and the commit's last steps cannot fail for room (#1552 D3, @@ -2398,6 +3150,18 @@ where if let Some(key) = key { revision_gate.publish_key(key); } + // #1310: the kept base's lengths, built above, now that the edit is committed. + if let Some(kept) = self + .kept_holder_mut() + .and_then(|holder| holder.kept.as_mut()) + { + if let Some(ramps) = kept_ramps { + kept.edit_ramps = ramps; + } + if kept_monitor_ramp.is_some() { + kept.monitor_ramp = kept_monitor_ramp; + } + } LiveCommit::Done(Ok(committed.write_into(&mut self.response_scratch).expect( "prepared response capacity was admitted before the live commit", ))) @@ -2423,7 +3187,7 @@ where let next = transaction .in_place_estimate() .unwrap_or_else(|| committed.resource_estimate()); - compiled_model_admission_between(committed.resource_estimate(), next) + compiled_model_admission_over(self.held_models()?.as_slice(), next) .map_err(CommandError::CompileRejected)? } else { // A live delta may add a VCA, so it meets the counts a rebuild would (#1558 D6): the @@ -2431,9 +3195,12 @@ where host_core::check_counts(view.structure(), &prepare_caps(self.limits)).map_err( |diagnostics| CommandError::CompileRejected(prepare_failure(diagnostics)), )?; - compiled_model_admission( - committed, - transaction.compiled().ok_or(CommandError::Internal)?, + compiled_model_admission_over( + self.held_models()?.as_slice(), + transaction + .compiled() + .ok_or(CommandError::Internal)? + .resource_estimate(), ) .map_err(CommandError::CompileRejected)? }; @@ -2454,28 +3221,13 @@ where host_core::StagedCatalog::Full(full) => full.resources(), }, ); - let (current, pending) = { - let reports = self - .shared - .reports - .lock() - .map_err(|_| CommandError::Internal)?; - let row = |provider: &ProviderEpoch| { - reports - .iter() - .find_map(|(epoch, report)| (*epoch == provider.epoch).then_some(*report)) - .map(|report| LiveEpochResources { - report: PlanResources::from(report), - capi: provider.capi, - }) - .ok_or(CommandError::Internal) - }; - let current = row(&self.providers)?; - let pending = self.pending_providers.last().map(row).transpose()?; - (current, pending) - }; - validate_live_peak(current, pending, models, catalog_admission, self.limits) - .map_err(CommandError::CompileRejected)?; + validate_live_peak( + self.held_epochs()?.as_slice(), + models, + catalog_admission, + self.limits, + ) + .map_err(CommandError::CompileRejected)?; Ok(catalog) } @@ -2579,12 +3331,44 @@ where &self.response_scratch[..bytes] } + /// The epoch the newest pending candidate succeeds, which keeps the committed base a + /// replacement of that candidate is prepared from (#1310 D3): the epoch before the newest, or + /// `None` while nothing is pending. + fn kept_holder(&self) -> Option<&ProviderEpoch> { + match self.pending_providers.len() { + 0 => None, + 1 => Some(&self.providers), + len => self.pending_providers.get(len - 2), + } + } + + /// [`Self::kept_holder`], mutably. + fn kept_holder_mut(&mut self) -> Option<&mut ProviderEpoch> { + match self.pending_providers.len() { + 0 => None, + 1 => Some(&mut self.providers), + len => self.pending_providers.get_mut(len - 2), + } + } + /// The producers of the newest committed session: the pending candidate's while one waits /// for its swap, else the current plan's (issue #1273 D3). pub(crate) fn newest_providers(&self) -> &ProviderEpoch { self.pending_providers.last().unwrap_or(&self.providers) } + /// Whether the control plane still holds the plan of `epoch`: the running plan's, or a pending + /// candidate's (#1310, batch 15a review fold B1). A candidate a later structural edit + /// superseded is dropped with that edit's commit, so a host that keeps tables per plan drops + /// the superseded plan's when this turns false, on the core's word, never its own. + pub fn holds_plan_epoch(&self, epoch: u64) -> bool { + self.providers.epoch == epoch + || self + .pending_providers + .iter() + .any(|provider| provider.epoch == epoch) + } + /// Sets the replacement peak ceiling ([`ControlLimits::maximum_replacement_peak_bytes`]) once /// the adapter knows its own fixed bytes exactly (#1290a D3): they are known only after the /// first plan is prepared, and the limits the session was built with are a plain value. @@ -2927,6 +3711,8 @@ where .effects .get_mut(position)?; let pushed = producer.try_push_observe(tap_index, armed, window_blocks); + // #1310 (root ruling B15-Q4): a record on a plan render has not adopted. + let on_candidate = !self.pending_providers.is_empty(); if pushed.is_ok() { // #1584 D6a: the push is the acknowledgement, so the state every later successor is // prepared from is recorded with it, after the only fallible step and without @@ -2946,6 +3732,7 @@ where } else { PendingArm::Disarm }; + entry.queued |= on_candidate; // The count and the bytes follow the slot's state (#1676, #1685): an arm of an // armed tap only changes its window, and a disarm of a tap that is not armed // removes nothing. @@ -3014,6 +3801,16 @@ where self.monitor.master() } + /// The observation records the last rebuild queued again on the plan it published, from the + /// unadopted candidate it superseded (#1310, root ruling B15-Q4), each with the tap's bound state. + /// Read right after a structural command publishes a plan; empty when it superseded nothing. + /// A host that mirrors each plan's armed taps copies these onto the new plan's mirror, as for a + /// record it admitted: the plan arms them at its first block, whatever its carry brings. + #[must_use] + pub fn observation_forwarded(&self) -> &[ObservationForward] { + &self.observation_forwarded + } + /// The taps the host has armed (#1584 D6a), for a test or an adapter that reports them. #[must_use] pub fn observation_bindings(&mut self) -> &controls::ObservationBindings { diff --git a/crates/control-plane/src/lib.rs b/crates/control-plane/src/lib.rs index b10a2d5ee..e3e10ccc0 100644 --- a/crates/control-plane/src/lib.rs +++ b/crates/control-plane/src/lib.rs @@ -61,9 +61,10 @@ mod plan; pub use adapter::{ControlAdapter, ControlLimits, PlanResources}; pub use compile::{CompiledChildren, RuntimePreparer, compile_children}; pub use control::{ - CommandError, EventError, EventLane, ObservationChange, ObservationKeyError, ObservationKeyRef, - ObservationLimit, ObservationRefusal, REPREPARE_REQUIRED, ResolvedObservationTap, SessionState, - SourceFailure, resolve_observation_tap, + CommandError, EventError, EventLane, ObservationChange, ObservationForward, + ObservationKeyError, ObservationKeyRef, ObservationLimit, ObservationRefusal, + REPREPARE_REQUIRED, ResolvedObservationTap, SessionState, SourceFailure, + resolve_observation_tap, }; pub use error::{CompileFailure, ResourceFault}; /// One planar chunk a host submits to a source of the session ([`SessionState::submit`]). @@ -72,14 +73,16 @@ pub use plan::{PlanQueries, PlanState}; #[cfg(feature = "test-support")] pub use compile::{ - CapiResources, CompiledModelAdmission, LiveCatalogAdmission, LiveEpochResources, - observation_binding_table_bytes, observation_reserve_bytes, prepare_caps, validate_live_peak, + CapiResources, CarriedSourceBytes, CompiledModelAdmission, LiveCatalogAdmission, + LiveEpochResources, observation_binding_table_bytes, observation_reserve_bytes, prepare_caps, + validate_live_peak, }; #[cfg(feature = "test-support")] pub use control::{ EffectCellRow, ProviderEpoch, TestOwnerCounters, TestOwnerState, TestStructuralFaultPhase, - TestTransactionSnapshot, test_force_full_preparation, test_lifecycle_counters, - test_reset_lifecycle_observer, test_take_in_place_counts, test_verify_catalog_stages, + TestTransactionSnapshot, test_force_full_preparation, test_hold_reclamation, + test_lifecycle_counters, test_reset_lifecycle_observer, test_take_in_place_counts, + test_verify_catalog_stages, }; pub(crate) use compile::*; diff --git a/crates/control-plane/src/plan.rs b/crates/control-plane/src/plan.rs index 0fded391e..451171c73 100644 --- a/crates/control-plane/src/plan.rs +++ b/crates/control-plane/src/plan.rs @@ -178,6 +178,24 @@ impl<'a> ObservedReservation<'a> { self.inner.as_ref().expect("reservation is live").epoch().0 } + /// The bound candidate, unpublished (#1310 D4). + pub(crate) fn plan(&self) -> &PreparedRenderPlan { + self.inner.as_ref().expect("reservation is live").plan() + } + + /// The bound candidate, unpublished, for the donation after the commit (#1310 D5). + pub(crate) fn plan_mut(&mut self) -> &mut PreparedRenderPlan { + self.inner.as_mut().expect("reservation is live").plan_mut() + } + + /// The revisions the candidate completes as `superseded` when render adopts it (#1310 D6). + pub(crate) fn set_superseded(&mut self, superseded: u64) { + self.inner + .as_mut() + .expect("reservation is live") + .set_superseded(superseded); + } + pub(crate) fn commit(mut self) { self.inner .take() diff --git a/crates/engine/src/realtime/plan.rs b/crates/engine/src/realtime/plan.rs index d83fa9850..6ce2f9471 100644 --- a/crates/engine/src/realtime/plan.rs +++ b/crates/engine/src/realtime/plan.rs @@ -297,6 +297,13 @@ pub trait PreparedPlanExecutor: Send { fn as_any_mut(&mut self) -> Option<&mut dyn core::any::Any> { None } + /// This executor as shared `Any`, so the crate that built it can read its own executor type + /// through a shared plan off the render thread (#1344 D3: a donation check reads a withdrawn + /// candidate the control thread holds only by shared reference). `None` (the default) offers + /// nothing. + fn as_any(&self) -> Option<&dyn core::any::Any> { + None + } /// Take state from the plan this one displaces, in the block that makes this plan active. /// /// **Where it runs.** On the render thread, inside the render scope, in the one block that @@ -1215,6 +1222,15 @@ impl PreparedRenderPlan { self.executor.as_deref_mut()?.as_any_mut() } + /// This plan's executor as shared `Any` ([`PreparedPlanExecutor::as_any`]): the read-only twin + /// of [`Self::executor_any_mut`], for a check off the render thread that must not take the + /// plan mutably. `None` without an executor or when the executor offers none. + #[doc(hidden)] + #[must_use] + pub fn executor_any(&self) -> Option<&dyn core::any::Any> { + self.executor.as_deref()?.as_any() + } + /// The executor's [`PreparedPlanExecutor::prime_ready`]; `false` without an executor. pub(crate) fn prime_ready(&self, block_start: u64, lead_blocks: u32) -> bool { self.executor diff --git a/crates/engine/src/realtime/plan_exchange.rs b/crates/engine/src/realtime/plan_exchange.rs index a34e57d18..c785ca33a 100644 --- a/crates/engine/src/realtime/plan_exchange.rs +++ b/crates/engine/src/realtime/plan_exchange.rs @@ -637,6 +637,20 @@ impl PlanReplacementReservation<'_> { self.adoption } + /// The bound candidate, unpublished (#1310 D4): a control-side check reads it before the + /// commit that decides whether it is published. + #[must_use] + pub const fn plan(&self) -> &PreparedRenderPlan { + &self.plan + } + + /// The bound candidate, unpublished and owned by this control thread (#1310 D5): a source + /// donation moves rings into it after the protocol commit and before [`Self::commit`] + /// publishes it. Nothing here changes its envelope, which the reservation checked. + pub fn plan_mut(&mut self) -> &mut PreparedRenderPlan { + &mut self.plan + } + /// Set how many revisions the candidate folds in from candidates it replaced (#1314 D1, /// #1310): its adoption completes them as `superseded`. pub fn set_superseded(&mut self, superseded: u64) { diff --git a/crates/graph/src/lib.rs b/crates/graph/src/lib.rs index 414d9123d..e1cf6d6a2 100644 --- a/crates/graph/src/lib.rs +++ b/crates/graph/src/lib.rs @@ -2633,6 +2633,39 @@ pub trait GraphPreparedSourceSetDriver: Send { ) -> bool { false } + /// This driver as shared `Any`, so [`Self::can_swap_sources`] can downcast the other driver. + /// `None` (the default) offers nothing. + fn as_any(&self) -> Option<&dyn Any> { + None + } + /// Whether source `source_index`'s consumer has begun a block (#1344 D5), or `None` when the + /// index is out of range, the entry is vacant, or the driver cannot tell (the default). Read + /// off the render thread. + fn source_has_begun_block(&self, _source_index: usize) -> Option { + None + } + /// Whether [`Self::swap_sources`] can exchange every `(this index, other index)` pair (#1344 + /// D4): both entries occupied and of equal channel count, both drivers of one quantum. The + /// occupied-to-occupied twin of [`Self::adopt_sources`]'s check. Read-only, off the render + /// thread. `false` (the default) swaps nothing. + fn can_swap_sources( + &self, + _other: &dyn GraphPreparedSourceSetDriver, + _pairs: &[(u32, u32)], + ) -> bool { + false + } + /// Exchange each checked pair's two entries whole (#1344 D4): ring, queued blocks, generation, + /// read position and held seek. All or nothing: it re-checks the pairs and on `false` moves + /// nothing. Control-thread code, on two plans neither of which render owns; it allocates and + /// frees nothing. + fn swap_sources( + &mut self, + _other: &mut dyn GraphPreparedSourceSetDriver, + _pairs: &[(u32, u32)], + ) -> bool { + false + } } /// The source set as a bank's gather sees it during the unit loop (issue #918): a shared view of @@ -4363,6 +4396,185 @@ pub fn carry_program_retained_bytes(plan: &mut PreparedRenderPlan) -> u64 { }) } +/// Every predecessor key the hand-over installed in a successor graph plan reads (#1344 D1), one +/// list per executor carry section, each in its section's order. What the successor carried is +/// derived from this, never recorded family by family. +#[derive(Clone, Debug, Default, Eq, PartialEq)] +pub struct GraphCarryReads { + /// [`GraphCarryProgram::predecessor`]: the plan every key below belongs to. + pub predecessor: u64, + /// Source indices the program moves consumers from. + pub sources: Vec, + /// Lanes the builtin-input section copies from. + pub input_lanes: Vec, + /// `(owner, lane)` pairs the strip section copies from. + pub strip_lanes: Vec<(BuiltinStripOwner, GraphLaneLocation)>, + /// Effect instances the effect section moves from. + pub effect_sites: Vec, + /// `(kind, observer)` pairs the observer section takes: meters and spectrum captures. + pub observers: Vec<(ObserverKind, GraphObserverLocation)>, + /// Effect instances whose observation taps the observer section takes. + pub observation_sites: Vec, + /// Compensation lines the delay section continues. + pub delay_lines: Vec, + /// Strip delay lines the strip-delay section continues. + pub strip_delay_lines: Vec, + /// Live route units the live-send section takes. + pub live_routes: Vec, +} + +/// The predecessor keys the hand-over installed in this graph plan reads (#1344 D1); `None` for any +/// other plan and for a graph plan with no [`GraphCarryProgram`] (no section installs without one). +/// +/// Read off the render thread, before publication. It names every executor carry section without a +/// rest pattern, so a carry section added to the executor does not compile until this reports its +/// predecessor keys: the installer of a new family owns its row here. +#[must_use] +pub fn installed_carry_reads(plan: &PreparedRenderPlan) -> Option { + let executor = plan.executor_any()?.downcast_ref::()?; + let GraphExecutor { + runtime: _, + sample_rate_hz: _, + source_set: _, + source_input_buffers: _, + active_units: _, + identity: _, + carry, + carry_inputs, + carry_strips, + carry_effects, + // The payload scratch reads no predecessor key. + carry_scratch: _, + next_block_sample: _, + node_carry_refused: _, + observer_carry, + carry_delays, + compensation_lines: _, + carry_strip_delays, + carry_routes, + bind_identities: _, + live_snapshot: _, + jump_lengths: _, + } = executor; + let program = carry.as_ref()?; + let (observers, observation_sites) = observer_carry.as_deref().map_or_else( + || (Vec::new(), Vec::new()), + |section| { + let ObserverCarryProgram { + moves, + // A join names two successor observers: it reads no predecessor key. + joins: _, + taps, + record: _, + carried: _, + } = section; + ( + moves + .iter() + .map(|step| (step.kind, step.predecessor)) + .collect(), + taps.iter().map(|tap| tap.step.predecessor).collect(), + ) + }, + ); + Some(GraphCarryReads { + predecessor: program.predecessor, + sources: program + .sources + .iter() + .map(|&(_, predecessor)| predecessor) + .collect(), + input_lanes: carry_inputs.iter().map(|step| step.predecessor).collect(), + strip_lanes: carry_strips + .iter() + .map(|step| (step.owner, step.predecessor)) + .collect(), + effect_sites: carry_effects.iter().map(|step| step.predecessor).collect(), + observers, + observation_sites, + delay_lines: carry_delays.iter().map(|step| step.predecessor).collect(), + strip_delay_lines: carry_strip_delays + .iter() + .map(|step| step.predecessor) + .collect(), + live_routes: carry_routes.iter().map(|step| step.predecessor).collect(), + }) +} + +/// Why a source donation between two graph plans is refused (#1344 D4, D5). +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum GraphDonationRefusal { + /// A plan is not a graph plan, or has no source set. + NotAGraphPlan, + /// A donor consumer has begun a block: it has played PCM the host was acked for. + Consumed, + /// A pair is out of range, names a vacant entry or repeats an index, or the two entries differ + /// in channel count or quantum. + Mismatch, +} + +fn graph_source_driver(plan: &PreparedRenderPlan) -> Option<&dyn GraphPreparedSourceSetDriver> { + let executor = plan.executor_any()?.downcast_ref::()?; + Some(executor.source_set.as_ref()?.driver.as_ref()) +} + +fn graph_source_driver_mut( + plan: &mut PreparedRenderPlan, +) -> Option<&mut dyn GraphPreparedSourceSetDriver> { + let executor = plan.executor_any_mut()?.downcast_mut::()?; + Some(executor.source_set.as_mut()?.driver.as_mut()) +} + +/// Check a donation of source consumers from `donor` to `successor` (#1344 D4, D5): for each +/// `(successor index, donor index)` pair, both entries occupied, of equal channel count and +/// quantum, no index repeated, and the donor's consumer has never begun a block. Read-only; both +/// plans are control-owned and neither is published. +pub fn can_donate_sources( + successor: &PreparedRenderPlan, + donor: &PreparedRenderPlan, + pairs: &[(u32, u32)], +) -> Result<(), GraphDonationRefusal> { + let (Some(into), Some(from)) = (graph_source_driver(successor), graph_source_driver(donor)) + else { + return Err(GraphDonationRefusal::NotAGraphPlan); + }; + for (index, &(successor_index, donor_index)) in pairs.iter().enumerate() { + if pairs[..index] + .iter() + .any(|&(s, d)| s == successor_index || d == donor_index) + { + return Err(GraphDonationRefusal::Mismatch); + } + if from.source_has_begun_block(donor_index as usize) == Some(true) { + return Err(GraphDonationRefusal::Consumed); + } + } + if into.can_swap_sources(from, pairs) { + Ok(()) + } else { + Err(GraphDonationRefusal::Mismatch) + } +} + +/// Swap the source consumers [`can_donate_sources`] checked (#1344 D4): each `(successor index, +/// donor index)` pair exchanges its two entries' whole contents (ring, queued blocks, generation, +/// read position and held seek), so the donor ends with the successor's fresh rings. On the +/// control thread, on two plans neither of which render owns. Returns whether the pairs moved; +/// unchecked pairs that fail the driver's own check move nothing. +pub fn donate_sources( + successor: &mut PreparedRenderPlan, + donor: &mut PreparedRenderPlan, + pairs: &[(u32, u32)], +) -> bool { + let (Some(into), Some(from)) = ( + graph_source_driver_mut(successor), + graph_source_driver_mut(donor), + ) else { + return false; + }; + into.swap_sources(from, pairs) +} + /// Bytes of one entry of each executor table sized at bind (issue #936): `(active_units, /// source_input_buffers)`. fn executor_table_entry_bytes() -> (u64, u64) { @@ -4590,6 +4802,10 @@ impl PreparedPlanExecutor for GraphExecutor { Some(self) } + fn as_any(&self) -> Option<&dyn Any> { + Some(self) + } + // REALTIME_POLICY_BEGIN /// Move the sources the installed [`GraphCarryProgram`] names out of the predecessor graph /// plan, and copy the strip input lanes [`install_builtin_input_carry`] names (issue #1276), diff --git a/crates/host-core/src/lib.rs b/crates/host-core/src/lib.rs index ed1b8a454..74151ff4a 100644 --- a/crates/host-core/src/lib.rs +++ b/crates/host-core/src/lib.rs @@ -185,8 +185,9 @@ pub use route_controls::{ pub use session::SOURCE_CHANNELS_LAUNCH_MAXIMUM; pub use shape::{HostSessionShape, compiled_session_shape}; pub use source::{ - SourceControlError, SourceControlSet, SourceSubmission, control_table_bytes, - source_id_arena_bytes, + ConsumerDonation, DonationError, DonationPlan, DonationSide, DonationSideMut, ProducerDonation, + SourceControlError, SourceControlSet, SourceSubmission, apply_donation, check_donation, + control_table_bytes, drops_acked_fresh_source, source_id_arena_bytes, }; pub use spectrum::{ SPECTRUM_BIN_COUNT, SPECTRUM_FLOOR_DB, SPECTRUM_WINDOW_FRAMES, SpectrumAnalysisError, @@ -226,7 +227,8 @@ pub use effect_compiler::{ }; #[cfg(feature = "test-support")] pub use prepare::{ - test_only_prepare_host_runtime_on, test_only_prepare_host_runtime_successor_on, + test_only_carry_read_keys, test_only_inventory_rows, test_only_prepare_host_runtime_on, + test_only_prepare_host_runtime_successor_on, test_only_prepare_host_runtime_with_live_controls_on, test_only_prepare_host_runtime_with_live_controls_successor_on, test_only_prepare_host_runtime_with_selected_meters_on, diff --git a/crates/host-core/src/prepare.rs b/crates/host-core/src/prepare.rs index 6fa86c607..529261b5b 100644 --- a/crates/host-core/src/prepare.rs +++ b/crates/host-core/src/prepare.rs @@ -312,6 +312,10 @@ pub struct HostPrepareReport { pub carry_program_retained_bytes: u64, /// Bytes [`PreparedHost::inventory`] retains ([`PlanStateInventory::retained_bytes`]). pub inventory_retained_bytes: u64, + /// Bytes [`PreparedHost::carried_base`] retains ([`PlanStateInventory::retained_bytes`]); + /// zero for a plan prepared without a successor base (#1344 D1). A host that keeps the base + /// to prepare across this plan once it is withdrawn charges it; one that drops it does not. + pub carried_base_retained_bytes: u64, /// Summed effect state bytes. pub effect_scalar_state_bytes: u64, /// Summed effect scratch bytes. @@ -673,6 +677,16 @@ pub struct PreparedHost { /// What this plan holds, for preparing its successor ([`SuccessorBase`]). Control-side plain /// data; never read on the render thread. pub inventory: PlanStateInventory, + /// What this plan carries from the plan it was prepared to displace (#1344 D1): that plan's + /// inventory restricted to the rows the installed hand-over reads, with its identity, so + /// every row is a predecessor row. `None` for a plan prepared without a [`SuccessorBase`]. + /// + /// Derived once, at the end of preparation, from what is installed in the plan + /// (`graph::installed_carry_reads`), never recorded family by family. Should this plan be + /// committed and then withdrawn before render adopts it, a successor of the plan render runs + /// is prepared with this as its base inventory and the running plan's committed model + /// ([`SuccessorBase`], D2). Control-side plain data; never read on the render thread. + pub carried_base: Option, /// The control list of the compiled session (decision 17, C4), built before graph lowering /// consumes the prepared entries. #[cfg(feature = "control-provider")] @@ -932,6 +946,12 @@ impl PlanStateInventory { self.observations.len() } + /// Spectrum capture rows, one per capture (#1395). + #[must_use] + pub fn spectrum_count(&self) -> usize { + self.spectrum.len() + } + /// The captures this plan takes over from the plan it was prepared to displace (#1395 D1, /// D3): the pairs the control side applies at promotion with /// [`SpectrumService::adopt_successor`](crate::SpectrumService::adopt_successor), when the @@ -1057,6 +1077,270 @@ impl PlanStateInventory { } } +impl PlanStateInventory { + /// This inventory restricted to the rows `reads` names (#1344 D1): the rows of each family a + /// successor's installed hand-over reads, keyed as that section keys its predecessor. Keeps + /// this plan's identity, rate, quantum and capture kind; the pairing and the restart set + /// describe a successor against this plan, not this plan, so they are empty. Every row kept is + /// this plan's own row, unchanged and in its order. + fn restricted_to(&self, reads: &graph::GraphCarryReads) -> Self { + // Named without a rest pattern: a family added to the inventory does not compile until + // its restriction is written here. + let Self { + plan_identity, + sources, + input_sections, + strip_stages, + effect_lanes, + meters, + observations, + sample_rate_hz, + quantum_frames, + spectrum, + spectrum_collection, + spectrum_pairing: _, + delay_lines, + strip_delays, + live_routes, + restarted: _, + } = self; + fn keep( + rows: &[Row], + keys: &BTreeSet, + key: impl Fn(&Row) -> Key, + ) -> Box<[Row]> { + rows.iter() + .filter(|row| keys.contains(&key(row))) + .cloned() + .collect() + } + let observers = |wanted: engine::realtime::ObserverKind| -> BTreeSet<_> { + reads + .observers + .iter() + .filter(|(kind, _)| *kind == wanted) + .map(|(_, location)| *location) + .collect() + }; + let set = |keys: &[u32]| keys.iter().copied().collect::>(); + Self { + plan_identity: *plan_identity, + sources: keep(sources, &set(&reads.sources), |row| row.index), + input_sections: keep( + input_sections, + &reads.input_lanes.iter().copied().collect(), + |row| row.location, + ), + strip_stages: keep( + strip_stages, + &reads.strip_lanes.iter().copied().collect(), + |row| (row.owner, row.location), + ), + effect_lanes: keep( + effect_lanes, + &reads.effect_sites.iter().copied().collect(), + |row| row.location, + ), + meters: keep( + meters, + &observers(engine::realtime::ObserverKind::Meter), + |row| row.location, + ), + observations: keep( + observations, + &reads.observation_sites.iter().copied().collect(), + |row| row.location, + ), + sample_rate_hz: *sample_rate_hz, + quantum_frames: *quantum_frames, + spectrum: keep( + spectrum, + &observers(engine::realtime::ObserverKind::Spectrum), + |row| row.location, + ), + spectrum_collection: *spectrum_collection, + spectrum_pairing: Box::default(), + delay_lines: keep(delay_lines, &set(&reads.delay_lines), |row| row.line), + strip_delays: keep(strip_delays, &set(&reads.strip_delay_lines), |row| row.line), + live_routes: keep(live_routes, &set(&reads.live_routes), |row| row.unit), + restarted: Box::default(), + } + } +} + +/// Test-support view of an inventory (#1344 fold M3): every row as `(family, owner, key)`, where +/// `owner` names what the row belongs to by session identity (a strip, an effect, a route, a +/// source) and `key` is the predecessor key a successor's hand-over names it by, formatted as +/// [`test_only_carry_read_keys`] formats the same key. A gate compares the two to prove a +/// restricted base holds exactly the rows the hand-over reads, and that those rows belong to the +/// owners the test expects. +#[cfg(feature = "test-support")] +#[doc(hidden)] +#[must_use] +pub fn test_only_inventory_rows( + inventory: &PlanStateInventory, +) -> BTreeSet<(&'static str, String, String)> { + let PlanStateInventory { + plan_identity: _, + sources, + input_sections, + strip_stages, + effect_lanes, + meters, + observations, + sample_rate_hz: _, + quantum_frames: _, + spectrum, + spectrum_collection: _, + spectrum_pairing: _, + delay_lines, + strip_delays, + live_routes, + restarted: _, + } = inventory; + let mut rows = BTreeSet::new(); + let mut add = |family, owner: String, key: String| { + rows.insert((family, owner, key)); + }; + for row in sources.iter() { + add("sources", row.id.to_string(), format!("{:?}", row.index)); + } + for row in input_sections.iter() { + add( + "input sections", + row.id.to_string(), + format!("{:?}", row.location), + ); + } + for row in strip_stages.iter() { + add( + "strip stages", + format!("{} {:?}", row.id, row.owner), + format!("{:?}", (row.owner, row.location)), + ); + } + let effect_owner = |location: &graph::GraphEffectSite| { + effect_lanes + .iter() + .find(|lane| lane.location == *location) + .map_or_else( + || "unknown effect".to_owned(), + |lane| format!("{} {:?} {}", lane.id, lane.rack, lane.effect_id), + ) + }; + for row in effect_lanes.iter() { + add( + "effect lanes", + effect_owner(&row.location), + format!("{:?}", row.location), + ); + } + for row in meters.iter() { + add( + "meters", + format!("{} {:?} {:?}", row.id, row.tap, row.metrics), + format!("{:?}", row.location), + ); + } + for row in observations.iter() { + add( + "observations", + effect_owner(&row.location), + format!("{:?}", row.location), + ); + } + for row in spectrum.iter() { + add( + "spectrum", + format!("{:?} {:?}", row.target, row.channels), + format!("{:?}", row.location), + ); + } + for row in delay_lines.iter() { + add( + "delay lines", + format!("{:?}", row.edge), + format!("{:?}", row.line), + ); + } + for row in strip_delays.iter() { + add( + "strip delays", + row.strip.to_string(), + format!("{:?}", row.line), + ); + } + for row in live_routes.iter() { + add( + "live routes", + row.route_id.to_string(), + format!("{:?}", row.unit), + ); + } + rows +} + +/// Test-support view of a hand-over's reads (#1344 fold M3): every predecessor key as `(family, +/// key)`, formatted as [`test_only_inventory_rows`] formats it. +#[cfg(feature = "test-support")] +#[doc(hidden)] +#[must_use] +pub fn test_only_carry_read_keys( + reads: &graph::GraphCarryReads, +) -> BTreeSet<(&'static str, String)> { + let graph::GraphCarryReads { + predecessor: _, + sources, + input_lanes, + strip_lanes, + effect_sites, + observers, + observation_sites, + delay_lines, + strip_delay_lines, + live_routes, + } = reads; + let mut keys = BTreeSet::new(); + let mut add = |family, key: String| { + keys.insert((family, key)); + }; + sources + .iter() + .for_each(|key| add("sources", format!("{key:?}"))); + input_lanes + .iter() + .for_each(|key| add("input sections", format!("{key:?}"))); + strip_lanes + .iter() + .for_each(|key| add("strip stages", format!("{key:?}"))); + effect_sites + .iter() + .for_each(|key| add("effect lanes", format!("{key:?}"))); + for (kind, location) in observers { + let family = match kind { + engine::realtime::ObserverKind::Meter => "meters", + engine::realtime::ObserverKind::Spectrum => "spectrum", + // The observation taps' readers move with their effect instance; the restriction + // keys observation rows by `observation_sites`, below. + engine::realtime::ObserverKind::Observation => continue, + }; + add(family, format!("{location:?}")); + } + observation_sites + .iter() + .for_each(|key| add("observations", format!("{key:?}"))); + delay_lines + .iter() + .for_each(|key| add("delay lines", format!("{key:?}"))); + strip_delay_lines + .iter() + .for_each(|key| add("strip delays", format!("{key:?}"))); + live_routes + .iter() + .for_each(|key| add("live routes", format!("{key:?}"))); + keys +} + /// The plan a successor is prepared to displace (issue #1272 D2, umbrella #1269 P1). /// /// `committed` is the predecessor's base at the moment the successor is prepared. An input @@ -1073,6 +1357,16 @@ impl PlanStateInventory { /// are equal; a difference in its live values (a bypass, a `Block`-rate parameter) is written to /// the successor's cells as a retarget stamped with `revision`, as a strip's is. A strip whose /// effect chain changed restarts even if its remaining slots carry. +/// +/// **Across a withdrawn plan (#1344 D2).** When the plan render runs (P0) has a committed +/// successor A that was taken back before render adopted it, A's replacement B is prepared +/// against P0 with this same entry: `inventory` is A's [`PreparedHost::carried_base`] and +/// `committed` is P0's committed model. Every row of that inventory is a P0 row, so the joins run +/// unchanged and an owner carries from P0 only if A carried it and B leaves it unchanged against +/// P0's model; everything A restarted, added or removed is fresh in B. A strip A restarted has no +/// row there, so B restarts it too and lists it in [`PreparedHost::restarted_strips`]. The rings +/// A created fresh move to B by a checked donation ([`crate::check_donation`]), never through this +/// base. #[derive(Clone, Copy, Debug)] pub struct SuccessorBase<'a> { /// The predecessor's [`PreparedHost::inventory`]. @@ -3185,6 +3479,32 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( live_routes: live_routes.into_boxed_slice(), restarted: restarted.into_boxed_slice(), }; + // #1344 D1: what this plan carries, read back from the hand-over just installed. A successor + // with nothing installed carries nothing, so its base is the predecessor's identity alone. + let carried_base = successor.map(|base| { + let reads = graph::installed_carry_reads(&plan).unwrap_or_default(); + debug_assert!( + reads == graph::GraphCarryReads::default() + || reads.predecessor == base.inventory.plan_identity, + "the hand-over names the base it was prepared against" + ); + let restricted = base.inventory.restricted_to(&reads); + // #1395 D3: the observer section's spectrum moves are the pairs the control side applies. + debug_assert_eq!( + restricted + .spectrum + .iter() + .map(|row| row.entry) + .collect::>(), + inventory + .spectrum_pairing + .iter() + .map(|pair| pair.predecessor) + .collect::>(), + "the spectrum rows the hand-over reads are the ones the pairing names" + ); + restricted + }); let report = HostPrepareReport { sample_rate_hz: compiled.sample_rate().0, quantum_frames: compiled.quantum().0, @@ -3212,6 +3532,9 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( carried_source_overhead_bytes: carried_overhead_bytes, carry_program_retained_bytes, inventory_retained_bytes: inventory.retained_bytes(), + carried_base_retained_bytes: carried_base + .as_ref() + .map_or(0, PlanStateInventory::retained_bytes), effect_scalar_state_bytes: effect_state_bytes, effect_scalar_scratch_bytes: effect_scratch_bytes, builtin_processor_payload_bytes: builtin_resources.engine_owned_processor_payload_bytes, @@ -3328,6 +3651,7 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( sources, report, inventory, + carried_base, #[cfg(feature = "control-provider")] control_catalog, }, diff --git a/crates/host-core/src/render_session.rs b/crates/host-core/src/render_session.rs index 2b55aac08..a51e08896 100644 --- a/crates/host-core/src/render_session.rs +++ b/crates/host-core/src/render_session.rs @@ -192,6 +192,7 @@ impl PreparedHost { sources, report, inventory, + carried_base, #[cfg(feature = "control-provider")] control_catalog, } = self; @@ -203,6 +204,7 @@ impl PreparedHost { sources, report, inventory, + carried_base, #[cfg(feature = "control-provider")] control_catalog, }, diff --git a/crates/host-core/src/source.rs b/crates/host-core/src/source.rs index f4e551682..ce0acaf9c 100644 --- a/crates/host-core/src/source.rs +++ b/crates/host-core/src/source.rs @@ -5,13 +5,18 @@ //! module is the single definition; the rules are capi's, which were the stricter pair. use core::alloc::Layout; +use std::collections::BTreeMap; use engine::SampleRateHz; +use engine::realtime::PreparedRenderPlan; +use session::SessionModel; use source::{ HostChunkError, HostChunkProvider, HostPlanarChunk, SourceCommand, SourceFrame, SourceGeneration, SourceSeekError, SubmitReport, }; +use crate::PlanStateInventory; + /// One host-owned producer endpoint plus the immutable facts a submission is checked against. pub(crate) struct ControlSource { id_offset: usize, @@ -408,3 +413,320 @@ impl ControlSourceBuilder { SourceControlSet::new(ids.into_boxed_slice(), self.sources.into_boxed_slice()) } } + +/// One side of a source donation (#1344 D3): a prepared plan the control thread holds, its source +/// producers, what it holds ([`crate::PreparedHost::inventory`]) and the session model it was +/// compiled from. +/// +/// The successor side is a prepared plan not yet published; the donor side is a plan committed and +/// then withdrawn before render adopted it (an `UnadoptedCandidate`'s `plan()`). Neither is owned +/// by render, so nothing either side holds can change between [`check_donation`] and +/// [`apply_donation`]: holding the plan by reference is what makes it unpublished. +#[derive(Clone, Copy)] +pub struct DonationSide<'a> { + /// The plan, which owns the source consumers. + pub plan: &'a PreparedRenderPlan, + /// The plan's source producers. + pub sources: &'a SourceControlSet, + /// The plan's own inventory: each source's ring configuration and index. + pub inventory: &'a PlanStateInventory, + /// The normalized model the plan was compiled from: each source's declaration. + pub model: &'a SessionModel, +} + +/// The mutable halves [`apply_donation`] moves between: a plan and its source producers. +pub struct DonationSideMut<'a> { + /// The plan, which owns the source consumers. + pub plan: &'a mut PreparedRenderPlan, + /// The plan's source producers. + pub sources: &'a mut SourceControlSet, +} + +/// Why [`check_donation`] refuses (#1344 D5, D6). A refusal moves nothing. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum DonationError { + /// A donor ring the donation would move has been consumed: its consumer has begun a block, so + /// PCM the host was acked for may already be gone. A withdrawn candidate is never rendered + /// (#1311), so only a plan rendered on the control thread (a synchronous host, a test) is + /// refused this way. + Consumed, + /// The apply step could not perform a selected move: a producer is missing on one side, or + /// the two plans' source entries cannot exchange (another quantum or channel count, or not + /// graph plans). + Unswappable, + /// A successor ring the donation would replace has already been fed: its producer accepted + /// PCM, an end of region or a seek. The donation drops that producer and moves its ring to the + /// donor, so the host would lose what it was acked for. An unpublished successor is not + /// reachable by a host submit, so only a caller that fed it by hand (a test) is refused. + SuccessorFed, +} + +/// The producer half of a [`DonationPlan`]: `(successor entry, donor entry)` positions in the two +/// [`SourceControlSet`]s. +#[derive(Debug, Eq, PartialEq)] +pub struct ProducerDonation(Box<[(usize, usize)]>); + +/// The consumer half of a [`DonationPlan`]: `(successor source index, donor source index)` pairs +/// in the two plans' source sets. +#[derive(Debug, Eq, PartialEq)] +pub struct ConsumerDonation(Box<[(u32, u32)]>); + +/// What [`check_donation`] found movable, one pair per donated source, in successor source order. +/// It borrows neither side. +#[derive(Debug, Eq, PartialEq)] +pub struct DonationPlan { + /// The producers `donate_producers` swaps. + pub producers: ProducerDonation, + /// The consumers `donate_consumers` swaps. + pub consumers: ConsumerDonation, +} + +impl DonationPlan { + /// Sources the plan donates. + #[must_use] + pub fn len(&self) -> usize { + self.consumers.0.len() + } + + /// Whether the plan donates nothing. + #[must_use] + pub fn is_empty(&self) -> bool { + self.consumers.0.is_empty() + } +} + +/// Check a donation of fresh source rings from a withdrawn plan to its replacement (#1344 D3, +/// D5). Read-only: it moves nothing and has no side effect. +/// +/// It selects every source that both plans created fresh (each holds the source's ring: neither +/// prepared it vacant to carry from the plan render runs) with the same ID, the same declaration +/// (content, channels, bit depth, frames) and the same ring configuration. A source whose ID +/// matches but whose declaration or ring differs is a source the transaction redeclared, as the +/// carry rule treats it ([`crate::SuccessorBase`]): it is not selected and keeps its own fresh +/// ring. It then checks everything the apply needs, so the apply cannot fail: both producers +/// present, both consumers exchangeable, and every donated consumer unconsumed since its +/// generation start (it has never begun a block), so every acked chunk, the generation and any +/// held seek are still in its ring. +/// +/// A caller runs it before the protocol commit, and [`apply_donation`] after. +pub fn check_donation( + successor: DonationSide<'_>, + donor: DonationSide<'_>, +) -> Result { + fn declarations(model: &SessionModel) -> BTreeMap<&str, &session::Source> { + model + .sources + .iter() + .map(|source| (source.id.as_str(), source)) + .collect() + } + let successor_declarations = declarations(successor.model); + let donor_declarations = declarations(donor.model); + let mut producers = Vec::new(); + let mut consumers = Vec::new(); + for row in successor.inventory.sources() { + let Some(given) = donor.inventory.source(&row.id) else { + continue; + }; + let fresh = + |side: &DonationSide<'_>, index: u32| side.plan.can_prepare_source_seek(index as usize); + if !fresh(&successor, row.index) + || !fresh(&donor, given.index) + || row.ring != given.ring + || successor_declarations.get(&*row.id) != donor_declarations.get(&*row.id) + || !successor_declarations.contains_key(&*row.id) + { + continue; + } + let (Some(into), Some(from)) = ( + successor.sources.index_of(row.id.as_bytes()), + donor.sources.index_of(given.id.as_bytes()), + ) else { + return Err(DonationError::Unswappable); + }; + let (Some(fresh_producer), Some(_)) = ( + successor.sources.sources[into].provider.as_ref(), + donor.sources.sources[from].provider.as_ref(), + ) else { + return Err(DonationError::Unswappable); + }; + if !unfed(fresh_producer) { + return Err(DonationError::SuccessorFed); + } + producers.push((into, from)); + consumers.push((row.index, given.index)); + } + graph::can_donate_sources(successor.plan, donor.plan, &consumers).map_err(|refusal| { + match refusal { + graph::GraphDonationRefusal::Consumed => DonationError::Consumed, + graph::GraphDonationRefusal::NotAGraphPlan | graph::GraphDonationRefusal::Mismatch => { + DonationError::Unswappable + } + } + })?; + Ok(DonationPlan { + producers: ProducerDonation(producers.into_boxed_slice()), + consumers: ConsumerDonation(consumers.into_boxed_slice()), + }) +} + +/// Whether a replacement would drop PCM the host was acked for on a ring the withdrawn plan +/// `donor` created (#1310, root rulings of 2026-10-16 on the acked-batch rule, M-A of fold 3). +/// +/// True when some source X of `donor` meets all three: +/// +/// * **Fresh in the donor.** The donor's plan owns X's ring (`can_prepare_source_seek` on X's +/// inventory index): the donor added X, or redeclared it against the plan it was prepared to +/// displace. A source the donor carries unchanged is prepared vacant (its ring is the running +/// plan's, carried at the swap), so it is never fresh here: its chunks are in the ring the +/// running plan renders, and the replacement treats it as a direct swap from the running plan +/// would. This is the same freshness test `check_donation` applies to the donor. +/// * **Fed.** The donor's producer for X has acked anything since preparation: a chunk, a seek or +/// an end of region (it is no longer `unfed`). +/// * **Not donated.** `donation`, the plan `check_donation` returned for this donor and the +/// replacement, has no pair whose donor index is X's. So the replacement does not keep X as the +/// donor declares it: it reverts X to the running plan's declaration, redeclares it another way, +/// or removes it. The donor's ring and its acked chunks would drop with the donor. +/// +/// The caller refuses the replacement before its commit as retryable backpressure: once render +/// adopts the donor, X's ring is the running plan's and the same edit commits on it. Read-only; +/// it allocates nothing. Issue #1698 lifts the reverted case only (it carries X from the running +/// plan); the others stay refused until an issue gives the replacement a way to read those chunks +/// or the owner rules that a removed source's chunks may drop. +#[must_use] +pub fn drops_acked_fresh_source(donor: DonationSide<'_>, donation: &DonationPlan) -> bool { + donor.inventory.sources().iter().any(|row| { + donor.plan.can_prepare_source_seek(row.index as usize) + && !donation + .consumers + .0 + .iter() + .any(|&(_, from)| from == row.index) + && donor + .sources + .index_of(row.id.as_bytes()) + .and_then(|index| donor.sources.sources[index].provider.as_ref()) + .is_some_and(|provider| !unfed(provider)) + }) +} + +/// Whether a fresh producer is exactly as preparation made it: generation 1 (the prepared initial +/// generation), nothing written, no end of region. Such a producer has acked nothing, so dropping +/// it (with the ring it feeds) loses nothing the host is owed. +fn unfed(provider: &HostChunkProvider) -> bool { + let telemetry = provider.telemetry(); + telemetry.active_generation == SourceGeneration(1) + && telemetry.cumulative_written_frames == 0 + && !telemetry.end_of_region_submitted +} + +/// Whether every producer pair resolves to an occupied entry on both sides and the successor's +/// producer is still [`unfed`]: the producer half's precondition, re-read by [`apply_donation`] +/// before anything moves. A successor producer fed since [`check_donation`] has acked PCM into a +/// ring the donation would hand to the donor, so it refuses the whole donation. +fn producers_resolve( + successor: &SourceControlSet, + donor: &SourceControlSet, + producers: &ProducerDonation, +) -> bool { + producers.0.iter().all(|&(into, from)| { + successor + .sources + .get(into) + .is_some_and(|source| source.provider.as_ref().is_some_and(unfed)) + && donor + .sources + .get(from) + .is_some_and(|source| source.provider.is_some()) + }) +} + +/// The producer half of [`apply_donation`] (#1344 D3): move each checked pair's donor producer +/// into the successor's entry, so the successor feeds the donated ring. The donor's entry is left +/// vacant, so a later submit or seek to the donor is refused with [`SourceControlError::Vacated`] +/// and never acked into a ring nobody plays. The successor's own fresh producer, which +/// [`check_donation`] proved unfed, is dropped; its ring moves to the donor with +/// [`donate_consumers`] and drops with it. Infallible on a plan [`check_donation`] returned for +/// these sets; allocates nothing. Returns how many moved. Crate-private: only [`apply_donation`] +/// runs it, after re-reading its precondition. +pub(crate) fn donate_producers( + successor: &mut SourceControlSet, + donor: &mut SourceControlSet, + producers: ProducerDonation, +) -> usize { + let mut moved = 0; + for &(into, from) in &producers.0 { + if let (Some(into), Some(from)) = + (successor.sources.get_mut(into), donor.sources.get_mut(from)) + && let Some(provider) = from.provider.take() + { + debug_assert!( + into.provider.as_ref().is_some_and(unfed), + "a checked successor producer is unfed" + ); + into.provider = Some(provider); + moved += 1; + } + } + debug_assert_eq!(moved, producers.0.len(), "a checked producer pair resolves"); + moved +} + +/// The consumer half of [`apply_donation`] (#1344 D3, D4): swap each checked pair's whole source +/// entries between the two plans (`graph::donate_sources`), ring, queued blocks, generation, read +/// position and held seek. Infallible on a plan [`check_donation`] returned for these plans; on +/// the control thread, on two plans neither of which render owns. Returns how many moved. +/// Crate-private: only [`apply_donation`] runs it, after re-reading its precondition. +pub(crate) fn donate_consumers( + successor: &mut PreparedRenderPlan, + donor: &mut PreparedRenderPlan, + consumers: ConsumerDonation, +) -> usize { + if consumers.0.is_empty() { + return 0; + } + let moved = graph::donate_sources(successor, donor, &consumers.0); + debug_assert!(moved, "a checked consumer donation swaps"); + if moved { consumers.0.len() } else { 0 } +} + +/// Apply a donation [`check_donation`] returned (#1344 D3). Returns how many sources moved. The +/// donor ends with the successor's fresh, unfed rings and drops them with itself; its donated +/// producer entries are vacant. +/// +/// Neither side can have changed since the check: the successor is unpublished and the donor +/// withdrawn, both held by the control thread. Call it after the protocol commit, when the +/// donor is never republished. +/// +/// The order is chosen so that no failure can lose PCM the host was acked for, even if a caller +/// breaks that contract (passes sets or plans other than the ones checked, or feeds a successor +/// producer between the check and the apply). First it re-reads, without moving anything, both +/// halves' preconditions: every producer pair resolves, every successor producer is still unfed, +/// and the consumer pairs pass [`graph::can_donate_sources`]. If any fails, nothing moves and it +/// returns `0`: the donor keeps its fed rings and their producers, together, and the successor +/// keeps its own. Then +/// `donate_consumers`, which is all or nothing; only if it moved does `donate_producers` run. +/// So a donated ring and its producer are never separated. +pub fn apply_donation( + successor: DonationSideMut<'_>, + donor: DonationSideMut<'_>, + plan: DonationPlan, +) -> usize { + let DonationPlan { + producers, + consumers, + } = plan; + let checked = producers.0.len() == consumers.0.len() + && producers_resolve(successor.sources, donor.sources, &producers) + && graph::can_donate_sources(successor.plan, donor.plan, &consumers.0).is_ok(); + if !checked { + return 0; + } + let consumers = donate_consumers(successor.plan, donor.plan, consumers); + if consumers == 0 { + return 0; + } + let producers = donate_producers(successor.sources, donor.sources, producers); + debug_assert_eq!(producers, consumers, "both halves move every pair"); + consumers +} diff --git a/crates/host-core/tests/withdrawn_successor.rs b/crates/host-core/tests/withdrawn_successor.rs new file mode 100644 index 000000000..9b71b4663 --- /dev/null +++ b/crates/host-core/tests/withdrawn_successor.rs @@ -0,0 +1,1279 @@ +//! Issue #1344: prepare a successor across a withdrawn candidate plan. +//! +//! P0 is the plan render runs. A was prepared against P0, committed and taken back before render +//! adopted it. B replaces A: it is prepared against P0 with A's `carried_base` and P0's committed +//! model (D2), and takes A's fresh rings by a checked donation (D3-D5). +//! +//! The twin of every case is the path render would have taken had A not been withdrawn: A' +//! (prepared against a second copy P0' of P0, rendered identically) adopts P0' render-free, B' is +//! prepared against A' normally and adopts A'. B adopting P0 must render what B' renders. +//! +//! * **Gate 1, base across a withdrawn plan.** P0 has sources `a`, `b` and tracks `t1`-`t3` on +//! `a` (`tb` reads `b`); `t3`'s transient-shaper insert and every input section carry audible +//! state. Plans are prepared with live controls, so inserts bank with live lanes and can carry. +//! A replaces `t1`'s insert, removes `b`, adds `c`; B replaces `t2`'s (the spec's +//! insert-quality edit does not prepare: no launch effect has a second quality). B renders +//! bit-identically to B' over 16 blocks, with `a` and `t3` carried, `t1` and `t2` restarted and +//! `b` absent; B prepared with no base does not. (An owner A restarts and B reverts to its P0 +//! form starts fresh here, as in the sequential twin; root ruled that a defect and its fix is a +//! successor issue, so no assertion pins it either way.) +//! * **Gate 2, `carried_base` follows the installed hand-over.** Over a session with a row in +//! every inventory family, where A drops a strip and a source that sort first (so A's keys are +//! not P0's) and restarts what a session edit can restart, A's base holds exactly the P0 rows +//! whose keys A's hand-over reads (set equality of keys), and they belong to exactly the owners +//! both plans hold less those A restarted; a plan with no base has none. +//! * **Gate 3, a source absent from P0 is never carried from A.** A adds `d` with 4 seconds, B with +//! 5: B's `d` is a fresh ring at generation 1, frame 0, and no donation applies. +//! * **Gate 4, donation.** `c` fed through A's producers and seeked (anchored) moves to B whole: +//! B plays it as B' does, and without the donation it would not. +//! * **Gate 5, refusals move nothing.** A donor whose `c` consumer began a block is refused with +//! `Consumed`; a redeclared or re-sized `c` is never donated. Nothing moves either way. + +#![cfg(all(feature = "test-support", feature = "control-provider"))] + +#[path = "support/successor.rs"] +mod successor; + +use bench_support::alloc as bench_alloc; +use engine::realtime::{CarryOutcome, PlanarBufferMut, PreparedRenderPlan, RenderIo}; +use graph_compiler::Backend; +use host_core::{ + DonationError, DonationSide, DonationSideMut, HostLiveControlRequest, HostLiveLanes, + LiveValuesSuperseded, PlanStateInventory, PreparedHost, SourceControlSet, SourceSubmission, + SpectrumCaptureRequest, SpectrumChannels, SpectrumTarget, SuccessorBase, apply_donation, + check_donation, drops_acked_fresh_source, prepare_host_runtime_with_live_controls_and_spectrum, + prepare_host_runtime_with_live_controls_and_spectrum_successor, spectrum_capture_resources_for, + test_only_carry_read_keys, test_only_inventory_rows, +}; +use session::{ + DualMonoFader, Effect, EffectIdentity, EffectQuality, LinkMode, RouteDestination, RouteSource, + SendTap, SessionModel, SidechainDeclaration, Source, StableId, Submix, parse_session_json, +}; +use std::collections::BTreeSet; +use successor::{QUANTUM, SUCCESSOR_REVISION, Session, caps}; + +const FIXTURE: &str = include_str!("../../../fixtures/session/v1/parametric-eq-nine-track.json"); +/// Blocks P0 renders before the swap. +const BEFORE: usize = 4; +/// Blocks B renders after it. +const AFTER: usize = 16; +/// The absolute sample of B's first block. +const SWAP_SAMPLE: u64 = (BEFORE * QUANTUM) as u64; + +fn id(value: &str) -> StableId { + StableId::parse(value).expect("stable ID") +} + +fn compressor(instance: &str) -> Effect { + Effect { + id: id(instance), + identity: EffectIdentity::Native { + effect_id: id("miso.compressor"), + }, + quality: EffectQuality::Normal, + bypass: false, + link_mode: LinkMode::DualMono, + params: Vec::new(), + sidechain: SidechainDeclaration::None, + } +} + +/// A source declared like the fixture's, under `source`, with its own content and `frames`. +fn source(template: &Source, source: &str, frames: u64) -> Source { + Source { + id: id(source), + content: template + .content + .replace("7e94", &format!("{:0>4}", &source[..source.len().min(4)])), + frames, + ..template.clone() + } +} + +/// Add track `track` on `source`, a copy of the template track with its own route to the output +/// and, when `insert`, a compressor insert. +fn add_track( + model: &mut SessionModel, + template: &SessionModel, + track: &str, + source: &str, + insert: bool, +) { + let mut added = template.tracks[0].clone(); + added.id = id(track); + added.source_id = id(source); + added.inserts.effects = if insert { + vec![compressor("comp")] + } else { + Vec::new() + }; + model.tracks.push(added); + let mut route = template.routes[0].clone(); + route.id = id(&format!("{track}-main")); + route.source = RouteSource::Track { + track_id: id(track), + tap: SendTap::PostPan, + }; + model.routes.push(route); +} + +fn remove_track(model: &mut SessionModel, track: &str) { + model + .tracks + .retain(|existing| existing.id.as_str() != track); + model + .routes + .retain(|route| route.id.as_str() != format!("{track}-main")); +} + +/// Replace `track`'s insert with another effect under the same insert ID: a prepared change that +/// restarts the insert (no launch effect declares a second quality, so a quality edit does not +/// prepare). +fn replace_insert(model: &mut SessionModel, track: &str) { + let track = model + .tracks + .iter_mut() + .find(|existing| existing.id.as_str() == track) + .expect("the track"); + track.inserts.effects[0].identity = EffectIdentity::Native { + effect_id: id("miso.gate-expander"), + }; +} + +/// The fixture with no console slot, so its sessions prepare at four lanes on an eight-lane +/// target too. +fn template() -> SessionModel { + let mut template = parse_session_json(FIXTURE).expect("fixture parses"); + template.console.pre_insert.clear(); + for track in &mut template.tracks { + track.console.clear(); + } + template +} + +/// P0: sources `a` and `b`; tracks `t1`, `t2`, `t3` on `a` and `tb` on `b`; `t1` and `t2` each +/// with a compressor insert, and `t3` with a transient shaper: an insert with audible state that +/// every plan here carries, in a bank of its own so that `t1`'s and `t2`'s edits leave its layout +/// alone (batch 15a fold, M3); no console slot. +fn p0_model() -> SessionModel { + let template = template(); + let mut model = template.clone(); + model.tracks.clear(); + model.routes.clear(); + model.sources = vec![ + source(&template.sources[0], "a", 48_000), + source(&template.sources[0], "b", 48_000), + ]; + for (track, insert) in [("t1", true), ("t2", true), ("t3", true)] { + add_track(&mut model, &template, track, "a", insert); + } + model.tracks[2].inserts.effects[0].identity = EffectIdentity::Native { + effect_id: id("miso.transient-shaper"), + }; + add_track(&mut model, &template, "tb", "b", false); + model +} + +/// A: P0 with `t1`'s insert replaced, `b` and `tb` removed, and `c` added with track `tc`. +fn a_model() -> SessionModel { + let template = template(); + let mut model = p0_model(); + replace_insert(&mut model, "t1"); + remove_track(&mut model, "tb"); + model.sources.retain(|source| source.id.as_str() != "b"); + model + .sources + .push(source(&template.sources[0], "c", 48_000)); + add_track(&mut model, &template, "tc", "c", false); + model +} + +/// B: A with `t2`'s insert replaced too. +fn b_model() -> SessionModel { + let mut model = a_model(); + replace_insert(&mut model, "t2"); + model +} + +fn backends() -> Vec { + vec![ + #[cfg(target_feature = "avx2")] + Backend::Simd8, + Backend::Simd4, + ] +} + +/// One stereo PCM signal per source: no exact-zero sample, distinct per `seed`. +fn planes(seed: u32, frames: usize) -> [Vec; 2] { + let plane = |channel: u32| { + (0..frames as u32) + .map(|frame| { + let step = frame + .wrapping_mul(7_919) + .wrapping_add(seed.wrapping_mul(104_729)) + .wrapping_add(channel * 31) + % 1_999; + (step as f32 - 999.5) / 2_000.0 + }) + .collect::>() + }; + [plane(0), plane(1)] +} + +/// Submit one block of `signal` to `source` through `sources`: generation `generation`, source +/// frames `block * QUANTUM..`. +fn submit( + sources: &mut SourceControlSet, + source: &str, + signal: &[Vec; 2], + generation: u64, + block: usize, +) -> Result<(), host_core::SourceControlError> { + let range = block * QUANTUM..(block + 1) * QUANTUM; + sources + .submit( + source.as_bytes(), + SourceSubmission { + generation, + start_frame: range.start as u64, + sample_rate_hz: 48_000, + planes: &[&signal[0][range.clone()], &signal[1][range]], + frames: QUANTUM as u32, + end_of_region: false, + }, + ) + .map(|_| ()) +} + +/// One rendered block's bits, and the allocator calls `(allocations, reallocations, +/// deallocations)` the render made on this thread. +fn render(plan: &mut PreparedRenderPlan) -> (Vec, (u64, u64, u64)) { + let mut output = [f32::NAN; QUANTUM * 2]; + let sample = plan.next_absolute_sample(); + let mark = bench_alloc::current_thread_counters(); + plan.render_contiguous( + RenderIo { + output: PlanarBufferMut::try_new(&mut output, 2, QUANTUM, QUANTUM).expect("output"), + }, + sample, + ) + .unwrap_or_else(|error| panic!("render: {error:?}")); + let delta = bench_alloc::current_thread_delta_since(mark); + ( + output.iter().map(|sample| sample.to_bits()).collect(), + (delta.allocations, delta.reallocations, delta.deallocations), + ) +} + +/// `session` prepared against `inventory` and `committed` at `backend`, with the neutral base +/// values of the amendment. +fn successor_of( + session: &Session, + inventory: &PlanStateInventory, + committed: &SessionModel, + backend: Backend, +) -> PreparedHost { + host_core::test_only_prepare_host_runtime_with_live_controls_successor_on( + &session.compiled, + &caps(), + &HostLiveControlRequest::default(), + SuccessorBase { + inventory, + committed, + revision: SUCCESSOR_REVISION, + edit_ramps: &session::EditRamps::new(), + monitor: host_core::MonitorBase::NONE, + observation: &host_core::NO_OBSERVATION, + }, + backend, + ) + .map(|(host, _)| host) + .unwrap_or_else(|failure| panic!("successor: {failure:?}")) +} + +/// `session` prepared fresh at `backend`, with live controls: its effects bank with live lanes, +/// so an insert's state can carry. +fn fresh(session: &Session, backend: Backend) -> PreparedHost { + host_core::test_only_prepare_host_runtime_with_live_controls_on( + &session.compiled, + &caps(), + &HostLiveControlRequest::default(), + backend, + ) + .map(|(host, _)| host) + .unwrap_or_else(|failure| panic!("prepare: {failure:?}")) +} + +/// Publish the revision the successor's carry retargets were stamped with on its own gate, as a +/// host's commit does before the successor renders. +fn commit(host: &mut PreparedHost) { + host.plan + .take_revision_gate() + .expect("the successor's gate handle") + .publish(SUCCESSOR_REVISION); +} + +/// P0 prepared and rendered [`BEFORE`] blocks with `a` and `b` fed one block ahead. +fn running_p0(p0: &Session, backend: Backend, signal: &[[Vec; 2]; 2]) -> PreparedHost { + let mut host = fresh(p0, backend); + for block in 0..=BEFORE { + submit(&mut host.sources, "a", &signal[0], 1, block).expect("feed a"); + submit(&mut host.sources, "b", &signal[1], 1, block).expect("feed b"); + } + for _ in 0..BEFORE { + render(&mut host.plan); + } + host +} + +/// What the host does to `c` on the withdrawn candidate's producers before B is prepared (gate +/// 4): four blocks of generation 1, then an anchored seek to generation 2 at frame 0 two blocks +/// after the swap, with that generation's first [`AFTER`] blocks. +fn feed_c(sources: &mut SourceControlSet, signal: &[[Vec; 2]; 2]) { + for block in 0..4 { + submit(sources, "c", &signal[0], 1, block).expect("feed c, generation 1"); + } + sources + .seek_at(b"c", 2, 0, SWAP_SAMPLE + 2 * QUANTUM as u64) + .expect("anchored seek on c"); + for block in 0..AFTER { + submit(sources, "c", &signal[1], 2, block).expect("feed c, generation 2"); + } +} + +/// The twin: A' adopts P0' render-free, B' is prepared against A' and adopts it, every producer +/// following its ring; then B' renders [`AFTER`] blocks with `a` fed one block ahead. +fn twin_run( + sessions: &[Session; 3], + backend: Backend, + signal: &[[Vec; 2]; 2], + c_signal: Option<&[[Vec; 2]; 2]>, +) -> Vec> { + let [p0, a, b] = sessions; + let mut p0_twin = running_p0(p0, backend, signal); + let mut a_twin = successor_of(a, &p0_twin.inventory, &p0.model, backend); + commit(&mut a_twin); + a_twin.sources.adopt_persisting(&mut p0_twin.sources); + assert_eq!( + a_twin.plan.adopt_predecessor_plan(&mut p0_twin.plan), + CarryOutcome::Carried + ); + if let Some(c_signal) = c_signal { + feed_c(&mut a_twin.sources, c_signal); + } + let mut b_twin = successor_of(b, &a_twin.inventory, &a.model, backend); + commit(&mut b_twin); + b_twin.sources.adopt_persisting(&mut a_twin.sources); + assert_eq!( + b_twin.plan.adopt_predecessor_plan(&mut a_twin.plan), + CarryOutcome::Carried + ); + play(&mut b_twin, signal) +} + +/// Render [`AFTER`] blocks of `host`, feeding `a` one block ahead; every block allocates and +/// frees nothing. +fn play(host: &mut PreparedHost, signal: &[[Vec; 2]; 2]) -> Vec> { + bench_alloc::assert_installed(); + (0..AFTER) + .map(|block| { + submit(&mut host.sources, "a", &signal[0], 1, BEFORE + block + 1).expect("feed a"); + let (bits, allocator) = render(&mut host.plan); + assert_eq!(allocator, (0, 0, 0), "block {block}: allocator calls"); + bits + }) + .collect() +} + +fn sessions() -> [Session; 3] { + [ + Session::compile(p0_model()), + Session::compile(a_model()), + Session::compile(b_model()), + ] +} + +fn signal() -> [[Vec; 2]; 2] { + let frames = (BEFORE + AFTER + 2) * QUANTUM; + [planes(1, frames), planes(2, frames)] +} + +/// Gate 1 at one width. +fn base_across_a_withdrawn_plan(backend: Backend) { + let sessions = sessions(); + let [p0, a, b] = &sessions; + let signal = signal(); + let twin = twin_run(&sessions, backend, &signal, None); + + let mut running = running_p0(p0, backend, &signal); + // A: prepared against P0, committed and taken back; render never adopts it. + let withdrawn = successor_of(a, &running.inventory, &p0.model, backend); + let base = withdrawn.carried_base.as_ref().expect("A has a base"); + let mut replacement = successor_of(b, base, &p0.model, backend); + commit(&mut replacement); + // `a` carries from P0: its producer moves, and its ring moves at adoption. + assert_eq!( + replacement.sources.adopt_persisting(&mut running.sources), + 1 + ); + let a_index = running.inventory.source("a").expect("P0 has a").index; + assert_eq!( + graph::installed_carry_reads(&replacement.plan) + .expect("B has a hand-over") + .sources, + vec![a_index], + "B carries a's ring from P0" + ); + assert!(replacement.inventory.source("b").is_none(), "b is absent"); + assert_eq!( + replacement.restarted_strips(), + [Box::::from("t1"), Box::from("t2")], + "t1 and t2 restart" + ); + assert_eq!( + replacement.plan.adopt_predecessor_plan(&mut running.plan), + CarryOutcome::Carried + ); + let blocks = play(&mut replacement, &signal); + for (block, (left, right)) in blocks.iter().zip(&twin).enumerate() { + assert_eq!(left, right, "{backend:?} block {block}: B equals B'"); + } + + // Not vacuous: B prepared with no base starts `a`, `t3` and every carried owner fresh. + let mut running = running_p0(p0, backend, &signal); + let mut fresh = self::fresh(b, backend); + for block in 0..=BEFORE { + submit(&mut fresh.sources, "a", &signal[0], 1, block).expect("feed a"); + } + assert_eq!( + fresh.plan.adopt_predecessor_plan(&mut running.plan), + CarryOutcome::NotRequested + ); + let fresh_blocks = play(&mut fresh, &signal); + assert_ne!(fresh_blocks, twin, "{backend:?}: carried state is audible"); +} + +#[test] +fn a_successor_prepared_across_a_withdrawn_plan_renders_as_its_twin() { + for backend in backends() { + base_across_a_withdrawn_plan(backend); + } +} + +/// Gate 2's session: three sources (`a`, `b`, `c`), sends into a submix (live routes), a true-peak limiter on +/// `t0001`, whose latency the other paths compensate (compensation lines), strip delays on `t` and +/// `t0002`, a compressor on `t`, `t0000` and `t0001` (effect lanes and observation taps), a console +/// slot on every strip, meters, and a spectrum capture on `t0000`. Track `t` and its source `a` +/// sort first, so a plan without them has other keys for every row after them. +fn rich_model() -> SessionModel { + let template = parse_session_json(FIXTURE).expect("fixture parses"); + let mut model = template.clone(); + model.tracks.clear(); + model.routes.clear(); + model.sources = vec![ + source(&template.sources[0], "a", 48_000), + source(&template.sources[0], "b", 48_000), + source(&template.sources[0], "c", 48_000), + ]; + for (track, source) in [ + ("t", "a"), + ("t0000", "b"), + ("t0001", "b"), + ("t0002", "c"), + ("t0003", "b"), + ] { + add_track(&mut model, &template, track, source, true); + } + for track in &mut model.tracks { + if track.id.as_str() == "t0001" { + track.inserts.effects.push(Effect { + id: id("limiter"), + identity: EffectIdentity::Native { + effect_id: id("miso.true-peak-limiter"), + }, + ..compressor("limiter") + }); + } + if matches!(track.id.as_str(), "t0002" | "t0003") { + track.inserts.effects.clear(); + } + if matches!(track.id.as_str(), "t" | "t0002") { + track.builtins.left.delay_samples = 64; + track.builtins.right.delay_samples = 32; + } + } + let bus = id("bus"); + model + .submixes + .push(Submix::unity(bus.clone(), &model.console)); + for track in ["t", "t0000", "t0001", "t0003"] { + let mut send = model.routes[0].clone(); + send.id = id(&format!("{track}-bus")); + send.source = RouteSource::Track { + track_id: id(track), + tap: SendTap::PreFader, + }; + send.destination = RouteDestination::SubmixInput { + submix_id: bus.clone(), + }; + model.routes.push(send); + } + let mut out = model.routes[0].clone(); + out.id = id("bus-main"); + out.source = RouteSource::Submix { + submix_id: bus, + tap: SendTap::PostPan, + }; + model.routes.push(out); + // A fader of its own on `t0003`, so the strips' values are not all equal. + model.tracks[4].fader = DualMonoFader { + left_db: -6.0, + right_db: -6.0, + left_mute: false, + right_mute: false, + }; + model +} + +/// Gate 2's A: P0 without track `t` (and its routes) and source `a`, which sort first, so every +/// family loses `t`'s (or `a`'s) row; and a restart where a session edit restarts an owner that +/// stays: `c` redeclared (sources) and `t0000`'s insert replaced (its effect lane and observation +/// taps). Under live input lanes an input-section or strip-stage value edit carries and +/// retargets, so those families restart only by removal; the plan's one spectrum capture carries, +/// since restarting it would leave the family unread. +fn rich_a_model() -> SessionModel { + let mut model = rich_model(); + remove_track(&mut model, "t"); + model.routes.retain(|route| route.id.as_str() != "t-bus"); + model.sources.retain(|source| source.id.as_str() != "a"); + for source in &mut model.sources { + if source.id.as_str() == "c" { + source.frames = 2 * 48_000; + } + } + replace_insert(&mut model, "t0000"); + model +} + +fn live_request() -> HostLiveControlRequest { + HostLiveControlRequest { + meter_period_frames: core::num::NonZeroU32::new(QUANTUM as u32), + observation_taps: 1, + ..HostLiveControlRequest::default() + } +} + +fn spectrum_request() -> SpectrumCaptureRequest { + let target = SpectrumTarget::TrackPostMatrix("t0000".into()); + SpectrumCaptureRequest { + maximum_capture_bytes: spectrum_capture_resources_for(&target).retained_bytes, + target, + channels: SpectrumChannels::Stereo, + } +} + +fn prepare_rich(session: &Session, base: Option>) -> PreparedHost { + let live = live_request(); + let superseded = LiveValuesSuperseded::default(); + let spectrum = spectrum_request(); + match base { + None => prepare_host_runtime_with_live_controls_and_spectrum( + &session.compiled, + &caps(), + &live, + &spectrum, + HostLiveLanes::ALL, + &superseded, + ), + Some(base) => prepare_host_runtime_with_live_controls_and_spectrum_successor( + &session.compiled, + &caps(), + &live, + &spectrum, + HostLiveLanes::ALL, + &superseded, + base, + ), + } + .map(|(host, _, _)| host) + .unwrap_or_else(|failure| panic!("prepare: {failure:?}")) +} + +/// `(family, owner)` of every row of `rows`. +fn owners(rows: &BTreeSet<(&'static str, String, String)>) -> BTreeSet<(&'static str, String)> { + rows.iter() + .map(|(family, owner, _)| (*family, owner.clone())) + .collect() +} + +/// `(family, key)` of every row of `rows`. +fn keys(rows: &BTreeSet<(&'static str, String, String)>) -> BTreeSet<(&'static str, String)> { + rows.iter() + .map(|(family, _, key)| (*family, key.clone())) + .collect() +} + +/// Gate 2, tightened by the batch 15a fold (M3). Red if the restriction keeps a row the hand-over +/// does not read (an unfiltered `restricted_to`: the removed `t`, `a` and every restarted +/// owner would stay), or if `installed_carry_reads` reports successor keys for predecessor keys +/// (A lacks `t` and `a`, which sort first, so A's keys name other P0 rows: the owners differ). +#[test] +fn the_carried_base_holds_exactly_the_rows_the_hand_over_reads() { + let p0 = Session::compile(rich_model()); + let a = Session::compile(rich_a_model()); + let running = prepare_rich(&p0, None); + assert!( + running.carried_base.is_none(), + "a plan with no base has none" + ); + assert_eq!(running.report.carried_base_retained_bytes, 0); + let withdrawn = prepare_rich( + &a, + Some(SuccessorBase { + inventory: &running.inventory, + committed: &p0.model, + revision: SUCCESSOR_REVISION, + edit_ramps: &session::EditRamps::new(), + monitor: host_core::MonitorBase::NONE, + observation: &host_core::NO_OBSERVATION, + }), + ); + let base = withdrawn.carried_base.as_ref().expect("A has a base"); + assert_eq!(base.plan_identity(), running.inventory.plan_identity()); + assert_eq!( + withdrawn.report.carried_base_retained_bytes, + base.retained_bytes() + ); + assert!(base.spectrum_pairing().is_empty()); + let reads = graph::installed_carry_reads(&withdrawn.plan).expect("A has a hand-over"); + assert_eq!(reads.predecessor, running.inventory.plan_identity()); + + let p0_rows = test_only_inventory_rows(&running.inventory); + let a_rows = test_only_inventory_rows(&withdrawn.inventory); + let base_rows = test_only_inventory_rows(base); + let read_keys = test_only_carry_read_keys(&reads); + // Every family is exercised by the hand-over. + for family in [ + "sources", + "input sections", + "strip stages", + "effect lanes", + "meters", + "observations", + "spectrum", + "delay lines", + "strip delays", + "live routes", + ] { + assert!( + read_keys.iter().any(|(read, _)| *read == family), + "{family}: the session exercises the family" + ); + } + // Every base row is P0's row, unchanged. + assert!(base_rows.is_subset(&p0_rows), "base rows are P0 rows"); + // The base holds exactly the keys the hand-over reads: no more, no fewer. + assert_eq!(keys(&base_rows), read_keys, "base keys equal read keys"); + // Not vacuous: A's own keys differ from P0's, so a successor-key read names other rows. + assert_ne!(keys(&a_rows), keys(&p0_rows), "A's keys are not P0's"); + // The owners: those both plans hold, less those A restarted. Independent of keys. + let restarted = |(family, owner): &(&'static str, String)| match *family { + "sources" => owner == "c", + "effect lanes" | "observations" => owner.starts_with("t0000 ") && owner.ends_with(" comp"), + _ => false, + }; + let expected: BTreeSet<_> = owners(&p0_rows) + .intersection(&owners(&a_rows)) + .filter(|owner| !restarted(owner)) + .cloned() + .collect(); + assert_eq!( + owners(&base_rows), + expected, + "base owners are A's carried owners" + ); + for owner in owners(&p0_rows) { + if restarted(&owner) || !owners(&a_rows).contains(&owner) { + assert!( + !owners(&base_rows).contains(&owner), + "{owner:?}: removed or restarted, so not in the base" + ); + } + } +} + +/// Gate 3: P0 lacks `d`; A adds it with 4 seconds of frames, B with 5. +#[test] +fn a_source_absent_from_the_running_plan_is_never_carried_from_the_withdrawn_one() { + let template = template(); + let with_d = |frames: u64| { + let mut model = a_model(); + model + .sources + .push(source(&template.sources[0], "d", frames)); + add_track(&mut model, &template, "td", "d", false); + model + }; + let p0 = Session::compile(p0_model()); + let a = Session::compile(with_d(4 * 48_000)); + let b = Session::compile(with_d(5 * 48_000)); + let backend = Backend::current(); + let signal = signal(); + let running = running_p0(&p0, backend, &signal); + let mut withdrawn = successor_of(&a, &running.inventory, &p0.model, backend); + // The host fed A's `d` and was acked. + for block in 0..2 { + submit(&mut withdrawn.sources, "d", &signal[0], 1, block).expect("feed A's d"); + } + let base = withdrawn.carried_base.as_ref().expect("A has a base"); + assert!( + base.source("d").is_none(), + "P0 has no d, so A's base has none" + ); + let mut replacement = successor_of(&b, base, &p0.model, backend); + let d_index = replacement.inventory.source("d").expect("B has d").index; + assert_eq!( + graph::installed_carry_reads(&replacement.plan) + .expect("B carries a") + .sources, + vec![running.inventory.source("a").expect("P0 has a").index], + "B carries a from P0 and nothing for d" + ); + assert!( + replacement.plan.can_prepare_source_seek(d_index as usize), + "B's d owns a ring" + ); + // No donation applies: d was redeclared. + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + let plan = check_donation(into, from).expect("the check passes"); + // The check has no side effect: run again, it finds the same. + assert_eq!(Ok(&plan), check_donation(into, from).as_ref()); + // Only `c` (unchanged from A) is donated; `d` is not. + assert_eq!(plan.len(), 1); + // B's `d` is a fresh ring at generation 1, frame 0: A's has frames 0..256 and would refuse a + // chunk that does not continue them. + submit(&mut replacement.sources, "d", &signal[1], 1, 0).expect("B's d starts at frame 0"); + assert!( + submit(&mut withdrawn.sources, "d", &signal[1], 1, 0).is_err(), + "A's d continues where the host left it" + ); +} + +/// Gate 4 at one width: `c` donated from the withdrawn A to B plays as B' plays it. +fn donation_plays_the_fed_ring(backend: Backend) { + let sessions = sessions(); + let [p0, a, b] = &sessions; + let signal = signal(); + let c_signal = [ + planes(3, (AFTER + 4) * QUANTUM), + planes(4, (AFTER + 4) * QUANTUM), + ]; + let twin = twin_run(&sessions, backend, &signal, Some(&c_signal)); + + let run = |donate: bool| { + let mut running = running_p0(p0, backend, &signal); + let mut withdrawn = successor_of(a, &running.inventory, &p0.model, backend); + feed_c(&mut withdrawn.sources, &c_signal); + let base = withdrawn.carried_base.clone().expect("A has a base"); + let mut replacement = successor_of(b, &base, &p0.model, backend); + commit(&mut replacement); + if donate { + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + let plan = check_donation(into, from).expect("the donation checks"); + assert_eq!(plan.len(), 1, "c is the one source both created fresh"); + assert_eq!( + apply_donation( + DonationSideMut { + plan: &mut replacement.plan, + sources: &mut replacement.sources, + }, + DonationSideMut { + plan: &mut withdrawn.plan, + sources: &mut withdrawn.sources, + }, + plan, + ), + 1 + ); + // The producer moved with its ring: the host continues generation 2 on B. + submit(&mut replacement.sources, "c", &c_signal[1], 2, AFTER) + .expect("B's c continues generation 2"); + // A's entry is vacant: a later submit to the donor is refused, never acked into + // the fresh ring A now holds and drops (#1344 fold m1). + assert_eq!( + submit(&mut withdrawn.sources, "c", &c_signal[1], 2, AFTER), + Err(host_core::SourceControlError::Vacated), + "the donor's c is vacant" + ); + } + replacement.sources.adopt_persisting(&mut running.sources); + assert_eq!( + replacement.plan.adopt_predecessor_plan(&mut running.plan), + CarryOutcome::Carried + ); + play(&mut replacement, &signal) + }; + let donated = run(true); + for (block, (left, right)) in donated.iter().zip(&twin).enumerate() { + assert_eq!( + left, right, + "{backend:?} block {block}: B plays c as B' does" + ); + } + assert_ne!( + run(false), + twin, + "{backend:?}: without the donation, c's acked PCM and seek are not heard" + ); +} + +#[test] +fn a_donated_ring_plays_its_acked_pcm_and_held_seek() { + for backend in backends() { + donation_plays_the_fed_ring(backend); + } +} + +/// The two sides of a donation from `withdrawn` (compiled from `donor_model`) to `replacement` +/// (compiled from `model`). +fn sides<'a>( + replacement: &'a PreparedHost, + model: &'a SessionModel, + withdrawn: &'a PreparedHost, + donor_model: &'a SessionModel, +) -> (DonationSide<'a>, DonationSide<'a>) { + ( + DonationSide { + plan: &replacement.plan, + sources: &replacement.sources, + inventory: &replacement.inventory, + model, + }, + DonationSide { + plan: &withdrawn.plan, + sources: &withdrawn.sources, + inventory: &withdrawn.inventory, + model: donor_model, + }, + ) +} + +/// What a refusal must leave alone: which of B's and A's `c` entries own a ring, and the +/// producers, read by feeding each set the chunk only its own ring accepts. +fn assert_untouched(replacement: &mut PreparedHost, withdrawn: &mut PreparedHost, what: &str) { + for (side, host) in [("B", &*replacement), ("A", &*withdrawn)] { + let index = host.inventory.source("c").expect("c").index; + assert!( + host.plan.can_prepare_source_seek(index as usize), + "{what}: {side}'s c owns its ring" + ); + } + let c_signal = planes(5, 8 * QUANTUM); + submit(&mut replacement.sources, "c", &c_signal, 1, 0) + .unwrap_or_else(|error| panic!("{what}: B keeps its fresh producer: {error:?}")); + submit(&mut withdrawn.sources, "c", &c_signal, 1, 4) + .unwrap_or_else(|error| panic!("{what}: A keeps its fed producer: {error:?}")); +} + +#[test] +fn a_refused_or_mismatched_donation_moves_nothing() { + let backend = Backend::current(); + let [p0, a, b] = &sessions(); + let signal = signal(); + let feed_four = |sources: &mut SourceControlSet| { + for block in 0..4 { + submit(sources, "c", &signal[0], 1, block).expect("feed c"); + } + }; + + // Consumed: a control-thread render of A began its `c` consumer. + let running = running_p0(p0, backend, &signal); + let mut withdrawn = successor_of(a, &running.inventory, &p0.model, backend); + feed_four(&mut withdrawn.sources); + render(&mut withdrawn.plan); + let base = withdrawn.carried_base.clone().expect("A has a base"); + let mut replacement = successor_of(b, &base, &p0.model, backend); + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + assert_eq!(check_donation(into, from), Err(DonationError::Consumed)); + assert_untouched(&mut replacement, &mut withdrawn, "consumed"); + + // A redeclared `c` (other content), and a `c` on a ring of another size: not donated. + let mut redeclared = b_model(); + let c = redeclared + .sources + .iter_mut() + .find(|source| source.id.as_str() == "c") + .expect("c"); + c.content = c.content.replace("0c", "0e"); + let redeclared = Session::compile(redeclared); + let mut small = caps(); + small.source_ring_frames /= 2; + for (what, session, caps) in [ + ("declaration", &redeclared, caps()), + ("ring configuration", b, small), + ] { + let running = running_p0(p0, backend, &signal); + let mut withdrawn = successor_of(a, &running.inventory, &p0.model, backend); + feed_four(&mut withdrawn.sources); + let base = withdrawn.carried_base.clone().expect("A has a base"); + let mut replacement = session + .prepare_successor(&base, &p0.model, &caps, backend) + .unwrap_or_else(|failure| panic!("{what}: successor: {failure:?}")); + let (into, from) = sides(&replacement, &session.model, &withdrawn, &a.model); + let plan = check_donation(into, from).expect("nothing to refuse"); + assert!(plan.is_empty(), "{what}: c is not donated"); + assert_eq!( + apply_donation( + DonationSideMut { + plan: &mut replacement.plan, + sources: &mut replacement.sources, + }, + DonationSideMut { + plan: &mut withdrawn.plan, + sources: &mut withdrawn.sources, + }, + plan, + ), + 0 + ); + assert_untouched(&mut replacement, &mut withdrawn, what); + } +} + +/// Fold m1 (#1344, batch 15a). Red on a check that passes a fed successor producer the apply +/// then drops, and on an apply that moves the producers before a consumer swap that fails (the +/// donor's fed producer would leave its ring). The swap the producer half used to make (the donor +/// kept B's fresh producer and acked into the ring it then dropped) is caught by gate 4 instead: +/// its assert that the donor's `c` is `Vacated` after the donation +/// (`a_donated_ring_plays_its_acked_pcm_and_held_seek`). +#[test] +fn a_donation_never_separates_an_acked_ring_from_its_producer() { + let backend = Backend::current(); + let [p0, a, b] = &sessions(); + let signal = signal(); + let prepare = || { + let running = running_p0(p0, backend, &signal); + let mut withdrawn = successor_of(a, &running.inventory, &p0.model, backend); + for block in 0..4 { + submit(&mut withdrawn.sources, "c", &signal[0], 1, block).expect("feed A's c"); + } + let base = withdrawn.carried_base.clone().expect("A has a base"); + let replacement = successor_of(b, &base, &p0.model, backend); + (withdrawn, replacement) + }; + + // A successor whose fresh `c` producer was fed is refused: the apply would drop it. + let (mut withdrawn, mut replacement) = prepare(); + submit(&mut replacement.sources, "c", &signal[1], 1, 0).expect("feed B's c by hand"); + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + assert_eq!(check_donation(into, from), Err(DonationError::SuccessorFed)); + submit(&mut replacement.sources, "c", &signal[1], 1, 1).expect("B keeps its producer"); + submit(&mut withdrawn.sources, "c", &signal[0], 1, 4).expect("A keeps its producer"); + + // A contract violation at apply: the checked plan is applied with a donor plan whose `c` is + // vacant (B' prepared against A carries it), so the consumer half cannot move. Nothing moves: + // A's fed producer stays with A's ring, and B keeps its fresh producer. + let (mut withdrawn, mut replacement) = prepare(); + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + let plan = check_donation(into, from).expect("the donation checks"); + assert_eq!(plan.len(), 1); + let mut vacant_c = successor_of(b, &withdrawn.inventory, &a.model, backend); + assert!( + !vacant_c + .plan + .can_prepare_source_seek(vacant_c.inventory.source("c").expect("c").index as usize), + "the stand-in donor's c is vacant" + ); + assert_eq!( + apply_donation( + DonationSideMut { + plan: &mut replacement.plan, + sources: &mut replacement.sources, + }, + DonationSideMut { + plan: &mut vacant_c.plan, + sources: &mut withdrawn.sources, + }, + plan, + ), + 0, + "a failing half moves nothing" + ); + assert_untouched( + &mut replacement, + &mut withdrawn, + "apply on another donor plan", + ); +} + +/// Two fresh sides of gate 4's donation: A fed four blocks of `c`, B prepared across it. +fn fed_withdrawn_and_replacement( + sessions: &[Session; 3], + backend: Backend, + signal: &[[Vec; 2]; 2], +) -> (PreparedHost, PreparedHost) { + let [p0, a, b] = sessions; + let running = running_p0(p0, backend, signal); + let mut withdrawn = successor_of(a, &running.inventory, &p0.model, backend); + for block in 0..4 { + submit(&mut withdrawn.sources, "c", &signal[0], 1, block).expect("feed A's c"); + } + let base = withdrawn.carried_base.clone().expect("A has a base"); + let replacement = successor_of(b, &base, &p0.model, backend); + (withdrawn, replacement) +} + +/// Batch 15a review fold 2, m-3: a successor producer that acked anything is fed, not only one +/// that acked PCM. A seek alone (the producer's generation moves off 1) and a seek to the region's +/// end closed by a zero-frame end-of-region chunk each make the check refuse with `SuccessorFed`. +/// +/// Test value: red if `unfed` reads only the written frames (either case would be donated and +/// its producer dropped with what it acked). +#[test] +fn a_seek_or_an_empty_end_of_region_on_the_successor_refuses_the_donation() { + let backend = Backend::current(); + let sessions = sessions(); + let [_, a, b] = &sessions; + let signal = signal(); + + let (withdrawn, mut replacement) = fed_withdrawn_and_replacement(&sessions, backend, &signal); + replacement + .sources + .seek(b"c", 2, 0) + .expect("a seek on B's c by hand"); + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + assert_eq!( + check_donation(into, from), + Err(DonationError::SuccessorFed), + "seek only" + ); + + // The host seeks to the region's end and closes it with a zero-frame end-of-region chunk: + // nothing written, and the producer has acked the end of the region. (A zero-frame + // end-of-region chunk at generation 1 needs an empty region, which the validator refuses, so + // the chunk only follows a seek.) + let (withdrawn, mut replacement) = fed_withdrawn_and_replacement(&sessions, backend, &signal); + let region = replacement.sources.region(b"c").expect("c's region"); + replacement + .sources + .seek(b"c", 2, region.end) + .expect("a seek to the region's end"); + let empty: [f32; 0] = []; + replacement + .sources + .submit( + b"c", + SourceSubmission { + generation: 2, + start_frame: region.end, + sample_rate_hz: 48_000, + planes: &[&empty, &empty], + frames: 0, + end_of_region: true, + }, + ) + .expect("an empty end-of-region chunk on B's c"); + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + assert_eq!( + check_donation(into, from), + Err(DonationError::SuccessorFed), + "zero-frame end of region" + ); +} + +/// Batch 15a review fold 2, m-2: the host feeds B's `c` after the donation was checked and +/// before it is applied (a contract breach the apply must survive). The apply re-reads that the +/// successor's producer is unfed, refuses, and moves nothing: B keeps its fed ring and the +/// producer that acked into it (its next chunk continues from the acked one), and A keeps its +/// own (not vacated). +/// +/// Test value: red if the apply's re-read does not include the successor producer's `unfed` +/// state, or if the re-read is removed: the consumer swap would then hand B's acked chunk to A +/// and drop B's fed producer (`donate_producers`' `debug_assert!` fires in a debug build). +#[test] +fn a_successor_fed_between_the_check_and_the_apply_moves_nothing() { + let backend = Backend::current(); + let sessions = sessions(); + let [_, a, b] = &sessions; + let signal = signal(); + let (mut withdrawn, mut replacement) = + fed_withdrawn_and_replacement(&sessions, backend, &signal); + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + let plan = check_donation(into, from).expect("the donation checks"); + assert_eq!(plan.len(), 1); + submit(&mut replacement.sources, "c", &signal[1], 1, 0).expect("B's c acks a chunk"); + assert_eq!( + apply_donation( + DonationSideMut { + plan: &mut replacement.plan, + sources: &mut replacement.sources, + }, + DonationSideMut { + plan: &mut withdrawn.plan, + sources: &mut withdrawn.sources, + }, + plan, + ), + 0, + "a fed successor refuses the apply" + ); + for (side, host) in [("B", &replacement), ("A", &withdrawn)] { + let index = host.inventory.source("c").expect("c").index; + assert!( + host.plan.can_prepare_source_seek(index as usize), + "{side}'s c owns its ring" + ); + } + submit(&mut replacement.sources, "c", &signal[1], 1, 1) + .expect("B's producer continues after its acked chunk"); + submit(&mut withdrawn.sources, "c", &signal[0], 1, 4) + .expect("A's producer continues after its acked chunks"); +} + +/// P0 with source `b` declared with `frames`: another declaration of `b`, so a plan prepared +/// against P0 restarts `b` in a fresh ring. +fn with_b_frames(frames: u64) -> SessionModel { + let mut model = p0_model(); + model + .sources + .iter_mut() + .find(|source| source.id.as_str() == "b") + .expect("b") + .frames = frames; + model +} + +/// `model` without source `source` and its track `track`. +fn without(mut model: SessionModel, source: &str, track: &str) -> SessionModel { + remove_track(&mut model, track); + model + .sources + .retain(|existing| existing.id.as_str() != source); + model +} + +/// Whether the acked-fresh-source guard refuses B (`b`) across the withdrawn A (`a`), A prepared +/// against the running P0. `feed` names the source the host feeds two blocks of through A's +/// producers (none when empty); `carry` hands P0's producers of the sources A carries over to A +/// first, as the control plane's commit does. +fn guard_refuses(a: SessionModel, b: SessionModel, feed: &str, carry: bool) -> bool { + let backend = Backend::current(); + let p0 = Session::compile(p0_model()); + let a = Session::compile(a); + let b = Session::compile(b); + let signal = signal(); + let mut running = running_p0(&p0, backend, &signal); + let mut withdrawn = successor_of(&a, &running.inventory, &p0.model, backend); + if carry { + assert!(withdrawn.sources.adopt_persisting(&mut running.sources) > 0); + } + if !feed.is_empty() { + // A fresh ring starts at frame 0; a carried one continues P0's, fed through block BEFORE. + let first = if carry && feed == "a" { BEFORE + 1 } else { 0 }; + for block in first..first + 2 { + submit(&mut withdrawn.sources, feed, &signal[0], 1, block).expect("feed A"); + } + } + let base = withdrawn.carried_base.clone().expect("A has a base"); + let replacement = successor_of(&b, &base, &p0.model, backend); + let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); + let plan = check_donation(into, from).expect("the donation checks"); + // Read-only: a second call finds the same. + let refused = drops_acked_fresh_source(from, &plan); + assert_eq!(refused, drops_acked_fresh_source(from, &plan)); + refused +} + +/// Batch 15a fold 3 (#1310, root ruling M-A of 2026-10-16): the guard refuses exactly when a ring +/// A created (A added or redeclared the source) was fed and the donation does not move it to B. +/// (a)-(c): A redeclares P0's `b` and the host feeds it; B reverts `b` to P0's declaration, +/// redeclares it a third way, or removes it. (d): A adds `c` and the host feeds it; B removes or +/// redeclares it. Not refused: B keeps `c` as A declares it (donated); A's fresh `c` unfed; a +/// source A carries from P0 (`a`), fed through A's producer. +/// +/// Test value: red if the guard reads only the reverted case (b-d pass), ignores the donation +/// plan (the donated `c` is refused), ignores the producer's fed state (the unfed `c` is refused) +/// or ignores freshness (the carried, fed `a` is refused). +#[test] +fn the_guard_refuses_exactly_a_fed_fresh_ring_the_donation_leaves_behind() { + let redeclared_b = || with_b_frames(40_000); + let c_as = |frames: u64| { + let mut model = a_model(); + model + .sources + .iter_mut() + .find(|source| source.id.as_str() == "c") + .expect("c") + .frames = frames; + model + }; + for (label, a, b, feed, carry, refused) in [ + ( + "(a) b reverted", + redeclared_b(), + p0_model(), + "b", + false, + true, + ), + ( + "(b) b redeclared a third way", + redeclared_b(), + with_b_frames(44_000), + "b", + false, + true, + ), + ( + "(c) b removed", + redeclared_b(), + without(p0_model(), "b", "tb"), + "b", + false, + true, + ), + ( + "(d) c added, removed", + a_model(), + without(a_model(), "c", "tc"), + "c", + false, + true, + ), + ( + "(d) c added, redeclared", + a_model(), + c_as(40_000), + "c", + false, + true, + ), + ("c donated", a_model(), b_model(), "c", false, false), + ( + "c fresh and unfed", + a_model(), + without(a_model(), "c", "tc"), + "", + false, + false, + ), + ( + "a carried and fed", + a_model(), + without(a_model(), "c", "tc"), + "a", + true, + false, + ), + ( + "b reverted, unfed", + redeclared_b(), + p0_model(), + "", + false, + false, + ), + ] { + assert_eq!(guard_refuses(a, b, feed, carry), refused, "{label}"); + } +} diff --git a/crates/protocol/src/controller.rs b/crates/protocol/src/controller.rs index 205573f26..b2469b2df 100644 --- a/crates/protocol/src/controller.rs +++ b/crates/protocol/src/controller.rs @@ -1245,6 +1245,8 @@ impl Drop for PreparedStructuralCommand { /// A committed structural response kept private until the caller has published its matched plan. pub struct CommittedCommandFrame { bytes: Vec, + /// The compiled session the commit displaced (#1310 D3), until the caller takes it. + displaced: Option, } impl CommittedCommandFrame { @@ -1252,6 +1254,14 @@ impl CommittedCommandFrame { pub fn write_into(&self, output: &mut [u8]) -> Result { copy_complete_frame(&self.bytes, output) } + + /// The compiled session this commit displaced (#1310 D3): `Some` once for a transaction + /// installed as a whole compilation (every rebuild), so a host keeps the predecessor's model + /// while a successor prepared from it is pending; `None` for a transaction committed in place + /// or of monitor writes only, which displace none. A frame dropped untaken drops it. + pub fn take_displaced(&mut self) -> Option { + self.displaced.take() + } } /// Affinity/use rejection before the non-fallible prepared state application begins. @@ -2286,8 +2296,10 @@ impl ProtocolController

{ monitor_only || path != EditPath::Monitor, "only a transaction of monitor writes has the monitor path" ); + let mut displaced = None; if !monitor_only { - let commit = self.session.commit_prepared(session); + let (commit, previous) = self.session.commit_prepared_displacing(session); + displaced = previous; debug_assert_eq!( commit.applied_operations, prepared.applied_operations as usize @@ -2367,7 +2379,7 @@ impl ProtocolController

{ self.structural_generation .store(prepared.generation.saturating_add(1), Ordering::Release); prepared.armed = false; - Ok(CommittedCommandFrame { bytes }) + Ok(CommittedCommandFrame { bytes, displaced }) } fn process_command_frame_into_legacy( diff --git a/crates/protocol/src/model.rs b/crates/protocol/src/model.rs index d6bb2bd32..562ccb31f 100644 --- a/crates/protocol/src/model.rs +++ b/crates/protocol/src/model.rs @@ -1390,6 +1390,18 @@ impl SessionStore { &mut self, prepared: PreparedSessionTransaction, ) -> SessionCommit { + self.commit_prepared_displacing(prepared).0 + } + + /// [`Self::commit_prepared`], returning the compiled session the commit displaced (#1310 D3): + /// `Some` for a transaction installed as a whole compilation (every rebuild), so a host can + /// keep the predecessor's model while a successor prepared from it is pending; `None` for one + /// committed in place (#1613), which displaces no compilation. Moving it out is + /// allocation-free; dropping it is the caller's. + pub(crate) fn commit_prepared_displacing( + &mut self, + prepared: PreparedSessionTransaction, + ) -> (SessionCommit, Option) { debug_assert_eq!(self.revision(), prepared.base_revision); debug_assert!( !prepared.monitor_only, @@ -1397,20 +1409,26 @@ impl SessionStore { ); let revision = prepared.revision(); let applied_operations = prepared.applied_operations; - match prepared.form { - TransactionForm::Compiled(compiled) => self.compiled = compiled, + let displaced = match prepared.form { + TransactionForm::Compiled(compiled) => { + Some(core::mem::replace(&mut self.compiled, compiled)) + } TransactionForm::InPlace(mut edit) => { self.compiled.swap_owners(&mut edit.owners, &edit.slots); self.compiled.finish_in_place(edit.revision, edit.estimate); + None } TransactionForm::Monitor(_) => { unreachable!("a monitor-only transaction commits no model") } - } - SessionCommit { - revision, - applied_operations, - } + }; + ( + SessionCommit { + revision, + applied_operations, + }, + displaced, + ) } /// The in-place preparation of [`Self::prepare_transaction_in_place`]: `None` unless it can @@ -1730,6 +1748,37 @@ mod tests { ) } + /// #1310 D3. A commit installed as a whole compilation returns the compiled session it + /// displaced, the model the plan that renders was prepared from; a commit made in place + /// displaces none. + /// + /// Red if the displaced compilation is the committed one (a host would keep the successor's + /// model as the running plan's), or if an in-place commit claims to displace one. + #[test] + fn a_compiled_commit_returns_the_compilation_it_displaced() { + let mut store = store(); + let before = store.compiled().normalized_model().clone(); + let revision = ExpectedRevision::Exact(store.revision()); + let gain = route_row("to-main", "gain_db", -3.123_456); + let prepared = store + .prepare_transaction_with_monitor(revision, std::slice::from_ref(&gain)) + .expect("prepares"); + assert!(!prepared.is_in_place()); + let (commit, displaced) = store.commit_prepared_displacing(prepared); + assert_eq!(commit.revision, store.revision()); + let displaced = displaced.expect("a compiled commit displaces the compilation"); + assert_eq!(displaced.normalized_model(), &before); + assert_ne!(store.compiled().normalized_model(), &before); + + let revision = ExpectedRevision::Exact(store.revision()); + let again = route_row("to-main", "gain_db", -6.5); + let prepared = store + .prepare_transaction_in_place(revision, std::slice::from_ref(&again)) + .expect("prepares"); + assert!(prepared.is_in_place()); + assert!(store.commit_prepared_displacing(prepared).1.is_none()); + } + /// #1613. A transaction of values is prepared in place, holds the edited owners and no /// compilation, commits to the session the full path commits to, and is materialized into the /// full preparation on demand; a transaction of solo writes only holds nothing; a structural edit diff --git a/crates/session/src/edit_ramp.rs b/crates/session/src/edit_ramp.rs index 7e2c06a4c..47a3fd3cc 100644 --- a/crates/session/src/edit_ramp.rs +++ b/crates/session/src/edit_ramp.rs @@ -9,7 +9,7 @@ //! applies a transaction's edits) and the host-core classifier (which reads them to pick each live //! record's length) share them without a new dependency between the two. -use std::collections::BTreeMap; +use std::collections::{BTreeMap, BTreeSet}; use crate::StableId; @@ -91,9 +91,18 @@ fn owner_of(target: &EditRampTarget) -> Owner { /// carried a length. An explicit 0 is an entry: it steps that edit and is not "unset". Transaction /// metadata, never model state: it is neither compared with nor merged into a session model, and /// no canonical JSON shows it. +/// +/// It also remembers which rows the transaction wrote with no length and which targets it +/// rewrote whole, so [`Self::overlay`] can lay a later transaction over an earlier one with the +/// later winning on every row it wrote (#1310). Neither changes [`Self::get`]: such a row has no +/// length, and the session's `control_smoothing` applies. #[derive(Clone, Debug, Default, Eq, PartialEq)] pub struct EditRamps { entries: BTreeMap, + /// Rows the latest edit wrote with no length. + unset: BTreeSet, + /// Targets an upsert rewrote whole ([`Self::clear_target`]). + cleared: BTreeSet, } impl EditRamps { @@ -102,6 +111,8 @@ impl EditRamps { pub const fn new() -> Self { Self { entries: BTreeMap::new(), + unset: BTreeSet::new(), + cleared: BTreeSet::new(), } } @@ -117,10 +128,12 @@ impl EditRamps { let key = Key::new(target, row); match samples { Some(samples) => { + self.unset.remove(&key); self.entries.insert(key, samples); } None => { self.entries.remove(&key); + self.unset.insert(key); } } } @@ -128,6 +141,12 @@ impl EditRamps { /// Remove every entry of `target`: an upsert rewrites all of its rows with no length. pub fn clear_target(&mut self, target: &EditRampTarget) { let owner = owner_of(target); + self.clear_owner(&owner); + self.cleared.insert(owner); + } + + /// Remove every entry and every unset row of `owner`. + fn clear_owner(&mut self, owner: &Owner) { // Entries sort by owner, then row, so the target's are one run. let start = Key { owner: owner.clone(), @@ -137,12 +156,39 @@ impl EditRamps { .entries .range(start..) .map(|(key, _)| key) - .take_while(|key| key.owner == owner) + .take_while(|key| key.owner == *owner) .cloned() .collect(); for key in doomed { self.entries.remove(&key); } + self.unset.retain(|key| key.owner != *owner); + } + + /// Lay `later`, a later transaction's lengths, over these (#1310): `later` wins on every row it + /// wrote, with a length or without one, and on every row of a target it rewrote whole; every + /// other row keeps its length here. The result is what one transaction holding both sets of + /// edits, in order, records. + pub fn overlay(&mut self, later: &Self) { + for owner in &later.cleared { + self.clear_owner(owner); + self.cleared.insert(owner.clone()); + } + for (key, samples) in &later.entries { + self.unset.remove(key); + self.entries.insert(key.clone(), *samples); + } + for key in &later.unset { + self.entries.remove(key); + self.unset.insert(key.clone()); + } + } + + /// Whether the transaction wrote no live row at all: no length, no row without one, and no + /// target rewritten whole. [`Self::overlay`] of such a transaction changes nothing. + #[must_use] + pub fn is_untouched(&self) -> bool { + self.entries.is_empty() && self.unset.is_empty() && self.cleared.is_empty() } /// Every entry, ordered by target kind, then ID, then row. @@ -177,6 +223,34 @@ mod tests { StableId::parse(text).expect("valid stable ID") } + /// #1310: the later transaction wins on every row it wrote, a row it wrote with no length + /// included, and on every row of a target it rewrote; the earlier lengths of other rows stay. + /// Red if the overlay keeps an earlier length on a row the later one wrote without one. + #[test] + fn an_overlay_lets_the_later_transaction_win_on_every_row_it_wrote() { + let strip = |name: &str| EditRampTarget::Strip(id(name)); + let mut earlier = EditRamps::new(); + earlier.set(&strip("a"), EditRampRow::Fader, Some(5)); + earlier.set(&strip("a"), EditRampRow::Matrix, Some(6)); + earlier.set(&strip("b"), EditRampRow::Fader, Some(7)); + earlier.set(&strip("c"), EditRampRow::Fader, Some(8)); + let mut later = EditRamps::new(); + later.set(&strip("a"), EditRampRow::Fader, None); + later.set(&strip("b"), EditRampRow::Fader, Some(9)); + later.clear_target(&strip("c")); + assert!(!later.is_untouched()); + earlier.overlay(&later); + assert_eq!(earlier.get(&strip("a"), EditRampRow::Fader), None); + assert_eq!( + earlier.entries(), + vec![ + (strip("a"), EditRampRow::Matrix, 6), + (strip("b"), EditRampRow::Fader, 9), + ] + ); + assert!(EditRamps::new().is_untouched()); + } + #[test] fn clearing_a_target_drops_its_rows_and_nothing_else() { let strip = |name: &str| EditRampTarget::Strip(id(name)); diff --git a/crates/source/src/lib.rs b/crates/source/src/lib.rs index f6c477082..94f868806 100644 --- a/crates/source/src/lib.rs +++ b/crates/source/src/lib.rs @@ -615,6 +615,7 @@ impl PcmSourceRing { native_decoder_sanitized_samples: 0, generation_changed: false, held_seek: None, + begun_block: false, }; // The producer starts with exactly the configured blocks; the retained block above never // enters the recycle queue until the consumer has a played block to hold instead. @@ -1016,6 +1017,9 @@ pub struct PcmSourceConsumer { generation_changed: bool, /// An observed [`SourceCommand::SeekAt`] whose anchor block has not begun yet (issue #1274). held_seek: Option, + /// Whether any block has begun on this consumer (#1344 D5): render-owned, set at block entry + /// and never cleared. + begun_block: bool, } /// A [`SourceCommand::SeekAt`] the consumer has popped and holds until its anchor block. @@ -1117,6 +1121,7 @@ impl PcmSourceConsumer { } fn begin_block_with(&mut self, clock: SeekClock) -> SourceReadReport { + self.begun_block = true; self.begin_block_observe(clock); self.begin_block_play(clock) } @@ -1279,6 +1284,15 @@ impl PcmSourceConsumer { self.channel_count } + /// Whether a block has ever begun on this consumer (#1344 D5). Until the first block it has + /// popped no PCM and played nothing, so every acked chunk, the generation and any seek are + /// still in its ring or held by it: the consumer can move to another plan whole with nothing + /// lost. A seek prepared between blocks ([`Self::prepare_seek`]) is not a block. + #[must_use] + pub const fn has_begun_block(&self) -> bool { + self.begun_block + } + /// Exact configured source ring capacity in transfer blocks. #[must_use] pub const fn transfer_block_capacity(&self) -> usize { @@ -1937,6 +1951,93 @@ impl GraphPreparedSourceSetDriver for SourceGraphSourceSetDriver { true } // REALTIME_POLICY_END + + fn as_any(&self) -> Option<&dyn core::any::Any> { + Some(self) + } + + fn source_has_begun_block(&self, source_index: usize) -> Option { + self.sources + .get(source_index)? + .consumer + .as_ref() + .map(PcmSourceConsumer::has_begun_block) + } + + /// Every pair names an occupied entry on both sides, of equal channel count, no index twice on + /// either side, and the two sets share one quantum. + fn can_swap_sources( + &self, + other: &dyn GraphPreparedSourceSetDriver, + pairs: &[(u32, u32)], + ) -> bool { + other + .as_any() + .and_then(|any| any.downcast_ref::()) + .is_some_and(|other| self.swappable(other, pairs)) + } + + /// Exchange each pair's entries whole after [`Self::can_swap_sources`]'s check. The other + /// set's pending generation change (a seek prepared on a donated consumer) comes with its + /// consumers, as in [`Self::adopt_sources`]. + fn swap_sources( + &mut self, + other: &mut dyn GraphPreparedSourceSetDriver, + pairs: &[(u32, u32)], + ) -> bool { + let Some(other) = other + .as_any_mut() + .and_then(|any| any.downcast_mut::()) + else { + return false; + }; + if !self.swappable(other, pairs) { + return false; + } + for &(own, theirs) in pairs { + core::mem::swap( + &mut self.sources[own as usize], + &mut other.sources[theirs as usize], + ); + } + // Set-wide, so over-approximate: a seek prepared on a donor entry this donation does not + // move still flags this set's next block. Safe: the flag only marks that block's + // observation validity as crossing a generation change, which an observer reads as a + // discontinuity; a false mark costs one block of conservative observation and never + // hides a real change, and this set's own flag is kept. It is also unreachable in + // production, where `prepare_source_seek` runs only on the plan render runs (the active + // plan of the exchange) and a donor is a withdrawn candidate that never was. A per-entry + // flag would put a new read into `begin_block` and `adopt_sources` on render for no + // audible gain. + if !pairs.is_empty() { + self.pending_generation_change |= other.pending_generation_change; + } + true + } +} + +impl SourceGraphSourceSetDriver { + /// [`GraphPreparedSourceSetDriver::can_swap_sources`] against a set of this type. + fn swappable(&self, other: &Self, pairs: &[(u32, u32)]) -> bool { + other.quantum_frames == self.quantum_frames + && pairs.iter().enumerate().all(|(index, &(own, theirs))| { + let fresh = pairs[..index].iter().all(|&(earlier_own, earlier_theirs)| { + earlier_own != own && earlier_theirs != theirs + }); + match ( + self.sources.get(own as usize), + other.sources.get(theirs as usize), + ) { + (Some(own), Some(theirs)) => { + fresh + && own.consumer.is_some() + && theirs.consumer.is_some() + && own.channel_count == theirs.channel_count + } + _ => false, + } + }) + } } /// Seal one or more prepared source consumers into a graph-owned fan-out source set. @@ -4292,5 +4393,223 @@ mod tests { let mut same_quantum = driver(vacant(), FRAMES); assert!(same_quantum.adopt_sources(&mut predecessor, &[(0, 0)])); } + + /// A hand-built driver over `entries` at `quantum_frames`. + fn swap_driver( + entries: Vec, + quantum_frames: u32, + ) -> SourceGraphSourceSetDriver { + SourceGraphSourceSetDriver { + sources: entries.into_boxed_slice(), + mappings: Vec::new().into_boxed_slice(), + quantum_frames, + block_validity: GraphObservationValidity::CLEAR, + pending_generation_change: false, + } + } + + fn occupied(channel_count: u32) -> (HostChunkProvider, GraphSourceEntry) { + let (host, source) = filled_source(); + ( + host, + GraphSourceEntry { + consumer: source.consumer, + channel_count, + }, + ) + } + + fn vacant_entry() -> GraphSourceEntry { + GraphSourceEntry { + consumer: None, + channel_count: 2, + } + } + + /// Which entries own a consumer. + fn occupancy(driver: &SourceGraphSourceSetDriver) -> Vec { + driver + .sources + .iter() + .map(|entry| entry.consumer.is_some()) + .collect() + } + + /// #1344 fold m2. Each D4 refusal of `swappable`, through `can_swap_sources` and + /// `swap_sources`: red if any one condition is dropped (a swap would then move a vacancy, + /// put a consumer under mappings of another channel count, swap one entry twice so a pair + /// trades back, index out of range, or cross quanta), or if a refused swap moves anything. + #[test] + fn a_source_swap_refuses_every_unswappable_pair_and_moves_nothing() { + struct Foreign; + impl GraphPreparedSourceSetDriver for Foreign { + fn claim_count(&self) -> usize { + 0 + } + fn begin_block( + &mut self, + _first_sample: u64, + _frames: u32, + ) -> Result<(), engine::realtime::RenderError> { + Ok(()) + } + fn copy_track_input( + &mut self, + _claim_index: usize, + _left: &mut [f32], + _right: &mut [f32], + ) -> Result<(), engine::realtime::RenderError> { + Ok(()) + } + } + let mut hosts = Vec::new(); + let mut set = |entries: Vec<(u32, bool)>, quantum| { + let entries = entries + .into_iter() + .map(|(channels, full)| { + if full { + let (host, entry) = occupied(channels); + hosts.push(host); + entry + } else { + GraphSourceEntry { + channel_count: channels, + ..vacant_entry() + } + } + }) + .collect(); + swap_driver(entries, quantum) + }; + let full = (2, true); + let empty = (2, false); + let mono = (1, true); + type Case = ( + &'static str, + SourceGraphSourceSetDriver, + SourceGraphSourceSetDriver, + Vec<(u32, u32)>, + ); + let cases: Vec = vec![ + ( + "own index out of range", + set(vec![full], FRAMES), + set(vec![full], FRAMES), + vec![(1, 0)], + ), + ( + "other index out of range", + set(vec![full], FRAMES), + set(vec![full], FRAMES), + vec![(0, 1)], + ), + ( + "own entry vacant", + set(vec![empty], FRAMES), + set(vec![full], FRAMES), + vec![(0, 0)], + ), + ( + "other entry vacant", + set(vec![full], FRAMES), + set(vec![empty], FRAMES), + vec![(0, 0)], + ), + ( + "channel counts differ", + set(vec![full], FRAMES), + set(vec![mono], FRAMES), + vec![(0, 0)], + ), + ( + "quanta differ", + set(vec![full], FRAMES), + set(vec![full], 2 * FRAMES), + vec![(0, 0)], + ), + ( + "own index repeated", + set(vec![full], FRAMES), + set(vec![full, full], FRAMES), + vec![(0, 0), (0, 1)], + ), + ( + "other index repeated", + set(vec![full, full], FRAMES), + set(vec![full], FRAMES), + vec![(0, 0), (1, 0)], + ), + ]; + for (what, mut own, mut other, pairs) in cases { + let before = (occupancy(&own), occupancy(&other)); + assert!(!own.can_swap_sources(&other, &pairs), "{what}: refused"); + assert!(!own.swap_sources(&mut other, &pairs), "{what}: not swapped"); + assert_eq!( + (occupancy(&own), occupancy(&other)), + before, + "{what}: nothing moved" + ); + } + // A driver of another type is refused both ways. + let mut own = set(vec![full], FRAMES); + assert!(!own.can_swap_sources(&Foreign, &[(0, 0)])); + assert!(!own.swap_sources(&mut Foreign, &[(0, 0)])); + assert_eq!(occupancy(&own), [true]); + // Not vacuous: a checked pair swaps both entries whole. + let mut own = set(vec![full, empty], FRAMES); + let mut other = set(vec![empty, full], FRAMES); + assert!(own.can_swap_sources(&other, &[(0, 1)])); + assert!(own.swap_sources(&mut other, &[(0, 1)])); + assert_eq!( + (occupancy(&own), occupancy(&other)), + (vec![true, false], vec![false, true]) + ); + let mut own = set(vec![(2, true), (1, true)], FRAMES); + let mut other = set(vec![(1, true), (2, false)], FRAMES); + assert!(own.can_swap_sources(&other, &[(1, 0)])); + assert!(own.swap_sources(&mut other, &[(1, 0)])); + assert_eq!(occupancy(&own), [true, true]); + } + + /// #1344 fold m2. Red if `swap_sources` drops the other set's pending generation change + /// (a seek prepared on a donated consumer since its last block): the successor's first + /// block would not report the generation change the consumer already took. An empty + /// donation takes nothing. + #[test] + fn a_source_swap_carries_a_prepared_seek_into_the_next_block() { + let swapped = |pairs: &[(u32, u32)]| { + let (_own_host, own_entry) = occupied(2); + let (producer, consumer, _) = PcmSourceRing::prepare(PcmSourceRingConfig { + channel_count: 2, + quantum_frames: QuantumFrames(FRAMES), + frame_capacity: BLOCKS as u64 * u64::from(FRAMES), + initial_generation: SourceGeneration(1), + }) + .expect("ring"); + let mut host = producer.into_host_chunk_provider(RATE); + let mut own = swap_driver(vec![own_entry], FRAMES); + let mut other = swap_driver( + vec![GraphSourceEntry { + consumer: Some(consumer), + channel_count: 2, + }], + FRAMES, + ); + host.try_seek(SourceCommand::Seek { + generation: SourceGeneration(2), + frame: SourceFrame(0), + }) + .expect("seek"); + let planes = frame_samples(0); + host.submit(chunk(2, 0, &[&planes[0], &planes[1]], FRAMES, false)) + .expect("generation 2 PCM"); + assert!(other.prepare_source_seek(0, 2, 0), "the seek is ready"); + assert!(own.swap_sources(&mut other, pairs)); + own.begin_block(0, FRAMES).expect("begin"); + own.observation_validity().source_generation_changed + }; + assert!(swapped(&[(0, 0)]), "the donated seek is reported"); + assert!(!swapped(&[]), "an empty donation takes nothing"); + } } } diff --git a/docs/CONTROL_PROTOCOL_SEMANTICS.md b/docs/CONTROL_PROTOCOL_SEMANTICS.md index 66fab7c52..b890f79f7 100644 --- a/docs/CONTROL_PROTOCOL_SEMANTICS.md +++ b/docs/CONTROL_PROTOCOL_SEMANTICS.md @@ -6,6 +6,8 @@ This defines controller behavior for the [BTLV bytes](CONTROL_BTLV_V1.md) and [r The committed `SessionModel` owns its revision, and the revision lies in `0..=9223372036854775807` (2^63 - 1, the session schema's domain; the wire field stays `u64`); command `revision-any` is therefore the header flag plus a zero carrier, never a revision sentinel. Structural edits require an exact revision and at least one edit; a configured zero edit limit disables the transaction command and its paired session events in capabilities and dispatch. Transactions clone/apply/compile a candidate typed session and replace `(SessionModel, CompiledSession, SessionRevision)` only when all validation, capacity reservation, and compilation steps succeed. One success increments once; malformed input, empty input, validation/compile/resource failure, backpressure, and replay leave model and revision untouched. A mutation at revision 2^63 - 1 returns `REVISION_EXHAUSTED` before the candidate is built; the controller does not turn a compiled session into a `PreparedRenderPlan`. +A transaction that needs a new plan while an earlier one's replacement plan is still unadopted commits as any other (#1310): the host takes the waiting replacement back, prepares one successor of the running plan that carries both, and publishes it, or, should the new transaction be refused, publishes the waiting replacement again unchanged. Host-fed sources the earlier transaction added keep their accepted PCM and queued seeks. No accepted chunk is dropped across a supersession: when the waiting replacement created a source's ring (the earlier transaction added the source, or redeclared it) and the host has fed that ring (a chunk, a seek or an end of region), a new transaction that does not keep the source as the replacement declares it (it declares it as the running plan does, redeclares it another way, or removes it) is refused with backpressure before it commits. Nothing changes (the replacement stays published, no event, no revision), and a retry after render adopts the replacement commits. A host that does not render keeps getting backpressure for such a transaction until it renders. A source the replacement carries unchanged from the running plan is not refused this way. The successor issue #1698 lifts the refusal only where the new transaction declares the source as the running plan does; the other cases stay refused until an issue lets the successor read those chunks or the owner rules that a removed source's chunks may drop. A source the caller's own transaction removes discards its queued audio, whether the running plan or the waiting replacement (carrying it unchanged from the running plan) holds it: the caller asked for the removal, so this is not an unintended drop. No counter or telemetry field reports those frames (source telemetry leaves with the source; the watermark outcome flags and the session counters name none); a counter for the discarded frames is a successor issue (placeholder #1699). The applied-revision watermark completes the displaced revisions as `superseded` when the successor is adopted. The resource admission counts every plan the session still holds (the running plan, the waiting replacement, and a plan render has displaced until the service step reclaims it) and the successor. A resource refusal is backpressure when the successor fit against the newest held plan alone, so a retry after a render normally commits; the split is approximate, and such a retry may still be refused within a small band (a successor prepared on the adopted replacement differs by a few hundred bytes from one prepared across it). It is a compile rejection when the successor does not fit even then. + All v1 operations are absolute and idempotent. A new logical command has a nonzero, strictly increasing endpoint-local request ID. The bounded replay cache stores exact canonical request and response bytes. Identical repeated bytes return the cached response without execution; same ID with changed bytes returns `REQUEST_ID_REUSE`; evicted/retired IDs return `REPLAY_EXPIRED`. Completed backpressure is replayed too, so retrying it uses a new ID. Cache capacity is reserved before execution; an unretainable response is rejected before state changes. The guarantee ends with the endpoint lifetime; reconnect recovery belongs to an adapter. ## The response preview (decision 17, C13, #1573) diff --git a/docs/C_ABI_V1_QUALIFICATION.md b/docs/C_ABI_V1_QUALIFICATION.md index b73e7ce77..a8c4ed2e2 100644 --- a/docs/C_ABI_V1_QUALIFICATION.md +++ b/docs/C_ABI_V1_QUALIFICATION.md @@ -553,7 +553,8 @@ model's delta (`host_core::classify_live_delta`, #1053 D1), never by its opcodes its parameters (two cells for a per-lane parameter, one for a shared one) and an EQ's prepared targets (one cell per target slot and channel) are written under the commit's stamp, so no live edit is refused for room and none returns `BACKPRESSURE` with `control.live.backpressure`, - which no longer exists. A full plan queue is still `control.plan.backpressure`. + which no longer exists. A full plan exchange is `INTERNAL` since #1398 D3 (the exchange holds + every candidate a session can meet), never `BACKPRESSURE`. - **No ack before a drop (#1053 D6, #1312 D7, D11).** Classification, the live admission (#1053 D8), the producer lookup and preflight of every effect record and the protocol's commit predicate all run before the commit's stamp, and the stamp is the last fallible step; nothing @@ -800,3 +801,89 @@ and #1602 removed them with their ceilings (4,300,720 bytes of cache and 947,248 journal outside every compile limit) and their epoch row. The engine's memory outside the compile limits is its handle alone; the C header states it in its "Engine memory" paragraph. A host that needs fast bounded designs again re-earns a cache in a new issue. + +## Plan capacities for a superseding candidate (#1398) + +A structural transaction is never refused because render has swapped plans but not yet published +the new epoch: the refusal that waited for the epoch atomic is gone, and a transaction submitted in +that window is admitted against the plan that renders. + +- **Capacities.** The plan-resource report table holds 4 rows (a lagging epoch's, the running + plan's, a withdrawn candidate's and its successor's), the pending and retired provider lists 2 + each, and the plan exchange 3 retirement credits (a displaced plan not yet reclaimed, a withdrawn + candidate's credit and its successor's reservation). A full exchange, report table or pending + list while a candidate is reserved is now `INTERNAL`, never `BACKPRESSURE`; an incompatible + envelope, an exhausted epoch and a destroyed plan stay `BACKPRESSURE`. +- **Admission.** A structural candidate is admitted against every plan the session holds + unreclaimed (the running plan and every pending or withdrawn candidate) plus itself, and every + compiled model it holds plus its own, in every cap the replacement peak checks; the live + admission sums the same held plans. Since supersession (#1310) a transaction submitted while a + candidate is pending commits, and that admission holds three plans. A source ring is counted + once, in the plan it was carried from: the candidate's carried rings leave its row, and so do + those of every held candidate after the running plan (batch 15a review fold). +- **Sizing rule.** The header states it beside `maximum_graph_session_plus_plan_bytes`: each + resource cap must be at least three times one plan's row, and the graph cap must also cover three + compiled models. The repository's reference limits (`tools/audit/src/capi.rs`, + `crates/capi/tests/plan_swap_race.rs`, `crates/capi/tests/resource_lifecycle.rs`) admit three + plans of the nine-track EQ reference session with room to spare (graph 316,350 bytes against + 100,000,000; capi 168,810 against 10,000,000), so none was raised. The rule is measured from the + row of a running successor plan, which carries its sources and owns their rings. The browser's caps are its + one memory budget, which charges the same tables through the same control-plane code; its + replacement-peak term must cover the three-plan peak when a warm candidate is admitted (#1403). +- **Resource movement.** The larger tables add fixed bytes to every session's + `capi_retained_bytes` (+1,904 bytes, #1398 D1's tables). On the nine-track EQ reference session + the row measures 165,842 bytes at the batch 15a head, 165,922 after its review fold and 168,810 + after fold 2 (x86-64; the ceiling in `crates/capi/tests/resource_lifecycle.rs` is 175,296). A caller whose + `maximum_capi_retained_bytes` was exact must raise it. No symbol, result code or event is added. + +## Supersession of an unadopted candidate (#1310) + +- **Behavior.** A structural transaction submitted while an earlier replacement is unadopted + commits (`RESULT_OK`); `RESULT_BACKPRESSURE` for a pending candidate is gone. The session takes + the candidate back, prepares one successor of the running plan that carries both transactions + (from the candidate's carried base and the running plan's kept model, monitor overlay and edit + ramps), moves the rings of the sources the candidate added into it, and publishes it. A refusal + after the withdrawal republishes the candidate unchanged. The watermark completes the displaced + revisions with `MISO_ENGINE_V1_OUTCOME_SUPERSEDED` when the successor is adopted. A transaction + that finds the candidate already taken by render builds on it, as before; the session first + treats every pending candidate as adopted, so it never holds two taken candidates beside a third. +- **Refusal at the three-plan peak.** A successor that does not fit beside the running plan, the + withdrawn candidate and their models is refused. The refusal is `RESULT_BACKPRESSURE` when the + successor fit against the newest held plan alone and the committed model, so a retry after a + render normally commits; it is `RESULT_COMPILE_REJECTED` with the cap's diagnostic when it does + not fit even then. The split is approximate: a successor prepared on the adopted candidate + differs from one prepared across the withdrawn candidate (about +64 graph and +158 capi bytes + on gate 4's session), and the second check counts a donated ring twice, so a retry may still be + refused within that small band. A host that is not rendering keeps getting + `RESULT_BACKPRESSURE` until it renders. +- **A fed ring the candidate created is backpressure (batch 15a folds 2 and 3, root rulings of + 2026-10-16).** When the candidate created a source's ring (it added the source or redeclared + it), the host has fed that ring (an accepted chunk, seek or end of region), and the new + transaction does not keep the source as the candidate declares it (it declares it as the running + plan does, redeclares it another way, or removes it), the successor would not take that ring and + the accepted PCM would drop with the candidate. The core (`host_core::drops_acked_fresh_source`, + checked right after the donation check, before the commit) refuses the transaction with + `RESULT_BACKPRESSURE`: nothing is committed, the candidate is republished, no event is queued. + After render adopts the candidate, the retry commits on it; a host that does not render keeps + getting `RESULT_BACKPRESSURE` until it renders. A source the candidate carries unchanged from the + running plan is not refused. Both hosts answer alike + (`control_parity::a_supersession_that_would_drop_acked_pcm_is_backpressure_alike_on_both_ + hosts`, cases reverted, redeclared a third way and removed). #1698 lifts only the reverted case. +- **Displaced plans are charged until reclaimed (batch 15a fold 2).** When a transaction finds the + candidate taken by render and builds on it, the plan render displaced may not yet be reclaimed + (a render preempted between its claim and its retirement). The session keeps that plan's + resource row and charges it in both admissions until a later call reclaims it; a refusal it + causes is `RESULT_BACKPRESSURE`, which the next service call clears. The session parks one such + epoch per retirement credit (three), and every call reclaims what render handed back before it + promotes from the published epoch, so two taken candidates and a published adoption never leave + the session answering `RESULT_INTERNAL`. +- **Admission.** While a successor of the running plan is pending, the running plan's model is + kept, so the replacement and the live admissions count it: three plans and three models when a + candidate is superseded, three models when a live edit lands while one waits. +- **Resource movement.** Each candidate epoch keeps its carried base (#1344), charged in its capi + epoch row (`carried_base_retained_bytes`); the boot plan has none. The provider epoch itself grew + (its kept base and carried base, and the batch 15a review fold's carried source bytes), and the + pending and retired provider lists charge that size, so the reference session's boot + `capi_retained_bytes` moves: 165,842 bytes at the batch 15a head, 165,922 after its review fold, + 168,810 after fold 2 (x86-64; a third retired slot and a 248-byte parked row in each of the + five slots, +2,888), within the 175,296 ceiling. No symbol, result code or event is added. diff --git a/docs/rulings/one-control-abstraction-2026-10-13.md b/docs/rulings/one-control-abstraction-2026-10-13.md index 7e58ecd6b..5b0dc379b 100644 --- a/docs/rulings/one-control-abstraction-2026-10-13.md +++ b/docs/rulings/one-control-abstraction-2026-10-13.md @@ -293,7 +293,19 @@ the browser moves onto the same frames. The phase after S13 delivers them (slice smaller one-host differences found by the same audit (plan latency and tail, the agent lattice, source submit diagnostics, render-failure recovery, session admission caps, CPU telemetry, the SDK builder's checks, canonical-PCM decode and content identity) are recorded for root to file as -their own issues under this rule. +their own issues under this rule. Source submission, seeks and the seek report are transport under +C1, with one core function behind both hosts, so their direct exports are not C13 exceptions +(ruling 2026-10-15, #1316). +Widened (root, 2026-10-16): each direct export approved in batch 15 is transport of bytes or host +lifecycle, with one core function behind it on both hosts, so none is a C13 exception. They are: +the source seek report, `miso_engine_v1_source_seek_report` (#1316, B15-R2); +`miso_engine_web_v1_declare_discontinuity` (#1323, B15-R7), the browser form of the host's +discontinuity declaration; the two #1386 document exports, `miso_engine_web_v1_replace_document_ptr` +(the browser's staging of the replacement document's bytes) and +`miso_engine_v1_submit_document_command` (the C ABI's transport of the same command); and #1688's +plan latency read, `miso_engine_web_v1_plan_latency_read` with its record pointer +`miso_engine_web_v1_plan_latency_ptr`, a copy of the core's `PlanQueries::resources()` row. A new +direct export that is not in this list needs its own root ruling. ### How to add a control diff --git a/hosts/host-web/src/lib.rs b/hosts/host-web/src/lib.rs index cc1bedd3a..c7b2fe732 100644 --- a/hosts/host-web/src/lib.rs +++ b/hosts/host-web/src/lib.rs @@ -1457,8 +1457,13 @@ struct PlanShape { observation_present: Box<[bool]>, } -/// How many replacement plans' tables the host holds at once: the plan exchange admits one -/// unadopted candidate at a time, and one more waits for the service step after its adoption. +/// How many replacement plans' tables the host holds at once. After every commit the core holds +/// at most one unadopted candidate (a structural edit supersedes a waiting one, #1310), and the +/// host drops a superseded candidate's tables as soon as the core no longer holds its epoch +/// ([`ReadyOwnership::drop_superseded_shapes`]); the tables of plans adopted since the last +/// service step wait for it. So a command meets at most one entry after the reclaim and adds one: +/// four leaves room to spare, and the room check in [`ReadyOwnership::run_command`] never refuses +/// a structural edit, paused or not. const PENDING_SHAPES: usize = 4; /// One replacement plan's tables, committed with the plan (#1290a). @@ -1613,6 +1618,7 @@ impl ReadyOwnership { request: &[u8], output_capacity: u64, analysis: Option<&mut SpectrumAnalysis>, + applied_at_sample: u64, ) -> Result { self.reclaim_shapes(analysis); if self.pending_shapes.len() == self.pending_shapes.capacity() { @@ -1631,7 +1637,21 @@ impl ReadyOwnership { // `Ok` only once it has left the plan's tables in the record (it refuses the // preparation, before anything is committed, when it cannot). So a committed rebuild // always has its tables, and no refusal can follow a commit. - let shape = prepared.expect("a committed rebuild's preparer left the plan's tables"); + let mut shape = + prepared.expect("a committed rebuild's preparer left the plan's tables"); + // #1310 (root ruling B15-Q4): the records the core queued again on this plan from the + // candidate it superseded are noted as admitted records are, so the hand-over keeps + // them; the core says which they are and in what state. + for forward in self.control.observation_forwarded() { + if let Some(&effect) = shape.producer_slots.get(forward.position) { + shape.note_observe( + effect as usize, + forward.tap_index, + forward.armed, + applied_at_sample, + ); + } + } self.pending_shapes.push(PendingShape { epoch, shape: Box::new(shape), @@ -1646,6 +1666,9 @@ impl ReadyOwnership { .name_successor(core::ptr::from_ref(&entry.shape.meter_header)); } self.blocking_rebuilds = self.blocking_rebuilds.saturating_add(1); + // #1310: the commit may have superseded the candidate that waited; its tables and + // spectrum captures go now, on the control side, as the core says. + self.drop_superseded_shapes(); } if result.is_ok() { self.stamp_structure_epoch(); @@ -1708,6 +1731,18 @@ impl ReadyOwnership { fn reclaim_shapes(&mut self, analysis: Option<&mut SpectrumAnalysis>) { self.promote_spectrum(analysis); self.pending_shapes.retain(|entry| !entry.adopted); + self.drop_superseded_shapes(); + } + + /// Drop the tables, and the spectrum captures, of every candidate the core no longer holds + /// (#1310, batch 15a review fold B1): a structural edit that superseded a waiting candidate + /// dropped its plan with the commit, and no block will adopt it. The core decides + /// ([`control_plane::SessionState::holds_plan_epoch`]); an adopted entry waits for + /// [`Self::reclaim_shapes`], which promotes its spectrum first. Control side only: it frees. + fn drop_superseded_shapes(&mut self) { + let control = &self.control; + self.pending_shapes + .retain(|entry| entry.adopted || control.holds_plan_epoch(entry.epoch)); } /// Promote the plans the render half has adopted (#1290d2; #1395 D3, D5): install each @@ -1724,7 +1759,13 @@ impl ReadyOwnership { /// audio thread, which keeps none, passes `None`. fn promote_spectrum(&mut self, mut analysis: Option<&mut SpectrumAnalysis>) { for entry in &mut self.pending_shapes { - let (Some(carried), Some(handover)) = (entry.adoption, entry.spectrum.take()) else { + // A candidate render has not adopted keeps its hand-over: a service step or command + // between its commit and its adoption block must not drop it (batch 15a review fold + // 2). Only an adopted entry's is taken. + let Some(carried) = entry.adoption else { + continue; + }; + let Some(handover) = entry.spectrum.take() else { continue; }; let Some(service) = self.spectrum.as_mut() else { @@ -2001,27 +2042,14 @@ impl ReadyOwnership { /// (#1676 attempt 2: the queue push itself is the core's /// [`SessionState::admit_and_push_observation_batch`], which admits the whole batch first). fn record_observe(&mut self, staged: StagedObserve, applied_at_sample: u64) { - let effect = staged.effect as usize; // Issue #143: the armed set is control-plane state and is updated here, where the record // is admitted, so it can never disagree with what the render side was told. - if let Some(mask) = self.shape.observation_armed.get_mut(effect) - && staged.tap_index < u32::BITS - { - let bit = 1_u32 << staged.tap_index; - *mask = if staged.armed { - *mask | bit - } else { - *mask & !bit - }; - if let Some(samples) = self.shape.observation_arm_samples.get_mut(effect) - && let Some(sample) = samples.get_mut(staged.tap_index as usize) - { - *sample = applied_at_sample; - } - if let Some(commanded) = self.shape.observation_commanded.get_mut(effect) { - *commanded |= bit; - } - } + self.shape.note_observe( + staged.effect as usize, + staged.tap_index, + staged.armed, + applied_at_sample, + ); } } @@ -2337,6 +2365,22 @@ impl AudioWorkletEngineHost { self.plan.as_ref().map(|plan| plan.owner().active_epoch().0) } + /// Render one block through the plan owner at `absolute_sample` rather than the host's clock, + /// and nothing else (#1398 gate 1b); `false` when the owner refused it or the host is not + /// ready. A mistimed block adopts a published successor before the owner checks its time, and + /// is refused without publishing the epoch atomic: the window in which render has swapped + /// plans and the control plane has not seen the epoch. The host's own records (clock, shape, + /// status) do not move. + #[cfg(feature = "test-support")] + pub fn render_plan_at_for_test(&mut self, absolute_sample: u64) -> bool { + let quantum = self.status.quantum_frames as usize; + let (Some(plan), Some(buffers)) = (self.plan.as_mut(), self.buffers.as_mut()) else { + return false; + }; + PlanarBufferMut::try_new(&mut buffers.output_pcm, 2, quantum, quantum) + .is_ok_and(|output| plan.render(absolute_sample, output).is_ok()) + } + /// The staging buffer for one control command frame of `len` bytes (issue #1562a D1), or /// `None` when `len` is zero or exceeds the boot's `maximum_control_frame_bytes` /// (16 KiB) or the host is not ready. The page writes the frame here and @@ -2389,11 +2433,12 @@ impl AudioWorkletEngineHost { if len == 0 || u64::from(len) > CONTROL_FRAME_BYTES { return self.record(RESULT_INVALID_ARGUMENT); } + let applied_at_sample = self.status.next_absolute_sample; let Some((buffers, ready)) = self.buffers.as_mut().zip(self.ready.as_mut()) else { return self.fail(RESULT_INTERNAL, b"web.internal.ready\t$\n"); }; let request = &buffers.control_request[..len as usize]; - match ready.run_command(request, CONTROL_FRAME_BYTES, analysis) { + match ready.run_command(request, CONTROL_FRAME_BYTES, analysis, applied_at_sample) { Ok(length) => { self.control_response = ControlResponse::Frame(length); self.record(RESULT_OK) @@ -5927,6 +5972,26 @@ fn build_plan_shape( } impl PlanShape { + /// Note one observation record queued on this plan's effect slot `effect` for `tap_index` + /// (#143, #1381b): the armed bit, the sample its window is fresh from, and the commanded bit + /// that keeps the swap's hand-over from overwriting it with the predecessor's state. + fn note_observe(&mut self, effect: usize, tap_index: u32, armed: bool, applied_at_sample: u64) { + if let Some(mask) = self.observation_armed.get_mut(effect) + && tap_index < u32::BITS + { + let bit = 1_u32 << tap_index; + *mask = if armed { *mask | bit } else { *mask & !bit }; + if let Some(samples) = self.observation_arm_samples.get_mut(effect) + && let Some(sample) = samples.get_mut(tap_index as usize) + { + *sample = applied_at_sample; + } + if let Some(commanded) = self.observation_commanded.get_mut(effect) { + *commanded |= bit; + } + } + } + /// Name `successor` as the header that replaces this plan's (null for none): the `u32` word /// the page reads, and beside it the full-width pointer a gate compares. #[inline] diff --git a/hosts/host-web/src/tests.rs b/hosts/host-web/src/tests.rs index dbdec3531..f45643aff 100644 --- a/hosts/host-web/src/tests.rs +++ b/hosts/host-web/src/tests.rs @@ -7065,6 +7065,333 @@ mod structural_apply { assert_eq!(rig.host.service(None), RESULT_OK); } + /// #1310 on the browser path, which runs the same control plane: a structural edit while a + /// candidate waits unadopted commits and supersedes it. The first candidate never renders; the + /// replacement carries both transactions and is adopted at the next block. + /// + /// Test value: red if the second structural edit is refused while the first waits (the + /// one-candidate backpressure the C ABI dropped), or if the replacement leaves out the first + /// transaction's source. + #[test] + fn a_structural_edit_behind_an_unadopted_candidate_supersedes_it() { + let model = parse_session_json(include_str!("../tests/browser-v1/command-session.json")) + .expect("accepted command fixture"); + let mut source = model.sources[0].clone(); + source.id = StableId::parse("a-source").expect("source ID"); + let mut track = model.tracks[0].clone(); + track.id = StableId::parse("a-track").expect("track ID"); + track.source_id = source.id.clone(); + let mut route = model.routes[0].clone(); + route.id = StableId::parse("a-track-main").expect("route ID"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("a track route") + }; + *track_id = track.id.clone(); + let mut later = source.clone(); + later.id = StableId::parse("b-source").expect("source ID"); + let first = [ + SessionEdit::UpsertSource { source }, + SessionEdit::UpsertTrack { track }, + SessionEdit::UpsertRoute { route }, + ]; + let second = [SessionEdit::UpsertSource { source: later }]; + let mut rig = Rig::boot(&model, options(), None); + rig.step(); + assert!(matches!( + rig.apply(&first, 1), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + )); + assert!( + matches!( + rig.apply(&second, 2), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + ), + "a structural edit behind an unadopted candidate supersedes it" + ); + rig.step(); + assert_eq!( + rig.epoch(), + 2, + "the replacement is adopted; the first never rendered" + ); + assert_eq!(rig.host.session_source_count(), 3); + assert_eq!(rig.host.service(None), RESULT_OK); + rig.step(); + } + + /// #1310 on a paused page (batch 15a review fold B1): eight structural edits with no render + /// between them each commit, each superseding the candidate before it, as on the C ABI. The + /// host keeps one table entry per candidate the core holds, so the fifth edit finds room; + /// then the next block adopts the newest plan, which holds all eight edits, and from that + /// block on the output equals a fresh boot of the committed model. + /// + /// Test value: red if the host keeps a superseded candidate's tables until a later adoption + /// (the fifth paused edit is refused with `BACKPRESSURE`, as before the fold), or if the + /// adopted plan misses an edit. + #[test] + fn paused_structural_edits_all_commit_and_the_newest_renders_as_a_fresh_boot() { + const EDITS: u64 = 8; + let model = gate_model(); + let mut rig = Rig::boot(&model, options(), None); + for _ in 0..APPLY_BEFORE { + rig.step(); + } + let base = rig.host.committed_revision(); + let mut results = Vec::new(); + for edit in 0..EDITS { + let committed = rig + .host + .ready + .as_ref() + .expect("ready") + .control + .committed_session() + .normalized_model() + .clone(); + let edits = add_muted_track(&committed, &format!("a-muted-{edit}")); + let (result, response) = + rig.entry(&frame(&edits, edit + 1, rig.host.committed_revision())); + results.push(result); + // A client drains the committed events as it goes, so the event lane never refuses. + while rig.host.control_event(EVENT_LANE_RELIABLE) == RESULT_OK + && !rig.host.control_response().is_empty() + {} + if result == RESULT_OK { + assert_eq!( + outcome(&response), + Outcome::Applied { + path: EditPath::Rebuild, + revision: base + edit + 1 + }, + "edit {edit}" + ); + } + } + assert_eq!( + results, + vec![RESULT_OK; EDITS as usize], + "every paused structural edit commits" + ); + assert_eq!(rig.epoch(), 0, "nothing rendered: the boot plan still runs"); + assert!( + rig.host.ready.as_ref().expect("ready").pending_shapes.len() <= 1, + "the host keeps the tables of the one candidate the core holds" + ); + let document = canonical_session_json( + rig.host + .ready + .as_ref() + .expect("ready") + .control + .committed_session() + .normalized_model(), + ) + .expect("canonical session"); + let committed = parse_session_json(&document).expect("the committed model parses"); + let mut fresh = Rig::boot(&committed, options(), None); + for _ in 0..BLOCKS { + fresh.step(); + } + let before = rig.watermark(); + while rig.block < BLOCKS { + rig.step(); + } + assert!(!rig.pending(), "render adopted the newest plan"); + assert_eq!(rig.host.service(None), RESULT_OK); + assert_eq!( + rig.host.live_control_tracks().len(), + model.tracks.len() + EDITS as usize, + "the adopted plan holds every edit's track" + ); + let watermark = rig.watermark(); + assert_eq!(watermark.revision, base + EDITS); + assert_eq!( + watermark.superseded_count, + before.superseded_count + EDITS - 1, + "the first seven complete as superseded" + ); + assert_eq!( + first_difference( + &rig.bits[APPLY_BEFORE as usize..], + &fresh.bits[APPLY_BEFORE as usize..] + ), + None, + "from the adoption block on, every block equals a fresh boot of the committed model" + ); + assert!( + fresh.bits[APPLY_BEFORE as usize..] + .iter() + .any(|block| block.iter().any(|word| f32::from_bits(*word) != 0.0)), + "the compared blocks are not silent" + ); + } + + /// #1310 on the browser with a spectrum capture (batch 15a review fold 2, m-5): a candidate + /// whose tables carry a spectrum hand-over (its prepared captures and pairing) is superseded + /// before render adopts it. The superseding command drops the candidate's entry, captures + /// included, on the control side; the replacement's hand-over is the one that waits, and the + /// service step after its adoption promotes it, so the stream keeps reading. The render-call + /// allocation count of the same script is pinned to zero in `structural_apply_realtime.rs`. + /// + /// Test value: red if a superseded candidate's tables and hand-over outlive the command that + /// superseded it (two entries wait), if the replacement's hand-over is lost (none waits, or + /// the stream stops after the swap), or if a service step before the adoption block drops a + /// waiting candidate's hand-over. + #[test] + fn a_superseded_candidates_spectrum_handover_drops_with_the_superseding_command() { + let model = gate_model(); + let mut rig = Rig::boot(&model, options(), single_capture()); + rig.host + .start_spectrum_stream(SpectrumSmoothingConfig::default(), None) + .expect("the stream starts"); + for _ in 0..APPLY_BEFORE { + rig.step(); + } + let first = add_muted_track(&model, ADDED); + let second = add_muted_track(&with_muted_track(&model, ADDED), "a-muted-b"); + assert!(matches!( + rig.apply(&first, 1), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + )); + assert_eq!(rig.spectrum_promotions_waiting(), 1, "A's hand-over waits"); + assert_eq!(rig.host.service(None), RESULT_OK); + assert_eq!( + rig.spectrum_promotions_waiting(), + 1, + "a service step before the adoption block keeps A's hand-over" + ); + assert!(matches!( + rig.apply(&second, 2), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + )); + assert_eq!( + rig.host.ready.as_ref().expect("ready").pending_shapes.len(), + 1, + "the superseded candidate's entry dropped with the command" + ); + assert_eq!( + rig.spectrum_promotions_waiting(), + 1, + "only the replacement's hand-over waits" + ); + rig.step(); + assert_eq!(rig.epoch(), 2, "the replacement is adopted"); + assert_eq!(rig.host.service(None), RESULT_OK); + assert_eq!( + rig.spectrum_promotions_waiting(), + 0, + "the service step promoted the replacement's captures" + ); + let mut windows = 0; + while rig.block < SPECTRUM_BLOCKS { + rig.step(); + let read = rig.read_stream(); + if read.window.is_some() { + windows += 1; + } + } + assert!(windows > 0, "the stream publishes windows after the swap"); + } + + /// #1310 (root ruling B15-Q4) on the browser, the host that admits observation records: a + /// subscription acked against a candidate that a later structural edit supersedes arms the + /// replacement and publishes windows there. The replacement carries "track"'s compressor from + /// the plan that renders, which never saw the record, so the core queues the record again on + /// the replacement and the host copies it onto the replacement's tables. + /// + /// Test value: red if the record is lost with the superseded candidate, in the engine (the + /// carried tap keeps the running plan's unarmed state and no window publishes) or in the + /// host's tables (the hand-over copies the running plan's unarmed bit and the read reports + /// the tap unarmed). + #[test] + fn a_subscription_queued_on_a_superseded_candidate_publishes_on_its_replacement() { + let mut model = + parse_session_json(include_str!("../tests/browser-v1/command-session.json")) + .expect("accepted command fixture"); + model.tracks[0].inserts.effects[0] = observed_compressor_insert(); + let mut source = model.sources[0].clone(); + source.id = StableId::parse("a-source").expect("source ID"); + let mut track = model.tracks[0].clone(); + track.id = StableId::parse("a-track").expect("track ID"); + track.source_id = source.id.clone(); + let mut route = model.routes[0].clone(); + route.id = StableId::parse("a-track-main").expect("route ID"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("a track route") + }; + *track_id = track.id.clone(); + let mut later = source.clone(); + later.id = StableId::parse("b-source").expect("source ID"); + let first = [ + SessionEdit::UpsertSource { source }, + SessionEdit::UpsertTrack { track }, + SessionEdit::UpsertRoute { route }, + ]; + let second = [SessionEdit::UpsertSource { source: later }]; + let mut rig = Rig::boot(&model, options(), None); + rig.step(); + assert!(matches!( + rig.apply(&first, 1), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + )); + assert_eq!( + observe_key(&mut rig.host, "track", RACK_INSERTS, "comp", 1, 2, true), + RESULT_OK, + "the subscription is acked against the pending candidate" + ); + assert!(matches!( + rig.apply(&second, 2), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + )); + rig.step(); + assert_eq!(rig.epoch(), 2, "the replacement is adopted"); + assert_eq!( + ( + armed_mask(&rig.host, "track", "comp"), + armed_mask(&rig.host, "a-track", "comp") + ), + (Some(1), Some(0)), + "the replacement's tables hold the subscription" + ); + for _ in 0..6 { + rig.step(); + } + let binding = (0..rig.host.observation_binding_count() as u32) + .filter_map(|index| rig.host.observation_binding(index)) + .find(|binding| binding.address.track_index == 1) + .expect("the old track's compressor is bound"); + let mut address = binding.address; + address.tap_id = 1; + let read = rig + .host + .read_observation_addresses(&[address]) + .expect("a read"); + assert_eq!(&*read[0].track_id, "track"); + assert!( + read[0].window.is_some(), + "the subscription publishes windows on the replacement: {:?}", + read[0].status + ); + } + /// Drive `blocks` more blocks with the lease on, polling after each; the windows folded. fn poll_blocks(rig: &mut Rig, blocks: u64) -> u32 { let mut windows = 0; diff --git a/hosts/host-web/tests/control_parity.rs b/hosts/host-web/tests/control_parity.rs index 736621aa6..5278f290e 100644 --- a/hosts/host-web/tests/control_parity.rs +++ b/hosts/host-web/tests/control_parity.rs @@ -3029,3 +3029,333 @@ fn both_hosts_designate_the_master_strip_after_an_empty_boot_alike() { ); } } + +/// #1310 on a paused host (batch 15a review fold B1): eight structural edits with no render +/// between them each commit on both hosts, each superseding the candidate before it, and from the +/// block that adopts the newest plan the two hosts render the same bits. +/// +/// Test value: red if one host refuses a paused structural edit the other commits (the browser +/// kept a superseded candidate's tables until a later adoption and refused the fifth edit with +/// `BACKPRESSURE`), or if the two hosts adopt different plans. +#[test] +fn paused_structural_edits_commit_alike_on_both_hosts() { + const EDITS: u64 = 8; + const PAUSE_AT: u64 = 2; + for rate in [48_000, 96_000] { + let model = model_at(rate); + let mut browser = Rig::new(&model); + let mut c_abi = CAbi::new(&model); + for block in 0..PAUSE_AT { + let b = browser.step(block); + assert_eq!(b, c_abi.step(block), "{rate} Hz: block {block}"); + } + let routes: Vec = model + .routes + .iter() + .filter(|route| { + matches!( + &route.source, + session::RouteSource::Track { track_id, .. } if track_id.as_str() == T1.id + ) + }) + .cloned() + .collect(); + assert!(!routes.is_empty(), "t1 routes somewhere"); + for edit in 0..EDITS { + // A copy of `t1` with no insert, on `t1`'s source, routed as `t1` is. + let mut track = model.tracks[T1.index as usize].clone(); + track.id = id(&format!("p{edit}")); + track.inserts.effects.clear(); + let mut edits = vec![SessionEdit::UpsertTrack { + track: track.clone(), + }]; + for (index, route) in routes.iter().enumerate() { + let mut route = route.clone(); + route.id = id(&format!("p{edit}-r{index}")); + let session::RouteSource::Track { track_id, .. } = &mut route.source else { + unreachable!("filtered on a track source") + }; + *track_id = track.id.clone(); + edits.push(SessionEdit::UpsertRoute { route }); + } + let (b_verdict, _) = browser.transaction_reply(&edits); + let (c_verdict, _) = c_abi.transaction(&edits); + c_abi.drain_reliable(); + assert_eq!( + b_verdict, + Verdict::Applied { + path: EditPath::Rebuild, + revision: model.revision + edit + 1 + }, + "{rate} Hz: paused edit {edit} commits on the browser" + ); + assert_eq!(b_verdict, c_verdict, "{rate} Hz: paused edit {edit}"); + } + for block in PAUSE_AT..PAUSE_AT + 8 { + let b = browser.step(block); + assert_eq!(b, c_abi.step(block), "{rate} Hz: block {block}"); + } + assert_eq!( + browser.host.live_control_tracks().len(), + model.tracks.len() + EDITS as usize, + "{rate} Hz: the browser adopted the plan of every edit" + ); + } +} + +/// #1310, root rulings of 2026-10-16 (no ack before a drop; M-A of fold 3), on both hosts. T1 (A) +/// redeclares `src-t1`, so A restarts it at frame 0 in a ring A creates, and the host's next +/// chunk for it is acked into A's ring. T2 (B) does not keep `src-t1` as A declares it: (a) it +/// puts `src-t1` back as the running plan declares it, (b) it redeclares it a third way, or (c) +/// it removes `src-t1` with its track `t1` and `t1`'s routes. B does not take A's ring by +/// donation, so A's acked chunk would drop with A, and both hosts refuse T2 with +/// `BACKPRESSURE`, the same code and the same revision. The next block adopts A on both, and the +/// retried T2 commits alike; the two hosts render the same bits in every block. +/// +/// This session's plan has a compensated latency of more than one block, and it grows with the +/// rate (a plain run first plays at frame 459 at 48 kHz, 939 at 96 kHz): the first block a plain +/// run plays is `lead`. So T1 commits only once the output plays, the adoption block must +/// play, and the acked chunk reaches the output `lead` blocks after the adoption block. The retry +/// waits for that block. A control run, whose host never feeds A's `src-t1` ring and so never +/// attempts the early T2, renders the same blocks until then and a different block there, on both +/// hosts: the acked chunk is heard, at A's restart position. +/// +/// Test value: red if either host lets such a supersession commit (the guard missing in the +/// core, or the fold-2 guard that refused only case (a)), if the two hosts refuse it differently +/// or render differently, or if A's acked chunk is not heard (dropped, or not played at A's +/// restart position). +#[test] +fn a_supersession_that_would_drop_acked_pcm_is_backpressure_alike_on_both_hosts() { + let plays = |pcm: &[u32]| pcm.iter().any(|word| f32::from_bits(*word) != 0.0); + for rate in [48_000, 96_000] { + let model = model_at(rate); + let name = format!("src-{}", T1.id); + let original = model + .sources + .iter() + .find(|source| source.id.as_str() == name) + .expect("t1's source") + .clone(); + let redeclare = |way: u8| { + let mut redeclared = original.clone(); + let last = redeclared.content.pop().expect("a content identity"); + // `way` distinct digits, none equal to the original's last one. + let digit = (b'0'..=b'9') + .filter(|&digit| char::from(digit) != last) + .nth(usize::from(way)) + .expect("a digit"); + redeclared.content.push(char::from(digit)); + redeclared + }; + let first = [SessionEdit::UpsertSource { + source: redeclare(0), + }]; + let mut removal: Vec = model + .routes + .iter() + .filter(|route| { + matches!( + &route.source, + session::RouteSource::Track { track_id, .. } if track_id.as_str() == T1.id + ) + }) + .map(|route| SessionEdit::RemoveRoute { + route_id: route.id.clone(), + }) + .collect(); + removal.push(SessionEdit::RemoveTrack { + track_id: id(T1.id), + }); + removal.push(SessionEdit::RemoveSource { + source_id: original.id.clone(), + }); + for (case, second, t1_in_b) in [ + ( + "(a) reverted", + vec![SessionEdit::UpsertSource { + source: original.clone(), + }], + true, + ), + ( + "(b) redeclared a third way", + vec![SessionEdit::UpsertSource { + source: redeclare(1), + }], + true, + ), + ("(c) removed", removal.clone(), false), + ] { + // The plan's latency in whole blocks: the first block a plain run plays. + let lead = { + let mut browser = Rig::new(&model); + (0..BLOCKS) + .find(|&block| plays(&browser.step(block))) + .expect("the session plays") + }; + let adopt = lead + 1; + let heard_at = adopt + lead; + let retry = heard_at + 1; + + // Each host is fed `src-t1` (if `t1`) from `t1_start` (its own frame, in blocks) and the + // other two sources at the render position, then renders `block`. + let feed_and_render = |browser: &mut Rig, + c_abi: &mut CAbi, + block: u64, + t1: bool, + t1_start: u64| + -> Vec { + for (feed, strip) in [T1, T2, T3].into_iter().enumerate() { + if strip.id == T1.id && !t1 { + continue; + } + let source = format!("src-{}", strip.id); + let start = if strip.id == T1.id { t1_start } else { block }; + let [left, right] = planes(feed as u64, block); + assert_eq!( + browser.host.submit_source( + source.as_bytes(), + 1, + start * u64::from(QUANTUM), + rate, + &[&left, &right], + QUANTUM, + false, + ), + host_web::RESULT_OK, + "{rate} Hz {case}: browser {source}" + ); + c_abi.submit_from(&source, feed as u64, block, start, 1); + } + assert_eq!(browser.host.render_next(), host_web::RESULT_OK); + let b = bits(browser.host.output_pcm().expect("browser output")); + assert_eq!(b, c_abi.render(block), "{rate} Hz {case}: block {block}"); + b + }; + + // `feed_a`: the host feeds A's `src-t1` ring the acked chunk, and T2 is attempted before + // the adoption. Otherwise the host never feeds A's ring before the retry. + let run = |feed_a: bool| -> Vec> { + let mut browser = Rig::new(&model); + let mut c_abi = CAbi::new(&model); + let mut blocks = Vec::new(); + for block in 0..adopt { + let b = browser.step(block); + assert_eq!(b, c_abi.step(block), "{rate} Hz {case}: block {block}"); + blocks.push(b); + } + + let (b_first, _) = browser.transaction_reply(&first); + let (c_first, _) = c_abi.transaction(&first); + assert!( + matches!( + b_first, + Verdict::Applied { + path: EditPath::Rebuild, + .. + } + ), + "{rate} Hz {case}: the redeclaration is a rebuild: {b_first:?}" + ); + assert_eq!(b_first, c_first, "{rate} Hz {case}: T1"); + let revision = c_abi.revision; + + if feed_a { + // A's `src-t1` chunk at frame 0, acked before any block adopts A: the source's + // PCM of the adoption block, as the host would send it. + let [left, right] = planes(0, adopt); + assert_eq!( + browser.host.submit_source( + name.as_bytes(), + 1, + 0, + rate, + &[&left, &right], + QUANTUM, + false, + ), + host_web::RESULT_OK + ); + c_abi.submit_from(&name, 0, adopt, 0, 1); + + // T2 before any block adopts A. + let b_frame = browser.encode( + CommandPayload::SessionTransactionApply(&second), + ExpectedRevision::Exact(SessionRevision(browser.revision)), + ); + let (b_code, b_reply) = browser.send_frame(&b_frame); + let (c_code, c_reply) = c_abi.send_status( + CommandPayload::SessionTransactionApply(&second), + ExpectedRevision::Exact(SessionRevision(c_abi.revision)), + ); + assert_eq!( + (b_code, c_code), + (host_web::RESULT_BACKPRESSURE, capi::RESULT_BACKPRESSURE), + "{rate} Hz {case}: the supersession would drop A's acked chunk: {} / {}", + String::from_utf8_lossy(&b_reply), + String::from_utf8_lossy(&c_reply) + ); + assert_eq!(c_abi.revision, revision, "{rate} Hz {case}: no commit"); + assert_eq!(browser.revision, revision, "{rate} Hz {case}: no commit"); + assert_eq!( + drain_events(&mut browser.host), + 0, + "{rate} Hz {case}: the refusal queues no event" + ); + } + + // The adoption block, and the blocks until the acked chunk reaches the output. Only + // the other two sources are fed: the one difference between the two runs is the + // acked chunk. + for block in adopt..retry { + blocks.push(feed_and_render(&mut browser, &mut c_abi, block, false, 0)); + } + + // The retry commits on A, alike; B restarts `src-t1` at frame 0 again. + let (b_second, _) = browser.transaction_reply(&second); + let (c_second, _) = c_abi.transaction(&second); + assert!( + matches!(b_second, Verdict::Applied { .. }), + "{rate} Hz {case}: the retry commits: {b_second:?}" + ); + assert_eq!(b_second, c_second, "{rate} Hz {case}: T2 retried"); + for block in retry..retry + 5 { + blocks.push(feed_and_render( + &mut browser, + &mut c_abi, + block, + t1_in_b, + block - retry, + )); + } + blocks + }; + + let fed = run(true); + let unfed = run(false); + let adopt = adopt as usize; + let heard_at = heard_at as usize; + assert!( + plays(&fed[adopt]), + "{rate} Hz {case}: the adoption block (block {adopt}) plays" + ); + assert!( + plays(&fed[heard_at]), + "{rate} Hz {case}: the block that carries the acked chunk (block {heard_at}) plays" + ); + assert_eq!( + fed[..heard_at], + unfed[..heard_at], + "{rate} Hz {case}: the refusal changes nothing, and the acked chunk is not heard early" + ); + assert_ne!( + fed[heard_at], unfed[heard_at], + "{rate} Hz {case}: the acked chunk is heard (block {heard_at}, latency {lead} blocks)" + ); + assert!( + fed[retry as usize..].iter().all(|block| plays(block)), + "{rate} Hz: the blocks after the retry play" + ); + } + } +} diff --git a/hosts/host-web/tests/plan_exchange_threads.rs b/hosts/host-web/tests/plan_exchange_threads.rs index 28115693a..2d48f1e4d 100644 --- a/hosts/host-web/tests/plan_exchange_threads.rs +++ b/hosts/host-web/tests/plan_exchange_threads.rs @@ -23,8 +23,8 @@ use std::alloc::System; use std::sync::Mutex; use host_web::{ - RESULT_INVALID_ARGUMENT, RESULT_OK, RenderLockedAllocator, WATERMARK_BYTES, WebBootOptions, - companion_drops_on_this_thread, native_staging, + AudioWorkletEngineHost, RESULT_INVALID_ARGUMENT, RESULT_OK, RenderLockedAllocator, + WATERMARK_BYTES, WebBootOptions, companion_drops_on_this_thread, native_staging, }; use protocol::{ CommandPayload, DecodeScratch, DecodedSuccessResponsePayload, DecodedTypedResponseFrame, @@ -95,9 +95,13 @@ fn add_muted_track(model: &SessionModel, id: &str) -> [SessionEdit; 2] { } fn transaction_bytes(edits: &[SessionEdit], revision: u64) -> Vec { + transaction_bytes_with_id(edits, revision, 1) +} + +fn transaction_bytes_with_id(edits: &[SessionEdit], revision: u64, request_id: u64) -> Vec { let codec = ProtocolCodec::default(); let frame = TypedCommandFrame { - request_id: RequestId::new(1).expect("nonzero request"), + request_id: RequestId::new(request_id).expect("nonzero request"), expected_revision: ExpectedRevision::Exact(SessionRevision(revision)), payload: CommandPayload::SessionTransactionApply(edits), }; @@ -486,3 +490,69 @@ fn a_rebuild_adds_the_longest_id_and_feeds_the_new_source() { .join() .expect("the run completes"); } + +/// #1398 gate 1b, the browser twin of the C ABI's +/// `control_calls_inside_a_plan_swapping_render_call_keep_replacement_live`: a structural edit +/// made after render has adopted the previous one's plan, but before the epoch atomic names it, is +/// admitted against the running plan. Boot, commit a structural edit (a muted track), and render +/// the block that adopts it as a mistimed block, which swaps plans and is refused before the +/// atomic is published. The second structural edit, through the browser's control path, returns +/// OK, and the next block adopts its plan. +/// +/// Test value: red if the lag refusal is kept on the browser path (the base returns +/// `BACKPRESSURE`), or if the browser's budget does not charge #1398 D1's tables and so refuses +/// the second plan; the C ABI's gate 1 tests do not boot the browser. +#[test] +fn a_structural_edit_inside_the_swap_window_is_admitted_on_the_browser_path() { + let _turn = TURN.lock().expect("the turn"); + std::thread::spawn(|| { + let mut host = AudioWorkletEngineHost::boot(document().as_bytes(), options()) + .unwrap_or_else(|_| panic!("the fixture boots")); + let model = parse_session_json(&document()).expect("the fixture parses"); + let command = |host: &mut AudioWorkletEngineHost, request: &[u8]| { + let length = u32::try_from(request.len()).expect("a small frame"); + host.control_request_mut(length) + .expect("a control frame fits the staging")[..request.len()] + .copy_from_slice(request); + let result = host.control_command(length, None); + (result, host.control_response().to_vec()) + }; + + let first = transaction_bytes(&add_muted_track(&model, "a-muted"), model.revision); + let (result, response) = command(&mut host, &first); + assert_eq!(result, RESULT_OK, "the first structural edit publishes"); + assert_rebuild_applied(&response, model.revision + 1); + assert_eq!(host.active_plan_epoch_for_test(), Some(0)); + + // The block that adopts it, mistimed: the owner swaps, then refuses the block, so the + // epoch atomic still names the boot plan. + assert!( + !host.render_plan_at_for_test(1), + "a mistimed block is refused" + ); + assert_eq!( + host.active_plan_epoch_for_test(), + Some(1), + "the owner adopted the first plan" + ); + + // Inside the window: the second structural edit is admitted against the running plan. + let second = + transaction_bytes_with_id(&add_muted_track(&model, "b-muted"), model.revision + 1, 2); + let (result, response) = command(&mut host, &second); + assert_eq!(result, RESULT_OK, "the window's structural edit publishes"); + assert_rebuild_applied(&response, model.revision + 2); + + // The next block adopts its plan. + assert_eq!(host.render_next(), RESULT_OK); + assert_eq!( + host.active_plan_epoch_for_test(), + Some(2), + "the window's plan renders" + ); + assert_eq!(host.service(None), RESULT_OK); + assert_eq!(host.dispose(), RESULT_OK); + }) + .join() + .expect("the run completes"); +} diff --git a/hosts/host-web/tests/structural_apply_realtime.rs b/hosts/host-web/tests/structural_apply_realtime.rs index 5ef6f5fa8..94dc91807 100644 --- a/hosts/host-web/tests/structural_apply_realtime.rs +++ b/hosts/host-web/tests/structural_apply_realtime.rs @@ -21,8 +21,9 @@ use std::sync::Mutex; use controls::{ControlAddress, Owner, Section}; use host_web::{ - RESULT_OK, RenderLockedAllocator, WebBootOptions, companion_drops_on_this_thread, - native_staging, + ABI_VERSION, RESULT_OK, RenderLockedAllocator, SPECTRUM_CAPTURE_BYTES, SPECTRUM_CHANNEL_BOTH, + SPECTRUM_REQUEST_BYTES, SPECTRUM_TARGET_TRACK_POST_PAN, WebBootOptions, WebSpectrumRequest, + companion_drops_on_this_thread, native_staging, }; use protocol::{ CommandPayload, DecodeScratch, DecodedSuccessResponsePayload, DecodedTypedResponseFrame, @@ -95,7 +96,18 @@ fn muted_track(model: &SessionModel) -> (session::Track, session::Route) { } fn add_muted_track(model: &SessionModel) -> Vec { - let (track, route) = muted_track(model); + add_muted_track_as(model, ADDED) +} + +/// [`add_muted_track`] under the track ID `id`. +fn add_muted_track_as(model: &SessionModel, id: &str) -> Vec { + let (mut track, mut route) = muted_track(model); + track.id = StableId::parse(id).expect("track ID"); + route.id = StableId::parse(&format!("{id}-main")).expect("route ID"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("a track route") + }; + *track_id = track.id.clone(); vec![ SessionEdit::UpsertTrack { track }, SessionEdit::UpsertRoute { route }, @@ -558,3 +570,82 @@ fn a_strip_count_change_publishes_the_successors_frame_on_the_first_block_after_ } assert!(adoption_frame.iter().any(|word| *word != 0.0)); } + +/// #1310 on the browser with a spectrum capture (batch 15a review fold 2, m-5): T1 commits a +/// candidate whose tables carry a spectrum hand-over, and T2, before any block adopts it, +/// supersedes it. The superseded candidate's plan and its captures drop in T2's command, on the +/// control side; render then adopts the replacement and the next service step promotes its +/// captures. `src/tests.rs` pins which hand-over waits; this binary pins the cost. +/// +/// Test value: red if a render call allocates or frees across the supersession, the adoption +/// block included (a superseded candidate's captures or tables reclaimed by the render half), +/// or if an attachment drops inside a render call. +#[test] +fn a_superseded_candidate_with_a_spectrum_hand_over_costs_no_render_call_an_allocation() { + let _turn = TURN.lock().expect("the turn"); + let (render_allocations, drops_in_render, epochs) = std::thread::spawn(|| { + let base = model(); + let target = base.tracks[3].id.as_str().to_owned(); + native_staging::spectrum_request( + WebSpectrumRequest { + struct_size: SPECTRUM_REQUEST_BYTES, + abi_version: ABI_VERSION, + target: SPECTRUM_TARGET_TRACK_POST_PAN, + channels: SPECTRUM_CHANNEL_BOTH, + target_id_bytes: target.len() as u32, + maximum_capture_bytes: SPECTRUM_CAPTURE_BYTES as u64, + ..WebSpectrumRequest::default() + }, + target.as_bytes(), + ); + let handle = boot(&base); + let mut render_allocations = 0; + let mut drops_in_render = 0; + let mut epochs = Vec::new(); + for block in 0..BLOCKS { + feed(handle, block); + if block == APPLY_BEFORE { + for (request, edits) in [ + (1, add_muted_track(&base)), + (2, add_muted_track_as(&with_muted_track(&base), "a-muted-b")), + ] { + let revision = host_web::miso_engine_web_v1_control_revision(handle); + let (result, response) = + native_staging::control_command(handle, &frame(&edits, request, revision)) + .expect("a live handle"); + assert_eq!(result, RESULT_OK, "transaction {request}"); + assert_eq!(applied(&response), (EditPath::Rebuild, revision + 1)); + } + assert_eq!( + native_staging::active_plan_epoch(handle), + Some(0), + "neither candidate rendered" + ); + } + let allocations = host_web::miso_engine_web_v1_render_allocation_count(); + let drops = companion_drops_on_this_thread(); + assert_eq!( + host_web::miso_engine_web_v1_render(handle, QUANTUM_FRAMES), + RESULT_OK, + "block {block}" + ); + drops_in_render += companion_drops_on_this_thread() - drops; + render_allocations += + host_web::miso_engine_web_v1_render_allocation_count() - allocations; + epochs.push(native_staging::active_plan_epoch(handle).expect("a plan")); + assert_eq!(host_web::miso_engine_web_v1_service(handle), RESULT_OK); + } + assert_eq!(host_web::miso_engine_web_v1_dispose(handle), RESULT_OK); + (render_allocations, drops_in_render, epochs) + }) + .join() + .expect("the run completes"); + let mut expected = vec![0; APPLY_BEFORE]; + expected.extend(vec![2; BLOCKS - APPLY_BEFORE]); + assert_eq!( + epochs, expected, + "the replacement is adopted; T1's candidate never rendered" + ); + assert_eq!(render_allocations, 0, "a render call allocated or freed"); + assert_eq!(drops_in_render, 0, "render dropped an attachment"); +}