From 82a50b8b78d48eecce6cb9be1c3115066419dd75 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 19:00:07 +0000 Subject: [PATCH 01/12] Record #1620's measurement: the first live commit's excess is the full path, not reusable state (stopped before coding) Co-Authored-By: Claude Opus 5.5 --- ...he-first-live-commit-of-a-large-session.md | 36 +++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md index b11fbb585..0ec3cbfe5 100644 --- a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md +++ b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md @@ -52,3 +52,39 @@ AudioWorklet's message handler on a 64- or 256-track session. ## Size Half a day, one sonnet-high implementer. + +## Decision record: #1620 (batch 14b) -- measurement, stopped before coding + +Anchor correction: the heap gate lives at `tools/parameter-metadata/tests/control_command_heap.rs` +(the batch 9 CI fold moved it to use `bench_support::alloc`), not under `hosts/host-web/tests/`. + +Measured on `8b3ae13ec` (debug, `verify_stages(false)`), the nine-edit transaction of that test: + +| tracks | first call: allocations / requested / peak / left | steady call peak | first-call excess | session canonical text | +|---|---|---|---|---| +| 1 | 834 / 129,249 / 77,347 / 1,078 | 72,588 | 4,759 | 5,474 | +| 64 | 1,848 / 549,392 / 197,929 / 1,078 | 72,588 | 125,341 | 96,887 | +| 256 | 4,922 / 1,866,704 / 663,179 / 1,078 | 72,588 | 590,591 | 375,479 | + +**What the excess is.** Not lazily built state that later calls reuse. Traced with a temporary +backtrace hook on every allocation of 4 KiB or more (reverted): the first call leaves the in-place +path (#1613) and takes the full preparation. `SessionStore::prepare_in_place` +(`crates/protocol/src/model.rs`) returns `None` because `session::owners_counts_equal` is false: the +fixture's track holds a sparse `eq-simd1` parameter list (IDs 1 to 6, ID 4 per lane), and the edits +write IDs 65, 66, 67, 81, 82, 83 and ID 4 on `both`, which adds parameter entries. A count change is +not a change of values to the in-place path, so the store clones the whole model, runs +`compile_session` (a second model clone, the validation maps, the stable sorts) and writes the +whole canonical text. At 256 tracks the peak is that text's `String` doubling to 524,288 bytes +(final 376,405) plus about 66 KB of model clones and sort buffers; at 64 tracks the text reaches +131,072. The 1,078 bytes left are the committed model's added parameter entries. From the second +call on, the entries exist, the counts are equal and every call is in place (72,588 at every size). + +**Why neither option fits.** Option A (build the reused state at boot) and Option B (lazy, one exact +allocation) both assume reusable state; there is none. The cost is the full compile path, taken by +any live edit that first writes a parameter the sparse model does not hold, on any revision, not +only the first. The candidate fixes are outside this spec's authorized paths and need a ruling: +(C1) let the in-place path accept a `SetControl` that adds a parameter entry (adjust +`owners_counts_equal` and the in-place resource estimate in `protocol`/`session` so an added entry +is charged exactly); or (C2) densify every effect's parameter list at preparation (changes the +committed model and its canonical snapshot, a contract change). Per the coordinator's ruling the +work stopped here; no code changed. From 17a4deaffe5f9e7f68019a409d503680f126caf7 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 19:00:53 +0000 Subject: [PATCH 02/12] #1620: root ruling C1 (in-place admission of added parameter entries) Co-Authored-By: Claude Opus 5.5 --- ...und-the-first-live-commit-of-a-large-session.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md index 0ec3cbfe5..dddf01c82 100644 --- a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md +++ b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md @@ -88,3 +88,17 @@ only the first. The candidate fixes are outside this spec's authorized paths and is charged exactly); or (C2) densify every effect's parameter list at preparation (changes the committed model and its canonical snapshot, a contract change). Per the coordinator's ruling the work stopped here; no code changed. + +## Root ruling (2026-10-15): C1 + +- **Design.** The in-place path admits a `SetControl` that adds a parameter entry, and charges the + added entry exactly. The committed model and the canonical snapshot equal the full path's for + the same transaction. The rejected alternative (C2, fill every parameter list at preparation) + would change the committed model and its canonical snapshot. +- **Authorized paths** gain `crates/session/src/in_place.rs` and `crates/protocol/src/model.rs` + (the in-place admission and its resource estimate), and the heap gate's real home, + `tools/parameter-metadata/tests/control_command_heap.rs`. +- **Gates.** (1) A differential gate: for edits that add parameter entries, the in-place commit's + model and canonical snapshot equal the full path's, bit for bit. (2) The first-call peak at 64 + and 256 tracks is measured and recorded, with a ceiling near the steady peak. (3) The first + call's allocation count does not depend on the track count. From 5fb2f7b5f14d9e4159aaf0286c2d9da7c4ad24fd Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 19:15:24 +0000 Subject: [PATCH 03/12] Build the effect control owner's candidate at begin (#1624) Co-Authored-By: Claude Opus 5.5 --- ...ffect-control-owners-candidate-at-begin.md | 55 +++++++ crates/capi/tests/resource_lifecycle.rs | 17 ++- crates/control-plane/src/control.rs | 3 +- crates/effect-compiler/src/control.rs | 138 +++++++++++++----- crates/effect-compiler/src/prepare.rs | 7 +- crates/host-core/tests/prepare.rs | 7 +- 6 files changed, 172 insertions(+), 55 deletions(-) diff --git a/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md b/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md index 8691201dc..b2cb8367b 100644 --- a/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md +++ b/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md @@ -41,3 +41,58 @@ document. ## Size Half a day. + +## Decision record: #1624 (batch 14b) + +**Stopped once before coding.** The spec's Option A (one scratch sized to the largest owner) cannot +hold: `commit_live` step 3 begins every touched EQ owner before any commits, so the open candidates +overlap (up to every EQ, 768 on the worst document). The owner is in +`crates/effect-compiler/src/control.rs`, not in the spec's paths. The coordinator ruled option B' +and approved the added paths (`effect-compiler/src/control.rs`, `prepare.rs`, +`host-core/tests/prepare.rs`, the snapshot doc in `control-plane/src/control.rs`). + +**Shipped (B').** `EffectControlOwner` holds `candidate: Option` (rows + touched mask). +`begin` builds it from the committed rows; `commit`, `discard` and a publication fault drop it, and +`OpenOwners::drop` already discards every begun owner on a refusal. `candidate()` and `dirty()` +return an empty slice while no candidate is open. `resource_facts` drops the two terms (an idle +owner retains the box and its committed rows only). `new` keeps a transient all-false mask for its +validation call. + +**Departure.** The coordinator asked for the rows and mask in one allocation. They are two exact +`Box<[T]>` (720 and 60 bytes per EQ): `EffectTargetRequest` takes `values: &[InitialParameterValue]` +and `changed: &[bool]` as separate slices, and one allocation would need `unsafe` or a contract +change. Cost: two allocations per begun owner per transaction. + +**Measured (x86-64, eight lanes).** + +| Figure | Before | After | +|---|---|---| +| Wasm boot growth, 1 MiB worst document (`check-web-boot-budget.mjs`, shipped simd128 module) | 20,512,768 (313 pages) | 19,857,408 (303 pages), -655,360; the multiplier 20 holds (pin plus one page is 21,037,056) | +| `capi_retained_bytes`, 9-EQ reference session | 166,374 | 159,354 (-7,020 = 9 x 780) | +| effect-control resources (producer table + owner payload) | 19,377 | 12,357 | +| routed-submix `capi` observed (one EQ) | 136,634 | 135,854 | +| Native retained, worst document | not run | 768 EQs x 780 = 599,040 bytes by arithmetic | +| `control_command_heap` steady, 1 track (seven EQ edits on one EQ): allocations / requested / peak | 756 / 103,138 / 72,841 | 758 / 103,918 / 72,841 | +| `control_command_heap` first call, 1 track: allocations / requested / peak | 837 / 129,514 / 77,600 | 839 / 130,294 / 77,600 | + +The peak is unmoved (it is reached elsewhere in the call than the open owner); the steady requested +bytes are 82 under their 104,000 ceiling, so a second EQ in that transaction would break it (the +ceiling is the test's own and was not raised). + +**Re-pins ("#1624", ceilings only fall).** `resource_lifecycle.rs`: `capi_retained_bytes` ceiling +182,400 -> 175,296 at both widths (159,354 + 10 %, up to 64); `REFERENCE_EFFECT_CONTROL_BUDGET` +21,376 -> 13,632 (12,357 + 10 %, up to 64). The four-lane value is the same (lane-independent +rows; CI prints confirm). + +**Gates.** `cargo test --locked` green for effect-compiler, effect-contract, control-plane +(`--all-features`), host-core (`--all-features`), host-web (`--all-features`; includes +`render_locked_*` and `retained_ceilings`, unchanged), capi (including `reference_session_retained_rows_stay_within_their_budgets` +and `capi_retained_bytes_charge_every_byte_the_compile_retains`), and +`parameter-metadata --test control_command_heap`. The EQ live-edit tests (C ABI and browser parity) +pass unchanged. New test (effect-compiler): +`the_candidate_lives_only_between_begin_and_commit_or_discard`, red if `begin` leaves nothing or +`discard` / `commit` keep the candidate (it asserts empty slices when idle and a fresh committed +copy after a discard). `revision_overflow_is_refused_before_candidate_mutation` now asserts no +candidate is built on a refused begin. Updated arithmetic: `prepare.rs` resource tests and +`host-core/tests/prepare.rs` drop the candidate and mask terms. `realtime-policy` ok; clippy +`-D warnings` and fmt clean on the touched crates. diff --git a/crates/capi/tests/resource_lifecycle.rs b/crates/capi/tests/resource_lifecycle.rs index f951ac83e..dcc2e049c 100644 --- a/crates/capi/tests/resource_lifecycle.rs +++ b/crates/capi/tests/resource_lifecycle.rs @@ -1220,6 +1220,12 @@ impl Budget { /// on the lane width), so its ceiling is 165,782 plus 10 %, rounded up to 64: 182,360.2 -> 182,400. /// Batch 13b adds #1639's jump-length cell to every plan: measured 165,934 on the integrated tree /// (x86-64), within the ceiling, so the ceiling stays. +/// +/// #1624 (batch 14b): an effect control owner builds its candidate rows and touched mask in `begin` +/// and frees them at `commit` or `discard`, so an idle owner retains only its committed rows (780 +/// bytes less per EQ: 60 rows of 12 bytes and 60 flags). `capi_retained_bytes` measured 166,374 +/// before and 159,354 after (x86-64, -7,020 = nine EQs; the row does not depend on the lane width), +/// so its ceiling is 159,354 plus 10 %, rounded up to 64: 175,289.4 -> 175,296 (was 182,400). const REFERENCE_BUDGETS: [Budget; 19] = [ Budget { row: "graph_session_plus_plan_bytes", @@ -1326,8 +1332,8 @@ const REFERENCE_BUDGETS: [Budget; 19] = [ Budget { row: "capi_retained_bytes", value: |report| report.capi_retained_bytes, - eight_lanes: 182_400, - four_lanes: 182_400, + eight_lanes: 175_296, + four_lanes: 175_296, }, Budget { row: "largest_named_allocation_bytes", @@ -1348,8 +1354,11 @@ const REFERENCE_MODEL_BUDGET: u64 = 25_344; /// effect lane and per control cell, not per block). Its measured total for the nine console-slot /// EQs is 19,377 bytes at both widths (the row is lane-width independent: 15,345 before #1345's /// per-tap cells replaced the control queue and the target FIFO), plus 10 %, rounded up to a -/// 64-byte multiple: 21,314.7 -> 21,376. The same convention as the model budget above. -const REFERENCE_EFFECT_CONTROL_BUDGET: u64 = 21_376; +/// 64-byte multiple: 21,314.7 -> 21,376. The same convention as the model budget above. #1624: the +/// owners no longer retain a candidate and a mask between transactions (9 x 780 bytes), so the +/// total measured 12,357 (was 19,377); the ceiling is 12,357 plus 10 %, rounded up to 64: +/// 13,592.7 -> 13,632. +const REFERENCE_EFFECT_CONTROL_BUDGET: u64 = 13_632; /// Sets one C cap row. type SetCap = fn(&mut CompileLimits, u64); diff --git a/crates/control-plane/src/control.rs b/crates/control-plane/src/control.rs index 4b7466594..000ba72eb 100644 --- a/crates/control-plane/src/control.rs +++ b/crates/control-plane/src/control.rs @@ -271,7 +271,8 @@ pub struct TestTransactionSnapshot { pub effect_cells: Vec, /// Every EQ owner's state, current epoch first, then each pending candidate: `(epoch, strip /// ID, effect instance ID, committed revision, phase, committed value bits, candidate value - /// bits)`. + /// bits)`. The candidate is empty while the owner is idle: it exists only between `begin` and + /// `commit` or `discard` (#1624). pub effect_owners: Vec, } diff --git a/crates/effect-compiler/src/control.rs b/crates/effect-compiler/src/control.rs index 73fa2415e..064eef7b6 100644 --- a/crates/effect-compiler/src/control.rs +++ b/crates/effect-compiler/src/control.rs @@ -67,13 +67,21 @@ pub struct EffectControlOwner { factory: Arc, sample_rate: u32, committed: Box<[InitialParameterValue]>, - candidate: Box<[InitialParameterValue]>, - dirty: Box<[bool]>, + /// The open transaction's rows and touched mask. Built by `begin`, dropped by `commit`, + /// `discard` and a publication fault, so an idle owner retains only `committed` (#1624). + candidate: Option, committed_revision: u64, pending_revision: Option, phase: EffectControlOwnerPhase, } +/// The rows an open transaction edits and the mask of those it touched; both are exactly as long +/// as the owner's committed rows. +struct OpenCandidate { + rows: Box<[InitialParameterValue]>, + dirty: Box<[bool]>, +} + /// Engine-owned allocations retained by one prepared-target owner. #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub(crate) struct EffectControlOwnerResources { @@ -108,12 +116,13 @@ impl EffectControlOwner { let Some(capability) = factory.target_preparation() else { return Err(EffectControlOwnerError::Unsupported); }; - let dirty = vec![false; preparation.initial_values.len()].into_boxed_slice(); + // Transient: the owner keeps no mask between transactions. + let untouched = vec![false; preparation.initial_values.len()]; capability.validate_targets( EffectTargetRequest { sample_rate: preparation.sample_rate, values: preparation.initial_values.as_ref(), - changed: &dirty, + changed: &untouched, }, &[], )?; @@ -121,8 +130,7 @@ impl EffectControlOwner { Ok(Self { factory, sample_rate: preparation.sample_rate, - candidate: committed.clone(), - dirty, + candidate: None, committed, committed_revision: 0, pending_revision: None, @@ -157,16 +165,17 @@ impl EffectControlOwner { &self.committed } - /// The current candidate rows. The slice is borrowed and cannot be retained by preparation. + /// The current candidate rows, empty while no transaction is open. The slice is borrowed and + /// cannot be retained by preparation. #[must_use] pub fn candidate(&self) -> &[InitialParameterValue] { - &self.candidate + self.candidate.as_ref().map_or(&[], |open| &open.rows) } - /// Touched mask for the current candidate. + /// Touched mask for the current candidate, empty while no transaction is open. #[must_use] pub fn dirty(&self) -> &[bool] { - &self.dirty + self.candidate.as_ref().map_or(&[], |open| &open.dirty) } /// Computes the actual dynamic allocations retained by this owner. The factory allocation is @@ -177,15 +186,12 @@ impl EffectControlOwner { let owner_box = u64::try_from(core::mem::size_of::()) .map_err(|_| EffectControlResourceError::Arithmetic)?; let committed = checked_slice_bytes::(self.committed.len())?; - let candidate = checked_slice_bytes::(self.candidate.len())?; - let dirty = checked_slice_bytes::(self.dirty.len())?; - let mut owned_payload_bytes = owner_box; - owned_payload_bytes = owned_payload_bytes + // An open transaction's candidate is transient (built by `begin`, dropped by `commit` or + // `discard`), so it is no retained allocation. + let owned_payload_bytes = owner_box .checked_add(committed) - .and_then(|bytes| bytes.checked_add(candidate)) - .and_then(|bytes| bytes.checked_add(dirty)) .ok_or(EffectControlResourceError::Arithmetic)?; - let largest_owned_allocation_bytes = owner_box.max(committed).max(candidate).max(dirty); + let largest_owned_allocation_bytes = owner_box.max(committed); let header = Layout::new::(); let (strong_weak, _) = header @@ -211,8 +217,10 @@ impl EffectControlOwner { { return Err(EffectControlOwnerError::Revision); } - self.candidate.copy_from_slice(&self.committed); - self.dirty.fill(false); + self.candidate = Some(OpenCandidate { + rows: self.committed.clone(), + dirty: vec![false; self.committed.len()].into_boxed_slice(), + }); self.phase = EffectControlOwnerPhase::Open; Ok(()) } @@ -243,6 +251,10 @@ impl EffectControlOwner { channel: ParameterChannel, value: f32, ) -> Result<(), EffectControlOwnerError> { + let candidate = self + .candidate + .as_mut() + .ok_or(EffectControlOwnerError::Revision)?; let parameter = self .factory .descriptor() @@ -260,28 +272,28 @@ impl EffectControlOwner { | (ParameterChannelPolicy::PerLane, ParameterChannel::Left) | (ParameterChannelPolicy::PerLane, ParameterChannel::Right) => { let value = normalize_zero(value); - let row = self - .candidate + let row = candidate + .rows .iter() .position(|item| { item.parameter_index == parameter_index && item.channel == channel }) .ok_or(EffectControlOwnerError::Shape)?; - self.candidate[row].value = value; - self.dirty[row] = true; + candidate.rows[row].value = value; + candidate.dirty[row] = true; Ok(()) } (ParameterChannelPolicy::PerLane, ParameterChannel::Both) => { for lane in [ParameterChannel::Left, ParameterChannel::Right] { - let row = self - .candidate + let row = candidate + .rows .iter() .position(|item| { item.parameter_index == parameter_index && item.channel == lane }) .ok_or(EffectControlOwnerError::Shape)?; - self.candidate[row].value = normalize_zero(value); - self.dirty[row] = true; + candidate.rows[row].value = normalize_zero(value); + candidate.dirty[row] = true; } Ok(()) } @@ -292,10 +304,14 @@ impl EffectControlOwner { /// Borrows the complete candidate request for off-audio preparation. pub fn preparation_request(&self) -> Result, EffectControlOwnerError> { self.require_open()?; + let open = self + .candidate + .as_ref() + .ok_or(EffectControlOwnerError::Invariant)?; Ok(EffectTargetRequest { sample_rate: self.sample_rate, - values: &self.candidate, - changed: &self.dirty, + values: &open.rows, + changed: &open.dirty, }) } @@ -341,8 +357,7 @@ impl EffectControlOwner { ) { return Err(EffectControlOwnerError::Revision); } - self.candidate.copy_from_slice(&self.committed); - self.dirty.fill(false); + self.candidate = None; self.pending_revision = None; self.phase = EffectControlOwnerPhase::Idle; Ok(()) @@ -393,6 +408,7 @@ impl EffectControlOwner { .is_err() { self.phase = EffectControlOwnerPhase::Faulted; + self.candidate = None; self.pending_revision = None; return Err(EffectControlOwnerError::Invariant); } @@ -411,8 +427,11 @@ impl EffectControlOwner { .pending_revision .take() .ok_or(EffectControlOwnerError::Invariant)?; - self.committed.copy_from_slice(&self.candidate); - self.dirty.fill(false); + let open = self + .candidate + .take() + .ok_or(EffectControlOwnerError::Invariant)?; + self.committed.copy_from_slice(&open.rows); self.committed_revision = next_revision; self.phase = EffectControlOwnerPhase::Idle; Ok(self.committed_revision) @@ -446,23 +465,64 @@ mod tests { let mut owner = EffectControlOwner { factory: Arc::new(ParametricEqFactory), sample_rate: 48_000, - candidate: committed.clone(), - dirty: vec![false; committed.len()].into_boxed_slice(), + candidate: None, committed, committed_revision: u64::MAX, pending_revision: None, phase: EffectControlOwnerPhase::Idle, }; let committed_before = owner.committed.clone(); - let candidate_before = owner.candidate.clone(); - let dirty_before = owner.dirty.clone(); assert_eq!( owner.begin(u64::MAX), Err(EffectControlOwnerError::Revision) ); assert_eq!(owner.phase, EffectControlOwnerPhase::Idle); assert_eq!(owner.committed, committed_before); - assert_eq!(owner.candidate, candidate_before); - assert_eq!(owner.dirty, dirty_before); + assert!(owner.candidate.is_none(), "a refused begin builds nothing"); + } + + /// An idle owner retains only its committed rows: the candidate and the touched mask exist + /// from `begin` to `commit` or `discard` and nowhere else (#1624). + #[test] + fn the_candidate_lives_only_between_begin_and_commit_or_discard() { + let committed: Box<[InitialParameterValue]> = + default_initial_values(&PARAMETRIC_EQ_DESCRIPTOR).collect(); + let rows = committed.len(); + let mut owner = EffectControlOwner { + factory: Arc::new(ParametricEqFactory), + sample_rate: 48_000, + candidate: None, + committed, + committed_revision: 0, + pending_revision: None, + phase: EffectControlOwnerPhase::Idle, + }; + let idle_bytes = owner.resource_facts().expect("facts").owned_payload_bytes; + assert!(owner.candidate().is_empty() && owner.dirty().is_empty()); + + owner.begin(0).expect("begin"); + assert_eq!(owner.candidate(), &owner.committed[..]); + assert_eq!(owner.dirty().len(), rows); + assert!(owner.dirty().iter().all(|touched| !touched)); + owner.edit(2, ParameterChannel::Both, 100.0).expect("edit"); + assert!(owner.dirty().iter().any(|touched| *touched)); + owner.discard().expect("discard"); + assert!(owner.candidate().is_empty() && owner.dirty().is_empty()); + + // A second candidate starts from the committed rows, not from the discarded edit. + owner.begin(0).expect("begin again"); + assert_eq!(owner.candidate(), &owner.committed[..]); + assert!(owner.dirty().iter().all(|touched| !touched)); + // A poisoned candidate is still the open one until discarded. + assert!(owner.edit(9_999, ParameterChannel::Both, 0.0).is_err()); + assert_eq!(owner.phase(), EffectControlOwnerPhase::Poisoned); + assert_eq!(owner.dirty().len(), rows); + owner.discard().expect("discard poisoned"); + assert!(owner.candidate().is_empty()); + assert_eq!( + owner.resource_facts().expect("facts").owned_payload_bytes, + idle_bytes, + "the retained facts do not depend on a candidate" + ); } } diff --git a/crates/effect-compiler/src/prepare.rs b/crates/effect-compiler/src/prepare.rs index 575328eac..35c67b463 100644 --- a/crates/effect-compiler/src/prepare.rs +++ b/crates/effect-compiler/src/prepare.rs @@ -1297,8 +1297,7 @@ mod owner_tests { let values = preparation().initial_values.len(); let owner_box = size_of::() as u64; let row_backing = (values * size_of::()) as u64; - let dirty_backing = (values * size_of::()) as u64; - let owner_payload = owner_box + (2 * row_backing) + dirty_backing; + let owner_payload = owner_box + row_backing; // The EQ lane's writer holds one cell writer per `(slot, channel)` of its target cells and // none for parameters: its parameters ride its targets. let writer_backing = (2 @@ -1328,7 +1327,6 @@ mod owner_tests { "effect-b".len() as u64, owner_box, row_backing, - dirty_backing, writer_backing, factory_layout, ] @@ -1387,8 +1385,7 @@ mod owner_tests { } let values = preparation().initial_values.len(); let owner_payload = size_of::() as u64 - + (2 * (values * size_of::()) as u64) - + (values * size_of::()) as u64 + + (values * size_of::()) as u64 + (2 * EQ_SLOTS * size_of::>()) as u64; diff --git a/crates/host-core/tests/prepare.rs b/crates/host-core/tests/prepare.rs index 4952f7889..c93ce24e6 100644 --- a/crates/host-core/tests/prepare.rs +++ b/crates/host-core/tests/prepare.rs @@ -430,12 +430,7 @@ fn effect_control_report_uses_actual_native_capacity_strings_and_owners() { largest_owned = largest_owned.max(writer_array); let owner = producer.owner().expect("live EQ owner"); assert!(std::sync::Arc::ptr_eq(shared_factory, owner.factory())); - for bytes in [ - size_of_val(owner), - size_of_val(owner.committed()), - size_of_val(owner.candidate()), - size_of_val(owner.dirty()), - ] { + for bytes in [size_of_val(owner), size_of_val(owner.committed())] { payload += bytes as u64; largest_owned = largest_owned.max(bytes as u64); } From c60b78b842079d45d46d2061469dc0a2f58c1497 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 19:17:26 +0000 Subject: [PATCH 04/12] Size the per-node live effect window by its live cells (#1621) Co-Authored-By: Claude Opus 5.5 --- ...de-live-effect-window-by-its-live-cells.md | 46 +++++++ crates/effect-contract/src/lib.rs | 29 ++-- crates/effect-contract/src/live.rs | 8 +- crates/graph-compiler/src/estimate.rs | 16 +-- crates/graph-compiler/src/lib.rs | 23 ++-- .../graph-compiler/tests/bypass_resources.rs | 27 ++++ crates/graph/src/lib.rs | 125 ++++++++++++++---- crates/graph/src/runtime.rs | 66 ++++----- 8 files changed, 244 insertions(+), 96 deletions(-) diff --git a/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md b/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md index 2917ade30..3b7e44769 100644 --- a/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md +++ b/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md @@ -42,3 +42,49 @@ banked effect does since #1611, so an unbanked EQ (no parameter cells) holds no ## Size Half a day. + +## Decision record: #1621 (batch 14b) + +**Shipped.** `LiveControlEffect::new` (`crates/graph/src/runtime.rs`) sizes its staging window by +`control.parameter_cells()`: an unbanked EQ (prepared-target owner, no parameter cell) and a +channel-less lane hold no span storage; a compressor-shaped lane holds exactly its cells. The estimate +(`effect_control_resource`, `crates/graph-compiler/src/estimate.rs`) charges the same +`parameter_cells * size_of::()`. Per-lane arithmetic is untouched. + +**Departure (one, mechanical).** The bind-time check `EffectProcessBlock::check_automation_window` +demanded `window == automation_capacity`, which the cell-sized window cannot meet. The banked stage +already replaced that equality with "the lanes' cells never exceed the capacity" (#1611 D3). The same +rule now holds per node: `check_automation_window` (in `crates/effect-contract/src/lib.rs`) accepts a +window `<= automation_capacity`; the other half of the contract, that the window holds every cell the +lane reads (no drop, #1345 D4), stays `EffectControlLane::check_window` and still runs at bind. The +pairing-rule argument is unchanged: the hazard (#1012) was a window larger than the capacity, which +could stage a twin pair across the effect's `span_index < automation_capacity` cut-off; a window no +longer than the capacity cannot. Doc comments in `effect-contract/src/lib.rs` and `live.rs` say so. +Ack-before-drop: no. A cell-sized window holds one span per cell, `check_window` refuses a shorter +one at bind, and preparation raises `automation_capacity` to at least the descriptor's cells. + +**Gates and tests.** +- Gate 1: `graph` `tests::a_per_node_window_is_its_lanes_cells_not_the_capacity` (EQ-shaped lane: 0; + compressor-shaped lane: 4 = its cells; gain fixture lane: 2; channel-less lane: 0; capacity 4,096). + `graph-compiler` `bypass_resources::a_per_node_live_window_does_not_grow_with_the_automation_capacity` + (real EQ, compressor and limiter inserts; allocator-measured retained bytes and the estimate are equal + at 128 and 4,096 spans). Both are red when the window returns to the capacity (checked by mutation). +- Gate 2: `graph-compiler` `effect_control_resource_uses_independent_queue_and_owner_arithmetic` follows + the new window formula (scalar-target window 0). Its `expected.largest > plain` assertion for the + scalar case was removed: it held only because the capacity-sized window was the largest owner + allocation; no per-node owner allocation outgrows the plain plan's largest any more, and the + exact-cap accept/refuse legs and the `largest` equality stay. The charge-equals-freed test + `a_bypass_or_live_controls_are_charged_at_least_what_they_retain`, `bypass_shunt_identity` and + `bypass_cohorts` stay green. +- Gate 3: `cargo test --locked` for graph, graph-compiler, rack, effect-contract, effect-compiler, + control-plane, controls, host-core (all features), host-web (all features), capi (all, incl. + `resource_lifecycle`) green; clippy `-D warnings` and fmt clean; `realtime-policy` ok; + `audit capi` `pcm_digest` `bf7dcedd1d92621f`; `check-browser-expected-resources.py` green. + +**Measured numbers.** No pinned row or ceiling moved, so none was re-pinned. Before/after on the same +tree: `resource_lifecycle` nine-track EQ, soft-clip and browser-identity rows identical (EQ plan 218,401 +observed, ceilings unchanged); the browser fixture's eight budget rows identical (for example +`graphIncrementalPlanBytes` 30,154). The one printed row that moved is the unpinned "routed submix" +plan row printed by `capi_retained_bytes_charge_every_byte_the_compile_retains`: observed 104,589 -> +98,781 and its estimate 196,180 -> 190,372 (-5,808 each, slack 91,591 unchanged; the per-node EQ/ +compressor windows of that session, 128-span capacity). diff --git a/crates/effect-contract/src/lib.rs b/crates/effect-contract/src/lib.rs index 93a073cff..b3c4ea2d5 100644 --- a/crates/effect-contract/src/lib.rs +++ b/crates/effect-contract/src/lib.rs @@ -1659,18 +1659,17 @@ pub enum ProcessBlockError { ZeroFrames, Shape, Automation, - /// A live-control staging window whose span count is not the effect's - /// `automation_capacity`; refused at preparation by - /// [`EffectProcessBlock::check_automation_window`] (the per-node window). A banked live slot - /// sizes its window by its lanes' parameter cells instead and reports its own rack error. + /// A live-control staging window with more spans than the effect's `automation_capacity`; + /// refused at preparation by [`EffectProcessBlock::check_automation_window`]. A banked live + /// slot checks the same bound and reports its own rack error. AutomationWindow, } /// The one statement of the staging-window bound, shared by both block types. /// -/// See [`EffectProcessBlock::check_automation_window`] for why the bound is equality. +/// See [`EffectProcessBlock::check_automation_window`] for why the window may not exceed the capacity. const fn check_window(window: usize, automation_capacity: u32) -> Result<(), ProcessBlockError> { - if window as u64 == automation_capacity as u64 { + if window as u64 <= automation_capacity as u64 { Ok(()) } else { Err(ProcessBlockError::AutomationWindow) @@ -1714,14 +1713,15 @@ impl<'a> EffectProcessBlock<'a> { self.left.len() } - /// Refuse, at preparation, a live-control staging window that is not exactly this instance's - /// `automation_capacity` spans (issue #1004 finding 1, issue #1012). + /// Refuse, at preparation, a live-control staging window longer than this instance's + /// `automation_capacity` spans (issue #1004 finding 1, issue #1012, #1621). /// /// Every caller that owns the window it hands [`EffectControlLane::stage`] for a prepared /// instance -- `graph`'s per-node live-control effect today -- calls this once, off the render - /// thread, where it allocates the window. The blocks this type builds afterwards carry only the - /// staged prefix of that window, per block, on the render thread, and never see the window or - /// the capacity, so this is where the bound can be checked at all. + /// thread, where it allocates the window. The window is the lane's live parameter cells + /// (#1621 D1), which preparation keeps within the capacity. The blocks this type builds + /// afterwards carry only the staged prefix of that window, per block, on the render thread, and + /// never see the window or the capacity, so this is where the bound can be checked at all. /// /// # Why the bound is a precondition of the pairing rule /// @@ -1731,12 +1731,13 @@ impl<'a> EffectProcessBlock<'a> { /// one span at a time, so a window larger than the capacity could stage a twin across that /// cut-off: the effect would apply `Left p v` and refuse `Right p v`, the witness would still /// hold, and the mono collapse would render the left channel's state for the right one. A - /// window equal to the capacity cannot stage a span the effect refuses on capacity. A smaller - /// one would be sound for the pair but is refused too: it would drop a cell the lane read. + /// window no longer than the capacity cannot stage a span the effect refuses on capacity. A + /// shorter one is sound for the pair; that it holds every cell the lane reads is + /// [`EffectControlLane::check_window`]'s bound, so no cell is dropped. /// /// # Errors /// - /// [`ProcessBlockError::AutomationWindow`] when `window.len()` differs from + /// [`ProcessBlockError::AutomationWindow`] when `window.len()` exceeds /// `metadata.automation_capacity`. pub const fn check_automation_window( window: &[PreparedAutomationSpan], diff --git a/crates/effect-contract/src/live.rs b/crates/effect-contract/src/live.rs index 45821889e..1c95a252b 100644 --- a/crates/effect-contract/src/live.rs +++ b/crates/effect-contract/src/live.rs @@ -56,8 +56,8 @@ //! no deferral and no drop. So the staging window is at least [`EffectControlLane::parameter_cells`] //! spans, and every effect accepts that many: preparation raises an instance's //! `automation_capacity` to at least [`EffectLaneShape::live_parameter_cells`] of its descriptor, -//! the window is exactly that capacity (the pairing rule of issue #1012), and a window shorter than -//! the lane needs is refused at bind ([`EffectControlLane::check_window`]). +//! the window is the lane's cells, never past that capacity (the pairing rule of issue #1012, #1621), +//! and a window shorter than the lane needs is refused at bind ([`EffectControlLane::check_window`]). //! //! # Observation subscriptions stay a FIFO (D2) //! @@ -1201,8 +1201,8 @@ impl EffectControlLane { /// declaring what it does to the witness. /// /// `staging` is the caller's preallocated window for this lane, at least - /// [`Self::parameter_cells`] spans long (checked at bind with [`Self::check_window`]) and - /// exactly the effect's `automation_capacity`, which the pairing rule depends on + /// [`Self::parameter_cells`] spans long (checked at bind with [`Self::check_window`]) and no + /// longer than the effect's `automation_capacity`, which the pairing rule depends on /// ([`EffectProcessBlock::check_automation_window`](crate::EffectProcessBlock::check_automation_window)): /// a staged twin pair is never cut off by the effect's `span_index < automation_capacity` /// limit. diff --git a/crates/graph-compiler/src/estimate.rs b/crates/graph-compiler/src/estimate.rs index 50af6638d..58e08519c 100644 --- a/crates/graph-compiler/src/estimate.rs +++ b/crates/graph-compiler/src/estimate.rs @@ -176,8 +176,9 @@ pub(crate) fn resource_estimate( /// allocates beyond the live-control-free path, in both of its forms: /// /// * **Per node**, `graph`'s `runtime::LiveControlEffect`: the boxed owner, which the -/// live-control-free `NodeKind::Effect` does not allocate at all; the staging window of -/// `automation_capacity` spans, for a live channel only; and the shunt, always. +/// live-control-free `NodeKind::Effect` does not allocate at all; the staging window of the +/// lane's live parameter cells (#1621 D1; none for a channel-less lane or a prepared-target +/// owner such as the parametric EQ); and the shunt, always. /// * **Banked**, `rack::LiveControlEffectBankStage`: its growth over the `rack::EffectBankStage` it /// replaces; the one-lane staging window and the packed window of the widest lane's live /// parameter cells (#1611 D3; at most `automation_capacity`) spans @@ -237,12 +238,11 @@ pub(crate) fn effect_control_resource( total = total.checked_add(lane)?; largest = largest.max(lane); // The per-node live-control owner: `runtime::LiveControlEffect::new` at the render - // quantum. - let window = if control.has_channel() { - u64::from(entry.metadata.automation_capacity).checked_mul(span_bytes)? - } else { - 0 - }; + // quantum. Its staging window holds the lane's live parameter cells (#1621 D1), the + // banked window's rule: none for a channel-less lane or a prepared-target owner. + let window = u64::try_from(control.parameter_cells()) + .ok()? + .checked_mul(span_bytes)?; let frames = usize::try_from(entry.metadata.quantum).ok()?; let latency = usize::try_from(entry.metadata.latency.0).ok()?; let shunt = bytes(BypassShunt::allocated_bytes(frames, latency)?)?; diff --git a/crates/graph-compiler/src/lib.rs b/crates/graph-compiler/src/lib.rs index 4b94a90d5..7667b9092 100644 --- a/crates/graph-compiler/src/lib.rs +++ b/crates/graph-compiler/src/lib.rs @@ -3195,11 +3195,10 @@ mod tests { let banked = bank_width != 0; // Issue #1100: every controlled instance also gets a live-control owner around its // effect. Each lane here has a live channel, so each owner holds a staging window of - // the effect's automation capacity, and a shunt of two dry blocks and two latency - // lines. + // the lane's live parameter cells (#1621 D1; none for a prepared-target owner), and a + // shunt of two dry blocks and two latency lines. let metadata = &effects.entries[0].metadata; let span = size_of::(); - let capacity = metadata.automation_capacity as usize; let quantum = metadata.quantum as usize; let latency = usize::try_from(metadata.latency.0).expect("EQ latency"); if banked && controlled != 0 { @@ -3255,7 +3254,13 @@ mod tests { + size_of::() + size_of::>>() + size_of::(); - let window = capacity * span; + let window = if target_staging { + 0 + } else { + effect_contract::EffectLaneShape::live_parameter_cells( + effects.entries[0].factory.descriptor(), + ) * span + }; let shunt = 2 * (quantum + latency) * size_of::(); expected_total = expected_total .checked_add((owner + window + shunt) as u64 * controlled as u64) @@ -3383,16 +3388,6 @@ mod tests { ); if controlled == 0 { assert_eq!(expected.total, 0, "no controls retain no owner bytes"); - } else if label == "scalar-target" { - // The per-node owner's staging window is the effect's automation capacity - // (4,096 spans here), so it is the allocation that outgrows the plain plan's - // largest. A bank's window is the lanes' live cells (#1611 D3) and is empty for - // a prepared-target owner. - assert!( - expected.largest > plain_resource.largest_allocation_bytes, - "the high-cap control owner must exercise {estimate} largest-allocation \ - accounting" - ); } } diff --git a/crates/graph-compiler/tests/bypass_resources.rs b/crates/graph-compiler/tests/bypass_resources.rs index 33dd657b8..cc1ae1ba3 100644 --- a/crates/graph-compiler/tests/bypass_resources.rs +++ b/crates/graph-compiler/tests/bypass_resources.rs @@ -514,6 +514,33 @@ fn a_lanes_own_accounting_is_what_the_allocator_sees() { } } +/// #1621: a per-node live-control effect holds its lane's cells, not the automation capacity. +/// +/// The same single-track session, with live controls on its per-node EQ, compressor and limiter, is +/// bound at 128 and at 4,096 automation spans. Every effect's capacity is at least its descriptor's +/// cells, so the larger capacity changes no window: the audited allocator reads the same retained +/// bytes, and the estimate charges the same. Before #1621 each window was +/// `automation_capacity` spans (40 bytes each), so the EQ alone, which has no parameter cell, held +/// 160 KiB more at 4,096. +/// +/// Red mutation: size `LiveControlEffect::new`'s window, or the estimate's, by the capacity again +/// -> both readings grow with it. +#[test] +fn a_per_node_live_window_does_not_grow_with_the_automation_capacity() { + let model = inserts(1, [0, 0, 0]); + let small = bind(&model, Backend::current(), 128, Some(0)); + let large = bind(&model, Backend::current(), 4096, Some(0)); + assert_eq!(small.shape.banks, 0, "one track banks nothing"); + assert_eq!( + large.live, small.live, + "the plan retains more at a larger capacity, so a window follows the capacity" + ); + assert_eq!( + large.estimate, small.estimate, + "the estimate charges more at a larger capacity, so a window follows the capacity" + ); +} + /// Gate 2, observation: an effect that declares a tap gets a queue, and the estimate charges /// exactly what bind retains for it, banked and per node. /// diff --git a/crates/graph/src/lib.rs b/crates/graph/src/lib.rs index 22d3bff20..414d9123d 100644 --- a/crates/graph/src/lib.rs +++ b/crates/graph/src/lib.rs @@ -8284,6 +8284,33 @@ mod tests { } } + /// The gain fixture's prepared metadata, at the four-frame quantum. + fn gain_metadata(latency: u64, automation_capacity: u32) -> PreparedEffectMetadata { + PreparedEffectMetadata { + descriptor: &GAIN_DESCRIPTOR, + sample_rate: 48_000, + quantum: 4, + quality: EffectQuality::Normal, + bypass: false, + link_mode: LinkMode::DualMono, + ports: PreparedPorts { + sidechain: PreparedSidechainPort::None, + }, + latency: LatencySamples(latency), + tail: TailSamples::Finite(0), + tail_every_peak: TailSamples::Infinite, + rest: effect_contract::RestBound::Unstated, + composition: effect_contract::CompositionBound::Unstated, + state_sizes: StatePayloadSizes { + common_bytes: 0, + left_bytes: 0, + right_bytes: 0, + }, + scratch_bytes: 0, + automation_capacity, + } + } + /// One track: source input -> one dynamic-rack effect -> output, at a four-frame quantum. /// /// `control` is the consumer half of the effect's live-control channel, or `None` for the @@ -8311,29 +8338,7 @@ mod tests { quantum: QuantumFrames(4), output_channels: core::num::NonZeroUsize::new(2).expect("two"), }; - let metadata = PreparedEffectMetadata { - descriptor: &GAIN_DESCRIPTOR, - sample_rate: 48_000, - quantum: 4, - quality: EffectQuality::Normal, - bypass: false, - link_mode: LinkMode::DualMono, - ports: PreparedPorts { - sidechain: PreparedSidechainPort::None, - }, - latency: LatencySamples(latency), - tail: TailSamples::Finite(0), - tail_every_peak: TailSamples::Infinite, - rest: effect_contract::RestBound::Unstated, - composition: effect_contract::CompositionBound::Unstated, - state_sizes: StatePayloadSizes { - common_bytes: 0, - left_bytes: 0, - right_bytes: 0, - }, - scratch_bytes: 0, - automation_capacity: 2, - }; + let metadata = gain_metadata(latency, 2); let main_edge = GraphEdge { id: GraphEdgeId::TrackMain { target: effect_node.clone(), @@ -8493,6 +8498,80 @@ mod tests { (writer, Box::new(lane)) } + /// #1621 D1 / gate 1: a per-node live-control effect's staging window is its lane's live + /// parameter cells, whatever automation capacity the effect was prepared with. A lane with no + /// parameter cell -- the parametric EQ's shape, whose parameters ride prepared targets, and a + /// channel-less session bypass -- holds no span storage; a compressor-shaped lane (shared and + /// per-lane parameters) holds exactly its cells. Every window holds every cell the lane reads + /// and stays within the capacity, so no staged value is dropped and no twin pair is cut off. + /// + /// Test value: red if `LiveControlEffect::new` sizes the window by `automation_capacity` + /// again (4,096 spans here), the defect #1621 removes; the other lanes' assertions are the + /// bank stage's rule (#1611 D3) stated for the per-node owner. + #[test] + fn a_per_node_window_is_its_lanes_cells_not_the_capacity() { + use effect_contract::{EffectLaneShape, EffectLaneSpec, ParameterChannelPolicy}; + const CAPACITY: u32 = 4096; + let superseded = engine::realtime::LiveValuesSuperseded::new(); + let window = |lane: EffectControlLane| { + let effect_id = EffectNodeId { + track_id: StableGraphId::parse("track").expect("ID"), + rack: RackId::Dynamic, + effect_id: StableGraphId::parse("gain").expect("ID"), + }; + let metadata = gain_metadata(0, CAPACITY); + let node = runtime::LiveControlEffect::new( + GraphPreparedEffect { + id: effect_id, + metadata, + processor: Box::new(LiveGain::new(metadata)), + response_snapshot_declared: false, + native_id: "miso.test.live-gain", + }, + Box::new(lane), + None, + 4, + ); + node.window_spans() + }; + let live = |shape: EffectLaneShape| { + let (_writer, lane) = effect_contract::effect_control_lane( + EffectLaneSpec { + shape, + bypass: false, + observation_depth: 0, + }, + &superseded, + ) + .expect("lane"); + lane + }; + + // The parametric EQ's shape: no parameter cell, four prepared-target slots. + let eq = live(EffectLaneShape::from_live_parameters([], 4)); + assert_eq!(eq.parameter_cells(), 0); + assert_eq!(window(eq), 0, "an unbanked EQ holds no span window"); + + // A compressor's shape: two shared parameters and one per-lane one, four cells. + let compressor = live(EffectLaneShape::from_live_parameters( + [ + (0, ParameterChannelPolicy::Shared), + (1, ParameterChannelPolicy::PerLane), + (3, ParameterChannelPolicy::Shared), + ], + 0, + )); + assert_eq!(compressor.parameter_cells(), 4); + assert_eq!(window(compressor), 4, "a compressor's window is its cells"); + + // The gain fixture's descriptor-derived lane: one per-lane parameter. + let (_writer, gain) = control_pair(); + assert_eq!(window(*gain), 2); + + // A channel-less session bypass stages nothing. + assert_eq!(window(EffectControlLane::without_channel(true)), 0); + } + /// Control's end of the fixture: its cell writer and the plan's gate handle. A push is one /// transaction: stamp the next revision, write the cell, publish the revision. struct LiveControl { diff --git a/crates/graph/src/runtime.rs b/crates/graph/src/runtime.rs index 803d1cb4e..046029a98 100644 --- a/crates/graph/src/runtime.rs +++ b/crates/graph/src/runtime.rs @@ -1316,15 +1316,16 @@ impl EffectNode { /// One prepared native effect plus everything its live-control channel needs (issue #140 A). /// -/// Sized once, at bind, from the effect's own prepared metadata: the staging window is exactly -/// `PreparedEffectMetadata::automation_capacity` spans when the lane has a live channel, and empty -/// when it does not (issue #1100), and the shunt's delay line is exactly -/// `PreparedEffectMetadata::latency` samples. Render allocates nothing and frees nothing. +/// Sized once, at bind, from the effect's own prepared metadata and lane: the staging window is +/// exactly the lane's live parameter cells ([`EffectControlLane::parameter_cells`], #1621 D1) -- +/// empty for a lane with no channel (issue #1100) and for a prepared-target owner such as the +/// parametric EQ -- and the shunt's delay line is exactly `PreparedEffectMetadata::latency` +/// samples. Render allocates nothing and frees nothing. pub(crate) struct LiveControlEffect { pub(crate) effect: EffectNode, control: Box, - /// `automation_capacity` spans for a live channel, and none for a channel-less lane; only - /// `[..staged]` is ever handed to the effect. + /// One span per live parameter cell of the lane (none for a channel-less lane or a lane with no + /// cell, #1621); only `[..staged]` is ever handed to the effect. spans: Box<[PreparedAutomationSpan]>, /// Latency-preserving dry path, so live bypass keeps the effect's declared latency exactly /// and therefore leaves every compiled PDC route timing correct. @@ -1418,25 +1419,31 @@ impl LiveControlEffect { }) } - fn new( + /// The spans of this instance's staging window (#1621). + #[cfg(test)] + pub(crate) fn window_spans(&self) -> usize { + self.spans.len() + } + + pub(crate) fn new( effect: GraphPreparedEffect, control: Box, observation: Option>, frames: usize, ) -> Self { - // Issue #1100, the per-node twin of the rack's #1087 fix: only a live channel ever stages - // a span. A channel-less lane -- a session bypass with no live controls attached -- drains - // nothing into an empty window, and there is no twin pair for the pairing rule to protect. - // The window is the effect's automation capacity, which a caller may set as high as - // `u32::MAX` spans, so this is what keeps a live-control-free bypass from costing - // live-control staging, or aborting bind. - let live = control.has_channel(); - let capacity = if live { - effect.metadata.automation_capacity as usize - } else { - 0 - }; - let latency = usize::try_from(effect.metadata.latency.0).unwrap_or(usize::MAX); + // Issue #1100, the per-node twin of the rack's #1087 fix, and #1621 D1, the per-node twin of + // the bank stage's #1611 D3: the window is what the lane can stage, not the effect's + // `automation_capacity`. A lane stages at most one span per live parameter cell + // ([`EffectControlLane::parameter_cells`]), a channel-less lane has no cell, and a + // prepared-target owner (the parametric EQ) has none either -- its parameters ride its + // targets -- so those hold no span storage at all, however large a caller set the capacity + // (up to `u32::MAX` spans). A window that holds every cell drops none (#1345 D4), and one + // no longer than the capacity can never stage a twin pair across the effect's + // `span_index < automation_capacity` cut-off, which the live drain's pairing rule depends + // on (issue #1012). Both are checked here, at bind, off the render thread; this node + // builder is infallible, so a violation is a bind-time panic, as `stage_for`'s validated + // width is. + let capacity = control.parameter_cells(); let spans: Box<[PreparedAutomationSpan]> = vec![ PreparedAutomationSpan { channel: effect_contract::ParameterChannel::Both, @@ -1447,19 +1454,12 @@ impl LiveControlEffect { capacity ] .into_boxed_slice(); - // The live drain's pairing rule needs the window to be exactly the effect's automation - // capacity (issue #1012), and a block stages one span for every live parameter cell it - // reads, with no deferral (#1345 D4), so the window holds the lane's cells. Checked here, - // at bind, off the render thread; this node builder is infallible, so a violation is a - // bind-time panic, as `stage_for`'s validated width is. A channel-less lane stages - // nothing. - if live { - effect_contract::EffectProcessBlock::check_automation_window(&spans, &effect.metadata) - .expect("a live-control effect's staging window is its automation capacity"); - control - .check_window(spans.len()) - .expect("a live-control effect's staging window holds every live parameter cell"); - } + effect_contract::EffectProcessBlock::check_automation_window(&spans, &effect.metadata) + .expect("a live-control effect's staging window is within its automation capacity"); + control + .check_window(spans.len()) + .expect("a live-control effect's staging window holds every live parameter cell"); + let latency = usize::try_from(effect.metadata.latency.0).unwrap_or(usize::MAX); let layout = rack::EffectLaneLayout::of(&effect.metadata.program_key()); Self { observation, From 2831fe90de4bb960217e86f8e29965a06e155e76 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 19:28:30 +0000 Subject: [PATCH 05/12] Bound the first live commit of a large session: in-place admission of added parameter entries (#1620, ruling C1) Co-Authored-By: Claude Opus 5.5 --- ...he-first-live-commit-of-a-large-session.md | 50 ++++++ crates/control-plane/src/control.rs | 9 +- crates/host-core/tests/in_place_commit.rs | 150 ++++++++++++++++++ crates/protocol/src/model.rs | 39 ++--- crates/session/src/compile.rs | 5 +- crates/session/src/estimate.rs | 70 +++++++- crates/session/src/in_place.rs | 83 ++++++---- crates/session/src/lib.rs | 2 +- crates/session/tests/in_place.rs | 71 ++++++--- .../tests/control_command_heap.rs | 37 +++-- 10 files changed, 418 insertions(+), 98 deletions(-) diff --git a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md index dddf01c82..f45ae3fc2 100644 --- a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md +++ b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md @@ -102,3 +102,53 @@ work stopped here; no code changed. model and canonical snapshot equal the full path's, bit for bit. (2) The first-call peak at 64 and 256 tracks is measured and recorded, with a ceiling near the steady peak. (3) The first call's allocation count does not depend on the track count. + +### Implementation (batch 14b) + +**What shipped (C1).** `session::owners_counts_equal` is replaced by `session::owners_growth` +(`crates/session/src/in_place.rs`): `Some(OwnersGrowth)` when the edited owners hold as many +console entries, inserts and VCA members as the committed ones and no effect's parameter list is +shorter, with the entries added and the longest grown list; `None` for any other change of a count +(a shorter list included, so every item the estimate takes a maximum over only grows). The new +`ResourceEstimate::after_owners_edit` (`crates/session/src/estimate.rs`) charges the growth exactly +as `estimate_session_resources` and `with_canonical_bytes` do: per added entry one +`size_of::()` in the model and one structural item (1,024 bytes, now the named +constant `CANONICAL_BYTES_PER_ITEM`) in the canonical bound; the single largest allocation is the +largest of the committed one, the new canonical bound and the grown lists. It runs the compile's +own `check_caps` (now `pub(crate)`, its unused model argument dropped) and the platform and +canonical-bound checks on the result. `SessionStore::prepare_in_place` +(`crates/protocol/src/model.rs`) uses both and passes the store's caps: a growth that does not fit +returns `None` and the full path gives its typed refusal before any commit (no ack before a drop). +The entries land where the full path puts them: `controls`' parameter write already keeps the list +in canonical `(ID, channel)` order, the order `compile_session` sorts into. + +Outside the listed paths (mechanical): the in-place admission comment in +`crates/control-plane/src/control.rs` (the in-place transaction may now add parameter entries, never +a count `check_counts` reads); `crates/session/tests/in_place.rs` moved to the new function; the +differential gate lives in `crates/host-core/tests/in_place_commit.rs`, the home of the #1613 `Pair` +harness that compares the two paths. + +**Gates.** + +1. Differential: `in_place_commit::edits_that_add_parameter_entries_commit_in_place_as_the_full_path_does` + (console EQ above IDs 1 to 6 on one lane and both; an empty insert EQ list gaining an entry, then + one below and one between; a split of a `both` entry; one transaction over seven owners of tracks + and the submix, console and inserts; an abandoned one). Every transaction is prepared and + classified in place and the stores (model, canonical snapshot, estimate, revision) equal the + full path's after each. Refusal: `an_added_entry_over_the_caps_is_refused_before_any_commit` + (a store whose compiled-model or single-allocation cap is the committed figure: a fader commits + in place, an added entry is refused by both entries with the same `Validation` error and the + store is unchanged). Session unit: `owners_with_values_the_compiler_refuses_are_not_valid_and_counts_are_compared`. +2. `tools/parameter-metadata/tests/control_command_heap.rs` (debug): the first call is gated at + every size, `FIRST_PEAK_CEILING` 74,000 and `FIRST_REQUESTED_CEILING` 105,000 (was 78,000 peak + at one track only); `first.left <= 256` (was 4,096). Re-pinned for #1620 from these + measurements, on this head before and after the change: + + | tracks | first call before: allocations / requested / peak / left | first call after | steady (unchanged) | + |---|---|---|---| + | 1 | 839 / 130,294 / 77,600 / 1,078 | 757 / 103,033 / 72,865 / 80 | 758 / 103,918 / 72,841 / 80 | + | 64 | 1,853 / 550,437 / 198,182 / 1,078 | 757 / 103,033 / 72,865 / 80 | the same | + | 256 | 4,927 / 1,867,749 / 663,179 / 1,078 | 757 / 103,033 / 72,865 / 80 | the same | + + The steady ceilings (73,000 peak, 104,000 requested) are kept. +3. The same test asserts the first call's allocation count equal at 1, 64 and 256 tracks. diff --git a/crates/control-plane/src/control.rs b/crates/control-plane/src/control.rs index 000ba72eb..9e74a50fe 100644 --- a/crates/control-plane/src/control.rs +++ b/crates/control-plane/src/control.rs @@ -2411,10 +2411,11 @@ where let transaction = prepared.get().prospective_session(); let committed = self.controller.session().compiled(); let models = if transaction.is_in_place() { - // #1613: a transaction prepared in place changes values, never a count (the - // preparation proved it), so the counts a rebuild would check are those of the - // committed model, which passed them when it was admitted. The model charge is the one - // the full path would make: the committed estimate and the next one. + // #1613: a transaction prepared in place changes values and at most adds parameter + // entries (#1620), never a count a rebuild checks (the preparation proved it), so those + // counts are the committed model's, which passed them when it was admitted. The model + // charge is the one the full path would make: the committed estimate and the next one, + // which charges the added entries. let next = transaction .in_place_estimate() .unwrap_or_else(|| committed.resource_estimate()); diff --git a/crates/host-core/tests/in_place_commit.rs b/crates/host-core/tests/in_place_commit.rs index e105e9f4f..b8f72442e 100644 --- a/crates/host-core/tests/in_place_commit.rs +++ b/crates/host-core/tests/in_place_commit.rs @@ -455,3 +455,153 @@ fn what_the_in_place_preparation_cannot_prove_is_prepared_by_the_full_path() { ); } } + +/// #1620, gate 1. Transactions whose `SetControl`s add parameter entries to sparse lists: on the +/// console EQ that holds IDs 1 to 6 (above them, one lane and both), on an insert EQ that holds +/// none (then below and between the entries it gained), splitting a `both` entry into two lanes, +/// and across several effects of tracks and a submix, console and inserts, in one transaction. +/// Each is prepared in place, classified from its rows and committed; the stores (model, canonical +/// snapshot, estimate, revision) equal the full path's after each. +/// +/// Red if an added entry is charged differently from a compile (estimate), lands where the full +/// path's sort does not put it (model, snapshot), or sends the transaction to the full path. +#[test] +fn edits_that_add_parameter_entries_commit_in_place_as_the_full_path_does() { + let mut pair = Pair::new(rich_session(10)); + let mut tally = Tally::default(); + let row = |strip: &str, section: Section, control: u32, lane: Lane, value: f32| { + set( + address(Owner::Strip(id(strip)), section, control, lane), + value, + ) + }; + let console = |strip: &str, slot: &str, control: u32, lane: Lane, value: f32| { + row(strip, Section::Console(id(slot)), control, lane, value) + }; + let insert = |strip: &str, effect: &str, control: u32, lane: Lane, value: f32| { + row(strip, Section::Inserts(id(effect)), control, lane, value) + }; + let script: Vec<(Vec, bool)> = vec![ + // Above the console EQ's IDs 1 to 6: a `both` entry and a one-lane entry. + ( + vec![ + console("t0000", "eq", 66, Lane::Both, 400.0), + console("t0000", "eq", 82, Lane::Left, 6_000.0), + ], + true, + ), + // An empty insert list gains an entry, then one below and one between. + (vec![insert("t0000", "eq", 82, Lane::Both, 6_000.0)], true), + ( + vec![ + insert("t0000", "eq", 4, Lane::Right, -3.0), + insert("t0000", "eq", 66, Lane::Left, 300.0), + ], + true, + ), + // A one-lane write on a `both` entry splits it. + (vec![insert("t0000", "eq", 82, Lane::Left, 5_000.0)], true), + // Abandoned: the store stays as it was. + (vec![insert("t0000", "eq", 3, Lane::Left, 200.0)], false), + // Several effects at once: tracks and the submix, console and inserts, both lanes. + ( + vec![ + insert("t0001", "comp", 1, Lane::Left, -30.0), + insert("t0002", "gate", 1, Lane::Both, 0.0), + console("t0003", "post-clip", 1, Lane::Both, 12.0), + console("bus", "eq", 4, Lane::Right, -2.0), + insert("t0004", "multi", 3, Lane::Both, -40.0), + console("t0009", "eq", 82, Lane::Both, 7_000.0), + set(strip_row("t0005", FADER_DB, Lane::Both), -4.0), + ], + true, + ), + // The other lane of an entry that exists on one lane only. + (vec![insert("t0001", "comp", 1, Lane::Right, -20.0)], true), + ]; + let mut committed = 0; + for (edits, commit) in &script { + let before = pair.full.compiled().resource_estimate().parameter_count; + let prepared = prepare_in_place(&pair.fast, edits).expect("prepares"); + assert!(prepared.is_in_place(), "{edits:?} is prepared in place"); + assert!( + prepared + .in_place_estimate() + .expect("in place") + .parameter_count + > before, + "{edits:?} adds parameter entries" + ); + drop(prepared); + pair.step(edits, &["added"], *commit, &mut tally); + committed += usize::from(*commit); + } + assert_eq!( + tally.in_place, + script.len(), + "every transaction was prepared and classified in place" + ); + assert_eq!( + (tally.full_at_preparation, tally.full_at_classification), + (0, 0) + ); + assert_eq!(tally.abandoned, script.len() - committed); + pair.assert_equal("after the script"); +} + +/// #1620: an added entry is charged against the store's caps before anything is committed. With a +/// cap at the committed session's figure, a change of values still commits in place, and an edit +/// that adds an entry is refused, by both entries alike, with the store unchanged. +/// +/// Red if the in-place path commits an added entry that a compile's caps refuse (an ack the full +/// path would not give). +#[test] +fn an_added_entry_over_the_caps_is_refused_before_any_commit() { + let model = rich_session(10); + let estimate = store(model.clone()).compiled().resource_estimate(); + // The compile's figures before the canonical text is written, which its caps bound. + let upper_bound = estimate.compiled_model_bytes - estimate.canonical_bytes + + estimate.canonical_upper_bound_bytes; + let tight = [ + session::CompileCaps { + max_compiled_model_bytes: upper_bound, + ..caps() + }, + session::CompileCaps { + max_single_allocation_bytes: estimate.single_allocation_bytes, + ..caps() + }, + ]; + let fader = set(strip_row("t0005", FADER_DB, Lane::Both), -4.0); + let added = set( + address( + Owner::Strip(id("t0000")), + Section::Console(id("eq")), + 66, + Lane::Both, + ), + 400.0, + ); + for caps in tight { + let mut store = SessionStore::new(model.clone(), caps).expect("compiles at its caps"); + let current = store.revision(); + let revision = ExpectedRevision::Exact(current); + assert!( + prepare_in_place(&store, std::slice::from_ref(&fader)) + .expect("a fader moves") + .is_in_place() + ); + let edits = [fader.clone(), added.clone()]; + let full = store.prepare_transaction_with_monitor(revision, &edits); + assert!( + matches!(full, Err(protocol::SessionStoreError::Validation { .. })), + "the compile refuses the added entry: {:?}", + full.as_ref().err() + ); + assert_eq!(prepare_in_place(&store, &edits).err(), full.err()); + let before = store.canonical_snapshot().to_owned(); + assert!(store.apply_transaction_in_place(revision, &edits).is_err()); + assert_eq!(store.revision(), current); + assert_eq!(store.canonical_snapshot(), before); + } +} diff --git a/crates/protocol/src/model.rs b/crates/protocol/src/model.rs index 9ab9d264f..d2923b01e 100644 --- a/crates/protocol/src/model.rs +++ b/crates/protocol/src/model.rs @@ -958,7 +958,8 @@ struct InPlaceEdit { /// The revision the commit gives the model. revision: u64, /// The resource estimate of the model after the commit: the committed one with the canonical - /// text's length moved by the change of the edited owners and of the revision's digits. + /// text's length moved by the change of the edited owners and of the revision's digits, and the + /// parameter entries the edits added charged (#1620). estimate: session::ResourceEstimate, /// The transaction's edits, for [`PreparedSessionTransaction::materialize`]. edits: Vec, @@ -1216,9 +1217,11 @@ impl SessionStore { /// - every edit is a `SetControl`, on a model row of a strip, a route or a VCA, or on a solo row; /// - `controls::set` accepts each edit in order, and the edited owners pass the validators of /// `compile_session` (`session::owners_values_valid`); - /// - the edited owners hold as many console entries, inserts, parameter entries and members as - /// the committed ones, so the resource estimate counts the same and the canonical text keeps - /// its structure (a parameter entry that does not exist yet is the full path); + /// - the edited owners hold as many console entries, inserts and members as the committed ones, + /// and no fewer parameter entries in any effect; an entry a `SetControl` adds to a sparse list + /// is charged exactly in the estimate (`session::owners_growth`, + /// `ResourceEstimate::after_owners_edit`, #1620), which must meet the store's caps as a + /// compile's would; /// - no ramp is longer than a second, and the canonical text stays within its preflight bound. /// /// # Errors @@ -1475,14 +1478,13 @@ impl SessionStore { }); } let (owners, slots) = owners.finish(); - if !session::owners_values_valid(slots.in_owners(&owners)) - || !session::owners_counts_equal( - slots.in_committed(committed), - slots.in_owners(&owners), - ) - { + if !session::owners_values_valid(slots.in_owners(&owners)) { return None; } + // Added parameter entries are charged exactly (#1620); any other change of a count is the + // full path's. + let growth = + session::owners_growth(slots.in_committed(committed), slots.in_owners(&owners))?; let estimate = self.compiled.resource_estimate(); let before = session::canonical_owner_bytes(slots.in_committed(committed)); let after = session::canonical_owner_bytes(slots.in_owners(&owners)); @@ -1494,20 +1496,9 @@ impl SessionStore { .checked_add(u64::try_from(after).ok()?)? .checked_sub(digits(current.0))? .checked_add(digits(revision))?; - // The full path refuses a canonical text over its preflight bound. - if canonical_bytes > estimate.canonical_upper_bound_bytes { - return None; - } - let compiled_model_bytes = estimate - .compiled_model_bytes - .checked_sub(estimate.canonical_bytes)? - .checked_add(canonical_bytes)?; - let estimate = session::ResourceEstimate { - canonical_bytes, - compiled_model_bytes, - estimated_bytes: compiled_model_bytes.checked_add(estimate.requested_runtime_bytes)?, - ..estimate - }; + // `None` where the full path refuses: a cap, the canonical text over its preflight bound, + // an overflow. The full path then gives the refusal, before anything is committed. + let estimate = estimate.after_owners_edit(growth, canonical_bytes, self.caps)?; Some(PreparedSessionTransaction { base_revision: current, form: TransactionForm::InPlace(InPlaceEdit { diff --git a/crates/session/src/compile.rs b/crates/session/src/compile.rs index 6a51dc7e6..00ac8a112 100644 --- a/crates/session/src/compile.rs +++ b/crates/session/src/compile.rs @@ -132,7 +132,7 @@ pub fn compile_session( caps: CompileCaps, ) -> Result { let estimate = estimate_session(session)?; - check_caps(session, estimate, caps)?; + check_caps(estimate, caps)?; validate_session(session)?; let canonical_json = write_canonical(session); let estimate = with_canonical_bytes(estimate, canonical_json.len())?; @@ -213,8 +213,7 @@ pub fn compile_session( resource_estimate: estimate, }) } -fn check_caps( - _session: &SessionModel, +pub(crate) fn check_caps( estimate: ResourceEstimate, caps: CompileCaps, ) -> Result<(), DiagnosticSet> { diff --git a/crates/session/src/estimate.rs b/crates/session/src/estimate.rs index 8678b5841..70ce50435 100644 --- a/crates/session/src/estimate.rs +++ b/crates/session/src/estimate.rs @@ -50,6 +50,10 @@ pub struct ResourceEstimate { pub estimated_bytes: u64, } +/// The canonical-writer bound's allowance for one structural item (an entity, a route, a VCA +/// member, an effect, a parameter entry, an automation or a segment). +const CANONICAL_BYTES_PER_ITEM: u64 = 1_024; + /// Estimate issue-004-owned model/runtime resources without cloning or canonicalizing the model. pub fn estimate_session_resources( session: &SessionModel, @@ -218,7 +222,12 @@ pub fn estimate_session_resources( 4_096, checked_add( checked_mul(retained_string_bytes, 10, "$.canonical", &mut errors), - checked_mul(structural_items, 1_024, "$.canonical", &mut errors), + checked_mul( + structural_items, + CANONICAL_BYTES_PER_ITEM, + "$.canonical", + &mut errors, + ), "$.canonical", &mut errors, ), @@ -300,6 +309,65 @@ pub fn estimate_session_resources( pub(crate) use estimate_session_resources as estimate_session; +impl ResourceEstimate { + /// The estimate of a committed session after an in-place commit of values (#1613) that added + /// `growth`'s parameter entries (#1620), with the canonical text now `canonical_bytes` long: + /// exactly what [`crate::compile_session`] of the edited model gives, or `None` where it would + /// refuse (a cap, the canonical bound, an overflow) or where this cannot be exact. + /// + /// `self` is the committed session's estimate and `growth` comes from + /// [`crate::owners_growth`] over the edited owners, so every other count is the committed one. + /// An added parameter entry adds one entry to its effect's list, one structural item to the + /// canonical bound and nothing else; the largest single allocation is the largest of the + /// committed one, the new canonical bound and the grown lists, since no item shrank. + #[must_use] + pub fn after_owners_edit( + &self, + growth: crate::OwnersGrowth, + canonical_bytes: u64, + caps: crate::CompileCaps, + ) -> Option { + let parameters = u64::try_from(growth.parameters).ok()?; + let entry = size::(); + let canonical_upper_bound_bytes = self + .canonical_upper_bound_bytes + .checked_add(parameters.checked_mul(CANONICAL_BYTES_PER_ITEM)?)?; + // The compile's bound before the canonical text is written: the committed one, which + // counted the committed canonical bound in place of the text, plus the added entries and + // their structural items. + let compiled_model_upper_bound = self + .compiled_model_bytes + .checked_sub(self.canonical_bytes)? + .checked_add(canonical_upper_bound_bytes)? + .checked_add(parameters.checked_mul(entry)?)?; + let largest_grown_list = u64::try_from(growth.largest_grown_list) + .ok()? + .checked_mul(entry)?; + let single_allocation_bytes = self + .single_allocation_bytes + .max(canonical_upper_bound_bytes) + .max(largest_grown_list); + let platform = u64::try_from(isize::MAX).unwrap_or(u64::MAX); + for bytes in [compiled_model_upper_bound, single_allocation_bytes] { + if usize::try_from(bytes).is_err() || bytes > platform { + return None; + } + } + let preflight = Self { + parameter_count: self.parameter_count.checked_add(parameters)?, + canonical_upper_bound_bytes, + canonical_bytes: 0, + compiled_model_bytes: compiled_model_upper_bound, + single_allocation_bytes, + estimated_bytes: compiled_model_upper_bound + .checked_add(self.requested_runtime_bytes)?, + ..*self + }; + crate::compile::check_caps(preflight, caps).ok()?; + with_canonical_bytes(preflight, usize::try_from(canonical_bytes).ok()?).ok() + } +} + pub(crate) fn with_canonical_bytes( mut estimate: ResourceEstimate, actual: usize, diff --git a/crates/session/src/in_place.rs b/crates/session/src/in_place.rs index 160f7bd40..1b293f6f1 100644 --- a/crates/session/src/in_place.rs +++ b/crates/session/src/in_place.rs @@ -1,10 +1,10 @@ //! The pieces of a value-only commit that the session owns (#1613): which owners an edit -//! touched, whether their values are valid, whether their counts are those of the owners they -//! replace, and how many bytes they take in the canonical text. +//! touched, whether their values are valid, what they add to the counts of the owners they replace +//! (parameter entries only, #1620), and how many bytes they take in the canonical text. //! //! A host that changes the value of a control writes it into a **clone of the owner** (a track, a //! submix, a route or a VCA), checks the clone with [`owners_values_valid`] and -//! [`owners_counts_equal`], measures the change of the canonical text with +//! [`owners_growth`], measures the change of the canonical text with //! [`canonical_owner_bytes`], and installs the clones with [`CompiledSession::swap_owners`]. The //! rest of the compiled session is the same value, so nothing else is cloned, validated or written. @@ -117,39 +117,62 @@ pub fn owners_values_valid<'a>(owners: impl IntoIterator>) - owners.into_iter().all(crate::validate::owner_values_valid) } -/// Whether the owners of `old` and `new`, pairwise, hold the same number of everything the resource -/// estimate counts: console entries, inserts, parameter entries per effect, VCA members. An edit that -/// adds or removes one is not a change of values. +/// What an edit of values adds to the counts of the resource estimate (#1620): parameter entries, +/// and nothing else. A `SetControl` of a parameter that an effect's sparse list does not hold yet +/// adds its entry (a one-lane write on a `both` entry splits it into two). +#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] +pub struct OwnersGrowth { + /// Parameter entries added across the edited owners. + pub parameters: usize, + /// The length of the longest parameter list that grew, or zero. + pub largest_grown_list: usize, +} + +/// What the owners of `new` add to those of `old`, pairwise, to the counts the resource estimate +/// keeps: `Some` when they hold the same number of console entries, inserts and VCA members and no +/// effect's parameter list is shorter, with the parameter entries added; `None` for any other change +/// of a count, which is not an edit of values. [`ResourceEstimate::after_owners_edit`] charges the +/// growth. +/// +/// [`ResourceEstimate::after_owners_edit`]: crate::ResourceEstimate::after_owners_edit #[must_use] -pub fn owners_counts_equal<'a>( +pub fn owners_growth<'a>( old: impl IntoIterator>, new: impl IntoIterator>, -) -> bool { +) -> Option { let (mut old, mut new) = (old.into_iter(), new.into_iter()); + let mut growth = OwnersGrowth::default(); loop { match (old.next(), new.next()) { - (None, None) => return true, - (Some(old), Some(new)) if counts_equal(old, new) => {} - _ => return false, + (None, None) => return Some(growth), + (Some(old), Some(new)) => grow(&mut growth, old, new)?, + _ => return None, } } } -fn counts_equal(old: OwnerRef<'_>, new: OwnerRef<'_>) -> bool { - let strip = |old: crate::StripRef<'_>, new: crate::StripRef<'_>| { - old.console.len() == new.console.len() - && old - .console - .iter() - .zip(new.console) - .all(|(old, new)| old.params.len() == new.params.len()) - && old.inserts.effects.len() == new.inserts.effects.len() - && old - .inserts - .effects - .iter() - .zip(&new.inserts.effects) - .all(|(old, new)| old.params.len() == new.params.len()) +fn grow(growth: &mut OwnersGrowth, old: OwnerRef<'_>, new: OwnerRef<'_>) -> Option<()> { + let mut list = |old: &[crate::EffectParam], new: &[crate::EffectParam]| { + let added = new.len().checked_sub(old.len())?; + if added != 0 { + growth.parameters = growth.parameters.checked_add(added)?; + growth.largest_grown_list = growth.largest_grown_list.max(new.len()); + } + Some(()) + }; + let mut strip = |old: crate::StripRef<'_>, new: crate::StripRef<'_>| { + if old.console.len() != new.console.len() + || old.inserts.effects.len() != new.inserts.effects.len() + { + return None; + } + for (old, new) in old.console.iter().zip(new.console) { + list(&old.params, &new.params)?; + } + for (old, new) in old.inserts.effects.iter().zip(&new.inserts.effects) { + list(&old.params, &new.params)?; + } + Some(()) }; match (old, new) { (OwnerRef::Track(old), OwnerRef::Track(new)) => { @@ -158,9 +181,11 @@ fn counts_equal(old: OwnerRef<'_>, new: OwnerRef<'_>) -> bool { (OwnerRef::Submix(old), OwnerRef::Submix(new)) => { strip(crate::StripRef::submix(old), crate::StripRef::submix(new)) } - (OwnerRef::Vca(old), OwnerRef::Vca(new)) => old.members.len() == new.members.len(), - (OwnerRef::Route(_), OwnerRef::Route(_)) => true, - _ => false, + (OwnerRef::Vca(old), OwnerRef::Vca(new)) => { + (old.members.len() == new.members.len()).then_some(()) + } + (OwnerRef::Route(_), OwnerRef::Route(_)) => Some(()), + _ => None, } } diff --git a/crates/session/src/lib.rs b/crates/session/src/lib.rs index 82593e7c2..ff33ad57c 100644 --- a/crates/session/src/lib.rs +++ b/crates/session/src/lib.rs @@ -31,7 +31,7 @@ pub use edit_ramp::{EditRampRow, EditRampTarget, EditRamps}; pub use estimate::{ResourceEstimate, estimate_session_resources}; pub use id::{STABLE_ID_MAXIMUM_BYTES, StableId}; pub use in_place::{ - OwnerRef, OwnerSlots, canonical_owner_bytes, owners_counts_equal, owners_values_valid, + OwnerRef, OwnerSlots, OwnersGrowth, canonical_owner_bytes, owners_growth, owners_values_valid, }; pub use model::*; pub use parse::parse_session_json; diff --git a/crates/session/tests/in_place.rs b/crates/session/tests/in_place.rs index 2395b4861..3613020b1 100644 --- a/crates/session/tests/in_place.rs +++ b/crates/session/tests/in_place.rs @@ -8,9 +8,9 @@ //! `owners_values_valid`. use session::{ - CompileCaps, DualMonoFader, MatrixOrPan, OwnerRef, OwnerSlots, SessionModel, StableId, Submix, - Vca, canonical_owner_bytes, canonical_session_json, compile_session, owners_counts_equal, - owners_values_valid, parse_session_json, + CompileCaps, DualMonoFader, MatrixOrPan, OwnerRef, OwnerSlots, OwnersGrowth, SessionModel, + StableId, Submix, Vca, canonical_owner_bytes, canonical_session_json, compile_session, + owners_growth, owners_values_valid, parse_session_json, }; const EXAMPLE: &str = include_str!("../../../fixtures/session/v1/canonical.json"); @@ -124,10 +124,13 @@ fn swapping_the_owners_in_gives_the_compiled_session_of_the_edited_model() { let recompiled = compile_session(&edited, caps()).expect("compiles"); let (slots, mut owners) = slots(compiled.normalized_model(), &edited); assert!(owners_values_valid(slots.in_owners(&owners))); - assert!(owners_counts_equal( - slots.in_committed(compiled.normalized_model()), - slots.in_owners(&owners) - )); + assert_eq!( + owners_growth( + slots.in_committed(compiled.normalized_model()), + slots.in_owners(&owners) + ), + Some(OwnersGrowth::default()) + ); compiled.swap_owners(&mut owners, &slots); compiled.finish_in_place( recompiled.normalized_model().revision, @@ -153,7 +156,8 @@ fn swapping_the_owners_in_gives_the_compiled_session_of_the_edited_model() { } /// The values the compiler refuses are refused by `owners_values_valid`, one owner kind at a time, -/// and a clone with another count of anything is not a change of values. +/// and a clone with another count of anything but added parameter entries is not a change of +/// values. #[test] fn owners_with_values_the_compiler_refuses_are_not_valid_and_counts_are_compared() { let model = model(); @@ -191,18 +195,41 @@ fn owners_with_values_the_compiler_refuses_are_not_valid_and_counts_are_compared let mut more = model.vcas[0].clone(); more.members.push(id("extra")); - assert!(!owners_counts_equal( - [OwnerRef::Vca(&model.vcas[0])], - [OwnerRef::Vca(&more)] - )); - let mut track = model.tracks[0].clone(); - track.inserts.effects[0].params.clear(); - assert!(!owners_counts_equal( - [OwnerRef::Track(&model.tracks[0])], - [OwnerRef::Track(&track)], - )); - assert!(owners_counts_equal( - [OwnerRef::Track(&model.tracks[0])], - [OwnerRef::Track(&model.tracks[0].clone())], - )); + assert_eq!( + owners_growth([OwnerRef::Vca(&model.vcas[0])], [OwnerRef::Vca(&more)]), + None + ); + let mut fewer = model.tracks[0].clone(); + fewer.inserts.effects[0].params.clear(); + assert_eq!( + owners_growth( + [OwnerRef::Track(&model.tracks[0])], + [OwnerRef::Track(&fewer)], + ), + None + ); + assert_eq!( + owners_growth( + [OwnerRef::Track(&model.tracks[0])], + [OwnerRef::Track(&model.tracks[0].clone())], + ), + Some(OwnersGrowth::default()) + ); + // #1620: an added parameter entry is growth, counted with the list it lengthens. + let mut added = model.tracks[0].clone(); + let params = &mut added.inserts.effects[0].params; + let mut entry = params[0].clone(); + entry.parameter_id = u32::MAX; + params.push(entry); + let length = params.len(); + assert_eq!( + owners_growth( + [OwnerRef::Track(&model.tracks[0])], + [OwnerRef::Track(&added)], + ), + Some(OwnersGrowth { + parameters: 1, + largest_grown_list: length, + }) + ); } diff --git a/tools/parameter-metadata/tests/control_command_heap.rs b/tools/parameter-metadata/tests/control_command_heap.rs index 88fa32468..20d64246e 100644 --- a/tools/parameter-metadata/tests/control_command_heap.rs +++ b/tools/parameter-metadata/tests/control_command_heap.rs @@ -9,8 +9,8 @@ //! heap live at the call's entry. //! //! The ceilings are the numbers measured on this head with a few KB of slack. The steady call is -//! the same at every track count: a live-only commit must not depend on the session's size. The -//! numbers print with `--nocapture`. +//! the same at every track count, and so is the first (#1620): a live-only commit must not depend +//! on the session's size. The numbers print with `--nocapture`. //! //! It lives in `parameter-metadata`, which links host-web and the audited allocator already: //! `scripts/check-bench-policy.sh` keeps `bench-support` out of every `hosts/` package. @@ -214,10 +214,12 @@ fn run(tracks: usize) -> (Heap, Heap) { /// 101,025 bytes in 753 allocations. Before #1615 they were 172,790 and 168,031, requesting 293,477 /// in 786. /// -/// A transaction on a larger session is the same number once the first call has built what it -/// keeps; the first call of 64 and 256 tracks still grows with the session and is recorded, not -/// gated. -const FIRST_PEAK_CEILING_ONE_TRACK: usize = 78_000; +/// #1620: the first call is in place too. It used to take the full path, because its edits add +/// parameter entries to the sparse EQ list (peak 77,600 / 198,182 / 663,179 bytes at 1 / 64 / 256 +/// tracks, 839 / 1,853 / 4,927 allocations); in place, it peaks at 72,865 bytes, requesting 103,033 +/// in 757 allocations, at every size. Its ceiling was 78,000 at one track and none above. +const FIRST_PEAK_CEILING: usize = 74_000; +const FIRST_REQUESTED_CEILING: usize = 105_000; const STEADY_PEAK_CEILING: usize = 73_000; const STEADY_REQUESTED_CEILING: usize = 104_000; @@ -228,20 +230,19 @@ fn a_live_only_nine_edit_transaction_has_a_small_peak_at_every_size() { // what this test is not measuring. host_core::verify_stages(false); let mut peaks = Vec::new(); + let mut firsts = Vec::new(); for tracks in [1_usize, 64, 256] { let (first, steady) = run(tracks); println!("tracks {tracks}: first {first:?}; steady {steady:?}"); - // A steady call keeps only its 80-byte response; the first also seeds some state. + // A call keeps only its 80-byte response. assert!( - steady.left <= 256 && first.left <= 4_096, + steady.left <= 256 && first.left <= 256, "a call leaves heap live" ); - if tracks == 1 { - assert!( - first.peak <= FIRST_PEAK_CEILING_ONE_TRACK, - "{tracks}: first {first:?}" - ); - } + assert!( + first.peak <= FIRST_PEAK_CEILING && first.requested <= FIRST_REQUESTED_CEILING, + "{tracks}: first {first:?}" + ); assert!( steady.peak <= STEADY_PEAK_CEILING, "{tracks}: steady {steady:?}" @@ -251,7 +252,15 @@ fn a_live_only_nine_edit_transaction_has_a_small_peak_at_every_size() { "{tracks}: steady {steady:?}" ); peaks.push(steady); + firsts.push(first); } + // #1620: nor does the first call, which adds parameter entries. + assert!( + firsts + .iter() + .all(|first| first.allocations == firsts[0].allocations), + "the first call's allocations depend on the track count: {firsts:?}" + ); // A live-only commit does not depend on the session's size. assert_eq!( peaks[1].peak, peaks[2].peak, From 4c38674b49543980cf283aa52045e5632b51e512 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 19:32:58 +0000 Subject: [PATCH 06/12] Generate the SDK's RESPONSE_GET codes and drop its own checks (#1623) Co-Authored-By: Claude Opus 5.5 --- ...ponse-get-codes-and-drop-its-own-checks.md | 68 +++ crates/capi/tests/response_get.rs | 26 ++ crates/host-core/tests/response_frame.rs | 108 ++++- crates/protocol/src/frame_table.rs | 328 ++++++++++++++ crates/protocol/src/lib.rs | 14 +- crates/protocol/src/message_wire.rs | 5 +- crates/protocol/src/message_wire/response.rs | 34 +- crates/protocol/src/schema.rs | 21 + sdk/assets/miso-engine-v1-response-frame.json | 90 ++++ sdk/codegen/assets.mjs | 15 +- sdk/codegen/generate.mjs | 21 + sdk/src/core/response-frame.ts | 261 +++++++---- sdk/src/core/response.ts | 125 ++---- sdk/src/core/session.ts | 46 +- sdk/src/generated/response-frame.ts | 422 ++++++++++++++++++ sdk/test/response-evals.mjs | 146 ++++++ tools/parameter-metadata/Cargo.toml | 7 +- tools/parameter-metadata/src/lib.rs | 16 +- .../parameter-metadata/src/response_frame.rs | 82 ++++ .../tests/response_frame.rs | 24 + 20 files changed, 1670 insertions(+), 189 deletions(-) create mode 100644 crates/protocol/src/frame_table.rs create mode 100644 sdk/assets/miso-engine-v1-response-frame.json create mode 100644 sdk/src/generated/response-frame.ts create mode 100644 tools/parameter-metadata/src/response_frame.rs create mode 100644 tools/parameter-metadata/tests/response_frame.rs diff --git a/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md b/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md index f66a5b1b0..8a4d48317 100644 --- a/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md +++ b/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md @@ -45,3 +45,71 @@ same code the C ABI gives. ## Size Half a day to a day. + +## Decision record: #1623 (batch 14b) + +Shipped. Anchors held except as noted (`response.ts` had moved: the checks were in `#query` and the +`gridName`/`qualityName`/`channelName`/`asF32` helpers). + +**D1.** The protocol crate gained `protocol::response_frame_table()` (`crates/protocol/src/frame_table.rs`). +It reads the field IDs, wire types and mandatory flags from the `schema.rs` specs (a name is paired +with its spec constant and the function refuses a message whose names do not cover its spec in +order) and reads each code table (configuration kind, grid, lane, quality, link mode) from the +strict reader that refuses codes outside it. `parameter-metadata --print-response-frame` writes it +as `sdk/assets/miso-engine-v1-response-frame.json`, a source-only SDK asset like the session keys; +`sdk/codegen/generate.mjs` emits `sdk/src/generated/response-frame.ts`. `response-frame.ts` takes +every ID, wire type, mandatory flag and code from it; its field helpers take a table row of one +literal wire type, so a schema change of a field's type is a `tsc` error. The hand copies +(`GRID_CODES`, `LANE_CODES`, `QUALITY_CODES`, `LINK_CODES`, `KIND_*`, every field number and +`MESSAGE_RESPONSE_GET`) are deleted. + +**D2.** Deleted from the SDK: the grid-kind, finite-endpoint, logarithmic-minimum, point-count, +rate and quantum, quality, channel-name-default-as-check, link-mode-fallback, request-ID-zero, +shared-lane and parameter-domain checks of the response path, the unknown-effect and +no-response-capability throws, the `1 << 20` request-frame bound and the `16 << 20` and `>= 1` +bounds on `maximumResultBytes`/`maximumAutomationSpansPerBlock`. What stays is a value with no wire +spelling, as `MisoUsageError`: a name the code tables lack, a number its field cannot carry +(`requireU32`/`requireU64`), a `fields` selector, a parameter name or label the catalog lacks, a +non-number parameter value, and the SDK's own `requestDeadlineMs` timer bound. + +**Departures and mechanical edits outside the authorized paths.** +- `crates/protocol`: `Wire::name`, `ResponseConfigurationKind` (a `control_code_table!` replacing the + literal `1`/`2` of the configuration kind in the encoder and decoder; same wire, same refusal), + `frame_table.rs`, two re-exports. Needed because `schema.rs` is crate-private. +- `tools/parameter-metadata/Cargo.toml`: `protocol` moved from dev-dependencies to dependencies + (already in the lockfile through `host-web`). +- `sdk/src/core/session.ts`: `parameterWireRows`, the encode-only half of `parameterRows` + (`parameterRows` itself is unchanged in behavior); the response path no longer calls the + authoring checks. +- A response limit the caller leaves out is now a field left out of the frame, so the core's own + default applies (the SDK defaults `1 << 20`, `1 << 30`, `1 << 30`, `4096`, `16 << 20` equalled the + core's and are gone). The frame of a default query is therefore shorter; the replies are equal. +- Rust rows added so the codes are held on both sides: `crates/capi/tests/response_get.rs` + (draft and high quality) and `crates/host-core/tests/response_frame.rs` + (`the_core_refuses_what_a_thin_client_sends_as_given`, `a_non_finite_endpoint_sent_raw_is_refused_by_the_decoder`). +- `effect()` still refuses draft/high quality, a shared parameter on one lane and an out-of-domain + value at declaration: that is session authoring (`session.ts`), not the response path, and a + response preview of a hand-built configuration now reaches the core. + +**Gates.** +1. `sdk/test/response-evals.mjs`: "a grid, a rate, a quality and a value the SDK used to refuse are + the core's refusals" (14 rows: logarithmic axis from zero `response.grid`, NaN and infinite + endpoints `protocol.failure`, draft and high quality `effect.quality.unsupported`, ...), + "a lane given twice is refused by the core with the C ABI's code" (`response.override.duplicate`, + raw frames), "a name or a number with no wire spelling stays a usage error". The codes equal the + rows of `capi` `every_reachable_refusal_is_typed_and_changes_nothing` and host-core + `the_core_refuses_what_a_thin_client_sends_as_given`. The session-less web entry and a C ABI + session cannot take one literal frame (the session endpoint refuses a rate and a quantum in the + request), so equality is by request content and the shared `ResponseQueryError::refusal` table. +2. `bash scripts/check-sdk-generated.sh` fails on a hand edit of the asset or of the generated + module (both tried red); `parameter-metadata` `response_frame_asset_is_current` holds the asset + to the generator. Protocol test `frame_table::tests::every_code_is_the_one_the_encoder_writes` + holds each name to the code its encoder writes. +3. `check-sdk-headless.sh` (563 tests pass), `check-sdk-types.sh`, `sdk-package.sh check`, + `check-sdk-deletions.py` (and `--self-test`), `check-workspace-policy.sh`, + `cargo test --locked -p parameter-metadata -p protocol`, host-core `response_frame`, capi + `response_get`; clippy `-D warnings` and fmt clean on protocol, parameter-metadata, host-core, + capi. + +**Successor found.** `control-frame.ts` still holds the `WIRE_*` type codes, `FLAG_REVISION_ANY`, +the header magic and the status names by hand; the same frame table could carry them. diff --git a/crates/capi/tests/response_get.rs b/crates/capi/tests/response_get.rs index 70372a772..445650048 100644 --- a/crates/capi/tests/response_get.rs +++ b/crates/capi/tests/response_get.rs @@ -651,6 +651,32 @@ fn every_reachable_refusal_is_typed_and_changes_nothing() { StatusCode::InvalidField, "response.grid", ), + ( + "draft quality", + ResponseGetRequest { + target: ResponseTarget::Configuration(ResponseConfiguration::Effect { + effect: EQ.to_owned(), + quality: session::EffectQuality::Draft, + link_mode: session::LinkMode::DualMono, + }), + ..request(eq_target(), 64) + }, + StatusCode::ValidationFailed, + "effect.quality.unsupported", + ), + ( + "high quality", + ResponseGetRequest { + target: ResponseTarget::Configuration(ResponseConfiguration::Effect { + effect: EQ.to_owned(), + quality: session::EffectQuality::High, + link_mode: session::LinkMode::DualMono, + }), + ..request(eq_target(), 64) + }, + StatusCode::ValidationFailed, + "effect.quality.unsupported", + ), ( "an unknown effect", ResponseGetRequest { diff --git a/crates/host-core/tests/response_frame.rs b/crates/host-core/tests/response_frame.rs index e3a7c12e3..8f74be1d7 100644 --- a/crates/host-core/tests/response_frame.rs +++ b/crates/host-core/tests/response_frame.rs @@ -4,6 +4,11 @@ //! a message that is not `RESPONSE_GET` and a header that does not decode are each answered or //! refused as the entry documents. //! * **The caller's limits.** The request's own `maximum_reply_bytes` lowers the reply bound. +//! * **The core's refusals, not the client's (#1623).** A logarithmic axis from zero, a quality the +//! effect does not publish, a lane given twice and a non-finite endpoint are each refused with a +//! status and code of the core. The browser SDK sends them as given and shows exactly these +//! (`sdk/test/response-evals.mjs`); `crates/capi/tests/response_get.rs` holds the C ABI session +//! to the same codes. //! //! The byte equality with a C ABI session is `crates/capi/tests/response_get.rs`. @@ -13,7 +18,7 @@ use host_core::{ResponseFrameError, serve_response_frame}; use protocol::{ CommandPayload, DecodeScratch, DecodedSuccessResponsePayload, DecodedTypedResponseFrame, ExpectedRevision, MessageId, ParameterChannel, ProtocolCodec, RequestId, ResponseConfiguration, - ResponseGetRequest, ResponseGrid, ResponseResult, ResponseTarget, StatusCode, + ResponseGetRequest, ResponseGrid, ResponseOverride, ResponseResult, ResponseTarget, StatusCode, TypedCommandFrame, }; @@ -154,6 +159,107 @@ fn a_request_the_entry_cannot_resolve_is_refused_with_its_code() { } } +/// The refusals a client sends as given, with the status and code the core answers. The SDK's +/// `response-evals.mjs` lists the same rows; `crates/capi/tests/response_get.rs` lists them for a +/// C ABI session. +/// +/// Red if a client-side check is put back in front of one of them and the core's answer drifts, or +/// if the core stops refusing one: nothing then stands between a client and the engine. +#[test] +fn the_core_refuses_what_a_thin_client_sends_as_given() { + let mut zero_log = request(); + zero_log.minimum_hz = 0.0; + let with_quality = |quality| { + let mut query = request(); + query.target = ResponseTarget::Configuration(ResponseConfiguration::Effect { + effect: "miso.parametric-eq".to_owned(), + quality, + link_mode: session::LinkMode::DualMono, + }); + query + }; + let mut twice = request(); + twice.overrides = vec![ + ResponseOverride { + control: 4, + lane: ParameterChannel::Both, + value: 1.0, + }, + ResponseOverride { + control: 4, + lane: ParameterChannel::Left, + value: 2.0, + }, + ]; + let mut one_point = request(); + one_point.points = 1; + for (name, query, status, code) in [ + ( + "a logarithmic axis from zero", + zero_log, + StatusCode::InvalidField, + "response.grid", + ), + ( + "one point", + one_point, + StatusCode::InvalidField, + "response.points", + ), + ( + "draft quality", + with_quality(session::EffectQuality::Draft), + StatusCode::ValidationFailed, + "effect.quality.unsupported", + ), + ( + "high quality", + with_quality(session::EffectQuality::High), + StatusCode::ValidationFailed, + "effect.quality.unsupported", + ), + ( + "a lane given twice", + twice, + StatusCode::InvalidField, + "response.override.duplicate", + ), + ] { + let reply = serve(&query); + let (got_status, message, codes) = refusal(&reply).unwrap_or_else(|| panic!("{name}")); + assert_eq!( + (got_status, message), + (status, MessageId::ResponseGet), + "{name}" + ); + assert_eq!(codes, [code], "{name}"); + } +} + +/// A frame the Rust encoder refuses to write (a NaN endpoint) is still a frame a client can send; +/// the decoder refuses it with the protocol's own code. +#[test] +fn a_non_finite_endpoint_sent_raw_is_refused_by_the_decoder() { + let mut frame = encode(CommandPayload::ResponseGet(&request())); + let finite = 20.0_f32.to_le_bytes(); + let at = frame + .windows(4) + .position(|window| window == finite) + .expect("the minimum endpoint is in the frame"); + frame[at..at + 4].copy_from_slice(&f32::NAN.to_le_bytes()); + let mut reply = Vec::new(); + serve_response_frame(&frame, &mut reply).expect("a command frame"); + let (status, message, codes) = refusal(&reply).expect("a refusal"); + assert_eq!( + (status, message, codes), + ( + StatusCode::InvalidField, + MessageId::ResponseGet, + vec!["protocol.failure".to_owned()] + ) + ); +} + #[test] fn another_message_is_unsupported_and_a_broken_header_has_no_reply() { let mut reply = Vec::new(); diff --git a/crates/protocol/src/frame_table.rs b/crates/protocol/src/frame_table.rs new file mode 100644 index 000000000..64c4587b8 --- /dev/null +++ b/crates/protocol/src/frame_table.rs @@ -0,0 +1,328 @@ +//! The `RESPONSE_GET` frame as data: the field IDs, wire types and enum codes a client builds its +//! codec from (decision 17, C13; #1623). +//! +//! The browser SDK writes and reads this frame without a Rust codec. Before this table it copied +//! the field IDs and the grid, lane, quality and link codes by hand, and a field or code that moved +//! in `schema.rs` moved in the engine alone. The table is read from the same `schema` specs the +//! codec writes and reads with, and each code table is read from the decoder that refuses any code +//! outside it, so a client generated from the table cannot spell a field or a code the engine does +//! not. +//! +//! Only the names are written here, once, beside the spec each one labels. A name is paired with +//! its spec constant, not with a position, and [`response_frame_table`] refuses a message whose +//! names do not cover its spec in order. + +use crate::{ + message_wire::{ParameterChannel, ResponseConfigurationKind, ResponseGrid}, + schema::{ + self, FieldSpec, MessageSpec, response_configuration, response_override, response_request, + response_result, + }, + session_wire::{parse_link, parse_quality}, + wire::MessageId, +}; + +/// One field of a message in the frame table. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct FrameField { + /// The field's name, as a client spells it. + pub name: &'static str, + /// The field ID. + pub id: u16, + /// The BTLV wire type code. + pub wire: u8, + /// The wire type's name. + pub wire_name: &'static str, + /// Whether a frame must carry the field. + pub mandatory: bool, + /// Whether the field may repeat. + pub repeated: bool, + /// The name of the nested message, for a message-typed field. + pub message: Option<&'static str>, +} + +/// One message of the frame: its fields, in field-ID order. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct FrameMessage { + /// The message's key in the table. + pub key: &'static str, + /// The schema's name for the message. + pub name: &'static str, + /// The fields, in ascending ID order. + pub fields: Vec, +} + +/// One allocated code of a code table. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct FrameCode { + /// The name a client spells the code with. + pub name: &'static str, + /// The wire code. + pub code: u8, +} + +/// The codes a one-byte enum field carries. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct FrameCodeTable { + /// The table's key in the table. + pub key: &'static str, + /// Every allocated code, ascending. + pub codes: Vec, +} + +/// The `RESPONSE_GET` command: its message ID, its four messages and its code tables. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct ResponseFrameTable { + /// The command's message ID. + pub command: u16, + /// The request, the configuration it may carry, one override row and the reply. + pub messages: Vec, + /// The code tables of the enum fields. + pub code_tables: Vec, +} + +fn message( + key: &'static str, + spec: &'static MessageSpec, + names: &[(&'static str, FieldSpec)], +) -> FrameMessage { + assert!( + names.len() == spec.fields.len() + && names + .iter() + .zip(spec.fields) + .all(|((_, named), field)| named.id == field.id), + "the names of {} do not cover its spec in order", + spec.name + ); + FrameMessage { + key, + name: spec.name, + fields: names + .iter() + .map(|(name, field)| FrameField { + name, + id: field.id, + wire: field.wire.raw(), + wire_name: field.wire.name(), + mandatory: field.mandatory, + repeated: field.repeated, + message: field.nested.map(|nested| nested.name), + }) + .collect(), + } +} + +/// The allocated codes of a table, read from the strict reader that refuses the rest. +fn codes(reader: impl Fn(u8) -> Option, name: impl Fn(T) -> &'static str) -> Vec { + (0..=u8::MAX) + .filter_map(|code| { + reader(code).map(|value| FrameCode { + name: name(value), + code, + }) + }) + .collect() +} + +const fn kind_name(kind: ResponseConfigurationKind) -> &'static str { + match kind { + ResponseConfigurationKind::Effect => "effect", + ResponseConfigurationKind::InputFilters => "inputFilters", + } +} + +const fn grid_name(grid: ResponseGrid) -> &'static str { + match grid { + ResponseGrid::Linear => "linear", + ResponseGrid::Logarithmic => "logarithmic", + } +} + +const fn lane_name(lane: ParameterChannel) -> &'static str { + match lane { + ParameterChannel::Left => "left", + ParameterChannel::Right => "right", + ParameterChannel::Both => "both", + } +} + +/// The `RESPONSE_GET` frame, as the codec writes and reads it. +#[must_use] +pub fn response_frame_table() -> ResponseFrameTable { + use response_configuration as configuration; + use response_override as row; + use response_request as request; + use response_result as result; + ResponseFrameTable { + command: MessageId::ResponseGet as u16, + messages: vec![ + message( + "request", + &request::SPEC, + &[ + ("configuration", request::CONFIGURATION), + ("address", request::ADDRESS), + ("stripId", request::STRIP_ID), + ("grid", request::GRID), + ("points", request::POINTS), + ("minimumHz", request::MINIMUM_HZ), + ("maximumHz", request::MAXIMUM_HZ), + ("channels", request::CHANNELS), + ("sections", request::SECTIONS), + ("override", request::OVERRIDE), + ("configurationId", request::CONFIGURATION_ID), + ("sampleRateHz", request::SAMPLE_RATE_HZ), + ("quantumFrames", request::QUANTUM_FRAMES), + ("maximumPreparedBytes", request::MAXIMUM_PREPARED_BYTES), + ("maximumStateBytes", request::MAXIMUM_STATE_BYTES), + ("maximumScratchBytes", request::MAXIMUM_SCRATCH_BYTES), + ("maximumAutomationSpans", request::MAXIMUM_AUTOMATION_SPANS), + ("maximumReplyBytes", request::MAXIMUM_REPLY_BYTES), + ], + ), + message( + "configuration", + &configuration::SPEC, + &[ + ("kind", configuration::KIND), + ("effect", configuration::EFFECT), + ("quality", configuration::QUALITY), + ("linkMode", configuration::LINK_MODE), + ], + ), + message( + "override", + &row::SPEC, + &[ + ("control", row::CONTROL), + ("lane", row::LANE), + ("value", row::VALUE), + ], + ), + message( + "result", + &result::SPEC, + &[ + ("requestTag", result::REQUEST_TAG), + ("sampleRateHz", result::SAMPLE_RATE_HZ), + ("configurationId", result::CONFIGURATION_ID), + ("floorDb", result::FLOOR_DB), + ("bypass", result::BYPASS), + ("sectionCount", result::SECTION_COUNT), + ("enabledLeft", result::ENABLED_LEFT), + ("enabledRight", result::ENABLED_RIGHT), + ("retainedBytes", result::RETAINED_BYTES), + ("frequenciesHz", result::FREQUENCIES_HZ), + ("leftDb", result::LEFT_DB), + ("rightDb", result::RIGHT_DB), + ("sectionsLeftDb", result::SECTIONS_LEFT_DB), + ("sectionsRightDb", result::SECTIONS_RIGHT_DB), + ], + ), + ], + code_tables: vec![ + FrameCodeTable { + key: "configurationKind", + codes: codes( + |code| ResponseConfigurationKind::from_code(code).ok(), + kind_name, + ), + }, + FrameCodeTable { + key: "grid", + codes: codes(|code| ResponseGrid::from_code(code).ok(), grid_name), + }, + FrameCodeTable { + key: "lane", + codes: codes( + |code| schema::parameter_channel_from_wire(code).ok(), + lane_name, + ), + }, + FrameCodeTable { + key: "quality", + codes: codes(|code| parse_quality(code).ok(), |quality| quality.token()), + }, + FrameCodeTable { + key: "linkMode", + codes: codes(|code| parse_link(code).ok(), |mode| mode.token()), + }, + ], + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::session_wire::{enum_link, enum_quality}; + + fn table<'a>(frame: &'a ResponseFrameTable, key: &str) -> &'a FrameCodeTable { + frame + .code_tables + .iter() + .find(|table| table.key == key) + .unwrap_or_else(|| panic!("no code table {key}")) + } + + fn code_of(frame: &ResponseFrameTable, key: &str, name: &str) -> u8 { + table(frame, key) + .codes + .iter() + .find(|row| row.name == name) + .unwrap_or_else(|| panic!("no code {key}.{name}")) + .code + } + + /// Each name is the code the encoder writes for that variant. A table read from the decoders + /// alone would call a swapped pair of variants consistent; the encoders are the other half. + #[test] + fn every_code_is_the_one_the_encoder_writes() { + let frame = response_frame_table(); + assert_eq!(frame.command, 0x000e); + for (name, kind) in [ + ("effect", ResponseConfigurationKind::Effect), + ("inputFilters", ResponseConfigurationKind::InputFilters), + ] { + assert_eq!(code_of(&frame, "configurationKind", name), kind.code()); + } + assert_eq!(table(&frame, "configurationKind").codes.len(), 2); + for (name, grid) in [ + ("linear", ResponseGrid::Linear), + ("logarithmic", ResponseGrid::Logarithmic), + ] { + assert_eq!(code_of(&frame, "grid", name), grid.code()); + } + assert_eq!(table(&frame, "grid").codes.len(), 2); + for (name, lane) in [ + ("left", ParameterChannel::Left), + ("right", ParameterChannel::Right), + ("both", ParameterChannel::Both), + ] { + assert_eq!( + code_of(&frame, "lane", name), + schema::parameter_channel_wire(lane) + ); + } + assert_eq!(table(&frame, "lane").codes.len(), 3); + for quality in [ + session::EffectQuality::Draft, + session::EffectQuality::Normal, + session::EffectQuality::High, + ] { + assert_eq!( + code_of(&frame, "quality", quality.token()), + enum_quality(quality) + ); + } + assert_eq!(table(&frame, "quality").codes.len(), 3); + for mode in [ + session::LinkMode::DualMono, + session::LinkMode::Maximum, + session::LinkMode::Average, + ] { + assert_eq!(code_of(&frame, "linkMode", mode.token()), enum_link(mode)); + } + assert_eq!(table(&frame, "linkMode").codes.len(), 3); + } +} diff --git a/crates/protocol/src/lib.rs b/crates/protocol/src/lib.rs index 963631b6d..99cb2132c 100644 --- a/crates/protocol/src/lib.rs +++ b/crates/protocol/src/lib.rs @@ -8,6 +8,7 @@ mod btlv; #[cfg(any(test, feature = "test-support"))] mod control_rows; mod controller; +mod frame_table; mod message_wire; mod model; mod queue; @@ -27,11 +28,11 @@ pub use message_wire::{ Diagnostic, DiagnosticEvent, DiagnosticSeverity, DiagnosticsPage, DiagnosticsRequest, EditPath, EnumChoice, MeterBatch, MeterComponent, MeterRecord, NonOkResponse, ObservationEnded, ObservationSection, ObservationTapKey, ParameterChannel, ParameterRack, ParameterUnit, - PathSegment, RESPONSE_BYPASS_CONTROL, ResponseConfiguration, ResponseCurves, - ResponseGetRequest, ResponseGrid, ResponseOverride, ResponseResult, ResponseTarget, - SessionCommitted, SessionSnapshot, SessionSnapshotRequest, TelemetryConfiguration, - TransactionApplied, TransportSetRequest, TransportSnapshot, TransportState, - TransportStateEvent, response_result_encoded_len, + PathSegment, RESPONSE_BYPASS_CONTROL, ResponseConfiguration, ResponseConfigurationKind, + ResponseCurves, ResponseGetRequest, ResponseGrid, ResponseOverride, ResponseResult, + ResponseTarget, SessionCommitted, SessionSnapshot, SessionSnapshotRequest, + TelemetryConfiguration, TransactionApplied, TransportSetRequest, TransportSnapshot, + TransportState, TransportStateEvent, response_result_encoded_len, }; /// The control core's address types, to name the address of a [`SessionEdit::SetControl`]. pub mod control_core { @@ -45,6 +46,9 @@ pub mod control_core { MonitorRow, MonitorWrite, Owner, Section, }; } +pub use frame_table::{ + FrameCode, FrameCodeTable, FrameField, FrameMessage, ResponseFrameTable, response_frame_table, +}; pub use model::{ PreparedSessionTransaction, SessionCommit, SessionEdit, SessionEditError, SessionEditOpcode, SessionStore, SessionStoreError, apply_session_edit, diff --git a/crates/protocol/src/message_wire.rs b/crates/protocol/src/message_wire.rs index 4d937f051..e6863b7b9 100644 --- a/crates/protocol/src/message_wire.rs +++ b/crates/protocol/src/message_wire.rs @@ -389,8 +389,9 @@ macro_rules! control_code_table { mod response; pub use response::{ - RESPONSE_BYPASS_CONTROL, ResponseConfiguration, ResponseCurves, ResponseGetRequest, - ResponseGrid, ResponseOverride, ResponseResult, ResponseTarget, response_result_encoded_len, + RESPONSE_BYPASS_CONTROL, ResponseConfiguration, ResponseConfigurationKind, ResponseCurves, + ResponseGetRequest, ResponseGrid, ResponseOverride, ResponseResult, ResponseTarget, + response_result_encoded_len, }; pub(crate) use response::{write_response_request, write_response_result}; diff --git a/crates/protocol/src/message_wire/response.rs b/crates/protocol/src/message_wire/response.rs index e27bc926a..93ebc302b 100644 --- a/crates/protocol/src/message_wire/response.rs +++ b/crates/protocol/src/message_wire/response.rs @@ -28,6 +28,15 @@ control_code_table! { } } +control_code_table! { + /// What kind of target a response configuration names. + #[allow(missing_docs)] + ResponseConfigurationKind { + Effect = 1, + InputFilters = 2, + } +} + /// The control ID that a response override uses for the effect-wide bypass: lane `both`, value /// exactly `0` or `1`. pub const RESPONSE_BYPASS_CONTROL: u32 = 0; @@ -374,7 +383,11 @@ pub(crate) fn write_response_request( quality, link_mode, } => { - write_spec!(sink, response_configuration::KIND, &[1])?; + write_spec!( + sink, + response_configuration::KIND, + &[ResponseConfigurationKind::Effect.code()] + )?; write_spec!(sink, response_configuration::EFFECT, effect.as_bytes())?; write_spec!( sink, @@ -388,7 +401,11 @@ pub(crate) fn write_response_request( ) } ResponseConfiguration::InputFilters => { - write_spec!(sink, response_configuration::KIND, &[2]) + write_spec!( + sink, + response_configuration::KIND, + &[ResponseConfigurationKind::InputFilters.code()] + ) } } })?; @@ -568,9 +585,12 @@ fn decode_response_configuration( let effect = optional_spec!(message, response_configuration::EFFECT)?; let quality = optional_spec!(message, response_configuration::QUALITY)?; let link_mode = optional_spec!(message, response_configuration::LINK_MODE)?; - match read_u8(one_spec!(message, response_configuration::KIND)?)? { + match ResponseConfigurationKind::from_code(read_u8(one_spec!( + message, + response_configuration::KIND + )?)?)? { // The effect, its quality and its link mode are present exactly together. - 1 => { + ResponseConfigurationKind::Effect => { let (Some(effect), Some(quality), Some(link_mode)) = (effect, quality, link_mode) else { return Err(DecodeError::InvalidField); @@ -583,10 +603,12 @@ fn decode_response_configuration( .map_err(|_| DecodeError::InvalidField)?, }) } - 2 if effect.is_none() && quality.is_none() && link_mode.is_none() => { + ResponseConfigurationKind::InputFilters + if effect.is_none() && quality.is_none() && link_mode.is_none() => + { Ok(ResponseConfiguration::InputFilters) } - _ => Err(DecodeError::InvalidField), + ResponseConfigurationKind::InputFilters => Err(DecodeError::InvalidField), } } diff --git a/crates/protocol/src/schema.rs b/crates/protocol/src/schema.rs index 548fe7ec9..4477a4893 100644 --- a/crates/protocol/src/schema.rs +++ b/crates/protocol/src/schema.rs @@ -135,6 +135,27 @@ impl Wire { self as u8 } + /// The type's spelling in the frame table a client generates its codec from. + pub(crate) const fn name(self) -> &'static str { + match self { + Self::U8 => "u8", + Self::U16 => "u16", + Self::U32 => "u32", + Self::U64 => "u64", + Self::I64 => "i64", + Self::F32 => "f32", + Self::F64 => "f64", + Self::Bool => "bool", + Self::Utf8 => "utf8", + Self::Bytes => "bytes", + Self::Message => "message", + Self::PackedU16 => "packedU16", + Self::PackedU32 => "packedU32", + Self::PackedU64 => "packedU64", + Self::PackedF32 => "packedF32", + } + } + pub(crate) const fn from_raw(value: u8) -> Option { match value { 1 => Some(Self::U8), diff --git a/sdk/assets/miso-engine-v1-response-frame.json b/sdk/assets/miso-engine-v1-response-frame.json new file mode 100644 index 000000000..bc5fa01da --- /dev/null +++ b/sdk/assets/miso-engine-v1-response-frame.json @@ -0,0 +1,90 @@ +{ + "schema": "miso.response.frame.v1", + "command": 14, + "messages": { + "request": { + "name": "ResponseRequest", + "fields": { + "configuration": { "id": 1, "wire": 11, "wireName": "message", "mandatory": false, "repeated": false, "message": "ResponseConfiguration" }, + "address": { "id": 2, "wire": 11, "wireName": "message", "mandatory": false, "repeated": false, "message": "ControlAddress" }, + "stripId": { "id": 3, "wire": 9, "wireName": "utf8", "mandatory": false, "repeated": false, "message": null }, + "grid": { "id": 4, "wire": 1, "wireName": "u8", "mandatory": true, "repeated": false, "message": null }, + "points": { "id": 5, "wire": 3, "wireName": "u32", "mandatory": true, "repeated": false, "message": null }, + "minimumHz": { "id": 6, "wire": 6, "wireName": "f32", "mandatory": true, "repeated": false, "message": null }, + "maximumHz": { "id": 7, "wire": 6, "wireName": "f32", "mandatory": true, "repeated": false, "message": null }, + "channels": { "id": 8, "wire": 1, "wireName": "u8", "mandatory": true, "repeated": false, "message": null }, + "sections": { "id": 9, "wire": 8, "wireName": "bool", "mandatory": true, "repeated": false, "message": null }, + "override": { "id": 10, "wire": 11, "wireName": "message", "mandatory": false, "repeated": true, "message": "ResponseOverride" }, + "configurationId": { "id": 11, "wire": 4, "wireName": "u64", "mandatory": false, "repeated": false, "message": null }, + "sampleRateHz": { "id": 12, "wire": 3, "wireName": "u32", "mandatory": false, "repeated": false, "message": null }, + "quantumFrames": { "id": 13, "wire": 3, "wireName": "u32", "mandatory": false, "repeated": false, "message": null }, + "maximumPreparedBytes": { "id": 14, "wire": 4, "wireName": "u64", "mandatory": false, "repeated": false, "message": null }, + "maximumStateBytes": { "id": 15, "wire": 4, "wireName": "u64", "mandatory": false, "repeated": false, "message": null }, + "maximumScratchBytes": { "id": 16, "wire": 4, "wireName": "u64", "mandatory": false, "repeated": false, "message": null }, + "maximumAutomationSpans": { "id": 17, "wire": 3, "wireName": "u32", "mandatory": false, "repeated": false, "message": null }, + "maximumReplyBytes": { "id": 18, "wire": 3, "wireName": "u32", "mandatory": false, "repeated": false, "message": null } + } + }, + "configuration": { + "name": "ResponseConfiguration", + "fields": { + "kind": { "id": 1, "wire": 1, "wireName": "u8", "mandatory": true, "repeated": false, "message": null }, + "effect": { "id": 2, "wire": 9, "wireName": "utf8", "mandatory": false, "repeated": false, "message": null }, + "quality": { "id": 3, "wire": 1, "wireName": "u8", "mandatory": false, "repeated": false, "message": null }, + "linkMode": { "id": 4, "wire": 1, "wireName": "u8", "mandatory": false, "repeated": false, "message": null } + } + }, + "override": { + "name": "ResponseOverride", + "fields": { + "control": { "id": 1, "wire": 3, "wireName": "u32", "mandatory": true, "repeated": false, "message": null }, + "lane": { "id": 2, "wire": 1, "wireName": "u8", "mandatory": true, "repeated": false, "message": null }, + "value": { "id": 3, "wire": 6, "wireName": "f32", "mandatory": true, "repeated": false, "message": null } + } + }, + "result": { + "name": "ResponseResult", + "fields": { + "requestTag": { "id": 1, "wire": 4, "wireName": "u64", "mandatory": false, "repeated": false, "message": null }, + "sampleRateHz": { "id": 2, "wire": 3, "wireName": "u32", "mandatory": false, "repeated": false, "message": null }, + "configurationId": { "id": 3, "wire": 4, "wireName": "u64", "mandatory": false, "repeated": false, "message": null }, + "floorDb": { "id": 4, "wire": 6, "wireName": "f32", "mandatory": false, "repeated": false, "message": null }, + "bypass": { "id": 5, "wire": 8, "wireName": "bool", "mandatory": false, "repeated": false, "message": null }, + "sectionCount": { "id": 6, "wire": 3, "wireName": "u32", "mandatory": false, "repeated": false, "message": null }, + "enabledLeft": { "id": 7, "wire": 3, "wireName": "u32", "mandatory": false, "repeated": false, "message": null }, + "enabledRight": { "id": 8, "wire": 3, "wireName": "u32", "mandatory": false, "repeated": false, "message": null }, + "retainedBytes": { "id": 9, "wire": 4, "wireName": "u64", "mandatory": false, "repeated": false, "message": null }, + "frequenciesHz": { "id": 10, "wire": 15, "wireName": "packedF32", "mandatory": false, "repeated": false, "message": null }, + "leftDb": { "id": 11, "wire": 15, "wireName": "packedF32", "mandatory": false, "repeated": false, "message": null }, + "rightDb": { "id": 12, "wire": 15, "wireName": "packedF32", "mandatory": false, "repeated": false, "message": null }, + "sectionsLeftDb": { "id": 13, "wire": 15, "wireName": "packedF32", "mandatory": false, "repeated": false, "message": null }, + "sectionsRightDb": { "id": 14, "wire": 15, "wireName": "packedF32", "mandatory": false, "repeated": false, "message": null } + } + } + }, + "codes": { + "configurationKind": [ + { "name": "effect", "code": 1 }, + { "name": "inputFilters", "code": 2 } + ], + "grid": [ + { "name": "linear", "code": 1 }, + { "name": "logarithmic", "code": 2 } + ], + "lane": [ + { "name": "left", "code": 1 }, + { "name": "right", "code": 2 }, + { "name": "both", "code": 3 } + ], + "quality": [ + { "name": "draft", "code": 1 }, + { "name": "normal", "code": 2 }, + { "name": "high", "code": 3 } + ], + "linkMode": [ + { "name": "dual_mono", "code": 1 }, + { "name": "maximum", "code": 2 }, + { "name": "average", "code": 3 } + ] + } +} diff --git a/sdk/codegen/assets.mjs b/sdk/codegen/assets.mjs index e2b8dd2b3..264fa53d1 100644 --- a/sdk/codegen/assets.mjs +++ b/sdk/codegen/assets.mjs @@ -39,20 +39,25 @@ const sdkRoot = resolve(here, ".."); const repoRoot = resolve(sdkRoot, ".."); const assets = resolve(sdkRoot, "assets"); -/** Each asset, and the generator mode that renders it. One generator, three documents. */ +/** Each asset, and the generator mode that renders it. One generator, four documents. */ const documents = [ ["miso-engine-v1-parameter-metadata.json", "--print"], ["miso-engine-v1-abi-layout.json", "--print-abi-layout"], ["miso-engine-v1-session-keys.json", "--print-session-keys"], + ["miso-engine-v1-response-frame.json", "--print-response-frame"], ]; /** * The documents an engine artifact directory does not carry. The session key tables are a source - * of the SDK's canonical session writer, not a member of the release artifact set, so a check - * against an artifact directory leaves them out; `cargo test -p parameter-metadata` - * (`session_keys_asset_is_current`) holds them to the generator there. + * of the SDK's canonical session writer, and the response frame table (#1623) is a source of its + * response codec; neither is a member of the release artifact set, so a check against an artifact + * directory leaves them out. `cargo test -p parameter-metadata` (`session_keys_asset_is_current`, + * `response_frame_asset_is_current`) holds them to the generator there. */ -const SOURCE_ONLY = new Set(["miso-engine-v1-session-keys.json"]); +const SOURCE_ONLY = new Set([ + "miso-engine-v1-session-keys.json", + "miso-engine-v1-response-frame.json", +]); function render(mode) { return execFileSync( diff --git a/sdk/codegen/generate.mjs b/sdk/codegen/generate.mjs index 80d7f0113..924dbedae 100644 --- a/sdk/codegen/generate.mjs +++ b/sdk/codegen/generate.mjs @@ -38,6 +38,7 @@ const assets = resolve(sdkRoot, "assets"); const generated = resolve(sdkRoot, "src", "generated"); const SESSION_KEYS_ASSET = "miso-engine-v1-session-keys.json"; +const RESPONSE_FRAME_ASSET = "miso-engine-v1-response-frame.json"; const inputs = [ ["catalog", "miso-engine-v1-parameter-metadata.json", "CATALOG"], @@ -874,6 +875,18 @@ function renderSessionKeys(value) { + `export type SessionKeyKind = SessionKeys["records"][number]["keys"][number][1];\n`; } +function renderResponseFrame(value) { + const json = JSON.stringify(value, null, 2); + return `// @generated by sdk/codegen/generate.mjs from sdk/assets/${RESPONSE_FRAME_ASSET}; DO NOT EDIT.\n` + + `// That JSON is the protocol crate's \`RESPONSE_GET\` frame table, read from the codec's own\n` + + `// schema specs and strict code readers by \`parameter-metadata --print-response-frame\`; run\n` + + `// \`npm run assets\` then \`npm run codegen\` in sdk/ to refresh both. The response codec reads\n` + + `// it: no field ID, wire type or enum code of the frame is typed by hand.\n\n` + + deepFreezeSource() + + `export const RESPONSE_FRAME = deepFreeze(\n${json} as const,\n);\n\n` + + `export type ResponseFrame = typeof RESPONSE_FRAME;\n`; +} + function renderProvenance(catalog, abi) { const value = { abiVersion: abi.abiVersion, @@ -919,6 +932,14 @@ async function expectedModules() { throw new Error(`cannot read valid JSON input ${sessionKeysPath}: ${error.message}`); } modules.push([resolve(generated, "session-keys.ts"), renderSessionKeys(sessionKeys)]); + const responseFramePath = resolve(assets, RESPONSE_FRAME_ASSET); + let responseFrame; + try { + responseFrame = JSON.parse(await readFile(responseFramePath, "utf8")); + } catch (error) { + throw new Error(`cannot read valid JSON input ${responseFramePath}: ${error.message}`); + } + modules.push([resolve(generated, "response-frame.ts"), renderResponseFrame(responseFrame)]); modules.push([resolve(generated, "controls.ts"), renderControls(parsed.catalog)]); modules.push([resolve(generated, "control-builders.ts"), renderControlBuilders(parsed.catalog)]); modules.push([resolve(generated, "provenance.ts"), renderProvenance(parsed.catalog, parsed.abi)]); diff --git a/sdk/src/core/response-frame.ts b/sdk/src/core/response-frame.ts index 393b79e8e..17c22155a 100644 --- a/sdk/src/core/response-frame.ts +++ b/sdk/src/core/response-frame.ts @@ -1,3 +1,4 @@ +import { RESPONSE_FRAME } from "../generated/response-frame.ts"; import { FLAG_REVISION_ANY, WIRE_BOOL, @@ -22,35 +23,57 @@ import { u8, } from "./control-frame.ts"; import type { ControlDiagnostic, ControlStatusName, RawField } from "./control-frame.ts"; +import { MisoUsageError } from "./errors.ts"; /** - * The SDK's one codec of the `RESPONSE_GET` frame (`0x000e`) a response preview sends and the - * reply it reads (decision 17, C13; #1574 D4). It is the frame a C ABI host sends: the core decides - * every refusal, and this file checks nothing the core checks (no point count, no domain, no rate). - * The field numbers are the protocol schema's (`crates/protocol/src/schema.rs`: - * `response_request`, `response_configuration`, `response_override`, `response_result`). + * The SDK's one codec of the `RESPONSE_GET` frame a response preview sends and the reply it reads + * (decision 17, C13; #1574 D4; #1623). It is the frame a C ABI host sends: the core decides every + * refusal, and this file checks nothing the core checks (no point count, no domain, no rate, no + * quality, no addressing). It refuses only a value that has no wire spelling: a name the code + * tables do not hold, or a number its field cannot carry. + * + * Every field ID, wire type, mandatory flag and enum code is the generated frame table's + * (`RESPONSE_FRAME`, written from the protocol schema by `parameter-metadata + * --print-response-frame`). A field helper takes the table row of the wire type it writes, so a + * schema change of a field's type is a type error here, not a frame the core refuses. */ -const MESSAGE_RESPONSE_GET = 0x000e; +const REQUEST = RESPONSE_FRAME.messages.request.fields; +const CONFIGURATION = RESPONSE_FRAME.messages.configuration.fields; +const OVERRIDE = RESPONSE_FRAME.messages.override.fields; +const RESULT = RESPONSE_FRAME.messages.result.fields; -const KIND_EFFECT = 1; -const KIND_INPUT_FILTERS = 2; -const GRID_CODES = new Map([["linear", 1], ["logarithmic", 2]]); -const LANE_CODES = new Map([["left", 1], ["right", 2], ["both", 3]]); -const QUALITY_CODES = new Map([["draft", 1], ["normal", 2], ["high", 3]]); -const LINK_CODES = new Map([["dual_mono", 1], ["maximum", 2], ["average", 3]]); +type Named = Table[number]["name"]; +type CodeRow = { readonly name: string; readonly code: number }; -export type ResponseFrameLane = "left" | "right" | "both"; +function codes(rows: readonly CodeRow[]): ReadonlyMap { + return new Map(rows.map((row) => [row.name, row.code] as const)); +} + +const KIND_CODES = codes(RESPONSE_FRAME.codes.configurationKind); +const GRID_CODES = codes(RESPONSE_FRAME.codes.grid); +const LANE_CODES = codes(RESPONSE_FRAME.codes.lane); +const QUALITY_CODES = codes(RESPONSE_FRAME.codes.quality); +const LINK_CODES = codes(RESPONSE_FRAME.codes.linkMode); + +/** A lane, by the name the lane code table holds. */ +export type ResponseFrameLane = Named; +/** How a response axis spaces its points. */ +export type ResponseFrameGrid = Named; +/** The preparation quality a configuration names. */ +export type ResponseFrameQuality = Named; +/** The channel-link mode a configuration names. */ +export type ResponseFrameLinkMode = Named; /** What a response is computed for. */ export type ResponseFrameTarget = | { - readonly kind: "effect"; + readonly kind: Extract, "effect">; readonly effect: string; - readonly quality: "draft" | "normal" | "high"; - readonly linkMode: "dual_mono" | "maximum" | "average"; + readonly quality: ResponseFrameQuality; + readonly linkMode: ResponseFrameLinkMode; } - | { readonly kind: "inputFilters" }; + | { readonly kind: Extract, "inputFilters"> }; /** One override row: a control ID, the lane it applies to and its value. Control 0 is the bypass. */ export interface ResponseFrameOverride { @@ -59,11 +82,14 @@ export interface ResponseFrameOverride { readonly value: number; } -/** A `RESPONSE_GET` request for a configuration target. */ +/** + * A `RESPONSE_GET` request for a configuration target. A limit left out is a field left out of the + * frame, so the core's own default applies; the SDK holds no copy of it. + */ export interface ResponseFrameRequest { readonly requestId: bigint; readonly target: ResponseFrameTarget; - readonly grid: "linear" | "logarithmic"; + readonly grid: ResponseFrameGrid; readonly points: number; readonly minimumHz: number; readonly maximumHz: number; @@ -73,11 +99,11 @@ export interface ResponseFrameRequest { readonly configurationId: bigint; readonly sampleRateHz: number; readonly quantumFrames: number; - readonly maximumPreparedBytes: bigint; - readonly maximumStateBytes: bigint; - readonly maximumScratchBytes: bigint; - readonly maximumAutomationSpans: number; - readonly maximumReplyBytes: number; + readonly maximumPreparedBytes?: bigint | undefined; + readonly maximumStateBytes?: bigint | undefined; + readonly maximumScratchBytes?: bigint | undefined; + readonly maximumAutomationSpans?: number | undefined; + readonly maximumReplyBytes?: number | undefined; } /** The curves of a successful reply. */ @@ -107,9 +133,16 @@ export type ResponseFrameReply = readonly diagnostics: readonly ControlDiagnostic[]; }; -function code(table: ReadonlyMap, name: string): number { +/** A table row of the wire type `Wire`: the field's ID and whether a frame must carry it. */ +type Row = { + readonly id: number; + readonly wire: Wire; + readonly mandatory: boolean; +}; + +function code(table: ReadonlyMap, name: string, what: string): number { const found = table.get(name); - if (found === undefined) throw malformed(`the request names ${name}, which has no code`); + if (found === undefined) throw new MisoUsageError(`${what} ${String(name)} has no wire code`); return found; } @@ -125,6 +158,35 @@ function f32Bytes(value: number): Uint8Array { return out; } +function putU8(row: Row, value: number): Uint8Array { + return field(row.id, row.wire, row.mandatory, u8(value)); +} + +function putBool(row: Row, value: boolean): Uint8Array { + return field(row.id, row.wire, row.mandatory, u8(value ? 1 : 0)); +} + +function putText(row: Row, value: string): Uint8Array { + return field(row.id, row.wire, row.mandatory, text(value)); +} + +function putMessage(row: Row, body: Uint8Array): Uint8Array { + return field(row.id, row.wire, row.mandatory, body); +} + +function putU32(row: Row, value: number, what: string): Uint8Array { + return field(row.id, row.wire, row.mandatory, u32(requireU32(value, what))); +} + +function putU64(row: Row, value: bigint, what: string): Uint8Array { + return field(row.id, row.wire, row.mandatory, u64Bytes(requireU64(value, what))); +} + +/** An `f32` field: any number has a spelling (`NaN` and the infinities too); the core refuses it. */ +function putF32(row: Row, value: number): Uint8Array { + return field(row.id, row.wire, row.mandatory, f32Bytes(value)); +} + /** A nested message body: `count:u32, reserved:u32`, then its fields. */ function messageBody(fields: readonly Uint8Array[]): Uint8Array { const header = new Uint8Array(8); @@ -133,56 +195,65 @@ function messageBody(fields: readonly Uint8Array[]): Uint8Array { } function configurationMessage(target: ResponseFrameTarget): Uint8Array { - if (target.kind === "inputFilters") { - return messageBody([field(1, WIRE_U8, true, u8(KIND_INPUT_FILTERS))]); - } + const kind = putU8(CONFIGURATION.kind, code(KIND_CODES, target.kind, "the configuration kind")); + if (target.kind === "inputFilters") return messageBody([kind]); return messageBody([ - field(1, WIRE_U8, true, u8(KIND_EFFECT)), - field(2, WIRE_UTF8, false, text(target.effect)), - field(3, WIRE_U8, false, u8(code(QUALITY_CODES, target.quality))), - field(4, WIRE_U8, false, u8(code(LINK_CODES, target.linkMode))), + kind, + putText(CONFIGURATION.effect, target.effect), + putU8(CONFIGURATION.quality, code(QUALITY_CODES, target.quality, "the quality")), + putU8(CONFIGURATION.linkMode, code(LINK_CODES, target.linkMode, "the link mode")), ]); } function overrideMessage(row: ResponseFrameOverride): Uint8Array { return messageBody([ - field(1, WIRE_U32, true, u32(requireU32(row.control, "a control ID"))), - field(2, WIRE_U8, true, u8(code(LANE_CODES, row.lane))), - field(3, WIRE_F32, true, f32Bytes(row.value)), + putU32(OVERRIDE.control, row.control, "a control ID"), + putU8(OVERRIDE.lane, code(LANE_CODES, row.lane, "the override lane")), + putF32(OVERRIDE.value, row.value), ]); } /** * Encode a `RESPONSE_GET` command for a configuration target. It takes any session revision. Throws - * before producing a byte when a number cannot be encoded. + * `MisoUsageError`, before producing a byte, when a name has no code in the frame table or a number + * is not one its field can carry. */ export function encodeResponseRequest(request: ResponseFrameRequest): Uint8Array { const requestId = requireU64(request.requestId, "requestId"); - if (requestId === 0n) throw malformed("the request ID is zero"); const fields: Uint8Array[] = [ - field(1, WIRE_MESSAGE, false, configurationMessage(request.target)), - field(4, WIRE_U8, true, u8(code(GRID_CODES, request.grid))), - field(5, WIRE_U32, true, u32(requireU32(request.points, "points"))), - field(6, WIRE_F32, true, f32Bytes(request.minimumHz)), - field(7, WIRE_F32, true, f32Bytes(request.maximumHz)), - field(8, WIRE_U8, true, u8(code(LANE_CODES, request.channels))), - field(9, WIRE_BOOL, true, u8(request.sections ? 1 : 0)), + putMessage(REQUEST.configuration, configurationMessage(request.target)), + putU8(REQUEST.grid, code(GRID_CODES, request.grid, "the grid")), + putU32(REQUEST.points, request.points, "points"), + putF32(REQUEST.minimumHz, request.minimumHz), + putF32(REQUEST.maximumHz, request.maximumHz), + putU8(REQUEST.channels, code(LANE_CODES, request.channels, "the channels")), + putBool(REQUEST.sections, request.sections), ]; for (const row of request.overrides) { - fields.push(field(10, WIRE_MESSAGE, false, overrideMessage(row))); + fields.push(putMessage(REQUEST.override, overrideMessage(row))); } fields.push( - field(11, WIRE_U64, false, u64Bytes(requireU64(request.configurationId, "configurationId"))), - field(12, WIRE_U32, false, u32(requireU32(request.sampleRateHz, "sampleRateHz"))), - field(13, WIRE_U32, false, u32(requireU32(request.quantumFrames, "quantumFrames"))), - field(14, WIRE_U64, false, u64Bytes(requireU64(request.maximumPreparedBytes, "maximumPreparedBytes"))), - field(15, WIRE_U64, false, u64Bytes(requireU64(request.maximumStateBytes, "maximumStateBytes"))), - field(16, WIRE_U64, false, u64Bytes(requireU64(request.maximumScratchBytes, "maximumScratchBytes"))), - field(17, WIRE_U32, false, u32(requireU32(request.maximumAutomationSpans, "maximumAutomationSpans"))), - field(18, WIRE_U32, false, u32(requireU32(request.maximumReplyBytes, "maximumReplyBytes"))), + putU64(REQUEST.configurationId, request.configurationId, "configurationId"), + putU32(REQUEST.sampleRateHz, request.sampleRateHz, "sampleRateHz"), + putU32(REQUEST.quantumFrames, request.quantumFrames, "quantumFrames"), ); + if (request.maximumPreparedBytes !== undefined) { + fields.push(putU64(REQUEST.maximumPreparedBytes, request.maximumPreparedBytes, "maximumPreparedBytes")); + } + if (request.maximumStateBytes !== undefined) { + fields.push(putU64(REQUEST.maximumStateBytes, request.maximumStateBytes, "maximumStateBytes")); + } + if (request.maximumScratchBytes !== undefined) { + fields.push(putU64(REQUEST.maximumScratchBytes, request.maximumScratchBytes, "maximumScratchBytes")); + } + if (request.maximumAutomationSpans !== undefined) { + fields.push(putU32(REQUEST.maximumAutomationSpans, request.maximumAutomationSpans, "maximumAutomationSpans")); + } + if (request.maximumReplyBytes !== undefined) { + fields.push(putU32(REQUEST.maximumReplyBytes, request.maximumReplyBytes, "maximumReplyBytes")); + } return commandFrame( - MESSAGE_RESPONSE_GET, + RESPONSE_FRAME.command, FLAG_REVISION_ANY, requestId, 0n, @@ -191,23 +262,46 @@ export function encodeResponseRequest(request: ResponseFrameRequest): Uint8Array ); } -function required(fields: readonly RawField[], id: number, wire: number, width: number, what: string): DataView { - const value = scalar(fields, id, wire, width); +function required( + fields: readonly RawField[], + row: Row, + width: number, + what: string, +): DataView { + const value = scalar(fields, row.id, row.wire, width); if (value === undefined) throw malformed(`${what} is missing`); return value; } -function packed(fields: readonly RawField[], id: number): Float32Array | undefined { - const found = fields.filter((row) => row.id === id); - if (found.length > 1) throw malformed(`field ${id} repeats`); - const row = found[0]; - if (row === undefined) return undefined; - if (row.wire !== WIRE_PACKED_F32 || row.value.length % 4 !== 0) { - throw malformed(`field ${id} is not a packed f32 array`); +function readU32(fields: readonly RawField[], row: Row, what: string): number { + return required(fields, row, 4, what).getUint32(0, true); +} + +function readU64(fields: readonly RawField[], row: Row, what: string): bigint { + return required(fields, row, 8, what).getBigUint64(0, true); +} + +function readF32(fields: readonly RawField[], row: Row, what: string): number { + return required(fields, row, 4, what).getFloat32(0, true); +} + +function readBool(fields: readonly RawField[], row: Row, what: string): boolean { + const value = required(fields, row, 1, what).getUint8(0); + if (value > 1) throw malformed(`${what} is not a boolean`); + return value === 1; +} + +function packed(fields: readonly RawField[], row: Row): Float32Array | undefined { + const found = fields.filter((candidate) => candidate.id === row.id); + if (found.length > 1) throw malformed(`field ${row.id} repeats`); + const entry = found[0]; + if (entry === undefined) return undefined; + if (entry.wire !== row.wire || entry.value.length % 4 !== 0) { + throw malformed(`field ${row.id} is not a packed f32 array`); } // The bytes are copied out of the frame: the result owns its arrays. - const out = new Float32Array(row.value.length / 4); - const view = new DataView(row.value.buffer, row.value.byteOffset, row.value.byteLength); + const out = new Float32Array(entry.value.length / 4); + const view = new DataView(entry.value.buffer, entry.value.byteOffset, entry.value.byteLength); for (let index = 0; index < out.length; index += 1) out[index] = view.getFloat32(index * 4, true); return out; } @@ -218,33 +312,34 @@ function packed(fields: readonly RawField[], id: number): Float32Array | undefin * (`phase: "lifecycle"`, `code: "internal"`). */ export function decodeResponseReply(bytes: Uint8Array): ResponseFrameReply { - const { requestId, status, fields } = decodeResponseEnvelope(bytes, MESSAGE_RESPONSE_GET); + const { requestId, status, fields } = decodeResponseEnvelope(bytes, RESPONSE_FRAME.command); if (status !== "ok") { const { diagnostics } = decodeRefusalBody(fields); return Object.freeze({ ok: false, requestId, status, diagnostics }); } - if (fields.some((row) => row.id === 1)) throw malformed("a configuration reply holds a request tag"); - const bypass = required(fields, 5, WIRE_BOOL, 1, "the bypass flag").getUint8(0); - if (bypass > 1) throw malformed("the bypass flag is not a boolean"); - const frequenciesHz = packed(fields, 10); + if (fields.some((row) => row.id === RESULT.requestTag.id)) { + throw malformed("a configuration reply holds a request tag"); + } + const bypass = readBool(fields, RESULT.bypass, "the bypass flag"); + const frequenciesHz = packed(fields, RESULT.frequenciesHz); if (frequenciesHz === undefined) throw malformed("the frequency axis is missing"); return Object.freeze({ ok: true, requestId, curves: Object.freeze({ - sampleRateHz: required(fields, 2, WIRE_U32, 4, "the sample rate").getUint32(0, true), - configurationId: required(fields, 3, WIRE_U64, 8, "the configuration ID").getBigUint64(0, true), - floorDb: required(fields, 4, WIRE_F32, 4, "the floor").getFloat32(0, true), - bypass: bypass === 1, - sectionCount: required(fields, 6, WIRE_U32, 4, "the section count").getUint32(0, true), - enabledLeft: required(fields, 7, WIRE_U32, 4, "the left enabled mask").getUint32(0, true), - enabledRight: required(fields, 8, WIRE_U32, 4, "the right enabled mask").getUint32(0, true), - retainedBytes: required(fields, 9, WIRE_U64, 8, "the retained bytes").getBigUint64(0, true), + sampleRateHz: readU32(fields, RESULT.sampleRateHz, "the sample rate"), + configurationId: readU64(fields, RESULT.configurationId, "the configuration ID"), + floorDb: readF32(fields, RESULT.floorDb, "the floor"), + bypass, + sectionCount: readU32(fields, RESULT.sectionCount, "the section count"), + enabledLeft: readU32(fields, RESULT.enabledLeft, "the left enabled mask"), + enabledRight: readU32(fields, RESULT.enabledRight, "the right enabled mask"), + retainedBytes: readU64(fields, RESULT.retainedBytes, "the retained bytes"), frequenciesHz, - leftDb: packed(fields, 11), - rightDb: packed(fields, 12), - sectionsLeftDb: packed(fields, 13), - sectionsRightDb: packed(fields, 14), + leftDb: packed(fields, RESULT.leftDb), + rightDb: packed(fields, RESULT.rightDb), + sectionsLeftDb: packed(fields, RESULT.sectionsLeftDb), + sectionsRightDb: packed(fields, RESULT.sectionsRightDb), }), }); } diff --git a/sdk/src/core/response.ts b/sdk/src/core/response.ts index d9a191989..ffe9482d0 100644 --- a/sdk/src/core/response.ts +++ b/sdk/src/core/response.ts @@ -8,7 +8,7 @@ import { responseRefusalResult, } from "./response-frame.ts"; import type { ResponseFrameCurves, ResponseFrameOverride, ResponseFrameRequest } from "./response-frame.ts"; -import { parameterRows } from "./session.ts"; +import { parameterWireRows } from "./session.ts"; import type { Channel, EffectDecl } from "./types.ts"; /** An explicit stopped input-filter configuration. */ @@ -99,8 +99,6 @@ export interface ResponsePreviewResult { type RawExports = Record & { readonly memory: WebAssembly.Memory }; -/** The most request frame bytes the engine's session-less response entry takes. */ -const MAXIMUM_REQUEST_FRAME_BYTES = 1 << 20; const FRAME_EXPORTS = [ "miso_engine_web_v1_response_frame_ptr", "miso_engine_web_v1_response_frame_submit", @@ -167,16 +165,9 @@ function finiteInteger(value: number, name: string, minimum: number, maximum: nu return value; } +/** A limit as the `u64` it travels as; a number with no such spelling is a usage error. */ function u64(value: number | bigint, name: string): bigint { - const normalized = typeof value === "bigint" ? value : BigInt(finiteInteger(value, name, 0, Number.MAX_SAFE_INTEGER)); - if (normalized < 0n || normalized > 0xffff_ffff_ffff_ffffn) throw new MisoUsageError(`${name} must fit u64`); - return normalized; -} - -function channelName(channel: Channel | undefined): Channel { - if (channel === undefined) return "both"; - if (channel === "left" || channel === "right" || channel === "both") return channel; - throw new MisoUsageError("channels must be left, right, or both"); + return typeof value === "bigint" ? value : BigInt(finiteInteger(value, name, 0, Number.MAX_SAFE_INTEGER)); } function wantsSections(fields: ResponsePreviewQuery["fields"]): boolean { @@ -185,26 +176,9 @@ function wantsSections(fields: ResponsePreviewQuery["fields"]): boolean { throw new MisoUsageError("fields must be total or totalAndSections"); } -function gridName(kind: unknown): "linear" | "logarithmic" { - if (kind === "linear" || kind === "logarithmic") return kind; - throw new MisoUsageError("grid.kind must be linear or logarithmic"); -} - -function qualityName(value: EffectDecl["options"]["quality"]): "normal" { - if (value !== "normal") throw new MisoUsageError("response previews support only normal native quality"); - return "normal"; -} - -function linkModeName(value: EffectDecl["options"]["linkMode"]): "dual_mono" | "maximum" | "average" { - if (value === "dual_mono") return "dual_mono"; - if (value === "maximum") return "maximum"; - return "average"; -} - -function asF32(value: number | undefined, name: string): number { - const normalized = value ?? 0; - if (!Number.isFinite(normalized)) throw new MisoUsageError(`${name} must be finite`); - return Math.fround(normalized); +/** An input-filter cutoff as the `f32` the frame carries; a cutoff left out is 0, filter off. */ +function cutoff(value: number | undefined): number { + return Math.fround(value ?? 0); } function isInputFilters( @@ -230,15 +204,14 @@ function normalizedConfiguration(configuration: ResponsePreviewConfiguration): R if (isInputFilters(configuration)) { return Object.freeze({ kind: "inputFilters", - left: Object.freeze({ hpfHz: asF32(configuration.left.hpfHz, "left.hpfHz"), lpfHz: asF32(configuration.left.lpfHz, "left.lpfHz") }), - right: Object.freeze({ hpfHz: asF32(configuration.right.hpfHz, "right.hpfHz"), lpfHz: asF32(configuration.right.lpfHz, "right.lpfHz") }), + left: Object.freeze({ hpfHz: cutoff(configuration.left.hpfHz), lpfHz: cutoff(configuration.left.lpfHz) }), + right: Object.freeze({ hpfHz: cutoff(configuration.right.hpfHz), lpfHz: cutoff(configuration.right.lpfHz) }), }); } const descriptor = CATALOG.effects.find((candidate) => candidate.id === configuration.effectId); - if (descriptor === undefined) throw new MisoUsageError(`unknown generated effect ${configuration.effectId}`); - if (descriptor.response === null) { - throw new MisoUsageError(`effect ${configuration.effectId} does not publish a response capability`); - } + // An effect the catalog does not hold, or one with no response, is the core's refusal to make + // (`response.effect.unknown`, `response.effect.unsupported`): the frame names it as given. + if (descriptor === undefined) return Object.freeze({ ...configuration }); const parameters = Object.fromEntries(descriptor.parameters.map((row) => { const supplied = configuration.parameters[row.name]; if (supplied !== undefined) return [row.name, supplied]; @@ -257,13 +230,13 @@ function normalizedConfiguration(configuration: ResponsePreviewConfiguration): R /** A synchronous query client around one analysis-only Wasm instance. */ export class ResponsePreviewModule { readonly #exports: RawExports; + /** The limits the caller set; one left out is left out of the frame, for the core's default. */ readonly #responseLimits: { - readonly maximumPreparedBytes: bigint; - readonly maximumTotalStateBytes: bigint; - readonly maximumScratchBytes: bigint; - readonly maximumAutomationSpansPerBlock: number; - readonly maximumResultBytes: number; - readonly requestDeadlineMs: number; + readonly maximumPreparedBytes: bigint | undefined; + readonly maximumTotalStateBytes: bigint | undefined; + readonly maximumScratchBytes: bigint | undefined; + readonly maximumAutomationSpansPerBlock: number | undefined; + readonly maximumResultBytes: number | undefined; }; #closed = false; #busy = false; @@ -271,13 +244,17 @@ export class ResponsePreviewModule { constructor(instance: WebAssembly.Instance, responseLimits: ResponsePreviewLimits = {}) { this.#exports = rawExports(instance); + // The deadline is the SDK's own timer; every other limit is the core's to bound. + finiteInteger(responseLimits.requestDeadlineMs ?? 5000, "requestDeadlineMs", 1, 2_147_483_647); this.#responseLimits = { - maximumPreparedBytes: u64(responseLimits.maximumPreparedBytes ?? (1 << 20), "maximumPreparedBytes"), - maximumTotalStateBytes: u64(responseLimits.maximumTotalStateBytes ?? (1 << 30), "maximumTotalStateBytes"), - maximumScratchBytes: u64(responseLimits.maximumScratchBytes ?? (1 << 30), "maximumScratchBytes"), - maximumAutomationSpansPerBlock: finiteInteger(responseLimits.maximumAutomationSpansPerBlock ?? 4096, "maximumAutomationSpansPerBlock", 1, 0xffff_ffff), - maximumResultBytes: finiteInteger(responseLimits.maximumResultBytes ?? (16 << 20), "maximumResultBytes", 1, 16 << 20), - requestDeadlineMs: finiteInteger(responseLimits.requestDeadlineMs ?? 5000, "requestDeadlineMs", 1, 2_147_483_647), + maximumPreparedBytes: responseLimits.maximumPreparedBytes === undefined + ? undefined : u64(responseLimits.maximumPreparedBytes, "maximumPreparedBytes"), + maximumTotalStateBytes: responseLimits.maximumTotalStateBytes === undefined + ? undefined : u64(responseLimits.maximumTotalStateBytes, "maximumTotalStateBytes"), + maximumScratchBytes: responseLimits.maximumScratchBytes === undefined + ? undefined : u64(responseLimits.maximumScratchBytes, "maximumScratchBytes"), + maximumAutomationSpansPerBlock: responseLimits.maximumAutomationSpansPerBlock, + maximumResultBytes: responseLimits.maximumResultBytes, }; for (const name of FRAME_EXPORTS) callable(this.#exports, name); } @@ -302,26 +279,20 @@ export class ResponsePreviewModule { } #query(request: ResponsePreviewQuery): ResponsePreviewResult { - if (request.configurationId < 0n || request.configurationId > 0xffff_ffff_ffff_ffffn) throw new MisoUsageError("configurationId must fit u64"); - finiteInteger(request.sampleRateHz, "sampleRateHz", 1, 0xffff_ffff); - finiteInteger(request.quantumFrames, "quantumFrames", 1, 0xffff_ffff); - finiteInteger(request.grid.points, "grid.points", 2, 0xffff_ffff); - const grid = gridName(request.grid.kind); - if (!Number.isFinite(request.grid.minimumHz) || !Number.isFinite(request.grid.maximumHz)) throw new MisoUsageError("grid endpoints must be finite"); - if (grid === "logarithmic" && request.grid.minimumHz <= 0) throw new MisoUsageError("logarithmic grid minimumHz must be positive"); - const channels = channelName(request.channels); const sections = wantsSections(request.fields); const configuration = normalizedConfiguration(request.configuration); const requestId = this.#nextRequestId; this.#nextRequestId += 1n; + // Nothing of the request is checked here but what has no spelling on the wire: the grid, the + // axis, the rate, the quality and every lane are the core's to accept or refuse. const frame = encodeResponseRequest({ requestId, ...this.#target(configuration), - grid, + grid: request.grid.kind, points: request.grid.points, - minimumHz: asF32(request.grid.minimumHz, "grid.minimumHz"), - maximumHz: asF32(request.grid.maximumHz, "grid.maximumHz"), - channels, + minimumHz: Math.fround(request.grid.minimumHz), + maximumHz: Math.fround(request.grid.maximumHz), + channels: request.channels ?? "both", sections, configurationId: request.configurationId, sampleRateHz: request.sampleRateHz, @@ -332,7 +303,6 @@ export class ResponsePreviewModule { maximumAutomationSpans: this.#responseLimits.maximumAutomationSpansPerBlock, maximumReplyBytes: this.#responseLimits.maximumResultBytes, }); - if (frame.length > MAXIMUM_REQUEST_FRAME_BYTES) throw new MisoUsageError("the response request exceeds the frame bound"); const memory = this.#exports.memory; const requestPtr = callable(this.#exports, "miso_engine_web_v1_response_frame_ptr")(frame.length); if (requestPtr === 0) throw responseError(resultValue("refusedBudget")); @@ -370,7 +340,7 @@ export class ResponsePreviewModule { throw new MisoUsageError("the generated catalog has no input-filter rows"); } const row = (control: number, lane: "left" | "right", value: number | undefined): ResponseFrameOverride => ({ - control, lane, value: asF32(value, `${lane}.cutoff`), + control, lane, value: cutoff(value), }); return { target: { kind: "inputFilters" }, @@ -383,23 +353,26 @@ export class ResponsePreviewModule { }; } const descriptor = CATALOG.effects.find((candidate) => candidate.id === configuration.effectId); - if (descriptor === undefined) throw new MisoUsageError(`unknown generated effect ${configuration.effectId}`); - const overrides: ResponseFrameOverride[] = Object.entries(configuration.parameters) - .flatMap(([name, value]) => parameterRows( - descriptor, - name, - value, - configuration.options.channel, - `response.${name}`, - )) - .map((row) => ({ control: Number(row.parameter_id), lane: row.channel as Channel, value: Number(row.value) })); + const named = Object.entries(configuration.parameters); + const overrides: ResponseFrameOverride[] = []; + if (descriptor === undefined && named.length > 0) { + // The catalog has no parameter names for this effect, so a name cannot be spelled as an ID. + throw new MisoUsageError(`unknown generated effect ${configuration.effectId} has no parameter names`); + } + if (descriptor !== undefined) { + for (const [name, value] of named) { + for (const row of parameterWireRows(descriptor, name, value, configuration.options.channel, `response.${name}`)) { + overrides.push({ control: Number(row.parameter_id), lane: row.channel as Channel, value: Number(row.value) }); + } + } + } if (configuration.options.bypass) overrides.push({ control: BYPASS_CONTROL, lane: "both", value: 1 }); return { target: { kind: "effect", effect: configuration.effectId, - quality: qualityName(configuration.options.quality), - linkMode: linkModeName(configuration.options.linkMode), + quality: configuration.options.quality, + linkMode: configuration.options.linkMode, }, overrides, }; diff --git a/sdk/src/core/session.ts b/sdk/src/core/session.ts index ba085b966..eea230b55 100644 --- a/sdk/src/core/session.ts +++ b/sdk/src/core/session.ts @@ -463,10 +463,36 @@ export function parameterRows( path: string, ): readonly ModelRecord[] { const row = effectParameter(descriptor, name, path); - const pair = row.channelPolicyName === "perLane" ? lanePair(raw) : undefined; if (row.channelPolicyName === "shared" && channel !== "both") { fail(path, `${name} is a shared parameter and must be addressed as 'both'`); } + return wireRows(row, raw, channel, path, true); +} + +/** + * The same rows as `parameterRows`, with only the refusals of a value that has no wire spelling: a + * name the descriptor does not hold, a label it does not declare, a value that is not a number. The + * lane a shared parameter is addressed on and the parameter's domain are the engine's to refuse + * (a response preview sends them as given, #1623). + */ +export function parameterWireRows( + descriptor: EffectDescriptor, + name: string, + raw: unknown, + channel: Channel, + path: string, +): readonly ModelRecord[] { + return wireRows(effectParameter(descriptor, name, path), raw, channel, path, false); +} + +function wireRows( + row: EffectParameterRow, + raw: unknown, + channel: Channel, + path: string, + checkDomain: boolean, +): readonly ModelRecord[] { + const pair = row.channelPolicyName === "perLane" ? lanePair(raw) : undefined; const addressed: readonly (readonly [Channel, unknown])[] = pair === undefined ? [[channel, raw]] : [["left", pair[0]], ["right", pair[1]]]; @@ -474,12 +500,20 @@ export function parameterRows( parameter_id: row.id, channel: lane, unit: row.unitName, - value: parameterScalar(row, value, `${path}${pair === undefined ? "" : `.${lane}`}`), + value: parameterScalar(row, value, `${path}${pair === undefined ? "" : `.${lane}`}`, checkDomain), })); } -/** Map a display-unit value onto the `f32` the wire carries, per the row's declared domain. */ -function parameterScalar(row: EffectParameterRow, value: unknown, path: string): number { +/** + * Map a display-unit value onto the `f32` the wire carries, per the row's declared domain. With + * `checkDomain` false a number outside the domain, or not finite, is carried as given. + */ +function parameterScalar( + row: EffectParameterRow, + value: unknown, + path: string, + checkDomain: boolean, +): number { if (row.domainName === "boolean") return bool(value, path) ? 1 : 0; if (row.domainName === "enumeration") { if (typeof value !== "string") fail(path, "expected an enumeration label"); @@ -489,6 +523,10 @@ function parameterScalar(row: EffectParameterRow, value: unknown, path: string): } return Math.fround(choice.value); } + if (!checkDomain) { + if (typeof value !== "number") fail(path, "expected a number"); + return Math.fround(value); + } const normalized = f32(value, path); if (normalized < Math.fround(row.minimum) || normalized > Math.fround(row.maximum)) { fail(path, `${row.name} is outside its catalog domain [${row.minimum}, ${row.maximum}]`); diff --git a/sdk/src/generated/response-frame.ts b/sdk/src/generated/response-frame.ts new file mode 100644 index 000000000..a2b18cd5c --- /dev/null +++ b/sdk/src/generated/response-frame.ts @@ -0,0 +1,422 @@ +// @generated by sdk/codegen/generate.mjs from sdk/assets/miso-engine-v1-response-frame.json; DO NOT EDIT. +// That JSON is the protocol crate's `RESPONSE_GET` frame table, read from the codec's own +// schema specs and strict code readers by `parameter-metadata --print-response-frame`; run +// `npm run assets` then `npm run codegen` in sdk/ to refresh both. The response codec reads +// it: no field ID, wire type or enum code of the frame is typed by hand. + +function deepFreeze(value: T): T { + if (value !== null && typeof value === "object") { + Object.freeze(value); + for (const child of Object.values(value as Record)) { + deepFreeze(child); + } + } + return value; +} + +export const RESPONSE_FRAME = deepFreeze( +{ + "schema": "miso.response.frame.v1", + "command": 14, + "messages": { + "request": { + "name": "ResponseRequest", + "fields": { + "configuration": { + "id": 1, + "wire": 11, + "wireName": "message", + "mandatory": false, + "repeated": false, + "message": "ResponseConfiguration" + }, + "address": { + "id": 2, + "wire": 11, + "wireName": "message", + "mandatory": false, + "repeated": false, + "message": "ControlAddress" + }, + "stripId": { + "id": 3, + "wire": 9, + "wireName": "utf8", + "mandatory": false, + "repeated": false, + "message": null + }, + "grid": { + "id": 4, + "wire": 1, + "wireName": "u8", + "mandatory": true, + "repeated": false, + "message": null + }, + "points": { + "id": 5, + "wire": 3, + "wireName": "u32", + "mandatory": true, + "repeated": false, + "message": null + }, + "minimumHz": { + "id": 6, + "wire": 6, + "wireName": "f32", + "mandatory": true, + "repeated": false, + "message": null + }, + "maximumHz": { + "id": 7, + "wire": 6, + "wireName": "f32", + "mandatory": true, + "repeated": false, + "message": null + }, + "channels": { + "id": 8, + "wire": 1, + "wireName": "u8", + "mandatory": true, + "repeated": false, + "message": null + }, + "sections": { + "id": 9, + "wire": 8, + "wireName": "bool", + "mandatory": true, + "repeated": false, + "message": null + }, + "override": { + "id": 10, + "wire": 11, + "wireName": "message", + "mandatory": false, + "repeated": true, + "message": "ResponseOverride" + }, + "configurationId": { + "id": 11, + "wire": 4, + "wireName": "u64", + "mandatory": false, + "repeated": false, + "message": null + }, + "sampleRateHz": { + "id": 12, + "wire": 3, + "wireName": "u32", + "mandatory": false, + "repeated": false, + "message": null + }, + "quantumFrames": { + "id": 13, + "wire": 3, + "wireName": "u32", + "mandatory": false, + "repeated": false, + "message": null + }, + "maximumPreparedBytes": { + "id": 14, + "wire": 4, + "wireName": "u64", + "mandatory": false, + "repeated": false, + "message": null + }, + "maximumStateBytes": { + "id": 15, + "wire": 4, + "wireName": "u64", + "mandatory": false, + "repeated": false, + "message": null + }, + "maximumScratchBytes": { + "id": 16, + "wire": 4, + "wireName": "u64", + "mandatory": false, + "repeated": false, + "message": null + }, + "maximumAutomationSpans": { + "id": 17, + "wire": 3, + "wireName": "u32", + "mandatory": false, + "repeated": false, + "message": null + }, + "maximumReplyBytes": { + "id": 18, + "wire": 3, + "wireName": "u32", + "mandatory": false, + "repeated": false, + "message": null + } + } + }, + "configuration": { + "name": "ResponseConfiguration", + "fields": { + "kind": { + "id": 1, + "wire": 1, + "wireName": "u8", + "mandatory": true, + "repeated": false, + "message": null + }, + "effect": { + "id": 2, + "wire": 9, + "wireName": "utf8", + "mandatory": false, + "repeated": false, + "message": null + }, + "quality": { + "id": 3, + "wire": 1, + "wireName": "u8", + "mandatory": false, + "repeated": false, + "message": null + }, + "linkMode": { + "id": 4, + "wire": 1, + "wireName": "u8", + "mandatory": false, + "repeated": false, + "message": null + } + } + }, + "override": { + "name": "ResponseOverride", + "fields": { + "control": { + "id": 1, + "wire": 3, + "wireName": "u32", + "mandatory": true, + "repeated": false, + "message": null + }, + "lane": { + "id": 2, + "wire": 1, + "wireName": "u8", + "mandatory": true, + "repeated": false, + "message": null + }, + "value": { + "id": 3, + "wire": 6, + "wireName": "f32", + "mandatory": true, + "repeated": false, + "message": null + } + } + }, + "result": { + "name": "ResponseResult", + "fields": { + "requestTag": { + "id": 1, + "wire": 4, + "wireName": "u64", + "mandatory": false, + "repeated": false, + "message": null + }, + "sampleRateHz": { + "id": 2, + "wire": 3, + "wireName": "u32", + "mandatory": false, + "repeated": false, + "message": null + }, + "configurationId": { + "id": 3, + "wire": 4, + "wireName": "u64", + "mandatory": false, + "repeated": false, + "message": null + }, + "floorDb": { + "id": 4, + "wire": 6, + "wireName": "f32", + "mandatory": false, + "repeated": false, + "message": null + }, + "bypass": { + "id": 5, + "wire": 8, + "wireName": "bool", + "mandatory": false, + "repeated": false, + "message": null + }, + "sectionCount": { + "id": 6, + "wire": 3, + "wireName": "u32", + "mandatory": false, + "repeated": false, + "message": null + }, + "enabledLeft": { + "id": 7, + "wire": 3, + "wireName": "u32", + "mandatory": false, + "repeated": false, + "message": null + }, + "enabledRight": { + "id": 8, + "wire": 3, + "wireName": "u32", + "mandatory": false, + "repeated": false, + "message": null + }, + "retainedBytes": { + "id": 9, + "wire": 4, + "wireName": "u64", + "mandatory": false, + "repeated": false, + "message": null + }, + "frequenciesHz": { + "id": 10, + "wire": 15, + "wireName": "packedF32", + "mandatory": false, + "repeated": false, + "message": null + }, + "leftDb": { + "id": 11, + "wire": 15, + "wireName": "packedF32", + "mandatory": false, + "repeated": false, + "message": null + }, + "rightDb": { + "id": 12, + "wire": 15, + "wireName": "packedF32", + "mandatory": false, + "repeated": false, + "message": null + }, + "sectionsLeftDb": { + "id": 13, + "wire": 15, + "wireName": "packedF32", + "mandatory": false, + "repeated": false, + "message": null + }, + "sectionsRightDb": { + "id": 14, + "wire": 15, + "wireName": "packedF32", + "mandatory": false, + "repeated": false, + "message": null + } + } + } + }, + "codes": { + "configurationKind": [ + { + "name": "effect", + "code": 1 + }, + { + "name": "inputFilters", + "code": 2 + } + ], + "grid": [ + { + "name": "linear", + "code": 1 + }, + { + "name": "logarithmic", + "code": 2 + } + ], + "lane": [ + { + "name": "left", + "code": 1 + }, + { + "name": "right", + "code": 2 + }, + { + "name": "both", + "code": 3 + } + ], + "quality": [ + { + "name": "draft", + "code": 1 + }, + { + "name": "normal", + "code": 2 + }, + { + "name": "high", + "code": 3 + } + ], + "linkMode": [ + { + "name": "dual_mono", + "code": 1 + }, + { + "name": "maximum", + "code": 2 + }, + { + "name": "average", + "code": 3 + } + ] + } +} as const, +); + +export type ResponseFrame = typeof RESPONSE_FRAME; diff --git a/sdk/test/response-evals.mjs b/sdk/test/response-evals.mjs index bf76ca102..046779f4a 100644 --- a/sdk/test/response-evals.mjs +++ b/sdk/test/response-evals.mjs @@ -5,6 +5,7 @@ import { MisoEngineAsset } from "../src/core/asset.ts"; import { MisoEngineError, MisoUsageError, resultName } from "../src/core/errors.ts"; import { RESPONSE_CAPABILITIES, ResponsePreviewModule } from "../src/core/response.ts"; import { createResponsePreview as createHeadlessResponsePreview } from "../src/headless/response.ts"; +import { decodeResponseReply, encodeResponseRequest } from "../src/core/response-frame.ts"; import { effect } from "../src/core/session.ts"; import { moduleBytes } from "./support.mjs"; @@ -265,3 +266,148 @@ test("candidate Wasm: the engine refuses 4097 points with its own code", { await preview.close(); } }); + +/** A configuration the way a caller who skips `effect()` writes it: no catalog check has run. */ +function forged(effectId, parameters, options = {}) { + return { + effectId, + parameters, + options: { bypass: false, quality: "normal", linkMode: "dual_mono", channel: "both", ...options }, + }; +} + +/** + * Send one raw request frame through the module's session-less entry, as a client with no codec of + * the core's would, and read the reply. + */ +function submitFrame(instance, frame) { + const exports = instance.exports; + const pointer = exports.miso_engine_web_v1_response_frame_ptr(frame.length); + assert.notEqual(pointer, 0); + new Uint8Array(exports.memory.buffer, pointer, frame.length).set(frame); + assert.equal(exports.miso_engine_web_v1_response_frame_submit(frame.length), 0); + const replyPointer = exports.miso_engine_web_v1_response_frame_reply_ptr(); + const replyBytes = exports.miso_engine_web_v1_response_frame_reply_bytes(); + return decodeResponseReply(new Uint8Array(exports.memory.buffer.slice(replyPointer, replyPointer + replyBytes))); +} + +/** + * #1623 gate 1: the SDK checks nothing the core checks. Each row is a request the SDK used to + * refuse itself with a `MisoUsageError`; it now reaches the core, and the refusal is the core's + * own, with the dotted code and the result class the C ABI gives the same request + * (`crates/capi/tests/response_get.rs` and `crates/host-core/tests/response_frame.rs` hold those + * rows to the same codes and statuses). + */ +test("candidate Wasm: a grid, a rate, a quality and a value the SDK used to refuse are the core's refusals", { + skip: !process.env.MISO_ENGINE_SDK_ARTIFACTS_HEX, +}, async () => { + const asset = await MisoEngineAsset.load(await moduleBytes()); + const preview = await createHeadlessResponsePreview({ asset }); + const eq = (parameters = {}, options = {}) => forged("miso.parametric-eq", parameters, options); + const grid = (override) => ({ ...query.grid, kind: "logarithmic", points: 8, minimumHz: 20, maximumHz: 20_000, ...override }); + const rows = [ + ["a logarithmic axis from zero", { grid: grid({ minimumHz: 0 }) }, "response.grid", "invalidArgument"], + ["a negative logarithmic minimum", { grid: grid({ minimumHz: -5 }) }, "response.grid", "invalidArgument"], + ["a NaN endpoint", { grid: grid({ maximumHz: Number.NaN }) }, "protocol.failure", "invalidArgument"], + ["an infinite endpoint", { grid: grid({ kind: "linear", maximumHz: Number.POSITIVE_INFINITY }) }, "protocol.failure", "invalidArgument"], + ["one point", { grid: grid({ points: 1 }) }, "response.points", "invalidArgument"], + ["a sample rate that is no launch rate", { sampleRateHz: 12_345 }, "response.rate", "invalidArgument"], + ["a zero quantum", { quantumFrames: 0 }, "response.rate", "invalidArgument"], + ["draft quality", { configuration: eq({}, { quality: "draft" }) }, "effect.quality.unsupported", "invalidArgument"], + ["high quality", { configuration: eq({}, { quality: "high" }) }, "effect.quality.unsupported", "invalidArgument"], + ["a non-finite value", { configuration: eq({ "band-1-gain": Number.NaN }) }, "protocol.failure", "invalidArgument"], + ["a gain outside its domain", { configuration: eq({ "band-1-gain": 500 }) }, "effect.parameter.initial", "invalidArgument"], + ["an effect the engine does not know", { configuration: forged("miso.no-such-effect", {}) }, "response.effect.unknown", "invalidArgument"], + ["an effect with no response", { configuration: forged("miso.compressor", {}) }, "response.effect.unsupported", "unsupported"], + [ + "a shared parameter on one lane, of an effect with no response", + { configuration: forged("miso.compressor", { threshold: -20 }, { channel: "left" }) }, + "response.effect.unsupported", + "unsupported", + ], + ]; + try { + for (const [name, change, code, result] of rows) { + assert.throws( + () => preview.query({ ...query, ...change }), + (error) => error instanceof MisoEngineError && !(error instanceof MisoUsageError) + && error.code === code && resultName(error.result, "call") === result, + name, + ); + } + // The same request with a left-only scalar lane is accepted: the core, not the SDK, says + // which lane an effect takes. + const left = preview.query({ + ...query, + configuration: eq({ "band-1-gain": 3 }, { channel: "left" }), + grid: grid({}), + }); + assert.equal(left.frequenciesHz.length, 8); + } finally { + await preview.close(); + } +}); + +test("candidate Wasm: a lane given twice is refused by the core with the C ABI's code", { + skip: !process.env.MISO_ENGINE_SDK_ARTIFACTS_HEX, +}, async () => { + const asset = await MisoEngineAsset.load(await moduleBytes()); + const instance = await asset.instantiate(); + const frame = (overrides) => encodeResponseRequest({ + requestId: 1n, + target: { kind: "effect", effect: "miso.parametric-eq", quality: "normal", linkMode: "dual_mono" }, + grid: "linear", + points: 8, + minimumHz: 20, + maximumHz: 20_000, + channels: "both", + sections: false, + overrides, + configurationId: 1n, + sampleRateHz: 48_000, + quantumFrames: 128, + }); + for (const overrides of [ + [{ control: 4, lane: "left", value: 1 }, { control: 4, lane: "left", value: 2 }], + [{ control: 4, lane: "both", value: 1 }, { control: 4, lane: "left", value: 2 }], + ]) { + const reply = submitFrame(instance, frame(overrides)); + assert.equal(reply.ok, false); + assert.equal(reply.status, "invalid_field"); + assert.deepEqual(reply.diagnostics.map((row) => row.code), ["response.override.duplicate"]); + } + // A limit left out of the request is a field left out of the frame, for the core's default. + const accepted = submitFrame(instance, frame([{ control: 4, lane: "both", value: 3 }])); + assert.equal(accepted.ok, true); + assert.equal(accepted.curves.frequenciesHz.length, 8); +}); + +test("a name or a number with no wire spelling stays a usage error before any byte is made", () => { + const request = { + requestId: 1n, + target: { kind: "inputFilters" }, + grid: "linear", + points: 8, + minimumHz: 20, + maximumHz: 20_000, + channels: "both", + sections: false, + overrides: [], + configurationId: 1n, + sampleRateHz: 48_000, + quantumFrames: 128, + }; + assert.ok(encodeResponseRequest(request) instanceof Uint8Array); + for (const [name, change] of [ + ["a grid with no code", { grid: "bogus" }], + ["a lane with no code", { channels: "middle" }], + ["an override lane with no code", { overrides: [{ control: 1, lane: "middle", value: 0 }] }], + ["a quality with no code", { target: { kind: "effect", effect: "e", quality: "ultra", linkMode: "dual_mono" } }], + ["a link mode with no code", { target: { kind: "effect", effect: "e", quality: "normal", linkMode: "sum" } }], + ["a point count that is no u32", { points: 2 ** 32 }], + ["a fractional rate", { sampleRateHz: 48_000.5 }], + ["a configuration ID that is not a bigint", { configurationId: 1 }], + ]) { + assert.throws(() => encodeResponseRequest({ ...request, ...change }), MisoUsageError, name); + } +}); diff --git a/tools/parameter-metadata/Cargo.toml b/tools/parameter-metadata/Cargo.toml index dd97fd0eb..1fd3e2557 100644 --- a/tools/parameter-metadata/Cargo.toml +++ b/tools/parameter-metadata/Cargo.toml @@ -33,6 +33,8 @@ effect-compiler.workspace = true effect-contract.workspace = true host-core.workspace = true host-web.workspace = true +# The `RESPONSE_GET` frame table the SDK generates its codec from (`response_frame`, #1623). +protocol.workspace = true session.workspace = true # The canonical writer corpus manifest is read as data. serde_json = "=1.0.151" @@ -41,11 +43,10 @@ serde_json = "=1.0.151" workspace = true [dev-dependencies] -# Test-only (#1615): `tests/control_command_heap.rs` drives host-web's control command with -# `protocol` frames and measures its peak heap with `bench_support::alloc`. It lives here because +# `tests/control_command_heap.rs` (#1615) drives host-web's control command with `protocol` frames +# and measures its peak heap with `bench_support::alloc`. It lives here because # `scripts/check-bench-policy.sh` keeps `bench-support` out of every `hosts/` package, dev # dependencies included, and this tool already links both host-web and bench-support. -protocol.workspace = true # Test-only: `host_core::verify_stages(false)` turns off the debug stage oracle, so the heap numbers # are the release path's. host-core = { workspace = true, features = ["test-support"] } diff --git a/tools/parameter-metadata/src/lib.rs b/tools/parameter-metadata/src/lib.rs index e86e4e426..e5d5c70a6 100644 --- a/tools/parameter-metadata/src/lib.rs +++ b/tools/parameter-metadata/src/lib.rs @@ -80,6 +80,7 @@ macro_rules! append { } pub mod abi_layout; +pub mod response_frame; pub mod session_keys; use std::io::Write as _; @@ -135,7 +136,7 @@ pub const PLANE_OBSERVATION: &str = "observation"; fn usage() -> ! { eprintln!( "usage: parameter_metadata --write DIRECTORY | --check DIRECTORY \ -| --print | --print-abi-layout | --print-session-keys" +| --print | --print-abi-layout | --print-session-keys | --print-response-frame" ); std::process::exit(2) } @@ -1106,8 +1107,8 @@ const fn smoothing_name(rule: SmoothingRule) -> &'static str { } } -/// Command-line entry point: `--write DIR`, `--check DIR`, `--print`, `--print-abi-layout` or -/// `--print-session-keys`. +/// Command-line entry point: `--write DIR`, `--check DIR`, `--print`, `--print-abi-layout`, +/// `--print-session-keys` or `--print-response-frame`. /// /// `--write`/`--check` cover **both** emitted documents. They are one tool because they are one /// transcription discipline over one engine: splitting them into two binaries would let an @@ -1117,7 +1118,8 @@ const fn smoothing_name(rule: SmoothingRule) -> &'static str { /// /// `--print-session-keys` is the SDK's session key tables (issue #1569). It is an SDK source asset, /// not a member of the release artifact set, so `--write`/`--check` leave it out and -/// `sdk/codegen/assets.mjs` renders it itself. +/// `sdk/codegen/assets.mjs` renders it itself. `--print-response-frame` is the same kind of asset: +/// the `RESPONSE_GET` frame table of the protocol crate (issue #1623). pub fn run() { let mut arguments = std::env::args().skip(1); let mode = arguments.next().unwrap_or_else(|| usage()); @@ -1140,6 +1142,12 @@ pub fn run() { } print!("{}", session_keys::render()); } + "--print-response-frame" => { + if arguments.next().is_some() { + usage(); + } + print!("{}", response_frame::render()); + } "--write" | "--check" => { let directory = PathBuf::from(arguments.next().unwrap_or_else(|| usage())); if arguments.next().is_some() { diff --git a/tools/parameter-metadata/src/response_frame.rs b/tools/parameter-metadata/src/response_frame.rs new file mode 100644 index 000000000..00c90461b --- /dev/null +++ b/tools/parameter-metadata/src/response_frame.rs @@ -0,0 +1,82 @@ +//! Build-time transcription of the `RESPONSE_GET` frame (issue #1623). +//! +//! The browser SDK writes the `RESPONSE_GET` request and reads its reply without the Rust codec. +//! Before this document it held the field IDs and the grid, lane, quality and link codes in a hand +//! copy, and the same copy refused requests the core also refuses. The frame table of the +//! `protocol` crate ([`protocol::response_frame_table`]) is read from the codec's own schema specs +//! and strict readers; this module writes it down as the SDK's source asset, and `--check`-style +//! comparison (`sdk/codegen/assets.mjs`, `response_frame_asset_is_current`) makes a hand edit or a +//! schema change that did not reach the SDK fail before a consumer can observe it. + +use protocol::{FrameCodeTable, FrameField, FrameMessage, response_frame_table}; + +/// The emitted file name. It is an SDK source asset, not a member of the release artifact set. +pub const OUTPUT_NAME: &str = "miso-engine-v1-response-frame.json"; +/// The schema tag every consumer must check before reading a field. +pub const SCHEMA: &str = "miso.response.frame.v1"; + +fn field(out: &mut String, row: &FrameField, last: bool) { + let message = row + .message + .map_or_else(|| "null".to_owned(), |name| format!("\"{name}\"")); + append!( + out, + " \"{}\": {{ \"id\": {}, \"wire\": {}, \"wireName\": \"{}\", \"mandatory\": {}, \"repeated\": {}, \"message\": {message} }}{}\n", + row.name, + row.id, + row.wire, + row.wire_name, + row.mandatory, + row.repeated, + if last { "" } else { "," } + ); +} + +fn message(out: &mut String, row: &FrameMessage, last: bool) { + append!(out, " \"{}\": {{\n", row.key); + append!(out, " \"name\": \"{}\",\n", row.name); + out.push_str(" \"fields\": {\n"); + for (index, entry) in row.fields.iter().enumerate() { + field(out, entry, index + 1 == row.fields.len()); + } + out.push_str(" }\n"); + append!(out, " }}{}\n", if last { "" } else { "," }); +} + +fn codes(out: &mut String, table: &FrameCodeTable, last: bool) { + append!(out, " \"{}\": [\n", table.key); + for (index, row) in table.codes.iter().enumerate() { + append!( + out, + " {{ \"name\": \"{}\", \"code\": {} }}{}\n", + row.name, + row.code, + if index + 1 == table.codes.len() { + "" + } else { + "," + } + ); + } + append!(out, " ]{}\n", if last { "" } else { "," }); +} + +/// The document `--print-response-frame` writes. +#[must_use] +pub fn render() -> String { + let frame = response_frame_table(); + let mut out = String::new(); + out.push_str("{\n"); + append!(&mut out, " \"schema\": \"{SCHEMA}\",\n"); + append!(&mut out, " \"command\": {},\n", frame.command); + out.push_str(" \"messages\": {\n"); + for (index, row) in frame.messages.iter().enumerate() { + message(&mut out, row, index + 1 == frame.messages.len()); + } + out.push_str(" },\n \"codes\": {\n"); + for (index, table) in frame.code_tables.iter().enumerate() { + codes(&mut out, table, index + 1 == frame.code_tables.len()); + } + out.push_str(" }\n}\n"); + out +} diff --git a/tools/parameter-metadata/tests/response_frame.rs b/tools/parameter-metadata/tests/response_frame.rs new file mode 100644 index 000000000..4261f34d4 --- /dev/null +++ b/tools/parameter-metadata/tests/response_frame.rs @@ -0,0 +1,24 @@ +//! Issue #1623: the `RESPONSE_GET` frame table the SDK generates its codec from is the protocol +//! crate's own. + +use std::path::Path; + +use parameter_metadata::response_frame; + +/// `sdk/assets/miso-engine-v1-response-frame.json` is what `--print-response-frame` writes. The SDK +/// checks of `check-sdk-generated.sh` compare the generated TypeScript with this asset; this holds +/// the asset to the protocol schema even where the check runs against an engine artifact directory, +/// which does not carry it. +#[test] +fn response_frame_asset_is_current() { + let path = Path::new(env!("CARGO_MANIFEST_DIR")) + .join("../../sdk/assets") + .join(response_frame::OUTPUT_NAME); + let checked = std::fs::read_to_string(&path).expect("read the checked response frame table"); + assert_eq!( + checked, + response_frame::render(), + "{} is stale; run `node codegen/assets.mjs` in sdk/", + path.display() + ); +} From 8ff0cce3920467285f412b2ce8d84d4ad9e754e3 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 19:42:53 +0000 Subject: [PATCH 07/12] Carry, then retarget a strip input section across a plan swap (#1630) Co-Authored-By: Claude Opus 5.5 --- ...-strip-input-section-across-a-plan-swap.md | 52 ++++ crates/host-core/src/live_delta.rs | 163 +------------ crates/host-core/src/prepare.rs | 222 +++++++++++++++++- crates/host-core/tests/successor_swap.rs | 192 +++++++++++++-- 4 files changed, 436 insertions(+), 193 deletions(-) diff --git a/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md b/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md index cb5df007d..9bd10802e 100644 --- a/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md +++ b/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md @@ -50,3 +50,55 @@ then ramps to the new value, on both hosts, instead of restarting the section. ## Size Half a day to a day. + +## Decision record: #1630 (batch 14b) + +**Shipped.** The input join in `crates/host-core/src/prepare.rs` (the `carried_inputs` block of +the successor's preparation) now keys on the section's structure. Every strip's input section has +the same stages (trim, polarity, two filter sections per channel), so the structure is the control +kind (`row.live == live_input`, as before) plus the rate the designs are for. On a plan with input +cells, a surviving strip of the same kind always carries, and the records the live path gives the +same edit (`input_records`, #1556) become its retarget (`InputJoin::retarget`). They are written +through the successor's own `InputCells`, stamped on its held gate in the same block as the strip, +effect and route retargets, with the session's ramps or the transaction's edit ramp for +`(Strip, Input)` (#1394 D6). A plan without input cells cannot retarget, so it still carries only +bit-equal sections (`same_input_section`, unchanged #1276 behaviour). + +**Departures and code truth.** + +- `input_records` and `filter_edits` moved from `live_delta.rs` (built only with + `control-provider`) to `prepare.rs` (every build), as the strip-retarget helpers did for #1277 + D5; the classifier imports them and maps `OutOfDomain` to `LiveRebuild::Domain`. One function, + so a retarget and a live edit cannot disagree. This edits `crates/host-core/src/live_delta.rs`, + outside the authorized paths, mechanically (move, imports, one doc link). +- The join adds a sample-rate check (`base.inventory.sample_rate_hz == compiled rate`): a carried + lane's coefficients and filter state are designed for the predecessor's rate, so a rate change + is a structural difference (D1, "their designs' shape"). Before, a rate change with bit-equal + values carried. +- A value the live path refuses (out of domain) restarts the section instead of carrying; a + compiled model does not reach it. +- No builtins or `crates/engine/src/realtime/` edit was needed: the input stage's carry entry + (#1276 `export_lane`/`import_lane`) and the successor's cell drain already order carry before + retarget on the swap block. No new realtime region. + +**Superseded tests.** `a_restarting_input_lane_does_not_count_its_replaced_writes` and +`a_pending_trim_record_in_a_later_bank_survives_the_swap` restarted one strip by a trim difference +between committed and B, which now carries. They now restart it with a committed model that lacks +the strip (`without_track`), so they keep their discrimination. Gate 4 of #1276 +(`a_strip_whose_input_section_changed_starts_at_rest`) runs on plans without input cells and stays. + +**Gates** (`crates/host-core/tests/successor_swap.rs`, `control-provider`, both widths each): + +1. `a_trim_change_carries_and_retargets_the_input_section` +2. `a_filter_cutoff_change_carries_and_retargets_the_input_section` (HPF on both lanes, LPF one + lane), `a_polarity_change_carries_and_retargets_the_input_section`. + +Each: the carry-then-retarget run equals, in every block, a run that wrote the classifier's records +for the same edit to the predecessor before the swap; it differs at the swap block from a run whose +section restarts and from a run without the edit. Red on the old join (first difference at block 6) +and red when the retarget write is dropped. +3. The same runs: swap block 0 allocations, reallocations and frees (`bench_support::alloc` thread + counters after `audit::warm_up`) and 0 audited allocations. `cargo test --locked -p host-core + --all-features`, `-p capi`, `-p host-web --features host-web/test-support`: pass. Clippy + `-p host-core --all-targets` with and without `--all-features`: clean. `audit capi`: + `pcm_digest` `bf7dcedd1d92621f`, 0 violations. `realtime_policy`: ok, 149 regions (unchanged). diff --git a/crates/host-core/src/live_delta.rs b/crates/host-core/src/live_delta.rs index 3d59c6487..b893b6476 100644 --- a/crates/host-core/src/live_delta.rs +++ b/crates/host-core/src/live_delta.rs @@ -23,21 +23,15 @@ use std::cell::OnceCell; use std::collections::{BTreeMap, BTreeSet}; -use crate::control_preparation::{ - INPUT_FILTER_TARGET_CAPACITY, InputFilterEdit, InputFilterPreparer, -}; use crate::route_controls::RouteControlRecord; -use builtins::{ - BuiltinLaneSelector, PreparedInputFilterTarget, checked_trim_gain, validate_input_filter_pair, - validate_prepared_input_filter_target, -}; +use builtins::{checked_trim_gain, validate_prepared_input_filter_target}; use builtins_compiler::{TrackControlRecord, TrackFaderRecord, TrackInputRecord}; use controls::{ ChangedControl, ControlAddress, ControlClass, EffectiveStrip, Family, LocalInputs, ModelView, Owner, Section, }; use effect_compiler::{LiveEffectAddress, launch_native_effect_registry, resolve_initial_values}; -use effect_contract::{EffectControlRecord, ParameterChannel, PreparedEffectTarget}; +use effect_contract::{EffectControlRecord, PreparedEffectTarget}; use session::{ EditRamps, Effect, EffectIdentity, EffectParam, LoweredRacks, Route, RouteDestination, SessionModel, StableId, StripRef, canonical_session_json, @@ -60,7 +54,7 @@ pub fn take_fast_classification_count() -> usize { // every build (a successor's carry-then-retarget, #1277 D5); the classifier calls them. use crate::prepare::{ EffectiveDiff, LevelRules, OutOfDomain, RouteEditRamps, StripChange, StripEditRamps, - effect_instance_records, effective_fader, fader_records, lane_records, level_differs, + effect_instance_records, effective_fader, fader_records, input_records, level_differs, matrix_record, route_record_between, route_source_strip, same_params, strip_at, }; use crate::prepare::{GraphJumpLengths, LiveRampRow, LiveRamps, MonitorBase, session_jump_lengths}; @@ -312,8 +306,9 @@ pub use crate::prepare::LiveRebuild; /// the same dB bits), a `Mute` per lane whose mute changes (one `Both` when both change to the /// same value), a matrix record when a lowered coefficient's bits change, and the strip's input /// records (a `TrimDb` and a `PolarityInvert` by the same rules, then the filter targets -/// [`InputFilterPreparer`] designs for the changed cutoffs, #1556). A send into a submix gets one -/// record when its folded target or its gate changes (`route_records`). A fader move on a +/// [`InputFilterPreparer`](crate::InputFilterPreparer) designs for the changed cutoffs, #1556). +/// A send into a submix gets one record when its folded target or its gate changes +/// (`route_records`). A fader move on a /// lane that stays muted still gets its record, so the stage remembers the gain for a later /// unmute. An effect record is emitted only for a resolved value whose bits change: a `Both` /// value rewritten as equal per-lane values gives none. @@ -1420,7 +1415,8 @@ fn lower<'a>( ctx.ramps, ctx.edit, ) - .map(Lowered::Input), + .map(Lowered::Input) + .map_err(|_| LiveRebuild::Domain), Family::Setting | Family::Route => Ok(Lowered::Nothing), } } @@ -1451,149 +1447,6 @@ fn route_records<'a>( })) } -/// The input records that bring a strip's input section from `before`'s values to `after`'s -/// (#1556 D2-D4), in write order: trims, then polarities, then filter targets, each left before -/// right. A record is emitted only when the value the render plane holds changes. -/// -/// - `TrimDb`: one per lane whose [`checked_trim_gain`] bits change, or one `Both` when both lanes -/// change to the same `trim_db` bits. The comparison is of the gain's bits, so `0.0` to `-0.0` -/// (the same gain) gives no record. -/// - `PolarityInvert`: one per lane whose value changes, or one `Both` when both change to the -/// same value. -/// - `PreparedFilter`: the targets [`InputFilterPreparer`] designs, off the render thread, from the -/// pre-commit normalized pairs and the changed values (see [`filter_edits`]). -/// -/// # Errors -/// -/// [`LiveRebuild::Domain`] when a trim is outside the domain, or a pair or a designed target is -/// refused. -fn input_records( - before: &session::DualMonoBuiltins, - after: &session::DualMonoBuiltins, - sample_rate_hz: u32, - ramps: LiveRamps, - edit: StripEditRamps, -) -> Result, LiveRebuild> { - let lanes_before = [&before.left, &before.right]; - let lanes_after = [&after.left, &after.right]; - let mut records = Vec::with_capacity(4 + INPUT_FILTER_TARGET_CAPACITY); - - let gain_bits = |db: f32| { - checked_trim_gain(db) - .map(f32::to_bits) - .map_err(|_| LiveRebuild::Domain) - }; - let mut trim_changed = [false; 2]; - for lane in 0..2 { - trim_changed[lane] = - gain_bits(lanes_before[lane].trim_db)? != gain_bits(lanes_after[lane].trim_db)?; - } - for (lanes, lane) in lane_records( - trim_changed, - lanes_after[0].trim_db.to_bits() == lanes_after[1].trim_db.to_bits(), - ) { - records.push(TrackInputRecord::TrimDb { - lanes, - db: lanes_after[lane].trim_db, - smoothing_samples: ramps.resolve(LiveRampRow::InputTrim, edit.input), - }); - } - - let polarity_changed = - [0, 1].map(|lane| lanes_before[lane].polarity_invert != lanes_after[lane].polarity_invert); - for (lanes, lane) in lane_records( - polarity_changed, - lanes_after[0].polarity_invert == lanes_after[1].polarity_invert, - ) { - records.push(TrackInputRecord::PolarityInvert { - lanes, - inverted: lanes_after[lane].polarity_invert, - smoothing_samples: ramps.resolve(LiveRampRow::PolarityInvert, edit.input), - }); - } - - // The filters: compare the values each lane's pair normalizes to, so `-0.0` to `0.0` is no - // change (#1556 D4). - let pair = |lane: &session::ChannelBuiltins| { - validate_input_filter_pair(sample_rate_hz, lane.hpf_hz, lane.lpf_hz) - .map_err(|_| LiveRebuild::Domain) - }; - let pairs_before = [pair(lanes_before[0])?, pair(lanes_before[1])?]; - let pairs_after = [pair(lanes_after[0])?, pair(lanes_after[1])?]; - let mut changed = [(false, false); 2]; - for lane in 0..2 { - changed[lane] = ( - pairs_before[lane].hpf_hz.to_bits() != pairs_after[lane].hpf_hz.to_bits(), - pairs_before[lane].lpf_hz.to_bits() != pairs_after[lane].lpf_hz.to_bits(), - ); - } - let edits = filter_edits(changed, pairs_after); - if !edits.is_empty() { - let seeds = [ - pairs_before[0].hpf_hz, - pairs_before[0].lpf_hz, - pairs_before[1].hpf_hz, - pairs_before[1].lpf_hz, - ]; - let mut out = [PreparedInputFilterTarget { - lanes: BuiltinLaneSelector::Both, - section: 0, - pair: [0.0; 2], - coefficients: [0.0; 6], - }; INPUT_FILTER_TARGET_CAPACITY]; - let (_, count) = InputFilterPreparer - .prepare(sample_rate_hz, &seeds, &edits, &mut out) - .map_err(|_| LiveRebuild::Domain)?; - records.extend( - out[..count] - .iter() - .map(|&target| TrackInputRecord::PreparedFilter { target }), - ); - } - Ok(records) -} - -/// The semantic edits that carry a strip's changed filter values (#1556 D4): per lane, `3` or `4` -/// for a one-sided change and the atomic pair `0` when both of the lane's values change (so a -/// pair that two one-sided edits would refuse, an HPF raised past the old LPF while the LPF -/// rises, is one valid edit). The same edit on both lanes is one `Both` edit, which is how the -/// preparer later merges equal targets. -fn filter_edits( - changed: [(bool, bool); 2], - after: [builtins::InputFilterPair; 2], -) -> Vec { - let lane_edit = |lane: usize| { - let pair = after[lane]; - match changed[lane] { - (false, false) => None, - (true, false) => Some((3, pair.hpf_hz, 0.0)), - (false, true) => Some((4, pair.lpf_hz, 0.0)), - (true, true) => Some((0, pair.hpf_hz, pair.lpf_hz)), - } - }; - let (left, right) = (lane_edit(0), lane_edit(1)); - let edit = |(parameter_id, value0, value1): (u32, f32, f32), channel| InputFilterEdit { - parameter_id, - channel, - value0, - value1, - }; - match (left, right) { - (Some(left), Some(right)) - if left.0 == right.0 - && left.1.to_bits() == right.1.to_bits() - && left.2.to_bits() == right.2.to_bits() => - { - vec![edit(left, ParameterChannel::Both)] - } - _ => left - .map(|left| edit(left, ParameterChannel::Left)) - .into_iter() - .chain(right.map(|right| edit(right, ParameterChannel::Right))) - .collect(), - } -} - /// Step 4: a changed effect's `params` must resolve, as preparation resolves them. An unknown /// parameter ID, a unit mismatch or a duplicate lane has no control row to move, yet a rebuild /// refuses it, so a live commit must not ack it. diff --git a/crates/host-core/src/prepare.rs b/crates/host-core/src/prepare.rs index f5e10b4b5..c4120b401 100644 --- a/crates/host-core/src/prepare.rs +++ b/crates/host-core/src/prepare.rs @@ -8,12 +8,13 @@ use core::num::{NonZeroU32, NonZeroUsize}; use std::collections::BTreeSet; use builtins::{ - BuiltinLaneSelector, MeterConfig, MeterHandle, MeterMetricSet, MeterTap, checked_fader_gain, + BuiltinLaneSelector, MeterConfig, MeterHandle, MeterMetricSet, MeterTap, + PreparedInputFilterTarget, checked_fader_gain, checked_trim_gain, validate_input_filter_pair, }; use builtins_compiler::{ BuiltinCompileCaps, LiveValuesSuperseded, MeterConsumer, MeterRequest, SelectedMeterRequest, TrackControlProducer, TrackControlRecord, TrackControlRequest, TrackFaderRecord, - lower_matrix_or_pan, prepare_selected_session_builtins_between_render_calls, + TrackInputRecord, lower_matrix_or_pan, prepare_selected_session_builtins_between_render_calls, prepare_selected_session_builtins_with_live_controls, prepare_session_builtins_between_render_calls, prepare_session_builtins_with_live_controls, session_structural_symmetry, @@ -54,6 +55,7 @@ use std::collections::BTreeMap; use crate::control_preparation::{ EQ_TARGET_CAPACITY, EQ_VALUE_COUNT, EqTargetEdit, EqTargetPreparer, EqTargetPreparerError, + INPUT_FILTER_TARGET_CAPACITY, InputFilterEdit, InputFilterPreparer, }; use crate::diagnostics::{PrepareDiagnostics, PrepareRejection, diagnostic_lines}; use crate::route_controls::{ @@ -2048,10 +2050,11 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( index, }); } - // Issue #1276 D1: the strip input sections that carry, with the predecessor lane each one - // copies from. Every strip of this plan attaches the same control kind. + // Issue #1276 D1, #1630 D1: the strip input sections that carry, with the predecessor lane + // each one copies from and the records that retarget it. Every strip of this plan attaches + // the same control kind. let live_input = attach_live && lanes.strip_input; - let carried_inputs: Vec<(&str, graph::GraphLaneLocation)> = match successor { + let carried_inputs: Vec> = match successor { None => Vec::new(), Some(base) => { // `committed_input_section` binary-searches: a model that is not normalized would @@ -2068,13 +2071,37 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( .all(|pair| pair[0].id < pair[1].id), "a successor's committed model is normalized: strips in canonical ID order" ); + // #1630 D1: the join keys on the section's structure. Every strip's input section has + // the same stages (trim, polarity, two filter sections per channel), so the structure + // is the control kind and the rate its designs are for; a live difference carries and + // retargets, and a plan without input cells carries only bit-equal values. + let ramps = LiveRamps::for_session(model); + let rate = compiled.sample_rate().0; + let same_rate = base.inventory.sample_rate_hz == rate; model .strips() .filter_map(|strip| { let row = base.inventory.input_section(strip.id.as_str())?; let before = committed_input_section(base.committed, &strip)?; - (row.live == live_input && same_input_section(before, strip.builtins)) - .then_some((strip.id.as_str(), row.location)) + if !same_rate || row.live != live_input { + return None; + } + let retarget = if live_input { + // A retarget is the record of the same edit (#1394 D6): the live path's + // records, so the carried lane ramps as a live edit would. A value the + // live path refuses restarts the section. + let edit = StripEditRamps::of(base.edit_ramps, strip.id); + input_records(before, strip.builtins, rate, ramps, edit).ok()? + } else if same_input_section(before, strip.builtins) { + Vec::new() + } else { + return None; + }; + Some(InputJoin { + id: strip.id.as_str(), + predecessor: row.location, + retarget, + }) }) .collect() } @@ -2788,10 +2815,10 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( if !carried_inputs.is_empty() { let moves: Vec = carried_inputs .iter() - .filter_map(|(id, predecessor)| { - input_section_row(&input_sections, id).map(|row| graph::GraphLaneMove { + .filter_map(|join| { + input_section_row(&input_sections, join.id).map(|row| graph::GraphLaneMove { successor: row.location, - predecessor: *predecessor, + predecessor: join.predecessor, }) }) .collect(); @@ -3223,7 +3250,8 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( successor, strip_carry.retargets.is_empty() && effect_carry.retargets.is_empty() - && route_carry.retargets.is_empty(), + && route_carry.retargets.is_empty() + && carried_inputs.iter().all(|join| join.retarget.is_empty()), ) { let Some(Ok(stamp)) = plan.stamp_held_gate(base.revision) else { return Err(successor_revision_refusal()); @@ -3267,6 +3295,24 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( producer.producer.write(stamp, record); } } + // #1630 D2: the carried input sections' live differences, through the successor's own + // input cells, in the same block as the strips'. A carried lane with a retarget exists + // only on a plan that attaches input cells, and every record is the live path's (#1556 + // D5): a trim in domain and a validated filter target. + for join in carried_inputs + .iter() + .filter(|join| !join.retarget.is_empty()) + { + let Some(input) = canonical_index + .get(join.id) + .and_then(|&index| strip_controls[index].input.as_mut()) + else { + return Err(graph_failure("host.control.order")); + }; + for &record in &join.retarget { + input.write(stamp, record); + } + } } Ok(( @@ -3579,6 +3625,14 @@ pub fn pair_carried_meters( } // REALTIME_POLICY_END +/// One strip input section that carries (#1276 D1, #1630 D1): the predecessor lane it copies +/// from and the records that retarget it to this model's values, empty when they are equal. +struct InputJoin<'a> { + id: &'a str, + predecessor: graph::GraphLaneLocation, + retarget: Vec, +} + /// Strip `id`'s row of `rows`, which are sorted by strip ID. fn input_section_row<'a>( rows: &'a [InputSectionInventoryRow], @@ -3615,7 +3669,8 @@ fn committed_input_section<'a>( } /// Whether two strips' input sections are bit-equal (issue #1276 D1): polarity, trim and both -/// filter cutoffs, per channel. +/// filter cutoffs, per channel. Only a plan without input cells, which cannot retarget a carried +/// section, needs it (#1630 D1). fn same_input_section(left: &session::DualMonoBuiltins, right: &session::DualMonoBuiltins) -> bool { [(&left.left, &right.left), (&left.right, &right.right)] .into_iter() @@ -4234,6 +4289,149 @@ pub(crate) fn lane_records( records.into_iter().flatten() } +/// The input records that bring a strip's input section from `before`'s values to `after`'s +/// (#1556 D2-D4), in write order: trims, then polarities, then filter targets, each left before +/// right. A record is emitted only when the value the render plane holds changes. +/// +/// - `TrimDb`: one per lane whose [`checked_trim_gain`] bits change, or one `Both` when both lanes +/// change to the same `trim_db` bits. The comparison is of the gain's bits, so `0.0` to `-0.0` +/// (the same gain) gives no record. +/// - `PolarityInvert`: one per lane whose value changes, or one `Both` when both change to the +/// same value. +/// - `PreparedFilter`: the targets [`InputFilterPreparer`] designs, off the render thread, from the +/// pre-commit normalized pairs and the changed values (see [`filter_edits`]). +/// +/// # Errors +/// +/// [`OutOfDomain`] when a trim is outside the domain, or a pair or a designed target is +/// refused. +pub(crate) fn input_records( + before: &session::DualMonoBuiltins, + after: &session::DualMonoBuiltins, + sample_rate_hz: u32, + ramps: LiveRamps, + edit: StripEditRamps, +) -> Result, OutOfDomain> { + let lanes_before = [&before.left, &before.right]; + let lanes_after = [&after.left, &after.right]; + let mut records = Vec::with_capacity(4 + INPUT_FILTER_TARGET_CAPACITY); + + let gain_bits = |db: f32| { + checked_trim_gain(db) + .map(f32::to_bits) + .map_err(|_| OutOfDomain) + }; + let mut trim_changed = [false; 2]; + for lane in 0..2 { + trim_changed[lane] = + gain_bits(lanes_before[lane].trim_db)? != gain_bits(lanes_after[lane].trim_db)?; + } + for (lanes, lane) in lane_records( + trim_changed, + lanes_after[0].trim_db.to_bits() == lanes_after[1].trim_db.to_bits(), + ) { + records.push(TrackInputRecord::TrimDb { + lanes, + db: lanes_after[lane].trim_db, + smoothing_samples: ramps.resolve(LiveRampRow::InputTrim, edit.input), + }); + } + + let polarity_changed = + [0, 1].map(|lane| lanes_before[lane].polarity_invert != lanes_after[lane].polarity_invert); + for (lanes, lane) in lane_records( + polarity_changed, + lanes_after[0].polarity_invert == lanes_after[1].polarity_invert, + ) { + records.push(TrackInputRecord::PolarityInvert { + lanes, + inverted: lanes_after[lane].polarity_invert, + smoothing_samples: ramps.resolve(LiveRampRow::PolarityInvert, edit.input), + }); + } + + // The filters: compare the values each lane's pair normalizes to, so `-0.0` to `0.0` is no + // change (#1556 D4). + let pair = |lane: &session::ChannelBuiltins| { + validate_input_filter_pair(sample_rate_hz, lane.hpf_hz, lane.lpf_hz) + .map_err(|_| OutOfDomain) + }; + let pairs_before = [pair(lanes_before[0])?, pair(lanes_before[1])?]; + let pairs_after = [pair(lanes_after[0])?, pair(lanes_after[1])?]; + let mut changed = [(false, false); 2]; + for lane in 0..2 { + changed[lane] = ( + pairs_before[lane].hpf_hz.to_bits() != pairs_after[lane].hpf_hz.to_bits(), + pairs_before[lane].lpf_hz.to_bits() != pairs_after[lane].lpf_hz.to_bits(), + ); + } + let edits = filter_edits(changed, pairs_after); + if !edits.is_empty() { + let seeds = [ + pairs_before[0].hpf_hz, + pairs_before[0].lpf_hz, + pairs_before[1].hpf_hz, + pairs_before[1].lpf_hz, + ]; + let mut out = [PreparedInputFilterTarget { + lanes: BuiltinLaneSelector::Both, + section: 0, + pair: [0.0; 2], + coefficients: [0.0; 6], + }; INPUT_FILTER_TARGET_CAPACITY]; + let (_, count) = InputFilterPreparer + .prepare(sample_rate_hz, &seeds, &edits, &mut out) + .map_err(|_| OutOfDomain)?; + records.extend( + out[..count] + .iter() + .map(|&target| TrackInputRecord::PreparedFilter { target }), + ); + } + Ok(records) +} + +/// The semantic edits that carry a strip's changed filter values (#1556 D4): per lane, `3` or `4` +/// for a one-sided change and the atomic pair `0` when both of the lane's values change (so a +/// pair that two one-sided edits would refuse, an HPF raised past the old LPF while the LPF +/// rises, is one valid edit). The same edit on both lanes is one `Both` edit, which is how the +/// preparer later merges equal targets. +fn filter_edits( + changed: [(bool, bool); 2], + after: [builtins::InputFilterPair; 2], +) -> Vec { + let lane_edit = |lane: usize| { + let pair = after[lane]; + match changed[lane] { + (false, false) => None, + (true, false) => Some((3, pair.hpf_hz, 0.0)), + (false, true) => Some((4, pair.lpf_hz, 0.0)), + (true, true) => Some((0, pair.hpf_hz, pair.lpf_hz)), + } + }; + let (left, right) = (lane_edit(0), lane_edit(1)); + let edit = |(parameter_id, value0, value1): (u32, f32, f32), channel| InputFilterEdit { + parameter_id, + channel, + value0, + value1, + }; + match (left, right) { + (Some(left), Some(right)) + if left.0 == right.0 + && left.1.to_bits() == right.1.to_bits() + && left.2.to_bits() == right.2.to_bits() => + { + vec![edit(left, ParameterChannel::Both)] + } + _ => left + .map(|left| edit(left, ParameterChannel::Left)) + .into_iter() + .chain(right.map(|right| edit(right, ParameterChannel::Right))) + .collect(), + } +} + /// The refusal of a successor whose revision its gate cannot stamp (#1277 D10). fn successor_revision_refusal() -> PrepareDiagnostics { PrepareDiagnostics::fixed(PrepareRejection::Session, "host.successor.revision") diff --git a/crates/host-core/tests/successor_swap.rs b/crates/host-core/tests/successor_swap.rs index 66aceab6c..1381d6b1b 100644 --- a/crates/host-core/tests/successor_swap.rs +++ b/crates/host-core/tests/successor_swap.rs @@ -894,6 +894,17 @@ fn filtered_session(mono: bool) -> SessionModel { model } +/// `model` without track `id` and the routes it sources: as a committed base, B's strip of that ID +/// is new to the join, so its input section restarts at the swap. A live value difference no +/// longer restarts one (#1630 D1). +fn without_track(mut model: SessionModel, id: &str) -> SessionModel { + model.tracks.retain(|track| track.id.as_str() != id); + model.routes.retain( + |route| !matches!(&route.source, RouteSource::Track { track_id, .. } if track_id.as_str() == id), + ); + model +} + /// `model` plus [`MUTED_TRACK`], a muted copy of `eq0` that sorts first. fn with_muted_track(mut model: SessionModel) -> SessionModel { add_track(&mut model, "eq0", MUTED_TRACK, SOURCE, true); @@ -1229,10 +1240,10 @@ fn first_direct_difference(left: &DirectRun, right: &DirectRun) -> Option /// B is prepared from, is pending in A's queue at the swap. The carry drains it into the lane /// before copying, so B's first block starts the trim ramp exactly where A would have. `strip` is /// the written strip: the first bank's `eq2`, or a strip in a later predecessor bank. -/// `restarting` names a strip whose trim the committed model changes without B following it, so -/// its input section restarts at the swap (#1276 D1); it sits in the first bank at the lane of the -/// written strip in its own bank, and it is muted in every session so that its restart leaves the -/// output bits where the unrestarted reference puts them. +/// `restarting` names a strip the committed model lacks ([`without_track`]), so B's strip of that +/// ID is new to the join and its input section restarts at the swap; it sits in the first bank at +/// the lane of the written strip in its own bank, and it is muted in every session so that its +/// restart leaves the output bits where the unrestarted reference puts them. fn a_pending_trim_record_survives_the_swap( backend: Backend, strip: &'static str, @@ -1273,15 +1284,9 @@ fn a_pending_trim_record_survives_the_swap( track.builtins.right.trim_db = NEW_TRIM_DB; let b = Session::compile(with_muted_track(committed.clone())); if let Some(restarting) = restarting { - // Committed moves the restarting strip's trim; B keeps the old one, so that input section - // restarts instead of carrying. - let track = committed - .tracks - .iter_mut() - .find(|track| track.id.as_str() == restarting) - .expect("the restarting strip"); - track.builtins.left.trim_db += 3.0; - track.builtins.right.trim_db += 3.0; + // A live value difference carries and retargets (#1630 D1); a strip the committed model + // lacks has nothing to carry, so that input section restarts. + committed = without_track(committed, restarting); } let reference_session = Session::compile(with_muted_track(session())); let feed = Feed::new(SOURCE, 5); @@ -1328,23 +1333,16 @@ fn a_pending_trim_record_survives_the_swap( /// #1312 D2 on the input cells: `live_values_superseded` counts the writes the carry drain /// replaces on input lanes that carry, and none on an input lane the swap restarts, whichever -/// lanes share their bank. The committed model differs from B's in the first strip's trim, so -/// that strip's input section restarts (#1276 D1) while the strip beside it, in the same -/// predecessor bank at four and at eight lanes, carries. Three trim writes sit unread in each -/// strip's cells at the hand-over. +/// lanes share their bank. The committed model lacks the first strip ([`without_track`]), so that +/// strip's input section restarts while the strip beside it, in the same predecessor bank at four +/// and at eight lanes, carries. Three trim writes sit unread in each strip's cells at the +/// hand-over. fn a_restarting_input_lane_does_not_count_its_replaced_writes(backend: Backend) { const RESTARTING: &str = "eq0"; const CARRYING: &str = "eq1"; let a = Session::compile(filtered_session(false)); let b = Session::compile(with_muted_track(filtered_session(false))); - let mut committed = filtered_session(false); - let strip = committed - .tracks - .iter_mut() - .find(|track| track.id.as_str() == RESTARTING) - .expect("the restarting strip"); - strip.builtins.left.trim_db = 3.0; - strip.builtins.right.trim_db = 3.0; + let committed = without_track(filtered_session(false), RESTARTING); let feed = Feed::new(SOURCE, 5); let run = |strips: &[&'static str]| { let writes: Vec = strips @@ -1581,7 +1579,9 @@ fn collapsed_strips_keep_collapsing_across_a_swap_at_four_lanes() { /// Gate 4 of #1276, one edit. B applies `edit` to `eq0`'s input section in the same transaction /// that adds a muted track, so `eq0`'s section does not carry (Q1's default): from the swap block -/// on it renders exactly what a fresh plan of B renders from rest on the same PCM. +/// on it renders exactly what a fresh plan of B renders from rest on the same PCM. These plans +/// attach no input cells, so they cannot retarget a carried section; on plans that do, the section +/// carries and retargets (#1630, [`an_input_difference_carries_and_retargets`]). fn a_changed_section_starts_at_rest(name: &str, edit: fn(&mut session::DualMonoBuiltins)) { let one_track = |model: SessionModel| { let mut model = model; @@ -1794,6 +1794,146 @@ fn the_input_carry_allocates_and_frees_nothing() { } } +/// #1630 gates 1-3, one edit. B applies `edit` to [`RETARGETED`]'s input section in the same +/// transaction that adds a muted track, on plans with live input cells. Run 1 prepares B from A's +/// committed model: the section carries and the successor's cells retarget it (D1, D2). Run 2 +/// writes the classifier's records for the same edit to A's cells just before the swap and +/// prepares B from a base that holds them: a plan that made the edit live. The runs are +/// bit-identical in every block, and the swap block allocates and frees nothing (D3). +#[cfg(feature = "control-provider")] +fn an_input_difference_carries_and_retargets(name: &str, edit: fn(&mut session::DualMonoBuiltins)) { + use bench_support::alloc::{Mode, assert_installed, mode, set_mode}; + struct RestoreMode(Mode); + impl Drop for RestoreMode { + fn drop(&mut self) { + set_mode(self.0); + } + } + const RETARGETED: &str = "eq2"; + assert_installed(); + let _restore = RestoreMode(mode()); + set_mode(Mode::Count); + engine::realtime::audit::warm_up(); + let a = Session::compile(filtered_session(false)); + let mut moved = filtered_session(false); + edit( + &mut moved + .tracks + .iter_mut() + .find(|track| track.id.as_str() == RETARGETED) + .expect("the retargeted strip") + .builtins, + ); + let moved = Session::compile(moved).model; + let b = Session::compile(with_muted_track(moved.clone())); + // The records the live path gives the same edit (#1556), with the session's ramps. + let delta = host_core::live_delta::classify_live_delta( + &a.model, + &moved, + LiveRamps::for_session(&moved), + &EditRamps::new(), + ) + .unwrap_or_else(|reason| panic!("{name}: the edit is live: {reason:?}")); + assert_eq!(delta.strips.len(), 1, "{name}"); + assert!(!delta.strips[0].input.is_empty(), "{name}: an input record"); + let writes: Vec = delta.strips[0] + .input + .iter() + .map(|&record| LiveWrite { + block: SWAP_BLOCK, + strip: RETARGETED, + record, + }) + .collect(); + let feed = Feed::new(SOURCE, 17); + for backend in backends() { + let run = |committed: &SessionModel, writes: &[LiveWrite]| { + let run = direct_run( + &a, + Some((&b, committed)), + &feed, + QUANTUM, + writes, + backend, + false, + ); + assert_eq!(run.carry, CarryOutcome::Carried, "{name}, {backend:?}"); + run + }; + let retargeted = run(&a.model, &[]); + let written = run(&moved, &writes); + assert_eq!( + retargeted.swap_allocator, + ((0, 0, 0), (0, 0)), + "{name}, {backend:?}: allocator calls and audited allocations at the swap block" + ); + assert_eq!( + first_direct_difference(&retargeted, &written), + None, + "{name}, {backend:?}: carry then retarget sounds like live edit then structural edit" + ); + // Not vacuous: a section that restarts at the new values is audible at the swap block, + // and so is one that carries without the retarget (B without the edit). + let restarted = run(&without_track(a.model.clone(), RETARGETED), &[]); + assert_eq!( + first_direct_difference(&retargeted, &restarted), + Some(SWAP_BLOCK), + "{name}, {backend:?}: a restart is a transient the carry avoids" + ); + let unmoved = direct_run( + &a, + Some(( + &Session::compile(with_muted_track(a.model.clone())), + &a.model, + )), + &feed, + QUANTUM, + &[], + backend, + false, + ); + assert_eq!( + first_direct_difference(&retargeted, &unmoved), + Some(SWAP_BLOCK), + "{name}, {backend:?}: the retarget moves the output from the swap block on" + ); + } +} + +/// #1630 gate 1: a trim change. Red if the join still restarts a section whose trim differs (the +/// restart transient), or if the carried section keeps the old trim (no retarget). +#[cfg(feature = "control-provider")] +#[test] +fn a_trim_change_carries_and_retargets_the_input_section() { + an_input_difference_carries_and_retargets("trim_db", |builtins| { + builtins.left.trim_db += 1.5; + builtins.right.trim_db += 0.75; + }); +} + +/// #1630 gate 2: input filter cutoffs, one-sided and on both sides. Red if a filter retarget is +/// dropped or designed from other seeds than the live path's. +#[cfg(feature = "control-provider")] +#[test] +fn a_filter_cutoff_change_carries_and_retargets_the_input_section() { + an_input_difference_carries_and_retargets("hpf_hz", |builtins| { + builtins.left.hpf_hz = 160.0; + builtins.right.hpf_hz = 170.0; + }); + an_input_difference_carries_and_retargets("lpf_hz", |builtins| { + builtins.right.lpf_hz = 3_500.0; + }); +} + +/// #1630 gate 2: a polarity change. +#[cfg(feature = "control-provider")] +#[test] +fn a_polarity_change_carries_and_retargets_the_input_section() { + an_input_difference_carries_and_retargets("polarity_invert", |builtins| { + builtins.right.polarity_invert = true; + }); +} + /// Gate 3 of #1276, live term. On the all-mono session a left-only trim record that leaves the /// trim where it was is pending in A's queue at the swap. It moves no word, but it is a one-channel /// write, so its strip's chain must render dual from the swap block on, as the reference's does From 3c49a8ba8b0759cf1bebf5c909206874a4f380b7 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 21:29:44 +0000 Subject: [PATCH 08/12] Batch 14b review fold, part B: #1620, #1624, #1630 - #1624: begin reserves fallibly (EffectControlOwnerError::Allocation, refused before the stamp); one owner state holds the candidate; the candidate test commits; heap ceilings re-pinned by a written slack rule. - #1630: the input, fader and matrix carry drains count a carried value the successor's retarget replaces (D15-2, root ruling); a successor input ramp test; a debug assert on the join's records. - #1620: comments say the in-place form adds entries; OwnersGrowth fields are private; the left-bytes claim is dropped. Co-Authored-By: Claude Opus 5.5 --- ...he-first-live-commit-of-a-large-session.md | 43 ++- ...ffect-control-owners-candidate-at-begin.md | 27 ++ ...-strip-input-section-across-a-plan-swap.md | 40 ++- crates/builtins-compiler/src/lib.rs | 116 +++++++- crates/capi/tests/resource_lifecycle.rs | 10 +- crates/control-plane/src/control.rs | 10 +- crates/effect-compiler/src/control.rs | 221 +++++++------- crates/effect-compiler/src/prepare.rs | 51 ++++ crates/host-core/src/prepare.rs | 9 +- crates/host-core/tests/in_place_commit.rs | 4 +- crates/host-core/tests/successor_swap.rs | 277 +++++++++++++++++- crates/protocol/src/controller.rs | 4 +- crates/protocol/src/model.rs | 7 +- crates/session/src/estimate.rs | 4 +- crates/session/src/in_place.rs | 33 ++- crates/session/tests/in_place.rs | 23 +- .../tests/control_command_heap.rs | 41 +-- tools/realtime-policy/src/policy.rs | 8 +- 18 files changed, 745 insertions(+), 183 deletions(-) diff --git a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md index f45ae3fc2..f0704c408 100644 --- a/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md +++ b/.github/ISSUE_SPECS/1620-bound-the-first-live-commit-of-a-large-session.md @@ -124,9 +124,12 @@ in canonical `(ID, channel)` order, the order `compile_session` sorts into. Outside the listed paths (mechanical): the in-place admission comment in `crates/control-plane/src/control.rs` (the in-place transaction may now add parameter entries, never -a count `check_counts` reads); `crates/session/tests/in_place.rs` moved to the new function; the -differential gate lives in `crates/host-core/tests/in_place_commit.rs`, the home of the #1613 `Pair` -harness that compares the two paths. +a count `check_counts` reads); `crates/session/src/estimate.rs` (`ResourceEstimate::after_owners_edit` +and the named `CANONICAL_BYTES_PER_ITEM`), `crates/session/src/compile.rs` (`check_caps` made +`pub(crate)`, its unused model argument dropped) and `crates/session/src/lib.rs` (the exports); +`crates/session/tests/in_place.rs` moved to the new function; the differential gate lives in +`crates/host-core/tests/in_place_commit.rs`, the home of the #1613 `Pair` harness that compares the +two paths. **Gates.** @@ -141,14 +144,34 @@ harness that compares the two paths. store is unchanged). Session unit: `owners_with_values_the_compiler_refuses_are_not_valid_and_counts_are_compared`. 2. `tools/parameter-metadata/tests/control_command_heap.rs` (debug): the first call is gated at every size, `FIRST_PEAK_CEILING` 74,000 and `FIRST_REQUESTED_CEILING` 105,000 (was 78,000 peak - at one track only); `first.left <= 256` (was 4,096). Re-pinned for #1620 from these - measurements, on this head before and after the change: + at one track only). Re-pinned for #1620 from these measurements, on this head before and after + the change: - | tracks | first call before: allocations / requested / peak / left | first call after | steady (unchanged) | + | tracks | first call before: allocations / requested / peak | first call after | steady (unchanged) | |---|---|---|---| - | 1 | 839 / 130,294 / 77,600 / 1,078 | 757 / 103,033 / 72,865 / 80 | 758 / 103,918 / 72,841 / 80 | - | 64 | 1,853 / 550,437 / 198,182 / 1,078 | 757 / 103,033 / 72,865 / 80 | the same | - | 256 | 4,927 / 1,867,749 / 663,179 / 1,078 | 757 / 103,033 / 72,865 / 80 | the same | + | 1 | 839 / 130,294 / 77,600 | 757 / 103,033 / 72,865 | 758 / 103,918 / 72,841 | + | 64 | 1,853 / 550,437 / 198,182 | 757 / 103,033 / 72,865 | the same | + | 256 | 4,927 / 1,867,749 / 663,179 | 757 / 103,033 / 72,865 | the same | - The steady ceilings (73,000 peak, 104,000 requested) are kept. + The steady ceilings (73,000 peak, 104,000 requested) were kept here; the batch 14b review fold + re-pinned all four (below). 3. The same test asserts the first call's allocation count equal at 1, 64 and 256 tracks. + +**Review fold (batch 14b).** (1) The comments that still said the in-place form takes only existing +values now say it also adds parameter entries: `protocol/src/model.rs` (`TransactionForm::InPlace`, +`prepare_transaction_in_place`), `protocol/src/controller.rs`, `control-plane/src/control.rs` +(`commit_live`) and `host-core/tests/in_place_commit.rs`. (2) `control_command_heap.rs` drops the +`first.left <= 256` check: it had no unique catch (the first in-place commit frees the boot's +canonical text, and the live-bytes window stops at zero), so the left-bytes claim is removed from +gate 2 above; `steady.left <= 256` stays. (3) `OwnersGrowth`'s fields are private with `const` +getters (`parameters()`, `largest_grown_list()`) and it no longer derives `Default`: only +`owners_growth` builds one, so no caller outside `session` can hand the public +`ResourceEstimate::after_owners_edit` a growth smaller than the owners' real one; the session tests +compare the getters' pair. (4) The heap ceilings follow one slack rule (measured plus 2,000 bytes, +rounded up to the next 1,000): first call 75,000 peak / 106,000 requested, steady 75,000 / 106,000 +(were 74,000 / 105,000 and 73,000 / 104,000); measured, debug, every size: first 757 / 103,033 / +72,865, steady 758 / 103,918 / 72,841 (the #1624 re-pin, recorded there). (5) Original gate 2 +(`scripts/check-web-boot-budget.mjs`) was not run in attempt 1; run in this fold on a fresh +`build-web-audioworklet.sh --module-only` simd128 module (`765e340fa583b48c`, 4,096,024 B, built from this fold's working tree): `node +scripts/check-web-boot-budget.mjs` passes at its current pin ("web boot budget high-water gate +passed; mismatches: 0"). diff --git a/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md b/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md index b2cb8367b..af9a23dd8 100644 --- a/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md +++ b/.github/ISSUE_SPECS/1624-build-the-effect-control-owners-candidate-at-begin.md @@ -96,3 +96,30 @@ copy after a discard). `revision_overflow_is_refused_before_candidate_mutation` candidate is built on a refused begin. Updated arithmetic: `prepare.rs` resource tests and `host-core/tests/prepare.rs` drop the candidate and mask terms. `realtime-policy` ok; clippy `-D warnings` and fmt clean on the touched crates. + +**Review fold (batch 14b).** (a) `begin` allocates fallibly: the rows and the touched mask are +`try_reserve_exact`'d, then filled within that capacity and boxed in place, and a failed +reservation refuses with the new `EffectControlOwnerError::Allocation`, leaving the owner idle and +unchanged. `commit_live` (`crates/control-plane/src/control.rs`, step 3) maps it, as every failed +`try_reserve_exact` of that commit, to `CommandError::Internal` before the stamp: nothing is written +or acknowledged, and `OpenOwners` discards the owners begun so far. (b) The candidate lives inside +the phase: a private `OwnerState` enum (`Idle`, `Open(candidate)`, `Poisoned(candidate)`, +`Faulted`, `Published { candidate, next_revision }`) replaces the `candidate`, `pending_revision` +and `phase` fields, so a phase and a candidate cannot disagree and the two "impossible state" +error paths are gone; `phase()` still returns the public `EffectControlOwnerPhase`. (c) +`the_candidate_lives_only_between_begin_and_commit_or_discard` moved to `prepare.rs`'s +`owner_tests` (it needs the `OwnerRig` to publish): it now publishes and commits, and asserts the +candidate is held while published and gone after the commit, with the committed rows equal to the +edited candidate. The always-true `resource_facts` comparison is dropped. Red evidence: a `commit` +that leaves the published state (candidate kept) fails it at the post-commit phase check. (d) +`tools/parameter-metadata/tests/control_command_heap.rs` (debug, `verify_stages(false)`), measured +at 1, 64 and 256 tracks: steady 758 allocations / 103,918 requested / 72,841 peak, first call 757 / +103,033 / 72,865. Re-pinned under the file's one slack rule, now written down (measured plus 2,000 +bytes, rounded up to the next 1,000), reason "#1624 (B' adds 780 B and 2 allocations per begun EQ +owner)": steady peak 73,000 -> 75,000 and requested 104,000 -> 106,000 (room 159 and 82 bytes +before, 2,159 and 2,082 after); first call 74,000 -> 75,000 and 105,000 -> 106,000. The doc's old +#1615 figures (71,252 / 101,025 / 753) are kept as history beside the current measurement. (e) The +655,360-byte saving above is boot and idle retention only. A transaction that begins every EQ of +the worst document still allocates about 780 bytes per EQ for its candidates, transiently, so the +wasm high-water mark of such a transaction gains about 600 KB (768 EQs x 780 = 599,040 bytes) +over the idle figure. diff --git a/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md b/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md index 9bd10802e..f85bcc66f 100644 --- a/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md +++ b/.github/ISSUE_SPECS/1630-carry-then-retarget-a-strip-input-section-across-a-plan-swap.md @@ -73,10 +73,11 @@ bit-equal sections (`same_input_section`, unchanged #1276 behaviour). outside the authorized paths, mechanically (move, imports, one doc link). - The join adds a sample-rate check (`base.inventory.sample_rate_hz == compiled rate`): a carried lane's coefficients and filter state are designed for the predecessor's rate, so a rate change - is a structural difference (D1, "their designs' shape"). Before, a rate change with bit-equal - values carried. + is a structural difference (D1, "their designs' shape"). The check is defensive: the control + plane and `adopt_predecessor_plan` already refuse a successor at another rate, so it cannot + change any output today. - A value the live path refuses (out of domain) restarts the section instead of carrying; a - compiled model does not reach it. + compiled model does not reach it, and a debug build asserts so (review fold). - No builtins or `crates/engine/src/realtime/` edit was needed: the input stage's carry entry (#1276 `export_lane`/`import_lane`) and the successor's cell drain already order carry before retarget on the swap block. No new realtime region. @@ -102,3 +103,36 @@ and red when the retarget write is dropped. --all-features`, `-p capi`, `-p host-web --features host-web/test-support`: pass. Clippy `-p host-core --all-targets` with and without `--all-features`: clean. `audit capi`: `pcm_digest` `bf7dcedd1d92621f`, 0 violations. `realtime_policy`: ok, 149 regions (unchanged). + +**Review fold (batch 14b).** (f) Root ruling: D15-2's `live_values_superseded` is exact, so a +carried value the successor's retarget replaces in the adopting block is counted, as an effect +lane's carried write is. The input and strip carry drains apply a predecessor's unread value to lane +state at the swap; until now, when the successor's own cell (the retarget) applied in the adopting +block, before any sample, that carried value was replaced unheard and not counted. Each render-side +cell reader (`InputReader`, `FaderReader`, `MatrixReader`, `crates/builtins-compiler/src/lib.rs`) +now keeps `CarryMarks`: the cells its last drain applied, and the cells a swap carried in. The +adopting lane takes the predecessor reader's applied cells (`adopt_input_lane`, the fader and matrix +banks' `adopt_lane`, which the fused bank shares); the successor's first drain counts one +superseded value per carried cell it applies (`LiveValuesSuperseded::add_carried`) and clears the +marks, so from its second block a carried value counts as heard. No `crates/engine/src/realtime` +or `graph` edit; the new code sits in the existing realtime-policy regions plus one new region (the +`CarryMarks` methods and `adopt_carry_marks`): `realtime_policy` ok at 150 regions, and its +floor (`tools/realtime-policy/src/policy.rs`) rises 149 -> 150 in this change. The marks add four bytes per +reader: `crates/capi/tests/resource_lifecycle.rs`'s eight-lane `builtin_bank_bytes` measured 15,321 +-> 15,705 (+384), over its 15,680 ceiling, re-pinned by that file's rule (plus 10 %, up to 64) to +17,280; the four-lane ceiling was not re-measured here. Tests (`successor_swap.rs`, both widths, both strip +forms): `a_carried_input_value_the_retarget_replaces_is_counted_on_every_width` and +`a_carried_strip_value_the_retarget_replaces_is_counted_on_every_width_and_form`: a one-lane write +pending at the swap and committed in the base, then moved again by B, counts exactly 1 on the +successor (0 on the predecessor), is bit-identical in every block to a run whose base never held +the write, and counts 0 when B leaves the value (no retarget). Red with the count removed: +`(0, 0)` against `(0, 1)` in both. The harness reads the successor's own count +(`successor_superseded`), since the test seam gives each plan its own counter. (g) +`an_input_difference_carries_and_retargets` also runs every case with a transaction that gave the +strip's `(Strip, Input)` row a 37-sample ramp, through the successor path (`direct_run_ramped`, +`SuccessorBase::edit_ramps`): it equals the run that writes the classifier's records for the +ramped edit, and for trim and polarity (whose records the row changes) differs at the swap block +from the session-ramp retarget. Red when the join ignores the edit ramps (`StripEditRamps::default()` +at the `input_records` call): trim and polarity fail, first difference at block 6. (h) The +`input_records(..)` call in the join is now `debug_assert!`ed `Ok` before `.ok()?`: an internal +refusal would silently restart the section. diff --git a/crates/builtins-compiler/src/lib.rs b/crates/builtins-compiler/src/lib.rs index 5659b61d7..1784af3a6 100644 --- a/crates/builtins-compiler/src/lib.rs +++ b/crates/builtins-compiler/src/lib.rs @@ -398,8 +398,63 @@ struct FaderReader { cells: [CellReader; FADER_CELLS], dirty: DirtyReader, superseded: LiveValuesSuperseded, + /// The carry marks (#1630 f, D15-2); see [`CarryMarks`]. + marks: CarryMarks, } +/// What one reader knows about values a plan swap carried (#1630 f, decision 15 D15-2). Bit `n` +/// is the cell with dirty bit `n`. +/// +/// `applied` is the cells the reader's last drain applied. The swap's carry drain on the +/// predecessor sets it, and the successor's adopting lane takes it as `carried`: the cells whose +/// value the swap carried into lane state and no sample has rendered yet. The successor's first +/// drain, in the adopting block before any sample, counts each carried cell it applies a value of +/// its own to as one superseded value (the carried value was replaced unheard, as an effect lane's +/// carried write is), and clears `carried`, whether or not it applied anything: from the second +/// block on, the carried value has been heard. +#[derive(Clone, Copy, Default)] +struct CarryMarks { + carried: u16, + applied: u16, +} + +// REALTIME_POLICY_BEGIN +impl CarryMarks { + /// Start a drain: take the carried cells and forget the last drain's applied ones. + fn begin(&mut self) -> u16 { + self.applied = 0; + core::mem::take(&mut self.carried) + } + + /// The drain applied cell `bit`; a carried value of that cell was replaced unheard. + fn applied(&mut self, bit: usize, carried: u16, superseded: &LiveValuesSuperseded) { + let mask = 1_u16 << bit; + self.applied |= mask; + if carried & mask != 0 { + superseded.add_carried(1); + } + } + + /// The successor's adopting lane takes the cells the predecessor's carry drain applied. + fn adopt(&mut self, predecessor: &Self) { + self.carried = predecessor.applied; + } +} + +/// An adopted lane's reader takes its predecessor reader's carry marks; a lane without a reader on +/// either side has none to take. +fn adopt_carry_marks( + into: Option<&mut Option>, + from: Option<&Option>, + marks_mut: impl FnOnce(&mut R) -> &mut CarryMarks, + marks: impl FnOnce(&R) -> &CarryMarks, +) { + if let (Some(Some(into)), Some(Some(from))) = (into, from) { + marks_mut(into).adopt(marks(from)); + } +} +// REALTIME_POLICY_END + /// Which cell kind a drain applies. #[derive(Clone, Copy)] enum FaderKind { @@ -426,6 +481,7 @@ impl FaderReader { [u32; FADER_CELL_WORDS], ) -> Result<(), BuiltinParameterError>, ) -> Result<(), BuiltinParameterError> { + let carried = self.marks.begin(); let pending = self.dirty.pending(&self.lane.dirty); if pending == 0 { return Ok(()); @@ -442,6 +498,7 @@ impl FaderReader { index as u32, &self.superseded, ) { + self.marks.applied(index, carried, &self.superseded); *slot = Some(words); } } @@ -476,6 +533,8 @@ struct MatrixReader { cell: CellReader, dirty: DirtyReader, superseded: LiveValuesSuperseded, + /// The carry marks (#1630 f, D15-2); see [`CarryMarks`]. + marks: CarryMarks, } // REALTIME_POLICY_BEGIN @@ -487,6 +546,7 @@ impl MatrixReader { snapshot: LiveSnapshot, mut apply: impl FnMut(Matrix2x2, u32) -> Result<(), BuiltinParameterError>, ) -> Result<(), BuiltinParameterError> { + let carried = self.marks.begin(); let pending = self.dirty.pending(&self.lane.dirty); if pending == 0 { return Ok(()); @@ -500,6 +560,7 @@ impl MatrixReader { ) else { return Ok(()); }; + self.marks.applied(0, carried, &self.superseded); apply( Matrix2x2 { ll: f32::from_bits(words[0]), @@ -526,6 +587,7 @@ fn fader_lane(superseded: &LiveValuesSuperseded) -> (FaderCells, FaderReader) { cells: core::array::from_fn(|_| CellReader::new()), dirty: DirtyReader::new(), superseded: superseded.clone(), + marks: CarryMarks::default(), }, ) } @@ -543,6 +605,7 @@ fn matrix_lane(superseded: &LiveValuesSuperseded) -> (MatrixCells, MatrixReader) cell: CellReader::new(), dirty: DirtyReader::new(), superseded: superseded.clone(), + marks: CarryMarks::default(), }, ) } @@ -714,6 +777,8 @@ struct InputReader { filter: [CellReader; 2 * FILTER_SECTIONS], dirty: DirtyReader, superseded: LiveValuesSuperseded, + /// The carry marks (#1630 f, D15-2); see [`CarryMarks`]. + marks: CarryMarks, } /// The value(s) a pair of channel cells applied in one drain: one `Both` value when the two words @@ -736,7 +801,9 @@ fn paired_channels( } /// Read one pending cell under `snapshot`: its newest value in range, if it was not applied -/// before. The replaced count joins `superseded`. +/// before. The replaced count joins `superseded`, and an applied cell marks `marks` (a carried +/// value it replaces joins the count too, #1630 f). +#[allow(clippy::too_many_arguments)] fn read_input_cell( pending: u64, bit: usize, @@ -745,12 +812,16 @@ fn read_input_cell( snapshot: LiveSnapshot, dirty: &mut DirtyReader, superseded: &LiveValuesSuperseded, + (marks, carried): (&mut CarryMarks, u16), ) -> Option<[u32; N]> { if pending & (1 << bit) == 0 { return None; } match reader.read_counted(cell, snapshot, dirty, bit as u32, superseded) { - CellRead::Applied { words, .. } => Some(words), + CellRead::Applied { words, .. } => { + marks.applied(bit, carried, superseded); + Some(words) + } CellRead::Unchanged => None, } } @@ -775,6 +846,7 @@ impl InputReader { witness: &mut ChannelSymmetryWitness, mut apply: impl FnMut(TrackInputRecord) -> Result<(), BuiltinParameterError>, ) -> Result<(), BuiltinParameterError> { + let carried = self.marks.begin(); let pending = self.dirty.pending(&self.lane.dirty); if pending == 0 { return Ok(()); @@ -786,6 +858,7 @@ impl InputReader { filter, dirty, superseded, + marks, } = self; let trim_words: [Option<[u32; TRIM_CELL_WORDS]>; 2] = core::array::from_fn(|channel| { read_input_cell( @@ -796,6 +869,7 @@ impl InputReader { snapshot, dirty, superseded, + (&mut *marks, carried), ) }); let polarity_words: [Option<[u32; POLARITY_CELL_WORDS]>; 2] = @@ -808,6 +882,7 @@ impl InputReader { snapshot, dirty, superseded, + (&mut *marks, carried), ) }); let filter_words: [Option<[u32; FILTER_CELL_WORDS]>; 2 * FILTER_SECTIONS] = @@ -820,6 +895,7 @@ impl InputReader { snapshot, dirty, superseded, + (&mut *marks, carried), ) }); let mut first_error = Ok(()); @@ -896,6 +972,7 @@ fn input_lane_cells(superseded: &LiveValuesSuperseded) -> (InputCells, InputRead filter: core::array::from_fn(|_| CellReader::new()), dirty: DirtyReader::new(), superseded: superseded.clone(), + marks: CarryMarks::default(), }, ) } @@ -1299,6 +1376,12 @@ impl GraphPreparedBuiltinBankProcessor for BuiltinBankProcessor { return false; } *slot = live; + adopt_carry_marks( + self.controls.get_mut(lane), + predecessor.controls.get(predecessor_lane), + |reader| &mut reader.marks, + |reader| &reader.marks, + ); true } @@ -1469,12 +1552,22 @@ impl FaderBankProcessor { } /// Copy `predecessor`'s lane `predecessor_lane` into lane `lane` (#1277 D7, D8). The cell - /// reader stays: the successor's own cells take every later value. + /// reader stays: the successor's own cells take every later value, and it takes the carry + /// marks of the values the carry drain applied (#1630 f). fn adopt_lane(&mut self, lane: usize, predecessor: &Self, predecessor_lane: usize) -> bool { - predecessor + let adopted = predecessor .bank .export_lane(predecessor_lane) - .is_some_and(|state| self.bank.import_lane(lane, &state).is_ok()) + .is_some_and(|state| self.bank.import_lane(lane, &state).is_ok()); + if adopted { + adopt_carry_marks( + self.controls.get_mut(lane), + predecessor.controls.get(predecessor_lane), + |reader| &mut reader.marks, + |reader| &reader.marks, + ); + } + adopted } // REALTIME_POLICY_END } @@ -1621,10 +1714,19 @@ impl MatrixBankProcessor { /// The matrix bank's counterpart of [`FaderBankProcessor::adopt_lane`]. fn adopt_lane(&mut self, lane: usize, predecessor: &Self, predecessor_lane: usize) -> bool { - predecessor + let adopted = predecessor .bank .export_lane(predecessor_lane) - .is_some_and(|state| self.bank.import_lane(lane, &state).is_ok()) + .is_some_and(|state| self.bank.import_lane(lane, &state).is_ok()); + if adopted { + adopt_carry_marks( + self.controls.get_mut(lane), + predecessor.controls.get(predecessor_lane), + |reader| &mut reader.marks, + |reader| &reader.marks, + ); + } + adopted } // REALTIME_POLICY_END } diff --git a/crates/capi/tests/resource_lifecycle.rs b/crates/capi/tests/resource_lifecycle.rs index dcc2e049c..41e5e4136 100644 --- a/crates/capi/tests/resource_lifecycle.rs +++ b/crates/capi/tests/resource_lifecycle.rs @@ -1226,6 +1226,14 @@ impl Budget { /// bytes less per EQ: 60 rows of 12 bytes and 60 flags). `capi_retained_bytes` measured 166,374 /// before and 159,354 after (x86-64, -7,020 = nine EQs; the row does not depend on the lane width), /// so its ceiling is 159,354 plus 10 %, rounded up to 64: 175,289.4 -> 175,296 (was 182,400). +/// +/// #1630 (batch 14b review fold) raises the eight-lane builtin bank row, a structural move: each +/// strip's input, fader and matrix cell readers keep carry marks (two `u16`s each), so a carried +/// value the successor's retarget replaces is counted (D15-2). `builtin_bank_bytes` measured 15,321 +/// before and 15,705 after (x86-64, eight lanes, +384), so its ceiling is 15,705 plus 10 %, rounded +/// up to 64: 17,275.5 -> 17,280 (was 15,680). The four-lane ceiling (21,056) was not re-measured +/// (no AArch64 run in this fold); `builtin_processor_payload_bytes` moved 36,171 -> 36,387 inside +/// its ceiling. const REFERENCE_BUDGETS: [Budget; 19] = [ Budget { row: "graph_session_plus_plan_bytes", @@ -1272,7 +1280,7 @@ const REFERENCE_BUDGETS: [Budget; 19] = [ Budget { row: "builtin_bank_bytes", value: |report| report.builtin_bank_bytes, - eight_lanes: 15_680, + eight_lanes: 17_280, four_lanes: 21_056, }, Budget { diff --git a/crates/control-plane/src/control.rs b/crates/control-plane/src/control.rs index 9e74a50fe..e90e06eb7 100644 --- a/crates/control-plane/src/control.rs +++ b/crates/control-plane/src/control.rs @@ -1943,9 +1943,9 @@ where /// epoch advances if a descriptor or an address changed: a VCA added or removed live adds or /// removes its rows (#1558 D5), a member change, an offset or a mute changes none. /// - /// **A token prepared in place (#1613).** A transaction of `SetControl`s on existing values holds - /// the owners it edited and no compilation (`PreparedSessionTransaction::is_in_place`). The - /// classification, the admission, the catalog stage and the producers' preflight read the + /// **A token prepared in place (#1613).** A transaction of `SetControl`s, on existing values or + /// adding parameter entries (#1620), holds the owners it edited and no compilation + /// (`PreparedSessionTransaction::is_in_place`). The classification, the admission, the catalog stage and the producers' preflight read the /// committed model and those owners as one view (`controls::ModelView`), and nothing writes the /// committed model before the commit: the protocol commit swaps the owners in as the last write. /// If the fast classification cannot judge the transaction, the token is materialized (the full @@ -2167,6 +2167,10 @@ where else { return LiveCommit::Done(Err(CommandError::Internal)); }; + // `begin` allocates the candidate fallibly (#1624): its `Allocation` refusal maps to + // `INTERNAL`, as every failed `try_reserve_exact` of this commit does, before the + // stamp. Nothing is written or acknowledged, and `OpenOwners` discards the owners + // begun so far. if producer.begin_owner(base_revision).is_err() { return LiveCommit::Done(Err(CommandError::Internal)); } diff --git a/crates/effect-compiler/src/control.rs b/crates/effect-compiler/src/control.rs index 064eef7b6..1563f6f2d 100644 --- a/crates/effect-compiler/src/control.rs +++ b/crates/effect-compiler/src/control.rs @@ -54,6 +54,8 @@ pub enum EffectControlOwnerError { Target(EffectTargetError), /// A target the preflight accepted had no cell to take it. Invariant, + /// The candidate's rows or touched mask could not be allocated; nothing changed. + Allocation, } impl From for EffectControlOwnerError { @@ -67,12 +69,46 @@ pub struct EffectControlOwner { factory: Arc, sample_rate: u32, committed: Box<[InitialParameterValue]>, - /// The open transaction's rows and touched mask. Built by `begin`, dropped by `commit`, - /// `discard` and a publication fault, so an idle owner retains only `committed` (#1624). - candidate: Option, committed_revision: u64, - pending_revision: Option, - phase: EffectControlOwnerPhase, + /// The transaction phase, which carries the open transaction's candidate. A candidate exists + /// exactly in the phases that may use it, so the phase and the candidate cannot disagree; it + /// is built by `begin` and dropped by `commit`, `discard` and a publication fault, so an idle + /// owner retains only `committed` (#1624). + state: OwnerState, +} + +/// The owner's phase, with the candidate in the phases that have one. +enum OwnerState { + Idle, + Open(OpenCandidate), + Poisoned(OpenCandidate), + Faulted, + /// Every target was published; `next_revision` is the revision the one commit takes. + Published { + candidate: OpenCandidate, + next_revision: u64, + }, +} + +impl OwnerState { + const fn phase(&self) -> EffectControlOwnerPhase { + match self { + Self::Idle => EffectControlOwnerPhase::Idle, + Self::Open(_) => EffectControlOwnerPhase::Open, + Self::Poisoned(_) => EffectControlOwnerPhase::Poisoned, + Self::Faulted => EffectControlOwnerPhase::Faulted, + Self::Published { .. } => EffectControlOwnerPhase::Published, + } + } + + const fn candidate(&self) -> Option<&OpenCandidate> { + match self { + Self::Open(candidate) + | Self::Poisoned(candidate) + | Self::Published { candidate, .. } => Some(candidate), + Self::Idle | Self::Faulted => None, + } + } } /// The rows an open transaction edits and the mask of those it touched; both are exactly as long @@ -96,7 +132,7 @@ impl core::fmt::Debug for EffectControlOwner { .field("sample_rate", &self.sample_rate) .field("value_count", &self.committed.len()) .field("committed_revision", &self.committed_revision) - .field("phase", &self.phase) + .field("phase", &self.state.phase()) .finish_non_exhaustive() } } @@ -130,11 +166,9 @@ impl EffectControlOwner { Ok(Self { factory, sample_rate: preparation.sample_rate, - candidate: None, committed, committed_revision: 0, - pending_revision: None, - phase: EffectControlOwnerPhase::Idle, + state: OwnerState::Idle, }) } @@ -156,7 +190,7 @@ impl EffectControlOwner { #[must_use] pub const fn phase(&self) -> EffectControlOwnerPhase { - self.phase + self.state.phase() } /// The last committed canonical rows. @@ -169,13 +203,13 @@ impl EffectControlOwner { /// cannot be retained by preparation. #[must_use] pub fn candidate(&self) -> &[InitialParameterValue] { - self.candidate.as_ref().map_or(&[], |open| &open.rows) + self.state.candidate().map_or(&[], |open| &open.rows) } /// Touched mask for the current candidate, empty while no transaction is open. #[must_use] pub fn dirty(&self) -> &[bool] { - self.candidate.as_ref().map_or(&[], |open| &open.dirty) + self.state.candidate().map_or(&[], |open| &open.dirty) } /// Computes the actual dynamic allocations retained by this owner. The factory allocation is @@ -210,18 +244,29 @@ impl EffectControlOwner { } /// Starts one candidate at the caller's checked base revision. + /// + /// The candidate's rows and touched mask are allocated fallibly: a failed allocation refuses + /// with [`EffectControlOwnerError::Allocation`] and leaves the owner idle and unchanged. pub fn begin(&mut self, base_revision: u64) -> Result<(), EffectControlOwnerError> { - if self.phase != EffectControlOwnerPhase::Idle + if !matches!(self.state, OwnerState::Idle) || base_revision != self.committed_revision || self.committed_revision.checked_add(1).is_none() { return Err(EffectControlOwnerError::Revision); } - self.candidate = Some(OpenCandidate { - rows: self.committed.clone(), - dirty: vec![false; self.committed.len()].into_boxed_slice(), + let length = self.committed.len(); + let mut rows = Vec::new(); + let mut dirty = Vec::new(); + if rows.try_reserve_exact(length).is_err() || dirty.try_reserve_exact(length).is_err() { + return Err(EffectControlOwnerError::Allocation); + } + // Within the reserved capacity: neither extension reallocates, and each boxes in place. + rows.extend_from_slice(&self.committed); + dirty.resize(length, false); + self.state = OwnerState::Open(OpenCandidate { + rows: rows.into_boxed_slice(), + dirty: dirty.into_boxed_slice(), }); - self.phase = EffectControlOwnerPhase::Open; Ok(()) } @@ -235,28 +280,35 @@ impl EffectControlOwner { channel: ParameterChannel, value: f32, ) -> Result<(), EffectControlOwnerError> { - if self.phase != EffectControlOwnerPhase::Open { + let OwnerState::Open(candidate) = &mut self.state else { return Err(EffectControlOwnerError::Revision); - } - let result = self.edit_inner(parameter_index, channel, value); + }; + let result = Self::edit_inner( + self.factory.as_ref(), + candidate, + parameter_index, + channel, + value, + ); if result.is_err() { - self.phase = EffectControlOwnerPhase::Poisoned; + // The candidate moves into the poisoned phase unchanged; only discard may follow. + if let OwnerState::Open(candidate) = + core::mem::replace(&mut self.state, OwnerState::Faulted) + { + self.state = OwnerState::Poisoned(candidate); + } } result } fn edit_inner( - &mut self, + factory: &dyn NativeEffectFactory, + candidate: &mut OpenCandidate, parameter_index: u32, channel: ParameterChannel, value: f32, ) -> Result<(), EffectControlOwnerError> { - let candidate = self - .candidate - .as_mut() - .ok_or(EffectControlOwnerError::Revision)?; - let parameter = self - .factory + let parameter = factory .descriptor() .parameters .get(parameter_index as usize) @@ -303,11 +355,9 @@ impl EffectControlOwner { /// Borrows the complete candidate request for off-audio preparation. pub fn preparation_request(&self) -> Result, EffectControlOwnerError> { - self.require_open()?; - let open = self - .candidate - .as_ref() - .ok_or(EffectControlOwnerError::Invariant)?; + let OwnerState::Open(open) = &self.state else { + return Err(EffectControlOwnerError::Revision); + }; Ok(EffectTargetRequest { sample_rate: self.sample_rate, values: &open.rows, @@ -342,7 +392,7 @@ impl EffectControlOwner { } fn require_open(&self) -> Result<(), EffectControlOwnerError> { - if self.phase == EffectControlOwnerPhase::Open { + if matches!(self.state, OwnerState::Open(_)) { Ok(()) } else { Err(EffectControlOwnerError::Revision) @@ -351,15 +401,10 @@ impl EffectControlOwner { /// Discards an open or poisoned candidate. pub fn discard(&mut self) -> Result<(), EffectControlOwnerError> { - if !matches!( - self.phase, - EffectControlOwnerPhase::Open | EffectControlOwnerPhase::Poisoned - ) { + if !matches!(self.state, OwnerState::Open(_) | OwnerState::Poisoned(_)) { return Err(EffectControlOwnerError::Revision); } - self.candidate = None; - self.pending_revision = None; - self.phase = EffectControlOwnerPhase::Idle; + self.state = OwnerState::Idle; Ok(()) } @@ -407,33 +452,35 @@ impl EffectControlOwner { .write(stamp, EffectControlRecord::PreparedTarget(*target)) .is_err() { - self.phase = EffectControlOwnerPhase::Faulted; - self.candidate = None; - self.pending_revision = None; + self.state = OwnerState::Faulted; return Err(EffectControlOwnerError::Invariant); } } - self.pending_revision = Some(next_revision); - self.phase = EffectControlOwnerPhase::Published; - Ok(()) + // The preflight held the open phase, so the candidate moves into the published phase. + match core::mem::replace(&mut self.state, OwnerState::Faulted) { + OwnerState::Open(candidate) => { + self.state = OwnerState::Published { + candidate, + next_revision, + }; + Ok(()) + } + _ => Err(EffectControlOwnerError::Invariant), + } } /// Commits the candidate after a complete publication. pub fn commit(&mut self) -> Result { - if self.phase != EffectControlOwnerPhase::Published { + let OwnerState::Published { + candidate, + next_revision, + } = &self.state + else { return Err(EffectControlOwnerError::Revision); - } - let next_revision = self - .pending_revision - .take() - .ok_or(EffectControlOwnerError::Invariant)?; - let open = self - .candidate - .take() - .ok_or(EffectControlOwnerError::Invariant)?; - self.committed.copy_from_slice(&open.rows); - self.committed_revision = next_revision; - self.phase = EffectControlOwnerPhase::Idle; + }; + self.committed.copy_from_slice(&candidate.rows); + self.committed_revision = *next_revision; + self.state = OwnerState::Idle; Ok(self.committed_revision) } } @@ -465,64 +512,20 @@ mod tests { let mut owner = EffectControlOwner { factory: Arc::new(ParametricEqFactory), sample_rate: 48_000, - candidate: None, committed, committed_revision: u64::MAX, - pending_revision: None, - phase: EffectControlOwnerPhase::Idle, + state: OwnerState::Idle, }; let committed_before = owner.committed.clone(); assert_eq!( owner.begin(u64::MAX), Err(EffectControlOwnerError::Revision) ); - assert_eq!(owner.phase, EffectControlOwnerPhase::Idle); + assert_eq!(owner.phase(), EffectControlOwnerPhase::Idle); assert_eq!(owner.committed, committed_before); - assert!(owner.candidate.is_none(), "a refused begin builds nothing"); - } - - /// An idle owner retains only its committed rows: the candidate and the touched mask exist - /// from `begin` to `commit` or `discard` and nowhere else (#1624). - #[test] - fn the_candidate_lives_only_between_begin_and_commit_or_discard() { - let committed: Box<[InitialParameterValue]> = - default_initial_values(&PARAMETRIC_EQ_DESCRIPTOR).collect(); - let rows = committed.len(); - let mut owner = EffectControlOwner { - factory: Arc::new(ParametricEqFactory), - sample_rate: 48_000, - candidate: None, - committed, - committed_revision: 0, - pending_revision: None, - phase: EffectControlOwnerPhase::Idle, - }; - let idle_bytes = owner.resource_facts().expect("facts").owned_payload_bytes; - assert!(owner.candidate().is_empty() && owner.dirty().is_empty()); - - owner.begin(0).expect("begin"); - assert_eq!(owner.candidate(), &owner.committed[..]); - assert_eq!(owner.dirty().len(), rows); - assert!(owner.dirty().iter().all(|touched| !touched)); - owner.edit(2, ParameterChannel::Both, 100.0).expect("edit"); - assert!(owner.dirty().iter().any(|touched| *touched)); - owner.discard().expect("discard"); - assert!(owner.candidate().is_empty() && owner.dirty().is_empty()); - - // A second candidate starts from the committed rows, not from the discarded edit. - owner.begin(0).expect("begin again"); - assert_eq!(owner.candidate(), &owner.committed[..]); - assert!(owner.dirty().iter().all(|touched| !touched)); - // A poisoned candidate is still the open one until discarded. - assert!(owner.edit(9_999, ParameterChannel::Both, 0.0).is_err()); - assert_eq!(owner.phase(), EffectControlOwnerPhase::Poisoned); - assert_eq!(owner.dirty().len(), rows); - owner.discard().expect("discard poisoned"); - assert!(owner.candidate().is_empty()); - assert_eq!( - owner.resource_facts().expect("facts").owned_payload_bytes, - idle_bytes, - "the retained facts do not depend on a candidate" + assert!( + owner.candidate().is_empty(), + "a refused begin builds nothing" ); } } diff --git a/crates/effect-compiler/src/prepare.rs b/crates/effect-compiler/src/prepare.rs index 35c67b463..1a9c9e75e 100644 --- a/crates/effect-compiler/src/prepare.rs +++ b/crates/effect-compiler/src/prepare.rs @@ -1279,6 +1279,57 @@ mod owner_tests { }; 12] } + /// An idle owner holds no candidate: the candidate and its touched mask exist from `begin` + /// to `commit` or `discard` and nowhere else, and each new candidate starts from the + /// committed rows (#1624). + #[test] + fn the_candidate_lives_only_between_begin_and_commit_or_discard() { + let mut rig = OwnerRig::new(); + let rows = rig.owner.committed().len(); + assert!(rig.owner.candidate().is_empty() && rig.owner.dirty().is_empty()); + + rig.owner.begin(0).expect("begin"); + assert_eq!(rig.owner.candidate(), rig.owner.committed()); + assert_eq!(rig.owner.dirty().len(), rows); + assert!(rig.owner.dirty().iter().all(|touched| !touched)); + rig.owner + .edit(2, ParameterChannel::Both, 100.0) + .expect("edit"); + assert!(rig.owner.dirty().iter().any(|touched| *touched)); + rig.owner.discard().expect("discard"); + assert!(rig.owner.candidate().is_empty() && rig.owner.dirty().is_empty()); + + // A second candidate starts from the committed rows, not from the discarded edit. + rig.owner.begin(0).expect("begin again"); + assert_eq!(rig.owner.candidate(), rig.owner.committed()); + assert!(rig.owner.dirty().iter().all(|touched| !touched)); + // A poisoned candidate is still the open one until discarded. + assert!(rig.owner.edit(9_999, ParameterChannel::Both, 0.0).is_err()); + assert_eq!(rig.owner.phase(), EffectControlOwnerPhase::Poisoned); + assert_eq!(rig.owner.dirty().len(), rows); + rig.owner.discard().expect("discard poisoned"); + assert!(rig.owner.candidate().is_empty()); + + // A published candidate lives until its one commit, which drops it. + rig.owner.begin(0).expect("begin a committed transaction"); + rig.owner + .edit(2, ParameterChannel::Both, 100.0) + .expect("edit"); + let edited = rig.owner.candidate().to_vec(); + let mut targets = blank_targets(); + let count = rig + .owner + .prepare_targets_into(&mut targets) + .expect("prepare"); + rig.publish(0, &targets[..count]).expect("publish"); + assert_eq!(rig.owner.phase(), EffectControlOwnerPhase::Published); + assert_eq!(rig.owner.candidate(), &edited[..]); + assert_eq!(rig.owner.commit(), Ok(1)); + assert_eq!(rig.owner.phase(), EffectControlOwnerPhase::Idle); + assert!(rig.owner.candidate().is_empty() && rig.owner.dirty().is_empty()); + assert_eq!(rig.owner.committed(), &edited[..]); + } + #[test] fn effect_control_resources_charge_actual_tables_and_shared_owner_once() { let factory: Arc = Arc::new(ParametricEqFactory); diff --git a/crates/host-core/src/prepare.rs b/crates/host-core/src/prepare.rs index c4120b401..6fa86c607 100644 --- a/crates/host-core/src/prepare.rs +++ b/crates/host-core/src/prepare.rs @@ -2091,7 +2091,14 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( // records, so the carried lane ramps as a live edit would. A value the // live path refuses restarts the section. let edit = StripEditRamps::of(base.edit_ramps, strip.id); - input_records(before, strip.builtins, rate, ramps, edit).ok()? + let records = input_records(before, strip.builtins, rate, ramps, edit); + // A compiled model's values are in the live domain, so a refusal here is + // an internal fault that would silently restart the section. + debug_assert!( + records.is_ok(), + "a compiled strip's input values have live records: {records:?}" + ); + records.ok()? } else if same_input_section(before, strip.builtins) { Vec::new() } else { diff --git a/crates/host-core/tests/in_place_commit.rs b/crates/host-core/tests/in_place_commit.rs index b8f72442e..565efd118 100644 --- a/crates/host-core/tests/in_place_commit.rs +++ b/crates/host-core/tests/in_place_commit.rs @@ -1,7 +1,7 @@ //! #1613: a live-only transaction committed without cloning and compiling the model. //! -//! The protocol prepares a transaction of `SetControl`s on existing values **in place** -//! (`SessionStore::prepare_transaction_in_place`): the owners the edits name, edited, and nothing +//! The protocol prepares a transaction of `SetControl`s, on existing values or adding parameter +//! entries (#1620), **in place** (`SessionStore::prepare_transaction_in_place`): the owners the edits name, edited, and nothing //! of the model. The classifier reads the committed model and those owners as one view //! (`classify_live_staged`), the catalog stage reads them through the same view, and the commit //! swaps the owners in. This file holds the differential gate: the in-place chain must equal the diff --git a/crates/host-core/tests/successor_swap.rs b/crates/host-core/tests/successor_swap.rs index 1381d6b1b..7623ea88e 100644 --- a/crates/host-core/tests/successor_swap.rs +++ b/crates/host-core/tests/successor_swap.rs @@ -1033,17 +1033,20 @@ impl LivePlan { predecessor, committed, &host_core::NO_OBSERVATION, + &EditRamps::new(), backend, ) } - /// A successor prepared with the taps the host armed (#1584 D6a). + /// A successor prepared with the taps the host armed (#1584 D6a) and the ramp lengths the + /// transaction's edits gave their rows (#1394 D6). fn successor_observed( session: &Session, request: &host_core::HostLiveControlRequest, predecessor: &host_core::PlanStateInventory, committed: &SessionModel, observation: &controls::ObservationBindings, + edit_ramps: &EditRamps, backend: Backend, ) -> Self { let (prepared, handles) = @@ -1055,7 +1058,7 @@ impl LivePlan { inventory: predecessor, committed, revision: successor::SUCCESSOR_REVISION, - edit_ramps: &EditRamps::new(), + edit_ramps, monitor: host_core::MonitorBase::NONE, observation, }, @@ -1141,6 +1144,10 @@ struct DirectRun { transitions: [u64; 3], /// The predecessor's `live_values_superseded` count after the hand-over (#1312 D2). carry_superseded: u64, + /// The successor's own `live_values_superseded` count after its first block: the carried + /// values its adopting block replaced (#1630 f). The test seam gives each plan its own count; + /// a host passes both plans the session's. + successor_superseded: u64, } /// Render `a` for [`SWAP_BLOCK`] blocks of `quantum` frames, then -- when `b` is given -- prepare @@ -1155,6 +1162,30 @@ fn direct_run( writes: &[LiveWrite], backend: Backend, successor_dual: bool, +) -> DirectRun { + direct_run_ramped( + a, + b, + &EditRamps::new(), + feed, + quantum, + writes, + backend, + successor_dual, + ) +} + +/// [`direct_run`] whose successor is prepared with the transaction's `edit_ramps` (#1394 D6). +#[allow(clippy::too_many_arguments)] +fn direct_run_ramped( + a: &Session, + b: Option<(&Session, &SessionModel)>, + edit_ramps: &EditRamps, + feed: &Feed, + quantum: usize, + writes: &[LiveWrite], + backend: Backend, + successor_dual: bool, ) -> DirectRun { use bench_support::alloc as bench_alloc; let mut plan = LivePlan::fresh(a, backend); @@ -1163,6 +1194,7 @@ fn direct_run( let mut swap_allocator = ((0, 0, 0), (0, 0)); let mut collapses = [[0; 2]; 2]; let mut carry_superseded = 0; + let mut successor_superseded = 0; plan.submit(feed, quantum, 0); for block in 0..BLOCKS { for write in writes.iter().filter(|write| write.block == block) { @@ -1171,8 +1203,15 @@ fn direct_run( let swap = block == SWAP_BLOCK && b.is_some(); if let (true, Some((session, committed))) = (swap, b) { collapses[0] = plan.prepared.plan.bank_collapse_counters(); - let mut successor = - LivePlan::successor(session, &plan.prepared.inventory, committed, backend); + let mut successor = LivePlan::successor_observed( + session, + &LivePlan::request(), + &plan.prepared.inventory, + committed, + &host_core::NO_OBSERVATION, + edit_ramps, + backend, + ); successor .prepared .plan @@ -1202,6 +1241,7 @@ fn direct_run( (audit.allocations, audit.deallocations), ); carry_superseded = plan.handles.live_values_superseded.get(); + successor_superseded = successor.handles.live_values_superseded.get(); blocks.push(output.iter().map(|sample| sample.to_bits()).collect()); plan = successor; continue; @@ -1219,6 +1259,7 @@ fn direct_run( collapses, transitions: plan.prepared.plan.bank_collapse_transitions(), carry_superseded, + successor_superseded, } } @@ -1897,6 +1938,58 @@ fn an_input_difference_carries_and_retargets(name: &str, edit: fn(&mut session:: Some(SWAP_BLOCK), "{name}, {backend:?}: the retarget moves the output from the swap block on" ); + // #1394 D6 through the successor path (#1630 g): the transaction gave the strip's + // `(Strip, Input)` row a 37-sample ramp. The retarget is the record of that edit, so the + // run equals the one that writes the classifier's records for it. Red if the join reads + // another strip's lengths, or none. + let mut edit_ramps = EditRamps::new(); + edit_ramps.set( + &session::EditRampTarget::Strip(id(RETARGETED)), + session::EditRampRow::Input, + Some(37), + ); + let ramped_delta = host_core::live_delta::classify_live_delta( + &a.model, + &moved, + LiveRamps::for_session(&moved), + &edit_ramps, + ) + .unwrap_or_else(|reason| panic!("{name}: the ramped edit is live: {reason:?}")); + let ramped_writes: Vec = ramped_delta.strips[0] + .input + .iter() + .map(|&record| LiveWrite { + block: SWAP_BLOCK, + strip: RETARGETED, + record, + }) + .collect(); + let ramped = direct_run_ramped( + &a, + Some((&b, &a.model)), + &edit_ramps, + &feed, + QUANTUM, + &[], + backend, + false, + ); + assert_eq!(ramped.carry, CarryOutcome::Carried, "{name}, {backend:?}"); + let ramped_written = run(&moved, &ramped_writes); + assert_eq!( + first_direct_difference(&ramped, &ramped_written), + None, + "{name}, {backend:?}: the retarget ramps over the edit's `(Strip, Input)` length" + ); + // Not vacuous where the row ramps (trim and polarity; a filter target does not take it): + // the edit's length is audible against the session's. + if ramped_delta.strips[0].input != delta.strips[0].input { + assert_eq!( + first_direct_difference(&ramped, &retargeted), + Some(SWAP_BLOCK), + "{name}, {backend:?}: the edit's length is audible" + ); + } } } @@ -1934,6 +2027,88 @@ fn a_polarity_change_carries_and_retargets_the_input_section() { }); } +/// #1630 f, D15-2 on the input cells: a carried live value that the successor's retarget replaces +/// in the adopting block is counted once. A left-only trim write sits unread in A's cells at the +/// hand-over and is committed in the base; B moves that trim again. The carry drain applies the +/// written value, and the successor's retarget replaces it before any sample renders: one value, +/// one cell, superseded. The run is bit-identical to one whose base never held the write, and with +/// no retarget (B leaves the trim at the written value) nothing is counted. Red without the +/// adopting block's count (0, not 1). +fn a_carried_input_value_the_retarget_replaces_is_counted(backend: Backend) { + const STRIP: &str = "eq2"; + const WRITTEN_DB: f32 = 1.75; + const MOVED_DB: f32 = -3.5; + let with_left_trim = |db: f32| { + let mut model = filtered_session(false); + model + .tracks + .iter_mut() + .find(|track| track.id.as_str() == STRIP) + .expect("the strip") + .builtins + .left + .trim_db = db; + model + }; + let a = Session::compile(filtered_session(false)); + let written = with_left_trim(WRITTEN_DB); + let b = Session::compile(with_muted_track(with_left_trim(MOVED_DB))); + let writes = [LiveWrite { + block: SWAP_BLOCK, + strip: STRIP, + record: builtins_compiler::TrackInputRecord::TrimDb { + lanes: builtins::BuiltinLaneSelector::Left, + db: WRITTEN_DB, + smoothing_samples: 64, + }, + }]; + let feed = Feed::new(SOURCE, 19); + let run = |b: &Session, committed: &SessionModel, writes: &[LiveWrite]| { + let run = direct_run( + &a, + Some((b, committed)), + &feed, + QUANTUM, + writes, + backend, + false, + ); + assert_eq!(run.carry, CarryOutcome::Carried, "{backend:?}"); + run + }; + let replaced = run(&b, &written, &writes); + assert_eq!( + (replaced.carry_superseded, replaced.successor_superseded), + (0, 1), + "{backend:?}: the carried value the retarget replaced is counted once" + ); + let unwritten = run(&b, &a.model, &[]); + assert_eq!(unwritten.successor_superseded, 0, "{backend:?}"); + assert_eq!( + first_direct_difference(&replaced, &unwritten), + None, + "{backend:?}: the replaced carried value moves no bit" + ); + // A carried value with no retarget is heard, not superseded. + let kept = run( + &Session::compile(with_muted_track(written.clone())), + &written, + &writes, + ); + assert_eq!( + (kept.carry_superseded, kept.successor_superseded), + (0, 0), + "{backend:?}: a carried value nothing replaces is not counted" + ); +} + +#[test] +fn a_carried_input_value_the_retarget_replaces_is_counted_on_every_width() { + for backend in backends() { + a_carried_input_value_the_retarget_replaces_is_counted(backend); + } +} + /// Gate 3 of #1276, live term. On the all-mono session a left-only trim record that leaves the /// trim where it was is pending in A's queue at the swap. It moves no word, but it is a one-channel /// write, so its strip's chain must render dual from the swap block on, as the reference's does @@ -2882,6 +3057,8 @@ struct CellRun { /// cells held unread that the carry drain replaced (#1312 D2). Zero for a run without a swap /// that rendered no block after its writes. carry_superseded: u64, + /// The successor's own count after its first block, as [`DirectRun::successor_superseded`]. + successor_superseded: u64, } /// Render `a` and write `writes` to the plan that renders their block next; with a `swap`, prepare @@ -2903,6 +3080,7 @@ fn cell_run( let mut restarted = Vec::new(); let mut swap_allocator = ((0, 0, 0), (0, 0)); let mut carry_superseded = 0; + let mut successor_superseded = 0; plan.live.submit(feed, QUANTUM, 0); for block in 0..BLOCKS { plan.write_block(writes, block); @@ -2944,6 +3122,7 @@ fn cell_run( (audit.allocations, audit.deallocations), ); carry_superseded = plan.live.handles.live_values_superseded.get(); + successor_superseded = successor.live.handles.live_values_superseded.get(); blocks.push(output.iter().map(|sample| sample.to_bits()).collect()); plan = successor; continue; @@ -2959,6 +3138,7 @@ fn cell_run( restarted, swap_allocator, carry_superseded, + successor_superseded, } } @@ -3504,6 +3684,94 @@ fn a_carried_lane_counts_its_replaced_writes_at_eight_lanes() { } } +/// #1630 f, D15-2 on the strip cells (the #1277 carry): a carried live value that the successor's +/// retarget replaces in the adopting block is counted once. A left-only fader write sits unread in +/// [`MOVED`]'s cells at the hand-over and is committed in the base; B moves that fader again. The +/// carry drain applies the written value, and the successor's retarget replaces it before any +/// sample renders: one value, one cell, superseded. The run is bit-identical to one whose base +/// never held the write, and with no retarget nothing is counted. Red without the adopting block's +/// count (0, not 1). +fn a_carried_strip_value_the_retarget_replaces_is_counted(backend: Backend, form: Form) { + const WRITTEN_DB: f32 = -7.5; + const MOVED_DB: f32 = -2.25; + let base = strip_state_session(); + let right = base + .tracks + .iter() + .find(|track| track.id.as_str() == MOVED) + .expect("the moved strip") + .fader + .right_db; + let a = Session::compile(base.clone()); + let written = with_fader(base.clone(), MOVED, [WRITTEN_DB, right]); + let b = Session::compile(with_muted_track(with_fader( + base.clone(), + MOVED, + [MOVED_DB, right], + ))); + let feed = feed_for_strips(); + let writes = [CellWrite { + block: SWAP_BLOCK, + strip: MOVED, + record: CellRecord::Fader(TrackFaderRecord::FaderDb { + lanes: BuiltinLaneSelector::Left, + db: WRITTEN_DB, + smoothing_samples: 64, + }), + }]; + let run = |to: &Session, committed: &SessionModel, writes: &[CellWrite]| { + let run = cell_run( + &a, + form, + Some(Swap { + to, + committed, + form, + successor_writes: &[], + edit_ramps: &EditRamps::new(), + }), + &feed, + writes, + backend, + ); + assert_eq!(run.carry, CarryOutcome::Carried, "{backend:?} {form:?}"); + assert!(run.restarted.is_empty(), "{backend:?} {form:?}"); + run + }; + let replaced = run(&b, &written, &writes); + assert_eq!( + (replaced.carry_superseded, replaced.successor_superseded), + (0, 1), + "{backend:?} {form:?}: the carried value the retarget replaced is counted once" + ); + let unwritten = run(&b, &base, &[]); + assert_eq!(unwritten.successor_superseded, 0, "{backend:?} {form:?}"); + assert_eq!( + first_cell_difference(&replaced, &unwritten), + None, + "{backend:?} {form:?}: the replaced carried value moves no bit" + ); + let kept = run( + &Session::compile(with_muted_track(written.clone())), + &written, + &writes, + ); + assert_eq!( + (kept.carry_superseded, kept.successor_superseded), + (0, 0), + "{backend:?} {form:?}: a carried value nothing replaces is not counted" + ); +} + +#[test] +fn a_carried_strip_value_the_retarget_replaces_is_counted_on_every_width_and_form() { + for backend in backends() { + for form in forms() { + a_carried_strip_value_the_retarget_replaces_is_counted(backend, form); + } + } +} + /// #1558 D4 (was #1276's gate 5, when a VCA was a prepared difference): B adds a VCA with a +3 dB /// offset over X. X's stage carries and is retargeted to the effective value, over the `VcaOffset` /// row's session length, and `restarted_strips()` is empty. Run 1 prepares B from A's base. Run 2 @@ -7878,6 +8146,7 @@ mod observation_carry { &plan.prepared.inventory, &a.model, swap.bindings.unwrap_or(&host_core::NO_OBSERVATION), + &EditRamps::new(), backend, ); subscribe(&mut successor, swap.successor_records); diff --git a/crates/protocol/src/controller.rs b/crates/protocol/src/controller.rs index 85bfafcf1..205573f26 100644 --- a/crates/protocol/src/controller.rs +++ b/crates/protocol/src/controller.rs @@ -1706,8 +1706,8 @@ impl ProtocolController

{ } /// [`Self::prepare_command_frame`] for a host that commits a transaction of values without a - /// compilation (#1613): a `SESSION_TRANSACTION_APPLY` whose edits are all `SetControl`s on - /// existing values is prepared with [`SessionStore::prepare_transaction_in_place`], so its token + /// compilation (#1613): a `SESSION_TRANSACTION_APPLY` whose edits are all `SetControl`s, on + /// existing values or adding parameter entries (#1620), is prepared with [`SessionStore::prepare_transaction_in_place`], so its token /// holds the edited owners and no compilation ([`PreparedSessionTransaction::is_in_place`]). /// Every other command, and every transaction the in-place preparation cannot prove equal to /// the full one, is prepared exactly as [`Self::prepare_command_frame`] prepares it, and its diff --git a/crates/protocol/src/model.rs b/crates/protocol/src/model.rs index c65ed17b2..d6bb2bd32 100644 --- a/crates/protocol/src/model.rs +++ b/crates/protocol/src/model.rs @@ -939,8 +939,8 @@ enum TransactionForm { /// The complete prospective compilation: the clone of the model with the edits applied, /// validated, normalized and written as canonical text. Compiled(CompiledSession), - /// A transaction of `SetControl`s on existing values (#1613): the owners the edits changed and - /// nothing else of the model. The committed compilation is not cloned, compiled or serialized; + /// A transaction of `SetControl`s that change values or add parameter entries (#1613, #1620): + /// the owners the edits changed and nothing else of the model. The committed compilation is not cloned, compiled or serialized; /// the commit installs the owners in it. InPlace(InPlaceEdit), /// A transaction of solo writes only: the model, the revision and the compilation stay as @@ -1202,7 +1202,8 @@ impl SessionStore { /// [`Self::prepare_transaction_with_monitor`] for a host that commits a transaction of values /// without a compilation (#1613). /// - /// A transaction whose edits are all `SetControl`s on values that exist is prepared **in + /// A transaction whose edits are all `SetControl`s, on values that exist or adding parameter + /// entries (#1620, charged exactly by `ResourceEstimate::after_owners_edit`), is prepared **in /// place**: the owners the edits name are cloned and edited with the very `controls::set` the /// full path uses, checked with the session's own value validators, and held with the exact /// resource estimate and the revision the commit gives. The committed model is not cloned, diff --git a/crates/session/src/estimate.rs b/crates/session/src/estimate.rs index 70ce50435..5263e4dda 100644 --- a/crates/session/src/estimate.rs +++ b/crates/session/src/estimate.rs @@ -327,7 +327,7 @@ impl ResourceEstimate { canonical_bytes: u64, caps: crate::CompileCaps, ) -> Option { - let parameters = u64::try_from(growth.parameters).ok()?; + let parameters = u64::try_from(growth.parameters()).ok()?; let entry = size::(); let canonical_upper_bound_bytes = self .canonical_upper_bound_bytes @@ -340,7 +340,7 @@ impl ResourceEstimate { .checked_sub(self.canonical_bytes)? .checked_add(canonical_upper_bound_bytes)? .checked_add(parameters.checked_mul(entry)?)?; - let largest_grown_list = u64::try_from(growth.largest_grown_list) + let largest_grown_list = u64::try_from(growth.largest_grown_list()) .ok()? .checked_mul(entry)?; let single_allocation_bytes = self diff --git a/crates/session/src/in_place.rs b/crates/session/src/in_place.rs index 1b293f6f1..0d8cf5dfa 100644 --- a/crates/session/src/in_place.rs +++ b/crates/session/src/in_place.rs @@ -120,12 +120,37 @@ pub fn owners_values_valid<'a>(owners: impl IntoIterator>) - /// What an edit of values adds to the counts of the resource estimate (#1620): parameter entries, /// and nothing else. A `SetControl` of a parameter that an effect's sparse list does not hold yet /// adds its entry (a one-lane write on a `both` entry splits it into two). -#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] +/// +/// Only [`owners_growth`] builds one: its fields are private, so no caller can hand +/// [`ResourceEstimate::after_owners_edit`] a growth smaller than the owners' real one and have the +/// estimate under-charge. +/// +/// [`ResourceEstimate::after_owners_edit`]: crate::ResourceEstimate::after_owners_edit +#[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct OwnersGrowth { /// Parameter entries added across the edited owners. - pub parameters: usize, + parameters: usize, + /// The length of the longest parameter list that grew, or zero. + largest_grown_list: usize, +} + +impl OwnersGrowth { + const NONE: Self = Self { + parameters: 0, + largest_grown_list: 0, + }; + + /// Parameter entries added across the edited owners. + #[must_use] + pub const fn parameters(&self) -> usize { + self.parameters + } + /// The length of the longest parameter list that grew, or zero. - pub largest_grown_list: usize, + #[must_use] + pub const fn largest_grown_list(&self) -> usize { + self.largest_grown_list + } } /// What the owners of `new` add to those of `old`, pairwise, to the counts the resource estimate @@ -141,7 +166,7 @@ pub fn owners_growth<'a>( new: impl IntoIterator>, ) -> Option { let (mut old, mut new) = (old.into_iter(), new.into_iter()); - let mut growth = OwnersGrowth::default(); + let mut growth = OwnersGrowth::NONE; loop { match (old.next(), new.next()) { (None, None) => return Some(growth), diff --git a/crates/session/tests/in_place.rs b/crates/session/tests/in_place.rs index 3613020b1..4bb3b25b0 100644 --- a/crates/session/tests/in_place.rs +++ b/crates/session/tests/in_place.rs @@ -128,8 +128,9 @@ fn swapping_the_owners_in_gives_the_compiled_session_of_the_edited_model() { owners_growth( slots.in_committed(compiled.normalized_model()), slots.in_owners(&owners) - ), - Some(OwnersGrowth::default()) + ) + .map(growth_pair), + Some((0, 0)) ); compiled.swap_owners(&mut owners, &slots); compiled.finish_in_place( @@ -212,8 +213,9 @@ fn owners_with_values_the_compiler_refuses_are_not_valid_and_counts_are_compared owners_growth( [OwnerRef::Track(&model.tracks[0])], [OwnerRef::Track(&model.tracks[0].clone())], - ), - Some(OwnersGrowth::default()) + ) + .map(growth_pair), + Some((0, 0)) ); // #1620: an added parameter entry is growth, counted with the list it lengthens. let mut added = model.tracks[0].clone(); @@ -226,10 +228,13 @@ fn owners_with_values_the_compiler_refuses_are_not_valid_and_counts_are_compared owners_growth( [OwnerRef::Track(&model.tracks[0])], [OwnerRef::Track(&added)], - ), - Some(OwnersGrowth { - parameters: 1, - largest_grown_list: length, - }) + ) + .map(growth_pair), + Some((1, length)) ); } + +/// The growth's two counts, the only view of it outside `session`. +fn growth_pair(growth: OwnersGrowth) -> (usize, usize) { + (growth.parameters(), growth.largest_grown_list()) +} diff --git a/tools/parameter-metadata/tests/control_command_heap.rs b/tools/parameter-metadata/tests/control_command_heap.rs index 20d64246e..9004b8f98 100644 --- a/tools/parameter-metadata/tests/control_command_heap.rs +++ b/tools/parameter-metadata/tests/control_command_heap.rs @@ -8,7 +8,8 @@ //! allocator's thread-scoped counters and live-heap window (`bench_support::alloc`), relative to the //! heap live at the call's entry. //! -//! The ceilings are the numbers measured on this head with a few KB of slack. The steady call is +//! The ceilings are the numbers measured on this head plus 2,000 bytes, rounded up to the next +//! 1,000 (the slack rule below). The steady call is //! the same at every track count, and so is the first (#1620): a live-only commit must not depend //! on the session's size. The numbers print with `--nocapture`. //! @@ -206,22 +207,24 @@ fn run(tracks: usize) -> (Heap, Heap) { (first.expect("a first call"), steady_first) } -/// Ceilings, measured on this head (decision record of #1615): bytes of heap live at the peak above -/// the call's entry, and bytes requested over the call, with a few KB of slack. Debug and release -/// builds give the same numbers (the debug stage check is off, above). +/// Ceilings: bytes of heap live at the peak above the call's entry, and bytes requested over the +/// call. The slack rule: each ceiling is the number measured on this head plus 2,000 bytes, +/// rounded up to the next 1,000. Debug and release builds give the same numbers (the debug stage +/// check is off, above). /// -/// One track: the first call peaks at 76,011 bytes and every later call at 71,252, requesting -/// 101,025 bytes in 753 allocations. Before #1615 they were 172,790 and 168,031, requesting 293,477 -/// in 786. +/// Measured at every size (1, 64 and 256 tracks): the first call peaks at 72,865 bytes, requesting +/// 103,033 in 757 allocations; every steady call peaks at 72,841, requesting 103,918 in 758. /// -/// #1620: the first call is in place too. It used to take the full path, because its edits add -/// parameter entries to the sparse EQ list (peak 77,600 / 198,182 / 663,179 bytes at 1 / 64 / 256 -/// tracks, 839 / 1,853 / 4,927 allocations); in place, it peaks at 72,865 bytes, requesting 103,033 -/// in 757 allocations, at every size. Its ceiling was 78,000 at one track and none above. -const FIRST_PEAK_CEILING: usize = 74_000; -const FIRST_REQUESTED_CEILING: usize = 105_000; -const STEADY_PEAK_CEILING: usize = 73_000; -const STEADY_REQUESTED_CEILING: usize = 104_000; +/// History. #1615: the steady call peaked at 71,252 bytes, requesting 101,025 in 753 allocations +/// (before #1615: 168,031, requesting 293,477 in 786). #1620: the first call is in place too; it +/// used to take the full path, because its edits add parameter entries to the sparse EQ list +/// (peak 77,600 / 198,182 / 663,179 bytes at 1 / 64 / 256 tracks, 839 / 1,853 / 4,927 +/// allocations). #1624 (B' adds 780 B and 2 allocations per begun EQ owner): the steady ceilings +/// rose from 73,000 / 104,000 and the first call's from 74,000 / 105,000 under this rule. +const FIRST_PEAK_CEILING: usize = 75_000; +const FIRST_REQUESTED_CEILING: usize = 106_000; +const STEADY_PEAK_CEILING: usize = 75_000; +const STEADY_REQUESTED_CEILING: usize = 106_000; #[test] fn a_live_only_nine_edit_transaction_has_a_small_peak_at_every_size() { @@ -234,11 +237,9 @@ fn a_live_only_nine_edit_transaction_has_a_small_peak_at_every_size() { for tracks in [1_usize, 64, 256] { let (first, steady) = run(tracks); println!("tracks {tracks}: first {first:?}; steady {steady:?}"); - // A call keeps only its 80-byte response. - assert!( - steady.left <= 256 && first.left <= 256, - "a call leaves heap live" - ); + // A steady call keeps only its 80-byte response. The first call is not checked: it + // frees the boot's canonical text, and the live-bytes window stops at zero. + assert!(steady.left <= 256, "a call leaves heap live: {steady:?}"); assert!( first.peak <= FIRST_PEAK_CEILING && first.requested <= FIRST_REQUESTED_CEILING, "{tracks}: first {first:?}" diff --git a/tools/realtime-policy/src/policy.rs b/tools/realtime-policy/src/policy.rs index 1e6e6a4a9..48bbcff04 100644 --- a/tools/realtime-policy/src/policy.rs +++ b/tools/realtime-policy/src/policy.rs @@ -221,10 +221,12 @@ impl Policy { // reads of `CellProgram` and `Program::cell` in `program.rs`) and #1638's three // (`crates/builtins`: the fader operation sink, and the bank's and the scalar track's // `process_with`) and #1639's two (`crates/graph`: the jump-length cell's word decode - // and its block-entry read). The floors had fallen behind the tree before #1637; they are set - // to its counts again. A change that adds a marker raises them in the same change. + // and its block-entry read) and #1630's one (`crates/builtins-compiler`: the cell + // readers' carry marks, D15-2). The floors had fallen behind the tree before #1637; they + // are set to its counts again. A change that adds a marker raises them in the same + // change. marked_files_floor: 32, - marked_regions_floor: 149, + marked_regions_floor: 150, render_thread_exports: table::parse(RENDER_THREAD_EXPORTS) .unwrap_or_else(|error| panic!("render-thread-exports.json: {error}")), unwrapped_exports: UNWRAPPED_EXPORTS From 81aa9ef1da49f0b0e20993b7d43b54f930c38b11 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 21:29:44 +0000 Subject: [PATCH 09/12] Batch 14b review fold, part A: #1621, #1623 - #1621: the paired-spans test refuses only a window longer than the capacity; stale prose. - #1623: the core refuses a non-finite endpoint (response.grid) and a non-finite override (response.override.value) on both hosts, with a parity row; the encoder stays strict; the sdk CI job checks the generated response-frame and session-key assets; the SDK refuses a non-number and an unknown reply section. Co-Authored-By: Claude Opus 5.5 --- ...de-live-effect-window-by-its-live-cells.md | 13 ++ ...ponse-get-codes-and-drop-its-own-checks.md | 29 ++++ .github/workflows/qualification.yml | 5 + crates/capi/tests/response_get.rs | 13 +- .../effect-compiler/tests/native_session.rs | 5 +- crates/effect-contract/src/lib.rs | 2 +- crates/host-core/src/control_provider.rs | 28 +++- crates/host-core/src/response.rs | 42 ++++-- crates/host-core/tests/response_frame.rs | 111 ++++++++++++--- crates/host-core/tests/response_query.rs | 51 +++++++ crates/protocol/src/message_wire/response.rs | 50 ++++--- crates/protocol/src/message_wire/tests.rs | 114 ++++++++++----- docs/CONTROL_PROTOCOL_REGISTRY.md | 11 +- hosts/host-web/tests/control_parity.rs | 132 ++++++++++++++++++ sdk/src/core/response-frame.ts | 15 +- sdk/src/core/response.ts | 23 ++- sdk/test/response-evals.mjs | 73 ++++++++-- tools/console-workload/tests/paired_spans.rs | 46 +++--- 18 files changed, 621 insertions(+), 142 deletions(-) diff --git a/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md b/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md index 3b7e44769..edff35760 100644 --- a/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md +++ b/.github/ISSUE_SPECS/1621-size-the-per-node-live-effect-window-by-its-live-cells.md @@ -88,3 +88,16 @@ observed, ceilings unchanged); the browser fixture's eight budget rows identical plan row printed by `capi_retained_bytes_charge_every_byte_the_compile_retains`: observed 104,589 -> 98,781 and its estimate 196,180 -> 190,372 (-5,808 each, slack 91,591 unchanged; the per-node EQ/ compressor windows of that session, 128-span capacity). + +**Review fold (batch 14b).** B1: `console-workload` `paired_spans` +`every_launch_effect_refuses_a_staging_window_that_is_not_its_capacity` still refused a window of +`capacity - 1`, which this issue made legal; it is now +`every_launch_effect_refuses_a_staging_window_longer_than_its_capacity`: it raises the direct +factory request to the live cells as preparation does (D4), accepts a window of the capacity and of +`EffectLaneShape::live_parameter_cells`, and refuses `capacity + 1`. The lower bound is the lane's +(`effect-contract` `live_window` `a_window_shorter_than_the_live_cells_is_refused_with_the_count_it_needs`). +Red when `check_window` returns `Ok(())` unconditionally (checked by mutation). Stale prose fixed in +`effect-contract/src/lib.rs` (`check_window` is the per-node bound, not "shared by both block types") +and `effect-compiler/tests/native_session.rs` (the window is the lane's cells, which the capacity +equals there). Gate 3 adds `cargo test --locked --release -p console-workload` (green; `paired_spans` +13 pass). diff --git a/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md b/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md index 8a4d48317..b559cbc48 100644 --- a/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md +++ b/.github/ISSUE_SPECS/1623-generate-the-sdks-response-get-codes-and-drop-its-own-checks.md @@ -113,3 +113,32 @@ non-number parameter value, and the SDK's own `requestDeadlineMs` timer bound. **Successor found.** `control-frame.ts` still holds the `WIRE_*` type codes, `FLAG_REVISION_ANY`, the header magic and the status names by hand; the same frame table could carry them. + +**Review fold (batch 14b).** M1: `check-sdk-generated.sh` with an artifact directory skips the +SOURCE_ONLY assets (session keys, response frame), and an sdk-only change routes to the `sdk` job, so +that job now also runs `cargo test --locked -p parameter-metadata --test response_frame --test +session_keys` (a step inside the existing job: no expectation-table change; `check-ci-path-routing`, +`check-test-support-ci` and `check-script-reachability` pass). M2 (root ruling, thick core): the +codec's decoder no longer refuses a request's value rules (an empty effect ID, a NaN or infinite +endpoint or override value, a bypass row off `both` or not 0/1); the encoder still does, since the +core refuses every such request too, and the wire format is unchanged. host-core +`check_response_request` refuses a non-finite endpoint with `response.grid` and a non-finite override +value or input-filter cutoff with the new `response.override.value` (`ResponseQueryError::OverrideValue`, +`INVALID_FIELD`), after the axis and before any shaping; an empty effect ID is `response.effect.unknown`, +a bypass off `both` `response.override.lane`, a bypass between 0 and 1 `response.override.bypass`. +Registered in `docs/CONTROL_PROTOCOL_REGISTRY.md` (the SDK lists no codes). Tests: protocol +`response_get_request_value_rules_are_refused_on_encode_only`; host-core +`a_non_finite_or_bad_value_sent_raw_is_refused_with_the_core_code`, `bounds_pass_at_the_limit_and_refuse_one_past_it` +(override value and cutoff legs), `response_get_asks_the_core_first_and_names_a_bad_bypass`; host-web +`control_parity` `a_response_request_with_a_bad_value_is_refused_alike_on_both_hosts` (one frame, +both hosts: same result, reply bytes, code, unchanged control rows and document); SDK +`response-evals.mjs` gate-1 rows now `response.grid`/`response.override.value`, plus a raw-frame test +for the empty ID and the bypass rows. All red with the decoder check restored or the core check +removed (checked by mutation). Minors: `putF32` refuses a non-number with `MisoUsageError`, and +`response.ts` no longer coerces (`Math.fround` on the grid removed; `cutoff()` passes a non-number +on); a reply with more sections than the generated capability declares is an `internal` error; the +duplicate eval row "a shared parameter on one lane" is deleted and the left-lane comment no longer +overclaims; capi `every_reachable_refusal_is_typed_and_changes_nothing` pins `effect.parameter.initial`. +Successor (with `control-frame.ts`): `sdk/src/core/live-response.ts:227-234` and +`sdk/src/browser/engine.ts:271-286` (the strip-target and live response paths) still hand-code +grid/lane codes and keep their own grid checks (finite endpoints, logarithmic minimum). diff --git a/.github/workflows/qualification.yml b/.github/workflows/qualification.yml index b87d419d1..3377b0cd9 100644 --- a/.github/workflows/qualification.yml +++ b/.github/workflows/qualification.yml @@ -278,6 +278,11 @@ jobs: bash scripts/check-sdk-types.sh bash scripts/check-sdk-headless.sh target/ci/qualification-artifacts bash scripts/sdk-package.sh check target/ci/qualification-artifacts + # check-sdk-generated.sh, given the artifact directory, skips the SOURCE_ONLY assets + # (sdk/codegen/assets.mjs: session keys and response frame), whose currency tests otherwise run + # only on the full route; an sdk-only change routes here, so they run here too (#1623). + - name: Check the source-only SDK assets are current + run: cargo test --locked -p parameter-metadata --test response_frame --test session_keys artifact-gates: name: AudioWorklet artifact gates diff --git a/crates/capi/tests/response_get.rs b/crates/capi/tests/response_get.rs index 445650048..562853632 100644 --- a/crates/capi/tests/response_get.rs +++ b/crates/capi/tests/response_get.rs @@ -735,7 +735,7 @@ fn every_reachable_refusal_is_typed_and_changes_nothing() { "a gain outside its domain", eq(&|r| r.overrides = vec![row(BAND_1_GAIN, ParameterChannel::Both, 500.0)]), StatusCode::ValidationFailed, - "", + "effect.parameter.initial", ), ( "no preparation budget", @@ -804,16 +804,7 @@ fn every_reachable_refusal_is_typed_and_changes_nothing() { }; assert_eq!(actual, status, "{name}"); assert_eq!(codes.len(), 1, "{name}: one diagnostic"); - if code.is_empty() { - // The owner's own code, such as `effect.parameter.out_of_domain`. - assert!( - codes[0].split('.').count() >= 2 && !codes[0].starts_with("response."), - "{name}: {}", - codes[0] - ); - } else { - assert_eq!(codes[0], code, "{name}"); - } + assert_eq!(codes[0], code, "{name}"); assert_eq!( endpoint.state(), before, diff --git a/crates/effect-compiler/tests/native_session.rs b/crates/effect-compiler/tests/native_session.rs index 7e33df4a8..13876e542 100644 --- a/crates/effect-compiler/tests/native_session.rs +++ b/crates/effect-compiler/tests/native_session.rs @@ -652,11 +652,12 @@ fn every_launch_effect_is_prepared_with_a_window_for_all_its_live_cells() { assert_eq!( lane.check_window(capacity), Ok(()), - "{id}: the window is the capacity and holds the lane's cells" + "{id}: a window of the prepared capacity (here the lane's cells) holds the lane's cells" ); // One transaction writes every live parameter cell of the instance (the two channels of a // per-lane parameter apart, so no pair folds into fewer spans), and one block, staging into - // a window of exactly the prepared capacity, applies every one of them. + // a window of exactly the lane's live cells (which the prepared capacity equals here), + // applies every one of them. let (gate, mut pin) = engine::realtime::RevisionGate::pair(0); let stamp = pin.stamp(1).expect("a legal revision"); let mut written = 0; diff --git a/crates/effect-contract/src/lib.rs b/crates/effect-contract/src/lib.rs index b3c4ea2d5..b9d4d248f 100644 --- a/crates/effect-contract/src/lib.rs +++ b/crates/effect-contract/src/lib.rs @@ -1665,7 +1665,7 @@ pub enum ProcessBlockError { AutomationWindow, } -/// The one statement of the staging-window bound, shared by both block types. +/// The one statement of the per-node staging-window bound: the window may not exceed the capacity. /// /// See [`EffectProcessBlock::check_automation_window`] for why the window may not exceed the capacity. const fn check_window(window: usize, automation_capacity: u32) -> Result<(), ProcessBlockError> { diff --git a/crates/host-core/src/control_provider.rs b/crates/host-core/src/control_provider.rs index 71a8853da..5253bca2d 100644 --- a/crates/host-core/src/control_provider.rs +++ b/crates/host-core/src/control_provider.rs @@ -610,9 +610,9 @@ fn contract_channel(lane: ParameterChannel) -> effect_contract::ParameterChannel /// The checks every `RESPONSE_GET` meets first, whatever the endpoint: the target is a /// configuration (the address and strip targets are served by later slices), then the core's own -/// checks in the core's order (rate, points, override count, axis), so the refusal of a request -/// that is wrong in several ways is the same on every host. It returns the configuration the -/// request targets and the grid it names. +/// checks in the core's order (rate, points, override count, axis, override values), so the +/// refusal of a request that is wrong in several ways is the same on every host. It returns the +/// configuration the request targets and the grid it names. fn response_preflight( sample_rate_hz: u32, request: &ResponseGetRequest, @@ -636,7 +636,13 @@ fn response_preflight( maximum_hz: request.maximum_hz, }, }; - check_response_request(sample_rate_hz, grid, request.overrides.len()).map_err(query_refusal)?; + check_response_request( + sample_rate_hz, + grid, + request.overrides.len(), + request.overrides.iter().all(|row| row.value.is_finite()), + ) + .map_err(query_refusal)?; Ok((configuration, grid)) } @@ -2453,9 +2459,10 @@ mod tests { } /// A request that is wrong in several ways is refused with the core's refusal first, in the - /// core's order (rate, points, override count, axis), before the provider's own rules (a - /// rate on a session endpoint, a zero byte budget, the bypass); and a bypass value between 0 - /// and 1 is `response.override.bypass`, not "no such parameter". + /// core's order (rate, points, override count, axis, override values), before the provider's + /// own rules (a rate on a session endpoint, a zero byte budget, the bypass); a bypass value + /// between 0 and 1 is `response.override.bypass`, not "no such parameter"; and a NaN override + /// value, a bypass row's included, is `response.override.value` (#1623). /// /// Red if the provider shapes the overrides before it asks the core, so a request with one bad /// point count and one bad bypass value answers the bypass, or the bypass shares the code of an @@ -2498,6 +2505,13 @@ mod tests { "the axis is the core's and precedes the session-rate rule" ); both_wrong.minimum_hz = 20.0; + both_wrong.overrides.push(bypass(f32::NAN)); + assert_eq!( + refusal(&both_wrong), + (StatusCode::InvalidField, "response.override.value"), + "a non-finite override value is the core's, after the axis and before the session rate" + ); + both_wrong.overrides.pop(); assert_eq!( refusal(&both_wrong), (StatusCode::InvalidField, "response.rate.session"), diff --git a/crates/host-core/src/response.rs b/crates/host-core/src/response.rs index 52fb825d1..c7e8988f5 100644 --- a/crates/host-core/src/response.rs +++ b/crates/host-core/src/response.rs @@ -757,6 +757,8 @@ pub enum ResponseQueryError { Overrides, /// The sample rate is not a launch rate. SampleRate, + /// An override value (or, for the input filters, a cutoff) is NaN or infinite. + OverrideValue, } /// The class of a refused response query, which a host maps to its own status. @@ -788,6 +790,7 @@ impl ResponseQueryError { Self::Points => (InvalidField, "response.points"), Self::Overrides => (LimitExceeded, "response.override.count"), Self::SampleRate => (InvalidField, "response.rate"), + Self::OverrideValue => (InvalidField, "response.override.value"), Self::Preview(error) => match error { ResponsePreviewError::InvalidShape => (InvalidField, "response.rate"), ResponsePreviewError::InvalidGrid => (InvalidField, "response.grid"), @@ -825,14 +828,16 @@ impl ResponseQueryError { /// The checks of a response query that need no owner, in the order every host reports them: an /// unsupported rate, a point count outside `2..=4096`, more than [`RESPONSE_MAXIMUM_OVERRIDES`] -/// override rows, then the frequency axis. [`prepare_response_query`] runs them first; a host that -/// shapes the request before calling it (the control provider folds override rows into a bypass -/// and cutoffs) runs them before its own shaping, so the core's refusal wins whichever request -/// word is wrong first. +/// override rows, the frequency axis (a NaN or infinite endpoint is `response.grid`), then an +/// override value that is NaN or infinite (`response.override.value`, #1623). +/// [`prepare_response_query`] runs them first; a host that shapes the request before calling it +/// (the control provider folds override rows into a bypass and cutoffs) runs them before its own +/// shaping, so the core's refusal wins whichever request word is wrong first. pub(crate) fn check_response_request( sample_rate_hz: u32, grid: ResponsePreviewGrid, override_count: usize, + override_values_finite: bool, ) -> Result<(), ResponseQueryError> { if !engine::is_launch_sample_rate(engine::SampleRateHz(sample_rate_hz)) { return Err(ResponseQueryError::SampleRate); @@ -845,7 +850,11 @@ pub(crate) fn check_response_request( } // The axis rules are the query's, not the owner's: refuse a bad axis here, before any owner is // prepared, so a host answers it the same way and with the same code. - check_response_axis(grid, sample_rate_hz).map_err(ResponseQueryError::Preview) + check_response_axis(grid, sample_rate_hz).map_err(ResponseQueryError::Preview)?; + if !override_values_finite { + return Err(ResponseQueryError::OverrideValue); + } + Ok(()) } /// A response query whose owner is prepared; `curves` evaluates it. @@ -892,11 +901,26 @@ pub struct ResponseCurves { pub fn prepare_response_query( query: &ResponseQuery<'_>, ) -> Result { - let override_count = match query.target { - ResponsePreviewTarget::Effect { overrides, .. } => overrides.len(), - ResponsePreviewTarget::InputFilters { .. } => 0, + // An input-filters target carries its cutoffs, already folded from its rows: they are its + // override values. + let (override_count, finite) = match query.target { + ResponsePreviewTarget::Effect { overrides, .. } => ( + overrides.len(), + overrides.iter().all(|row| row.value.is_finite()), + ), + ResponsePreviewTarget::InputFilters { + left_hpf_hz, + left_lpf_hz, + right_hpf_hz, + right_lpf_hz, + } => ( + 0, + [left_hpf_hz, left_lpf_hz, right_hpf_hz, right_lpf_hz] + .iter() + .all(|cutoff| cutoff.is_finite()), + ), }; - check_response_request(query.sample_rate_hz, query.grid, override_count)?; + check_response_request(query.sample_rate_hz, query.grid, override_count, finite)?; let preview = prepare_response_preview(ResponsePreviewRequest { configuration_id: query.configuration_id, sample_rate_hz: query.sample_rate_hz, diff --git a/crates/host-core/tests/response_frame.rs b/crates/host-core/tests/response_frame.rs index 8f74be1d7..35630bb21 100644 --- a/crates/host-core/tests/response_frame.rs +++ b/crates/host-core/tests/response_frame.rs @@ -5,8 +5,8 @@ //! refused as the entry documents. //! * **The caller's limits.** The request's own `maximum_reply_bytes` lowers the reply bound. //! * **The core's refusals, not the client's (#1623).** A logarithmic axis from zero, a quality the -//! effect does not publish, a lane given twice and a non-finite endpoint are each refused with a -//! status and code of the core. The browser SDK sends them as given and shows exactly these +//! effect does not publish, a lane given twice, a non-finite endpoint or override value and a +//! bypass value between 0 and 1 are each refused with a status and code of the core. The browser SDK sends them as given and shows exactly these //! (`sdk/test/response-evals.mjs`); `crates/capi/tests/response_get.rs` holds the C ABI session //! to the same codes. //! @@ -236,28 +236,99 @@ fn the_core_refuses_what_a_thin_client_sends_as_given() { } } -/// A frame the Rust encoder refuses to write (a NaN endpoint) is still a frame a client can send; -/// the decoder refuses it with the protocol's own code. -#[test] -fn a_non_finite_endpoint_sent_raw_is_refused_by_the_decoder() { - let mut frame = encode(CommandPayload::ResponseGet(&request())); - let finite = 20.0_f32.to_le_bytes(); - let at = frame +/// `frame` with the one four-byte occurrence of `from` replaced by `to`. +fn patched(mut frame: Vec, from: f32, to: f32) -> Vec { + let from = from.to_le_bytes(); + let at: Vec = frame .windows(4) - .position(|window| window == finite) - .expect("the minimum endpoint is in the frame"); - frame[at..at + 4].copy_from_slice(&f32::NAN.to_le_bytes()); - let mut reply = Vec::new(); - serve_response_frame(&frame, &mut reply).expect("a command frame"); - let (status, message, codes) = refusal(&reply).expect("a refusal"); + .enumerate() + .filter(|(_, window)| *window == from) + .map(|(at, _)| at) + .collect(); + assert_eq!(at.len(), 1, "the value is in the frame exactly once"); + frame[at[0]..at[0] + 4].copy_from_slice(&to.to_le_bytes()); + frame +} + +/// #1623 (root ruling, thick core): a frame the Rust encoder refuses to write -- a NaN or infinite +/// endpoint, a NaN override value, a bypass value between 0 and 1 -- is still a frame a client can +/// send. The decoder reads it as sent and the core refuses it with its own typed code, never the +/// codec's generic `protocol.failure`. +/// +/// Red if the decoder refuses the value again (each code becomes `protocol.failure`), or if the core +/// stops checking the value before the owner is prepared. +#[test] +fn a_non_finite_or_bad_value_sent_raw_is_refused_with_the_core_code() { + let mut with_rows = request(); + with_rows.overrides = vec![ + ResponseOverride { + control: protocol::RESPONSE_BYPASS_CONTROL, + lane: ParameterChannel::Both, + value: 1.0, + }, + ResponseOverride { + control: 4, + lane: ParameterChannel::Both, + value: 3.25, + }, + ]; + let frame = encode(CommandPayload::ResponseGet(&with_rows)); assert_eq!( - (status, message, codes), + refusal(&{ + let mut reply = Vec::new(); + serve_response_frame(&frame, &mut reply).expect("a command frame"); + reply + }), + None, + "the unpatched frame is answered" + ); + for (name, from, to, status, code) in [ ( + "a NaN minimum", + 20.0, + f32::NAN, StatusCode::InvalidField, - MessageId::ResponseGet, - vec!["protocol.failure".to_owned()] - ) - ); + "response.grid", + ), + ( + "an infinite maximum", + 20_000.0, + f32::INFINITY, + StatusCode::InvalidField, + "response.grid", + ), + ( + "a NaN override value", + 3.25, + f32::NAN, + StatusCode::InvalidField, + "response.override.value", + ), + ( + "an infinite override value", + 3.25, + f32::NEG_INFINITY, + StatusCode::InvalidField, + "response.override.value", + ), + ( + "a bypass of one half", + 1.0, + 0.5, + StatusCode::InvalidField, + "response.override.bypass", + ), + ] { + let mut reply = Vec::new(); + serve_response_frame(&patched(frame.clone(), from, to), &mut reply) + .expect("a command frame"); + let (got_status, message, codes) = refusal(&reply).unwrap_or_else(|| panic!("{name}")); + assert_eq!( + (got_status, message, codes), + (status, MessageId::ResponseGet, vec![code.to_owned()]), + "{name}" + ); + } } #[test] diff --git a/crates/host-core/tests/response_query.rs b/crates/host-core/tests/response_query.rs index fc944d234..6f43f2fb3 100644 --- a/crates/host-core/tests/response_query.rs +++ b/crates/host-core/tests/response_query.rs @@ -348,6 +348,52 @@ fn bounds_pass_at_the_limit_and_refuse_one_past_it() { ] { prepare_response_query(&axis(rate, grid)).expect("an axis inside the rules"); } + // A NaN or infinite override value, or input-filter cutoff, is refused before any owner is + // prepared, with its own error (#1623), and after the axis. + for value in [f32::NAN, f32::INFINITY, f32::NEG_INFINITY] { + let mut rows = overrides.clone(); + rows[1].value = value; + assert_eq!( + refused(query(48_000, effect(&rows), both, false, POINTS)), + Some(ResponseQueryError::OverrideValue), + "override value {value}" + ); + assert_eq!( + refused(ResponseQuery { + grid: linear(f32::NAN, 1_000.0), + ..query(48_000, effect(&rows), both, false, POINTS) + }), + Some(ResponseQueryError::Preview( + ResponsePreviewError::InvalidGrid + )), + "the axis precedes the override value {value}" + ); + let ResponsePreviewTarget::InputFilters { + left_lpf_hz, + right_hpf_hz, + right_lpf_hz, + .. + } = input_filters() + else { + unreachable!() + }; + assert_eq!( + refused(query( + 48_000, + ResponsePreviewTarget::InputFilters { + left_hpf_hz: value, + left_lpf_hz, + right_hpf_hz, + right_lpf_hz, + }, + both, + false, + POINTS + )), + Some(ResponseQueryError::OverrideValue), + "cutoff {value}" + ); + } // The axis is checked before the owner: an unknown effect on a bad axis is the axis's error, // and the same effect on a good axis is its own. let unknown = |grid: ResponsePreviewGrid| ResponseQuery { @@ -575,6 +621,11 @@ fn every_query_error_has_its_class_and_code() { InvalidField, "response.rate", ), + ( + ResponseQueryError::OverrideValue, + InvalidField, + "response.override.value", + ), ( preview(Preview::InvalidShape), InvalidField, diff --git a/crates/protocol/src/message_wire/response.rs b/crates/protocol/src/message_wire/response.rs index 93ebc302b..0caad821c 100644 --- a/crates/protocol/src/message_wire/response.rs +++ b/crates/protocol/src/message_wire/response.rs @@ -208,11 +208,15 @@ fn finite_all(values: &[f32]) -> bool { values.iter().all(|value| value.is_finite()) } -/// A request's own rules, shared by both directions. -fn response_request_is_valid(limits: crate::ProtocolLimits, value: &ResponseGetRequest) -> bool { - let target_ok = match &value.target { +/// A request's shape rules, shared by both directions: what a provider needs to read the request +/// at all. A request that breaks one is refused with the codec's own `INVALID_FIELD`. +fn response_request_shape_is_valid( + limits: crate::ProtocolLimits, + value: &ResponseGetRequest, +) -> bool { + match &value.target { ResponseTarget::Configuration(ResponseConfiguration::Effect { effect, .. }) => { - !effect.is_empty() && text_fits(limits, effect) + text_fits(limits, effect) } ResponseTarget::Configuration(ResponseConfiguration::InputFilters) => true, ResponseTarget::Address(address) => { @@ -222,25 +226,37 @@ fn response_request_is_valid(limits: crate::ProtocolLimits, value: &ResponseGetR ResponseTarget::Strip(id) => { stable_id_fits(limits, id) && !value.sections && value.overrides.is_empty() } + } +} + +/// A request's value rules, checked on encode only: no effect ID is empty, nothing the request +/// carries as a number is NaN or infinite (the axis endpoints and every override value), and an +/// effect's bypass row (control 0) is on lane `both` with a value of exactly 0 or 1. +/// +/// The decoder does not check these (#1623): a client that sends such a request gets the core's +/// typed refusal for it (`response.effect.unknown`, `response.grid`, `response.override.value`, +/// `response.override.lane`, `response.override.bypass`) from host-core's response provider, the +/// same on every host, not the codec's generic failure. The encoder keeps them because every +/// request they refuse is one the core would refuse too. +fn response_request_values_are_valid(value: &ResponseGetRequest) -> bool { + let effect = match &value.target { + ResponseTarget::Configuration(ResponseConfiguration::Effect { effect, .. }) => Some(effect), + _ => None, }; - // Nothing the request carries as a number is NaN or infinite: the axis endpoints and every - // override value are finite before any host looks at them, so a host never prepares an owner - // for a request whose axis or values cannot be meant. + if effect.is_some_and(|effect| effect.is_empty()) { + return false; + } + let effect = effect.is_some(); let finite = value.minimum_hz.is_finite() && value.maximum_hz.is_finite() && value.overrides.iter().all(|row| row.value.is_finite()); - // The bypass is the effect-wide switch: lane `both`, value exactly 0 or 1. - let overrides_ok = value.overrides.iter().all(|row| { - let effect = matches!( - value.target, - ResponseTarget::Configuration(ResponseConfiguration::Effect { .. }) - ); + let bypass_ok = value.overrides.iter().all(|row| { !(effect && row.control == RESPONSE_BYPASS_CONTROL) || (row.lane == ParameterChannel::Both && (row.value.to_bits() == 0.0_f32.to_bits() || row.value.to_bits() == 1.0_f32.to_bits())) }); - target_ok && overrides_ok && finite + finite && bypass_ok } /// A reply's own rules, shared by both directions. @@ -304,7 +320,9 @@ pub(crate) fn write_response_request( sink: &mut dyn Sink, value: &ResponseGetRequest, ) -> Result<(), EncodeError> { - if !response_request_is_valid(sink.limits(), value) { + if !response_request_shape_is_valid(sink.limits(), value) + || !response_request_values_are_valid(value) + { return Err(EncodeError::InvalidField); } let configuration = matches!(value.target, ResponseTarget::Configuration(_)); @@ -571,7 +589,7 @@ fn decode_response_request( maximum_automation_spans: optional_u32(response_request::MAXIMUM_AUTOMATION_SPANS)?, maximum_reply_bytes: optional_u32(response_request::MAXIMUM_REPLY_BYTES)?, }; - if !response_request_is_valid(limits, &value) { + if !response_request_shape_is_valid(limits, &value) { return Err(DecodeError::InvalidField); } Ok(value) diff --git a/crates/protocol/src/message_wire/tests.rs b/crates/protocol/src/message_wire/tests.rs index 01a4b9079..e328a53f8 100644 --- a/crates/protocol/src/message_wire/tests.rs +++ b/crates/protocol/src/message_wire/tests.rs @@ -3160,9 +3160,9 @@ fn response_get_round_trips_every_target_reply_and_optional_field() { /// Gate 1: each shape rule of the request is refused as `INVALID_FIELD` on decode, and on encode /// where the typed value can say it: two targets, no target, a strip with `sections` or an -/// override, a strip ID that is not a stable ID, a bypass override on one lane or with a value -/// that is not 0 or 1, a NaN or infinite axis endpoint or override value, and a code outside its table (grid, channels, lane, kind, quality, link -/// mode). +/// override, a strip ID that is not a stable ID, and a code outside its table (grid, channels, +/// lane, kind, quality, link mode). The value rules are +/// `response_get_request_value_rules_are_refused_on_encode_only`. /// /// Red if the codec admits a request the provider could not read, or only one direction checks. #[test] @@ -3176,7 +3176,7 @@ fn response_get_request_shape_rules_are_refused_both_ways() { type Typed = fn(&mut ResponseGetRequest); type Wire = fn(&mut Vec); // (name, base, typed mutation if representable, wire edit) - let rules: [(&str, bool, Option, Wire); 17] = [ + let rules: [(&str, bool, Option, Wire); 12] = [ ("no target", true, None, |f| remove(f, 1)), ("two targets", true, None, |f| { insert(f, leaf(3, 9, b"vocal")) @@ -3219,36 +3219,6 @@ fn response_get_request_shape_rules_are_refused_both_ways() { Some(|r| r.target = ResponseTarget::Strip("Not A Stable ID".to_owned())), |f| set(f, 3, b"Not A Stable ID"), ), - ( - "a bypass override on the left lane", - true, - Some(|r| r.overrides[0].lane = ParameterChannel::Left), - |f| set(nth(f, 10, 0), 2, &[1]), - ), - ( - "a bypass value of one half", - true, - Some(|r| r.overrides[0].value = 0.5), - |f| set(nth(f, 10, 0), 3, &0.5_f32.to_le_bytes()), - ), - ( - "a NaN minimum frequency", - true, - Some(|r| r.minimum_hz = f32::NAN), - |f| set(f, 6, &f32::NAN.to_le_bytes()), - ), - ( - "an infinite maximum frequency", - true, - Some(|r| r.maximum_hz = f32::INFINITY), - |f| set(f, 7, &f32::INFINITY.to_le_bytes()), - ), - ( - "a NaN override value", - true, - Some(|r| r.overrides[1].value = f32::NAN), - |f| set(nth(f, 10, 1), 3, &f32::NAN.to_le_bytes()), - ), ("a grid code outside the table", true, None, |f| { set(f, 4, &[3]) }), @@ -3307,6 +3277,82 @@ fn response_get_request_shape_rules_are_refused_both_ways() { ); } +/// #1623: the value rules of a request -- an empty effect ID, a NaN or infinite axis endpoint or +/// override value, and a bypass override on one lane or with a value that is not 0 or 1 -- are +/// refused on encode, and decoded as sent: the core refuses each with its own typed code +/// (host-core's `response_frame` and host-web's `control_parity` tests), which a decode refusal +/// would replace with the generic `protocol.failure`. +/// +/// Red if the decoder refuses one again (the typed refusal becomes unreachable), or the encoder +/// stops refusing one. +#[test] +fn response_get_request_value_rules_are_refused_on_encode_only() { + let codec = ProtocolCodec::default(); + let effect = full_effect_request(); + let effect_bytes = encode_response_request(&codec, &effect); + + type Typed = fn(&mut ResponseGetRequest); + type Wire = fn(&mut Vec); + let rules: [(&str, Typed, Wire); 6] = [ + ( + "an empty effect ID", + |r| { + if let ResponseTarget::Configuration(ResponseConfiguration::Effect { + effect, .. + }) = &mut r.target + { + effect.clear(); + } + }, + |f| set(nth(f, 1, 0), 2, b""), + ), + ( + "a bypass override on the left lane", + |r| r.overrides[0].lane = ParameterChannel::Left, + |f| set(nth(f, 10, 0), 2, &[1]), + ), + ( + "a bypass value of one half", + |r| r.overrides[0].value = 0.5, + |f| set(nth(f, 10, 0), 3, &0.5_f32.to_le_bytes()), + ), + ( + "a NaN minimum frequency", + |r| r.minimum_hz = f32::NAN, + |f| set(f, 6, &f32::NAN.to_le_bytes()), + ), + ( + "an infinite maximum frequency", + |r| r.maximum_hz = f32::INFINITY, + |f| set(f, 7, &f32::INFINITY.to_le_bytes()), + ), + ( + "a NaN override value", + |r| r.overrides[1].value = f32::NAN, + |f| set(nth(f, 10, 1), 3, &f32::NAN.to_le_bytes()), + ), + ]; + for (name, typed, wire) in rules { + let mut request = effect.clone(); + typed(&mut request); + assert_eq!( + codec.encoded_response_request_len(&request), + Err(EncodeError::InvalidField), + "{name}: encode" + ); + let (patched, count) = edited(&effect_bytes, wire); + let decoded = codec + .decode_response_request(&patched, count) + .unwrap_or_else(|error| panic!("{name}: decode refused it: {error:?}")); + // `Debug` compares a NaN with itself, where `PartialEq` would not. + assert_eq!( + format!("{decoded:?}"), + format!("{request:?}"), + "{name}: decoded as sent" + ); + } +} + /// Gate 1: each shape rule of the reply is refused on encode and on decode: a curve reply with /// neither side, a side of the wrong length, section curves without their side's total, section /// curves of the wrong length, a lone tag with any curve field, and a curve reply without its diff --git a/docs/CONTROL_PROTOCOL_REGISTRY.md b/docs/CONTROL_PROTOCOL_REGISTRY.md index 832011314..930b9db72 100644 --- a/docs/CONTROL_PROTOCOL_REGISTRY.md +++ b/docs/CONTROL_PROTOCOL_REGISTRY.md @@ -64,13 +64,13 @@ A control row's own rules: all numbers are finite and `minimum <= maximum`; choi `RESPONSE_GET` (`000e`, capability flag bit 14) asks for the frequency response of one native effect or of the input filters, for parameters the caller chooses. It is a read: it writes no session, revision, catalog or cell, it is sent with revision `Any`, and its reply is cached for exact replay like every reply. -**Request** (`ResponseRequest`): exactly one target of `1:configuration MESSAGE O`, `2:address MESSAGE O` (a `ControlAddress` with `control` and `lane` absent) and `3:strip-id UTF8 O` (a stable ID); `4:grid U8 R` (linear `1`, logarithmic `2`), `5:points U32 R`, `6:minimum-hz F32 R`, `7:maximum-hz F32 R`, `8:channels U8 R` (left `1`, right `2`, both `3`), `9:sections BOOL R`, `10:override MESSAGE O*`, `11:configuration-id U64 O`, `12:sample-rate-hz U32 O`, `13:quantum-frames U32 O`, `14:maximum-prepared-bytes U64 O`, `15:maximum-state-bytes U64 O`, `16:maximum-scratch-bytes U64 O`, `17:maximum-automation-spans U32 O`, `18:maximum-reply-bytes U32 O`. A configuration is `1:kind U8 R` (effect `1`, input filters `2`), `2:effect UTF8 O`, `3:quality U8 O` (draft `1`, normal `2`, high `3`), `4:link-mode U8 O` (dual mono `1`, maximum `2`, average `3`); fields 2 to 4 are present exactly for kind `1`. An override is `1:control U32 R`, `2:lane U8 R` (left `1`, right `2`, both `3`), `3:value F32 R`. The codec refuses with `INVALID_FIELD`: not exactly one target, a code outside its table, a `strip-id` that is not a stable ID, for a strip target `sections` true or any override, and for an effect configuration a bypass override (control `0`) that is not lane `both` with a value of exactly `0` or `1`. +**Request** (`ResponseRequest`): exactly one target of `1:configuration MESSAGE O`, `2:address MESSAGE O` (a `ControlAddress` with `control` and `lane` absent) and `3:strip-id UTF8 O` (a stable ID); `4:grid U8 R` (linear `1`, logarithmic `2`), `5:points U32 R`, `6:minimum-hz F32 R`, `7:maximum-hz F32 R`, `8:channels U8 R` (left `1`, right `2`, both `3`), `9:sections BOOL R`, `10:override MESSAGE O*`, `11:configuration-id U64 O`, `12:sample-rate-hz U32 O`, `13:quantum-frames U32 O`, `14:maximum-prepared-bytes U64 O`, `15:maximum-state-bytes U64 O`, `16:maximum-scratch-bytes U64 O`, `17:maximum-automation-spans U32 O`, `18:maximum-reply-bytes U32 O`. A configuration is `1:kind U8 R` (effect `1`, input filters `2`), `2:effect UTF8 O`, `3:quality U8 O` (draft `1`, normal `2`, high `3`), `4:link-mode U8 O` (dual mono `1`, maximum `2`, average `3`); fields 2 to 4 are present exactly for kind `1`. An override is `1:control U32 R`, `2:lane U8 R` (left `1`, right `2`, both `3`), `3:value F32 R`. The codec refuses with `INVALID_FIELD`, on encode and on decode: not exactly one target, a code outside its table, a `strip-id` that is not a stable ID, and for a strip target `sections` true or any override. The encoder also refuses an empty effect ID, a NaN or infinite endpoint or override value, and for an effect configuration a bypass override (control `0`) that is not lane `both` with a value of exactly `0` or `1`; the decoder reads these as sent, so the core refuses each with its typed code below (#1623): `response.effect.unknown`, `response.grid`, `response.override.value`, `response.override.lane` and `response.override.bypass`. **Overrides** follow the lane rule of decision 17. For an effect, control `0` is the effect-wide bypass; any other control is a descriptor parameter ID. For the input filters, control `3` is `hpf_hz` and `4` is `lpf_hz` (the builtin parameter IDs); an absent lane takes the row's default. `both` on a per-lane parameter sets both lanes; `left` or `right` on a shared parameter is refused. A (control, lane) pair given twice, or `both` beside `left` or `right` of the same control, is a duplicate. At most 256 overrides. **Result** (`ResponseResult`), all fields optional on the wire and shaped as one of two forms. The tag form is `1:request-tag U64 O` alone (the strip target, served by #1578). The curve form is `2:sample-rate-hz U32`, `3:configuration-id U64`, `4:floor-db F32`, `5:bypass BOOL`, `6:section-count U32`, `7:enabled-left U32`, `8:enabled-right U32`, `9:retained-bytes U64`, `10:frequencies-hz PACKED_F32`, with at least one of `11:left-db PACKED_F32` and `12:right-db PACKED_F32`, and `13:sections-left-db`, `14:sections-right-db` (`PACKED_F32`, section-major) each present only with its side's total. Fields 11 and 12 have the length of field 10 (at least 2 points); fields 13 and 14 have `section-count` times that length. The enabled masks hold bit `min(index, 31)` per enabled section. All values are finite. The codec refuses every other shape, on encode and on decode. `protocol::response_result_encoded_len(points, channels, section_count, sections)` is the exact frame length of a curve reply. -**Refusals** carry one error diagnostic whose `code` is the table below. Overrides, owner and axis codes come from one table in `host_core` (`ResponseQueryError::refusal`). A request that is wrong in several ways is refused with the core's refusal first, in the core's order: rate, points, override count, axis, then the owner's. +**Refusals** carry one error diagnostic whose `code` is the table below. Overrides, owner and axis codes come from one table in `host_core` (`ResponseQueryError::refusal`). A request that is wrong in several ways is refused with the core's refusal first, in the core's order: rate, points, override count, axis, override values, then the owner's. | Status | Code | Cause | | --- | --- | --- | @@ -79,13 +79,14 @@ A control row's own rules: all numbers are finite and `minimum <= maximum`; choi | `INVALID_FIELD` | `response.rate` | a rate or quantum that is not a launch value (a session endpoint cannot reach it) | | `INVALID_FIELD` | `response.rate.session` | `sample-rate-hz` or `quantum-frames` present on a session endpoint | | `INVALID_FIELD` | `response.override.control` | a control that is no parameter of the target | -| `INVALID_FIELD` | `response.override.bypass` | a bypass value other than exactly `0` or `1` (a typed provider caller; the wire codec refuses it earlier, without a code) | -| `INVALID_FIELD` | `response.override.lane` | a lane on a shared parameter | +| `INVALID_FIELD` | `response.override.value` | an override value (or input-filter cutoff) that is NaN or infinite | +| `INVALID_FIELD` | `response.override.bypass` | a bypass value other than exactly `0` or `1` | +| `INVALID_FIELD` | `response.override.lane` | a lane on a shared parameter, or a bypass override on `left` or `right` | | `INVALID_FIELD` | `response.override.duplicate` | a (control, lane) pair twice, or `both` beside a lane | | `LIMIT_EXCEEDED` | `response.override.count` | more than 256 overrides | | `LIMIT_EXCEEDED` | `response.limit` | the owner cannot be prepared within the byte, state, scratch or span bounds | | `LIMIT_EXCEEDED` | `response.reply_too_large` | the reply frame would exceed the smaller of the endpoint's frame limit, the replay cache's response bound and `maximum-reply-bytes`; refused before any curve is computed | -| `NOT_FOUND` | `response.effect.unknown` | no such native effect | +| `NOT_FOUND` | `response.effect.unknown` | no such native effect, an empty effect ID included | | `UNAVAILABLE` | `response.effect.unsupported` | the effect has no response | | `UNAVAILABLE` | `response.target.unavailable` | an address or strip target (#1575, #1578) | | `VALIDATION_FAILED` | the owner's code | the owner refused the configuration | diff --git a/hosts/host-web/tests/control_parity.rs b/hosts/host-web/tests/control_parity.rs index 568f61100..736621aa6 100644 --- a/hosts/host-web/tests/control_parity.rs +++ b/hosts/host-web/tests/control_parity.rs @@ -1277,6 +1277,11 @@ impl CAbi { ) .expect("command frame"); bytes.truncate(length); + self.send_raw(&bytes) + } + + /// Submit one command frame as given, and answer as [`Self::send_status`] does. + fn send_raw(&mut self, bytes: &[u8]) -> (u32, Vec) { let mut response = vec![0_u8; 4_096]; let mut out = BytesOut { struct_size: BYTES_OUT_SIZE, @@ -2020,6 +2025,133 @@ fn a_stale_effect_row_edit_is_refused_alike_on_both_hosts() { } } +/// #1623 (root ruling, thick core): a `RESPONSE_GET` frame whose value the Rust encoder refuses +/// -- a NaN or infinite axis endpoint, a NaN override value, a bypass value between 0 and 1 -- is +/// decoded as sent and refused by the core on both hosts, with one result, the same reply bytes +/// and the same typed code (`response.grid`, `response.override.value`, +/// `response.override.bypass`), and changes nothing: the control rows and the document are the +/// bytes they were on each host. +/// +/// Test value: red if either host's codec refuses the value again (the code becomes the generic +/// `protocol.failure`), if one host checks a value the other does not, or if a refusal changes +/// state. +#[test] +fn a_response_request_with_a_bad_value_is_refused_alike_on_both_hosts() { + let request = protocol::ResponseGetRequest { + target: protocol::ResponseTarget::Configuration(protocol::ResponseConfiguration::Effect { + effect: "miso.parametric-eq".to_owned(), + quality: session::EffectQuality::Normal, + link_mode: session::LinkMode::DualMono, + }), + grid: protocol::ResponseGrid::Logarithmic, + points: 64, + minimum_hz: 20.0, + maximum_hz: 20_000.0, + channels: protocol::ParameterChannel::Both, + sections: false, + overrides: vec![ + protocol::ResponseOverride { + control: protocol::RESPONSE_BYPASS_CONTROL, + lane: protocol::ParameterChannel::Both, + value: 1.0, + }, + protocol::ResponseOverride { + control: 4, + lane: protocol::ParameterChannel::Both, + value: 3.25, + }, + ], + configuration_id: None, + sample_rate_hz: None, + quantum_frames: None, + maximum_prepared_bytes: None, + maximum_state_bytes: None, + maximum_scratch_bytes: None, + maximum_automation_spans: None, + maximum_reply_bytes: None, + }; + let model = model_at(48_000); + let mut browser = Rig::new(&model); + let mut c_abi = CAbi::new(&model); + // One frame, sent to both hosts, with a request ID past both legs' counters (the C ABI leg + // pages its control dumps in more frames), which the C ABI leg's counter then takes: each host + // sees ascending IDs, and both see the same frame. + let next_frame = |browser: &mut Rig, c_abi: &mut CAbi| loop { + let frame = browser.encode(CommandPayload::ResponseGet(&request), ExpectedRevision::Any); + let id = u64::from_le_bytes(frame[24..32].try_into().expect("request ID")); + if id > c_abi.request { + c_abi.request = id; + break frame; + } + }; + let frame = next_frame(&mut browser, &mut c_abi); + // The unpatched request is answered alike. + let (b, c) = (browser.send_frame(&frame), c_abi.send_raw(&frame)); + assert_eq!(b, c, "one answer to the unpatched request"); + assert_eq!(b.0, C_RESULT_OK); + let before_b = dump(|payload, expected| browser.command(payload, expected)); + let before_c = dump(|payload, expected| c_abi.send(payload, expected)); + assert_eq!(before_b, before_c); + let patched = |frame: Vec, from: f32, to: f32| { + let from = from.to_le_bytes(); + let at: Vec = frame + .windows(4) + .enumerate() + .filter(|(_, window)| *window == from) + .map(|(at, _)| at) + .collect(); + assert_eq!(at.len(), 1, "the value is in the frame exactly once"); + let mut bytes = frame; + bytes[at[0]..at[0] + 4].copy_from_slice(&to.to_le_bytes()); + bytes + }; + for (name, from, to, code) in [ + ("a NaN minimum", 20.0, f32::NAN, "response.grid"), + ( + "an infinite maximum", + 20_000.0, + f32::INFINITY, + "response.grid", + ), + ( + "a NaN override value", + 3.25, + f32::NAN, + "response.override.value", + ), + ("a bypass of one half", 1.0, 0.5, "response.override.bypass"), + ] { + let bytes = patched(next_frame(&mut browser, &mut c_abi), from, to); + let (b, c) = (browser.send_frame(&bytes), c_abi.send_raw(&bytes)); + assert_eq!(b, c, "{name}: one answer on both hosts"); + assert_eq!(b.0, C_RESULT_OK, "{name}: a refusal is a non-OK frame"); + let mut fields = [0_u16; 1024]; + let DecodedTypedResponseFrame::NonOk { header, payload } = ProtocolCodec::default() + .decode_typed_response(&b.1, &mut DecodeScratch::new(&mut fields)) + .expect("a typed response") + else { + panic!("{name}: expected a refusal"); + }; + assert_eq!(header.status, protocol::StatusCode::InvalidField, "{name}"); + let codes: Vec<_> = payload + .diagnostics + .iter() + .map(|d| d.code.as_str()) + .collect(); + assert_eq!(codes, [code], "{name}"); + assert_eq!( + dump(|payload, expected| browser.command(payload, expected)), + before_b, + "{name}: the browser host is unchanged" + ); + assert_eq!( + dump(|payload, expected| c_abi.send(payload, expected)), + before_c, + "{name}: the C ABI is unchanged" + ); + } +} + /// Block the bypass of `a_live_bypass_crossfades_alike_on_both_hosts` is submitted before. const BYPASS_BLOCK: u64 = 6; diff --git a/sdk/src/core/response-frame.ts b/sdk/src/core/response-frame.ts index 17c22155a..1ecc29628 100644 --- a/sdk/src/core/response-frame.ts +++ b/sdk/src/core/response-frame.ts @@ -182,8 +182,13 @@ function putU64(row: Row, value: bigint, what: string): Uint8Ar return field(row.id, row.wire, row.mandatory, u64Bytes(requireU64(value, what))); } -/** An `f32` field: any number has a spelling (`NaN` and the infinities too); the core refuses it. */ -function putF32(row: Row, value: number): Uint8Array { +/** + * An `f32` field: any number has a spelling (`NaN` and the infinities too), and the core refuses + * the ones it does not take. A value that is not a number has none: it is a `MisoUsageError`, + * never coerced (`"20"` is not 20, and `null` is not 0). + */ +function putF32(row: Row, value: number, what: string): Uint8Array { + if (typeof value !== "number") throw new MisoUsageError(`${what} must be a number`); return field(row.id, row.wire, row.mandatory, f32Bytes(value)); } @@ -209,7 +214,7 @@ function overrideMessage(row: ResponseFrameOverride): Uint8Array { return messageBody([ putU32(OVERRIDE.control, row.control, "a control ID"), putU8(OVERRIDE.lane, code(LANE_CODES, row.lane, "the override lane")), - putF32(OVERRIDE.value, row.value), + putF32(OVERRIDE.value, row.value, "an override value"), ]); } @@ -224,8 +229,8 @@ export function encodeResponseRequest(request: ResponseFrameRequest): Uint8Array putMessage(REQUEST.configuration, configurationMessage(request.target)), putU8(REQUEST.grid, code(GRID_CODES, request.grid, "the grid")), putU32(REQUEST.points, request.points, "points"), - putF32(REQUEST.minimumHz, request.minimumHz), - putF32(REQUEST.maximumHz, request.maximumHz), + putF32(REQUEST.minimumHz, request.minimumHz, "minimumHz"), + putF32(REQUEST.maximumHz, request.maximumHz, "maximumHz"), putU8(REQUEST.channels, code(LANE_CODES, request.channels, "the channels")), putBool(REQUEST.sections, request.sections), ]; diff --git a/sdk/src/core/response.ts b/sdk/src/core/response.ts index ffe9482d0..e471a7c3c 100644 --- a/sdk/src/core/response.ts +++ b/sdk/src/core/response.ts @@ -176,9 +176,14 @@ function wantsSections(fields: ResponsePreviewQuery["fields"]): boolean { throw new MisoUsageError("fields must be total or totalAndSections"); } -/** An input-filter cutoff as the `f32` the frame carries; a cutoff left out is 0, filter off. */ +/** + * An input-filter cutoff as the `f32` the frame carries; a cutoff left out is 0, filter off. A + * value that is not a number (`null`, a string) is passed on as given, for the frame encoder to + * refuse with a `MisoUsageError` (#1623), never coerced. + */ function cutoff(value: number | undefined): number { - return Math.fround(value ?? 0); + if (value === undefined) return 0; + return typeof value === "number" ? Math.fround(value) : value; } function isInputFilters( @@ -290,8 +295,8 @@ export class ResponsePreviewModule { ...this.#target(configuration), grid: request.grid.kind, points: request.grid.points, - minimumHz: Math.fround(request.grid.minimumHz), - maximumHz: Math.fround(request.grid.maximumHz), + minimumHz: request.grid.minimumHz, + maximumHz: request.grid.maximumHz, channels: request.channels ?? "both", sections, configurationId: request.configurationId, @@ -390,9 +395,17 @@ export class ResponsePreviewModule { const capability = isInputFilters(configuration) ? RESPONSE_CAPABILITIES.find((candidate) => candidate.owner === "builtins") : RESPONSE_CAPABILITIES.find((candidate) => candidate.owner === "effect" && candidate.target === configuration.effectId); + // The reply names no more sections than the generated capability declares: a reply that does + // is the engine and the SDK out of step, never a request the caller can change. + const declared = capability?.sections ?? []; + if (sectionCount > declared.length) { + throw new MisoEngineError("the response reply has more sections than the target declares", { + phase: "output", code: "internal", result: resultValue("internal"), + }); + } const returned: ResponsePreviewSection[] = []; for (let index = 0; sections && index < sectionCount; index += 1) { - const declaration = capability?.sections[index]; + const declaration = declared[index]; if (declaration === undefined) continue; const leftDb = curves.sectionsLeftDb?.slice(index * points, (index + 1) * points); const rightDb = curves.sectionsRightDb?.slice(index * points, (index + 1) * points); diff --git a/sdk/test/response-evals.mjs b/sdk/test/response-evals.mjs index 046779f4a..f9b977c22 100644 --- a/sdk/test/response-evals.mjs +++ b/sdk/test/response-evals.mjs @@ -308,23 +308,23 @@ test("candidate Wasm: a grid, a rate, a quality and a value the SDK used to refu const rows = [ ["a logarithmic axis from zero", { grid: grid({ minimumHz: 0 }) }, "response.grid", "invalidArgument"], ["a negative logarithmic minimum", { grid: grid({ minimumHz: -5 }) }, "response.grid", "invalidArgument"], - ["a NaN endpoint", { grid: grid({ maximumHz: Number.NaN }) }, "protocol.failure", "invalidArgument"], - ["an infinite endpoint", { grid: grid({ kind: "linear", maximumHz: Number.POSITIVE_INFINITY }) }, "protocol.failure", "invalidArgument"], + ["a NaN endpoint", { grid: grid({ maximumHz: Number.NaN }) }, "response.grid", "invalidArgument"], + ["an infinite endpoint", { grid: grid({ kind: "linear", maximumHz: Number.POSITIVE_INFINITY }) }, "response.grid", "invalidArgument"], ["one point", { grid: grid({ points: 1 }) }, "response.points", "invalidArgument"], ["a sample rate that is no launch rate", { sampleRateHz: 12_345 }, "response.rate", "invalidArgument"], ["a zero quantum", { quantumFrames: 0 }, "response.rate", "invalidArgument"], ["draft quality", { configuration: eq({}, { quality: "draft" }) }, "effect.quality.unsupported", "invalidArgument"], ["high quality", { configuration: eq({}, { quality: "high" }) }, "effect.quality.unsupported", "invalidArgument"], - ["a non-finite value", { configuration: eq({ "band-1-gain": Number.NaN }) }, "protocol.failure", "invalidArgument"], + ["a non-finite value", { configuration: eq({ "band-1-gain": Number.NaN }) }, "response.override.value", "invalidArgument"], + [ + "a non-finite cutoff", + { configuration: { kind: "inputFilters", left: { hpfHz: Number.POSITIVE_INFINITY }, right: {} } }, + "response.override.value", + "invalidArgument", + ], ["a gain outside its domain", { configuration: eq({ "band-1-gain": 500 }) }, "effect.parameter.initial", "invalidArgument"], ["an effect the engine does not know", { configuration: forged("miso.no-such-effect", {}) }, "response.effect.unknown", "invalidArgument"], ["an effect with no response", { configuration: forged("miso.compressor", {}) }, "response.effect.unsupported", "unsupported"], - [ - "a shared parameter on one lane, of an effect with no response", - { configuration: forged("miso.compressor", { threshold: -20 }, { channel: "left" }) }, - "response.effect.unsupported", - "unsupported", - ], ]; try { for (const [name, change, code, result] of rows) { @@ -335,8 +335,15 @@ test("candidate Wasm: a grid, a rate, a quality and a value the SDK used to refu name, ); } - // The same request with a left-only scalar lane is accepted: the core, not the SDK, says - // which lane an effect takes. + // A number the frame cannot spell is the SDK's usage error, never coerced: "20" is not 20 and + // a null cutoff is not 0 (filter off). + for (const [name, change] of [ + ["a string endpoint", { grid: grid({ minimumHz: "20" }) }], + ["a null cutoff", { configuration: { kind: "inputFilters", left: { hpfHz: null }, right: {} } }], + ]) { + assert.throws(() => preview.query({ ...query, ...change }), MisoUsageError, name); + } + // A left-only value of a per-lane parameter is sent as given and accepted. const left = preview.query({ ...query, configuration: eq({ "band-1-gain": 3 }, { channel: "left" }), @@ -382,6 +389,46 @@ test("candidate Wasm: a lane given twice is refused by the core with the C ABI's assert.equal(accepted.curves.frequenciesHz.length, 8); }); +/** + * #1623 (root ruling, thick core): a frame the Rust encoder would refuse to write -- an empty + * effect ID, a non-finite override value, a bypass row off lane `both` or between 0 and 1 -- is + * decoded as sent, and the core answers it with its own typed code, never `protocol.failure` + * (`crates/host-core/tests/response_frame.rs` and `hosts/host-web/tests/control_parity.rs` hold + * the same codes for both hosts). + */ +test("candidate Wasm: a value the Rust encoder refuses is the core's typed refusal", { + skip: !process.env.MISO_ENGINE_SDK_ARTIFACTS_HEX, +}, async () => { + const asset = await MisoEngineAsset.load(await moduleBytes()); + const instance = await asset.instantiate(); + const frame = (effect, overrides) => encodeResponseRequest({ + requestId: 1n, + target: { kind: "effect", effect, quality: "normal", linkMode: "dual_mono" }, + grid: "linear", + points: 8, + minimumHz: 20, + maximumHz: 20_000, + channels: "both", + sections: false, + overrides, + configurationId: 1n, + sampleRateHz: 48_000, + quantumFrames: 128, + }); + const eq = "miso.parametric-eq"; + for (const [name, effect, overrides, status, code] of [ + ["an empty effect ID", "", [], "not_found", "response.effect.unknown"], + ["a NaN override value", eq, [{ control: 4, lane: "both", value: Number.NaN }], "invalid_field", "response.override.value"], + ["a bypass of one half", eq, [{ control: 0, lane: "both", value: 0.5 }], "invalid_field", "response.override.bypass"], + ["a bypass on one lane", eq, [{ control: 0, lane: "left", value: 1 }], "invalid_field", "response.override.lane"], + ]) { + const reply = submitFrame(instance, frame(effect, overrides)); + assert.equal(reply.ok, false, name); + assert.equal(reply.status, status, name); + assert.deepEqual(reply.diagnostics.map((row) => row.code), [code], name); + } +}); + test("a name or a number with no wire spelling stays a usage error before any byte is made", () => { const request = { requestId: 1n, @@ -407,6 +454,10 @@ test("a name or a number with no wire spelling stays a usage error before any by ["a point count that is no u32", { points: 2 ** 32 }], ["a fractional rate", { sampleRateHz: 48_000.5 }], ["a configuration ID that is not a bigint", { configurationId: 1 }], + ["a minimum that is a string", { minimumHz: "20" }], + ["a maximum that is null", { maximumHz: null }], + ["an override value that is a string", { overrides: [{ control: 3, lane: "left", value: "20" }] }], + ["an override value that is null", { overrides: [{ control: 3, lane: "left", value: null }] }], ]) { assert.throws(() => encodeResponseRequest({ ...request, ...change }), MisoUsageError, name); } diff --git a/tools/console-workload/tests/paired_spans.rs b/tools/console-workload/tests/paired_spans.rs index 2f563b4bb..57cd5a568 100644 --- a/tools/console-workload/tests/paired_spans.rs +++ b/tools/console-workload/tests/paired_spans.rs @@ -1964,14 +1964,17 @@ fn every_launch_effect_applies_a_twin_pair_with_channel_symmetric_validity() { /// and has its own rule: `rack`'s `a_live_slot_sizes_its_spans_by_its_lanes_cells_not_the_capacity` /// and `a_bank_window_below_a_lanes_live_cells_is_refused_at_bind`.) /// -/// A window of exactly `automation_capacity` spans is accepted; one span more (which could stage a -/// twin across the effect's cut-off) or one span fewer (which would drop admitted records) is -/// refused with the typed `ProcessBlockError::AutomationWindow`. +/// A window of exactly `automation_capacity` spans, or of the effect's live parameter cells (the +/// window preparation allocates, #1621 D1), is accepted; one span more than the capacity (which +/// could stage a twin across the effect's cut-off) is refused with the typed +/// `ProcessBlockError::AutomationWindow`. A shorter window is sound for the pair; that it still +/// holds every cell is the lane's bound, `effect-contract`'s `live_window` test +/// `a_window_shorter_than_the_live_cells_is_refused_with_the_count_it_needs`. /// /// Red mutation (issue #1012): make the contract's `check_window` return `Ok(())` -/// unconditionally -> both refusals below fail, for every effect. +/// unconditionally -> the refusal below fails, for every effect. #[test] -fn every_launch_effect_refuses_a_staging_window_that_is_not_its_capacity() { +fn every_launch_effect_refuses_a_staging_window_longer_than_its_capacity() { let registry = registry(); let window = |spans: usize| { vec![ @@ -1996,28 +1999,39 @@ fn every_launch_effect_refuses_a_staging_window_that_is_not_its_capacity() { .into_iter() .find(|mode| descriptor.supported_link_modes.contains(*mode)) .expect("a link mode"); - let Some(request) = prepare_request(descriptor, &values, mode) else { + let Some(mut request) = prepare_request(descriptor, &values, mode) else { continue; }; + // Preparation raises a request below the live cells to them (#1621 D4), so the window it + // allocates always fits; this direct factory call raises it the same way. + let cells = EffectLaneShape::live_parameter_cells(descriptor); + let raised = u32::try_from(cells).expect("cells fit a u32"); + request.limits.maximum_automation_spans_per_block = request + .limits + .maximum_automation_spans_per_block + .max(raised); let scalar = factory.prepare(request).expect("scalar prepare").metadata; let capacity = scalar.automation_capacity as usize; assert!( - capacity > 0, - "{}: a live effect has capacity", + capacity > 0 && cells <= capacity, + "{}: a live effect has capacity for its {cells} live cells, got {capacity}", descriptor.id.as_str() ); - assert_eq!( - EffectProcessBlock::check_automation_window(&window(capacity), &scalar), - Ok(()) - ); - for wrong in [capacity + 1, capacity - 1] { + for accepted in [capacity, cells] { assert_eq!( - EffectProcessBlock::check_automation_window(&window(wrong), &scalar), - Err(ProcessBlockError::AutomationWindow), - "{}: a scalar window of {wrong} spans against capacity {capacity}", + EffectProcessBlock::check_automation_window(&window(accepted), &scalar), + Ok(()), + "{}: a scalar window of {accepted} spans against capacity {capacity}", descriptor.id.as_str() ); } + assert_eq!( + EffectProcessBlock::check_automation_window(&window(capacity + 1), &scalar), + Err(ProcessBlockError::AutomationWindow), + "{}: a scalar window of {} spans against capacity {capacity}", + descriptor.id.as_str(), + capacity + 1 + ); checked.push(descriptor.id.as_str().to_owned()); } assert_eq!(checked.len(), registry.len(), "{checked:?}"); From 10496f15a3857973634a7a24bb167b45a61cd073 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 21:30:29 +0000 Subject: [PATCH 10/12] #1620: the session policy's preflight anchor follows check_caps(estimate, caps) Co-Authored-By: Claude Opus 5.5 --- scripts/check-session-policy.sh | 2 +- scripts/test-session-policy.sh | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/scripts/check-session-policy.sh b/scripts/check-session-policy.sh index e46f9ba40..0dd461cb9 100755 --- a/scripts/check-session-policy.sh +++ b/scripts/check-session-policy.sh @@ -41,7 +41,7 @@ anchor() { printf '%s' "$line" } estimate_line=$(anchor estimate 'let estimate = estimate_session\(session\)') -caps_line=$(anchor caps 'check_caps\(session, estimate, caps\)') +caps_line=$(anchor caps 'check_caps\(estimate, caps\)') validate_line=$(anchor validate 'validate_session\(session\)') canonical_line=$(anchor canonical 'let canonical_json = write_canonical\(session\)') clone_line=$(anchor clone 'let mut normalized = session\.clone\(\)') diff --git a/scripts/test-session-policy.sh b/scripts/test-session-policy.sh index f48d64f19..3e0dbe7f7 100755 --- a/scripts/test-session-policy.sh +++ b/scripts/test-session-policy.sh @@ -12,7 +12,7 @@ make_base() { cat >"$d/crates/session/src/compile.rs" <<'EOF' fn compile(session: (), caps: ()) { let estimate = estimate_session(session); - check_caps(session, estimate, caps); + check_caps(estimate, caps); validate_session(session); let canonical_json = write_canonical(session); let mut normalized = session.clone(); @@ -32,11 +32,11 @@ for kind in engine session toml serde publication estimate; do d="$tmp/$kind"; c estimate) printf 'String::with_capacity(1);\n' >>"$d/crates/session/src/estimate.rs";; esac; red "$kind" "$d"; done for f in crates/engine/Cargo.toml crates/session/Cargo.toml crates/session/src/estimate.rs crates/session/src/compile.rs; do d="$tmp/missing-${f//\//-}"; cp -a "$base" "$d"; rm "$d/$f"; red "missing $f" "$d"; done -for f in 'let estimate = estimate_session(session);' 'check_caps(session, estimate, caps);' 'validate_session(session);' 'let canonical_json = write_canonical(session);' 'let mut normalized = session.clone();'; do d="$tmp/anchor-$RANDOM"; cp -a "$base" "$d"; sed -i "s|$f||" "$d/crates/session/src/compile.rs"; red anchor "$d"; done -d="$tmp/order"; cp -a "$base" "$d"; sed -i '0,/check_caps(session, estimate, caps);/{s//let canonical_json = write_canonical(session);\n check_caps(session, estimate, caps);/}' "$d/crates/session/src/compile.rs"; red order "$d" +for f in 'let estimate = estimate_session(session);' 'check_caps(estimate, caps);' 'validate_session(session);' 'let canonical_json = write_canonical(session);' 'let mut normalized = session.clone();'; do d="$tmp/anchor-$RANDOM"; cp -a "$base" "$d"; sed -i "s|$f||" "$d/crates/session/src/compile.rs"; red anchor "$d"; done +d="$tmp/order"; cp -a "$base" "$d"; sed -i '0,/check_caps(estimate, caps);/{s//let canonical_json = write_canonical(session);\n check_caps(estimate, caps);/}' "$d/crates/session/src/compile.rs"; red order "$d" d="$tmp/duplicate"; cp -a "$base" "$d"; cat >>"$d/crates/session/src/compile.rs" <<'EOF' let estimate = estimate_session(session); - check_caps(session, estimate, caps); + check_caps(estimate, caps); validate_session(session); let canonical_json = write_canonical(session); let mut normalized = session.clone(); @@ -65,7 +65,7 @@ descs=('engine reverse dependency' 'session engine workspace dependency' 'sessio outs=('1:session.workspace = true' '4:engine.workspace = true' '5:json-syntax = { version = "=0.12.5", default-features = false }' '4:toml = "1"' '1:use engine::PlanPublisher;' '1:String::with_capacity(1);') for i in "${!tokens[@]}"; do d="$tmp/direct-$i"; cp -a "$base" "$d"; rgshim "$d" "${tokens[i]}" "${outs[i]}"; red direct "$d" "${descs[i]} scan errored (rg exit 9)" env PATH="$d/bin:$PATH"; done # Plausible correct first rows, malformed/zero rows, and early clone distinguish first-match order. -an=(estimate caps validate canonical clone); pats=('estimate_session' 'check_caps' 'validate_session' 'write_canonical' 'session\.clone'); rows=('2: let estimate = estimate_session(session);' '3: check_caps(session, estimate, caps);' '4: validate_session(session);' '5: let canonical_json = write_canonical(session);' '6: let mut normalized = session.clone();') +an=(estimate caps validate canonical clone); pats=('estimate_session' 'check_caps' 'validate_session' 'write_canonical' 'session\.clone'); rows=('2: let estimate = estimate_session(session);' '3: check_caps(estimate, caps);' '4: validate_session(session);' '5: let canonical_json = write_canonical(session);' '6: let mut normalized = session.clone();') for i in "${!an[@]}"; do d="$tmp/ae-$i"; cp -a "$base" "$d"; rgshim "$d" "${pats[i]}" "${rows[i]}"; red anchor "$d" "${an[i]} scan errored (rg exit 9)" env PATH="$d/bin:$PATH"; done for row in 'x: let estimate = estimate_session(session);' '0: let estimate = estimate_session(session);'; do d="$tmp/al-$RANDOM"; cp -a "$base" "$d"; rgshim "$d" 'estimate_session' "$row" 0; red line "$d" 'estimate anchor line is not a positive decimal' env PATH="$d/bin:$PATH"; done d="$tmp/early-clone"; cp -a "$base" "$d"; sed -i '2i let mut normalized = session.clone();' "$d/crates/session/src/compile.rs"; red early-clone "$d" 'resource preflight/cap ordering changed' From 1fe8b5a29d8706480563efef5ff64336efdac254 Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 21:46:05 +0000 Subject: [PATCH 11/12] Batch 14b fold verification: a later write after adoption is not a second carried value; add_carried's doc names both callers (#1630) Co-Authored-By: Claude Opus 5.5 --- crates/engine/src/realtime/latest_cell.rs | 13 ++-- crates/host-core/tests/successor_swap.rs | 85 ++++++++++++++++++----- 2 files changed, 74 insertions(+), 24 deletions(-) diff --git a/crates/engine/src/realtime/latest_cell.rs b/crates/engine/src/realtime/latest_cell.rs index f1fa71738..debb67193 100644 --- a/crates/engine/src/realtime/latest_cell.rs +++ b/crates/engine/src/realtime/latest_cell.rs @@ -978,8 +978,8 @@ impl LiveValuesSuperseded { /// caller is [`CellReader::read_counted`], which render's drains call: only a cell read writes /// the count. `read_counted` is public, so a reader built over a stamped cell could call it /// elsewhere; the rule rests on who holds the readers, not on a type barrier (#1347 attempt 2, - /// n1). [`LiveValuesSuperseded::add_carried`] is the second render path that writes the count - /// (#1280 D2). + /// n1). [`LiveValuesSuperseded::add_carried`] is the second render path that writes the count, + /// from a plan swap and from a successor's first block drain (#1280 D2). fn add(&self, count: u64) { if count != 0 { let now = self.0.load(core::sync::atomic::Ordering::Relaxed); @@ -990,9 +990,12 @@ impl LiveValuesSuperseded { } } - /// Render, at a plan swap: add `count` writes the retiring plan's cell held and its successor - /// takes over (#1280 D2). The same load, saturating add and store as `add`, and render is still - /// the only writer: the swap runs on the render thread between two blocks. A zero count changes + /// Render: add `count` carried writes (#1280 D2). Two render-thread callers. A plan swap adds + /// the writes the retiring plan's cell held and its successor takes over. A successor's first + /// block drain adds one for each carried value it replaces before any sample renders, through + /// the builtins compiler's `CarryMarks::applied`. The same load, saturating add and store as + /// `add`, and render is still the only writer: the swap runs on the render thread between two + /// blocks, and the drain runs inside the successor's first block. A zero count changes /// nothing. pub fn add_carried(&self, count: u64) { if count != 0 { diff --git a/crates/host-core/tests/successor_swap.rs b/crates/host-core/tests/successor_swap.rs index 7623ea88e..47c173db8 100644 --- a/crates/host-core/tests/successor_swap.rs +++ b/crates/host-core/tests/successor_swap.rs @@ -1148,6 +1148,9 @@ struct DirectRun { /// values its adopting block replaced (#1630 f). The test seam gives each plan its own count; /// a host passes both plans the session's. successor_superseded: u64, + /// The last plan's own count after the final block: the successor's, when `b` is given. A + /// carried value counted again by a later drain shows here and not in `successor_superseded`. + final_superseded: u64, } /// Render `a` for [`SWAP_BLOCK`] blocks of `quantum` frames, then -- when `b` is given -- prepare @@ -1260,6 +1263,7 @@ fn direct_run_ramped( transitions: plan.prepared.plan.bank_collapse_transitions(), carry_superseded, successor_superseded, + final_superseded: plan.handles.live_values_superseded.get(), } } @@ -2053,15 +2057,29 @@ fn a_carried_input_value_the_retarget_replaces_is_counted(backend: Backend) { let a = Session::compile(filtered_session(false)); let written = with_left_trim(WRITTEN_DB); let b = Session::compile(with_muted_track(with_left_trim(MOVED_DB))); - let writes = [LiveWrite { - block: SWAP_BLOCK, - strip: STRIP, - record: builtins_compiler::TrackInputRecord::TrimDb { - lanes: builtins::BuiltinLaneSelector::Left, - db: WRITTEN_DB, - smoothing_samples: 64, + // The second write: a second value to the same cell, two blocks after the adopting block, + // through the successor's normal write path. The carry marks are spent by then, so it is a plain + // write, never a second carried value. + let writes = [ + LiveWrite { + block: SWAP_BLOCK, + strip: STRIP, + record: builtins_compiler::TrackInputRecord::TrimDb { + lanes: builtins::BuiltinLaneSelector::Left, + db: WRITTEN_DB, + smoothing_samples: 64, + }, }, - }]; + LiveWrite { + block: SWAP_BLOCK + 2, + strip: STRIP, + record: builtins_compiler::TrackInputRecord::TrimDb { + lanes: builtins::BuiltinLaneSelector::Left, + db: 0.5, + smoothing_samples: 64, + }, + }, + ]; let feed = Feed::new(SOURCE, 19); let run = |b: &Session, committed: &SessionModel, writes: &[LiveWrite]| { let run = direct_run( @@ -2082,8 +2100,13 @@ fn a_carried_input_value_the_retarget_replaces_is_counted(backend: Backend) { (0, 1), "{backend:?}: the carried value the retarget replaced is counted once" ); - let unwritten = run(&b, &a.model, &[]); + assert_eq!( + replaced.final_superseded, 1, + "{backend:?}: a later write to the cell is not a second carried value" + ); + let unwritten = run(&b, &a.model, &writes[1..]); assert_eq!(unwritten.successor_superseded, 0, "{backend:?}"); + assert_eq!(unwritten.final_superseded, 0, "{backend:?}"); assert_eq!( first_direct_difference(&replaced, &unwritten), None, @@ -2100,6 +2123,7 @@ fn a_carried_input_value_the_retarget_replaces_is_counted(backend: Backend) { (0, 0), "{backend:?}: a carried value nothing replaces is not counted" ); + assert_eq!(kept.final_superseded, 0, "{backend:?}"); } #[test] @@ -3059,6 +3083,8 @@ struct CellRun { carry_superseded: u64, /// The successor's own count after its first block, as [`DirectRun::successor_superseded`]. successor_superseded: u64, + /// The last plan's count after the final block, as [`DirectRun::final_superseded`]. + final_superseded: u64, } /// Render `a` and write `writes` to the plan that renders their block next; with a `swap`, prepare @@ -3139,6 +3165,7 @@ fn cell_run( swap_allocator, carry_superseded, successor_superseded, + final_superseded: plan.live.handles.live_values_superseded.get(), } } @@ -3710,15 +3737,29 @@ fn a_carried_strip_value_the_retarget_replaces_is_counted(backend: Backend, form [MOVED_DB, right], ))); let feed = feed_for_strips(); - let writes = [CellWrite { - block: SWAP_BLOCK, - strip: MOVED, - record: CellRecord::Fader(TrackFaderRecord::FaderDb { - lanes: BuiltinLaneSelector::Left, - db: WRITTEN_DB, - smoothing_samples: 64, - }), - }]; + // The second write: a second value to the same cell, two blocks after the adopting block, + // through the successor's normal write path. The carry marks are spent by then, so it is a plain + // write, never a second carried value. + let writes = [ + CellWrite { + block: SWAP_BLOCK, + strip: MOVED, + record: CellRecord::Fader(TrackFaderRecord::FaderDb { + lanes: BuiltinLaneSelector::Left, + db: WRITTEN_DB, + smoothing_samples: 64, + }), + }, + CellWrite { + block: SWAP_BLOCK + 2, + strip: MOVED, + record: CellRecord::Fader(TrackFaderRecord::FaderDb { + lanes: BuiltinLaneSelector::Left, + db: -4.0, + smoothing_samples: 64, + }), + }, + ]; let run = |to: &Session, committed: &SessionModel, writes: &[CellWrite]| { let run = cell_run( &a, @@ -3744,8 +3785,13 @@ fn a_carried_strip_value_the_retarget_replaces_is_counted(backend: Backend, form (0, 1), "{backend:?} {form:?}: the carried value the retarget replaced is counted once" ); - let unwritten = run(&b, &base, &[]); + assert_eq!( + replaced.final_superseded, 1, + "{backend:?} {form:?}: a later write to the cell is not a second carried value" + ); + let unwritten = run(&b, &base, &writes[1..]); assert_eq!(unwritten.successor_superseded, 0, "{backend:?} {form:?}"); + assert_eq!(unwritten.final_superseded, 0, "{backend:?} {form:?}"); assert_eq!( first_cell_difference(&replaced, &unwritten), None, @@ -3761,6 +3807,7 @@ fn a_carried_strip_value_the_retarget_replaces_is_counted(backend: Backend, form (0, 0), "{backend:?} {form:?}: a carried value nothing replaces is not counted" ); + assert_eq!(kept.final_superseded, 0, "{backend:?} {form:?}"); } #[test] From 538537d41a4fd5ccee6431acd3c93c690682ecca Mon Sep 17 00:00:00 2001 From: BL Date: Thu, 15 Oct 2026 22:16:35 +0000 Subject: [PATCH 12/12] #1630 carry marks: regenerate the graph and builtins fixtures and re-pin the audit's strip preparation bytes (1,136 -> 1,160) Co-Authored-By: Claude Opus 5.5 --- fixtures/builtins/v1/MANIFEST.tsv | 2 +- fixtures/builtins/v1/resources.jsonl | 18 +++++++++--------- fixtures/graph/MANIFEST.tsv | 2 +- fixtures/graph/v1/direct-route.resources.json | 2 +- tools/audit/src/builtins_graph.rs | 5 ++++- tools/audit/src/fixture_builtins.rs | 14 ++++++++++++-- 6 files changed, 28 insertions(+), 15 deletions(-) diff --git a/fixtures/builtins/v1/MANIFEST.tsv b/fixtures/builtins/v1/MANIFEST.tsv index a4dc52465..89c4ba082 100644 --- a/fixtures/builtins/v1/MANIFEST.tsv +++ b/fixtures/builtins/v1/MANIFEST.tsv @@ -48,4 +48,4 @@ pcm/partition.f32le 64 b651d0ec619d8c2947372ad9ceba33799b09f0ddbb59c54a9ebd1bc7e pcm/polarity-gain.f32le 64 31326535e1d116fde7e4ee5066f41a14c8f625dbc1fa4530b69eca1a4d157f1a pcm/reset.f32le 96 76795b3b6044cdde0fcc3662f26c927544b8d3f9f676ec1d706212e7da40b7f9 reference/filter-response.csv 406312 1889e345bdbfb799275bfac8fa995cd07f076003a870e5a48192bc212e4887a7 -resources.jsonl 2371 7a9c9284ce38dcd18a6fea18841c6f401129d5bb0f3066f959464002a5d8ece6 +resources.jsonl 2371 9074570ee3b0881696491cbf1c53cc14a921c02e32187e1b4f6677de389686e3 diff --git a/fixtures/builtins/v1/resources.jsonl b/fixtures/builtins/v1/resources.jsonl index ce542a8e7..805971002 100644 --- a/fixtures/builtins/v1/resources.jsonl +++ b/fixtures/builtins/v1/resources.jsonl @@ -1,9 +1,9 @@ -{"tracks":1,"meters":0,"queue_capacity":1,"meter_items":0,"engine_owned_processor_payload_bytes":2207,"engine_owned_meter_payload_bytes":0,"engine_owned_retained_payload_bytes":2207,"maximum_single_allocation_bytes":1136,"retained_allocation_count":16} -{"tracks":1,"meters":1,"queue_capacity":1,"meter_items":2,"engine_owned_processor_payload_bytes":2207,"engine_owned_meter_payload_bytes":1174,"engine_owned_retained_payload_bytes":3381,"maximum_single_allocation_bytes":1136,"retained_allocation_count":31} -{"tracks":1,"meters":7,"queue_capacity":4,"meter_items":35,"engine_owned_processor_payload_bytes":2207,"engine_owned_meter_payload_bytes":11578,"engine_owned_retained_payload_bytes":13785,"maximum_single_allocation_bytes":1136,"retained_allocation_count":85} -{"tracks":4,"meters":0,"queue_capacity":1,"meter_items":0,"engine_owned_processor_payload_bytes":8894,"engine_owned_meter_payload_bytes":0,"engine_owned_retained_payload_bytes":8894,"maximum_single_allocation_bytes":4544,"retained_allocation_count":43} -{"tracks":4,"meters":1,"queue_capacity":1,"meter_items":2,"engine_owned_processor_payload_bytes":8894,"engine_owned_meter_payload_bytes":1186,"engine_owned_retained_payload_bytes":10080,"maximum_single_allocation_bytes":4544,"retained_allocation_count":58} -{"tracks":4,"meters":7,"queue_capacity":4,"meter_items":35,"engine_owned_processor_payload_bytes":8894,"engine_owned_meter_payload_bytes":11662,"engine_owned_retained_payload_bytes":20556,"maximum_single_allocation_bytes":4544,"retained_allocation_count":112} -{"tracks":65537,"meters":0,"queue_capacity":1,"meter_items":0,"engine_owned_processor_payload_bytes":147948095,"engine_owned_meter_payload_bytes":0,"engine_owned_retained_payload_bytes":147948095,"maximum_single_allocation_bytes":74450032,"retained_allocation_count":589840} -{"tracks":65537,"meters":1,"queue_capacity":1,"meter_items":2,"engine_owned_processor_payload_bytes":147948095,"engine_owned_meter_payload_bytes":1186,"engine_owned_retained_payload_bytes":147949281,"maximum_single_allocation_bytes":74450032,"retained_allocation_count":589855} -{"tracks":65537,"meters":7,"queue_capacity":4,"meter_items":35,"engine_owned_processor_payload_bytes":147948095,"engine_owned_meter_payload_bytes":11662,"engine_owned_retained_payload_bytes":147959757,"maximum_single_allocation_bytes":74450032,"retained_allocation_count":589909} +{"tracks":1,"meters":0,"queue_capacity":1,"meter_items":0,"engine_owned_processor_payload_bytes":2231,"engine_owned_meter_payload_bytes":0,"engine_owned_retained_payload_bytes":2231,"maximum_single_allocation_bytes":1160,"retained_allocation_count":16} +{"tracks":1,"meters":1,"queue_capacity":1,"meter_items":2,"engine_owned_processor_payload_bytes":2231,"engine_owned_meter_payload_bytes":1174,"engine_owned_retained_payload_bytes":3405,"maximum_single_allocation_bytes":1160,"retained_allocation_count":31} +{"tracks":1,"meters":7,"queue_capacity":4,"meter_items":35,"engine_owned_processor_payload_bytes":2231,"engine_owned_meter_payload_bytes":11578,"engine_owned_retained_payload_bytes":13809,"maximum_single_allocation_bytes":1160,"retained_allocation_count":85} +{"tracks":4,"meters":0,"queue_capacity":1,"meter_items":0,"engine_owned_processor_payload_bytes":8990,"engine_owned_meter_payload_bytes":0,"engine_owned_retained_payload_bytes":8990,"maximum_single_allocation_bytes":4640,"retained_allocation_count":43} +{"tracks":4,"meters":1,"queue_capacity":1,"meter_items":2,"engine_owned_processor_payload_bytes":8990,"engine_owned_meter_payload_bytes":1186,"engine_owned_retained_payload_bytes":10176,"maximum_single_allocation_bytes":4640,"retained_allocation_count":58} +{"tracks":4,"meters":7,"queue_capacity":4,"meter_items":35,"engine_owned_processor_payload_bytes":8990,"engine_owned_meter_payload_bytes":11662,"engine_owned_retained_payload_bytes":20652,"maximum_single_allocation_bytes":4640,"retained_allocation_count":112} +{"tracks":65537,"meters":0,"queue_capacity":1,"meter_items":0,"engine_owned_processor_payload_bytes":149520983,"engine_owned_meter_payload_bytes":0,"engine_owned_retained_payload_bytes":149520983,"maximum_single_allocation_bytes":76022920,"retained_allocation_count":589840} +{"tracks":65537,"meters":1,"queue_capacity":1,"meter_items":2,"engine_owned_processor_payload_bytes":149520983,"engine_owned_meter_payload_bytes":1186,"engine_owned_retained_payload_bytes":149522169,"maximum_single_allocation_bytes":76022920,"retained_allocation_count":589855} +{"tracks":65537,"meters":7,"queue_capacity":4,"meter_items":35,"engine_owned_processor_payload_bytes":149520983,"engine_owned_meter_payload_bytes":11662,"engine_owned_retained_payload_bytes":149532645,"maximum_single_allocation_bytes":76022920,"retained_allocation_count":589909} diff --git a/fixtures/graph/MANIFEST.tsv b/fixtures/graph/MANIFEST.tsv index 904f4ec34..3b1f5a9e5 100644 --- a/fixtures/graph/MANIFEST.tsv +++ b/fixtures/graph/MANIFEST.tsv @@ -2,7 +2,7 @@ path length sha256 v1/direct-route.canonical.txt 3742 442532b37fd70ef2bcc645681b7168128d0d8946299617ba6ecb6cf18f594be7 v1/direct-route.dot 1363 775348c03ec8e6c53f6862db274f2e3c4a56d78e3cc1b5c96a0e5489d8b58143 v1/direct-route.report.json 331 01a11f0c56a973782413c340833e32287e732384b275801c218318db6b39571f -v1/direct-route.resources.json 434 a42f6ddbf5706d14d11dab5fe2c850370e4b227e7e1bf6c1bbd5004acc396eed +v1/direct-route.resources.json 434 da6443bbac593e405f04d0714d1e3a27f817380a1cb3f775ec7476139c883353 v1/invalid-scc-diagnostics.json 296 9b43dbd8d62935e3eb1d96e39c0114b0dfd0bec91d84a6a786cdc0902a4b4600 v1/main-sidechain-pdc.csv 358 c08654d06220ecec7e0730298725392bba3afeb2b4779fc20c597d003bfead29 v1/summation-residuals.json 843 1c90bc760e98fc852a7688f334249b0b675fc47d1ba115c209d8e54f79e523eb diff --git a/fixtures/graph/v1/direct-route.resources.json b/fixtures/graph/v1/direct-route.resources.json index a28290661..414d0a0d1 100644 --- a/fixtures/graph/v1/direct-route.resources.json +++ b/fixtures/graph/v1/direct-route.resources.json @@ -1 +1 @@ -{"schema":1,"fixture":"direct-route","logical_nodes":9,"materialized_nodes":9,"edges":8,"schedule_items":9,"dependency_levels":9,"colored_output_buffers":2,"audio_buffer_samples":8705,"delay_bytes":0,"graph_metadata_bytes":6476,"declared_effect_bytes":0,"builtin_bank_count":3,"builtin_bank_bytes":7047,"builtin_bank_scratch_bytes":24576,"largest_allocation_bytes":5336,"incremental_plan_bytes":48343,"session_plus_plan_bytes":48343} +{"schema":1,"fixture":"direct-route","logical_nodes":9,"materialized_nodes":9,"edges":8,"schedule_items":9,"dependency_levels":9,"colored_output_buffers":2,"audio_buffer_samples":8705,"delay_bytes":0,"graph_metadata_bytes":6476,"declared_effect_bytes":0,"builtin_bank_count":3,"builtin_bank_bytes":7239,"builtin_bank_scratch_bytes":24576,"largest_allocation_bytes":5336,"incremental_plan_bytes":48535,"session_plus_plan_bytes":48535} diff --git a/tools/audit/src/builtins_graph.rs b/tools/audit/src/builtins_graph.rs index 44794e333..e3c70ab72 100644 --- a/tools/audit/src/builtins_graph.rs +++ b/tools/audit/src/builtins_graph.rs @@ -63,8 +63,11 @@ const B_RENDERS: u64 = REFUSED_RANGE + 2; // `resources.jsonl` and its manifest row moved (+2 bytes, +1 allocation per row). // Re-pinned for #1609: `GraphNodeObserverBinding` is 88 bytes (the `block_end` flag), so the // meter-bearing `resources.jsonl` rows and its manifest row moved (+8 bytes per meter). +// Re-pinned for #1630 carry marks (batch 14b review fold): each input, fader and matrix cell +// reader keeps two u16 masks, so `StripPreparation` grew by 24 bytes per track (1136 -> 1160) and +// `resources.jsonl` and its manifest row moved. const ACCEPTED_MANIFEST_SHA256: &str = - "f99fdbf2e91782a98f044b35ec71d044303b8a11bc53a232707db3e17b4c795a"; + "58ba5b37f3ae85692472b2c36731c3a5ab56f9aef323e376c9bc820699fc75f7"; const ACCEPTED_GRAPH_PCM_SHA256: &str = "508c8e94244b99ae1ee59e4863088ba69c6462127eb0256f85ec72e775a17a19"; const ACCEPTED_GRAPH_METERS_SHA256: &str = diff --git a/tools/audit/src/fixture_builtins.rs b/tools/audit/src/fixture_builtins.rs index 79283ce91..92caeea9a 100644 --- a/tools/audit/src/fixture_builtins.rs +++ b/tools/audit/src/fixture_builtins.rs @@ -372,7 +372,11 @@ const INPUT_PROCESSOR_BYTES: u64 = 696; // (`FaderReader` 56 bytes, `MatrixReader` 32) in place of two 40-byte ring consumers. // #1346: +48 (1088 before): it holds the input cell reader (`InputReader` 88 bytes) in place of // the 40-byte input ring consumer. -const STRIP_PREPARATION_BYTES: u64 = 1136; +// #1630 carry marks (batch 14b review fold): each input, fader and matrix cell reader keeps two +// u16 masks (`CarryMarks`, 4 bytes, which the readers' 8-byte alignment rounds to +8 each), so +// +24 (1136 before). The masks are plain `u16`s, not lane-width banks: the size is the same on +// every 64-bit target, AVX2 and NEON/simd128 alike. +const STRIP_PREPARATION_BYTES: u64 = 1160; const FADER_PROCESSOR_BYTES: u64 = 16; const MATRIX_PROCESSOR_BYTES: u64 = 136; // #1080 removed the controlled-activation flag #816 added, so the binding was 80 bytes again; @@ -5373,7 +5377,13 @@ mod tests { // Re-pinned by issue #1609: `GraphNodeObserverBinding` grows from 80 to 88 bytes after // the bind-time `block_end` acceptance flag is added; only the meter-bearing resource // rows (eight bytes per meter) and this joined manifest identity move. - "f99fdbf2e91782a98f044b35ec71d044303b8a11bc53a232707db3e17b4c795a", + // + // Re-pinned by issue #1630 carry marks (batch 14b review fold): each input, fader and + // matrix cell reader keeps two u16 masks, so `resources.jsonl` alone moves, by +24 + // bytes per track in every processor payload count and in the largest allocation + // (`StripPreparation` 1136 -> 1160). The allocation counts do not move. No PCM, + // meter, response or benchmark fixture moved. + "58ba5b37f3ae85692472b2c36731c3a5ab56f9aef323e376c9bc820699fc75f7", "accepted joined-corpus manifest identity" ); remove_temporary_root(root);