diff --git a/.github/ISSUE_SPECS/1290-replace-the-running-browser-session-in-the-rust-host.md b/.github/ISSUE_SPECS/1290-replace-the-running-browser-session-in-the-rust-host.md index 76efee23f..fb4212b07 100644 --- a/.github/ISSUE_SPECS/1290-replace-the-running-browser-session-in-the-rust-host.md +++ b/.github/ISSUE_SPECS/1290-replace-the-running-browser-session-in-the-rust-host.md @@ -569,3 +569,77 @@ follows the apply (#1296's); the engine side is complete. - **Doc fixes.** `miso_engine_web_v1_meter_header_ptr` no longer says "stable" (the address changes at each swap, each plan carrying its own header); the browser oracle's header size comment says 80 bytes. + +## Decision record: (d2) spectrum (batch 12, lane W) + +**Shipped.** A browser rebuild keeps a running spectrum stream, an armed one-shot and a collection's +selection. At a plan's promotion the host calls `SpectrumService::adopt_successor(captures, pairing, +carried)` with the plan's own pairing list and the engine's real carry outcome. + +- **Where the successor's captures wait (departure from the text).** The spec said to take them out of + the adopted plan's attachment. The attachment is the render half's once the plan is published, and + two adoptions before one service step would leave the first plan's captures unreachable. The + preparer now leaves a plan's captures and `inventory.spectrum_pairing()` in the `BootRecord`, beside + the plan's tables (`SpectrumHandover`); boot moves them into the service, and a command that + publishes a replacement keeps them in the plan's `PendingShape`. `RenderCompanions` lost its + `spectrum_capture` field (and `has_spectrum_capture`, `take_spectrum_capture`). The pairing is + charged to the plan's rows with its tables. +- **Carry outcome.** The adoption block reads `RealtimePlanOwner::carried_count()` and compares it with + the count at the previous adoption (`carried_swaps_seen`): a swap carried exactly when it advanced. + One `u64` load and compare in the swap block; `PendingShape::adoption` records `Some(carried)`. + The block that adopted the plan marks it; a candidate a later one superseded keeps `None` and its + captures drop with the entry. +- **Promotion.** `ReadyOwnership::promote_spectrum`, run first in `reclaim_shapes` (the service step + and the start of every command), installs each adopted plan's captures in commit order and drops + the displaced captures there, on the control side. Spectrum exports run in render-locked windows, + so promotion is deliberately not called from them. +- **Lane K.** #1618 adds a fourth argument, `analysis: Option<&mut SpectrumAnalysis>`, to + `adopt_successor`. The browser holds no analysis state (the worklet passes `None`; the caller of + `start_spectrum_stream` owns it), so the call in `promote_spectrum` passes `None` at the merge. + +**Gates -> tests** (`hosts/host-web/src/tests.rs`, `structural_apply`): +- gate 7, stream: `a_running_spectrum_stream_continues_across_a_rebuild` (an added track sorts first + and shifts every index; the stream is read after every block and equals a fresh boot of the + committed model window for window; one capture epoch, no `Failed`/`Gap`; service step right after + the adoption block, and 18 blocks later) +- gate 7, collection: `a_collection_selection_and_its_stream_continue_across_a_rebuild` (second entry + selected and streamed) +- armed one-shot: `an_armed_one_shot_completes_across_a_rebuild` (window spans the swap, equal to the + committed model's) +- `web_runtime_preparer_attaches_the_companions_on_every_branch` now reads the capture kind from the + record. +- Red check: forcing `swap_carried = false` turns the stream, collection and one-shot tests red. + +**Limits found.** +1. With the carried arm, promotion is not observable: the service keeps the same reader whether it + holds the predecessor's capture or the paired successor's shell. Only a swap that did not carry + distinguishes them, and the browser cannot make one yet: the only natural refusal is the + identity mismatch of a successor of a candidate that never rendered (supersession, #1310), and + a second command while a candidate waits is `Backpressure` today. No test covers the not-carried + arm on the browser; `a_refused_hand_over_installs_the_fresh_capture` (host-core) covers + `adopt_successor`'s side. Revisit when #1310 lands: add the two-rebuilds-before-one-block case. +2. A removed spectrum-observed strip: not tested here (#1593 keeps the taps). +3. Gate 8 (render-locked allocations) is unchanged; the adoption block adds one load and compare + (`realtime-policy` ok). + +## Fold (batch 12b review) + +- **m1/m7.** The batch 12 note above says the browser passes `None` to `adopt_successor` in + `promote_spectrum`. That left the analysis of an instance that owns both halves (the ffi staging's + `SpectrumAnalysis`) active after a promotion that does not keep the stream: `spectrum_stream_reset` + answered ok, and `spectrum_stream_analysis` adopted the stale metadata record again and re-reported + the last window. `service`, `control_command`, `run_command`, `reclaim_shapes` and + `promote_spectrum` now take `Option<&mut SpectrumAnalysis>`. The two exports lend the staging's + analysis through `with_promotion_analysis` (`try_borrow_mut`; `None` when the staging is borrowed, + so the audio thread's own instance, which keeps none, is unchanged). When the promotion ended the + analysis, the export also writes the stream record from the service and drops the staged window, as + `spectrum_stream_stop` does; without that the record still says `Ready`. +- **Gate:** `ffi::spectrum_ffi_tests::a_promotion_that_drops_the_stream_ends_the_analysis`, run for + both exports. The swap carries, and its pairing is emptied so it does not name the capture (the + host cannot make a refused hand-over yet; limit 1 above stands for that arm). After the promotion + the analysis is not `stream_active`, `spectrum_stream_reset` answers a wrong state, and + `spectrum_stream_analysis` does not answer ok. Red with the exports passing `None`. +- **m6.** `the_headers_structure_epoch_is_the_control_lists` now reads the epoch from a real + `CONTROL_LIST_GET` page through the control entry, after a rebuild and after a live commit that + moves the epoch (a VCA added). It is red when `stamp_structure_epoch` runs for rebuilds only (the + live stamp removed): the header keeps 2 while the page reads 3. diff --git a/.github/ISSUE_SPECS/1562-admit-browser-live-edits-in-the-worker-through-the-committed-model.md b/.github/ISSUE_SPECS/1562-admit-browser-live-edits-in-the-worker-through-the-committed-model.md index e5cabdd84..7797406c3 100644 --- a/.github/ISSUE_SPECS/1562-admit-browser-live-edits-in-the-worker-through-the-committed-model.md +++ b/.github/ISSUE_SPECS/1562-admit-browser-live-edits-in-the-worker-through-the-committed-model.md @@ -485,3 +485,230 @@ Checked against `ea336c039`. lanes compile at every merge. - **Frozen decisions.** D1-D6 and the batch 10 split hold. - **No edit under `crates/engine/src/realtime`.** + +## Decision record: #1562b-W (batch 12) + +Lane W. Checked against the batch 12 head `0c6f35a1d`. + +### What shipped + +- **D9 is gone in the browser.** `REBUILD_PENDING`, `WebRuntimePreparer`'s `refuse_rebuild` + override, the `allow_rebuild` switch and `structural_command_for_test` (with + `native_staging::structural_command`) are deleted. The browser prepares and publishes a + rebuild like the C ABI: a transaction's ack still comes only after the preparer left the plan's + tables (`run_command`). The core hook `RuntimePreparer::refuse_rebuild` and its call in + `SessionState::command` stay for lane K to delete. Gate 11 and its test are deleted. + The #1290a, #1381b and #1584 D6b tests, and the plan-exchange and structural-realtime integration + tests, drive the rebuild through `control_request_mut` and `control_command` (the unit module's + `Rig::entry`, `native_staging::control_command`). +- **Structure epoch in the meter header.** `WebMeterHeader` grew at its end: `structure_epoch: u64` + at offset 80 (88 bytes). `run_command` stamps the newest committed plan's header (the pending + replacement's, else the rendering plan's) with `SessionState::structure_epoch()` (a new read-only + accessor next to `committed_session`; the only edit under `crates/control-plane`), and the boot + stamps the boot plan's. The worklet posts it as `structureEpoch` in every `miso.meter.v1` frame + (two 32-bit words, no BigInt) and in the `miso.observationmap.v1` reply (read from the rendering + plan's header). Mirrors: `abi_layout.rs`, `check-abi-layout-v1.py` and its fixture, the + regenerated `sdk/assets` and `sdk/src/generated/abi.ts`, both `.d.ts`, the host's frame validator. + Lane K reads `meterHeader.structureEpoch` in `pollMeters`. +- **Maps follow the plan.** The worklet read `sources` and the observation bindings once at boot; + both are now read from the module at every `miso.sessionmap.v1` / `miso.observationmap.v1` + request (`readSourceRows`, `readObservationBindings`), so a rebuild that adds a source, a strip or + an effect is seen. The exports those reads call after construction are now `render_locked` and + listed (wrapped) in `render-thread-exports.json`; `check-session-map-shape.py` follows + `readSourceRows`. +- **Browser SDK follows the strip order.** `createBrowserStripOrders` (sdk/src/browser/live-controls.ts) + keeps the last eight strip orders by structure epoch. A meter frame is named by the order of the + epoch it carries (`createMeasurementFeeds(host, orders, available)`, in-order delivery, a frame + whose list the page never held is dropped), and the observation map is paired with the order of + the epoch the reply names. No fixed boot strip IDs remain in `meterProjection`'s callers. +- **Browser qualification.** A new page row, `structural-apply` (`runStructuralApplyQualification`): + a `delay_samples` edit through the SDK is the rebuild path, the applied revision is not early, the + output is the input delayed by exactly the edit (computed, not read back), the meter stream is the + successor's with one structure epoch, and the render-locked count is 0 (11 workloads now). Four + mutations: `structural-apply-path`, `-early`, `-output`, `-meter`. + +### Departures and notes + +- Gate 1's `delay_samples` row is its own test (`a_delay_samples_edit_is_a_rebuild_and_renders_as_a_fresh_plan`), + because a rebuild's reply and swap differ from the live rows' loop. +- The rebuild rows of the batch 7 parity amendment are scripted: output route gain and mute, + `delay_samples`, the multiband and delay prepared bypass. **No launch effect declares an + audio-rate parameter** (`AutomationRate::Sample` has no user in a launch effect), so that row has no + edit. +- `sdk/test` repairs that the browser change forces (the module accepts the rebuild in headless + and browser alike): `control-list-evals.mjs` (rebuild-only row commits as `rebuild`), + `console-evals.mjs` (the delay and multiband lift render the unbypassed session), and the fake + `observationMap` replies carry `structureEpoch`. `sdk/README.md` loses one sentence. +- Not done by lane W: the SDK's named effect rows and `pollMeters`' re-read (lane K), the managed + observation subscription remap (lane K). + +### Gates + +`control_admission.rs`: `a_delay_samples_edit_is_a_rebuild_and_renders_as_a_fresh_plan` (gate 1 row), +`a_structural_transaction_keeps_the_solo` (gate 3, structural part). `control_parity.rs`: +`both_hosts_give_the_same_result_for_the_same_control_edits` (now with the six rebuild steps), +`every_scripted_edit_is_heard`, and `the_status_words_carry_a_rebuilds_watermark_the_c_abi_reports` +(#1349 rebuild leg, four rates, equal to `miso_engine_v1_plan_watermark` after every block). +`tests.rs`: `the_headers_structure_epoch_is_the_control_lists`, the epoch assertions of +`a_strip_count_change_publishes_the_successors_frame_from_the_adoption_block`, and the layout test. +Worklet harness: the epoch in the frame (including past 2^32 and the adoption block), the frame +validator, the re-read session and observation maps. SDK evals: `measurement-evals.mjs` (three new: +frames named by their epoch, in order; an unnameable frame is dropped; the engine follows a rebuild). +Browser: `npm run qualify` with `--sdk-root`, `--check-matrix`, `--self-test-mutations` passes in +chromium 151.0.7922.34, firefox 153.0 and webkit 26.5 (the matrix rows are unchanged). + +## Decision record: #1562b-K (batch 12) + +Lane K. Checked against `2b79cec44` (batch 12a + #1562b-W). + +### What shipped + +- **The core hook is gone.** `RuntimePreparer::refuse_rebuild` (`crates/control-plane/src/compile.rs`) + and its call in `SessionState::command` are deleted. Both hosts take every rebuild: the host-web + rebuild tests (`control_admission`, `structural_apply_realtime`, the unit `structural_apply`) and + the capi tests pass without the hook. +- **A meter frame names its own strips.** `WasmBoundary.pollMeters` reads the structure epoch in the + meter header (offset 80, #1562b-W) and names the frame by the control list of that epoch. The + boundary keeps the last eight orders by epoch; a frame whose list it never read and cannot read now + is dropped (the browser's `createMeasurementFeeds` does the same). `MeterFrame` gains + `structureEpoch`, `trackIds` and `submixIds` (additive), so a frame that a rebuild leaves with the + same strip count is still named right, and a window the old plan completed is named by the old list + even after the SDK read the new one. +- **Named effect rows follow the core's section headers.** `ControlStripOrder` (the one cached read of + the list) now also holds the effect of each console slot and insert, from the list's section + headers (`stripEffectOf`). `LiveControlEdits` resolves a named effect row from it first, from the + booted session second. An edit that names an effect row remembers (effect, row name, strict or + not); `effectRowResolver`, installed as the transaction queue's new `prepare` hook on both engines, + checks it against the list just before the frame is encoded (a probe of the structure epoch, the + list only when it moved; edits with no effect row cost nothing). A named section follows its header; + a builder typed for an effect the insert no longer runs is refused with `MisoUsageError`, and + nothing is sent. The queue draws the transaction's request ID after `prepare`, because the core takes + IDs in ascending order and `prepare` reads the list with IDs of its own. +- **Live controls follow the strips.** `EngineLiveControls.syncStructure()` and + `LiveControlTransport.stripOrder` re-read the order when the epoch moved; an observation record is + re-addressed to its strip's index in the current list just before it is sent, so a manual + `observe()` edit built before a rebuild is also right. The headless `liveControls()` and + `withSession()` drop the booted session from the layout once a rebuild moved the strips (the engine + is then no longer that session); the browser's memoized controls drop it in `update()`. +- **Managed subscriptions follow a rebuild and end on `OBSERVATION_ENDED`.** + `ObservationSubscriptionOwner` takes the event through the new transport member + `observationEnded` (the headless engine and the browser engine decode the reliable lane with + `decodeObservationEnded`, `sdk/src/core/observation-ended.ts`). The event, one per commit with every + key, drops the bindings and tells each subscription once: `onEnd({ revision, ended, remaining })`; + `handle.ended` lists them; a subscription left with no selection is closed. An event that arrives + while an operation is in flight is applied when it ends. Arm and disarm records take their strip + index and effect index from the newest control list and observation map (`syncStructure()`, and + `#remap()` of every held binding), not from the address they were subscribed with. + +### Departures and notes + +- **Files outside `sdk/src/core` and the core hook** (mechanical or forced by the above): + `sdk/src/headless/engine.ts` and `sdk/src/browser/{engine,live-controls}.ts` (the transport wiring), + the three index barrels (`ObservationSubscriptionEnd`), `sdk/README.md`, and the test fakes that + must carry `onControlEvent` (hosts) and `syncStructure` (live controls). The browser's + `createBrowserStripOrders` is now one reader for each host, shared by the queue's check, the live + controls and the feeds. +- **A structural fixture.** The SDK has no builder for a structural edit, so + `hosts/host-web/tests/structural_frames_fixture.rs` writes five frames with the protocol codec + (`fixtures/structural-frames.json`: add a first strip, replace the last strip, swap an insert for + another effect, remove an insert, remove the insert before it), and `structure-evals.mjs` sends + them to a real core. It is a wire format, so the byte comparison is the gate. +- **The queue does not learn a foreign commit's revision.** A structural frame sent past the queue + (as the evals do) leaves the queue one revision behind and the next transaction is refused. The + evals call `queue.reset(controlRevision())`. A transaction API for structural edits (#1296) goes + through the queue and has no such gap. +- The existing `set checks only encodability` eval now asserts that a named insert row resolves from a + document-text boot (it was a refusal before) and that an unknown insert is still refused. + +### Gates + +`sdk/test/structure-evals.mjs`: `a rebuild that adds a strip: frames before name the old strips, +frames after the new`; `a window the old plan completed is named by the old list even after the new +list was read` (red when the frame is named by the newest cached order); `a rebuild that keeps the +strip count and changes the strips is named by the new list`; `headless` and `browser: a row named +before an insert changed effect lands on the new effect's row` (the compressor's `mix` is control 7, +the soft clip's 3; the typed builder and a name the new effect lacks are refused and send nothing); +`the end of a tap is reported once, with the rest of the subscription kept (headless)`; `one commit +that ends every tap of a subscription closes it and says so once`; `removing the insert ends its tap +the same way`; `a tap that survives a rebuild keeps publishing and is disarmed at the strip's new +index`; `an effect that moved to another index is disarmed where it is now` (red without `#remap`); +`through the browser transport, whose live controls were made before the rebuild` (red without +`syncStructure`/the record re-addressing). `check-sdk-headless.sh` 543 pass; `check-sdk-types.sh`, +`check-sdk-generated.sh`, `sdk-package.sh check` pass on `target/ci/b12k-artifacts`; chromium +151.0.7922.34 qualification with `--sdk-root`, `--check-matrix`, `--self-test-mutations` passes (all +render-allocation rows 0). Rust: control-plane (`test-support`), capi and host-web +(`host-web/test-support`) tests, clippy `-D warnings` (host-core with no features, default and all +features, control-plane, capi, host-web), fmt, `realtime-policy` (141 regions) and +`check-bench-policy.sh` pass. + +### Open + +- Between a rebuild's commit and the block that adopts it, `observationMap()` describes the plan that + renders, and a kind 7/8 record is queued on the newest plan. The evals render a block before they + subscribe. Which plan's index space the record addresses in that window is the core's to rule + (successor, with #1589). + +## Fold (batch 12b review) + +- **M1.** The "Open" item above (which plan's index space a record addresses between a rebuild's + commit and its adoption) is closed on the SDK side. The core admits against the newest plan + (`admit_on_newest_plan`), the strip index comes from the newest list, and the effect index came + from `observationMap()`, which describes the plan that renders. `ObservationMap` now carries the + `structureEpoch` of that plan (the browser reply's own; the headless engine reads the meter header + of the rendering plan and names the strips by the list of that epoch). Subscribe, update and close + compare it with the epoch of the list used to address the record (`syncStructure()` returns it). On + a mismatch an engine with a render wait waits for a render and reads again (at most 8 renders and + the engine's request deadline); the headless engine, and a wait that ends, refuse with the typed + retryable `backpressure` error (`sdk.observation.structure_behind`). `submitManagedAt(epoch, ...)` + checks the list epoch again at the send, so a commit between the check and the send is refused, not + misaddressed. No record is sent in the window. +- **Gates** (`sdk/test/structure-evals.mjs`, fixture frame `swap_t3_multi_and_limiter` from + `structural_frames_fixture.rs`: two effects with the same tap change indices): subscribe, close and + update in the window are refused and arm or disarm the right tap after a render; an owner with a + render wait sends the record of the adopted plan; a wait that gives up sends nothing. The browser + engine: `measurement-evals.mjs`, the map's epoch 1 against list epoch 2. All red on the previous SDK. +- **m2.** Meter frames the SDK cannot name by epoch are counted. The browser feed adds their windows to + `lossCount` of the next update (saturating) and sets the loss bit once; the headless `MeterFrame` + carries `unnamedWindows`. Eval: `a meter frame whose list the page cannot name is dropped, counted, + and the stream goes on`. +- **m3.** `BindingState.armRevision` is the committed revision read just before the arm was sent; an + `OBSERVATION_ENDED` event with a revision at or before it is ignored (it ended an earlier arm). + Eval: `an ended event older than the arm is ignored`. + +## Decision record: source staging sized by the launch maxima (batch 12b review B1) + +Since #1562b-W the browser accepts rebuilds, and a rebuild can add a source the boot session did not +have. The browser sized its ID staging (longest ID of the boot session) and its source PCM and plane +staging (`maximum_source_channels` of the boot session) at boot, so an acked rebuild that added a +longer ID tripped a `debug_assert` and then indexed past the buffer inside the render-locked +`miso_engine_web_v1_source_id` export (a wasm trap), and one that added a wider source made the +worklet's `readSourceRows` return `null` (sticky `RESULT_INTERNAL`); either way the acked source +could never be fed (an ack preceded a drop). Root ruling A: size the staging by what the grammar +allows, so nothing depends on the boot session. + +- **Constants.** `session::STABLE_ID_MAXIMUM_BYTES` (127; the one literal `StableId::parse` and + control-plane's observation rows already used) and `session::SOURCE_CHANNELS_MAXIMUM` (255). The + grammar has no source-channel limit narrower than the `u8` of `Source::channels`, so the `u8` domain + is the maximum; no new limit was invented. `project_buffers` no longer takes the shape's two values. +- **Typed error.** `copy_id_into_staging` returns `Result` (`NoStaging`, + `TooLong`) and the export answers zero on either (its existing "no such source"), never a panic. + With the staging at the maximum the second cannot happen for a compiled session. +- **Memory.** Staging grows by 129,536 bytes of source PCM, 4,048 of plane references and 113 of ID + staging (this fixture: `idStagingBytes` 14 to 127, `sourcePcmStagingBytes` 1,024 to 130,560; + `bridge_retained_bytes` 1,631,181 to 1,764,878, `bridge_metadata_bytes` 1,594,186 to 1,598,234; + wasm `memoryBytes` after boot 2,883,584 to 3,014,656, two pages). Re-pinned with the reason in + `retained_ceilings.rs` (`bridge_retained_bytes` ceiling `CAPTURE + 787,968`) and + `browser-v1/expected.json` (`bridgeRetainedBytes` 1,812,352 = 1,048,576 + 763,776, measured beyond + capture 694,314 plus 10 %, rounded up to 64; the two exact staging rows; `memoryBytes`). +- **Boot budget.** The 1 MiB, 512-track worst document's high-water growth + (`scripts/check-web-boot-budget.mjs`) measures 20,447,232 bytes before and after the change, under + the 20 MiB pin plus one page (21,037,056): the parse transient dominates and the staging fits in + the memory the parse already grew. +- **Gates.** `plan_exchange_threads::a_rebuild_adds_the_longest_id_with_more_channels_than_any_boot_source` + (exports: a rebuild adds a 127-byte ID with six channels; `source_id` returns 127 and the staged bytes + match, `source_channels` is 6, a six-plane chunk is accepted and heard; red on a staging sized from + the boot session, both for the ID and the PCM), `control_parity::a_rebuild_adds_the_longest_stereo_source_to_a_session_that_booted_with_a_mono_source_only` + (browser and C ABI render the same bits across the rebuild for the new source at 44.1, 48, 88.2 and + 96 kHz), `host-web` unit `staging_holds_the_grammars_longest_id_and_widest_source_and_refuses_an_overlong_id`. +- **Departure.** A session with no source or track does not prepare (`host.source.graph.prepare`), so + "start empty" is not reachable; the gates boot with one mono source. diff --git a/.github/ISSUE_SPECS/1584-subscribe-to-an-effects-observation-tap-with-a-transaction-edit.md b/.github/ISSUE_SPECS/1584-subscribe-to-an-effects-observation-tap-with-a-transaction-edit.md index 743c47f77..1fd3833fa 100644 --- a/.github/ISSUE_SPECS/1584-subscribe-to-an-effects-observation-tap-with-a-transaction-edit.md +++ b/.github/ISSUE_SPECS/1584-subscribe-to-an-effects-observation-tap-with-a-transaction-edit.md @@ -331,7 +331,7 @@ Checked against `ea336c039`. host-core and control-plane tests above, and `docs/CONTROL_PROTOCOL_{REGISTRY,SEMANTICS}.md`. - **D6b (lane W, after D6a).** `build_plan_shape` (`hosts/host-web/src/lib.rs:5437`) sets `observation_armed` for the bindings of `MonitorBase.next`, with arm sample 0 and the commanded bit - clear: a fresh tap has no stale window. Gates through `structural_command_for_test`: + clear: a fresh tap has no stale window. Gates through `control_command`, the host's production control entry: - After a restart of a subscribed compressor, the meter frame's gain-reduction word and the observation read return windows from the adoption block on. - After an insert removal, there is no slot for the removed insert, and the browser's event drain @@ -533,3 +533,54 @@ Root ruling A replaces D6a's one event per ended binding with one `OBSERVATION_E - **Internal invariant spelled accurately.** A reliable `OBSERVATION_ENDED` slot whose cell is empty is now `EventEgressError::ObservationEndedCellMissing` (was `DiagnosticStorageFull`); behaviour (an internal error) is unchanged. + +## Decision record: #1584 D6b (batch 12b, lane W) + +What shipped, in `hosts/host-web/src/lib.rs`: + +- `build_plan_shape` takes the bindings (`&[controls::ObservationBinding]`, from the `SuccessorBase` the + preparer received; empty at boot) and sets `observation_armed` for each binding whose strip, section + (console or inserts), effect ID and tap ID match an observed instance of the plan. The bit is the tap's + index in the descriptor's observations, as `push_observe` sets it. The arm sample stays 0 and the + commanded bit stays clear. The match is the one host-core uses for the lane's prepared arm, so the + table and the lane cannot disagree; a binding the plan lacks sets nothing. A carried reader still + takes the predecessor's bits (`carry_tap_state`), so the prepared arm never wins over carried state. + Nothing new runs in `process()`: the bits are written on the control side while the shape is built. +- Doc comment of `onControlEvent` names `OBSERVATION_ENDED` in + `hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts` and its copy + `sdk/src/browser/shipped-host.d.ts` (equal). The worklet drain (`drainControlEvents`) copies any + frame it dequeues and decodes none, so the event passes through with no JS change (the reliable lane + holds `SESSION_COMMITTED` and `OBSERVATION_ENDED`, within the drain's 16-call bound). The SDK reaction + stays with #1562b-K. + +Gates (`structural_apply` in `hosts/host-web/src/tests.rs`, through `control_command`, the production control entry): + +1. `a_restarted_effects_subscription_keeps_publishing_windows_on_the_browser`: the subscribed compressor's + link mode changes (rebuild, no carry); the successor has `observation_armed_taps() == 1`, its first + window opens at the adoption block's first sample with sequence 1, the windows continue (sequence 3 + after four more blocks), and the meter frame's gain-reduction word is positive. Red with the + bindings passed as `&[]` (armed taps 0 after the swap). +2. `a_removed_inserts_subscription_ends_with_an_event_and_has_no_slot`: after the insert removal the + browser's event drain returns `SESSION_COMMITTED` then one `OBSERVATION_ENDED` with the committing + revision and the key (`track`, inserts, `comp`, tap 1); the successor has no armed tap and + `read_observation_addresses` refuses the removed insert's address. +3. `tests/structural_apply_realtime.rs`: 3 passed, zero render-locked allocations. + +Runs: `cargo test -p host-web --features host-web/test-support` (all), `cargo test -p control-plane +--features test-support`, clippy `-D warnings` on host-web, fmt, the wasm build, `check-web-audioworklet.sh`, +`test-web-audioworklet.sh`, `check-browser-expected-resources.py --artifacts`, `check-sdk-generated.sh`, +`check-sdk-types.sh`, `check-sdk-headless.sh`, the chromium qualification (151.0.7922.34, all +render-allocation rows 0), `realtime-policy` (141 regions) and `check-bench-policy.sh` all pass. The +shipped module sha256 is `179a933a...c3c1ae`; no export, layout or JS file changed. + +## Fold (batch 12b review) + +- **n1.** The D6b gates run through `control_command`, the host's production control entry. The text + above said `structural_command_for_test`, which D9 of #1562 deleted. +- **m1/m7 (shared with #1290).** `miso_engine_web_v1_service` and `miso_engine_web_v1_control_command` + now lend the ffi staging's `SpectrumAnalysis` to the promotion their reclaim performs, so a swap + that does not keep the stream ends the instance's analysis in the same call (see the #1290 fold). +- **m3 (shared with #1562).** The SDK keeps the revision the core had committed when each managed + arm was sent and ignores an `OBSERVATION_ENDED` event of a commit at or before it: that event ended + an earlier arm of the same tap. `ControlTransactionQueue.readCommittedRevision()` reads it from the + core. diff --git a/.github/ISSUE_SPECS/1676-cap-the-observation-bindings-by-what-one-ended-event-holds.md b/.github/ISSUE_SPECS/1676-cap-the-observation-bindings-by-what-one-ended-event-holds.md new file mode 100644 index 000000000..774947f00 --- /dev/null +++ b/.github/ISSUE_SPECS/1676-cap-the-observation-bindings-by-what-one-ended-event-holds.md @@ -0,0 +1,62 @@ +# Cap the observation bindings by what one ended event holds + +Successor of #1584 D6a (batch 12a, root 2026-10-15), decision 17 +(`docs/rulings/one-control-abstraction-2026-10-13.md`). Code anchors verified on `45a2ab9df` +(PR #1675). Issue number: assigned at filing (written here as #1676). Feature sprint mode (owner, +2026-10-13): one implementer, normal tests, a batch review. + +## Product outcome + +A subscription the engine acknowledges can always be ended honestly: no commit that removes a +subscribed target is refused because its `OBSERVATION_ENDED` event would not fit one control +frame, and the memory reserved for that event follows the bindings the session can hold, not the +frame size. + +## Context + +- #1584 D6a ends every binding whose target is gone with one `OBSERVATION_ENDED` per commit + (root ruling A). The event must fit `maximum_control_frame_bytes`; a commit whose ended keys do + not fit is refused as a resource limit (`crates/control-plane/tests/observation_bindings.rs`, + `an_end_event_over_one_control_frame_refuses_the_removal_and_changes_nothing`). So a valid + removal can be refused, and the app must disarm first. +- The batch 12a review fold charges each `OBSERVATION_ENDED` cell a flat ceiling bounded by the + frame size (`ended_event_ceiling`, `crates/control-plane/src/compile.rs:572`; + `observation_reserve_bytes`, `:553`). The reserve grows with `maximum_control_frame_bytes` + (about 4x at the browser's 16 KiB frame against the 4 KiB reference); `capi_retained_bytes`' + ceiling rose from 181,696 to 205,312 with it. + +## Decisions frozen for this slice + +- **D1. Cap at admission.** A subscribe (an Observe that would add a binding) is refused with a + typed refusal before any ack when the bindings table already holds as many keys as one ended + event can carry in one control frame (computed once at preparation from the frame limit and the + largest encoded key the session's IDs allow). Nothing is queued and no state moves. +- **D2. Reserve by capacity.** The ended-event reserve per cell is the smaller of the frame bound + and the encoded size of the bindings capacity (the slots the plan has), so a session with few + observation slots reserves little whatever the frame size. +- **D3. Same on both hosts**, with a parity row. + +## Objective gates + +1. With a small `maximum_control_frame_bytes`, the subscribe that would exceed the cap is refused + with the typed code, and a removal of every subscribed target afterwards commits with one event + holding every key (the over-frame refusal can no longer be reached by a valid removal). +2. `capi_retained_bytes` on the reference session falls back toward its pre-fold figure at the + 4 KiB frame, and the reserve no longer scales with the frame for a plan with few slots; the + ceiling is re-pinned to the measured value (plus 10 %, rounded up to 64) at both lane widths. +3. The charge-equals-retained capi tests stay exact; `cargo test --locked -p control-plane + --features test-support -p capi -p host-web --features host-web/test-support`; clippy. + +## Authorized paths + +- `crates/control-plane/src/{compile.rs,control.rs}`, `crates/controls/src/observation.rs`, + `crates/protocol` (the refusal code, if a new one is needed), their tests, + `crates/capi/tests/resource_lifecycle.rs` (the ceiling), docs, this spec's decision record. + +## Dependencies + +- #1584 D6a (PR #1675). + +## Size + +Half a day. diff --git a/.github/ISSUE_SPECS/1677-address-observation-records-by-stable-key-in-the-core.md b/.github/ISSUE_SPECS/1677-address-observation-records-by-stable-key-in-the-core.md new file mode 100644 index 000000000..1bf1bbaf4 --- /dev/null +++ b/.github/ISSUE_SPECS/1677-address-observation-records-by-stable-key-in-the-core.md @@ -0,0 +1,41 @@ +# Address observation records by stable key in the core + +Successor of the batch 12b review (M1), decision 17. Code anchors verified on the batch 12b branch `codex/controls-batch-12b`. Issue number: assigned at filing (written here as #1677). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. + +## Product outcome + +An observation subscribe or unsubscribe names its tap by stable key (strip, section, slot or +insert ID, tap ID), the same key `OBSERVATION_ENDED` uses, and the core resolves it on the plan it +admits against. No client keeps an index space, so a structural edit can never make a record arm +another effect's tap. + +## Context + +- Today a browser observation record (kinds 7 and 8) carries a strip index and an effect index. + The SDK takes the strip index from the newest control list and the effect index from + `observationMap()`, and re-addresses records after a rebuild (`#remap`, `OBSERVED_STRIP` in + `sdk/src/core/observation-subscriptions.ts`); the batch 12b fold makes it wait when the map's + structure epoch differs from the list's. +- The core admits a record against the newest plan (`admit_on_newest_plan`). +- #1589 (keep only handles and callbacks in the SDK) is the related SDK slice. + +## Decisions to take (the implementer proposes, the coordinator rules) + +- The record shape: a stable key in place of the two indices, on both hosts, with the parity row. +- The core resolves the key on the plan it admits against and refuses an unknown key with a typed + code before any ack. +- The SDK's re-addressing and its epoch wait are deleted. + +## Objective gates + +1. A subscribe sent between a rebuild's commit and its adoption arms the named tap, never another. +2. An unknown key is refused with the typed code and nothing changes. +3. SDK headless and browser evals; host-web and control-plane tests; parity row. + +## Dependencies + +- #1584 D6a/D6b; #1589. + +## Size + +One day. diff --git a/.github/ISSUE_SPECS/1678-refuse-an-effect-row-edit-whose-effect-changed-in-the-core.md b/.github/ISSUE_SPECS/1678-refuse-an-effect-row-edit-whose-effect-changed-in-the-core.md new file mode 100644 index 000000000..5573131d9 --- /dev/null +++ b/.github/ISSUE_SPECS/1678-refuse-an-effect-row-edit-whose-effect-changed-in-the-core.md @@ -0,0 +1,39 @@ +# Refuse an effect-row edit whose effect changed, in the core + +Successor of the batch 12b review (m4), decision 17. Code anchors verified on the batch 12b branch `codex/controls-batch-12b`. Issue number: assigned at filing (written here as #1678). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. + +## Product outcome + +A `SET_CONTROL` edit on an effect section's row names the native effect it was written for, and +the core refuses it with a typed code when that slot or insert now runs another effect. Every +client (SDK, C ABI app) gets the same protection, and the SDK stops probing the control list +before each named-row edit. + +## Context + +- #1562b-K resolves named effect rows from the control list's section headers and checks each + effect-row edit just before encoding (`effectRowResolver`, `sdk/src/core/live-controls.ts:122`), + at the cost of a list probe per transaction; C ABI clients have no such check, so after a rebuild + a numeric effect row can land on another effect's parameter. + +## Decisions frozen for this slice + +- **D1.** An effect-section `SET_CONTROL` address may carry the expected native effect ID; the + core refuses a mismatch with a typed code before commit, nothing changes. +- **D2.** The SDK sends the expected ID from its builders and deletes the resolver hook. + +## Objective gates + +1. After a rebuild that changes an insert's effect, an edit written for the old effect is refused + with the typed code on both hosts (parity row); the same edit without the expected ID keeps + today's behaviour. +2. Wire re-pins with the repository tooling; SDK evals; protocol, control-plane, capi, host-web + tests. + +## Dependencies + +- #1562b-K. + +## Size + +Half a day to a day. diff --git a/.github/ISSUE_SPECS/1679-expose-the-armed-tap-mask-from-host-core.md b/.github/ISSUE_SPECS/1679-expose-the-armed-tap-mask-from-host-core.md new file mode 100644 index 000000000..9f3e1d85c --- /dev/null +++ b/.github/ISSUE_SPECS/1679-expose-the-armed-tap-mask-from-host-core.md @@ -0,0 +1,33 @@ +# Expose the armed tap mask from host-core + +Successor of the batch 12b review (m5), decision 17. Code anchors verified on the batch 12b branch `codex/controls-batch-12b`. Issue number: assigned at filing (written here as #1679). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. + +## Product outcome + +The browser copies a successor's armed taps from host-core instead of re-deriving host-core's +match, so one function decides which tap a binding arms. + +## Context + +- #1584 D6b's `build_plan_shape` (`hosts/host-web/src/lib.rs`, near `:5768-5790`) re-derives the + match host-core makes between bindings and taps (`crates/host-core/src/prepare.rs`, near + `:2271-2283`, `ObservationArm`). + +## Decisions frozen for this slice + +- **D1.** host-core's `EffectObservationHandle` exposes the armed tap mask it prepared; the + browser's table copies it; the duplicate match is deleted. + +## Objective gates + +1. The #1584 D6b browser tests pass unchanged; a test shows the browser's arm bits equal host-core's + mask for a session with several bindings, including an insert ID reused for another effect. +2. host-core and host-web tests; clippy. + +## Dependencies + +- #1584 D6b. + +## Size + +A few hours. diff --git a/crates/control-plane/src/compile.rs b/crates/control-plane/src/compile.rs index e4e7e7075..ef21b1103 100644 --- a/crates/control-plane/src/compile.rs +++ b/crates/control-plane/src/compile.rs @@ -468,9 +468,6 @@ pub(crate) fn observation_resources( }) } -/// The longest a stable ID is (`[a-z][a-z0-9._-]{0,126}`), which bounds the text of a key. -const STABLE_ID_MAXIMUM_BYTES: usize = 127; - /// The smallest encoded size of one key of an `OBSERVATION_ENDED` event: two one-character IDs, /// the section and the tap ID with their field headers, measured as what a second key adds to the /// frame. It bounds how many keys fit one frame. @@ -586,7 +583,7 @@ pub(crate) fn ended_event_ceiling(frame_bytes: u64) -> Result Result<(u64, u64), CompileFailure> { let arithmetic = || failure(ResourceFault::Arithmetic); - let text = 2 * STABLE_ID_MAXIMUM_BYTES; + let text = 2 * session::STABLE_ID_MAXIMUM_BYTES; let binding = size_of::() .checked_add(text) .ok_or_else(arithmetic)?; @@ -865,18 +862,6 @@ pub trait RuntimePreparer: Send { successor: Option>, superseded: &host_core::LiveValuesSuperseded, ) -> Result<(host_core::PreparedHost, host_core::HostLiveControlHandles), PrepareDiagnostics>; - - /// The code of the diagnostic this preparer refuses a structural transaction with, or `None` - /// to take it (issue #1562a D9). - /// - /// A command frame whose transaction the live classifier sends to the rebuild arm asks this - /// first, before the revision is stamped, the successor is prepared or anything is queued, so - /// a refusal here leaves the model, the revision, the overlay, the cells and the plan as they - /// were, and no acknowledgement has gone out. The code is one whole-session diagnostic line - /// (`code\t$\n`). The default takes every rebuild, as the C ABI does. - fn refuse_rebuild(&self) -> Option<&'static str> { - None - } } /// Prepare one plan plus its source producers, and project its resource accounting. diff --git a/crates/control-plane/src/control.rs b/crates/control-plane/src/control.rs index 910fd1f94..2085bd1dc 100644 --- a/crates/control-plane/src/control.rs +++ b/crates/control-plane/src/control.rs @@ -1344,15 +1344,6 @@ where // `commit_live` materializes an in-place token before it returns. LiveCommit::NeedsFull(_) => return Err(CommandError::Internal), }; - // #1562a D9: an adapter that cannot take a rebuild yet refuses here, with the - // token dropped unsent (nothing is stamped, prepared or queued, so the frame has - // no acknowledgement to drop). The C ABI's preparer takes every rebuild. - if let Some(code) = self.preparer.refuse_rebuild() { - drop(prepared); - return Err(CommandError::CompileRejected(crate::error::diagnostic( - code, - ))); - } // #1290a: a plan's rate and quantum are fixed for its session: the plan exchange // admits a replacement only at the same envelope, and would hand a different one // back as retryable backpressure. A transaction that changes either is refused @@ -2367,6 +2358,16 @@ where self.controller.session().compiled() } + /// The structure epoch of the committed control list (#1562b): the number the `CONTROL_LIST_GET` + /// pages carry. It advances when a commit changes an address, a header or a descriptor, a + /// rebuild that adds or removes a strip included. An adapter that publishes a plan's strip + /// order beside its meter frames stamps the plan with the epoch the commit left, so a client + /// that holds a list of another epoch knows to read it again. + #[must_use] + pub fn structure_epoch(&self) -> u64 { + self.controller.provider().structure_epoch() + } + /// What the session's one control catalog holds (#1381a): the control plane's provider owns it, /// and an adapter that charges its own report reads the figures here, not from a catalog of its /// own. diff --git a/crates/session/src/id.rs b/crates/session/src/id.rs index 510a0b2c8..a2da65eb4 100644 --- a/crates/session/src/id.rs +++ b/crates/session/src/id.rs @@ -2,6 +2,10 @@ use core::fmt; +/// The longest a stable ID is, in bytes: `[a-z][a-z0-9._-]{0,126}`. A host sizes any staging that +/// must hold an ID of a session it has not yet seen (a rebuild can add one) by this. +pub const STABLE_ID_MAXIMUM_BYTES: usize = 127; + /// A schema-validated, lowercase stable session identifier. #[derive(Clone, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)] pub struct StableId(String); @@ -11,7 +15,7 @@ impl StableId { #[must_use] pub fn parse(value: &str) -> Option { let bytes = value.as_bytes(); - if !(1..=127).contains(&bytes.len()) || !bytes[0].is_ascii_lowercase() { + if !(1..=STABLE_ID_MAXIMUM_BYTES).contains(&bytes.len()) || !bytes[0].is_ascii_lowercase() { return None; } if bytes[1..].iter().all(|byte| { diff --git a/crates/session/src/lib.rs b/crates/session/src/lib.rs index 741f59478..82593e7c2 100644 --- a/crates/session/src/lib.rs +++ b/crates/session/src/lib.rs @@ -29,7 +29,7 @@ pub use diagnostic::{ }; pub use edit_ramp::{EditRampRow, EditRampTarget, EditRamps}; pub use estimate::{ResourceEstimate, estimate_session_resources}; -pub use id::StableId; +pub use id::{STABLE_ID_MAXIMUM_BYTES, StableId}; pub use in_place::{ OwnerRef, OwnerSlots, canonical_owner_bytes, owners_counts_equal, owners_values_valid, }; diff --git a/crates/session/src/model.rs b/crates/session/src/model.rs index 92f8e57b0..d312258d0 100644 --- a/crates/session/src/model.rs +++ b/crates/session/src/model.rs @@ -211,6 +211,11 @@ closed_tokens! { } } +/// The most channels one source can declare: `Source::channels` is a `u8`, and the grammar has no +/// narrower launch limit. A host sizes any staging that must hold a source of a session it has not +/// yet seen (a rebuild can add one) by this. +pub const SOURCE_CHANNELS_MAXIMUM: u32 = u8::MAX as u32; + /// Explicit PCM output profile. #[derive(Clone, Debug, PartialEq)] pub struct OutputProfile { diff --git a/hosts/host-web/qualification/qualification.js b/hosts/host-web/qualification/qualification.js index 5cf2a2796..e83ed2491 100644 --- a/hosts/host-web/qualification/qualification.js +++ b/hosts/host-web/qualification/qualification.js @@ -399,6 +399,113 @@ async function runLiveControlQualification(createHost, sessionDocument) { }; } +/// The delay the structural row applies, in samples (#1562b). +const STRUCTURAL_DELAY_SAMPLES = 64; + +/// #1562b: a structural apply in a real browser. The live-control session's track gets a +/// `delay_samples` edit, which the core classifies as a rebuild: it prepares a replacement plan and +/// publishes it while the first plan has rendered nothing, so the very first quantum is the swap +/// block (the offline context cannot be suspended in every engine). The row holds what the swap +/// must keep: the reply is the rebuild path, the applied revision is not early, the output is the +/// submitted PCM delayed by exactly the edit (computed here, not read back), the meter stream is the +/// successor's from its first frame (the retired header's views were switched in the swap block, +/// and the frame carries the structure epoch of the list), and no allocator call ran on the +/// render thread. +async function runStructuralApplyQualification(createHost, sessionDocument) { + const context = new OfflineAudioContext(2, LIVE_CONTROL_FRAMES, SAMPLE_RATE); + const host = await createHost({ + context, + document: sessionDocument, + options: bootOptions( + LIVE_CONTROL_FRAMES, + LIVE_CONTROL_METER_BLOCKS, + ), + simd128ModuleUrl: ARTIFACT_URL, + workletModuleUrl: WORKLET_URL, + }); + host.node.connect(context.destination); + + const meterFrames = []; + const expected = [new Float32Array(LIVE_CONTROL_FRAMES), new Float32Array(LIVE_CONTROL_FRAMES)]; + const input = [new Float32Array(LIVE_CONTROL_FRAMES), new Float32Array(LIVE_CONTROL_FRAMES)]; + for (let block = 0; block < LIVE_CONTROL_BLOCKS; block += 1) { + const planes = sourcePlanes(block); + for (let channel = 0; channel < 2; channel += 1) { + input[channel].set(planes[channel], block * QUANTUM_FRAMES); + } + const acknowledgement = await host.submitSource({ + sourceId: "live-control-source", + generation: 1n, + startFrame: BigInt(block * QUANTUM_FRAMES), + sampleRateHz: SAMPLE_RATE, + planes, + frames: QUANTUM_FRAMES, + endOfRegion: block === LIVE_CONTROL_BLOCKS - 1, + }); + if (acknowledgement.result !== 0) throw new Error("structural-apply prefill rejected"); + } + for (let channel = 0; channel < 2; channel += 1) { + for (let frame = STRUCTURAL_DELAY_SAMPLES; frame < LIVE_CONTROL_FRAMES; frame += 1) { + expected[channel][frame] = input[channel][frame - STRUCTURAL_DELAY_SAMPLES]; + } + } + const meterLease = await host.meters({ + enabled: true, + onFrame: (frame) => meterFrames.push({ + sequence: frame.sequence, + trackCount: frame.trackCount, + structureEpoch: frame.structureEpoch, + peaks: frame.peaks.length, + firstSample: frame.firstSample, + endSample: frame.endSample, + }), + }); + const before = await host.status(); + const { applyDelayThroughSdk } = await import("/sdk/sdk-response-client.js"); + const command = await applyDelayThroughSdk(host, "track", STRUCTURAL_DELAY_SAMPLES); + const pending = await host.status(); + + const rendered = await context.startRendering(); + for (let spin = 0; spin < 8 && meterFrames.length === 0; spin += 1) { + await new Promise((resolve) => setTimeout(resolve, 4)); + } + const after = await host.status(); + const renderAllocations = await renderAllocationCount(host); + await host.dispose(); + const actual = [rendered.getChannelData(0), rendered.getChannelData(1)]; + + let exact = true; + for (let channel = 0; channel < 2; channel += 1) { + for (let frame = 0; frame < LIVE_CONTROL_FRAMES; frame += 1) { + if (!Object.is(actual[channel][frame], expected[channel][frame])) exact = false; + } + } + let increasing = true; + for (let index = 1; index < meterFrames.length; index += 1) { + if (meterFrames[index].sequence <= meterFrames[index - 1].sequence) increasing = false; + } + return { + commandOk: command.ok, + commandPath: command.path, + commandApplied: command.appliedOperations, + committedRevision: command.revision, + meterLeaseResult: meterLease.result, + initialRevision: before.appliedRevision.toString(), + pendingRevision: pending.appliedRevision.toString(), + finalRevision: after.appliedRevision.toString(), + finalSample: after.appliedSample.toString(), + meterFrames: meterFrames.length, + meterFrameWidth: meterFrames.length === 0 ? 0 : meterFrames[0].peaks, + meterTrackCount: meterFrames.length === 0 ? 0 : meterFrames[0].trackCount, + structureEpochs: [...new Set(meterFrames.map((frame) => frame.structureEpoch))], + sequenceIncreasing: increasing, + exactDelayedOutput: exact, + expectedDigest: await pcmDigest(expected, LIVE_CONTROL_FRAMES), + renderedDigest: await pcmDigest(actual, LIVE_CONTROL_FRAMES), + renderAllocations, + }; +} + /// Issue #143 E12: subscribe -> nonzero `trackGrDb` in a real browser, and an unarmed tap publishes none. /// /// Two offline renders of the same sixteen blocks over the same session, differing only in whether @@ -779,6 +886,7 @@ export async function runQualification() { boot: { ready: false, backend: null }, corpus: null, liveControls: null, + structuralApply: null, observation: null, stall: null, stagingReads: null, @@ -821,6 +929,18 @@ export async function runQualification() { name: error?.name, message: error?.message, ...error, diagnostic, })}`); } + let structural; + try { + structural = await runStructuralApplyQualification( + createMisoAudioWorkletHost, + liveControlDocument, + ); + } catch (error) { + const diagnostic = await diagnoseReady(liveControlDocument); + throw new Error(`structural-apply qualification failed: ${diagnosticJson({ + name: error?.name, message: error?.message, ...error, diagnostic, + })}`); + } let observation; try { observation = await runObservationQualification( @@ -866,6 +986,7 @@ export async function runQualification() { freshContextIdentity: digests[0] === digests[1], }, liveControls: live, + structuralApply: structural, observation, stall, stagingReads, @@ -876,6 +997,7 @@ export async function runQualification() { workload: `corpus-${index}-${segment}`, count, }))), { workload: "live-control", count: live.renderAllocations }, + { workload: "structural-apply", count: structural.renderAllocations }, { workload: "observation-armed", count: observation.armed.renderAllocations }, { workload: "observation-disarmed", count: observation.disarmed.renderAllocations }, { workload: "stall", count: stall.renderAllocations }, @@ -905,6 +1027,7 @@ export const qualificationBootContract = Object.freeze({ renderCorpusSegment, typedUnsupportedAttestation, runLiveControlQualification, + runStructuralApplyQualification, runObservationRun, runStallQualification, runStagingReadRun, diff --git a/hosts/host-web/qualification/run.mjs b/hosts/host-web/qualification/run.mjs index f7480e9c8..9cbc154fd 100644 --- a/hosts/host-web/qualification/run.mjs +++ b/hosts/host-web/qualification/run.mjs @@ -24,6 +24,10 @@ const MUTATIONS = [ "attestation", "boot", "native-corpus-digest", "main-thread-stall", // Issue #137 E8/E6: the live-control row and the live-control load carried across the stall. "control-path-applied", "control-path-meter", "control-path-command", "stall-live-controls-load", + // Issue #1562b: a structural apply that was not the rebuild path, that was applied early, that + // moved the output off the declared delay, or whose meter stream was not the successor's. + "structural-apply-path", "structural-apply-early", "structural-apply-output", + "structural-apply-meter", // Issue #143 E12: the observation row. `observation-armed` is the eval's named red mutation -- // a run whose armed tap published nothing, which is exactly what a browser that lost the // transport would produce. @@ -218,6 +222,30 @@ function validate(browserName, result) { gate(browserName, "control-path", live?.telemetryFrames >= 1, "no render-telemetry frame arrived over a full window"); + // Issue #1562b: a structural apply in a real browser. The core classifies a `delay_samples` edit + // as a rebuild and the browser prepares and publishes the replacement plan; the first quantum + // is the swap block, so what comes out is the declared delay of the input, exactly, the applied + // revision was not early, and the meter stream is the successor's from its first frame. + const structural = result.structuralApply; + gate(browserName, "structural-apply-path", structural?.commandOk === true + && structural?.commandPath === "rebuild" && structural?.commandApplied === 1 + && /^[0-9]+$/.test(structural?.committedRevision ?? ""), + "the structural SET_CONTROL transaction was not committed on the rebuild path"); + gate(browserName, "structural-apply-early", structural?.pendingRevision === structural?.initialRevision + && structural?.finalRevision === structural?.committedRevision + && BigInt(structural?.finalRevision ?? "0") > BigInt(structural?.initialRevision ?? "0") + && BigInt(structural?.finalSample ?? "-1") === 0n, + `the applied revision moved before the swap block or did not reach the committed one: ${JSON.stringify(structural)}`); + gate(browserName, "structural-apply-output", structural?.exactDelayedOutput === true + && structural?.renderedDigest === structural?.expectedDigest, + "the swapped plan did not render the input delayed by exactly the edit"); + gate(browserName, "structural-apply-meter", structural?.meterLeaseResult === 0 + && structural?.meterFrames >= 1 && structural?.meterFrameWidth === 4 + && structural?.meterTrackCount === 1 && structural?.sequenceIncreasing === true + && Array.isArray(structural?.structureEpochs) && structural.structureEpochs.length === 1 + && Number.isSafeInteger(structural.structureEpochs[0]) && structural.structureEpochs[0] >= 1, + `the meter stream after the swap was not the successor's with one structure epoch: ${JSON.stringify(structural)}`); + // Issue #143 E12: subscribe -> nonzero `trackGrDb`, an unarmed tap publishes none, with `firstSample` // monotonic and the windows tiling. The two runs render the *same* sixteen blocks, so their // audio must be identical: arming a declared tap may not move a sample. @@ -298,7 +326,7 @@ function validate(browserName, result) { // executor does behind its `call_indirect`, which the static gate cannot see. const allocations = result.renderAllocations; gate(browserName, "render-allocations", Array.isArray(allocations) - && allocations.length === 10 + && allocations.length === 11 && allocations.every((row) => typeof row?.workload === "string" && row.count === 0), `a worklet instance allocated on its render thread: ${JSON.stringify(allocations)}`); if (result.sdkResponse !== null) validateSdkResponse(browserName, result.sdkResponse); @@ -847,6 +875,12 @@ function mutate(result, mutation) { if (mutation === "control-path-meter") copy.liveControls.masterPeak = 0; if (mutation === "control-path-command") copy.liveControls.commandApplied = 0; if (mutation === "stall-live-controls-load") copy.stall.liveControlMeterFrames = 0; + if (mutation === "structural-apply-path") copy.structuralApply.commandPath = "live"; + if (mutation === "structural-apply-early") { + copy.structuralApply.pendingRevision = copy.structuralApply.committedRevision; + } + if (mutation === "structural-apply-output") copy.structuralApply.exactDelayedOutput = false; + if (mutation === "structural-apply-meter") copy.structuralApply.meterFrames = 0; // Issue #143 E12's named red mutation: `observationArmed = 0`. if (mutation === "observation-armed") copy.observation.armed.maximumTrackGrDb = 0; if (mutation === "observation-disarmed") copy.observation.disarmed.maximumTrackGrDb = 1; diff --git a/hosts/host-web/qualification/sdk-response-entry.ts b/hosts/host-web/qualification/sdk-response-entry.ts index 21adafa8f..6d1b4f984 100644 --- a/hosts/host-web/qualification/sdk-response-entry.ts +++ b/hosts/host-web/qualification/sdk-response-entry.ts @@ -2476,6 +2476,26 @@ export async function applyMatrixThroughSdk( return { ok: report.ok, path: report.path, appliedOperations: report.appliedOperations }; } +/** + * One `delay_samples` edit of a track as a `SET_CONTROL` transaction (#1562b): the plan needs a new + * latency, so the core classifies it as a rebuild, prepares a replacement plan and publishes it for + * the render to adopt at a block boundary. Returns the core's report, with the committed revision. + */ +export async function applyDelayThroughSdk( + host: MisoAudioWorkletHost, + trackId: string, + samples: number, +): Promise<{ ok: boolean; path: string; appliedOperations: number; revision: string }> { + const controls = await createBrowserLiveControls(host); + const report = await controls.submit(controls.edit.track(trackId).delaySamples(samples)); + return { + ok: report.ok, + path: report.path, + appliedOperations: report.appliedOperations, + revision: String(report.revision), + }; +} + /** * The track IDs of the host's running session, in the core's own order (the owner headers of its * control list, #1564 D5). The worklet's session map no longer carries them (#1565B), and the diff --git a/hosts/host-web/src/ffi.rs b/hosts/host-web/src/ffi.rs index d434f4b5e..aa87b599c 100644 --- a/hosts/host-web/src/ffi.rs +++ b/hosts/host-web/src/ffi.rs @@ -3213,7 +3213,7 @@ pub extern "C" fn miso_engine_web_v1_source_submit( let id_pointer = ids.as_ptr(); for (channel, slot) in plane_slots[..channel_count].iter_mut().enumerate() { let offset = channel * quantum; - // SAFETY: Preparation allocated `maximum_source_channels * quantum` stable PCM + // SAFETY: Preparation allocated `SOURCE_CHANNELS_MAXIMUM * quantum` stable PCM // samples. The checked channel bound/product and `frames <= quantum` prove this // exact plane prefix is readable for the synchronous source submission below. let plane = unsafe { slice::from_raw_parts(pcm_pointer.add(offset), frame_count) }; @@ -3288,6 +3288,43 @@ pub extern "C" fn miso_engine_web_v1_render(handle: u32, actual_frames: u32) -> }) } +/// Run a host operation that can promote an adopted plan's spectrum captures (`service`, and the +/// reclaim every `control_command` starts with), with the staging's analysis for the promotion to +/// end (#1618). A promotion that does not keep the stream ends the analysis with it; the staging's +/// stream record and staged window then describe a stream that no longer exists, so the record is +/// written from the service and the window is dropped, as `spectrum_stream_stop` does. Without +/// that, `spectrum_stream_analysis` would adopt the stale record again and re-report the window. +/// A staging that is borrowed has no analysis running: the operation gets `None`. +fn with_promotion_analysis( + host: &mut AudioWorkletEngineHost, + operation: impl FnOnce(&mut AudioWorkletEngineHost, Option<&mut SpectrumAnalysis>) -> R, +) -> R { + with_spectrum_staging(|slot| { + let Ok(mut staging) = slot.try_borrow_mut() else { + return operation(host, None); + }; + let staging = &mut *staging; + let was_running = staging + .analysis + .as_ref() + .is_some_and(SpectrumAnalysis::stream_active); + let result = operation(host, staging.analysis.as_mut()); + let running = staging + .analysis + .as_ref() + .is_some_and(SpectrumAnalysis::stream_active); + if was_running && !running { + staging.capture_len = 0; + staging.result_len = 0; + if let Some(metadata) = host.spectrum_stream_metadata().copied() { + write_stream_metadata(&mut staging.stream_metadata, &metadata); + } + staging.stream_metadata.result = RESULT_OK; + } + result + }) +} + /// Run the control plane's service step once (issue #1381a D5, #1348 D1): reclaim every plan and /// attachment render has displaced, in the caller's thread and never in render. Control side /// only: the control Worker, or in `single` mode the worklet's message handler, never `process()`. @@ -3297,7 +3334,9 @@ pub extern "C" fn miso_engine_web_v1_render(handle: u32, actual_frames: u32) -> /// that is not ready; an epoch failure as [`RESULT_INTERNAL`] with the session's diagnostic. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_service(handle: u32) -> u32 { - with_host_mut(handle, RESULT_INVALID_ARGUMENT, |host| host.service()) + with_host_mut(handle, RESULT_INVALID_ARGUMENT, |host| { + with_promotion_analysis(host, AudioWorkletEngineHost::service) + }) } /// Return the control request staging address for a frame of `len` bytes (issue #1562a D1), or @@ -3321,12 +3360,12 @@ pub extern "C" fn miso_engine_web_v1_control_request_ptr(handle: u32, len: u32) /// and a refused transaction is still an ok call with a non-ok frame); [`RESULT_INVALID_ARGUMENT`] /// for a wrong handle, a bad length or a frame the core cannot correlate; /// [`RESULT_WRONG_STATE`]; [`RESULT_BUFFER_TOO_SMALL`]; [`RESULT_BACKPRESSURE`]; -/// [`crate::RESULT_COMPILE_REJECTED`] for a typed refusal before commit (`web.apply.rebuild_pending`); +/// [`crate::RESULT_COMPILE_REJECTED`] for a typed refusal before commit (`web.apply.shape_unavailable`); /// [`RESULT_INTERNAL`]. After any failure the response is the refusal's diagnostic. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_control_command(handle: u32, len: u32) -> u32 { with_host_mut(handle, RESULT_INVALID_ARGUMENT, |host| { - host.control_command(len) + with_promotion_analysis(host, |host, analysis| host.control_command(len, analysis)) }) } @@ -3485,8 +3524,10 @@ pub extern "C" fn miso_engine_web_v1_meter_header_ptr(handle: u32) -> u32 { /// Return the number of prepared resident observation effects in the current owner map. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_observation_count(handle: u32) -> u32 { - with_host(handle, 0, |host| { - u32::try_from(host.observation_binding_count()).unwrap_or(0) + render_locked(|| { + with_host(handle, 0, |host| { + u32::try_from(host.observation_binding_count()).unwrap_or(0) + }) }) } @@ -3535,21 +3576,23 @@ fn copy_observation_id(handle: u32, index: u32, native: bool) -> u32 { /// Copy one bound observation effect's stable local slot ID into the observation-ID staging area. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_observation_effect_slot_id(handle: u32, index: u32) -> u32 { - copy_observation_id(handle, index, false) + render_locked(|| copy_observation_id(handle, index, false)) } /// Copy one bound observation effect's native contract ID into the observation-ID staging area. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_observation_native_effect_id(handle: u32, index: u32) -> u32 { - copy_observation_id(handle, index, true) + render_locked(|| copy_observation_id(handle, index, true)) } /// One bound observation effect's strip index (tracks, then submixes), or zero when out of range. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_observation_track_index(handle: u32, index: u32) -> u32 { - with_host(handle, 0, |host| { - host.observation_binding(index) - .map_or(0, |binding| binding.address.track_index) + render_locked(|| { + with_host(handle, 0, |host| { + host.observation_binding(index) + .map_or(0, |binding| binding.address.track_index) + }) }) } @@ -3557,9 +3600,11 @@ pub extern "C" fn miso_engine_web_v1_observation_track_index(handle: u32, index: /// range. Zero is the retired `simd1` code, so it names no rack (issue #1096). #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_observation_rack(handle: u32, index: u32) -> u32 { - with_host(handle, 0, |host| { - host.observation_binding(index) - .map_or(0, |binding| observation_rack_raw(binding.address.rack)) + render_locked(|| { + with_host(handle, 0, |host| { + host.observation_binding(index) + .map_or(0, |binding| observation_rack_raw(binding.address.rack)) + }) }) } @@ -3567,19 +3612,23 @@ pub extern "C" fn miso_engine_web_v1_observation_rack(handle: u32, index: u32) - /// console slot's index in the session's slot order, or an insert's index. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_observation_effect_index(handle: u32, index: u32) -> u32 { - with_host(handle, 0, |host| { - host.observation_binding(index) - .map_or(0, |binding| binding.address.effect_index) + render_locked(|| { + with_host(handle, 0, |host| { + host.observation_binding(index) + .map_or(0, |binding| binding.address.effect_index) + }) }) } /// Return one bound observation effect's declared tap count, or zero when out of range. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_observation_tap_count(handle: u32, index: u32) -> u32 { - with_host(handle, 0, |host| { - host.observation_binding(index) - .and_then(|binding| u32::try_from(binding.descriptors.len()).ok()) - .unwrap_or(0) + render_locked(|| { + with_host(handle, 0, |host| { + host.observation_binding(index) + .and_then(|binding| u32::try_from(binding.descriptors.len()).ok()) + .unwrap_or(0) + }) }) } @@ -3590,10 +3639,12 @@ pub extern "C" fn miso_engine_web_v1_observation_tap_id( index: u32, tap_index: u32, ) -> u32 { - with_host(handle, 0, |host| { - host.observation_binding(index) - .and_then(|binding| binding.descriptors.get(tap_index as usize)) - .map_or(0, |tap| tap.id.0) + render_locked(|| { + with_host(handle, 0, |host| { + host.observation_binding(index) + .and_then(|binding| binding.descriptors.get(tap_index as usize)) + .map_or(0, |tap| tap.id.0) + }) }) } @@ -3787,35 +3838,41 @@ pub extern "C" fn miso_engine_web_v1_observation_result_bytes() -> u32 { /// lower bound: an export that appears or disappears fails that gate. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_source_count(handle: u32) -> u32 { - with_host(handle, 0, |host| { - u32::try_from(host.session_source_count()).unwrap_or(0) + render_locked(|| { + with_host(handle, 0, |host| { + u32::try_from(host.session_source_count()).unwrap_or(0) + }) }) } /// Copy one canonical source ID into the ID staging buffer; returns its byte length. /// -/// Zero means "no such source". [`BUFFER_SOURCE_ID`] is sized for the longest source or track ID -/// in the compiled session. +/// Zero means "no such source". [`BUFFER_SOURCE_ID`] is sized for the longest ID the grammar +/// allows, so no session a rebuild commits outgrows it. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_source_id(handle: u32, index: u32) -> u32 { - with_host_mut(handle, 0, |host| host.copy_session_source_id(index)) + render_locked(|| with_host_mut(handle, 0, |host| host.copy_session_source_id(index))) } /// Return one source's declared channel count, or zero for an out-of-range index. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_source_channels(handle: u32, index: u32) -> u32 { - with_host(handle, 0, |host| { - host.session_source_shape(index) - .map_or(0, |shape| shape.channel_count) + render_locked(|| { + with_host(handle, 0, |host| { + host.session_source_shape(index) + .map_or(0, |shape| shape.channel_count) + }) }) } /// Return one source's exact length in source sample frames, or zero out of range. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_source_frames(handle: u32, index: u32) -> u64 { - with_host(handle, 0, |host| { - host.session_source_shape(index) - .map_or(0, |shape| shape.frames) + render_locked(|| { + with_host(handle, 0, |host| { + host.session_source_shape(index) + .map_or(0, |shape| shape.frames) + }) }) } @@ -3979,6 +4036,17 @@ pub mod native_staging { }) } + /// Read back the first `length` bytes of the source-ID staging, as the worklet reads them after + /// `miso_engine_web_v1_source_id` returns that length. `None` when the handle owns no staging + /// or `length` exceeds it. + pub fn staged_source_id(handle: u32, length: u32) -> Option> { + super::with_host_mut(handle, None, |host| { + host.source_id_mut() + .and_then(|staging| staging.get(..length as usize)) + .map(<[u8]>::to_vec) + }) + } + /// Fill the planar source-PCM staging with `value`, as the worklet copies a chunk's planes /// through `miso_engine_web_v1_buffer_ptr(handle, BUFFER_SOURCE_PCM)`. pub fn source_pcm(handle: u32, value: f32) -> bool { @@ -3987,19 +4055,6 @@ pub mod native_staging { }) } - /// Run one structural command frame through the control plane of the host `handle` names - /// (#1381a gates 2 and 4), which publishes the successor plan. The result is the control - /// plane's refusal rendered as its debug text, or `None` for an unknown handle. - #[cfg(feature = "test-support")] - pub fn structural_command(handle: u32, request: &[u8]) -> Option, String>> { - super::with_host_mut(handle, None, |host| { - Some( - host.structural_command_for_test(request) - .map_err(|error| format!("{error:?}")), - ) - }) - } - /// Stage one control frame as the worklet does through `miso_engine_web_v1_control_request_ptr`, /// run `miso_engine_web_v1_control_command` and read the response through the response /// exports' record (issue #1562a D1). Returns the export's result and the response bytes (the @@ -5691,6 +5746,176 @@ mod spectrum_ffi_tests { assert_eq!(miso_engine_web_v1_dispose(handle), RESULT_OK); } + /// The nine-track fixture with a muted copy of its first track (whose ID sorts first) and the + /// copy's route to the output, as one rebuild transaction at `revision` (#1618, m7). + fn adding_a_strip_frame(request: u64, revision: u64) -> Vec { + use protocol::{ + CommandPayload, ExpectedRevision, ProtocolCodec, RequestId, SessionEdit, + SessionRevision, TypedCommandFrame, + }; + let model = session::parse_session_json(include_str!( + "../../../fixtures/session/v1/parametric-eq-nine-track.json" + )) + .expect("the fixture parses"); + let mut track = model.tracks[0].clone(); + track.id = session::StableId::parse("a-muted").expect("track ID"); + track.fader.left_mute = true; + track.fader.right_mute = true; + let mut route = model.routes[0].clone(); + route.id = session::StableId::parse("a-muted-main").expect("route ID"); + let session::RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("a track route") + }; + *track_id = track.id.clone(); + let edits = [ + SessionEdit::UpsertTrack { track }, + SessionEdit::UpsertRoute { route }, + ]; + let codec = ProtocolCodec::default(); + let mut bytes = vec![0_u8; codec.limits().max_frame_bytes]; + let length = codec + .encode_command_frame_into( + &TypedCommandFrame { + request_id: RequestId::new(request).expect("nonzero request"), + expected_revision: ExpectedRevision::Exact(SessionRevision(revision)), + payload: CommandPayload::SessionTransactionApply(&edits), + }, + &mut bytes, + ) + .expect("a transaction frame"); + bytes.truncate(length); + bytes + } + + /// A rebuild whose promotion does not keep the spectrum stream ends the instance's analysis + /// with it (#1618, m1 and m7 of the batch 12b review), whichever control export runs the + /// promotion: the service step, or the reclaim a control command starts with. The swap carries + /// (the engine reports it so) but its pairing does not name the capture, so the successor's + /// fresh capture is installed and the service's stream ends. + /// + /// Test value: red if `miso_engine_web_v1_service` or `miso_engine_web_v1_control_command` + /// promotes without the staging's analysis: the stream ends but the analysis stays active, so + /// `spectrum_stream_reset` answers ok instead of a wrong state and `spectrum_stream_analysis` + /// re-analyzes the last window of a stream that no longer exists. + #[test] + fn a_promotion_that_drops_the_stream_ends_the_analysis() { + for by_service in [true, false] { + stage_left_output_request(); + let document = + include_str!("../../../fixtures/session/v1/parametric-eq-nine-track.json"); + let handle = test_boot( + document.as_bytes(), + WebBootOptions { + require_sample_rate_hz: 48_000, + require_quantum_frames: 128, + live_control_meter_blocks: 2, + live_control_observation_taps: 4, + ..WebBootOptions::explicit_defaults() + }, + ); + assert_ne!(handle, 0, "spectrum fixture must boot"); + assert_eq!( + miso_engine_web_v1_spectrum_stream_start(handle, 0.0), + RESULT_OK + ); + assert_eq!( + test_copy_staging(handle, BUFFER_SOURCE_ID, b"fixture-source"), + RESULT_OK + ); + assert_eq!(test_fill_source_pcm(handle, 0.25), RESULT_OK); + let mut block = 0_u64; + let mut render = |count: u64| { + for _ in 0..count { + assert_eq!( + miso_engine_web_v1_source_submit(handle, 14, 1, block * 128, 2, 128, 0), + RESULT_OK, + "source block {block}" + ); + assert_eq!(miso_engine_web_v1_render(handle, 128), RESULT_OK); + block += 1; + } + }; + render(16); + assert_eq!(miso_engine_web_v1_spectrum_stream_read(handle), RESULT_OK); + assert_eq!(miso_engine_web_v1_spectrum_stream_analysis(), RESULT_OK); + assert!(with_spectrum_staging(|slot| { + slot.borrow() + .analysis + .as_ref() + .is_some_and(SpectrumAnalysis::stream_active) + })); + + // A rebuild commits, and the block after it adopts the successor plan. + let revision = with_host(handle, 0, AudioWorkletEngineHost::committed_revision); + let (result, _) = + native_staging::control_command(handle, &adding_a_strip_frame(1, revision)) + .expect("the control staging"); + assert_eq!(result, RESULT_OK); + render(2); + // The pairing names nothing: the swap carried, but not this capture. + with_host_mut(handle, (), |host| { + let ready = host.ready.as_mut().expect("ready"); + let waiting = ready + .pending_shapes + .iter_mut() + .filter(|entry| entry.adoption == Some(true)) + .filter_map(|entry| entry.spectrum.as_mut()) + .map(|handover| handover.pairing = Box::default()) + .count(); + assert_eq!(waiting, 1, "the adopted plan waits for its promotion"); + }); + if by_service { + assert_eq!(miso_engine_web_v1_service(handle), RESULT_OK); + } else { + // Any control command reclaims first; a revision probe is the lightest. + let probe = { + use protocol::{ + CommandPayload, ExpectedRevision, ProtocolCodec, RequestId, + TypedCommandFrame, + }; + let codec = ProtocolCodec::default(); + let mut bytes = vec![0_u8; codec.limits().max_frame_bytes]; + let length = codec + .encode_command_frame_into( + &TypedCommandFrame { + request_id: RequestId::new(2).expect("nonzero request"), + expected_revision: ExpectedRevision::Any, + payload: CommandPayload::CapabilitiesGet, + }, + &mut bytes, + ) + .expect("a capabilities frame"); + bytes.truncate(length); + bytes + }; + let (result, _) = + native_staging::control_command(handle, &probe).expect("the control staging"); + assert_eq!(result, RESULT_OK); + } + + assert!( + with_spectrum_staging(|slot| { + slot.borrow() + .analysis + .as_ref() + .is_some_and(|analysis| !analysis.stream_active()) + }), + "the analysis ended with the stream (by_service = {by_service})" + ); + assert_eq!( + miso_engine_web_v1_spectrum_stream_reset(), + RESULT_WRONG_STATE, + "there is no stream to reset (by_service = {by_service})" + ); + assert_ne!( + miso_engine_web_v1_spectrum_stream_analysis(), + RESULT_OK, + "the analysis does not re-report the last window (by_service = {by_service})" + ); + assert_eq!(miso_engine_web_v1_dispose(handle), RESULT_OK); + } + } + /// Issue #1592 D4. Turns red if a stream start on a collection discards a caller's arm (the /// browser's old `arm`, then `stream_start`), or no longer replaces the arm that a select made. #[test] diff --git a/hosts/host-web/src/lib.rs b/hosts/host-web/src/lib.rs index b2cec073e..7f5dddaad 100644 --- a/hosts/host-web/src/lib.rs +++ b/hosts/host-web/src/lib.rs @@ -29,7 +29,6 @@ use core::mem::{MaybeUninit, size_of}; use core::num::{NonZeroU32, NonZeroUsize}; -use core::sync::atomic::{AtomicBool, Ordering}; use std::sync::{Arc, Mutex}; use builtins::{MeterMetricSet, MeterSnapshot, MeterTap}; @@ -58,7 +57,7 @@ use host_core::{ }; #[cfg(test)] use host_core::{SpectrumCaptureCollectionRequest, SpectrumCaptureRequest}; -use session::CompileCaps; +use session::{CompileCaps, SOURCE_CHANNELS_MAXIMUM, STABLE_ID_MAXIMUM_BYTES}; use host_core::EffectRack; /// Live effect addressing in the session's own terms (decision 12, issue #1096). @@ -976,6 +975,15 @@ pub struct WebMeterHeader { /// header and its frame in the message handler that commits the plan, so the render path /// switches views by reference and never cuts one (issue #1290d1). pub successor_pointer: u32, + /// The structure epoch of the control list this plan's strip order belongs to (issue #1562b): + /// the `structure_epoch` that `CONTROL_LIST_GET` pages carry, as the commit that prepared the + /// plan left it, and as each later commit that rode the plan's live lanes left it. Strips + /// (tracks, then submixes) are listed in the order the frame carries them, so a client that + /// holds a list of another epoch reads the list again before it names a frame's strips. A + /// plan that renders keeps the epoch of its own commit while a replacement waits: the + /// replacement's header carries the newer one, and the page sees it in the adoption block. + /// Appended in place, so every earlier offset is unmoved. + pub structure_epoch: u64, } /// `WebMeterHeader::reserved[1]` validity bits. The high 32 bits carry a saturating loss count. @@ -1015,6 +1023,7 @@ const fn empty_meter_header() -> WebMeterHeader { retired: 0, frame_pointer: 0, successor_pointer: 0, + structure_epoch: 0, } } @@ -1429,6 +1438,34 @@ struct PendingShape { /// Whether the adoption block has passed: the entry then holds tables nothing reads, which /// the service step drops on the control side. adopted: bool, + /// What the block that adopted this very plan took from its predecessor, set by that block + /// and `None` before it and for a candidate a later one superseded (#1290d2): `Some(true)` + /// when the engine reported the hand-over carried, `Some(false)` for a refused or unrequested + /// one. The promotion in [`ReadyOwnership::promote_spectrum`] reads it once. + adoption: Option, + /// The plan's prepared spectrum captures and the pairing list beside them, until the plan's + /// promotion installs them in the spectrum service (#1290d2); dropped with the entry when the + /// plan was superseded and never rendered. + spectrum: Option, +} + +/// The spectrum captures a replacement plan was prepared with, and the pairing that names the +/// observers its swap block moves (#1395; #1290d2). The captures are control-side objects: the +/// plan holds the observers, the service holds the readers, so a plan's captures never ride in +/// its attachment and cannot be lost with a retired plan. +struct SpectrumHandover { + captures: PreparedSpectrumCaptures, + /// [`host_core::PlanStateInventory::spectrum_pairing`] of the plan, copied at preparation. + pairing: Box<[host_core::SpectrumPair]>, +} + +/// The heap a handover's pairing list holds, charged to the plan's rows with its tables. A free +/// function: an inherent impl before the host's own would renumber the mangled render symbols the +/// call-graph gate pins. +fn spectrum_pairing_bytes(handover: &SpectrumHandover) -> Option { + u64::try_from(handover.pairing.len()) + .ok()? + .checked_mul(u64::try_from(size_of::()).ok()?) } /// Everything one compiled session owns on the browser side. @@ -1443,9 +1480,6 @@ struct ReadyOwnership { /// and effect producers, the plan's revision gate, the plan publisher and the retirer. It is /// the one owner of the producers; the browser holds none of its own. control: WebSession, - /// The preparer's rebuild switch (#1562a D9), for the test-support republish. - #[cfg(feature = "test-support")] - allow_rebuild: Arc, /// The limits the control half was built with (#1570b D6), for the cross-host proof. #[cfg(feature = "test-support")] limits: ControlLimits, @@ -1457,6 +1491,9 @@ struct ReadyOwnership { shape: Box, /// The epoch of the plan `shape` describes. shape_epoch: u64, + /// The plan owner's count of carried swaps as of the last adoption block the host handled + /// (#1290d2): a swap carried exactly when the count moved. + carried_swaps_seen: u64, /// Where the preparer leaves a replacement plan's tables (#1290a). record: Arc>, /// The tables of replacement plans between their commit and the service step after their @@ -1542,8 +1579,9 @@ impl ReadyOwnership { &mut self, request: &[u8], output_capacity: u64, + analysis: Option<&mut SpectrumAnalysis>, ) -> Result { - self.reclaim_shapes(); + self.reclaim_shapes(analysis); if self.pending_shapes.len() == self.pending_shapes.capacity() { return Err(control_plane::CommandError::Backpressure); } @@ -1551,11 +1589,9 @@ impl ReadyOwnership { let result = self.control.command(request, output_capacity); // Whatever the preparer left is taken: a command refused after its preparation leaves // tables nothing will read. - let prepared = self - .record - .lock() - .ok() - .and_then(|mut record| record.shape.take()); + let (prepared, handover) = self.record.lock().map_or((None, None), |mut record| { + (record.shape.take(), record.spectrum.take()) + }); let epoch = self.control.newest_plan_epoch(); if result.is_ok() && epoch != before { // A rebuild commits only after the preparer returned `Ok`, and the preparer returns @@ -1567,6 +1603,8 @@ impl ReadyOwnership { epoch, shape: Box::new(shape), adopted: false, + adoption: None, + spectrum: handover, }); // The rendering plan's header names the newest replacement's, so the page cuts its // views over the successor's frame in the handler that committed it (#1290d1). @@ -1576,21 +1614,75 @@ impl ReadyOwnership { } self.blocking_rebuilds = self.blocking_rebuilds.saturating_add(1); } + if result.is_ok() { + self.stamp_structure_epoch(); + } result } + /// Stamp the structure epoch the control list holds now on the newest committed plan's meter + /// header (#1562b): the replacement's while one waits, else the rendering plan's. A commit that + /// rode the newest plan's live lanes may have changed a descriptor or an address (a VCA added, + /// a `matrix_mode` switch), and a rebuild's commit changes the strip order. Control side only. + fn stamp_structure_epoch(&mut self) { + let epoch = self.control.structure_epoch(); + let header = match self + .pending_shapes + .iter_mut() + .rev() + .find(|entry| !entry.adopted) + { + Some(entry) => &mut entry.shape.meter_header, + None => &mut self.shape.meter_header, + }; + header.structure_epoch = epoch; + } + /// Drop the tables of plans render has adopted past (#1290a): the host's own, retired by the /// adoption block, and those of a candidate that a later one superseded. Control side only. - fn reclaim_shapes(&mut self) { + fn reclaim_shapes(&mut self, analysis: Option<&mut SpectrumAnalysis>) { + self.promote_spectrum(analysis); self.pending_shapes.retain(|entry| !entry.adopted); } + /// Promote the plans the render half has adopted (#1290d2; #1395 D3, D5): install each + /// adopted plan's spectrum captures in the service, paired with the service's current ones + /// by the plan's pairing list and the engine's carry outcome for that swap, and drop the + /// captures it displaces here, off the audio thread. Entries go in commit order, so two + /// adoptions before one service step promote one after the other, each against the captures + /// the one before left. A plan that was superseded before it rendered was never adopted + /// (`adoption` is `None`) and its captures drop with its entry. Control side only: it frees. + /// + /// `analysis` is the smoothing history of the instance that owns both halves (the ffi staging's + /// analysis, #1618): a promotion that does not keep the stream ends the stream's analysis in + /// the same call, so the analysis never reports facts of a stream that ended. The browser's + /// audio thread, which keeps none, passes `None`. + fn promote_spectrum(&mut self, mut analysis: Option<&mut SpectrumAnalysis>) { + for entry in &mut self.pending_shapes { + let (Some(carried), Some(handover)) = (entry.adoption, entry.spectrum.take()) else { + continue; + }; + let Some(service) = self.spectrum.as_mut() else { + continue; + }; + drop(service.adopt_successor( + handover.captures, + &handover.pairing, + carried, + analysis.as_deref_mut(), + )); + } + } + // REALTIME_POLICY_BEGIN /// Take the tables of the plan whose epoch is `epoch` (#1290a), at the block that adopted it. /// The tables it retires stay in the entry for [`Self::reclaim_shapes`]; an entry of an /// earlier epoch was superseded and is marked the same way. A pointer exchange: it allocates /// and frees nothing, and indexes nothing, so it adds no trap to the render export. /// + /// `swap_carried` is whether the engine reported this swap's hand-over as carried (the owner's + /// carried-swap count advanced), which the plan's spectrum promotion needs (#1290d2). + /// /// `carried` is the pairing the swap block's attachment hook made (#1381b): the readers the /// executors carried, which this hand-over follows with the per-reader state the tables hold. /// A swap that carried a meter keeps the meter delivery running, because the carried meters @@ -1605,7 +1697,7 @@ impl ReadyOwnership { /// the published ones from this block on. The retiring header is marked /// ([`WebMeterHeader::retired`]) and names the adopted one, so the page's render path switches /// to the views it cut over them in advance. Pointer exchanges and scalar writes only. - fn adopt_shape(&mut self, epoch: u64, carried: &[CarriedReader]) { + fn adopt_shape(&mut self, epoch: u64, swap_carried: bool, carried: &[CarriedReader]) { self.shape_epoch = epoch; let meters_carried = carried .iter() @@ -1626,6 +1718,7 @@ impl ReadyOwnership { if entry.epoch == epoch { core::mem::swap(&mut self.shape, &mut entry.shape); entry.adopted = true; + entry.adoption = Some(swap_carried); taken = true; // `self.shape` is the adopted plan's and `entry.shape` the retired one's. The // stream the page reads continues in the adopted plan's header (#1290d1): its @@ -2033,11 +2126,6 @@ impl AudioWorkletEngineHost { document_bytes, shape.sample_rate_hz, shape.quantum_frames, - shape.maximum_source_channels, - shape - .longest_source_id_bytes - .max(shape.longest_track_id_bytes) - .max(shape.longest_submix_id_bytes), options, ( spectrum_request.is_some(), @@ -2172,32 +2260,6 @@ impl AudioWorkletEngineHost { .get(reader as usize) } - /// Run one structural command frame through the control plane (#1381a gates 2 and 4, #1290a): - /// it prepares a successor of the committed model through the browser's preparer and publishes - /// it, for the render of the next block to adopt, with the rebuild arm open past - /// [`REBUILD_PENDING`]. No export reaches it: the production entry keeps the refusal until - /// #1562b deletes it. - /// - /// Returns the control plane's response frame. - /// - /// # Errors - /// - /// The control plane's refusal of the frame. - #[cfg(feature = "test-support")] - pub fn structural_command_for_test( - &mut self, - request: &[u8], - ) -> Result, control_plane::CommandError> { - let ready = self - .ready - .as_mut() - .ok_or(control_plane::CommandError::Internal)?; - ready.allow_rebuild.store(true, Ordering::Release); - let length = ready.run_command(request, CONTROL_FRAME_BYTES); - ready.allow_rebuild.store(false, Ordering::Release); - Ok(ready.control.command_response(length?).to_vec()) - } - /// The [`ControlLimits`] the control half was built with (#1570b D6), read-only: the cross-host /// proof builds the C ABI's compile limits from it field for field, so both hosts advertise the /// same capabilities. @@ -2245,17 +2307,21 @@ impl AudioWorkletEngineHost { /// length that is zero or over the frame ceiling, or a frame the control plane cannot /// correlate; [`RESULT_BUFFER_TOO_SMALL`] for a response over the control frame; /// [`RESULT_BACKPRESSURE`] for a queue or plan with no room now; [`RESULT_COMPILE_REJECTED`] - /// for a typed refusal before commit, `web.apply.rebuild_pending` for a transaction that needs - /// a plan rebuild (D9); [`RESULT_REPREPARE_REQUIRED`] for a transaction that changes the + /// for a typed refusal before commit (a transaction that needs a plan rebuild is prepared and + /// published like any other, #1562b); [`RESULT_REPREPARE_REQUIRED`] for a transaction that changes the /// session's sample rate or quantum (#1290a: the session is re-prepared from its document); /// [`RESULT_INTERNAL`] for a bookkeeping failure. For each failure the /// response is the refusal's diagnostic (`code\tpath\n` lines). /// - /// A frame refused after the core has seen it, `web.apply.rebuild_pending` included, has - /// consumed its request ID: the same bytes sent again are answered [`RESULT_OK`] with a non-ok - /// response frame whose code is `protocol.failure` (the replay cache), and a retry must take a - /// new request ID to meet the refusal again. - pub fn control_command(&mut self, len: u32) -> u32 { + /// A frame refused after the core has seen it has consumed its request ID: the same bytes sent + /// again are answered [`RESULT_OK`] with a non-ok response frame whose code is + /// `protocol.failure` (the replay cache), and a retry must take a new request ID to meet the + /// refusal again. + /// + /// `analysis` is the smoothing history of an instance that owns both halves of the spectrum + /// stream (see [`Self::service`]); a plan this command's reclaim promotes ends it when the + /// stream is not kept. + pub fn control_command(&mut self, len: u32, analysis: Option<&mut SpectrumAnalysis>) -> u32 { self.control_response = ControlResponse::None; if self.status.state != STATE_READY { return self.record(RESULT_WRONG_STATE); @@ -2267,7 +2333,7 @@ impl AudioWorkletEngineHost { return self.fail(RESULT_INTERNAL, b"web.internal.ready\t$\n"); }; let request = &buffers.control_request[..len as usize]; - match ready.run_command(request, CONTROL_FRAME_BYTES) { + match ready.run_command(request, CONTROL_FRAME_BYTES, analysis) { Ok(length) => { self.control_response = ControlResponse::Frame(length); self.record(RESULT_OK) @@ -2404,7 +2470,11 @@ impl AudioWorkletEngineHost { /// so a periodic tick never overwrites a refusal the host has yet to read. /// /// An epoch failure is sticky: the host fails with the session's diagnostic. - pub fn service(&mut self) -> u32 { + /// + /// `analysis` is the smoothing history of an instance that owns both halves of the spectrum + /// stream, which a promotion this step performs ends when it does not keep the stream (#1618); + /// the browser's audio thread passes `None`. + pub fn service(&mut self, analysis: Option<&mut SpectrumAnalysis>) -> u32 { if self.status.state != STATE_READY { return RESULT_WRONG_STATE; } @@ -2413,7 +2483,7 @@ impl AudioWorkletEngineHost { }; match ready.control.service() { Ok(()) => { - ready.reclaim_shapes(); + ready.reclaim_shapes(analysis); RESULT_OK } Err(error) => { @@ -3123,6 +3193,9 @@ impl AudioWorkletEngineHost { } /// Copy one canonical source ID into ID staging; returns its byte length (issue #207). + /// + /// Zero means "no such source", and also a staging that cannot hold the ID: staging is sized + /// for the longest ID the grammar allows, so the second cannot happen for a compiled session. pub(crate) fn copy_session_source_id(&mut self, index: u32) -> u32 { let Some(ready) = self.ready.as_ref() else { return 0; @@ -3136,17 +3209,21 @@ impl AudioWorkletEngineHost { return 0; }; Self::copy_id_into_staging(self.buffers.as_mut(), source.id.as_str().as_bytes()) - } - - fn copy_id_into_staging(buffers: Option<&mut PreparedBuffers>, bytes: &[u8]) -> u32 { - let length = bytes.len(); - let buffers = buffers.expect("a ready host owns its staging buffers"); - debug_assert!( - length <= buffers.source_id.len(), - "compiled ID exceeds its projected staging capacity" - ); - buffers.source_id[..length].copy_from_slice(bytes); - u32::try_from(length).expect("ID staging capacity is representable as u32") + .unwrap_or(0) + } + + /// Copy `bytes` into ID staging and return their length, or why they cannot be staged. Never + /// panics: this runs inside a render-locked export, where a panic is a trap. + fn copy_id_into_staging( + buffers: Option<&mut PreparedBuffers>, + bytes: &[u8], + ) -> Result { + let buffers = buffers.ok_or(IdStagingError::NoStaging)?; + let Some(target) = buffers.source_id.get_mut(..bytes.len()) else { + return Err(IdStagingError::TooLong); + }; + target.copy_from_slice(bytes); + u32::try_from(bytes.len()).map_err(|_| IdStagingError::TooLong) } pub(crate) fn diagnostic_buffer_mut(&mut self) -> Option<&mut [u8]> { @@ -3296,11 +3373,16 @@ impl AudioWorkletEngineHost { // drops the retired ones at its next step. let epoch = plan.owner().active_epoch().0; if epoch != ready.shape_epoch { + // #1290d2: whether the engine carried this swap, from the owner's count of + // carried swaps (a refused or unrequested hand-over does not advance it). + let carried_swaps = plan.owner().carried_count(); + let swap_carried = carried_swaps != ready.carried_swaps_seen; + ready.carried_swaps_seen = carried_swaps; // #1381b: what the swap block's hook paired travels with the tables. let carried = plan .active_attachment_mut::() .map_or(&[][..], |companions| companions.carried()); - ready.adopt_shape(epoch, carried); + ready.adopt_shape(epoch, swap_carried, carried); } self.status.next_absolute_sample = next_absolute_sample; self.status.rendered_quanta = self.status.rendered_quanta.saturating_add(1); @@ -4601,15 +4683,28 @@ fn spectrum_collection_staging_bytes( Ok((entry_bytes, target_id_bytes)) } +/// Why an ID could not be copied into ID staging. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum IdStagingError { + /// The host owns no staging buffers. + NoStaging, + /// The ID is longer than the staging buffer. + TooLong, +} + fn project_buffers( document_bytes: u32, sample_rate_hz: u32, quantum_frames: u32, - maximum_source_channels: u32, - id_staging_bytes: u64, options: WebBootOptions, spectrum_staging: (bool, (u64, u64)), ) -> Result { + // The staging holds what a rebuild may add, not only what the boot session declares (#1562b): + // a source, track or submix ID of the longest length the grammar allows, and a source of the + // most channels it allows. Sizing from the boot session would let an acked rebuild declare a + // source the host could never stage or feed. + let maximum_source_channels = SOURCE_CHANNELS_MAXIMUM; + let id_staging_bytes = STABLE_ID_MAXIMUM_BYTES as u64; let ( spectrum_configured, (spectrum_collection_entry_bytes, spectrum_collection_target_id_bytes), @@ -5122,21 +5217,21 @@ type WebSession = SessionState; type WebPlan = PlanState; /// The render-side halves a browser plan's preparation returns beside the plan (issue #1400 D5): -/// the meter consumers, the effect observation handles and the spectrum capture. +/// the meter consumers and the effect observation handles. /// /// They travel inside the plan as its [`HostAttachment`], so a plan the control plane prepares /// carries everything the render side reads, and they drop with the plan on the retirement path. /// The browser renders through the plan owner (#1381a D4) and reaches the active plan's companions /// through `PlanState::active_attachment_mut`: the meter poll and the observation reads read the -/// readers there. The spectrum capture is the one companion the host does not read there: -/// [`SpectrumService`] owns its captures, so boot moves the capture out into it -/// (`take_spectrum_capture`) until the spectrum readers move to the control plane (#1596). +/// readers there. A plan's spectrum captures are not among them: [`SpectrumService`] owns the +/// captures, so the preparer leaves a plan's in the `BootRecord` (boot moves them into the +/// service; a replacement's wait beside its tables for the promotion, #1290d2) until the spectrum +/// readers move to the control plane (#1596). /// -/// Field order is the drop order: the spectrum capture, then the meter consumers, then the effect -/// observation handles, the relative order the same three fields have in `ReadyOwnership`. No one -/// of them reads another while it drops; the order is kept so the two owners release alike. +/// Field order is the drop order: the meter consumers, then the effect observation handles, the +/// relative order the same two fields have in `ReadyOwnership`. Neither reads the other while it +/// drops; the order is kept so the two owners release alike. pub struct RenderCompanions { - spectrum_capture: Option, /// One consumer per selected meter, in strip order; empty when metering is off. meters: Vec, /// One reader set per observed effect instance, in `HostLiveControlHandles` order. @@ -5195,7 +5290,6 @@ impl RenderCompanions { // Every field is named, so a field added to the struct fails to compile here until its // heap is charged (or declared host-core's, as the capture is). let Self { - spectrum_capture: _, meters, effect_observations, carried, @@ -5217,18 +5311,6 @@ impl RenderCompanions { .checked_add(carried)?; Some((total, shell.max(meters).max(observations).max(carried))) } - - /// Whether the plan was prepared with a spectrum capture (one, or a collection). - #[must_use] - pub fn has_spectrum_capture(&self) -> bool { - self.spectrum_capture.is_some() - } - - /// Move the spectrum capture out, once, to the host's spectrum service (#1381a). Control side - /// only: boot calls it before the first render. - fn take_spectrum_capture(&mut self) -> Option { - self.spectrum_capture.take() - } } #[cfg(feature = "test-support")] @@ -5266,6 +5348,10 @@ struct BootRecord { /// plan's and every replacement plan's. Boot takes the first; a command that publishes a /// replacement takes that plan's, and one that fails after preparing leaves a table to drop. shape: Option, + /// The spectrum captures of the plan the preparer prepared last, with its pairing list + /// (#1290d2). Boot moves them into the spectrum service; a command that publishes a + /// replacement keeps them beside the plan's tables until the plan's promotion. + spectrum: Option, } impl RenderCompanions { @@ -5357,10 +5443,6 @@ impl HostAttachment for RenderCompanions { /// record another thread poisoned): refused before the transaction is committed (#1290a). const SHAPE_UNAVAILABLE: &str = "web.apply.shape_unavailable"; -/// The diagnostic code of a command frame whose transaction needs a plan rebuild, which the -/// browser cannot apply in place yet (issue #1562a D9; #1562b removes the refusal). -pub const REBUILD_PENDING: &str = "web.apply.rebuild_pending"; - /// The browser's [`control_plane::RuntimePreparer`] (issue #1400 D5): `prepare_web_host` for the /// session's boot options and spectrum request, with the render-side companions moved into the /// plan. @@ -5373,11 +5455,6 @@ pub struct WebRuntimePreparer { /// Where the boot preparation leaves its facts for the host (#1381a), or `None` for a preparer /// that is only called. record: Option>>, - /// Whether the command path takes a structural transaction (issue #1562a D9). It is `false`: - /// the browser has no in-place structural edit until #1562b, so a frame that classifies as a - /// rebuild is refused `web.apply.rebuild_pending` before anything is committed. Only the - /// test-support republish of #1381a's gates opens it, for one call. - allow_rebuild: Arc, } impl WebRuntimePreparer { @@ -5388,7 +5465,6 @@ impl WebRuntimePreparer { options, spectrum, record: None, - allow_rebuild: Arc::new(AtomicBool::new(false)), } } @@ -5400,10 +5476,6 @@ impl WebRuntimePreparer { } impl control_plane::RuntimePreparer for WebRuntimePreparer { - fn refuse_rebuild(&self) -> Option<&'static str> { - (!self.allow_rebuild.load(Ordering::Acquire)).then_some(REBUILD_PENDING) - } - fn prepare( &self, compiled: &CompiledSession, @@ -5446,7 +5518,12 @@ impl WebRuntimePreparer { effects: lanes.effects || live_controls.observation_taps != 0, ..lanes }; - let (mut host, mut handles, spectrum_capture) = prepare_web_host( + // #1584 D6b: the taps the host had armed, which host-core arms the successor's lanes from; + // the plan's tables must say the same. + let bindings: &[controls::ObservationBinding] = successor + .map(|base| base.observation.bindings()) + .unwrap_or_default(); + let (mut host, mut handles, spectrum_captures) = prepare_web_host( compiled, caps, &live_controls, @@ -5465,6 +5542,7 @@ impl WebRuntimePreparer { &handles.effect_observations, handles.meters.len(), handles.master_track, + bindings, ) .map_err(|code| { let rejection = if code.starts_with("web.resource.") { @@ -5486,8 +5564,25 @@ impl WebRuntimePreparer { meters.len() + effect_observations.len(), CarriedReader::UNUSED, ); + // #1290d2: the captures and the pairing that names the observers this plan's swap block + // moves leave with the tables, so they can be installed in the service at the plan's + // promotion whatever the plan's attachment has become by then. + let spectrum = match spectrum_captures { + Some(captures) => { + let pairs = host.inventory.spectrum_pairing(); + let mut pairing: Vec = Vec::new(); + pairing.try_reserve_exact(pairs.len()).map_err(|_| { + PrepareDiagnostics::fixed(PrepareRejection::Platform, "web.resource.allocation") + })?; + pairing.extend_from_slice(pairs); + Some(SpectrumHandover { + captures, + pairing: pairing.into_boxed_slice(), + }) + } + None => None, + }; let companions = RenderCompanions { - spectrum_capture, meters, effect_observations, carried: carried.into_boxed_slice(), @@ -5503,6 +5598,10 @@ impl WebRuntimePreparer { let (tables, tables_largest) = shape.retained_bytes().ok_or_else(arithmetic)?; retained = retained.checked_add(tables).ok_or_else(arithmetic)?; largest = largest.max(tables_largest); + if let Some(pairing) = spectrum.as_ref().and_then(spectrum_pairing_bytes) { + retained = retained.checked_add(pairing).ok_or_else(arithmetic)?; + largest = largest.max(pairing); + } } charge_attachment(&mut host.report, caps, retained, largest)?; host.plan.set_host_attachment(Box::new(companions)); @@ -5518,6 +5617,7 @@ impl WebRuntimePreparer { record.superseded = Some(handles.live_values_superseded.clone()); } record.shape = Some(shape); + record.spectrum = spectrum; } Ok((host, handles)) } @@ -5534,6 +5634,11 @@ impl WebRuntimePreparer { /// model's, the order host-core's addressing authority uses and the control plane's producers /// follow. /// +/// `bindings` are the taps the host had armed when this plan was prepared (#1584 D6b): the same +/// table host-core armed the plan's lanes from, so a bound tap's `observation_armed` bit is set +/// here with arm sample 0 and its commanded bit clear (a fresh tap has no stale window). A binding +/// whose effect or tap the plan lacks sets nothing, as host-core arms nothing for it. +/// /// Errors are the diagnostic code of the refusal. fn build_plan_shape( model: &session::SessionModel, @@ -5541,6 +5646,7 @@ fn build_plan_shape( observations: &[EffectObservationHandle], meter_count: usize, master_track: Option, + bindings: &[controls::ObservationBinding], ) -> Result { let track_count = model.tracks.len(); let submix_count = model.submixes.len(); @@ -5655,6 +5761,7 @@ fn build_plan_shape( .try_reserve_exact(observations.len()) .map_err(|_| "web.resource.allocation")?; let observation_present = vec![false; strip_count]; + let mut observation_armed = zero_u32(total_effects as usize)?; // The plan's published frame and the header that describes it (#1290d1). The frame's address // is the heap block's, so it stays put as the tables move. let meter_frame = zero_meter_frame(strip_count)?; @@ -5692,6 +5799,29 @@ fn build_plan_shape( .map_err(|_| "web.resource.allocation")?; arm_samples.resize(tap_count, 0); observation_arm_samples[slot] = arm_samples.into_boxed_slice(); + // #1584 D6b: the bits of the taps the host had armed, in the descriptor's tap order (the + // bit is the effect-local tap index, as `push_observe` sets it). + let console = handle.address.rack == LiveEffectRack::Console; + for binding in bindings { + if binding.key.section.is_console() != console + || binding.key.section.id().as_str() != &*handle.effect_id + || binding.key.strip.as_str() != &*handle.track_id + { + continue; + } + if let Some(index) = handle + .descriptor + .observations + .iter() + .position(|tap| tap.id.0 == binding.key.tap_id) + && let Some(bit) = u32::try_from(index) + .ok() + .and_then(|index| 1_u32.checked_shl(index)) + && let Some(word) = observation_armed.get_mut(slot) + { + *word |= bit; + } + } instance.reader = Some(u32::try_from(reader).map_err(|_| "web.live_controls.observation")?); reader_slots.push(u32::try_from(slot).map_err(|_| "web.live_controls.observation")?); } @@ -5703,7 +5833,7 @@ fn build_plan_shape( effect_instances: effect_instances.into_boxed_slice(), observation_tracks: observation_tracks.into_boxed_slice(), reader_slots: reader_slots.into_boxed_slice(), - observation_armed: zero_u32(total_effects as usize)?, + observation_armed, observation_arm_samples: observation_arm_samples.into_boxed_slice(), observation_commanded: zero_u32(total_effects as usize)?, master_track, @@ -6018,15 +6148,13 @@ fn compile_ready( let record = Arc::new(Mutex::new(BootRecord::default())); let preparer = WebRuntimePreparer::new(options, spectrum_request.cloned()).recording(Arc::clone(&record)); - #[cfg(feature = "test-support")] - let allow_rebuild = Arc::clone(&preparer.allow_rebuild); let limits = control_limits(caps, memory_budget); let control_plane::CompiledChildren { session: mut control, mut plan, } = control_plane::compile_children::(document, limits, Box::new(preparer)) .map_err(|failure| boot_refusal(&failure, &record))?; - let (engine, live_values_superseded, shape) = { + let (engine, live_values_superseded, shape, spectrum_boot) = { let mut record = record .lock() .map_err(|_| fixed_diagnostic("web.internal.boot_record"))?; @@ -6035,7 +6163,9 @@ fn compile_ready( record.superseded.take(), record.shape.take(), ) { - (Some(engine), Some(superseded), Some(shape)) => (engine, superseded, shape), + (Some(engine), Some(superseded), Some(shape)) => { + (engine, superseded, shape, record.spectrum.take()) + } _ => return Err(fixed_diagnostic("web.internal.boot_record").into()), } }; @@ -6088,12 +6218,11 @@ fn compile_ready( // prepared with `live_control_observation_taps == 0`. report.observation_retained_bytes = engine.observation_retained_bytes; // #1290a: the tables were built by the boot plan's own preparation (`build_plan_shape`), the - // same function every replacement plan's preparation runs. The companions' spectrum capture is - // the one thing moved out here, to the spectrum service. + // same function every replacement plan's preparation runs. The boot plan's spectrum captures + // go to the spectrum service below. let companions = plan .active_attachment_mut::() .ok_or_else(|| fixed_diagnostic("web.internal.companions"))?; - let spectrum_capture = companions.take_spectrum_capture(); let rows = shape .bridge_rows() .ok_or_else(|| fixed_diagnostic("web.resource.arithmetic"))?; @@ -6184,17 +6313,21 @@ fn compile_ready( .saturating_sub(report.bridge_retained_bytes) .saturating_add(engine.session_model_bytes), ); + let structure_epoch = control.structure_epoch(); let ready = ReadyOwnership { control, queries, #[cfg(feature = "test-support")] - allow_rebuild, - #[cfg(feature = "test-support")] limits, commands_enabled, live_values_superseded, - shape: Box::new(shape), + shape: { + let mut shape = Box::new(shape); + shape.meter_header.structure_epoch = structure_epoch; + shape + }, shape_epoch: 0, + carried_swaps_seen: 0, record, pending_shapes: { let mut pending = Vec::new(); @@ -6206,9 +6339,9 @@ fn compile_ready( blocking_rebuilds: 0, command_decoded: boxed_observe_staging(decoded_count)?, has_in_flight_commands: false, - spectrum: spectrum_capture.map(|captures| { + spectrum: spectrum_boot.map(|handover| { SpectrumService::new( - captures, + handover.captures, session_sample_rate_hz, session_quantum_frames, spectrum_hop, diff --git a/hosts/host-web/src/tests.rs b/hosts/host-web/src/tests.rs index fb6ad703e..48db80191 100644 --- a/hosts/host-web/src/tests.rs +++ b/hosts/host-web/src/tests.rs @@ -57,6 +57,46 @@ fn boot_options(quantum: u32) -> WebBootOptions { } } +/// The staging a boot allocates holds any ID and any source the grammar allows, whatever the boot +/// session declared, and an ID that does not fit is a typed error, never a panic (a panic inside +/// a render-locked export is a trap). +/// +/// Test value: red if the projection goes back to sizing from the boot session (a rebuild that adds +/// a longer ID or a wider source would then find no room), or if `copy_id_into_staging` indexes +/// past its buffer. A boot of a short, mono session never meets either; only a rebuild does. +#[test] +fn staging_holds_the_grammars_longest_id_and_widest_source_and_refuses_an_overlong_id() { + let projection = project_buffers(1, 48_000, 128, boot_options(128), (false, (0, 0))) + .expect("bridge projection"); + let mut buffers = allocate_buffers(projection).expect("staging"); + assert_eq!(buffers.source_id.len(), session::STABLE_ID_MAXIMUM_BYTES); + assert_eq!( + buffers.source_pcm.len(), + session::SOURCE_CHANNELS_MAXIMUM as usize * 128 + ); + assert_eq!( + buffers.plane_references.len(), + session::SOURCE_CHANNELS_MAXIMUM as usize + ); + let longest = [b'x'; session::STABLE_ID_MAXIMUM_BYTES]; + assert_eq!( + AudioWorkletEngineHost::copy_id_into_staging(Some(&mut buffers), &longest), + Ok(session::STABLE_ID_MAXIMUM_BYTES as u32) + ); + assert_eq!(&buffers.source_id[..], &longest[..]); + assert_eq!( + AudioWorkletEngineHost::copy_id_into_staging( + Some(&mut buffers), + &[b'x'; session::STABLE_ID_MAXIMUM_BYTES + 1] + ), + Err(IdStagingError::TooLong) + ); + assert_eq!( + AudioWorkletEngineHost::copy_id_into_staging(None, b"a"), + Err(IdStagingError::NoStaging) + ); +} + fn retained_projection(document: &[u8], options: WebBootOptions) -> u64 { let model = parse_host_session(core::str::from_utf8(document).expect("UTF-8 session")) .expect("accepted host session"); @@ -82,13 +122,6 @@ fn retained_projection(document: &[u8], options: WebBootOptions) -> u64 { u32::try_from(document.len()).expect("bounded fixture document"), shape.sample_rate_hz, shape.quantum_frames, - shape.maximum_source_channels, - shape - .longest_source_id_bytes - .max(shape.longest_track_id_bytes) - .max(shape.longest_submix_id_bytes) - .max(shape.longest_route_id_bytes) - .max(shape.longest_vca_id_bytes), options, (false, (0, 0)), ) @@ -285,9 +318,9 @@ fn frozen_layouts_and_values_are_exact() { // The meter header is a new fixed structure, not a change to an existing one. // Issue #1209 D2 appended `submix_count` and its pad: 64 bytes became 72, nothing moved. // Issue #1290d1 turned the pad into `retired` and appended the frame and successor pointers: - // 72 bytes became 80, nothing moved. - assert_eq!(size_of::(), 80); - assert_eq!(METER_HEADER_BYTES, 80); + // 72 bytes became 80, nothing moved. Issue #1562b appended `structure_epoch`: 80 became 88. + assert_eq!(size_of::(), 88); + assert_eq!(METER_HEADER_BYTES, 88); assert_eq!(offset_of!(WebMeterHeader, track_count), 8); assert_eq!(offset_of!(WebMeterHeader, windows), 12); assert_eq!(offset_of!(WebMeterHeader, first_sample), 16); @@ -300,6 +333,7 @@ fn frozen_layouts_and_values_are_exact() { assert_eq!(offset_of!(WebMeterHeader, retired), 68); assert_eq!(offset_of!(WebMeterHeader, frame_pointer), 72); assert_eq!(offset_of!(WebMeterHeader, successor_pointer), 76); + assert_eq!(offset_of!(WebMeterHeader, structure_epoch), 80); assert_eq!(offset_of!(WebCommandReport, result), 8); assert_eq!(offset_of!(WebCommandReport, rejected_index), 16); assert_eq!(offset_of!(WebCommandReport, applied_at_sample), 24); @@ -466,9 +500,10 @@ fn preparation_accepts_explicit_64_128_and_256_quanta_with_stable_buffers() { let source_id_ptr = host.source_id_mut().expect("ID").as_ptr(); let source_pcm_ptr = host.source_pcm_mut().expect("PCM").as_ptr(); let output_ptr = host.output_pcm().expect("output").as_ptr(); + // Source PCM staging holds the most channels a source can declare, not the boot session's. assert_eq!( host.source_pcm_mut().expect("PCM").len(), - 2 * quantum as usize + session::SOURCE_CHANNELS_MAXIMUM as usize * quantum as usize ); assert_eq!( host.output_pcm().expect("output").len(), @@ -828,11 +863,6 @@ fn observation_staging_resource_is_exact_for_the_taps_off_and_on_modes() { document.len() as u32, shape.sample_rate_hz, shape.quantum_frames, - shape.maximum_source_channels, - shape - .longest_source_id_bytes - .max(shape.longest_track_id_bytes) - .max(shape.longest_submix_id_bytes), options, (false, (0, 0)), ) @@ -1799,7 +1829,7 @@ fn native_set_control_timeline_digest_pins_the_wasm_parity() { host.control_request_mut(length) .expect("the control request staging")[..frame.len()] .copy_from_slice(frame); - assert_eq!(host.control_command(length), RESULT_OK); + assert_eq!(host.control_command(length, None), RESULT_OK); // A commit queues its reliable events; drain them the way the worklet does. for lane in [EVENT_LANE_RELIABLE, EVENT_LANE_LOSSY] { loop { @@ -3397,7 +3427,7 @@ fn the_meter_frame_carries_a_peak_pair_and_a_gain_word_per_submix() { let mut host = bus_meter_host(QUANTUM, true); let frame = bus_meter_frame(&mut host, BLOCKS); let header = *host.meter_header(); - assert_eq!(header.struct_size, 80); + assert_eq!(header.struct_size, 88); assert_eq!(header.track_count, 3); assert_eq!(header.submix_count, 2); assert_eq!(header.retired, 0); @@ -5772,7 +5802,9 @@ fn web_runtime_preparer_attaches_the_companions_on_every_branch() { CompanionCapture::Single, CompanionCapture::Collection, ] { - let preparer = WebRuntimePreparer::new(options, companion_request(capture)); + let record = Arc::new(Mutex::new(BootRecord::default())); + let preparer = WebRuntimePreparer::new(options, companion_request(capture)) + .recording(Arc::clone(&record)); let superseded = host_core::LiveValuesSuperseded::new(); let check = |host: &mut PreparedHost, handles: &host_core::HostLiveControlHandles, at: &str| { @@ -5793,16 +5825,15 @@ fn web_runtime_preparer_attaches_the_companions_on_every_branch() { strips, "{capture:?} {at}: one meter per strip" ); - let shape = match companions.spectrum_capture { + // #1290d2: the captures are control-side; the preparer leaves them, with the + // plan's pairing, in the record. + let handover = record.lock().expect("the record").spectrum.take(); + let shape = match handover.as_ref().map(|handover| &handover.captures) { None => CompanionCapture::None, Some(PreparedSpectrumCaptures::Single(_)) => CompanionCapture::Single, Some(PreparedSpectrumCaptures::Collection(_)) => CompanionCapture::Collection, }; assert_eq!(shape, capture, "{capture:?} {at}: the branch's capture"); - assert_eq!( - companions.has_spectrum_capture(), - capture != CompanionCapture::None - ); }; let (mut boot, boot_handles) = preparer @@ -5878,8 +5909,8 @@ fn web_runtime_preparer_attaches_the_companions_on_every_branch() { // prepares the successor from the committed model, publishes it and returns; the render half // adopts it at a block boundary. The native gates run the one instance on one thread, the // message handler's place between render calls (decision 17, "single instance first"), through -// the test-support switch that lets the rebuild arm run past `refuse_rebuild` (deleting that -// refusal is #1562b's). +// the production control entry (`control_request_mut` and `control_command`), as the page's +// `control` message does (#1562b). // // The sessions here have no console slot and no insert: an effect's state carries across a swap // only with #1280a, and a session that holds one would not render the same bits after the swap. @@ -6080,16 +6111,66 @@ mod structural_apply { self.block += 1; } - /// Apply a structural transaction through the control half and decode its response. + /// Run one frame through the production control entry, as the page's `control` message + /// does. Returns the entry's result word and what it left to read (the response frame + /// after an ok result, else the refusal's diagnostic text). + fn entry(&mut self, bytes: &[u8]) -> (u32, Vec) { + let staging = self + .host + .control_request_mut(bytes.len() as u32) + .expect("the control staging"); + staging[..bytes.len()].copy_from_slice(bytes); + let result = self.host.control_command(bytes.len() as u32, None); + (result, self.host.control_response().to_vec()) + } + + /// Apply a structural transaction through the control entry and decode its response. fn apply(&mut self, edits: &[SessionEdit], request: u64) -> Outcome { let revision = self.host.committed_revision(); - let response = self - .host - .structural_command_for_test(&frame(edits, request, revision)) - .expect("the control plane answers the frame"); + let (result, response) = self.entry(&frame(edits, request, revision)); + assert_eq!(result, RESULT_OK, "the control plane answers the frame"); outcome(&response) } + /// The structure epoch a `CONTROL_LIST_GET` page carries now: the control list's own, + /// read through the production control entry as a client reads it. + fn list_epoch(&mut self, request: u64) -> u64 { + let codec = ProtocolCodec::default(); + let mut bytes = vec![0_u8; codec.limits().max_frame_bytes]; + let length = codec + .encode_command_frame_into( + &TypedCommandFrame { + request_id: RequestId::new(request).expect("nonzero request"), + expected_revision: ExpectedRevision::Any, + payload: CommandPayload::ControlListGet(&protocol::ControlListRequest { + structure_epoch: 0, + after_handle: 0, + limit: 1, + owner: None, + }), + }, + &mut bytes, + ) + .expect("a control list frame"); + bytes.truncate(length); + let (result, response) = self.entry(&bytes); + assert_eq!( + result, RESULT_OK, + "the control plane answers the list frame" + ); + let mut fields = [0_u16; 64]; + let DecodedTypedResponseFrame::Success { + payload: DecodedSuccessResponsePayload::ControlListPage(page), + .. + } = codec + .decode_typed_response(&response, &mut DecodeScratch::new(&mut fields)) + .expect("a control list page") + else { + panic!("expected a control list page") + }; + page.structure_epoch + } + fn epoch(&self) -> u64 { self.host.active_plan_epoch_for_test().expect("a plan") } @@ -6166,7 +6247,7 @@ mod structural_apply { } assert_eq!(rig.epoch(), 1); assert!(!rig.pending(), "render adopted the plan"); - assert_eq!(rig.host.service(), RESULT_OK); + assert_eq!(rig.host.service(None), RESULT_OK); assert!(!rig.pending()); assert_eq!( rig.host.status().next_absolute_sample, @@ -6265,6 +6346,277 @@ mod structural_apply { gap_free_acceptance(capture_collection); } + // Issue #1290d2: the spectrum part of gate 7. A rebuild keeps a running stream, an armed + // one-shot and a collection's selection, because the host installs the successor's prepared + // captures at the plan's promotion (the service step after the adoption block) through + // `SpectrumService::adopt_successor`, with the plan's pairing and the engine's carry outcome. + + /// A run long enough for several windows of the 2048-frame capture (16 blocks each). + const SPECTRUM_BLOCKS: u64 = 48; + /// The block before which the rebuild runs: inside the window that opened before it. + const SPECTRUM_APPLY_BEFORE: u64 = 21; + + fn other_track_id() -> String { + gate_model().tracks[4].id.as_str().to_owned() + } + + fn two_entry_collection() -> Option { + let entry = |id: String| host_core::SpectrumCaptureCollectionEntry { + target: SpectrumTarget::TrackPostMatrix(id.into()), + channels: SpectrumChannels::Stereo, + }; + Some(SpectrumPreparationRequest::Collection( + SpectrumCaptureCollectionRequest { + entries: vec![entry(model_track_id()), entry(other_track_id())], + maximum_capture_bytes: u64::MAX, + }, + )) + } + + /// One read of the managed stream: the result word, the metadata and the held window. + #[derive(Clone, Copy, Debug)] + struct StreamRead { + result: u32, + metadata: SpectrumStreamMetadata, + window: Option, + } + + impl Rig { + fn read_stream(&mut self) -> StreamRead { + let result = self.host.read_spectrum_stream(None); + StreamRead { + result, + metadata: *self + .host + .spectrum_stream_metadata() + .expect("a spectrum service"), + window: self.host.spectrum_stream_window().copied(), + } + } + + /// Whether the spectrum service still waits for the promotion of an adopted plan. + fn spectrum_promotions_waiting(&self) -> usize { + self.host + .ready + .as_ref() + .expect("ready") + .pending_shapes + .iter() + .filter(|entry| entry.spectrum.is_some()) + .count() + } + } + + fn assert_same_stream(got: &[StreamRead], want: &[StreamRead], label: &str) { + assert_eq!(got.len(), want.len()); + for (block, (got, want)) in got.iter().zip(want).enumerate() { + let at = format!("{label}: block {block}"); + assert_eq!(got.result, want.result, "{at}: result"); + let (g, w) = (&got.metadata, &want.metadata); + assert_eq!(g.status, w.status, "{at}: status"); + assert_eq!(g.capture_epoch, w.capture_epoch, "{at}: capture epoch"); + assert_eq!(g.sequence, w.sequence, "{at}: sequence"); + assert_eq!(g.windows, w.windows, "{at}: windows"); + assert_eq!(g.dropped_captures, w.dropped_captures, "{at}: dropped"); + assert_eq!(g.captured_sample, w.captured_sample, "{at}: captured"); + assert_eq!(g.target_kind, w.target_kind, "{at}: target"); + assert_eq!(g.channels, w.channels, "{at}: channels"); + match (&got.window, &want.window) { + (None, None) => {} + (Some(g), Some(w)) => { + assert_eq!(g.first_sample, w.first_sample, "{at}: window start"); + assert_eq!(g.sequence, w.sequence, "{at}: window sequence"); + assert_eq!(g.stream_epoch, w.stream_epoch, "{at}: window epoch"); + assert!( + g.left + .iter() + .zip(&w.left) + .all(|(a, b)| a.to_bits() == b.to_bits()) + && g.right + .iter() + .zip(&w.right) + .all(|(a, b)| a.to_bits() == b.to_bits()), + "{at}: the planes equal the reference's bit for bit" + ); + } + _ => panic!("{at}: one side holds a window and the other does not"), + } + } + } + + /// `model` booted with `spectrum`, `setup` run on the host, then the stream read after every + /// block of [`SPECTRUM_BLOCKS`]. With `rebuild`, the muted track is added before block + /// [`SPECTRUM_APPLY_BEFORE`] (it sorts first, so every strip index shifts) and the service + /// step runs after block `service_after`. + fn run_stream( + model: &SessionModel, + spectrum: &impl Fn() -> Option, + setup: &impl Fn(&mut Rig), + rebuild: Option, + ) -> (Rig, Vec) { + let mut rig = Rig::boot(model, options(), spectrum()); + setup(&mut rig); + let mut reads = Vec::new(); + for block in 0..SPECTRUM_BLOCKS { + if rebuild.is_some() && block == SPECTRUM_APPLY_BEFORE { + let base = rig.host.committed_revision(); + assert_eq!( + rig.apply(&add_muted_track(model, ADDED), 1), + Outcome::Applied { + path: EditPath::Rebuild, + revision: base + 1 + } + ); + } + rig.step(); + if rebuild == Some(block) { + assert_eq!(rig.host.service(None), RESULT_OK); + } + reads.push(rig.read_stream()); + } + (rig, reads) + } + + fn start_stream(rig: &mut Rig) { + rig.host + .start_spectrum_stream(SpectrumSmoothingConfig::default(), None) + .expect("start the stream"); + } + + /// The swap leaves every read of a running stream equal to a fresh boot of the committed + /// model's: same epoch, no gap, no failure, and the same planes. + fn stream_continues( + spectrum: impl Fn() -> Option, + setup: impl Fn(&mut Rig), + service_after: u64, + ) { + let model = gate_model(); + let committed = with_muted_track(&model, ADDED); + let (_, reference) = run_stream(&committed, &spectrum, &setup, None); + let (mut rig, reads) = run_stream(&model, &spectrum, &setup, Some(service_after)); + assert_eq!(rig.epoch(), 1); + assert_same_stream(&reads, &reference, "the stream across the swap"); + for read in &reads { + assert!( + !matches!( + read.metadata.status, + SpectrumStreamStatus::Failed + | SpectrumStreamStatus::Gap + | SpectrumStreamStatus::Inactive + | SpectrumStreamStatus::Stopped + ), + "no failed epoch, no gap: {:?}", + read.metadata + ); + assert_eq!( + read.metadata.capture_epoch, reads[0].metadata.capture_epoch, + "one capture epoch throughout" + ); + } + let last = reads.last().expect("reads"); + assert_eq!(last.metadata.status, SpectrumStreamStatus::Ready); + assert!( + last.window.expect("a window").first_sample + >= (SPECTRUM_APPLY_BEFORE + 1) * u64::from(QUANTUM), + "the stream delivers windows from after the swap" + ); + assert_eq!( + rig.spectrum_promotions_waiting(), + 0, + "promoted at the service step" + ); + assert_eq!(rig.host.service(None), RESULT_OK); + assert_eq!( + rig.host.ready.as_ref().expect("ready").carried_swaps_seen, + 1, + "the engine carried the swap" + ); + } + + /// Gate 7, spectrum part (single capture). Test value: red if the successor's capture is not + /// installed at promotion or the carry outcome is read wrong, or if the stream restarts at the + /// swap (a new epoch, a gap or a failed read). The service step runs right after the adoption + /// block and long after it: between the two the carried capture is served by the reader the + /// service already holds. + #[test] + fn a_running_spectrum_stream_continues_across_a_rebuild() { + stream_continues(single_capture, start_stream, SPECTRUM_APPLY_BEFORE); + stream_continues(single_capture, start_stream, SPECTRUM_BLOCKS - 3); + } + + /// Gate 7, spectrum part (collection). Test value: red if a collection's selection or its + /// stream is lost at promotion: the second entry is selected and streamed before the swap, and + /// the stream reads like the committed model's. + #[test] + fn a_collection_selection_and_its_stream_continue_across_a_rebuild() { + let select_second = |rig: &mut Rig| { + let other: StableId = StableId::parse(&other_track_id()).expect("track ID"); + assert_eq!( + rig.host.select_spectrum_target( + SpectrumTarget::TrackPostMatrix(other.as_str().into()).as_ref(), + SpectrumChannels::Stereo, + None, + None + ), + Ok(false) + ); + start_stream(rig); + }; + stream_continues(two_entry_collection, select_second, SPECTRUM_APPLY_BEFORE); + stream_continues(two_entry_collection, select_second, SPECTRUM_BLOCKS - 3); + } + + /// Gate 7, spectrum part (one-shot). Test value: red if an armed one-shot is lost at the swap: + /// armed before it, completed after it, equal to the committed model's. + #[test] + fn an_armed_one_shot_completes_across_a_rebuild() { + const ARM_BEFORE: u64 = 12; + let model = gate_model(); + let committed = with_muted_track(&model, ADDED); + let shot = |model: &SessionModel, rebuild: bool| { + let mut rig = Rig::boot(model, options(), single_capture()); + let mut found = None; + for block in 0..SPECTRUM_BLOCKS { + if block == ARM_BEFORE { + assert_eq!(rig.host.arm_spectrum(), RESULT_OK); + } + if rebuild && block == SPECTRUM_APPLY_BEFORE { + rig.apply(&add_muted_track(model, ADDED), 1); + } + rig.step(); + if rebuild && block == SPECTRUM_APPLY_BEFORE { + assert_eq!(rig.host.service(None), RESULT_OK); + } + if found.is_none() && block >= ARM_BEFORE { + found = rig + .host + .read_spectrum(SpectrumChannels::Stereo) + .expect("read the one-shot"); + } + } + found.expect("the one-shot completed") + }; + let reference = shot(&committed, false); + let got = shot(&model, true); + assert!( + got.window.first_sample < SPECTRUM_APPLY_BEFORE * u64::from(QUANTUM) + && got.window.end_sample().expect("end") + > SPECTRUM_APPLY_BEFORE * u64::from(QUANTUM), + "the window spans the swap" + ); + assert_eq!(got.window.first_sample, reference.window.first_sample); + assert_eq!(got.token, reference.token); + assert!( + got.window + .left + .iter() + .chain(&got.window.right) + .zip(reference.window.left.iter().chain(&reference.window.right)) + .all(|(a, b)| a.to_bits() == b.to_bits()), + "the planes equal the committed model's bit for bit" + ); + } + /// The ramp of the live fader edit, in samples: longer than the blocks between the edit and /// the adoption block, so the ramp is in flight when the plan swaps. const FADER_RAMP: u32 = 400; @@ -6287,7 +6639,10 @@ mod structural_apply { .control_request_mut(bytes.len() as u32) .expect("the control staging"); staging[..bytes.len()].copy_from_slice(&bytes); - assert_eq!(rig.host.control_command(bytes.len() as u32), RESULT_OK); + assert_eq!( + rig.host.control_command(bytes.len() as u32, None), + RESULT_OK + ); assert_eq!( outcome(rig.host.control_response()), Outcome::Applied { @@ -6430,35 +6785,12 @@ mod structural_apply { rig.host.committed_revision(), ); bytes.truncate(bytes.len() - 5); - let error = rig - .host - .structural_command_for_test(&bytes) - .expect_err("a truncated frame is refused"); - format!("{error:?}") + let (result, response) = rig.entry(&bytes); + assert_ne!(result, RESULT_OK, "a truncated frame is refused"); + format!("{result} {}", String::from_utf8_lossy(&response)) }); } - /// Run one frame through the production control entry with the rebuild arm open, as the SDK's - /// `control` message will once #1562b deletes the refusal. Returns the entry's result word and - /// what it left to read. - fn entry_with_rebuild_open( - rig: &mut Rig, - edits: &[SessionEdit], - request: u64, - ) -> (u32, Vec) { - let bytes = frame(edits, request, rig.host.committed_revision()); - let staging = rig - .host - .control_request_mut(bytes.len() as u32) - .expect("the control staging"); - staging[..bytes.len()].copy_from_slice(&bytes); - let allow = Arc::clone(&rig.host.ready.as_ref().expect("ready").allow_rebuild); - allow.store(true, Ordering::Release); - let result = rig.host.control_command(bytes.len() as u32); - allow.store(false, Ordering::Release); - (result, rig.host.control_response().to_vec()) - } - /// Gate 3, case two. Test value: red if a transaction that changes the rate or the quantum /// reaches publication (the plan exchange answers it as retryable backpressure, a retry that /// can never succeed), or if the entry reports it as anything but a required re-prepare. @@ -6473,8 +6805,9 @@ mod structural_apply { }, ] { assert_refusal_changes_nothing(options(), |rig, _| { + let revision = rig.host.committed_revision(); let (result, response) = - entry_with_rebuild_open(rig, std::slice::from_ref(&edit), 1); + rig.entry(&frame(std::slice::from_ref(&edit), 1, revision)); assert_eq!(result, RESULT_REPREPARE_REQUIRED, "{edit:?}"); let text = String::from_utf8(response).expect("diagnostic text"); assert!( @@ -6502,18 +6835,13 @@ mod structural_apply { Rig::boot(&model, tight, None); // ... and refuses a second plan beside it. let refused = assert_refusal_changes_nothing(tight, |rig, model| { - let error = rig - .host - .structural_command_for_test(&frame( - &add_muted_track(model, ADDED), - 1, - rig.host.committed_revision(), - )) - .expect_err("a replacement that does not fit beside the running plan is refused"); - let control_plane::CommandError::CompileRejected(failure) = &error else { - panic!("a typed refusal, not {error:?}") - }; - String::from_utf8_lossy(&failure.diagnostic()).into_owned() + let revision = rig.host.committed_revision(); + let (result, response) = rig.entry(&frame(&add_muted_track(model, ADDED), 1, revision)); + assert_eq!( + result, RESULT_COMPILE_REJECTED, + "a replacement that does not fit beside the running plan is refused, typed" + ); + String::from_utf8_lossy(&response).into_owned() }); assert!( refused.contains("control.resource.replacement_peak"), @@ -6633,7 +6961,7 @@ mod structural_apply { 1, "the record took effect on the successor: the old track's compressor is armed" ); - assert_eq!(rig.host.service(), RESULT_OK); + assert_eq!(rig.host.service(None), RESULT_OK); for _ in 0..4 { rig.step(); } @@ -6644,7 +6972,7 @@ mod structural_apply { rig.host.read_observation_addresses(&[address]).is_ok(), "the reader the catalog names is the successor's" ); - assert_eq!(rig.host.service(), RESULT_OK); + assert_eq!(rig.host.service(None), RESULT_OK); } /// Drive `blocks` more blocks with the lease on, polling after each; the windows folded. @@ -6735,6 +7063,120 @@ mod structural_apply { ); } + /// The meter header's structure epoch is the control list's (#1562b): a rebuild that leaves + /// every address and descriptor alone (a strip's `delay_samples` value) keeps it, and the + /// header of the plan that renders reads the number `CONTROL_LIST_GET` pages carry, read here + /// through the control entry after a rebuild and after a live commit that moves the epoch (a + /// VCA added). + /// + /// Test value: red if the header's epoch advances with the revision or the plan epoch rather + /// than with the list (a client would read the list again at every commit), stays the boot's + /// when the list moved (the strip order of a frame would be named by a stale list), or is not + /// stamped by a live commit (`stamp_structure_epoch`): the header then keeps the epoch of the + /// last rebuild while the page's moved on. + #[test] + fn the_headers_structure_epoch_is_the_control_lists() { + let model = gate_model(); + let mut rig = Rig::boot(&model, options(), None); + let boot = rig.host.meter_header().structure_epoch; + assert_eq!(boot, 1, "the boot list is the first structure"); + let delay = SessionEdit::SetControl { + address: controls::ControlAddress { + owner: controls::Owner::Strip(model.tracks[0].id.clone()), + section: controls::Section::Builtins, + control: 11, + lane: session::ParameterChannel::Both, + }, + value: 64.0, + ramp_samples: None, + }; + let base = rig.host.committed_revision(); + assert_eq!( + rig.apply(std::slice::from_ref(&delay), 1), + Outcome::Applied { + path: EditPath::Rebuild, + revision: base + 1 + }, + "a delay is a rebuild" + ); + for _ in 0..2 { + rig.step(); + } + assert_eq!(rig.epoch(), 1); + assert_eq!( + rig.host.meter_header().structure_epoch, + boot, + "a value-only rebuild keeps the list's structure" + ); + assert_eq!( + rig.apply(&add_muted_track(&model, ADDED), 2), + Outcome::Applied { + path: EditPath::Rebuild, + revision: base + 2 + } + ); + assert_eq!( + rig.host.meter_header().structure_epoch, + boot, + "the plan that renders keeps its own list until the replacement is adopted" + ); + rig.step(); + assert_eq!(rig.epoch(), 2); + assert_eq!(rig.host.meter_header().structure_epoch, boot + 1); + let page = rig.list_epoch(3); + assert_eq!( + rig.host.meter_header().structure_epoch, + page, + "after a rebuild the header is the page's" + ); + + // A commit that rides the live lanes can move the list's epoch too (a VCA added: a new + // owner and new addresses), and stamps the plan that renders, which has no successor + // waiting. The header then reads what a `CONTROL_LIST_GET` page carries. + let vca = SessionEdit::UpsertVca { + vca: session::Vca { + id: StableId::parse("late-vca").expect("VCA ID"), + fader: session::DualMonoFader { + left_db: -3.0, + right_db: -3.0, + left_mute: false, + right_mute: false, + }, + members: vec![model.tracks[0].id.clone()], + }, + }; + // The two rebuilds above queued their events; a client drains them as it goes. + for lane in [EVENT_LANE_RELIABLE, EVENT_LANE_LOSSY] { + while rig.host.control_event(lane) == RESULT_OK + && !rig.host.control_response().is_empty() + {} + } + let before_vca = rig.list_epoch(4); + let revision = rig.host.committed_revision(); + assert_eq!( + rig.apply(std::slice::from_ref(&vca), 5), + Outcome::Applied { + path: EditPath::Live, + revision: revision + 1 + }, + "a VCA added is a live commit" + ); + assert!( + !rig.pending(), + "no replacement waits: the live commit rode the rendering plan" + ); + let after_vca = rig.list_epoch(6); + assert!( + after_vca > before_vca, + "a VCA added moves the list's structure epoch" + ); + assert_eq!( + rig.host.meter_header().structure_epoch, + after_vca, + "the rendering plan's header is stamped with the live commit's epoch" + ); + } + /// A swap that adds a strip publishes the successor's frame (#1290d1): the frame travels with /// the plan, so the window the adoption block closes is described by a `3(T + S) + 3` frame for /// the plan's own strips, bit for bit what a fresh boot of the committed model publishes for @@ -6781,9 +7223,24 @@ mod structural_apply { "the rendering plan names the newest candidate's header" ); assert_eq!(ready.shape.meter_header.retired, 0); + // The strip order changed, so the list's structure epoch did: the rendering plan keeps + // the epoch of its own list, the replacement carries the newer one (#1562b). + assert_eq!( + waiting.shape.meter_header.structure_epoch, + last.0.structure_epoch + 1 + ); + assert_eq!( + ready.shape.meter_header.structure_epoch, + last.0.structure_epoch + ); } let after = poll_cadence(&mut rig, 1, 1, 0); assert_eq!(rig.epoch(), 1); + assert_eq!( + after[0].0.structure_epoch, + last.0.structure_epoch + 1, + "the adoption block's frame names the successor's list" + ); assert_eq!( after.len(), 1, @@ -7034,7 +7491,7 @@ mod structural_apply { 1, "the subscription the plan had acknowledged is armed on the successor" ); - assert_eq!(rig.host.service(), RESULT_OK); + assert_eq!(rig.host.service(None), RESULT_OK); for _ in 0..4 { rig.step(); } @@ -7174,19 +7631,217 @@ mod structural_apply { panic!("poison the record"); }); assert!(poisoner.join().is_err()); - let error = rig - .host - .structural_command_for_test(&frame( - &add_muted_track(model, ADDED), - 1, - rig.host.committed_revision(), - )) - .expect_err("the rebuild is refused"); - let control_plane::CommandError::CompileRejected(failure) = &error else { - panic!("a typed refusal, not {error:?}") - }; - String::from_utf8_lossy(&failure.diagnostic()).into_owned() + let revision = rig.host.committed_revision(); + let (result, response) = rig.entry(&frame(&add_muted_track(model, ADDED), 1, revision)); + assert_eq!( + result, RESULT_COMPILE_REJECTED, + "the rebuild is refused, typed" + ); + String::from_utf8_lossy(&response).into_owned() }); assert!(refused.contains("web.apply.shape_unavailable"), "{refused}"); } + + /// The model of the observation gates: the command fixture with the observed compressor + /// (insert `comp`, tap 1) as the first track's only insert. + fn observed_model() -> SessionModel { + let mut model = + parse_session_json(include_str!("../tests/browser-v1/command-session.json")) + .expect("accepted command fixture"); + model.tracks[0].inserts.effects[0] = observed_compressor_insert(); + model + } + + fn comp_address() -> ObservationAddress { + ObservationAddress { + track_index: 0, + rack: LiveEffectRack::Inserts, + effect_index: 0, + tap_id: 1, + channels: ObservationReadChannels::Both, + } + } + + /// Every reliable event the control plane has queued, drained the way the worklet does, as + /// `(revision, payload)` for the two this gate reads. + fn drain_reliable(host: &mut AudioWorkletEngineHost) -> Vec<(u64, String)> { + let codec = ProtocolCodec::default(); + let mut events = Vec::new(); + loop { + assert_eq!(host.control_event(EVENT_LANE_RELIABLE), RESULT_OK); + if host.control_response().is_empty() { + return events; + } + let bytes = host.control_response().to_vec(); + let mut fields = vec![0_u16; 4_096]; + let decoded = codec + .decode_typed_event(&bytes, &mut DecodeScratch::new(&mut fields)) + .expect("an event frame"); + let revision = decoded.header.revision.0; + match decoded.payload { + protocol::DecodedEventPayload::SessionCommitted(_) => { + events.push((revision, "committed".to_owned())); + } + protocol::DecodedEventPayload::ObservationEnded(ended) => { + for key in &ended.keys { + events.push(( + revision, + format!( + "ended {} {:?} {} {}", + key.strip_id, key.section, key.effect_id, key.tap_id + ), + )); + } + } + _ => {} + } + } + } + + /// #1584 D6b, restart: a subscribed compressor whose link mode changes is prepared again (a + /// rebuild with no carry), and the successor's tables say what its lanes were armed with. The + /// tap is armed on the browser's side, the first window opens at the adoption block's first + /// sample, and the windows continue block after block. + /// + /// Test value: red if `build_plan_shape` leaves the successor's armed bits clear for a tap the + /// plan's lane armed (the read would answer `Unarmed` and the page's subscription would + /// silently stop at every restart of its effect, as `observation_armed_taps` reads zero). + #[test] + fn a_restarted_effects_subscription_keeps_publishing_windows_on_the_browser() { + let model = observed_model(); + let mut rig = Rig::boot(&model, options(), None); + assert_eq!(rig.host.set_meter_lease(true), RESULT_OK); + rig.step(); + assert_eq!( + observe(&mut rig.host, 0, RACK_INSERTS, 0, 1, 2, true), + RESULT_OK + ); + for _ in 0..8 { + rig.step(); + } + let before = rig + .host + .read_observation_addresses(&[comp_address()]) + .expect("a read")[0] + .window + .expect("the armed tap published a window") + .sequence; + assert!(before > 1); + + let mut restarted = model.tracks[0].inserts.effects[0].clone(); + restarted.link_mode = session::LinkMode::Maximum; + let base = rig.host.committed_revision(); + assert!(matches!( + rig.apply( + &[SessionEdit::PutTrackEffect { + track_id: model.tracks[0].id.clone(), + rack_name: session::RackName::Inserts, + final_position: 0, + effect: restarted, + }], + 1 + ), + Outcome::Applied { + path: EditPath::Rebuild, + revision + } if revision == base + 1 + )); + let adoption_sample = rig.block * u64::from(QUANTUM); + rig.step(); + assert_eq!(rig.epoch(), 1); + assert_eq!( + rig.host.observation_armed_taps(), + 1, + "the successor's tables carry the subscription the host had acknowledged" + ); + rig.step(); + let first = rig + .host + .read_observation_addresses(&[comp_address()]) + .expect("a read")[0] + .window + .expect("the restarted tap published its first window"); + assert_eq!( + first.first_sample, adoption_sample, + "from the adoption block on" + ); + assert_eq!( + first.sequence, 1, + "a restarted tap numbers its windows afresh" + ); + for _ in 0..4 { + rig.step(); + } + let later = rig + .host + .read_observation_addresses(&[comp_address()]) + .expect("a read")[0] + .window + .expect("the windows continue"); + assert_eq!(later.sequence, 3); + assert!(rig.host.poll_meters() > 0); + assert!( + gain_reduction(&rig.host).0[0] > 0.0, + "the meter frame's gain-reduction word follows the restarted tap" + ); + } + + /// #1584 D6b, removal: the removed insert has no slot on the successor, the browser's read + /// finds no tap there, and the event drain returns the commit and one `OBSERVATION_ENDED` + /// that names the tap. + /// + /// Test value: red if the browser's drain loses the event (the page would keep a subscription + /// whose effect is gone and never be told), or if a removed insert's tap stays armed in the + /// successor's tables. + #[test] + fn a_removed_inserts_subscription_ends_with_an_event_and_has_no_slot() { + let model = observed_model(); + let mut rig = Rig::boot(&model, options(), None); + rig.step(); + assert_eq!( + observe(&mut rig.host, 0, RACK_INSERTS, 0, 1, 2, true), + RESULT_OK + ); + for _ in 0..4 { + rig.step(); + } + assert_eq!(rig.host.observation_armed_taps(), 1); + assert!( + drain_reliable(&mut rig.host).is_empty(), + "nothing is queued yet" + ); + + let base = rig.host.committed_revision(); + assert!(matches!( + rig.apply( + &[SessionEdit::RemoveTrackEffect { + track_id: model.tracks[0].id.clone(), + rack_name: session::RackName::Inserts, + effect_id: model.tracks[0].inserts.effects[0].id.clone(), + }], + 1 + ), + Outcome::Applied { + path: EditPath::Rebuild, + revision + } if revision == base + 1 + )); + let events = drain_reliable(&mut rig.host); + assert_eq!( + events, + vec![ + (base + 1, "committed".to_owned()), + (base + 1, "ended track Inserts comp 1".to_owned()), + ] + ); + rig.step(); + assert_eq!(rig.epoch(), 1); + assert_eq!(rig.host.observation_armed_taps(), 0); + assert!( + rig.host + .read_observation_addresses(&[comp_address()]) + .is_err(), + "the successor has no insert, so no slot to read" + ); + } } diff --git a/hosts/host-web/tests/browser-v1/direct-oracle.mjs b/hosts/host-web/tests/browser-v1/direct-oracle.mjs index 9607eca0c..fc342299b 100644 --- a/hosts/host-web/tests/browser-v1/direct-oracle.mjs +++ b/hosts/host-web/tests/browser-v1/direct-oracle.mjs @@ -22,9 +22,9 @@ const COMMAND_RECORD_BYTES = ABI_LAYOUT.commandRecord.bytes; const COMMAND_OBSERVE_SUBSCRIBE = 7; const COMMAND_OBSERVE_UNSUBSCRIBE = 8; const BUFFER_METER_FRAME = 7; -// 80 bytes: `submix_count` at 64 and `retired` at 68 (issue #1209 D2, #1290d1), then -// `frame_pointer` at 72 and `successor_pointer` at 76 (issue #1290d1). -const METER_HEADER_BYTES = 80; +// Issue #1209 D2: 72 since `submix_count` and its pad were appended at 64 and 68; 80 with the frame +// and successor pointers (#1290d1); 88 with `structure_epoch` appended at 80 (#1562b). +const METER_HEADER_BYTES = 88; const OBSERVATION_WINDOW_BLOCKS = 2; const RESOURCE_NAMES = [ "optionsBytes", "statusBytes", "sessionDocumentBytes", "diagnosticBytes", "idStagingBytes", @@ -405,6 +405,7 @@ async function runObservationTimeline(modulePath, sessionDocument, sourceId, tap assert.equal(header.getUint32(68, true), 0, "the header is not retired"); assert.equal(header.getUint32(72, true), framePointer, "the header names its frame"); assert.equal(header.getUint32(76, true), 0, "no replacement plan waits"); + assert.equal(header.getBigUint64(80, true), 1n, "the boot list is the first structure"); assert.equal(header.getUint32(40, true), taps === 0n ? 0 : 1); const frame = new Float32Array(memoryBuffer, framePointer, 6); assert.equal(exports.miso_engine_web_v1_meter_lease(handle, 1), 0); diff --git a/hosts/host-web/tests/browser-v1/expected.json b/hosts/host-web/tests/browser-v1/expected.json index c008ddb8f..e74e3c65c 100644 --- a/hosts/host-web/tests/browser-v1/expected.json +++ b/hosts/host-web/tests/browser-v1/expected.json @@ -36,7 +36,7 @@ "quantumFrames": 128, "nextAbsoluteSample": "0", "renderedQuanta": "0", - "memoryBytes": 2883584 + "memoryBytes": 3014656 }, "beforeDisposeStatus": { "state": 2, @@ -46,7 +46,7 @@ "quantumFrames": 128, "nextAbsoluteSample": "384", "renderedQuanta": "3", - "memoryBytes": 2883584 + "memoryBytes": 3014656 }, "resources": { "sampleRateHz": 48000, @@ -56,8 +56,8 @@ "statusBytes": "80", "sessionDocumentBytes": "2021", "diagnosticBytes": "16384", - "idStagingBytes": "14", - "sourcePcmStagingBytes": "1024", + "idStagingBytes": "127", + "sourcePcmStagingBytes": "130560", "outputPcmBytes": "1024", "largestBridgeAllocationBytes": "1048576", "effectScalarStateBytes": "0", @@ -68,7 +68,7 @@ }, "resourceCeilings": { "bridgeMetadataBytes": "1628864", - "bridgeRetainedBytes": "1651520", + "bridgeRetainedBytes": "1812352", "sourceTotalBytes": "3648", "sourceOverheadBytes": "2496", "builtinRetainedBytes": "4288", @@ -114,7 +114,7 @@ "quantumFrames": 128, "nextAbsoluteSample": "1280", "renderedQuanta": "10", - "memoryBytes": 2883584 + "memoryBytes": 3014656 }, "pcmF32leSha256": "c3659578f50d02030fcef19df98be88d70e89f2a0c9bade3e23e9570864d1cfd" }, diff --git a/hosts/host-web/tests/common/mod.rs b/hosts/host-web/tests/common/mod.rs index 681af3db0..a82f69568 100644 --- a/hosts/host-web/tests/common/mod.rs +++ b/hosts/host-web/tests/common/mod.rs @@ -409,7 +409,7 @@ impl Rig { .control_request_mut(length) .expect("the control request staging"); staging[..frame.len()].copy_from_slice(frame); - let result = self.host.control_command(length); + let result = self.host.control_command(length, None); (result, self.host.control_response().to_vec()) } diff --git a/hosts/host-web/tests/control_admission.rs b/hosts/host-web/tests/control_admission.rs index c8e463b67..aa07ed9f9 100644 --- a/hosts/host-web/tests/control_admission.rs +++ b/hosts/host-web/tests/control_admission.rs @@ -5,13 +5,13 @@ //! //! Every test drives `AudioWorkletEngineHost` through `control_request_mut` and `control_command`, //! the calls the exports make, with the frames the C ABI would receive and nothing decoded by the -//! host. The rebuild path is #1562b's: until then a transaction the classifier sends there is -//! refused `web.apply.rebuild_pending` before it is committed (D9, gate 11). +//! host. A transaction the classifier sends to the rebuild arm is prepared and published like any +//! other (#1562b). #![cfg(feature = "test-support")] use controls::{ControlAddress, Lane, MonitorOverlay, Owner, Section}; -use host_web::{RESULT_COMPILE_REJECTED, RESULT_INVALID_ARGUMENT, RESULT_OK}; +use host_web::RESULT_INVALID_ARGUMENT; use protocol::{ CommandPayload, ControlListRequest, DecodeScratch, DecodedSuccessResponsePayload, DecodedTypedResponseFrame, EditPath, ExpectedRevision, ProtocolCodec, SessionEdit, @@ -350,7 +350,7 @@ fn live_rows() -> Vec { /// Test value: red if a browser edit reaches the cells without reaching the committed model (the /// document would not hold the value, and the fresh plan would render the old one), if a family /// lowers differently in the browser than in the model, or if a row the core calls live is a -/// rebuild in the browser (`web.apply.rebuild_pending`). +/// rebuild in the browser. #[test] fn every_live_row_commits_to_the_model_and_renders_as_a_fresh_plan() { const BLOCKS_RUN: u64 = 40; @@ -465,92 +465,137 @@ fn every_live_row_commits_to_the_model_and_renders_as_a_fresh_plan() { } } -/// Gate 11 and D9: a `SET_CONTROL` on `delay_samples` needs a plan rebuild, which the browser -/// cannot apply in place before #1562b. The frame is refused `web.apply.rebuild_pending` before -/// anything is committed. +/// Gate 1's `delay_samples` row (#1562b): a `SET_CONTROL` on `delay_samples` is a rebuild. The +/// browser prepares the successor and publishes it, the reply is `rebuild` at the next revision +/// with the plan still the boot plan, and from the block that adopts it every block is the bits of +/// a plan prepared from the committed document and fed the same PCM from frame 0. /// -/// Test value: red if a rebuild-path frame reaches `SessionState`'s rebuild arm (a revision would -/// advance, a plan would be published, a commit event would queue), or if the refusal leaves the -/// host unable to take the next live edit. +/// Test value: red if the browser still refuses a rebuild-path frame, answers it live, acknowledges +/// it without a plan that renders it (the document would hold the delay and the blocks would not +/// have it), or adopts a plan that is not the committed model's. #[test] -fn a_rebuild_frame_is_refused_before_commit_and_changes_nothing() { - const BLOCKS_RUN: u64 = 12; +fn a_delay_samples_edit_is_a_rebuild_and_renders_as_a_fresh_plan() { + const BLOCKS_RUN: u64 = 24; let model = model_at(RATE); let mut rig = Rig::new(&model); - let before_document = document(&mut rig); - let before_revision = rig.host.committed_revision(); - let epoch = rig.host.active_plan_epoch_for_test(); - drain_events(&mut rig.host); - - let edits = [set( + let before = rig.host.committed_revision(); + let epoch = rig.host.active_plan_epoch_for_test().expect("a plan"); + let verdict = rig.transaction(&[set( strip_row("t1", DELAY_SAMPLES, ParameterChannel::Both), 64.0, None, - )]; - let frame = rig.encode( - CommandPayload::SessionTransactionApply(&edits), - ExpectedRevision::Exact(protocol::SessionRevision(rig.revision)), + )]); + assert_eq!( + verdict, + Verdict::Applied { + path: EditPath::Rebuild, + revision: before + 1 + }, + "the reply is the rebuild path at the next revision" ); - let (result, diagnostic) = rig.send_frame(&frame); - assert_eq!(result, RESULT_COMPILE_REJECTED, "the call's result"); assert_eq!( - String::from_utf8(diagnostic).expect("UTF-8"), - "web.apply.rebuild_pending\t$\n", - "the response is the typed refusal" + rig.host.active_plan_epoch_for_test(), + Some(epoch), + "the submit never waits for render: the boot plan still renders" ); + let committed = document(&mut rig); + let edited: Vec> = (0..BLOCKS_RUN).map(|block| rig.step(block)).collect(); assert_eq!( - rig.host.committed_revision(), - before_revision, - "no revision" + rig.host.active_plan_epoch_for_test(), + Some(epoch + 1), + "render adopted the successor" ); - assert_eq!(rig.host.active_plan_epoch_for_test(), epoch, "no plan"); - assert_eq!(document(&mut rig), before_document, "no committed change"); + let parsed = parse_session_json(std::str::from_utf8(&committed).expect("UTF-8")) + .expect("the committed document parses"); + let held = controls::get( + &parsed, + &MonitorOverlay::default(), + &strip_row("t1", DELAY_SAMPLES, ParameterChannel::Left), + RATE, + ) + .expect("the delay row reads"); + assert!((held - 64.0).abs() < 1.0e-6, "the document holds {held}"); + let fresh = fresh_blocks(&committed, BLOCKS_RUN, |_| {}); assert_eq!( - drain_events(&mut rig.host), - 0, - "nothing was acknowledged: no commit event is queued" + first_difference(&edited, &fresh), + None, + "a block differs from the plan prepared from the committed model" ); - // The refusal consumed its request ID, as every refusal after the core has seen the frame - // does (the replay cache answers a resend `protocol.failure`); a retry takes a new ID and - // meets the same refusal. - let (resent, resent_bytes) = rig.send_frame(&frame); - assert_eq!(resent, RESULT_OK); - assert!(matches!( - decode_verdict(&resent_bytes), - Verdict::Refused { ref codes, .. } if codes == &["protocol.failure"] - )); - let retry = rig.encode( - CommandPayload::SessionTransactionApply(&edits), - ExpectedRevision::Exact(protocol::SessionRevision(rig.revision)), + let untouched = fresh_blocks(&document_of(&model), BLOCKS_RUN, |_| {}); + assert_ne!( + first_difference(&edited, &untouched), + None, + "the delay is heard" ); - let (again, _) = rig.send_frame(&retry); - assert_eq!(again, RESULT_COMPILE_REJECTED); +} - // The host is still ready, and the next live edit commits at the unchanged revision. - let verdict = rig.transaction(&[set( - strip_row("t1", FADER_DB, ParameterChannel::Both), - -6.0, - Some(0), - )]); - assert_eq!( - verdict, - Verdict::Applied { - path: EditPath::Live, - revision: before_revision + 1 - } - ); +fn document_of(model: &SessionModel) -> Vec { + canonical_session_json(model) + .expect("canonical session") + .into_bytes() +} + +/// Gate 3's structural part (#1562b): a structural transaction keeps the solo. Track `t3` is +/// muted by the solo of `t2`, so its delay changes nothing the output carries; the rebuild's +/// successor must still render under the overlay, or the soloed-out tracks would be heard. +/// +/// Test value: red if a rebuild drops or recomputes the monitor overlay (the others would unmute +/// from the swap block, which the bits against a run without the rebuild show), or if the solo +/// moved the revision or the document. +#[test] +fn a_structural_transaction_keeps_the_solo() { + const BLOCKS_RUN: u64 = 24; + const REBUILD_BEFORE: u64 = 8; + let model = without_inserts(RATE); + let solo = set(strip_own_solo("t2"), 1.0, Some(0)); - // The next blocks are the bits of a run that never saw the refused frame. - let mut with_refusal = Rig::new(&model); - let _ = with_refusal.send_frame(&frame); let mut without = Rig::new(&model); + without.transaction(std::slice::from_ref(&solo)); + let reference: Vec> = (0..BLOCKS_RUN).map(|block| without.step(block)).collect(); + + let mut rig = Rig::new(&model); + rig.transaction(std::slice::from_ref(&solo)); + let mut blocks = Vec::new(); for block in 0..BLOCKS_RUN { - assert_eq!( - with_refusal.step(block), - without.step(block), - "block {block} differs after a refused rebuild frame" - ); + if block == REBUILD_BEFORE { + let before = rig.host.committed_revision(); + let verdict = rig.transaction(&[set( + strip_row("t3", DELAY_SAMPLES, ParameterChannel::Both), + 64.0, + None, + )]); + assert_eq!( + verdict, + Verdict::Applied { + path: EditPath::Rebuild, + revision: before + 1 + }, + "the structural transaction rebuilds" + ); + } + blocks.push(rig.step(block)); } + assert_eq!( + rig.host.active_plan_epoch_for_test(), + Some(1), + "render adopted the successor" + ); + assert_eq!( + first_difference(&blocks, &reference), + None, + "the successor renders under the solo: every block equals a run with no rebuild" + ); + // The solo is still a monitor write after the swap. + let after = rig.host.committed_revision(); + let verdict = rig.transaction(&[set(strip_own_solo("t2"), 0.0, Some(0))]); + assert_eq!( + verdict, + Verdict::Applied { + path: EditPath::Monitor, + revision: after + }, + "clearing the solo after the swap is still a monitor write" + ); } /// Gate 4 (the acked-batch rule): a transaction of a valid fader edit, a solo and an out-of-domain diff --git a/hosts/host-web/tests/control_parity.rs b/hosts/host-web/tests/control_parity.rs index ad5ee1f68..9eb80af88 100644 --- a/hosts/host-web/tests/control_parity.rs +++ b/hosts/host-web/tests/control_parity.rs @@ -13,8 +13,12 @@ //! codes. Per block it compares the bits of the two output planes. At the end the two hosts report //! the same `live_values_superseded` count, and every page of `CONTROL_LIST_GET` and //! `CONTROL_STATE_GET` and the `SESSION_SNAPSHOT_GET` bytes are byte-identical. The whole run -//! repeats at 44.1, 48, 88.2 and 96 kHz. The rebuild part of the browser's parity (a -//! `delay_samples` edit) is #1562b's: until then the browser refuses it `web.apply.rebuild_pending`. +//! repeats at 44.1, 48, 88.2 and 96 kHz. The script holds the edits that take the rebuild arm (a +//! route into the output, the prepared bypass of the delay and the multiband compressor, and +//! `delay_samples`): the browser prepares and publishes the successor like the C ABI does, and the +//! two hosts answer the same bytes and render the same bits across the swap (#1562b). No launch +//! effect declares an audio-rate parameter, so that row of the batch 7 amendment has no edit to +//! script. //! //! It absorbs #1364 gate 3 (the edits carry no per-edit ramp, so each host resolves the session's //! own length, and one pan carries a window of its own while another does not) and #1399 (a fader, @@ -62,6 +66,8 @@ const EFFECTIVE_FADER_DB: u32 = 64; const ADDED_VCA: VcaRef = VcaRef { id: "w" }; const OUTER_VCA: VcaRef = VcaRef { id: "outer" }; const MATRIX_MODE_ROW: u32 = 1; +/// The builtin row of a strip's `delay_samples`. +const DELAY_SAMPLES_ROW: u32 = 11; const RATES: [u32; 4] = [44_100, 48_000, 88_200, 96_000]; /// Which lane(s) of a dual-mono value an edit writes. @@ -206,6 +212,19 @@ enum Edit { AddVca, /// A VCA that holds the VCA `v` and the added VCA `w`: a nested VCA. NestedVca, + /// The gain of the route from `t1` into the output (a rebuild: that route has no live lane). + OutputGain { + db: f32, + }, + /// The mute of the route from `t1` into the output (a rebuild). + OutputMute { + muted: bool, + }, + /// A strip's `delay_samples` on both lanes (a rebuild: it moves the plan's latency). + DelaySamples { + strip: Strip, + samples: f32, + }, } /// How a scripted edit must end on both hosts. @@ -213,6 +232,8 @@ enum Edit { enum Outcome { /// Accepted: `live` on both hosts (`monitor` for solo). Accepted, + /// Accepted by a plan rebuild: `rebuild` on both hosts (#1562b). + Rebuilt, /// Accepted and a no-op: the edit repeats the committed value, so both hosts answer /// `model_only` and write no record. Unchanged, @@ -247,6 +268,15 @@ const fn live_ramped(block: u64, edit: Edit, ramp: u32) -> Step { } } +const fn rebuilt(block: u64, edit: Edit) -> Step { + Step { + block, + edit, + ramp: None, + outcome: Outcome::Rebuilt, + } +} + const fn refused(block: u64, edit: Edit, code: &'static str) -> Step { Step { block, @@ -689,10 +719,35 @@ fn script() -> Vec { 480, ), unchanged(39, Edit::SendMute { muted: false }, 480), - // A submix's trim and pan (switched from its matrix first), one lane of a high-pass. A - // route into the output has no live lane: its gain and mute are a plan rebuild, which the - // C ABI applies and the browser refuses `web.apply.rebuild_pending` until #1562b (D9), so - // its rows join this script with that issue. + // The rebuild arm (#1562b): a route into the output has no live lane, so its gain and + // mute rebuild the plan; so does the prepared bypass of the delay and of the multiband + // compressor, and a strip's `delay_samples`, which moves the plan's latency. Each is a + // swap at the next block on both hosts, with the watermark advancing at that block. + rebuilt(11, Edit::OutputGain { db: -3.0 }), + rebuilt(17, Edit::OutputMute { muted: true }), + rebuilt(19, Edit::OutputMute { muted: false }), + rebuilt( + 21, + Edit::DelaySamples { + strip: T1, + samples: 64.0, + }, + ), + rebuilt( + 34, + Edit::Bypass { + insert: MULTIBAND, + bypassed: true, + }, + ), + rebuilt( + 43, + Edit::Bypass { + insert: DELAY, + bypassed: true, + }, + ), + // A submix's trim and pan (switched from its matrix first), one lane of a high-pass. live( 48, Edit::Trim { @@ -1139,6 +1194,28 @@ impl CAbi { members: vec![id(VCA.id), id(ADDED_VCA.id)], }, }], + Edit::OutputGain { db } => vec![set( + address( + Owner::Route(id("t1-main")), + Section::Own, + ROUTE_GAIN_DB, + ParameterChannel::Both, + ), + db, + )], + Edit::OutputMute { muted } => vec![set( + address( + Owner::Route(id("t1-main")), + Section::Own, + ROUTE_MUTE, + ParameterChannel::Both, + ), + toggle(muted), + )], + Edit::DelaySamples { strip, samples } => vec![set( + Self::strip_address(strip, DELAY_SAMPLES_ROW, Lanes::Both), + samples, + )], Edit::Solo { strip, on } => vec![set( address( Owner::Strip(id(strip.id)), @@ -1273,41 +1350,64 @@ impl CAbi { fn step(&mut self, block: u64) -> Vec { for (feed, strip) in [T1, T2, T3].into_iter().enumerate() { - let [left, right] = planes(feed as u64, block); - let pointers = [left.as_ptr(), right.as_ptr()]; - let chunk = SourceChunk { - struct_size: SOURCE_CHUNK_SIZE, - sample_rate_hz: self.rate, - generation: 1, - start_frame: block * u64::from(QUANTUM), - planes: pointers.as_ptr(), - plane_count: 2, - frames: QUANTUM, - end_of_region: 0, - reserved0: 0, - }; - let mut report = SubmitReport { - struct_size: SUBMIT_REPORT_SIZE, - reserved0: 0, - accepted_frames: 0, - cumulative_written_frames: 0, - active_generation: 0, - }; - let name = format!("src-{}", strip.id); - // SAFETY: The session is live and used by this thread alone; the chunk's planes and - // the report are owned storage that outlives the synchronous call. - let code = unsafe { - capi::miso_engine_v1_source_submit_planar_f32( - self.session, - name.as_ptr(), - name.len() as u64, - &chunk, - &mut report, - ) - }; - assert_eq!(code, C_RESULT_OK, "C ABI source {}", strip.id); - assert_eq!(report.accepted_frames, u64::from(QUANTUM)); + self.submit(&format!("src-{}", strip.id), feed as u64, block); } + self.render(block) + } + + /// Submit block `block`'s PCM of feed `feed` to the source `name`, at the source's own position + /// `block`. + fn submit(&mut self, name: &str, feed: u64, block: u64) { + self.submit_from(name, feed, block, block, 1); + } + + /// [`Self::submit`], with the chunk of `generation` starting at the source's frame + /// `start_block * QUANTUM`. + fn submit_from( + &mut self, + name: &str, + feed: u64, + block: u64, + start_block: u64, + generation: u64, + ) { + let [left, right] = planes(feed, block); + let pointers = [left.as_ptr(), right.as_ptr()]; + let chunk = SourceChunk { + struct_size: SOURCE_CHUNK_SIZE, + sample_rate_hz: self.rate, + generation, + start_frame: start_block * u64::from(QUANTUM), + planes: pointers.as_ptr(), + plane_count: 2, + frames: QUANTUM, + end_of_region: 0, + reserved0: 0, + }; + let mut report = SubmitReport { + struct_size: SUBMIT_REPORT_SIZE, + reserved0: 0, + accepted_frames: 0, + cumulative_written_frames: 0, + active_generation: 0, + }; + // SAFETY: The session is live and used by this thread alone; the chunk's planes and + // the report are owned storage that outlives the synchronous call. + let code = unsafe { + capi::miso_engine_v1_source_submit_planar_f32( + self.session, + name.as_ptr(), + name.len() as u64, + &chunk, + &mut report, + ) + }; + assert_eq!(code, C_RESULT_OK, "C ABI source {name}"); + assert_eq!(report.accepted_frames, u64::from(QUANTUM)); + } + + /// Render block `block` and return the output's bits. + fn render(&mut self, block: u64) -> Vec { let mut pcm = vec![f32::NAN; QUANTUM as usize * 2]; let output = PlanarOutput { struct_size: PLANAR_OUTPUT_SIZE, @@ -1435,6 +1535,18 @@ fn run(rate: u32, steps: &[Step]) -> (Vec>, u64) { "{label}: the path is {path:?}, got {c:?}" ); } + Outcome::Rebuilt => { + assert!( + matches!( + &c, + Verdict::Applied { + path: EditPath::Rebuild, + .. + } + ), + "{label}: the path is rebuild, got {c:?}" + ); + } Outcome::Unchanged => { let path = if solo { EditPath::Monitor @@ -1596,6 +1708,107 @@ fn the_status_words_carry_the_applied_revision_the_c_abi_reports() { } } +/// #1349's rebuild leg (#1562b): a rebuild, then a live edit before the swap. The status words +/// stay at the initial revision through the blocks before the swap block (neither commit is +/// applied until a plan that carries it renders), then both revisions are in effect together from +/// the swap block's first sample, and the words equal the C ABI's `miso_engine_v1_plan_watermark` +/// after every block. +/// +/// Test value: red if the browser publishes a rebuild's revision when it commits rather than when +/// the swap block renders (the words would move before block 3), if the live edit committed behind +/// a pending rebuild is reported at its own earlier block, or if the swap block's sample or +/// outcome flags are the browser's own computation and not the plan's. +#[test] +fn the_status_words_carry_a_rebuilds_watermark_the_c_abi_reports() { + for rate in RATES { + let model = model_at(rate); + let mut browser = Rig::new(&model); + let mut c_abi = CAbi::new(&model); + let initial = (model.revision, 0, 1); + for block in 0..3 { + let _ = (browser.step(block), c_abi.step(block)); + assert_eq!(status_watermark(&browser), initial, "{rate} Hz: {block}"); + assert_eq!(c_abi.watermark(), initial); + } + let rebuild = Edit::DelaySamples { + strip: T1, + samples: 64.0, + }; + let live_edit = Edit::Fader { + strip: T2, + lanes: Lanes::Both, + db: -6.0, + }; + let (rebuilt, _) = browser.transaction_reply(&CAbi::edits(rebuild, None)); + let (c_rebuilt, _) = c_abi.edit(rebuild, None); + assert_eq!(rebuilt, c_rebuilt); + assert!(matches!( + rebuilt, + Verdict::Applied { + path: EditPath::Rebuild, + revision + } if revision == model.revision + 1 + )); + assert_eq!( + status_watermark(&browser), + initial, + "{rate} Hz: never early" + ); + assert_eq!(c_abi.watermark(), initial); + // The live edit rides the pending successor's lanes. + let (edited, _) = browser.transaction_reply(&CAbi::edits(live_edit, None)); + let (c_edited, _) = c_abi.edit(live_edit, None); + assert_eq!(edited, c_edited); + assert!(matches!( + edited, + Verdict::Applied { + path: EditPath::Live, + revision + } if revision == model.revision + 2 + )); + assert_eq!( + status_watermark(&browser), + initial, + "{rate} Hz: still pending" + ); + assert_eq!(c_abi.watermark(), initial); + + let _ = (browser.step(3), c_abi.step(3)); + assert_eq!( + status_watermark(&browser), + c_abi.watermark(), + "{rate} Hz: the swap block" + ); + let (revision, first_sample, flags) = status_watermark(&browser); + assert_eq!( + revision, + model.revision + 2, + "{rate} Hz: both revisions apply" + ); + assert_eq!( + first_sample, + 3 * u64::from(QUANTUM), + "{rate} Hz: the swap block" + ); + let _ = flags; + assert_eq!(browser.host.read_watermark(), host_web::RESULT_OK); + let record = browser.host.watermark(); + assert_eq!( + (record.revision, record.first_sample, record.outcome_flags), + c_abi.watermark(), + "{rate} Hz: the control-side record" + ); + for block in 4..8 { + let _ = (browser.step(block), c_abi.step(block)); + assert_eq!( + status_watermark(&browser), + c_abi.watermark(), + "{rate} Hz: block {block}" + ); + } + } +} + /// Block the bypass of `a_live_bypass_crossfades_alike_on_both_hosts` is submitted before. const BYPASS_BLOCK: u64 = 6; @@ -1726,7 +1939,7 @@ fn every_scripted_edit_is_heard() { // A step that a later step of the same block replaces (same strip's fader, same strip's // pan, same insert parameter) is superseded by design, before any render reads it: that // is what the equal superseded counts check, so it is never heard. - if !matches!(full[omitted].outcome, Outcome::Accepted) + if !matches!(full[omitted].outcome, Outcome::Accepted | Outcome::Rebuilt) || full[omitted + 1..].iter().any(|later| { later.block == full[omitted].block && replaces(&full[omitted].edit, &later.edit) }) @@ -1903,3 +2116,167 @@ fn both_hosts_bound_the_vcas_alike() { assert_eq!(c_diagnostics, browser_diagnostics, "VCA count"); assert_eq!(c_diagnostics, b"host.resource.count\t$\n".to_vec()); } + +// --------------------------------------------------------------------------------------------- +// A rebuild adds a source the boot session did not declare (#1562b review B1) +// --------------------------------------------------------------------------------------------- + +/// The longest source ID the grammar allows. +fn longest_id() -> String { + format!("s{}", "x".repeat(session::STABLE_ID_MAXIMUM_BYTES - 1)) +} + +/// A session whose only source is mono (ID `m`), with one track and one route, no console slot, +/// no submix and no VCA, and the fixture's stereo source, track and route to add later. (A session +/// with no source or track does not prepare: its graph is empty.) +fn mono_model( + rate: u32, +) -> ( + SessionModel, + session::Source, + session::Track, + session::Route, +) { + let mut model = model_at(rate); + let (source, mut track, route) = ( + model.sources[0].clone(), + model.tracks[0].clone(), + model.routes[0].clone(), + ); + track.console.clear(); + track.inserts.effects.clear(); + model.console.pre_insert.clear(); + model.console.post_insert.clear(); + model.submixes.clear(); + model.vcas.clear(); + let (mut mono_source, mut mono_track, mut mono_route) = + (source.clone(), track.clone(), route.clone()); + mono_source.id = id("m"); + mono_source.channels = 1; + mono_track.id = id("m"); + mono_track.source_id = id("m"); + mono_track.left_source_channel = 0; + mono_track.right_source_channel = 0; + mono_route.id = id("m-main"); + let session::RouteSource::Track { track_id, .. } = &mut mono_route.source else { + panic!("the fixture routes a track") + }; + *track_id = id("m"); + model.sources = vec![mono_source]; + model.tracks = vec![mono_track]; + model.routes = vec![mono_route]; + (model, source, track, route) +} + +/// Both hosts boot a session whose only source is mono; a rebuild then adds a stereo source whose ID has the +/// longest length the grammar allows; the browser lists it in its session map and takes its PCM, +/// and both hosts render the same bits for it. +/// +/// Test value: red if a rebuild that adds a source (the longest ID, stereo, after a mono-only boot) +/// is rendered or fed differently by the browser than by the C ABI: the source rows, the swap, the +/// fresh-generation seek and the chunk path agree bit for bit. The staging the exports use for the +/// same session is gated in `plan_exchange_threads.rs`, since this row drives the host's safe API. +#[test] +fn a_rebuild_adds_the_longest_stereo_source_to_a_session_that_booted_with_a_mono_source_only() { + for rate in RATES { + let (model, mut source, mut track, mut route) = mono_model(rate); + let name = longest_id(); + source.id = id(&name); + source.channels = 2; + source.frames = u64::from(QUANTUM) * (BLOCKS + 8); + track.id = id("t1"); + track.source_id = source.id.clone(); + route.id = id("t1-main"); + let session::RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("the fixture routes a track") + }; + *track_id = track.id.clone(); + let edits = [ + SessionEdit::UpsertSource { source }, + SessionEdit::UpsertTrack { track }, + SessionEdit::UpsertRoute { route }, + ]; + + let mut browser = Rig::new(&model); + let mut c_abi = CAbi::new(&model); + assert_eq!( + browser.host.session_source_count(), + 1, + "{rate} Hz: boots mono" + ); + for block in 0..2 { + assert_eq!(browser.host.render_next(), host_web::RESULT_OK); + let _ = c_abi.render(block); + } + let (rebuilt, b_reply) = browser.transaction_reply(&edits); + let c_reply = c_abi.send( + CommandPayload::SessionTransactionApply(&edits), + ExpectedRevision::Exact(SessionRevision(c_abi.revision)), + ); + assert_eq!(b_reply, c_reply, "{rate} Hz: the same reply bytes"); + assert!( + matches!( + rebuilt, + Verdict::Applied { + path: EditPath::Rebuild, + .. + } + ), + "{rate} Hz: adding a source is a rebuild, got {rebuilt:?}" + ); + // The swap block: the successor adopts at the next block boundary. + assert_eq!(browser.host.render_next(), host_web::RESULT_OK); + let _ = c_abi.render(2); + + assert_eq!( + browser.host.session_source_count(), + 2, + "{rate} Hz: the map lists it" + ); + // Canonical order is by ID: `m` then the long one. + assert_eq!(browser.host.session_source_id(1), Some(name.as_str())); + assert_eq!( + browser + .host + .session_source_shape(1) + .map(|shape| shape.channel_count), + Some(2) + ); + + // The source enters at the render clock the successor adopted at, so it starts at its own + // frame 0 from block 3 on: a seek to a new generation, then the chunks of that generation. + assert_eq!( + browser.host.seek_source(name.as_bytes(), 2, 0), + host_web::RESULT_OK + ); + // SAFETY: The session is live and used by this thread alone; the name outlives the call. + let code = unsafe { + capi::miso_engine_v1_source_seek(c_abi.session, name.as_ptr(), name.len() as u64, 2, 0) + }; + assert_eq!(code, C_RESULT_OK, "{rate} Hz: C ABI seek"); + let mut heard = false; + for block in 3..11 { + let [left, right] = planes(0, block); + assert_eq!( + browser.host.submit_source( + name.as_bytes(), + 2, + (block - 3) * u64::from(QUANTUM), + rate, + &[&left, &right], + QUANTUM, + false, + ), + host_web::RESULT_OK, + "{rate} Hz: the browser takes the new source's PCM" + ); + c_abi.submit_from(&name, 0, block, block - 3, 2); + assert_eq!(browser.host.render_next(), host_web::RESULT_OK); + let b = bits(browser.host.output_pcm().expect("browser output")); + let c = c_abi.render(block); + assert_eq!(b, c, "{rate} Hz: block {block}: the two hosts render alike"); + heard |= b.iter().any(|word| f32::from_bits(*word) != 0.0); + } + assert!(heard, "{rate} Hz: the new source is heard"); + } +} diff --git a/hosts/host-web/tests/fixtures/structural-frames.json b/hosts/host-web/tests/fixtures/structural-frames.json new file mode 100644 index 000000000..ae2149367 --- /dev/null +++ b/hosts/host-web/tests/fixtures/structural-frames.json @@ -0,0 +1,12 @@ +{ + "schema": "miso.host-web.structural-frames", + "revision": "1", + "frames": [ + { "name": "replace_last_strip", "hex": "4d49534f43544c000100000030000100030000005806000001000000000000000100000000000000040000000000000001000b013800000002000000000000000100020102000000010500000000000002000b01180000000100000000000000010009010700000074332d6d61696e0001000b013800000002000000000000000100020102000000010200000000000002000b011800000001000000000000000100090102000000743300000000000001000b019004000002000000000000000100020102000000000200000000000002000b0170040000010000000000000001000b01600400000a000000000000000100090102000000743400000000000002000901060000007372632d74330000030001010100000000000000000000000400010101000000010000000000000005000b01c8000000020000000000000001000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000002000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000007000b0118020000030000000000000001000b01b0000000060000000000000001000901050000006d756c746900000002000b014000000002000000000000000100010101000000010000000000000002000901190000006d69736f2e6d756c746962616e642d636f6d70726573736f720000000000000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000001000b01a8000000060000000000000001000901070000006c696d697465720002000b013800000002000000000000000100010101000000010000000000000002000901160000006d69736f2e747275652d7065616b2d6c696d69746572000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000001000b01a00000000600000000000000010009010500000064656c617900000002000b0130000000020000000000000001000101010000000100000000000000020009010a0000006d69736f2e64656c617900000000000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000009000b014800000004000000000000000100060104000000000080c0000000000200060104000000000080c00000000003000801010000000000000000000000040008010100000000000000000000000a000b016800000006000000000000000100010101000000020000000000000002000601040000006666663f000000000300060104000000cdcccc3d000000000400060104000000cdcc4c3e000000000500060104000000cdcc4c3f00000000060003010400000000000000000000000b000b0128000000020000000000000001000901080000007072652d636f6d70020008010100000000000000000000000b000b013000000002000000000000000100090109000000706f73742d636c6970000000000000000200080101000000000000000000000001000b013801000002000000000000000100020102000000000500000000000002000b0118010000010000000000000001000b01080100000700000000000000010009010700000074342d6d61696e0002000b0138000000030000000000000001000101010000000100000000000000020009010200000074340000000000000300010101000000070000000000000003000b012800000002000000000000000100010101000000020000000000000002000901080000006d61696e2d6f757404000b0148000000040000000000000001000601040000000000803f00000000020006010400000000000000000000000300060104000000000000000000000004000601040000000000803f00000000050006010400000000000000000000000600080101000000000000000000000007000801010000000000000000000000" }, + { "name": "add_first_strip", "hex": "4d49534f43544c000100000030000100030000004006000001000000000000000100000000000000020000000000000001000b01f804000002000000000000000100020102000000000200000000000002000b01d8040000010000000000000001000b01c80400000a000000000000000100090102000000613000000000000002000901060000007372632d74310000030001010100000000000000000000000400010101000000010000000000000005000b01c8000000020000000000000001000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000002000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000007000b01a0020000020000000000000001000b01e80100000a000000000000000100090102000000657100000000000002000b013800000002000000000000000100010101000000010000000000000002000901120000006d69736f2e706172616d65747269632d657100000000000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000006000b0148000000040000000000000001000301040000000100000000000000020001010100000003000000000000000300010101000000050000000000000004000601040000000000803f0000000006000b014800000004000000000000000100030104000000020000000000000002000101010000000300000000000000030001010100000005000000000000000400060104000000000000400000000006000b01480000000400000000000000010003010400000003000000000000000200010101000000030000000000000003000101010000000200000000000000040006010400000000007a440000000006000b014800000004000000000000000100030104000000040000000000000002000101010000000300000000000000030001010100000001000000000000000400060104000000000000000000000007000b011800000001000000000000000100010101000000010000000000000001000b01a000000006000000000000000100090104000000636f6d700000000002000b0130000000020000000000000001000101010000000100000000000000020009010f0000006d69736f2e636f6d70726573736f720003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000009000b014800000004000000000000000100060104000000000000c0000000000200060104000000000000c00000000003000801010000000100000000000000040008010100000001000000000000000a000b01480000000400000000000000010001010100000001000000000000000200060104000000cdcc4cbf0000000003000601040000009a99193f000000000400030104000000c8000000000000000b000b0128000000020000000000000001000901080000007072652d636f6d70020008010100000000000000000000000b000b013000000002000000000000000100090109000000706f73742d636c6970000000000000000200080101000000000000000000000001000b013801000002000000000000000100020102000000000500000000000002000b0118010000010000000000000001000b01080100000700000000000000010009010700000061302d6d61696e0002000b0138000000030000000000000001000101010000000100000000000000020009010200000061300000000000000300010101000000070000000000000003000b012800000002000000000000000100010101000000020000000000000002000901080000006d61696e2d6f757404000b0148000000040000000000000001000601040000000000803f00000000020006010400000000000000000000000300060104000000000000000000000004000601040000000000803f00000000050006010400000000000000000000000600080101000000000000000000000007000801010000000000000000000000" }, + { "name": "swap_comp_for_soft_clip", "hex": "4d49534f43544c000100000030000100030000000005000001000000000000000100000000000000010000000000000001000b01f804000002000000000000000100020102000000000200000000000002000b01d8040000010000000000000001000b01c80400000a000000000000000100090102000000743100000000000002000901060000007372632d74310000030001010100000000000000000000000400010101000000010000000000000005000b01c8000000020000000000000001000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000002000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000007000b01a0020000020000000000000001000b01e80100000a000000000000000100090102000000657100000000000002000b013800000002000000000000000100010101000000010000000000000002000901120000006d69736f2e706172616d65747269632d657100000000000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000006000b0148000000040000000000000001000301040000000100000000000000020001010100000003000000000000000300010101000000050000000000000004000601040000000000803f0000000006000b014800000004000000000000000100030104000000020000000000000002000101010000000300000000000000030001010100000005000000000000000400060104000000000000400000000006000b01480000000400000000000000010003010400000003000000000000000200010101000000030000000000000003000101010000000200000000000000040006010400000000007a440000000006000b014800000004000000000000000100030104000000040000000000000002000101010000000300000000000000030001010100000001000000000000000400060104000000000000000000000007000b011800000001000000000000000100010101000000010000000000000001000b01a000000006000000000000000100090104000000636f6d700000000002000b0130000000020000000000000001000101010000000100000000000000020009010e0000006d69736f2e736f66742d636c6970000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000009000b014800000004000000000000000100060104000000000000c0000000000200060104000000000000c00000000003000801010000000000000000000000040008010100000000000000000000000a000b01480000000400000000000000010001010100000001000000000000000200060104000000cdcc4cbf0000000003000601040000009a99193f000000000400030104000000c8000000000000000b000b0128000000020000000000000001000901080000007072652d636f6d70020008010100000000000000000000000b000b013000000002000000000000000100090109000000706f73742d636c69700000000000000002000801010000000000000000000000" }, + { "name": "remove_comp", "hex": "4d49534f43544c000100000030000100030000005804000001000000000000000100000000000000010000000000000001000b015004000002000000000000000100020102000000000200000000000002000b0130040000010000000000000001000b01200400000a000000000000000100090102000000743100000000000002000901060000007372632d74310000030001010100000000000000000000000400010101000000010000000000000005000b01c8000000020000000000000001000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000002000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000007000b01f8010000010000000000000001000b01e80100000a000000000000000100090102000000657100000000000002000b013800000002000000000000000100010101000000010000000000000002000901120000006d69736f2e706172616d65747269632d657100000000000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000006000b0148000000040000000000000001000301040000000100000000000000020001010100000003000000000000000300010101000000050000000000000004000601040000000000803f0000000006000b014800000004000000000000000100030104000000020000000000000002000101010000000300000000000000030001010100000005000000000000000400060104000000000000400000000006000b01480000000400000000000000010003010400000003000000000000000200010101000000030000000000000003000101010000000200000000000000040006010400000000007a440000000006000b014800000004000000000000000100030104000000040000000000000002000101010000000300000000000000030001010100000001000000000000000400060104000000000000000000000007000b011800000001000000000000000100010101000000010000000000000009000b014800000004000000000000000100060104000000000000c0000000000200060104000000000000c00000000003000801010000000000000000000000040008010100000000000000000000000a000b01480000000400000000000000010001010100000001000000000000000200060104000000cdcc4cbf0000000003000601040000009a99193f000000000400030104000000c8000000000000000b000b0128000000020000000000000001000901080000007072652d636f6d70020008010100000000000000000000000b000b013000000002000000000000000100090109000000706f73742d636c69700000000000000002000801010000000000000000000000" }, + { "name": "remove_eq", "hex": "4d49534f43544c000100000030000100030000001003000001000000000000000100000000000000010000000000000001000b010803000002000000000000000100020102000000000200000000000002000b01e8020000010000000000000001000b01d80200000a000000000000000100090102000000743100000000000002000901060000007372632d74310000030001010100000000000000000000000400010101000000010000000000000005000b01c8000000020000000000000001000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000002000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000007000b01b0000000010000000000000001000b01a000000006000000000000000100090104000000636f6d700000000002000b0130000000020000000000000001000101010000000100000000000000020009010f0000006d69736f2e636f6d70726573736f720003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000009000b014800000004000000000000000100060104000000000000c0000000000200060104000000000000c00000000003000801010000000000000000000000040008010100000000000000000000000a000b01480000000400000000000000010001010100000001000000000000000200060104000000cdcc4cbf0000000003000601040000009a99193f000000000400030104000000c8000000000000000b000b0128000000020000000000000001000901080000007072652d636f6d70020008010100000000000000000000000b000b013000000002000000000000000100090109000000706f73742d636c69700000000000000002000801010000000000000000000000" }, + { "name": "swap_t3_multi_and_limiter", "hex": "4d49534f43544c000100000030000100030000009804000001000000000000000100000000000000010000000000000001000b019004000002000000000000000100020102000000000200000000000002000b0170040000010000000000000001000b01600400000a000000000000000100090102000000743300000000000002000901060000007372632d74330000030001010100000000000000000000000400010101000000010000000000000005000b01c8000000020000000000000001000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000002000b01580000000500000000000000010008010100000000000000000000000200060104000000000000000000000003000601040000000000000000000000040006010400000000000000000000000500030104000000000000000000000007000b0118020000030000000000000001000b01a8000000060000000000000001000901070000006c696d697465720002000b013800000002000000000000000100010101000000010000000000000002000901160000006d69736f2e747275652d7065616b2d6c696d69746572000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000001000b01b0000000060000000000000001000901050000006d756c746900000002000b014000000002000000000000000100010101000000010000000000000002000901190000006d69736f2e6d756c746962616e642d636f6d70726573736f720000000000000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000001000b01a00000000600000000000000010009010500000064656c617900000002000b0130000000020000000000000001000101010000000100000000000000020009010a0000006d69736f2e64656c617900000000000003000101010000000200000000000000040008010100000000000000000000000500010101000000010000000000000007000b011800000001000000000000000100010101000000010000000000000009000b014800000004000000000000000100060104000000000080c0000000000200060104000000000080c00000000003000801010000000000000000000000040008010100000000000000000000000a000b016800000006000000000000000100010101000000020000000000000002000601040000006666663f000000000300060104000000cdcccc3d000000000400060104000000cdcc4c3e000000000500060104000000cdcc4c3f00000000060003010400000000000000000000000b000b0128000000020000000000000001000901080000007072652d636f6d70020008010100000000000000000000000b000b013000000002000000000000000100090109000000706f73742d636c69700000000000000002000801010000000000000000000000" } + ] +} diff --git a/hosts/host-web/tests/plan_exchange_threads.rs b/hosts/host-web/tests/plan_exchange_threads.rs index 1a442d984..b91c3ab1f 100644 --- a/hosts/host-web/tests/plan_exchange_threads.rs +++ b/hosts/host-web/tests/plan_exchange_threads.rs @@ -206,9 +206,9 @@ fn run(republish: bool) -> Run { feed(handle, block); if republish && block == PUBLISH_BEFORE { assert_eq!(native_staging::active_plan_epoch(handle), Some(0)); - let response = native_staging::structural_command(handle, &request) - .expect("a live handle") - .expect("the structural transaction publishes"); + let (result, response) = + native_staging::control_command(handle, &request).expect("a live handle"); + assert_eq!(result, RESULT_OK, "the structural transaction publishes"); assert_rebuild_applied(&response, model.revision + 1); assert_eq!( native_staging::active_plan_epoch(handle), @@ -349,9 +349,9 @@ fn the_watermark_export_copies_the_applied_revision_and_refuses_a_wrong_handle() // A published revision is not applied until render has adopted it. let request = transaction_bytes(&add_muted_track(&model, "a-muted"), model.revision); - let response = native_staging::structural_command(handle, &request) - .expect("a live handle") - .expect("the transaction publishes"); + let (result, response) = + native_staging::control_command(handle, &request).expect("a live handle"); + assert_eq!(result, RESULT_OK, "the structural transaction publishes"); assert_rebuild_applied(&response, model.revision + 1); assert_eq!( host_web::miso_engine_web_v1_watermark_read(handle), @@ -399,3 +399,91 @@ fn the_watermark_export_copies_the_applied_revision_and_refuses_a_wrong_handle() .join() .expect("the watermark phase completes"); } + +/// A rebuild adds a source whose ID has the longest length the grammar allows and whose channel +/// count exceeds every boot source's (#1562b review B1). Through the exports the worklet calls: +/// `source_id` copies the ID into staging (it trapped on a staging sized from the boot session), +/// the channel and frame queries answer the session map, and the source takes a six-plane chunk +/// and is heard (the staging held two planes, so the worklet's source rows failed and the source +/// could never be fed). +/// +/// Test value: red if the browser's ID or PCM staging is sized from the boot session. The parity +/// row in `control_parity.rs` compares the bits against the C ABI through the host's safe API, +/// which bypasses the staging; only the exports exercise it. +#[test] +fn a_rebuild_adds_the_longest_id_with_more_channels_than_any_boot_source() { + let _turn = TURN.lock().expect("the turn"); + std::thread::spawn(|| { + let handle = boot(); + let model = parse_session_json(&document()).expect("the fixture parses"); + let name = format!("s{}", "x".repeat(session::STABLE_ID_MAXIMUM_BYTES - 1)); + let mut source = model.sources[0].clone(); + source.id = StableId::parse(&name).expect("source ID"); + source.channels = 6; + let mut track = model.tracks[0].clone(); + track.id = StableId::parse("wide").expect("track ID"); + track.source_id = source.id.clone(); + track.left_source_channel = 4; + track.right_source_channel = 5; + let mut route = model.routes[0].clone(); + route.id = StableId::parse("wide-main").expect("route ID"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("a track route") + }; + *track_id = track.id.clone(); + let edits = [ + SessionEdit::UpsertSource { source }, + SessionEdit::UpsertTrack { track }, + SessionEdit::UpsertRoute { route }, + ]; + let request = transaction_bytes(&edits, model.revision); + let (result, response) = + native_staging::control_command(handle, &request).expect("a live handle"); + assert_eq!(result, RESULT_OK, "the structural transaction publishes"); + assert_rebuild_applied(&response, model.revision + 1); + assert_eq!( + host_web::miso_engine_web_v1_render(handle, QUANTUM_FRAMES), + RESULT_OK, + "the swap block" + ); + + // The session map: canonical order puts the new ID after `fixture-source`. + assert_eq!(host_web::miso_engine_web_v1_source_count(handle), 2); + let length = host_web::miso_engine_web_v1_source_id(handle, 1); + assert_eq!( + length as usize, + name.len(), + "the ID is not truncated or refused" + ); + assert_eq!( + native_staging::staged_source_id(handle, length).as_deref(), + Some(name.as_bytes()) + ); + assert_eq!(host_web::miso_engine_web_v1_source_channels(handle, 1), 6); + + // Feed it a six-plane chunk of its own generation and hear it. + assert!(native_staging::source_id(handle, name.as_bytes())); + assert_eq!( + host_web::miso_engine_web_v1_source_seek(handle, length, 2, 0), + RESULT_OK + ); + assert!(native_staging::source_pcm(handle, 0.5)); + assert_eq!( + host_web::miso_engine_web_v1_source_submit(handle, length, 2, 0, 6, QUANTUM_FRAMES, 0), + RESULT_OK, + "a six-channel chunk fits the staging" + ); + assert_eq!( + host_web::miso_engine_web_v1_render(handle, QUANTUM_FRAMES), + RESULT_OK + ); + let heard = native_staging::output_pcm(handle) + .expect("output") + .iter() + .any(|sample| *sample != 0.0); + assert!(heard, "the new source is heard"); + assert_eq!(host_web::miso_engine_web_v1_dispose(handle), RESULT_OK); + }) + .join() + .expect("the run completes"); +} diff --git a/hosts/host-web/tests/protocol_cross_host.rs b/hosts/host-web/tests/protocol_cross_host.rs index 634b6b285..64b9d460e 100644 --- a/hosts/host-web/tests/protocol_cross_host.rs +++ b/hosts/host-web/tests/protocol_cross_host.rs @@ -215,7 +215,7 @@ impl Browser { .control_request_mut(length) .expect("the request staging")[..frame.len()] .copy_from_slice(frame); - let result = self.host.control_command(length); + let result = self.host.control_command(length, None); (result, self.host.control_response().to_vec()) } @@ -252,7 +252,7 @@ impl Browser { } fn service(&mut self) { - assert_eq!(self.host.service(), RESULT_OK, "browser service"); + assert_eq!(self.host.service(None), RESULT_OK, "browser service"); } } diff --git a/hosts/host-web/tests/retained_ceilings.rs b/hosts/host-web/tests/retained_ceilings.rs index 9fe2f9e8a..38b70dfe4 100644 --- a/hosts/host-web/tests/retained_ceilings.rs +++ b/hosts/host-web/tests/retained_ceilings.rs @@ -99,6 +99,14 @@ const CAPTURE: u64 = LIVE_RESPONSE_CAPTURE_BYTES as u64; /// because the plan's lanes are the control plane's (decision 17 C6), which attaches the strips' /// input lanes (trim, polarity and filters) where the browser's no-commands options attached none: /// 4,041 * 1.1 = 4,445.1, so 4,480. The four-lane values are inferred, as before. +/// +/// **Re-pinned: `bridge_retained_bytes` (#1562b review B1).** The browser sizes its source PCM +/// staging, its plane references and its ID staging by what the grammar allows (255 channels, 127 +/// ID bytes), not by the boot session, so a rebuild can add any valid source. For this fixture the +/// retained row rose from 1,631,181 to 1,764,878 bytes (x86-64; +133,697: the PCM staging from 2 +/// to 255 channels of the quantum, the plane references and the ID staging); the metadata row rose +/// by the plane references alone (1,594,186 to 1,598,234) and stays under its ceiling. The new +/// beyond-capture figure is 716,302; plus 10 % is 787,932.2, rounded up to 64: 787,968. const BUDGETS: [Budget; 12] = [ Budget { row: "bridge_metadata_bytes", @@ -109,8 +117,8 @@ const BUDGETS: [Budget; 12] = [ Budget { row: "bridge_retained_bytes", value: |report| report.bridge_retained_bytes, - eight_lanes: CAPTURE + 611_712, - four_lanes: CAPTURE + 611_712, + eight_lanes: CAPTURE + 787_968, + four_lanes: CAPTURE + 787_968, }, Budget { row: "source_total_bytes", diff --git a/hosts/host-web/tests/structural_apply_realtime.rs b/hosts/host-web/tests/structural_apply_realtime.rs index 3eb69a530..15151e706 100644 --- a/hosts/host-web/tests/structural_apply_realtime.rs +++ b/hosts/host-web/tests/structural_apply_realtime.rs @@ -264,12 +264,12 @@ fn run(script: Script) -> Run { let revision = host_web::miso_engine_web_v1_control_revision(handle); assert_eq!(native_staging::active_plan_epoch(handle), Some(0)); let control = host_web::miso_engine_web_v1_control_allocation_count(); - let response = native_staging::structural_command( + let (result, response) = native_staging::control_command( handle, &frame(&add_muted_track(&base), request, revision), ) - .expect("a live handle") - .expect("the structural transaction publishes"); + .expect("a live handle"); + assert_eq!(result, RESULT_OK, "the structural transaction publishes"); run.control_calls_in_apply = host_web::miso_engine_web_v1_control_allocation_count() - control; assert_eq!( @@ -449,12 +449,12 @@ fn run_metered(structural: bool) -> Metered { feed(handle, block); if structural && block == ADOPTION_WITH_WINDOW { let revision = host_web::miso_engine_web_v1_control_revision(handle); - native_staging::structural_command( + let (result, _) = native_staging::control_command( handle, &frame(&add_muted_track(&base), 1, revision), ) - .expect("a live handle") - .expect("the structural transaction publishes"); + .expect("a live handle"); + assert_eq!(result, RESULT_OK, "the structural transaction publishes"); } let allocations = host_web::miso_engine_web_v1_render_allocation_count(); assert_eq!( diff --git a/hosts/host-web/tests/structural_frames_fixture.rs b/hosts/host-web/tests/structural_frames_fixture.rs new file mode 100644 index 000000000..1a8cf6d81 --- /dev/null +++ b/hosts/host-web/tests/structural_frames_fixture.rs @@ -0,0 +1,243 @@ +//! Issue #1562b-K: the structural command frames the SDK's structure evals send. +//! +//! The SDK has no encoder for a structural edit (its transaction API is later work), so the evals +//! that follow a rebuild -- a strip added, an insert swapped for another effect, an insert removed -- +//! send frames the protocol codec wrote. This test writes them beside the #1560 parity session +//! (`fixtures/control-list.json`, which the evals boot), as `fixtures/structural-frames.json`. A +//! frame is a wire format, so a byte comparison is the right gate: it turns red when the codec or +//! the parity session changes without the frames following. +//! +//! The frames carry request ID 1 and the boot revision; the evals set both words of the header +//! (bytes 24 and 32) before they send a frame, because the SDK's endpoint allocates the IDs. +//! +//! Run with `MISO_ENGINE_BLESS_FIXTURES=1` to rewrite the fixture after a deliberate change. + +use std::fmt::Write as _; +use std::path::PathBuf; + +use protocol::{ + CommandPayload, ExpectedRevision, ProtocolCodec, RequestId, SessionEdit, SessionRevision, + TypedCommandFrame, +}; +use session::{RouteSource, SessionModel, StableId, canonical_session_json, parse_session_json}; + +mod common; +use common::model_at; + +const FIXTURE: &str = "tests/fixtures/structural-frames.json"; +const RATE: u32 = 48_000; + +fn id(value: &str) -> StableId { + StableId::parse(value).expect("literal stable ID") +} + +/// The booted model: the parse of the canonical parity document, as the evals boot it. +fn booted() -> SessionModel { + let document = canonical_session_json(&model_at(RATE)).expect("canonical parity session"); + parse_session_json(&document).expect("the canonical document parses") +} + +fn frame(edits: &[SessionEdit], revision: u64) -> Vec { + let codec = ProtocolCodec::default(); + let mut bytes = vec![0_u8; codec.limits().max_frame_bytes]; + let length = codec + .encode_command_frame_into( + &TypedCommandFrame { + request_id: RequestId::new(1).expect("nonzero request"), + expected_revision: ExpectedRevision::Exact(SessionRevision(revision)), + payload: CommandPayload::SessionTransactionApply(edits), + }, + &mut bytes, + ) + .expect("a transaction frame"); + bytes.truncate(length); + bytes +} + +/// `t1` with its `comp` insert changed by `change` (`None` removes it). +fn t1_with_comp(model: &SessionModel, change: Option) -> SessionEdit { + let mut track = model.tracks[0].clone(); + assert_eq!(track.id.as_str(), "t1"); + let at = track + .inserts + .effects + .iter() + .position(|effect| effect.id.as_str() == "comp") + .expect("t1 has a comp insert"); + match change { + Some(effect) => track.inserts.effects[at] = effect, + None => { + track.inserts.effects.remove(at); + } + } + SessionEdit::UpsertTrack { track } +} + +/// `t1` without its `eq` insert, so `comp` is the first insert. +fn t1_without_eq(model: &SessionModel) -> SessionEdit { + let mut track = model.tracks[0].clone(); + assert_eq!(track.id.as_str(), "t1"); + track + .inserts + .effects + .retain(|effect| effect.id.as_str() != "eq"); + assert_eq!(track.inserts.effects.len(), 1); + SessionEdit::UpsertTrack { track } +} + +/// `t3` with its first two inserts, `multi` and `limiter`, swapped. Both declare the same tap +/// (`Gain Reduction`, ID 1), so a record for one effect's index arms the other's tap when the +/// indices belong to another plan than the one the core admits it against (#1562b, batch 12b). +fn t3_with_multi_and_limiter_swapped(model: &SessionModel) -> SessionEdit { + let mut track = model + .tracks + .iter() + .find(|track| track.id.as_str() == "t3") + .expect("the parity session has t3") + .clone(); + assert_eq!(track.inserts.effects[0].id.as_str(), "multi"); + assert_eq!(track.inserts.effects[1].id.as_str(), "limiter"); + track.inserts.effects.swap(0, 1); + SessionEdit::UpsertTrack { track } +} + +fn frames() -> Vec<(&'static str, Vec)> { + let model = booted(); + let revision = model.revision; + // A muted copy of t1 that sorts before it, with its route to the output: every other strip + // moves one place later in the strip order, and the mixed output keeps its bits. + let mut added = model.tracks[0].clone(); + added.id = id("a0"); + added.fader.left_mute = true; + added.fader.right_mute = true; + let mut route = model + .routes + .iter() + .find(|route| route.id.as_str() == "t1-main") + .expect("t1's route to the output") + .clone(); + route.id = id("a0-main"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("a track route") + }; + *track_id = added.id.clone(); + // The soft clip of t2 under the compressor's ID: the same insert now runs another effect, + // which has no threshold row and no observation tap. + let mut clip = model.tracks[1] + .inserts + .effects + .iter() + .find(|effect| effect.id.as_str() == "clip") + .expect("t2 has a clip insert") + .clone(); + clip.id = id("comp"); + // t3 replaced by a copy under another ID, with its route: the strip count stays three and the + // strip order changes, which a count alone cannot show. + let t3 = model + .tracks + .iter() + .find(|track| track.id.as_str() == "t3") + .expect("the parity session has t3"); + let mut replacement = t3.clone(); + replacement.id = id("t4"); + let t3_route = model + .routes + .iter() + .find(|route| route.id.as_str() == "t3-main") + .expect("t3's route to the output"); + let mut replacement_route = t3_route.clone(); + replacement_route.id = id("t4-main"); + let RouteSource::Track { track_id, .. } = &mut replacement_route.source else { + panic!("a track route") + }; + *track_id = replacement.id.clone(); + vec![ + ( + "replace_last_strip", + frame( + &[ + SessionEdit::RemoveRoute { + route_id: t3_route.id.clone(), + }, + SessionEdit::RemoveTrack { + track_id: t3.id.clone(), + }, + SessionEdit::UpsertTrack { track: replacement }, + SessionEdit::UpsertRoute { + route: replacement_route, + }, + ], + revision, + ), + ), + ( + "add_first_strip", + frame( + &[ + SessionEdit::UpsertTrack { track: added }, + SessionEdit::UpsertRoute { route }, + ], + revision, + ), + ), + ( + "swap_comp_for_soft_clip", + frame(&[t1_with_comp(&model, Some(clip))], revision), + ), + ( + "remove_comp", + frame(&[t1_with_comp(&model, None)], revision), + ), + // The compressor stays and moves from the second insert to the first. + ("remove_eq", frame(&[t1_without_eq(&model)], revision)), + // `multi` and `limiter` change places: each moves to the other's index. + ( + "swap_t3_multi_and_limiter", + frame(&[t3_with_multi_and_limiter_swapped(&model)], revision), + ), + ] +} + +fn render() -> String { + let model = booted(); + let mut text = String::new(); + writeln!(text, "{{").expect("write to a string"); + writeln!( + text, + " \"schema\": \"miso.host-web.structural-frames\",\n \"revision\": \"{}\",\n \"frames\": [", + model.revision + ) + .expect("write to a string"); + let frames = frames(); + for (index, (name, bytes)) in frames.iter().enumerate() { + let hex = bytes.iter().fold(String::new(), |mut hex, byte| { + write!(hex, "{byte:02x}").expect("write to a string"); + hex + }); + let comma = if index + 1 == frames.len() { "" } else { "," }; + writeln!( + text, + " {{ \"name\": \"{name}\", \"hex\": \"{hex}\" }}{comma}" + ) + .expect("write to a string"); + } + writeln!(text, " ]\n}}").expect("write to a string"); + text +} + +/// Red if the codec or the parity session changes without the frames the SDK evals send following. +#[test] +fn the_structural_frames_fixture_is_what_the_codec_writes() { + let path = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join(FIXTURE); + let expected = render(); + if std::env::var_os("MISO_ENGINE_BLESS_FIXTURES").is_some() { + std::fs::create_dir_all(path.parent().expect("fixture directory")).expect("mkdir"); + std::fs::write(&path, &expected).expect("write the fixture"); + return; + } + let actual = std::fs::read_to_string(&path).unwrap_or_default(); + assert!( + actual == expected, + "{FIXTURE} differs from the codec's frames; rerun with MISO_ENGINE_BLESS_FIXTURES=1 after a deliberate change" + ); +} diff --git a/hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts b/hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts index 7536e5077..9002425ec 100644 --- a/hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts +++ b/hosts/host-web/web/miso-engine-v1-audio-worklet-host.d.ts @@ -358,6 +358,14 @@ export interface MisoObservationMap { readonly tag: "miso.observationmap.v1"; readonly requestId: number; readonly result: number; + /// The structure epoch of the control list whose strip order `bindings` follow (issue 1562b): + /// the `structureEpoch` of `CONTROL_LIST_GET` pages, as the plan that renders carries it. A + /// binding's `trackIndex` indexes the tracks, then the submixes, of the list of this epoch. A + /// rebuild that adds, removes or reorders a strip changes it at the block that adopts the + /// replacement plan; between that commit and the adoption the control list is already the + /// replacement's, so a client names the strips of this map by the list of this epoch and never + /// by the newest list. A positive safe integer. + readonly structureEpoch: number; readonly bindings: MisoObservationMapBinding[]; } @@ -513,6 +521,14 @@ export interface MisoMeterFrame { /// The tracks of the plan that rendered the window. A structural apply that adds or removes a /// track changes it between two frames of one stream; every array below is sized for it. readonly trackCount: number; + /// The structure epoch of the control list that the plan's strip order belongs to (issue + /// 1562b): the `structureEpoch` that `CONTROL_LIST_GET` pages carry, as the commit that prepared + /// the plan left it. The frame's strips (the tracks, then the submixes) are named by the list + /// of this epoch, so a page that holds a list of another epoch reads the list again before it + /// names a frame's strips. A structural apply that adds, removes or reorders a strip changes it + /// between two frames of one stream, in the frame of the block that adopted the plan, and a + /// replacement that keeps the structure keeps it. A positive safe integer. + readonly structureEpoch: number; /// `[track0 L, track0 R, .., trackN L, trackN R, master L, master R]` peak magnitudes. readonly peaks: Float32Array; /// Gain reduction per track, in **non-negative decibels**, `trackCount` long (issue 143). @@ -778,9 +794,9 @@ export interface MisoRenderAllocations { /// The reply to one control frame (issue 1562a D1). `response` is the response frame the engine's /// control plane built, unchanged, when `result` is `0`; for another result it is the refusal's -/// diagnostic text (`code\tpath\n` lines), for example `web.apply.rebuild_pending` with `result: 10` -/// for a transaction that needs a plan rebuild, which this module cannot apply in place yet. A -/// refused transaction is still `result: 0` with a non-ok response frame, as on the C ABI. +/// diagnostic text (`code\tpath\n` lines), for example `web.apply.shape_unavailable` with `result: 10` +/// for a preparation that could not hand its plan to the host. A transaction that needs a plan +/// rebuild is prepared and published like any other (issue 1562b). A refused transaction is still `result: 0` with a non-ok response frame, as on the C ABI. /// `revision` is the committed session revision after the call. export interface MisoControlReply { readonly tag: "miso.control.v1"; @@ -912,9 +928,11 @@ export interface MisoAudioWorkletHost { request: { enabled: boolean; onFrame: ((frame: MisoTelemetryFrame) => void) | null }, ): Promise; /// @internal Subscribe to the engine's protocol events (issue 1570b D5): every command's - /// `SESSION_COMMITTED`, `TRANSPORT_STATE`, `METER_BATCH`, `COUNTER_SNAPSHOT`, `DIAGNOSTIC` and the - /// other events of the control protocol, as the engine encoded them. The listener gets each - /// event with its lane (`0` reliable, `1` lossy) in arrival order. Returns the unsubscribe + /// `SESSION_COMMITTED`, `OBSERVATION_ENDED` (reliable: a commit removed the effect of a + /// subscribed observation tap, and the event names every ended tap key, #1584), `TRANSPORT_STATE`, + /// `METER_BATCH`, `COUNTER_SNAPSHOT`, `DIAGNOSTIC` and the other events of the control protocol, + /// as the engine encoded them. The listener gets each event with its lane (`0` reliable, `1` + /// lossy) in arrival order. Returns the unsubscribe /// function. An event is not the ack of a command: the command's reply is. onControlEvent(listener: (event: MisoControlEvent) => void): () => void; /// @internal The count of engine events that arrived while no listener was subscribed (issue diff --git a/hosts/host-web/web/miso-engine-v1-audio-worklet-host.js b/hosts/host-web/web/miso-engine-v1-audio-worklet-host.js index 2bdea402b..5d9adf040 100644 --- a/hosts/host-web/web/miso-engine-v1-audio-worklet-host.js +++ b/hosts/host-web/web/miso-engine-v1-audio-worklet-host.js @@ -878,9 +878,9 @@ class MisoAudioWorkletHost { // `submixPeaks` a pair per bus and `submixGrDb` one magnitude per bus, every value finite // and non-negative. if (!hasExactFields(message, [ - "tag", "sequence", "generation", "validity", "lossCount", "windows", "trackCount", "peaks", - "trackGrDb", "masterGrDb", "firstSample", "endSample", "submixCount", "submixPeaks", - "submixGrDb", + "tag", "sequence", "generation", "validity", "lossCount", "windows", "trackCount", + "structureEpoch", "peaks", "trackGrDb", "masterGrDb", "firstSample", "endSample", + "submixCount", "submixPeaks", "submixGrDb", ]) || !Number.isSafeInteger(message.sequence) || message.sequence <= 0 || typeof message.generation !== "bigint" || message.generation <= 0n @@ -891,6 +891,7 @@ class MisoAudioWorkletHost { || ((message.validity & 0x4) !== 0) !== (message.lossCount > 0) || !Number.isSafeInteger(message.windows) || message.windows <= 0 || !Number.isSafeInteger(message.trackCount) || message.trackCount < 0 + || !Number.isSafeInteger(message.structureEpoch) || message.structureEpoch <= 0 || !(message.peaks instanceof Float32Array) || message.peaks.length !== message.trackCount * 2 + 2 || !(message.trackGrDb instanceof Float32Array) @@ -986,7 +987,7 @@ class MisoAudioWorkletHost { : pending.response === "sessionMap" ? ["tag", "requestId", "result", "sources", "metersAttached"] : pending.response === "observationMap" - ? ["tag", "requestId", "result", "bindings"] + ? ["tag", "requestId", "result", "structureEpoch", "bindings"] : pending.response === "observationRead" ? ["tag", "requestId", "result", "rows"] : pending.response === "trackResponse" @@ -1060,6 +1061,9 @@ class MisoAudioWorkletHost { ); const validObservationMap = pending.response !== "observationMap" || ( message.result === RESULT_OK && Array.isArray(message.bindings) + // Issue #1562b: the structure epoch of the control list whose strip order the bindings + // follow (the rendering plan's), a positive safe integer. + && Number.isSafeInteger(message.structureEpoch) && message.structureEpoch > 0 && message.bindings.every(validObservationMapBinding) ); const validObservationRead = pending.response !== "observationRead" || ( @@ -1372,8 +1376,8 @@ class MisoAudioWorkletHost { /// The reply is `{ result, response, revision }`: `response` is the response frame bytes /// unchanged on `RESULT_OK` (a refused transaction is still an ok call with a non-ok response /// frame, as on the C ABI), else the refusal's diagnostic text (`code\tpath\n` lines), for - /// example `web.apply.rebuild_pending` for a transaction that needs a plan rebuild, which this - /// module cannot apply in place yet. `revision` is the committed revision after the call. + /// example `web.apply.shape_unavailable` for a preparation that could not hand its plan to the + /// host (a transaction that needs a plan rebuild is prepared and published, #1562b). `revision` is the committed revision after the call. /// /// **Request IDs.** The frame carries the sender's own request ID, and the engine's replay cache /// refuses a reused ID with different bytes. One allocator must therefore serve every frame sent diff --git a/hosts/host-web/web/miso-engine-v1-audio-worklet.js b/hosts/host-web/web/miso-engine-v1-audio-worklet.js index 26cda062c..4da234947 100644 --- a/hosts/host-web/web/miso-engine-v1-audio-worklet.js +++ b/hosts/host-web/web/miso-engine-v1-audio-worklet.js @@ -4,10 +4,13 @@ const BOOT_OPTIONS_BYTES = 64; // Issue #1209 D2 appended `submix_count` at 64 and its pad at 68. Issue #1290d1 made the pad // `retired` and appended the frame and successor pointers: the frame travels with the plan, so the // header names its own frame and, while a replacement plan waits, the replacement's header. -const METER_HEADER_BYTES = 80; +// Issue #1562b appended `structure_epoch` (u64 at 80): the control list's structure epoch of the +// plan's strip order. +const METER_HEADER_BYTES = 88; const METER_HEADER_RETIRED = 68; const METER_HEADER_FRAME_POINTER = 72; const METER_HEADER_SUCCESSOR_POINTER = 76; +const METER_HEADER_STRUCTURE_EPOCH = 80; const RESULT_OK = 0; const RESULT_INVALID_ARGUMENT = 1; const RESULT_UNSUPPORTED = 7; @@ -452,6 +455,107 @@ class MisoEngineAudioWorkletProcessor extends AudioWorkletProcessor { return RESULT_OK; } + /// Issue #207, #1562b: the compiled session's sources in canonical order, read from the engine's + /// exports, or `null` for a row the engine cannot have written. Every read is checked against + /// something the engine already guarantees, so a mis-wired export fails here rather than + /// surfacing as a plausible-looking wrong number: compilation refuses a zero channel count or + /// full-source frame count, and the staging holds the most channels a source can + /// declare (`maximumSourceChannels`), so a channel count above it is a mis-wired export. The + /// sources are the committed model's, so a rebuild that adds or removes one is seen by the next + /// read. Control side only; allocates. + readSourceRows() { + const count = this.exports.miso_engine_web_v1_source_count(this.handle); + if (!u32(count)) return null; + const rows = []; + for (let index = 0; index < count; index += 1) { + const length = this.exports.miso_engine_web_v1_source_id(this.handle, index); + if (!u32(length) || length === 0 || length > this.sourceIdCapacity) return null; + const bytes = new Uint8Array(this.memoryBuffer, this.sourceIdPointer, length); + let id = ""; + for (let byte = 0; byte < length; byte += 1) { + if (bytes[byte] > 0x7f) return null; + id += String.fromCharCode(bytes[byte]); + } + const channels = this.exports.miso_engine_web_v1_source_channels(this.handle, index); + const frames = this.exports.miso_engine_web_v1_source_frames(this.handle, index); + if (!u32(channels) || channels === 0 || channels > this.maximumSourceChannels + || !u64(frames) || frames === 0n) return null; + rows.push({ id, channels, frames }); + } + return rows; + } + + /// Issue #143, #1562b: the observation bindings of the plan that renders, in the prepared + /// owner's stable map order, or `null` for a binding the engine cannot have written. A binding + /// names a strip by its index in that plan's strip order (the tracks, then the submixes), which + /// is the control list's order at [`Self::renderingStructureEpoch`]. A rebuild changes the + /// bindings at the block that adopts it, so every map request reads them again: a reboot or a + /// swap cannot retain an old numeric owner address behind a stable SDK selection. Control side + /// only; allocates. + readObservationBindings() { + const count = this.exports.miso_engine_web_v1_observation_count(this.handle); + if (!u32(count)) return null; + const idPointer = this.exports.miso_engine_web_v1_observation_id_ptr(); + const idCapacity = this.exports.miso_engine_web_v1_observation_id_capacity(); + if (!u32(idPointer) || idPointer === 0 || !u32(idCapacity) || idCapacity === 0) return null; + const readId = (length) => { + if (!u32(length) || length === 0 || length > idCapacity) return null; + const bytes = new Uint8Array(this.memoryBuffer, idPointer, length); + let id = ""; + for (let byte = 0; byte < length; byte += 1) { + if (bytes[byte] > 0x7f) return null; + id += String.fromCharCode(bytes[byte]); + } + return id; + }; + const bindings = []; + for (let index = 0; index < count; index += 1) { + const trackIndex = this.exports.miso_engine_web_v1_observation_track_index(this.handle, index); + const rack = this.exports.miso_engine_web_v1_observation_rack(this.handle, index); + const effectIndex = this.exports.miso_engine_web_v1_observation_effect_index(this.handle, index); + const effectSlotId = readId( + this.exports.miso_engine_web_v1_observation_effect_slot_id(this.handle, index), + ); + const nativeEffectId = readId( + this.exports.miso_engine_web_v1_observation_native_effect_id(this.handle, index), + ); + const tapCount = this.exports.miso_engine_web_v1_observation_tap_count(this.handle, index); + if (!u32(trackIndex) || !effectRack(rack) || !u32(effectIndex) + || effectSlotId === null || nativeEffectId === null || !u32(tapCount) + || tapCount === 0 || tapCount > 32) return null; + const tapIds = []; + for (let tapIndex = 0; tapIndex < tapCount; tapIndex += 1) { + const tapId = this.exports.miso_engine_web_v1_observation_tap_id( + this.handle, index, tapIndex, + ); + if (!u32(tapId) || tapId === 0) return null; + tapIds.push(tapId); + } + bindings.push(Object.freeze({ + trackIndex, + rack, + effectIndex, + effectSlotId, + nativeEffectId, + tapIds: Object.freeze(tapIds), + })); + } + return Object.freeze(bindings); + } + + /// Issue #1562b: the structure epoch of the plan that renders, read from its meter header (the + /// header the module publishes through `meter_header_ptr` is the rendering plan's, with or + /// without a meter lease). It is the number of the control list whose strip order the plan's + /// bindings and frames follow. `0` for a header the memory cannot hold. Control side only. + renderingStructureEpoch() { + const pointer = this.exports.miso_engine_web_v1_meter_header_ptr(this.handle); + if (!u32(pointer) || pointer === 0 + || pointer + METER_HEADER_BYTES > this.memoryBuffer.byteLength) return 0; + const header = new DataView(this.memoryBuffer, pointer, METER_HEADER_BYTES); + return header.getUint32(METER_HEADER_STRUCTURE_EPOCH, true) + + header.getUint32(METER_HEADER_STRUCTURE_EPOCH + 4, true) * 4294967296; + } + /// Cut one persistent view over the module's memory and remember how, so a growth of that memory /// can cut it again. Control side only. bindView(field, Constructor, pointer, length) { @@ -538,6 +642,11 @@ class MisoEngineAudioWorkletProcessor extends AudioWorkletProcessor { lossCount: 0, windows: 0, trackCount, + // Issue #1562b: the structure epoch of the control list the frame's strip order belongs + // to, so a page that holds a list of another epoch reads the list again before it names + // a frame's strips. Refreshed with every post; a safe integer (it advances by one at a + // structural commit). + structureEpoch: 0, // The frozen `2T + 2` track and master peaks: an existing reader indexes it exactly as // before, whatever the submix count. peaks: new Float32Array(trackCount * 2 + 2), @@ -745,88 +854,19 @@ class MisoEngineAudioWorkletProcessor extends AudioWorkletProcessor { return false; } - // Issue #207: source introspection, read once here for the same reason the other - // are -- the construction path is the one that may allocate, and `process()` never touches - // this. A headless consumer of the session map cannot otherwise learn which sources a compiled - // session declares, how many channels each carries, or which frames it is waiting for; the - // ABI exposed track discovery and nothing at all about sources. - // - // Every read is checked against something the engine already guarantees, so a mis-wired export - // fails initialization here rather than surfacing as a plausible-looking wrong number: - // compilation refuses a zero channel count or full-source frame count, and preparation refuses - // a channel count above `maximumSourceChannels`. - this.sourceCount = this.exports.miso_engine_web_v1_source_count(this.handle); - if (!u32(this.sourceCount)) return false; - this.sources = []; - for (let index = 0; index < this.sourceCount; index += 1) { - const length = this.exports.miso_engine_web_v1_source_id(this.handle, index); - if (!u32(length) || length === 0 || length > this.sourceIdCapacity) return false; - const bytes = new Uint8Array(this.memoryBuffer, this.sourceIdPointer, length); - let id = ""; - for (let byte = 0; byte < length; byte += 1) { - if (bytes[byte] > 0x7f) return false; - id += String.fromCharCode(bytes[byte]); - } - const channels = this.exports.miso_engine_web_v1_source_channels(this.handle, index); - const frames = this.exports.miso_engine_web_v1_source_frames(this.handle, index); - if (!u32(channels) || channels === 0 || channels > this.maximumSourceChannels - || !u64(frames) || frames === 0n) return false; - this.sources.push({ id, channels, frames }); - } - - // Observation bindings and the fixed selection staging view are control-path state. They are - // read by request handlers only; the render callback never follows an observation pointer or - // allocates a reply row. The map is rebuilt at each boot so a reboot cannot retain an old - // numeric owner address behind a stable SDK selection. - const observationCount = this.exports.miso_engine_web_v1_observation_count(this.handle); - if (!u32(observationCount)) return false; - const observationIdPointer = this.exports.miso_engine_web_v1_observation_id_ptr(); - const observationIdCapacity = this.exports.miso_engine_web_v1_observation_id_capacity(); - if (!u32(observationIdPointer) || observationIdPointer === 0 - || !u32(observationIdCapacity) || observationIdCapacity === 0) return false; - const readObservationId = (length) => { - if (!u32(length) || length === 0 || length > observationIdCapacity) return null; - const bytes = new Uint8Array(this.memoryBuffer, observationIdPointer, length); - let id = ""; - for (let byte = 0; byte < length; byte += 1) { - if (bytes[byte] > 0x7f) return null; - id += String.fromCharCode(bytes[byte]); - } - return id; - }; - const bindings = []; - for (let index = 0; index < observationCount; index += 1) { - const trackIndex = this.exports.miso_engine_web_v1_observation_track_index(this.handle, index); - const rack = this.exports.miso_engine_web_v1_observation_rack(this.handle, index); - const effectIndex = this.exports.miso_engine_web_v1_observation_effect_index(this.handle, index); - const effectSlotId = readObservationId( - this.exports.miso_engine_web_v1_observation_effect_slot_id(this.handle, index), - ); - const nativeEffectId = readObservationId( - this.exports.miso_engine_web_v1_observation_native_effect_id(this.handle, index), - ); - const tapCount = this.exports.miso_engine_web_v1_observation_tap_count(this.handle, index); - if (!u32(trackIndex) || !effectRack(rack) || !u32(effectIndex) - || effectSlotId === null || nativeEffectId === null || !u32(tapCount) - || tapCount === 0 || tapCount > 32) return false; - const tapIds = []; - for (let tapIndex = 0; tapIndex < tapCount; tapIndex += 1) { - const tapId = this.exports.miso_engine_web_v1_observation_tap_id( - this.handle, index, tapIndex, - ); - if (!u32(tapId) || tapId === 0) return false; - tapIds.push(tapId); - } - bindings.push(Object.freeze({ - trackIndex, - rack, - effectIndex, - effectSlotId, - nativeEffectId, - tapIds: Object.freeze(tapIds), - })); - } - this.observations = Object.freeze(bindings); + // Issue #207: source introspection and the observation bindings are control-path state, read + // here once at boot and again by the request handlers (a rebuild can change both, #1562b). The + // construction path and the handlers are the ones that may allocate; `process()` never touches + // them. A headless consumer of the session map cannot otherwise learn which sources a + // compiled session declares, how many channels each carries, or which frames it is waiting + // for; the ABI exposed track discovery and nothing at all about sources. + const sources = this.readSourceRows(); + if (sources === null) return false; + this.sources = sources; + this.sourceCount = sources.length; + const observations = this.readObservationBindings(); + if (observations === null) return false; + this.observations = observations; const observationSelectionPointer = this.exports.miso_engine_web_v1_observation_selection_ptr(); const observationSelectionBytes = this.exports.miso_engine_web_v1_observation_selection_bytes(); const observationSelectionCapacity = this.exports.miso_engine_web_v1_observation_selection_capacity(); @@ -1081,6 +1121,14 @@ class MisoEngineAudioWorkletProcessor extends AudioWorkletProcessor { this.receiveTelemetryLease(message); } else if (message?.tag === "miso.sessionmap.v1" && exactFields(message, ["tag", "requestId"])) { + // The sources are the committed model's (a rebuild can change them), read again here. + const sources = this.readSourceRows(); + if (sources === null) { + this.sticky(RESULT_INTERNAL, message.requestId); + return; + } + this.sources = sources; + this.sourceCount = sources.length; this.port.postMessage({ tag: "miso.sessionmap.v1", requestId: message.requestId, @@ -1094,10 +1142,19 @@ class MisoEngineAudioWorkletProcessor extends AudioWorkletProcessor { }); } else if (message?.tag === "miso.observationmap.v1" && exactFields(message, ["tag", "requestId"])) { + // The bindings are the rendering plan's (a rebuild changes them at its adoption block), and + // the reply names the structure epoch of the control list whose strip order they follow. + const bindings = this.readObservationBindings(); + if (bindings === null) { + this.sticky(RESULT_INTERNAL, message.requestId); + return; + } + this.observations = bindings; this.port.postMessage({ tag: "miso.observationmap.v1", requestId: message.requestId, result: RESULT_OK, + structureEpoch: this.renderingStructureEpoch(), bindings: this.observations, }); } else if (message?.tag === "miso.observation.v1" @@ -1806,6 +1863,9 @@ class MisoEngineAudioWorkletProcessor extends AudioWorkletProcessor { message.validity = Number(metadata & 0xffffffffn); message.lossCount = Number(metadata >> 32n); message.windows = windows; + // Two 32-bit words, so the read allocates nothing. + message.structureEpoch = header.getUint32(METER_HEADER_STRUCTURE_EPOCH, true) + + header.getUint32(METER_HEADER_STRUCTURE_EPOCH + 4, true) * 4294967296; message.peaks.set(set.trackPeaks); message.peaks.set(set.masterPeaks, set.trackCount * 2); message.submixPeaks.set(set.submixPeaks); diff --git a/scripts/check-abi-layout-v1.py b/scripts/check-abi-layout-v1.py index 23e0026c3..7c850c48d 100644 --- a/scripts/check-abi-layout-v1.py +++ b/scripts/check-abi-layout-v1.py @@ -226,7 +226,7 @@ "bootOptions": 64, "status": 80, "resourceReport": 224, - "meterHeader": 80, + "meterHeader": 88, "commandReport": 48, "planWatermark": 96, "observationSelection": 32, diff --git a/scripts/check-session-map-shape.py b/scripts/check-session-map-shape.py index 25bbe4d02..f80050a53 100644 --- a/scripts/check-session-map-shape.py +++ b/scripts/check-session-map-shape.py @@ -13,7 +13,7 @@ * `scripts/check-web-audioworklet.sh` -- the shipped module's frozen export set. An export that exists in the crate and not in that list is an export nothing proves is shipped. * `hosts/host-web/web/miso-engine-v1-audio-worklet.js` -- the worklet, which calls the - exports once at construction and posts the map. + exports in `readSourceRows` (at construction and at each map request) and posts the map. * `hosts/host-web/web/miso-engine-v1-audio-worklet-host.js` -- the main-realm host, whose acknowledgement validator fails the WHOLE host on a field set it does not expect. This is the #151 failure shape exactly: a map that grew a field the validator did not know about would @@ -181,22 +181,22 @@ def check_boot_export_signatures(text: str, shipped: set[str]) -> None: def worklet_called_exports(text: str) -> set[str]: - """Every source query the worklet's construction path calls. + """Every source query the worklet's `readSourceRows` calls. The PCM feed path lives in `receiveSource`/`receiveSeek`, not here, so a `source_` name found in this body is an introspection query by construction. """ - body = block_after(text, "\n bindLiveControls(init) ", "{", "}") + body = block_after(text, "\n readSourceRows() ", "{", "}") called = set( re.findall(r"this\.exports\.(miso_engine_web_v1_source_[a-z_]+)\(", body) ) - require(called, "the worklet's construction path calls no source query at all") + require(called, "the worklet's readSourceRows calls no source query at all") return called def worklet_source_reads(text: str) -> dict[str, str]: """`field name -> export name`, as the worklet actually wires them together.""" - body = block_after(text, "\n bindLiveControls(init) ", "{", "}") + body = block_after(text, "\n readSourceRows() ", "{", "}") reads = { field: export for field, export in re.findall( @@ -219,8 +219,8 @@ def worklet_source_reads(text: str) -> dict[str, str]: def worklet_pushed_fields(text: str) -> list[str]: - body = block_after(text, "\n bindLiveControls(init) ", "{", "}") - match = re.search(r"this\.sources\.push\(\{([^}]*)\}\);", body) + body = block_after(text, "\n readSourceRows() ", "{", "}") + match = re.search(r"rows\.push\(\{([^}]*)\}\);", body) require(match, "the worklet does not push a per-source record with shorthand fields") fields = [part.strip() for part in match.group(1).split(",") if part.strip()] require( @@ -413,8 +413,8 @@ def mutate(source: pathlib.Path, before: str, after: str) -> dict[pathlib.Path, "the worklet records a field it does not post", mutate( WORKLET_JS, - "this.sources.push({ id, channels, frames });", - "this.sources.push({ id, channels });", + "rows.push({ id, channels, frames });", + "rows.push({ id, channels });", ), ), ( diff --git a/scripts/fixtures/abi-layout-v1-self-test.json b/scripts/fixtures/abi-layout-v1-self-test.json index b8736305a..e54b8a08f 100644 --- a/scripts/fixtures/abi-layout-v1-self-test.json +++ b/scripts/fixtures/abi-layout-v1-self-test.json @@ -177,7 +177,7 @@ ] }, "meterHeader": { - "bytes": 80, + "bytes": 88, "fields": [ { "name": "structSize", "offset": 0, "type": "u32" }, { "name": "abiVersion", "offset": 4, "type": "u32" }, @@ -192,7 +192,8 @@ { "name": "submixCount", "offset": 64, "type": "u32" }, { "name": "retired", "offset": 68, "type": "u32" }, { "name": "framePointer", "offset": 72, "type": "u32" }, - { "name": "successorPointer", "offset": 76, "type": "u32" } + { "name": "successorPointer", "offset": 76, "type": "u32" }, + { "name": "structureEpoch", "offset": 80, "type": "u64" } ] }, "commandReport": { diff --git a/scripts/test-web-audioworklet.mjs b/scripts/test-web-audioworklet.mjs index 1f405efd4..2cc73cdba 100644 --- a/scripts/test-web-audioworklet.mjs +++ b/scripts/test-web-audioworklet.mjs @@ -2091,7 +2091,7 @@ async function testMainRealm() { node.port.onmessage({ data: { tag: "miso.meter.v1", sequence: 1, generation: 1n, validity: 0xb, lossCount: 0, - windows: 1, trackCount: 2, + windows: 1, trackCount: 2, structureEpoch: 1, peaks: new Float32Array([0.125, 0.25, 0.375, 0.5, 0.625, 0.75]), trackGrDb: new Float32Array([6.5, 0]), masterGrDb: 6.5, firstSample: 512n, endSample: 768n, @@ -2145,6 +2145,11 @@ async function testMainRealm() { { validity: 0x7, lossCount: 0 }, { validity: 0x3, lossCount: 1 }, { lossCount: 0x1_0000_0000 }, + // Issue #1562b: the structure epoch is a positive safe integer. + { structureEpoch: MISSING }, + { structureEpoch: 0 }, + { structureEpoch: 1.5 }, + { structureEpoch: 1n }, { trackGrDb: new Float32Array(1) }, { trackGrDb: [0, 0] }, { trackGrDb: new Float32Array([-6.5, 0]) }, @@ -2182,7 +2187,7 @@ async function testMainRealm() { const rejected = rejecting.status(); const data = { tag: "miso.meter.v1", sequence: 1, generation: 1n, validity: 0xb, lossCount: 0, - windows: 1, trackCount: 2, + windows: 1, trackCount: 2, structureEpoch: 1, peaks: new Float32Array(6), trackGrDb: new Float32Array(2), masterGrDb: null, firstSample: 512n, endSample: 768n, submixCount: 1, submixPeaks: new Float32Array(2), submixGrDb: new Float32Array(1), ...broken, @@ -2329,7 +2334,7 @@ async function testMainRealm() { node.port.onmessage({ data: { tag: "miso.meter.v1", sequence: 10 + reason, generation: 1n, validity: 0xb, - lossCount: 0, windows: 1, trackCount: 2, + lossCount: 0, windows: 1, trackCount: 2, structureEpoch: 1, peaks: new Float32Array([0.5, 0.5, 0.5, 0.5, 0.5, 0.5]), trackGrDb: new Float32Array([3.25, 0]), masterGrDb: 3.25, firstSample: 1024n, endSample: 1280n, @@ -2387,7 +2392,7 @@ async function testMainRealm() { node.port.onmessage({ data: { tag: "miso.meter.v1", sequence: 2, generation: 1n, validity: 0x3, lossCount: 0, - windows: 1, trackCount: 2, + windows: 1, trackCount: 2, structureEpoch: 1, peaks: new Float32Array(6), trackGrDb: new Float32Array(2), masterGrDb: null, firstSample: 768n, endSample: 1024n, submixCount: 0, submixPeaks: new Float32Array(0), submixGrDb: new Float32Array(0), @@ -2402,7 +2407,7 @@ async function testMainRealm() { node.port.onmessage({ data: { tag: "miso.meter.v1", sequence: 3, generation: 1n, validity: 0x3, lossCount: 0, - windows: 1, trackCount: 2, + windows: 1, trackCount: 2, structureEpoch: 1, peaks: new Float32Array(4), trackGrDb: new Float32Array(2), masterGrDb: null, firstSample: 0n, endSample: 0n, submixCount: 0, submixPeaks: new Float32Array(0), submixGrDb: new Float32Array(0), @@ -2493,7 +2498,7 @@ async function testMainRealm() { FakeNode.latest.port.onmessage({ data: { tag: "miso.meter.v1", sequence: 1, generation: 1n, validity: 0xb, lossCount: 0, - windows: 1, trackCount: 2, + windows: 1, trackCount: 2, structureEpoch: 1, peaks: new Float32Array(6), trackGrDb: new Float32Array([1.5, 2.5]), masterGrDb: 2.5, firstSample: 0n, endSample: 256n, submixCount: 0, submixPeaks: new Float32Array(0), submixGrDb: new Float32Array(0), @@ -2606,21 +2611,23 @@ function createFakeExports(quantum, backend = 1, liveControlsAttached = true, su const submixCount = submixIds.length; const strips = trackIds.length + submixCount; const meterFrameFloats = strips * 3 + 3; - // Issue #1290d1: the header is 80 bytes and names its frame (offset 72); `retired` (68) and - // `successor` (76) are zero until a replacement plan is committed and adopted. The replacement + // Issue #1290d1: the header is 88 bytes (#1562b appended `structure_epoch` at 80) and names its + // frame (offset 72); `retired` (68) and `successor` (76) are zero until a replacement plan is + // committed and adopted. The replacement // plan's own header and frame, sized for one more track, sit beside it: `commitMeterSuccessor` // names it in the active header, as the host does when a command commits a rebuild, and // `adoptOnNextRender` makes the next render call model the swap block (the old header is marked // retired and names the new one, whose stream fields are the old header's). const successorHeaderPointer = 41300; const successorFramePointer = 44000; - const meterHeader = new DataView(memory.buffer, meterHeaderPointer, 80); - const successorHeader = new DataView(memory.buffer, successorHeaderPointer, 80); - for (const [header, tracks, framePointer] of [ - [meterHeader, trackIds.length, meterFramePointer], - [successorHeader, trackIds.length + 1, successorFramePointer], + const meterHeader = new DataView(memory.buffer, meterHeaderPointer, 88); + const successorHeader = new DataView(memory.buffer, successorHeaderPointer, 88); + for (const [header, tracks, framePointer, epoch] of [ + [meterHeader, trackIds.length, meterFramePointer, 1n], + [successorHeader, trackIds.length + 1, successorFramePointer, 2n], ]) { - header.setUint32(0, 80, true); + header.setUint32(0, 88, true); + header.setBigUint64(80, epoch, true); header.setUint32(64, submixCount, true); header.setUint32(4, 0x00010000, true); header.setUint32(8, tracks, true); @@ -3739,7 +3746,7 @@ async function testProcessor() { // A refusal carries its result and diagnostic bytes, and leaves the worklet ready. fake.calls.controlResult = 10; fake.calls.controlResponse = Uint8Array.from( - "web.apply.rebuild_pending\t$\n", (character) => character.charCodeAt(0), + "web.apply.shape_unavailable\t$\n", (character) => character.charCodeAt(0), ); processor.port.onmessage({ data: { tag: "miso.control.v1", requestId: 2, frame: new Uint8Array([7]) }, @@ -3749,7 +3756,7 @@ async function testProcessor() { assert.equal(refused.result, 10); assert.equal( String.fromCharCode(...refused.response), - "web.apply.rebuild_pending\t$\n", + "web.apply.shape_unavailable\t$\n", ); assert.equal(processor.ready, true, "a typed refusal is not sticky"); // A frame the staging buffer refuses is a typed result with no response, not sticky. @@ -4196,6 +4203,7 @@ async function testProcessor() { assert.deepEqual([...frame.submixGrDb], [2.25]); assert.equal(frame.firstSample, 512n); assert.equal(frame.endSample, 768n); + assert.equal(frame.structureEpoch, 1, "the plan's list epoch rides the frame"); // A nonzero poll is still withheld unless both complete peak-validity bits are present. // This exercises the real worklet guard against a malformed/future Rust publication. @@ -4213,6 +4221,11 @@ async function testProcessor() { assert.equal(lossyFrame.generation, 1n); assert.equal(lossyFrame.validity, 0xf); assert.equal(lossyFrame.lossCount, 2); + // The epoch is read as two words: one past 2^32 still arrives whole, with no BigInt. + fake.meterHeader.setBigUint64(80, (3n << 32n) | 7n, true); + fake.calls.meterWindows = 1; + assert.equal(processor.process([], [[left, right]]), true); + assert.equal(processor.port.posts.at(-1).message.structureEpoch, 3 * 4294967296 + 7); // Releasing the lease stops the frames immediately. processor.receive({ tag: "miso.meters.v1", requestId: 2, enabled: false }); @@ -4291,6 +4304,7 @@ async function testProcessor() { assert.deepEqual([...frame.submixGrDb], [2.25]); assert.equal(frame.masterGrDb, 7.5); assert.equal(frame.sequence, 3, "the page's sequence continues across the swap"); + assert.equal(frame.structureEpoch, 2, "the adopted plan names its own list"); // A memory growth re-cuts both sets, so a later swap still reads live memory. fake.growMemory(); @@ -4393,7 +4407,7 @@ async function testProcessor() { } { - // The session map answers from the identities read once at construction. Two submixes + // The session map answers from the engine's rows. Two submixes // (#1210 MINOR-1, #1214 MINOR-2): the SDK indexes bus commands by this list's order, so red // if the worklet enumerates or posts the submix IDs in any other order. const { processor, fake } = makeProcessor(["aa-bus", "zz-bus"]); @@ -4406,12 +4420,48 @@ async function testProcessor() { assert.equal(Object.hasOwn(map, "submixes"), false, "issue #1565: the strip order is the core's list"); assert.equal(Object.hasOwn(map, "routes"), false, "issue #1565: no route list"); assert.equal(Object.hasOwn(map, "vcas"), false, "issue #1565: no VCA list"); - // The identities were read once at construction and the reads are not repeated per request: - // a second map answers from the same numbers, and `process()` never sees any of this. + // A second map answers from the same numbers, and `process()` never sees any of this. processor.receive({ tag: "miso.sessionmap.v1", requestId: 2 }); assert.deepEqual(processor.port.posts.at(-1).message.sources, fake.sourceRows); } + { + // Issue #1562b: a rebuild can change the committed sources and, at its adoption block, the + // observation bindings, so both maps are read from the engine at each request. Red if the + // worklet answers from the boot's rows (a source added by a rebuild would never be listed), + // or if the observation map does not name the structure epoch of the plan that renders. + const { processor, fake } = makeProcessor(); + const left = new Float32Array(64); + const right = new Float32Array(64); + fake.sourceRows.push({ id: "zz-added", channels: 1, frames: 480n }); + processor.receive({ tag: "miso.sessionmap.v1", requestId: 1 }); + assert.deepEqual( + processor.port.posts.at(-1).message.sources.map((source) => source.id), + ["bass", "drums", "zz-added"], + "the session map lists a source a rebuild added", + ); + processor.receive({ tag: "miso.observationmap.v1", requestId: 2 }); + const before = processor.port.posts.at(-1).message; + assert.equal(before.tag, "miso.observationmap.v1"); + assert.equal(before.structureEpoch, 1, "the boot plan's list"); + assert.deepEqual([...before.bindings], []); + + // A replacement plan waits: the rendering plan's epoch is still the boot's. + fake.commitMeterSuccessor(); + processor.port.onmessage({ data: { tag: "miso.service.v1" } }); + processor.receive({ tag: "miso.observationmap.v1", requestId: 3 }); + assert.equal(processor.port.posts.at(-1).message.structureEpoch, 1, "not adopted yet"); + // The adopting block: the successor's header is the rendering plan's now. + fake.calls.adoptOnNextRender = true; + assert.equal(processor.process([], [[left, right]]), true); + processor.receive({ tag: "miso.observationmap.v1", requestId: 4 }); + assert.equal( + processor.port.posts.at(-1).message.structureEpoch, + 2, + "the adopted plan's list", + ); + } + { // Issue #241: every source read is checked against something compilation already guarantees, // so a mis-wired export fails initialization instead of reaching a consumer as a plausible diff --git a/sdk/README.md b/sdk/README.md index eabfb993f..1dbb5ed5e 100644 --- a/sdk/README.md +++ b/sdk/README.md @@ -313,12 +313,17 @@ reads draw their request IDs from the same allocator as value edits. as the typed builders (`faderDb`, `gainDb`, `parameter`, ...) and returns it for `submit()` or a writer. `address` is either a generated `ControlAddress` (owner, section, the row's machine name, lane) or the `address` of a list entry, which names the row by its numeric `control`. A row of an -insert or console slot depends on the effect it runs, so name it by its machine name only for a -session the SDK built (`withSession()`); the list entry's numeric address always works. `set` -checks only that the edit can be encoded. The value's domain, the lane, the owner and whether the +insert or console slot depends on the effect it runs, and the SDK reads that from the control list's +section headers, not from the session it booted: an insert or console slot that is in the list can +be named by its machine name, and a rebuild that makes an insert run another effect is followed. The +effect row is checked again against the list when the transaction is sent (a probe of the structure +epoch, and the list only when it moved): a name follows the header, and a builder typed for an effect +that insert no longer runs (`insert("comp", "miso.compressor")` after `comp` became a soft clip) is +refused with `MisoUsageError` and sends nothing. A section the list does not have cannot be named; +use the list entry's numeric address. `set` checks only that the edit can be encoded. The value's domain, the lane, the owner and whether the row can be set are the core's: an edit of a derived row rejects `control.read_only`, and one of a -rebuild-only row is refused with the core's own code (on the browser host today, -`web.apply.rebuild_pending`). +rebuild-only row is committed on the rebuild path, as on the C ABI: the core prepares a replacement +plan and the engine adopts it at a block boundary. The typed builders are generated from the same control tables (`src/generated/control-builders.ts`), one method per settable row: a strip's own rows and builtins (`faderDb`, `mute`, `trimDb`, `pan`, @@ -354,8 +359,8 @@ instead of growing with stem duration. | Observation | API and meaning of its time | | --- | --- | -| Track/master peaks | Browser `subscribeMeters()`; headless `meters(true)` then `pollMeters()`. Linear peak magnitudes cover `[firstSample, endSample)`. This span timestamps peaks only, not an exact gain-reduction join. | -| Resident effect values | `observationMap()`, `readObservations()`, or `subscribeObservations()`. Select stable track/rack/effect-slot/tap IDs. Ready values retain the effect owner's own window and native unit. | +| Track/master peaks | Browser `subscribeMeters()`; headless `meters(true)` then `pollMeters()`. Linear peak magnitudes cover `[firstSample, endSample)`. This span timestamps peaks only, not an exact gain-reduction join. A headless frame names its own strips (`trackIds`, `submixIds`) by the control list of the `structureEpoch` it carries, so it stays right across a rebuild, including one that keeps the strip count. | +| Resident effect values | `observationMap()`, `readObservations()`, or `subscribeObservations()`. Select stable track/rack/effect-slot/tap IDs. Ready values retain the effect owner's own window and native unit. A managed subscription follows a rebuild (the core keeps the tap armed on the effect that carries on). When a commit removes the effect, the core ends the tap with `OBSERVATION_ENDED`: pass `onEnd` to `subscribeObservations()` to be told which selections ended and which remain, once for each commit; `handle.ended` lists them, and a subscription left with no selection is closed. | | Live EQ/filter response | `queryTrackResponse()` or `subscribeTrackResponse()`. `capturedSample` marks capture of applied target state at a block boundary. The curve is a stationary EQ/input-filter subtotal; fader, pan, routing, and unsupported processors are excluded. | | Spectrum | Browser `querySpectrum()`; headless `armSpectrum()`, explicit renders, then `readSpectrum()`. Each result covers 2,048 captured PCM samples, `[capturedSample, endSample)`, with dBFS bins and a graph-wide source-underrun flag. | | Continuous spectrum | `subscribeSpectrum()` retains one managed stream. Metadata carries the capture span, hop, epochs, drops, and smoothing history. The default 100 ms power smoothing includes earlier windows; set `smoothingMs: 0` for unsmoothed analysis. | diff --git a/sdk/assets/miso-engine-v1-abi-layout.json b/sdk/assets/miso-engine-v1-abi-layout.json index b8736305a..e54b8a08f 100644 --- a/sdk/assets/miso-engine-v1-abi-layout.json +++ b/sdk/assets/miso-engine-v1-abi-layout.json @@ -177,7 +177,7 @@ ] }, "meterHeader": { - "bytes": 80, + "bytes": 88, "fields": [ { "name": "structSize", "offset": 0, "type": "u32" }, { "name": "abiVersion", "offset": 4, "type": "u32" }, @@ -192,7 +192,8 @@ { "name": "submixCount", "offset": 64, "type": "u32" }, { "name": "retired", "offset": 68, "type": "u32" }, { "name": "framePointer", "offset": 72, "type": "u32" }, - { "name": "successorPointer", "offset": 76, "type": "u32" } + { "name": "successorPointer", "offset": 76, "type": "u32" }, + { "name": "structureEpoch", "offset": 80, "type": "u64" } ] }, "commandReport": { diff --git a/sdk/src/browser/engine.ts b/sdk/src/browser/engine.ts index abe8e7f88..d2a1740a8 100644 --- a/sdk/src/browser/engine.ts +++ b/sdk/src/browser/engine.ts @@ -1,3 +1,4 @@ +import { decodeObservationEnded } from "../core/observation-ended.ts"; import { MisoEngineAsset } from "../core/asset.ts"; import { WasmBoundary } from "../core/boundary.ts"; import type { EngineCallResult, SessionShape } from "../core/boundary.ts"; @@ -29,7 +30,7 @@ import type { EngineLiveControls } from "../core/live-controls.ts"; import { scratchBootWithWorker } from "./scratch.ts"; import type { ScratchWorkerFactory } from "./scratch.ts"; import { createDefaultHost, BrowserBootError } from "./default-host.ts"; -import { createBrowserControlListTransport, createBrowserLiveControls, createBrowserStripOrder } from "./live-controls.ts"; +import { createBrowserControlListTransport, createBrowserLiveControls, createBrowserStripOrders } from "./live-controls.ts"; import { readControlList, readControlState } from "../core/control-list.ts"; import type { ControlList, ControlListFilter, ControlState } from "../core/control-list.ts"; import { createTrackResponse } from "./response.ts"; @@ -657,21 +658,24 @@ export async function createEngine(options: CreateEngineOptions): Promise => { - const order = await stripOrder(); - return [...order.tracks, ...order.submixes]; - }; let observationSubscriptions: ObservationSubscriptionOwner | undefined; - const observationMap = async (): Promise => { + // The map and the strip IDs that name its bindings' indices, from one plan: the map follows the + // plan that renders, whose list is the one the reply names (#1562b). + const observationMapWithStrips = async (): Promise<{ + readonly map: ObservationMap; + readonly stripIds: readonly string[]; + }> => { const reply = await host.observationMap(); if (reply.result !== constantValue("resultCodes", "ok")) { throw new MisoEngineError("the browser host refused the observation map", { @@ -680,14 +684,32 @@ export async function createEngine(options: CreateEngineOptions): Promise => (await observationMapWithStrips()).map; const readObservations = async ( selections: readonly ObservationSelection[], ): Promise => { validateObservationSelections(selections); - const map = await observationMap(); - const stripIds = await strips(); + const { map, stripIds } = await observationMapWithStrips(); const addresses = resolveObservationAddressesWithTracks(map, stripIds, selections); const reply = await host.readObservations({ selections: [...addresses] }); if (reply.result !== constantValue("resultCodes", "ok")) { @@ -1081,6 +1103,12 @@ export async function createEngine(options: CreateEngineOptions): Promise host.onControlEvent((event) => { + const ended = event.lane === 0 ? decodeObservationEnded(event.frame) : undefined; + if (ended !== undefined) listener(ended); + }), // The audio thread renders on its own clock, so a subscription refused with backpressure // (an earlier one is not yet drained) may wait for the host's render count to move and be // sent again. The wait is bounded by the engine's request deadline and ends in the same diff --git a/sdk/src/browser/index.ts b/sdk/src/browser/index.ts index ef053d045..f73afaf7c 100644 --- a/sdk/src/browser/index.ts +++ b/sdk/src/browser/index.ts @@ -74,6 +74,7 @@ export type { ObservationSubscriptionBounds, ObservationSubscriptionConfiguration, ObservationSubscriptionLimits, + ObservationSubscriptionEnd, ObservationSubscriptionNotification, ObservationSubscriptionReceipt, ObservationSubscriptionRequest, diff --git a/sdk/src/browser/live-controls.ts b/sdk/src/browser/live-controls.ts index 90f5104a0..5f3d1d27b 100644 --- a/sdk/src/browser/live-controls.ts +++ b/sdk/src/browser/live-controls.ts @@ -5,7 +5,7 @@ import { controlCallFailure, decodeControlLimits, encodeRevisionProbe } from ".. import { ControlRequestIds, ControlTransactionQueue } from "../core/control-queue.ts"; import { readControlStripOrder } from "../core/control-list.ts"; import type { ControlListTransport, ControlStripOrder } from "../core/control-list.ts"; -import { EngineLiveControls } from "../core/live-controls.ts"; +import { EngineLiveControls, effectRowResolver } from "../core/live-controls.ts"; import type { LiveControlBeforeSubmit } from "../core/live-controls.ts"; import { resultName } from "../core/errors.ts"; import type { SessionLike } from "../core/session.ts"; @@ -62,11 +62,72 @@ export async function createBrowserControlListTransport( * epoch of the order returned last, so a structure change is followed and nothing is read twice. */ export function createBrowserStripOrder(host: MisoAudioWorkletHost): () => Promise { - let known: ControlStripOrder | undefined; - return async () => { - known = await readControlStripOrder(await createBrowserControlListTransport(host), known); - return known; + return createBrowserStripOrders(host).current; +} + +/** The strip orders the page holds for the host's running session (#1562b). */ +export interface BrowserStripOrders { + /** The order of the newest committed list, read again when its structure epoch moved. */ + readonly current: () => Promise; + /** The order of the list of `epoch` if the page holds it now. */ + readonly held: (epoch: number | bigint) => ControlStripOrder | undefined; + /** + * The order of the list whose structure epoch is `epoch`: a meter frame or an observation map + * names its strips by the list of the plan that produced it, which is not the newest list while + * a replacement plan waits for its block. It is a list this page read, or the newest list when + * that has this epoch. `undefined` when the list of that epoch is gone: the newest list is + * another one and this page never held it. The caller drops what it cannot name. + */ + readonly of: (epoch: number | bigint) => Promise; +} + +/** The lists the page remembers: a structural commit and its adoption are the two in flight. */ +const HELD_STRIP_ORDERS = 8; + +/** + * The strip orders of one host (#1562b). A rebuild that adds, removes or reorders a strip moves the + * list's structure epoch at its commit, and the plan that renders keeps naming its strips by the + * previous list until the block that adopts the replacement. So each order is kept by its epoch and + * a frame is named by the order of the epoch it carries. Reads are the ones {@link + * createBrowserStripOrder} makes: a probe, and the whole list only when the epoch moved. + */ +export function createBrowserStripOrders(host: MisoAudioWorkletHost): BrowserStripOrders { + const known = hostStripOrders.get(host); + if (known !== undefined) return known; + const created = newBrowserStripOrders(host); + hostStripOrders.set(host, created); + return created; +} + +/** One reader per host: the queue's effect-row check, the live controls and the feeds share its cache. */ +const hostStripOrders = new WeakMap(); + +function newBrowserStripOrders(host: MisoAudioWorkletHost): BrowserStripOrders { + const orders = new Map(); + let newest: ControlStripOrder | undefined; + const current = async (): Promise => { + const read = await readControlStripOrder(await createBrowserControlListTransport(host), newest); + newest = read; + orders.delete(read.structureEpoch); + orders.set(read.structureEpoch, read); + while (orders.size > HELD_STRIP_ORDERS) { + const oldest = orders.keys().next(); + if (oldest.done) break; + orders.delete(oldest.value); + } + return read; }; + const held = (epoch: number | bigint): ControlStripOrder | undefined => orders.get(BigInt(epoch)); + return Object.freeze({ + current, + held, + of: async (epoch: number | bigint) => { + const known = held(epoch); + if (known !== undefined) return known; + const read = await current(); + return read.structureEpoch === BigInt(epoch) ? read : undefined; + }, + }); } const hostQueues = new WeakMap>(); @@ -110,6 +171,9 @@ async function openBrowserControlQueue(host: MisoAudioWorkletHost): Promise (await probe()).revision, + // An edit that names an effect's row is checked against the core's list before it is sent + // (#1562b-K); the reader is the page's one, so a probe of the structure epoch is all it costs. + prepare: effectRowResolver(() => createBrowserStripOrders(host).current()), }); } @@ -131,7 +195,8 @@ export async function createBrowserLiveControls( // The strip order is the core's control list (#1564 D5), not the worklet's index exports: the // map keeps the worklet's sources and meter lease, and takes the tracks and submixes from the // owner headers. - const order = await readControlStripOrder({ send: hostControlSend(host), requestIds: queue.requestIds }); + const orders = createBrowserStripOrders(host); + const order = await orders.current(); const map: SessionMap = Object.freeze({ tracks: order.tracks, sources: Object.freeze(remoteMap.sources.map((source) => Object.freeze({ ...source }))), @@ -140,6 +205,8 @@ export async function createBrowserLiveControls( }); return new EngineLiveControls(map, { queue, + stripOrder: orders.current, + order, observe: async (edits): Promise => { const ack = await host.command({ commands: edits.map(browserCommand) }); return Object.freeze({ diff --git a/sdk/src/browser/measurement.ts b/sdk/src/browser/measurement.ts index e80926559..271727e07 100644 --- a/sdk/src/browser/measurement.ts +++ b/sdk/src/browser/measurement.ts @@ -26,6 +26,12 @@ export interface MeterUpdate { readonly sequence: bigint; readonly generation: bigint; readonly validity: number; + /** + * Meter windows dropped before this one, saturating: the host's own count plus the windows the + * SDK could not name and dropped (the frame carried a structure epoch whose strip list the page no + * longer held, #1562b). The first update after such a drop also sets the loss bit (`4`) in + * `validity`. + */ readonly lossCount: number; readonly windows: number; /** This half-open span timestamps the peak window only. */ @@ -58,6 +64,23 @@ export type TelemetryListener = (update: TelemetryUpdate) => void; type LeaseResult = { readonly result: number }; +/** The strip order a meter frame's strips are named by (#1562b). */ +export interface MeterStripOrders { + /** The order of the list of `epoch` if the page holds it now, else `undefined`. */ + readonly held: (epoch: number) => StripNames | undefined; + /** + * The order of the list of `epoch`, read if it is not held, or `undefined` when the page cannot + * name that list's strips any more (a frame of a plan the page never saw the list of). + */ + readonly of: (epoch: number) => Promise; +} + +/** The strips of one list, tracks then submixes, by stable ID. */ +export interface StripNames { + readonly tracks: readonly string[]; + readonly submixes: readonly string[]; +} + type HostFeed = { readonly subscribe: (listener: (update: Update) => void) => Promise<() => void>; readonly close: () => void; @@ -73,7 +96,11 @@ function createHostFeed(options: { readonly name: "meters" | "telemetry"; readonly available: boolean; readonly lease: (onFrame: ((frame: Frame) => void) | null) => Promise; - readonly project: (frame: Frame) => Update; + /** + * The frame as an update. A projection that needs a read (a meter frame of a structure the page + * holds no strip order for) answers later; `undefined` drops the frame. + */ + readonly project: (frame: Frame) => Update | undefined | Promise; }): HostFeed { const listeners = new Set<{ readonly listener: (update: Update) => void }>(); const pendingAdmissions = new Map< @@ -116,15 +143,8 @@ function createHostFeed(options: { const wanted = !closed && listeners.size > 0; if (wanted === armed) return; if (wanted) { - const ack = await lease((frame) => { - let update: Update; - try { - update = options.project(frame); - } catch { - // Host validation is authoritative. A malformed custom host frame must not make - // one callback prevent ownership reconciliation or another callback's delivery. - return; - } + const deliver = (update: Update | undefined): void => { + if (update === undefined) return; for (const subscription of [...listeners]) { if (closed || !listeners.has(subscription)) continue; try { @@ -133,6 +153,40 @@ function createHostFeed(options: { // Consumer callback failures are isolated from sibling listeners and the lease. } } + }; + // Frames are delivered in the order they arrived. A frame whose projection must read + // (the first frame of a new structure) holds the frames behind it until it settles. + let tail: Promise | undefined; + const ack = await lease((frame) => { + const project = (): Promise | undefined => { + let projected: Update | undefined | Promise; + try { + projected = options.project(frame); + } catch { + // Host validation is authoritative. A malformed custom host frame must not make + // one callback prevent ownership reconciliation or another callback's delivery. + return undefined; + } + if (projected instanceof Promise) { + return projected.then(deliver, () => undefined); + } + deliver(projected); + return undefined; + }; + if (tail !== undefined) { + const chained: Promise = tail.then(project).then(() => { + if (tail === chained) tail = undefined; + }); + tail = chained; + return; + } + const waiting = project(); + if (waiting !== undefined) { + const chained: Promise = waiting.then(() => { + if (tail === chained) tail = undefined; + }); + tail = chained; + } }); if (ack.result !== constantValue("resultCodes", "ok")) { throw refusal(ack.result); @@ -289,17 +343,53 @@ export interface BrowserMeasurementFeeds { readonly close: () => void; } +/** The shipped header's validity bit for "one or more windows were dropped before this one". */ +const METER_VALIDITY_LOSS = 4; +const MAXIMUM_LOSS_COUNT = 0xffff_ffff; + export function createMeasurementFeeds( host: Pick, - trackIds: readonly string[], - submixIds: readonly string[], + orders: MeterStripOrders, available: boolean, ): BrowserMeasurementFeeds { + // The windows the page could not name (no list of their epoch is held), counted so the loss is + // reported and never silent: every update after a drop carries the total in `lossCount` (it + // saturates like the host's own count), and the first one sets the loss bit in `validity`. + let unnamedWindows = 0; + let unnamedReported = true; + const reportingUnnamed = (update: MeterUpdate): MeterUpdate => { + if (unnamedWindows === 0) return update; + const flagged = !unnamedReported; + unnamedReported = true; + return Object.freeze({ + ...update, + lossCount: Math.min(update.lossCount + unnamedWindows, MAXIMUM_LOSS_COUNT), + validity: flagged ? update.validity | METER_VALIDITY_LOSS : update.validity, + }); + }; const meters = createHostFeed({ name: "meters", available, lease: (onFrame) => host.meters({ enabled: onFrame !== null, onFrame }), - project: (frame) => meterProjection(frame, trackIds, submixIds), + // A frame names its strips by the list of the structure epoch it carries (#1562b): a rebuild + // that adds, removes or reorders a strip changes the epoch in the frame of the block that + // adopted the plan, and the order is read once for the new epoch. A frame of an epoch whose + // list the page cannot name is dropped and counted (a frame named by another list would key its + // readings by the wrong strips); the count reaches the listener with the next update. + project: (frame) => { + const held = orders.held(frame.structureEpoch); + if (held !== undefined) { + return reportingUnnamed(meterProjection(frame, held.tracks, held.submixes)); + } + return orders.of(frame.structureEpoch).then((named) => { + if (named === undefined) { + unnamedWindows = Math.min(unnamedWindows + Math.max(1, frame.windows), MAXIMUM_LOSS_COUNT); + unnamedReported = false; + return undefined; + } + return reportingUnnamed(meterProjection(frame, named.tracks, named.submixes)); + }); + }, }); const telemetry = createHostFeed({ name: "telemetry", diff --git a/sdk/src/browser/shipped-host.d.ts b/sdk/src/browser/shipped-host.d.ts index 7536e5077..9002425ec 100644 --- a/sdk/src/browser/shipped-host.d.ts +++ b/sdk/src/browser/shipped-host.d.ts @@ -358,6 +358,14 @@ export interface MisoObservationMap { readonly tag: "miso.observationmap.v1"; readonly requestId: number; readonly result: number; + /// The structure epoch of the control list whose strip order `bindings` follow (issue 1562b): + /// the `structureEpoch` of `CONTROL_LIST_GET` pages, as the plan that renders carries it. A + /// binding's `trackIndex` indexes the tracks, then the submixes, of the list of this epoch. A + /// rebuild that adds, removes or reorders a strip changes it at the block that adopts the + /// replacement plan; between that commit and the adoption the control list is already the + /// replacement's, so a client names the strips of this map by the list of this epoch and never + /// by the newest list. A positive safe integer. + readonly structureEpoch: number; readonly bindings: MisoObservationMapBinding[]; } @@ -513,6 +521,14 @@ export interface MisoMeterFrame { /// The tracks of the plan that rendered the window. A structural apply that adds or removes a /// track changes it between two frames of one stream; every array below is sized for it. readonly trackCount: number; + /// The structure epoch of the control list that the plan's strip order belongs to (issue + /// 1562b): the `structureEpoch` that `CONTROL_LIST_GET` pages carry, as the commit that prepared + /// the plan left it. The frame's strips (the tracks, then the submixes) are named by the list + /// of this epoch, so a page that holds a list of another epoch reads the list again before it + /// names a frame's strips. A structural apply that adds, removes or reorders a strip changes it + /// between two frames of one stream, in the frame of the block that adopted the plan, and a + /// replacement that keeps the structure keeps it. A positive safe integer. + readonly structureEpoch: number; /// `[track0 L, track0 R, .., trackN L, trackN R, master L, master R]` peak magnitudes. readonly peaks: Float32Array; /// Gain reduction per track, in **non-negative decibels**, `trackCount` long (issue 143). @@ -778,9 +794,9 @@ export interface MisoRenderAllocations { /// The reply to one control frame (issue 1562a D1). `response` is the response frame the engine's /// control plane built, unchanged, when `result` is `0`; for another result it is the refusal's -/// diagnostic text (`code\tpath\n` lines), for example `web.apply.rebuild_pending` with `result: 10` -/// for a transaction that needs a plan rebuild, which this module cannot apply in place yet. A -/// refused transaction is still `result: 0` with a non-ok response frame, as on the C ABI. +/// diagnostic text (`code\tpath\n` lines), for example `web.apply.shape_unavailable` with `result: 10` +/// for a preparation that could not hand its plan to the host. A transaction that needs a plan +/// rebuild is prepared and published like any other (issue 1562b). A refused transaction is still `result: 0` with a non-ok response frame, as on the C ABI. /// `revision` is the committed session revision after the call. export interface MisoControlReply { readonly tag: "miso.control.v1"; @@ -912,9 +928,11 @@ export interface MisoAudioWorkletHost { request: { enabled: boolean; onFrame: ((frame: MisoTelemetryFrame) => void) | null }, ): Promise; /// @internal Subscribe to the engine's protocol events (issue 1570b D5): every command's - /// `SESSION_COMMITTED`, `TRANSPORT_STATE`, `METER_BATCH`, `COUNTER_SNAPSHOT`, `DIAGNOSTIC` and the - /// other events of the control protocol, as the engine encoded them. The listener gets each - /// event with its lane (`0` reliable, `1` lossy) in arrival order. Returns the unsubscribe + /// `SESSION_COMMITTED`, `OBSERVATION_ENDED` (reliable: a commit removed the effect of a + /// subscribed observation tap, and the event names every ended tap key, #1584), `TRANSPORT_STATE`, + /// `METER_BATCH`, `COUNTER_SNAPSHOT`, `DIAGNOSTIC` and the other events of the control protocol, + /// as the engine encoded them. The listener gets each event with its lane (`0` reliable, `1` + /// lossy) in arrival order. Returns the unsubscribe /// function. An event is not the ack of a command: the command's reply is. onControlEvent(listener: (event: MisoControlEvent) => void): () => void; /// @internal The count of engine events that arrived while no listener was subscribed (issue diff --git a/sdk/src/core/boundary.ts b/sdk/src/core/boundary.ts index 06fd6d88e..b3b5fe18e 100644 --- a/sdk/src/core/boundary.ts +++ b/sdk/src/core/boundary.ts @@ -298,6 +298,23 @@ export interface MeterFrame { readonly sequence: bigint; readonly windows: number; readonly trackCount: number; + /** + * The structure epoch of the control list of the plan that rendered this frame (#1562b). The + * frame names its own strips (`trackIds`, `submixIds`) by the order of that epoch: the newest + * list's, except between a rebuild's commit and the block that adopts its replacement, when the + * plan that still renders keeps the previous list. + */ + readonly structureEpoch: bigint; + /** + * Meter windows this instance dropped before this frame because no list of their structure epoch + * was held or could be read (the frame could not be named), counted over the instance's life. It + * only grows; a gap in `sequence` that this count explains is not a host loss. + */ + readonly unnamedWindows: bigint; + /** The tracks of the frame, in the order of its arrays: the strip order of `structureEpoch`. */ + readonly trackIds: readonly string[]; + /** The submix strips of the frame, in the order of its submix arrays. */ + readonly submixIds: readonly string[]; /** `[track0 L, track0 R, .., master L, master R]`. */ readonly peaks: Float32Array; readonly trackGrDb: Float32Array; @@ -325,6 +342,14 @@ export class WasmBoundary { readonly #controlRequestIds = new ControlRequestIds(); /** The strip order read last, with the structure epoch it belongs to. */ #stripOrder: ControlStripOrder | undefined; + /** + * The orders read, by structure epoch, newest last. The plan that renders keeps naming its strips + * by the list it was prepared from until the block that adopts a rebuild's replacement, so a meter + * frame is named by the order of the epoch it carries (#1562b). + */ + readonly #stripOrders = new Map(); + /** Meter windows dropped because no list named their structure epoch. */ + #unnamedMeterWindows = 0n; #trackResponse: TrackResponseModule | undefined; #spectrum: SpectrumModule | undefined; #spectrumQuery: SpectrumQuery | undefined; @@ -392,6 +417,7 @@ export class WasmBoundary { const spectrumHopBoot = requireSpectrumHopCapability(this.#exports, snapshot.spectrumHopFrames); this.dispose(); this.#stripOrder = undefined; + this.#stripOrders.clear(); const staged = stage( this.#exports, document, @@ -512,9 +538,25 @@ export class WasmBoundary { { send: (frame) => this.controlFrame(frame), requestIds: this.#controlRequestIds }, this.#stripOrder, ); + this.#holdStripOrder(this.#stripOrder); return this.#stripOrder; } + /** The order of the control list whose structure epoch is `epoch`, if this boundary read it. */ + stripOrderOf(epoch: bigint): ControlStripOrder | undefined { + return this.#stripOrders.get(epoch); + } + + #holdStripOrder(order: ControlStripOrder): void { + this.#stripOrders.delete(order.structureEpoch); + this.#stripOrders.set(order.structureEpoch, order); + while (this.#stripOrders.size > HELD_STRIP_ORDERS) { + const oldest = this.#stripOrders.keys().next(); + if (oldest.done) break; + this.#stripOrders.delete(oldest.value); + } + } + /** The engine's state word, named through the generated vocabulary. */ state(): "ready" | "failed" | "disposed" { // Disposal clears the handle, so there is deliberately no engine status pointer left to ask. @@ -586,10 +628,42 @@ export class WasmBoundary { } /** - * Read the current prepared owner's stable resident-observation bindings. A caller that already - * holds the strip order of this call passes it in, so the structure epoch is probed once. + * The strip order of the plan that renders (#1562b): the list of the structure epoch in that + * plan's meter header, which is the newest list's except between a rebuild's commit and the block + * that adopts its replacement. The observation bindings and reads address the rendering plan, so + * they name their strips by this order and never by the newest list. */ - observationMap(knownOrder?: ControlStripOrder): ObservationMap { + #renderingStripOrder(): ControlStripOrder { + const newest = this.stripOrder(); + const header = new StructView( + this.#exports.memory, + "meterHeader", + Number(this.#exports.miso_engine_web_v1_meter_header_ptr(this.#live())), + ); + const epoch = header.u64("structureEpoch"); + if (epoch === 0n || epoch === newest.structureEpoch) return newest; + const held = this.#stripOrders.get(epoch); + if (held === undefined) { + // The plan that renders was prepared from a list this instance never read, and a newer one + // replaced it: the strips of its bindings cannot be named. A render adopts the newer plan. + throw new MisoEngineError("the rendering plan belongs to a control list that is gone", { + phase: "lifecycle", + code: "control.list.stale", + diagnostics: [{ code: "control.list.stale", path: "structureEpoch" }], + }); + } + return held; + } + + /** + * Read the current prepared owner's stable resident-observation bindings: the plan that renders, + * named by the strip order of its own structure epoch. + */ + observationMap(): ObservationMap { + return this.#observationMapOf(this.#renderingStripOrder()); + } + + #observationMapOf(order: ControlStripOrder): ObservationMap { const handle = this.#live(); const count = Number(this.#exports.miso_engine_web_v1_observation_count(handle)); if (!Number.isSafeInteger(count) || count < 0 || count > 0xffff_ffff) { @@ -611,7 +685,6 @@ export class WasmBoundary { }); } // Issue #1210 D5: a binding's index is a strip index, the tracks then the submixes. - const order = knownOrder ?? this.stripOrder(); const strips = [...order.tracks, ...order.submixes]; const raw: RawObservationBinding[] = []; const readId = (length: number, path: string): string => { @@ -669,7 +742,7 @@ export class WasmBoundary { tapIds, }); } - return enrichObservationMap(strips, raw); + return enrichObservationMap(strips, raw, order.structureEpoch); } /** Internal managed-subscription view of the optional prepared spectrum query. */ @@ -688,8 +761,8 @@ export class WasmBoundary { const handle = this.#live(); // Issue #1210 D5: selections resolve against the strips, the tracks then the submixes. The // order is read once and shared with the map: each read probes the structure epoch. - const order = this.stripOrder(); - const map = this.observationMap(order); + const order = this.#renderingStripOrder(); + const map = this.#observationMapOf(order); const strips = [...order.tracks, ...order.submixes]; const addresses = resolveObservationAddressesWithTracks(map, strips, selections); const selectionPointer = Number(this.#exports.miso_engine_web_v1_observation_selection_ptr()); @@ -1114,7 +1187,8 @@ export class WasmBoundary { pollMeters(): MeterFrame | undefined { const handle = this.#live(); // The strip order the last read of the core's list gave; read once, then only when a frame's - // header disagrees with it (a poll is a per-block call and must not probe the core each time). + // header names another structure epoch (a poll is a per-block call and must not probe the core + // each time). const known = this.#stripOrder ?? this.stripOrder(); const windows = Number(this.#exports.miso_engine_web_v1_meter_poll(handle)); if (windows === 0) return undefined; @@ -1125,19 +1199,33 @@ export class WasmBoundary { Number(this.#exports.miso_engine_web_v1_meter_header_ptr(handle)), ); let header = readHeader(); - let trackCount = known.tracks.length; - if (header.u32("trackCount") !== trackCount) { + // The frame names its strips by the list of the plan that rendered it (#1562b): the epoch in its + // header, which is the newest list's except while a rebuild's replacement waits for its block. + const structureEpoch = header.u64("structureEpoch"); + let order = known.structureEpoch === structureEpoch ? known : this.#stripOrders.get(structureEpoch); + if (order === undefined) { // The structure may have changed since the order was read. A control call can grow the // module's memory, so the header view is taken again after it. - trackCount = this.stripOrder().tracks.length; + const fresh = this.stripOrder(); header = readHeader(); + order = fresh.structureEpoch === structureEpoch ? fresh : this.#stripOrders.get(structureEpoch); + } + // The list of that epoch was never read and is gone: the frame cannot be named, and a frame + // named by another list would key readings by the wrong strips. The browser drops it the same + // way (`createMeasurementFeeds`). The windows are counted, never lost without a trace: the next + // frame reports the count. + if (order === undefined) { + this.#unnamedMeterWindows += BigInt(windows); + return undefined; } + const trackCount = order.tracks.length; const reportedWindows = header.u32("windows"); const masterGrPresent = header.u32("masterGrPresent"); const submixCount = header.u32("submixCount"); if (header.u32("structSize") !== structBytes("meterHeader") || header.u32("abiVersion") !== ABI_LAYOUT.abiVersion || header.u32("trackCount") !== trackCount + || header.u32("submixCount") !== order.submixes.length || reportedWindows !== windows || masterGrPresent > 1) { throw new MisoEngineError("the engine returned a malformed meter frame header", { @@ -1175,6 +1263,10 @@ export class WasmBoundary { sequence: header.u64("sequence"), windows, trackCount, + structureEpoch, + unnamedWindows: this.#unnamedMeterWindows, + trackIds: order.tracks, + submixIds: order.submixes, peaks, trackGrDb: frame.slice(gainBase, gainBase + trackCount), masterGrDb: masterGrPresent === 1 ? frame[words - 1]! : null, @@ -1432,6 +1524,7 @@ export class WasmBoundary { const result = Number(this.#exports.miso_engine_web_v1_dispose(this.#handle)); this.#handle = 0; this.#stripOrder = undefined; + this.#stripOrders.clear(); if (result !== constantValue("resultCodes", "ok")) { throw new MisoEngineError("the engine refused to dispose this handle", { phase: "lifecycle", @@ -1449,6 +1542,9 @@ export interface ControlEvent { readonly frame: Uint8Array; } +/** The strip orders a boundary remembers: a structural commit and its adoption are the two in flight. */ +const HELD_STRIP_ORDERS = 8; + /** The most `control_event` calls one drain makes on a lane (#1570b D4). */ const MAXIMUM_CONTROL_EVENTS_PER_LANE = 16; diff --git a/sdk/src/core/control-list.ts b/sdk/src/core/control-list.ts index bf8bb7239..872102466 100644 --- a/sdk/src/core/control-list.ts +++ b/sdk/src/core/control-list.ts @@ -825,14 +825,49 @@ export interface ControlStripOrder { readonly structureEpoch: bigint; readonly tracks: readonly string[]; readonly submixes: readonly string[]; + /** + * The effect each console slot and insert of each strip runs, from the list's section headers + * (#1562b-K). Read it with {@link stripEffectOf}; a rebuild can change an insert's effect, so the + * map belongs to this order's `structureEpoch`. + */ + readonly effects: ReadonlyMap; +} + +function effectKey(stripId: string, kind: "console" | "inserts", id: string): string { + return `${stripId}\u0000${kind}\u0000${id}`; +} + +/** + * The effect a strip's console slot or insert runs, as the core's section header names it, or + * `undefined` when `order` has no such section. + */ +export function stripEffectOf( + order: ControlStripOrder, + stripId: string, + section: { readonly kind: "console"; readonly slot: string } | { readonly kind: "inserts"; readonly effect: string }, +): string | undefined { + return order.effects?.get( + section.kind === "console" + ? effectKey(stripId, "console", section.slot) + : effectKey(stripId, "inserts", section.effect), + ); } /** The strip order of an already read list: its track and submix owner headers, in list order. */ export function stripOrderOf(list: ControlList): ControlStripOrder { const tracks: string[] = []; const submixes: string[] = []; + const effects = new Map(); for (const header of list.headers) { - if (header.row !== "owner" || (header.kind !== "track" && header.kind !== "submix")) continue; + if (header.row === "section") { + const { owner, section } = header.address; + if (owner.kind === "strip" && header.effect !== undefined) { + if (section.kind === "console") effects.set(effectKey(owner.id, "console", section.slot), header.effect); + else if (section.kind === "inserts") effects.set(effectKey(owner.id, "inserts", section.effect), header.effect); + } + continue; + } + if (header.kind !== "track" && header.kind !== "submix") continue; const owner = header.address.owner; if (owner.kind !== "strip") throw malformed("a strip header does not name a strip"); (header.kind === "track" ? tracks : submixes).push(owner.id); @@ -841,6 +876,7 @@ export function stripOrderOf(list: ControlList): ControlStripOrder { structureEpoch: list.structureEpoch, tracks: Object.freeze(tracks), submixes: Object.freeze(submixes), + effects, }); } diff --git a/sdk/src/core/control-queue.ts b/sdk/src/core/control-queue.ts index ba488aef2..188186cf6 100644 --- a/sdk/src/core/control-queue.ts +++ b/sdk/src/core/control-queue.ts @@ -63,6 +63,16 @@ export interface ControlQueueOptions { * reply, because the core may have committed the transaction whose reply was lost. */ readonly readRevision?: (() => bigint | PromiseLike) | undefined; + /** + * Called with the edits of each transaction, coalesced, just before the frame is encoded. It + * returns the edits to send: the same count, in the same order, each standing for the edit at its + * position (an edit that names an effect's row by name is resolved against the core's current + * control list here, #1562b-K). A rejection fails the submits of that transaction as an encoding + * fault does: when it carried several, each is sent alone and only the one at fault is rejected. + */ + readonly prepare?: (( + edits: readonly ControlFrameEdit[], + ) => readonly ControlFrameEdit[] | PromiseLike) | undefined; } /** What the queue counted since it was made. */ @@ -226,6 +236,7 @@ export class ControlTransactionQueue { readonly #ids: ControlRequestIds; readonly #limits: ControlLimits | undefined; readonly #readRevision: (() => bigint | PromiseLike) | undefined; + readonly #prepare: ControlQueueOptions["prepare"]; #revision: bigint; /** The batch being staged, which the next transaction sends. */ #next: Batch | undefined; @@ -251,6 +262,7 @@ export class ControlTransactionQueue { this.#revision = revision; this.#limits = options.limits; this.#readRevision = options.readRevision; + this.#prepare = options.prepare; } /** @@ -266,6 +278,16 @@ export class ControlTransactionQueue { return this.#revision; } + /** + * The revision the core has committed now, read from the core, or `undefined` when the queue was + * given no way to read it. Unlike {@link revision} it sees commits made by other senders (a + * structural edit). It does not change the queue's own revision. + */ + async readCommittedRevision(): Promise { + const revision = await this.#readRevision?.(); + return typeof revision === "bigint" ? revision : undefined; + } + /** The host's frame ceiling in bytes, or `undefined` when the queue was given none. */ get maximumFrameBytes(): number | undefined { return this.#limits?.maximumFrameBytes; @@ -473,13 +495,27 @@ export class ControlTransactionQueue { const group = this.#takeGroup(remaining, alone > 0 ? 1 : Number.POSITIVE_INFINITY); if (alone > 0) alone -= 1; const { edits, owners } = coalesce(group); - const requestId = this.#ids.next(); + let requestId = 0n; let reply: ControlReply; + // The edits as sent: `edits` unless `prepare` stood other edits in for some of them. + let sent: readonly ControlFrameEdit[] = edits; try { + if (this.#prepare !== undefined) { + sent = await this.#prepare(edits); + if (sent.length !== edits.length) { + throw new MisoEngineError("the queue's prepare hook changed the number of edits", { + phase: "lifecycle", + code: "internal", + }); + } + } + // The ID is drawn after `prepare`, which reads the core's list with IDs of its own: the + // core takes IDs in ascending order. + requestId = this.#ids.next(); const frame = encodeControlTransaction({ requestId, expectedRevision: this.#revision, - edits, + edits: sent, }); this.#transactions += 1; reply = decodeControlReply(await this.#send(frame)); @@ -524,9 +560,11 @@ export class ControlTransactionQueue { // The named edit, in its own submit's coordinates; `undefined` when it names none of them. let own: number | undefined; if (culprit !== undefined) { - const key = controlEditKey(edits[index!]!); + // The core names the edit as it was sent; the submit holds the edit as it was staged. + const key = controlEditKey(sent[index!]!); culprit.edits.forEach((edit, at) => { - if (controlEditKey(edit) === key) own = at; + const stood = edits.indexOf(edit); + if (controlEditKey(stood < 0 ? edit : sent[stood]!) === key) own = at; }); } if (culprit === undefined || !sameSession) { diff --git a/sdk/src/core/live-controls.ts b/sdk/src/core/live-controls.ts index 72a17f011..57ff450e6 100644 --- a/sdk/src/core/live-controls.ts +++ b/sdk/src/core/live-controls.ts @@ -17,10 +17,13 @@ import type { TapName, } from "../generated/catalog.ts"; import type { CommandReport, SessionMap } from "./boundary.ts"; +import { stripEffectOf } from "./control-list.ts"; +import type { ControlStripOrder } from "./control-list.ts"; import { validateControlEdit } from "./control-frame.ts"; import type { ControlFrameEdit, ControlWireAddress } from "./control-frame.ts"; import type { ControlAppliedReport, ControlTransactionQueue } from "./control-queue.ts"; -import { MisoUsageError } from "./errors.ts"; +import { MisoEngineError, MisoUsageError } from "./errors.ts"; +import { constantValue } from "./abi.ts"; import { writeCanonicalSessionDocument } from "../internal/session-json.ts"; import type { SessionLike, SessionModel } from "./session.ts"; import { requireU32 } from "../internal/ramp.ts"; @@ -61,12 +64,122 @@ export { EffectControlBuilders, RouteEdits, SessionEdits, StripEdits, SubmixEdit from "../generated/control-builders.ts"; export type { InputFilterValues, MatrixValues } from "../generated/control-builders.ts"; +/** + * A managed observation record cannot be addressed yet: the plan that renders is older than the + * newest committed one, so the effect indices it reports belong to another plan than the one the + * core admits the record against. It is `backpressure`, the typed retryable refusal: a render adopts + * the newer plan, and the caller sends the request again (a browser engine waits for that render + * itself, within its request deadline). + */ +export function structureBehindError(detail: string): MisoEngineError { + return new MisoEngineError( + `the plan that renders is older than the newest committed plan; render and try again: ${detail}`, + { + phase: "lifecycle", + code: "backpressure", + result: constantValue("resultCodes", "backpressure"), + diagnostics: [{ code: "sdk.observation.structure_behind", path: "" }], + }, + ); +} + /** What live controls send through: the engine's one transaction queue, and the observation records. */ export interface LiveControlTransport { /** Every value edit goes here, as a `SET_CONTROL` transaction. */ readonly queue: ControlTransactionQueue; /** Observation records (kinds 7 and 8) stay 48-byte commands until the browser-feature phase. */ readonly observe: (edits: readonly LaneEdit[]) => CommandReport | Promise; + /** + * The strips and effects of the core's control list as they are now (#1562b-K): a probe of the + * structure epoch, and a read of the list only when it moved. Absent for a scripted transport + * that has no list; live controls then keep the order they were made with. + */ + readonly stripOrder?: StripOrderSource; + /** The order the transport read when it made these live controls. */ + readonly order?: ControlStripOrder; +} + +/** Reads the core's strip order and effects; the caller caches by structure epoch. */ +export type StripOrderSource = () => ControlStripOrder | Promise; + +/** What a built edit remembers of the effect row it names, to name it again after a rebuild. */ +interface RowProvenance { + readonly stripId: string; + readonly section: + | { readonly kind: "console"; readonly slot: string } + | { readonly kind: "inserts"; readonly effect: string }; + /** The row's machine name in its effect's control table. */ + readonly name: string; + /** The effect the numeric `control` was resolved for. */ + readonly effectId: string; + /** + * `true` when the caller typed the effect (`insert("c", "miso.compressor")`): the edit is refused + * if the core's section header now names another. `false` for an address named by the section + * alone, which follows the header. + */ + readonly strict: boolean; +} + +const PROVENANCE = new WeakMap(); +/** The strip an observation edit names by index, to name it again after a rebuild. */ +const OBSERVED_STRIP = new WeakMap(); + +function noteRow(set: ControlEditSet, row: RowProvenance): ControlEditSet { + for (const edit of set.edits) PROVENANCE.set(edit, row); + return set; +} + +/** + * The hook that resolves the effect rows of a transaction against the core's control list + * (#1562b-K). A rebuild can change the effect an insert or console slot runs, and the numeric row + * of an effect's control depends on the effect, so an edit built before the rebuild could land on + * another effect's row with the same number. Before each transaction is encoded, an edit that names + * an effect row is checked against the section header the core lists now (a probe of the structure + * epoch, and the list again only when it moved): an address named by its section follows the header, + * and one built for a typed effect that is no longer there is refused. Edits that name no effect row + * cost nothing. + */ +export function effectRowResolver( + source: StripOrderSource, +): (edits: readonly ControlFrameEdit[]) => readonly ControlFrameEdit[] | Promise { + return (edits) => { + if (!edits.some((edit) => PROVENANCE.has(edit))) return edits; + return (async () => { + const order = await source(); + return edits.map((edit) => { + const row = PROVENANCE.get(edit); + if (row === undefined) return edit; + const actual = stripEffectOf(order, row.stripId, row.section); + // No such section in the list: the core refuses the address with its own code. + if (actual === undefined || actual === row.effectId) return edit; + const where = `${row.section.kind === "console" ? "console slot" : "insert"} ` + + `'${row.section.kind === "console" ? row.section.slot : row.section.effect}' of strip '${row.stripId}'`; + if (row.strict) { + throw new MisoUsageError( + `${where} runs ${actual} now, and this edit was built for ${row.effectId}`, + ); + } + const control = effectControlIdIfRow(actual, row.name); + if (control === undefined) { + throw new MisoUsageError(`${where} runs ${actual} now, which has no control '${row.name}'`); + } + const resolved = Object.freeze({ + ...edit, + address: Object.freeze({ ...edit.address, control }), + }); + PROVENANCE.set(resolved, { ...row, effectId: actual }); + return resolved; + }); + })(); + }; +} + +function effectControlIdIfRow(effectId: string, name: string): number | undefined { + try { + return effectControlId(effectId, name); + } catch { + return undefined; + } } /** @@ -142,6 +255,13 @@ function sameIds(declared: Iterable, compiled: readonly string[]): boole return left.length === right.length && left.every((id, index) => id === right[index]); } +/** Whether the strips of `session` are the strips `map` names (false once a rebuild moved them). */ +export function sessionDescribesStrips(session: SessionLike, map: SessionMap): boolean { + const model = modelOf(session); + return sameIds(model.tracks.map((track) => String(track.id)), map.tracks) + && sameIds((model.submixes ?? []).map((submix) => String(submix.id)), map.submixes); +} + function layoutOf(session: SessionLike, map: SessionMap): LiveControlLayout { const model = modelOf(session); const consoleRecord = model.console as Readonly>; @@ -212,6 +332,7 @@ function controlEdit( function observationEdit( kind: LaneEdit["kind"], + stripId: string, stripIndex: number, options: { readonly rack: number; @@ -220,7 +341,7 @@ function observationEdit( readonly smoothingSamples: number; }, ): LaneEdit { - return Object.freeze({ + const edit: LaneEdit = Object.freeze({ kind, trackIndex: stripIndex, rack: options.rack, @@ -230,6 +351,8 @@ function observationEdit( smoothingSamples: options.smoothingSamples, values: Object.freeze([0, 0, 0, 0]) as unknown as readonly [number, number, number, number], }); + OBSERVED_STRIP.set(edit, stripId); + return edit; } /** @@ -246,17 +369,20 @@ function observationEdit( * index (`effect()`) to its stable ID. */ export class LiveControlEdits { - readonly #tracks: ReadonlyMap; - readonly #submixes: ReadonlyMap; - readonly #layout: LiveControlLayout | undefined; + #tracks: ReadonlyMap; + #submixes: ReadonlyMap; + #layout: LiveControlLayout | undefined; + /** The core's strips and effects as last read; supplies the effect of a named section. */ + #order: ControlStripOrder | undefined; /** What a strip reaches `set` and its effect builders through. */ readonly #context: StripContext; - constructor(map: SessionMap, session?: SessionLike) { + constructor(map: SessionMap, session?: SessionLike, order?: ControlStripOrder) { this.#tracks = new Map(map.tracks.map((trackId, index) => [trackId, index] as const)); const trackCount = map.tracks.length; this.#submixes = new Map(map.submixes.map((submixId, index) => [submixId, trackCount + index] as const)); this.#layout = session === undefined ? undefined : layoutOf(session, map); + this.#order = order; this.#context = { set: (address, value, options) => this.set(address, value, options), effect: (instance: EffectInstance) => new EffectEdits( @@ -266,6 +392,26 @@ export class LiveControlEdits { }; } + /** + * Follow the core's control list (#1562b-K): the strip indices that observation records name, and + * the effect each named section runs, are the list's after a rebuild. Builders made before this + * call keep what they were made with. A session the SDK built that no longer describes the strips + * is dropped from the layout, since the engine is no longer that session. + */ + update(order: ControlStripOrder): void { + if (order === this.#order) return; + this.#order = order; + this.#tracks = new Map(order.tracks.map((trackId, index) => [trackId, index] as const)); + this.#submixes = new Map( + order.submixes.map((submixId, index) => [submixId, order.tracks.length + index] as const), + ); + const layout = this.#layout; + if (layout !== undefined + && !(sameIds(layout.tracks.keys(), order.tracks) && sameIds(layout.submixes.keys(), order.submixes))) { + this.#layout = undefined; + } + } + /** * Set any control of the running session: one edit for one lane of one row, the sugar every typed * builder (`faderDb`, `gainDb`, an effect's `parameter`, ...) is built on. Returns the edit to pass @@ -293,16 +439,34 @@ export class LiveControlEdits { const rampSamples = options.rampSamples === undefined ? undefined : requireU32(options.rampSamples, "rampSamples"); + const named = typeof address.control !== "number"; const edit = controlEdit( address.owner, address.section, - typeof address.control === "number" ? address.control : this.#rowId(address), + named ? this.#rowId(address as ControlAddress) : address.control as number, value, address.lane, rampSamples, ); validateControlEdit(edit); - return editSet(edit); + const set = editSet(edit); + const { owner, section } = address; + if (named && owner.kind === "strip" && (section.kind === "console" || section.kind === "inserts")) { + // A row named by its section follows the core's header for that section at send time. + const effectId = this.#effectOf(owner, section); + if (effectId !== undefined) { + noteRow(set, { + stripId: owner.id, + section: section.kind === "console" + ? { kind: "console", slot: section.slot } + : { kind: "inserts", effect: section.effect }, + name: address.control as string, + effectId, + strict: false, + }); + } + } + return set; } /** The numeric ID of the row a named address names. */ @@ -339,18 +503,27 @@ export class LiveControlEdits { } #effectRowId(owner: ControlOwner, section: ControlSection, name: string): number { - const instance = owner.kind === "strip" && this.#layout !== undefined - ? this.#instanceOf(this.#layout, owner.id, section) - : undefined; - if (instance === undefined) { + const effectId = this.#effectOf(owner, section); + if (effectId === undefined) { throw new MisoUsageError( `the control '${name}' of ${section.kind === "console" ? `console slot '${section.slot}'` : section.kind === "inserts" ? `insert '${section.effect}'` : section.kind}` - + " is named, and its numeric ID depends on the effect it runs, which is known only for a " - + "session the SDK built (liveControls().withSession(session)); address it by its numeric " + + " is named, and its numeric ID depends on the effect it runs, which the engine's control " + + "list does not name for this section (is the ID right?); address it by its numeric " + "row ID instead (the `address` of its row in engine.controls())", ); } - return effectControlId(instance.effectId, name); + return effectControlId(effectId, name); + } + + /** + * The effect a strip's console slot or insert runs: the core's section header when the list is + * known (it follows a rebuild), else the session the SDK built for this engine. + */ + #effectOf(owner: ControlOwner, section: ControlSection): string | undefined { + if (owner.kind !== "strip" || (section.kind !== "console" && section.kind !== "inserts")) return undefined; + const listed = this.#order === undefined ? undefined : stripEffectOf(this.#order, owner.id, section); + if (listed !== undefined) return listed; + return this.#layout === undefined ? undefined : this.#instanceOf(this.#layout, owner.id, section)?.effectId; } #instanceOf( @@ -363,6 +536,7 @@ export class LiveControlEdits { if (section.kind === "inserts") return instances?.inserts.find((row) => row.id === section.effect); return undefined; } + /** * A track strip's live edits. The ID is not checked here: an unknown strip is refused by the core * with `control.address.unknown`. @@ -478,16 +652,29 @@ export class EffectEdits extends EffectControlBuilders { options?: SetControlOptions, ): ControlEditSet { // The row is one of this effect's, by the machine name the generated table gives it; the - // instance knows its effect, so the address carries the row's number. - return this.#set( + // instance knows its effect, so the address carries the row's number. The edit also keeps the + // effect it was typed for, so a send after a rebuild checks it against the core's header. + const section = this.#section(); + return noteRow( + this.#set( + { + owner: { kind: "strip", id: this.#stripId }, + section, + control: effectControlId(this.effectId, control), + lane, + }, + value, + options, + ), { - owner: { kind: "strip", id: this.#stripId }, - section: this.#section(), - control: effectControlId(this.effectId, control), - lane, + stripId: this.#stripId, + section: section.kind === "console" + ? { kind: "console", slot: section.slot } + : { kind: "inserts", effect: (section as { readonly effect: string }).effect }, + name: control, + effectId: this.effectId, + strict: true, }, - value, - options, ); } @@ -503,7 +690,7 @@ export class EffectEdits extends EffectControlBuilders { + "address it with liveControls().withSession(session) or effect(rack, index, effectId)", ); } - return observationEdit(armed ? "observeSubscribe" : "observeUnsubscribe", this.#stripIndex, { + return observationEdit(armed ? "observeSubscribe" : "observeUnsubscribe", this.#stripId, this.#stripIndex, { rack: RACKS[this.#rack], effectIndex: this.#effectIndex, parameterId: observation.id, @@ -538,7 +725,7 @@ export class EngineLiveControls { session?: SessionLike, booted?: Uint8Array, ) { - this.edit = new LiveControlEdits(map, session); + this.edit = new LiveControlEdits(map, session, transport.order); this.#map = map; this.#transport = transport; this.#beforeSubmit = beforeSubmit; @@ -563,7 +750,30 @@ export class EngineLiveControls { ); } assertBootedSession(session, this.#booted); - return new EngineLiveControls(this.#map, this.#transport, this.#beforeSubmit, session, this.#booted); + // After a rebuild moved the strips the engine is no longer the booted session: the names then + // resolve from the core's control list alone. + const describes = sessionDescribesStrips(session, this.#map); + return new EngineLiveControls( + this.#map, + this.#transport, + this.#beforeSubmit, + describes ? session : undefined, + this.#booted, + ); + } + + /** + * Follow the core's control list: read the structure epoch, and when a rebuild moved it, take the + * list's strips and effects (#1562b-K). Builders made afterwards address the new order. A + * transport without a list keeps the order these controls were made with. Resolves the structure + * epoch of the list it followed, or `undefined` for a transport without one. + */ + async syncStructure(): Promise { + const source = this.#transport.stripOrder; + if (source === undefined) return undefined; + const order = await source(); + this.edit.update(order); + return order.structureEpoch; } /** The engine's one transaction queue, which every value edit goes through. */ @@ -606,7 +816,22 @@ export class EngineLiveControls { return this.#submitObservation(edits, true); } - async #submitObservation(edits: readonly LaneEdit[], managed: boolean): Promise { + /** + * @internal Managed submission whose effect indices come from the plan that renders at + * `structureEpoch`. A record names its strip by the newest list and its effect by an index, and + * the core admits it against the newest plan; so when the newest list is not of that epoch, the + * indices belong to another plan and the record is refused with the retryable + * {@link structureBehindError} before it is sent. + */ + async submitManagedAt(structureEpoch: bigint | undefined, edits: readonly LaneEdit[]): Promise { + return this.#submitObservation(edits, true, structureEpoch); + } + + async #submitObservation( + edits: readonly LaneEdit[], + managed: boolean, + structureEpoch?: bigint, + ): Promise { if (edits.length === 0 || edits.length > ABI_LAYOUT.constants.maximumCommandRecords) { throw new MisoUsageError( `an observation submit must contain 1..${ABI_LAYOUT.constants.maximumCommandRecords} edits`, @@ -617,7 +842,29 @@ export class EngineLiveControls { } const beforeSubmit = this.#beforeSubmit?.(edits, managed); if (beforeSubmit !== undefined) await beforeSubmit; - const report = await this.#transport.observe(edits); + // A record names its strip by index, and a rebuild can move the strips: name each again from the + // core's list as it is now. + const source = this.#transport.stripOrder; + let sent = edits; + if (source !== undefined) { + const order = await source(); + if (structureEpoch !== undefined && order.structureEpoch !== structureEpoch) { + throw structureBehindError( + `the records were addressed at structure epoch ${structureEpoch}, and the control list is at ${order.structureEpoch}`, + ); + } + this.edit.update(order); + const strips = [...order.tracks, ...order.submixes]; + sent = edits.map((edit) => { + const stripId = OBSERVED_STRIP.get(edit); + const index = stripId === undefined ? -1 : strips.indexOf(stripId); + if (stripId === undefined || index < 0 || index === edit.trackIndex) return edit; + const moved: LaneEdit = Object.freeze({ ...edit, trackIndex: index }); + OBSERVED_STRIP.set(moved, stripId); + return moved; + }); + } + const report = await this.#transport.observe(sent); const resultConsistent = report.ok === (report.result === 0); const wholeBatch = report.ok ? report.admitted === edits.length diff --git a/sdk/src/core/observation-ended.ts b/sdk/src/core/observation-ended.ts new file mode 100644 index 000000000..9c3902604 --- /dev/null +++ b/sdk/src/core/observation-ended.ts @@ -0,0 +1,90 @@ +import { CONTROLS } from "../generated/controls.ts"; +import { + DECODER, + WIRE_MESSAGE, + WIRE_U32, + WIRE_U64, + WIRE_U8, + WIRE_UTF8, + malformed, + nestedFields, + readFields, + scalar, +} from "./control-frame.ts"; + +/** + * The reliable `OBSERVATION_ENDED` event (`8011`, #1584 D6a): one per commit that ended host-local + * observation subscriptions, carrying the stable key of every ended tap. The core decides what + * ended; this file only reads the frame it encoded. + * + * The wire is `docs/CONTROL_PROTOCOL_REGISTRY.md`'s: the 48-byte event header (kind 3, message + * `8011`, request ID zero, the committing revision), then `sequence` (field 1, u64) and one repeated + * `key` message (field 2) of strip ID (1), section (2, the control section code: inserts or console), the insert's or console + * slot's stable ID (3) and the effect-local tap ID (4). + */ +const HEADER_BYTES = 48; +const KIND_EVENT = 3; +const MESSAGE_OBSERVATION_ENDED = 0x8011; + +/** One observation tap the core ended, by the stable IDs that name it. */ +export interface ObservationEndedKey { + /** The strip's ID, a track or a submix. */ + readonly stripId: string; + readonly rack: "console" | "inserts"; + /** The console slot's ID or the insert's ID. */ + readonly effectSlotId: string; + /** The effect-local tap ID. */ + readonly tapId: number; +} + +/** What one `OBSERVATION_ENDED` event says. */ +export interface ObservationEnded { + /** The event's own sequence on the reliable lane. */ + readonly sequence: bigint; + /** The revision of the commit that ended the taps. */ + readonly revision: bigint; + readonly keys: readonly ObservationEndedKey[]; +} + +function rackName(code: number): "console" | "inserts" { + const row = CONTROLS.address.sectionKinds.find((candidate) => candidate.code === code); + if (row?.name !== "console" && row?.name !== "inserts") { + throw malformed(`an ended tap names section ${code}, which is neither the console nor the inserts`); + } + return row.name; +} + +/** + * Read an event frame the engine encoded. Returns `undefined` for any event that is not + * `OBSERVATION_ENDED`, and raises `MisoEngineError` (`internal`) for an `OBSERVATION_ENDED` frame + * that is not valid. + */ +export function decodeObservationEnded(frame: Uint8Array): ObservationEnded | undefined { + if (frame.length < HEADER_BYTES) return undefined; + const view = new DataView(frame.buffer, frame.byteOffset, frame.byteLength); + if (frame[14] !== KIND_EVENT || view.getUint16(16, true) !== MESSAGE_OBSERVATION_ENDED) return undefined; + if (HEADER_BYTES + view.getUint32(20, true) !== frame.length) throw malformed("the event payload length is wrong"); + const revision = view.getBigUint64(32, true); + const fields = readFields(frame.subarray(HEADER_BYTES), view.getUint32(40, true)); + const sequence = scalar(fields, 1, WIRE_U64, 8)?.getBigUint64(0, true); + if (sequence === undefined) throw malformed("an ended-observation event has no sequence"); + const keys = fields.filter((row) => row.id === 2).map((row) => { + if (row.wire !== WIRE_MESSAGE) throw malformed("an ended tap is not a message"); + const inner = nestedFields(row.value); + const strip = inner.find((candidate) => candidate.id === 1); + const slot = inner.find((candidate) => candidate.id === 3); + const section = scalar(inner, 2, WIRE_U8, 1)?.getUint8(0); + const tapId = scalar(inner, 4, WIRE_U32, 4)?.getUint32(0, true); + if (strip?.wire !== WIRE_UTF8 || slot?.wire !== WIRE_UTF8 || section === undefined || tapId === undefined) { + throw malformed("an ended tap is incomplete"); + } + return Object.freeze({ + stripId: DECODER.decode(strip.value), + rack: rackName(section), + effectSlotId: DECODER.decode(slot.value), + tapId, + }); + }); + if (keys.length === 0) throw malformed("an ended-observation event has no key"); + return Object.freeze({ sequence, revision, keys: Object.freeze(keys) }); +} diff --git a/sdk/src/core/observation-subscriptions.ts b/sdk/src/core/observation-subscriptions.ts index f084d91d0..dc2156bfa 100644 --- a/sdk/src/core/observation-subscriptions.ts +++ b/sdk/src/core/observation-subscriptions.ts @@ -2,7 +2,7 @@ import { ABI_LAYOUT } from "../generated/abi.ts"; import { constantValue } from "./abi.ts"; import { CATALOG } from "../generated/catalog.ts"; import type { CommandReport } from "./boundary.ts"; -import { EngineLiveControls } from "./live-controls.ts"; +import { EngineLiveControls, structureBehindError } from "./live-controls.ts"; import { MisoEngineError, MisoUsageError } from "./errors.ts"; import { MAXIMUM_OBSERVATION_READS, @@ -14,6 +14,7 @@ import { type ObservationSelection, } from "./observation.ts"; import type { LaneEdit } from "./writer.ts"; +import type { ObservationEnded } from "./observation-ended.ts"; import { normalizeTrackResponseQuery } from "./live-response.ts"; import type { TrackResponseLimits, @@ -45,6 +46,14 @@ import type { EngineCallResult } from "./boundary.ts"; */ const MAXIMUM_ADMISSION_RETRIES = 8; +/** + * How many renders a subscribe, update or close waits through for the plan that renders to catch + * up with the newest committed one, so that the map's effect indices and the list's strip indices + * belong to one plan. A commit is adopted within a block or two, so more than a few means commits + * keep arriving faster than renders. + */ +const MAXIMUM_STRUCTURE_WAITS = 8; + /** The bounded configuration accepted by `subscribeObservations`. */ export interface ObservationSubscriptionLimits { readonly maximumHandles?: number; @@ -70,6 +79,26 @@ export interface ObservationSubscriptionRequest { readonly cadenceMs?: number; /** Receives availability notifications; call `handle.readLatest()` for owned rows. */ readonly onUpdate?: (notification: ObservationSubscriptionNotification) => void; + /** + * Receives the end of selections the engine ended (#1562b-K): a rebuild removed the effect that + * owned the tap, or the ID now names an effect without it. The core announces it with a reliable + * event and the SDK reports it here once for each commit. A subscription whose selections have all + * ended is closed. + */ + readonly onEnd?: (end: ObservationSubscriptionEnd) => void; +} + +/** What `onEnd` receives: the selections of one subscription that one commit ended. */ +export interface ObservationSubscriptionEnd { + readonly handle: ObservationSubscription; + readonly owner: bigint; + readonly epoch: bigint; + /** The revision of the commit that ended them. */ + readonly revision: bigint; + /** The selections of this subscription that this commit ended, as the subscription asked for them. */ + readonly ended: readonly ObservationSelection[]; + /** The selections still subscribed; empty when the whole subscription ended and was closed. */ + readonly remaining: readonly ObservationSelection[]; } export interface ObservationSubscriptionConfiguration { @@ -117,6 +146,8 @@ export interface ObservationSubscription { readonly configuration: ObservationSubscriptionConfiguration; readonly appliedAtSample: bigint; readonly bounds: ObservationSubscriptionBounds; + /** Every selection of this subscription the engine has ended so far, in the order it ended them. */ + readonly ended: readonly ObservationSelection[]; readLatest(): readonly ObservationReadResult[]; pump(): Promise; /** @@ -125,6 +156,11 @@ export interface ObservationSubscription { * subscription still holds the tap's queue slot and no render has drained it yet. The headless * engine refuses at once, so the caller renders and calls again. The browser engine first waits * for a render, up to `requestDeadlineMs`, and ends in the same error when none comes. + * + * The same holds between a rebuild's commit and the block that adopts the replacement plan: the + * records of a subscribe, update or close name effects of the plan that renders, and the core + * admits them against the newest plan, so none is sent until a render has adopted it. The wait + * ends in the typed, retryable `backpressure` error (`sdk.observation.structure_behind`). */ update(request: ObservationSubscriptionRequest): Promise; close(): Promise; @@ -142,6 +178,13 @@ export interface ObservationSubscriptionTransport { observationMap(): MaybePromise; readObservations(selections: readonly ObservationSelection[]): MaybePromise; liveControls(): MaybePromise; + /** + * Subscribe to the core's `OBSERVATION_ENDED` events (#1584): the listener gets each one, once, + * as decoded from the reliable lane. Returns the unsubscribe function. A transport that cannot + * read the lane leaves this absent, and a managed subscription then ends only when its owner + * closes it. + */ + observationEnded?(listener: (ended: ObservationEnded) => void): () => void; /** * A transport whose renders run on their own clock (a browser's audio thread) lets a managed * submission that was refused with `backpressure` wait for a render and be sent again. A @@ -197,6 +240,7 @@ interface EffectiveLimits { interface ResolvedSelection { readonly selection: ObservationSelection; readonly binding: ObservationBinding; + readonly windowBlocks: number; readonly baseKey: string; readonly key: string; readonly nativeSelection: ObservationSelection; @@ -207,13 +251,20 @@ interface BindingState { readonly baseKey: string; readonly selection: ObservationSelection; readonly nativeSelection: ObservationSelection; - readonly binding: ObservationBinding; + /** The owner's address as the newest map gave it: a rebuild can move the effect's index. */ + binding: ObservationBinding; readonly windowBlocks: number; refs: number; latest: ObservationReadResult | undefined; lastSequence: bigint | undefined; nativeMissed: bigint; appliedAtSample: bigint; + /** + * The revision the core had committed when this arm was sent, or `undefined` when the transport + * cannot read it. An `OBSERVATION_ENDED` event of a commit at or before it belongs to an earlier + * arm of the same tap (a late event), and does not end this one. + */ + readonly armRevision: bigint | undefined; } interface HandleState { @@ -222,7 +273,10 @@ interface HandleState { epoch: bigint; configuration: ObservationSubscriptionConfiguration; entries: readonly ResolvedSelection[]; + /** The selections the engine ended. */ + ended: readonly ObservationSelection[]; callback: ((notification: ObservationSubscriptionNotification) => void) | undefined; + onEnd: ((end: ObservationSubscriptionEnd) => void) | undefined; appliedAtSample: bigint; readonly cursor: Map; readonly nativeMissedSeen: Map; @@ -280,7 +334,11 @@ function canonicalLimits(overrides: ObservationSubscriptionLimits | undefined): function normalizedRequest( request: ObservationSubscriptionRequest, subscriptionLimits: EffectiveLimits, -): { readonly configuration: ObservationSubscriptionConfiguration; readonly callback: ((notification: ObservationSubscriptionNotification) => void) | undefined } { +): { + readonly configuration: ObservationSubscriptionConfiguration; + readonly callback: ((notification: ObservationSubscriptionNotification) => void) | undefined; + readonly onEnd: ((end: ObservationSubscriptionEnd) => void) | undefined; +} { if (request === null || typeof request !== "object") { throw new MisoUsageError("observation subscription request must be an object"); } @@ -309,6 +367,9 @@ function normalizedRequest( if (request.onUpdate !== undefined && typeof request.onUpdate !== "function") { throw new MisoUsageError("observation onUpdate must be a function"); } + if (request.onEnd !== undefined && typeof request.onEnd !== "function") { + throw new MisoUsageError("observation onEnd must be a function"); + } return { configuration: Object.freeze({ selections: Object.freeze(selections), @@ -316,6 +377,7 @@ function normalizedRequest( cadenceMs, }), callback: request.onUpdate, + onEnd: request.onEnd, }; } @@ -333,6 +395,7 @@ function mapSelection(map: ObservationMap, selection: ObservationSelection, wind return Object.freeze({ selection, binding, + windowBlocks, baseKey: keyFor(selection), key: bindingKey(selection, windowBlocks), nativeSelection: cloneSelection(selection, "both"), @@ -420,6 +483,7 @@ class ObservationSubscriptionImpl implements ObservationSubscription { get configuration(): ObservationSubscriptionConfiguration { return this.#state.configuration; } get appliedAtSample(): bigint { return this.#state.appliedAtSample; } get bounds(): ObservationSubscriptionBounds { return this.#owner.bounds; } + get ended(): readonly ObservationSelection[] { return this.#state.ended; } readLatest(): readonly ObservationReadResult[] { return this.#owner.readLatest(this.#state); @@ -498,6 +562,9 @@ export class ObservationSubscriptionOwner { #timer: unknown; #timerCadence: number | undefined; #disposed = false; + #unsubscribeEnded: (() => void) | undefined; + /** `OBSERVATION_ENDED` events that arrived while a mutation was in flight; applied after it. */ + #endedWhileBusy: ObservationEnded[] = []; constructor( transport: ObservationSubscriptionTransport, @@ -514,6 +581,7 @@ export class ObservationSubscriptionOwner { || transport.spectrumRead === undefined || transport.spectrumStop === undefined ? undefined : canonicalSpectrumSubscriptionLimits(spectrumSubscriptionLimits); + this.#unsubscribeEnded = transport.observationEnded?.((ended) => this.#onEnded(ended)); } get bounds(): ObservationSubscriptionBounds { @@ -559,30 +627,34 @@ export class ObservationSubscriptionOwner { throw new MisoUsageError(`observation subscriptions are capped at ${this.#subscriptionLimits.maximumHandles}`); } await this.#waitForPoll(); - const map = await this.#transport.observationMap(); - this.#assertEpoch(epoch); - const entries = normalized.configuration.selections.map((selection) => - mapSelection(map, selection, normalized.configuration.windowBlocks)); - if (entries.some((entry) => { - const managed = [...this.#bindings.values()].find((binding) => binding.baseKey === entry.baseKey); - return managed !== undefined && managed.key !== entry.key; - })) { - throw new MisoUsageError("observation window conflicts with a managed subscriber"); - } - const newEntries = this.#newEntries(entries); - if (this.#bindings.size + newEntries.length > this.#subscriptionLimits.maximumBindings) { - throw new MisoUsageError(`managed observation bindings are capped at ${this.#subscriptionLimits.maximumBindings}`); - } + const settled = await this.#settle(epoch, (map) => { + const entries = normalized.configuration.selections.map((selection) => + mapSelection(map, selection, normalized.configuration.windowBlocks)); + if (entries.some((entry) => { + const managed = [...this.#bindings.values()].find((binding) => binding.baseKey === entry.baseKey); + return managed !== undefined && managed.key !== entry.key; + })) { + throw new MisoUsageError("observation window conflicts with a managed subscriber"); + } + const newEntries = this.#newEntries(entries); + if (this.#bindings.size + newEntries.length > this.#subscriptionLimits.maximumBindings) { + throw new MisoUsageError(`managed observation bindings are capped at ${this.#subscriptionLimits.maximumBindings}`); + } + this.#assertCommandSize(newEntries.length); + return { entries, newEntries, records: newEntries.length }; + }); + const { entries, newEntries } = settled.resolved; + const { liveControls } = settled; const preflight = await this.#preflight(newEntries, epoch); - this.#assertCommandSize(newEntries.length); - const liveControls = newEntries.length === 0 ? undefined : await this.#transport.liveControls(); - this.#assertEpoch(epoch); const edits = newEntries.map((entry) => this.#edit(liveControls!, entry, true, normalized.configuration.windowBlocks)); - const report = await this.#submit(liveControls, edits, epoch); + const armRevision = await this.#committedRevision(liveControls); + const report = await this.#submit(liveControls, edits, epoch, settled.structureEpoch); this.#assertEpoch(epoch); const appliedAtSample = report?.appliedAtSample ?? this.#existingSample(entries); - const state = this.#newHandle(normalized.configuration, entries, normalized.callback, appliedAtSample); + const state = this.#newHandle( + normalized.configuration, entries, normalized.callback, normalized.onEnd, appliedAtSample, + ); newEntries.forEach((entry) => { const row = preflight.get(entry.key); this.#bindings.set(entry.key, { @@ -597,10 +669,12 @@ export class ObservationSubscriptionOwner { lastSequence: undefined, nativeMissed: 0n, appliedAtSample: report?.appliedAtSample ?? 0n, + armRevision, }); }); entries.filter((entry) => !newEntries.includes(entry)).forEach((entry) => { - this.#bindings.get(entry.key)!.refs += 1; + const held = this.#bindings.get(entry.key); + if (held !== undefined) held.refs += 1; }); this.#handles.set(state.id, state); this.#startTimer(); @@ -614,55 +688,54 @@ export class ObservationSubscriptionOwner { this.#assertHandle(state); const epoch = this.#epoch; await this.#waitForPoll(); - const map = await this.#transport.observationMap(); - this.#assertEpoch(epoch); - const entries = normalized.configuration.selections.map((selection) => - mapSelection(map, selection, normalized.configuration.windowBlocks)); - const currentByKey = new Map(state.entries.map((entry) => [entry.key, entry] as const)); - const desiredByKey = new Map(entries.map((entry) => [entry.key, entry] as const)); - const remove = state.entries.filter((entry) => !desiredByKey.has(entry.key)); - const add = entries.filter((entry) => !currentByKey.has(entry.key)); - const conflicts = entries.filter((entry) => { - const base = [...this.#bindings.values()].find((binding) => binding.baseKey === entry.baseKey); - if (base === undefined || base.key === entry.key) return false; - const ownedByThisHandle = state.entries.some((current) => current.key === base.key); - return !ownedByThisHandle || base.refs > 1; - }); - if (conflicts.length > 0) { - throw new MisoUsageError("observation window conflicts with a managed subscriber"); - } - const newEntries = add.filter((entry) => !this.#bindings.has(entry.key)); - if (this.#bindings.size + newEntries.length > this.#subscriptionLimits.maximumBindings) { - throw new MisoUsageError(`managed observation bindings are capped at ${this.#subscriptionLimits.maximumBindings}`); - } - // The bindings this handle alone holds and drops. - const release = remove.filter((entry) => { - const binding = this.#bindings.get(entry.key); - return binding !== undefined && binding.refs === 1; + const settled = await this.#settle(epoch, (map) => { + this.#remap(map); + const entries = normalized.configuration.selections.map((selection) => + mapSelection(map, selection, normalized.configuration.windowBlocks)); + const currentByKey = new Map(state.entries.map((entry) => [entry.key, entry] as const)); + const desiredByKey = new Map(entries.map((entry) => [entry.key, entry] as const)); + const remove = state.entries.filter((entry) => !desiredByKey.has(entry.key)); + const add = entries.filter((entry) => !currentByKey.has(entry.key)); + const conflicts = entries.filter((entry) => { + const base = [...this.#bindings.values()].find((binding) => binding.baseKey === entry.baseKey); + if (base === undefined || base.key === entry.key) return false; + const ownedByThisHandle = state.entries.some((current) => current.key === base.key); + return !ownedByThisHandle || base.refs > 1; + }); + if (conflicts.length > 0) { + throw new MisoUsageError("observation window conflicts with a managed subscriber"); + } + const newEntries = add.filter((entry) => !this.#bindings.has(entry.key)); + if (this.#bindings.size + newEntries.length > this.#subscriptionLimits.maximumBindings) { + throw new MisoUsageError(`managed observation bindings are capped at ${this.#subscriptionLimits.maximumBindings}`); + } + // The bindings this handle alone holds and drops. + const release = remove.filter((entry) => { + const binding = this.#bindings.get(entry.key); + return binding !== undefined && binding.refs === 1; + }); + // A window change moves one tap from one key to another. A submission names a tap once: an + // `Observe` record overwrites the tap's whole state at the block that drains it, so the + // arm with the new window is the re-arm, and the old key sends no disarm (#1345 D2). + const disarm = release.filter((entry) => !newEntries.some((added) => added.baseKey === entry.baseKey)); + this.#assertCommandSize(newEntries.length + disarm.length); + return { entries, remove, add, newEntries, release, disarm, records: disarm.length + newEntries.length }; }); - // A window change moves one tap from one key to another. A submission names a tap once: an - // `Observe` record overwrites the tap's whole state at the block that drains it, so the - // arm with the new window is the re-arm, and the old key sends no disarm (#1345 D2). - const disarm = release.filter((entry) => !newEntries.some((added) => added.baseKey === entry.baseKey)); - this.#assertCommandSize(newEntries.length + disarm.length); + const { entries, remove, add, newEntries, release, disarm } = settled.resolved; + const { liveControls } = settled; const preflightEntries = newEntries.filter((entry) => !state.entries.some((current) => { const binding = this.#bindings.get(current.key); return current.baseKey === entry.baseKey && binding?.refs === 1; })); const preflight = await this.#preflight(preflightEntries, epoch); - const liveControls = disarm.length + newEntries.length === 0 - ? undefined - : await this.#transport.liveControls(); - this.#assertEpoch(epoch); const edits: LaneEdit[] = [ - ...disarm.map((entry) => this.#edit( - liveControls!, entry, false, this.#bindings.get(entry.key)!.windowBlocks, - )), + ...disarm.map((entry) => this.#edit(liveControls!, entry, false, entry.windowBlocks)), ...newEntries.map((entry) => this.#edit( liveControls!, entry, true, normalized.configuration.windowBlocks, )), ]; - const report = await this.#submit(liveControls, edits, epoch); + const armRevision = await this.#committedRevision(liveControls); + const report = await this.#submit(liveControls, edits, epoch, settled.structureEpoch); this.#assertEpoch(epoch); const appliedAtSample = report?.appliedAtSample ?? this.#existingSample(entries); release.forEach((entry) => this.#bindings.delete(entry.key)); @@ -684,14 +757,17 @@ export class ObservationSubscriptionOwner { lastSequence: undefined, nativeMissed: 0n, appliedAtSample: report?.appliedAtSample ?? 0n, + armRevision, }); }); add.filter((entry) => !newEntries.includes(entry)).forEach((entry) => { - this.#bindings.get(entry.key)!.refs += 1; + const held = this.#bindings.get(entry.key); + if (held !== undefined) held.refs += 1; }); state.configuration = normalized.configuration; state.entries = entries; state.callback = normalized.callback; + state.onEnd = normalized.onEnd; state.appliedAtSample = appliedAtSample; state.cursor.clear(); state.nativeMissedSeen.clear(); @@ -718,17 +794,27 @@ export class ObservationSubscriptionOwner { this.#assertHandle(state); const epoch = this.#epoch; await this.#waitForPoll(); - const disarm = state.entries.filter((entry) => { + let disarm = state.entries.filter((entry) => { const binding = this.#bindings.get(entry.key); return binding !== undefined && binding.refs === 1; }); this.#assertCommandSize(disarm.length); - const liveControls = disarm.length === 0 ? undefined : await this.#transport.liveControls(); - this.#assertEpoch(epoch); - const edits = disarm.map((entry) => this.#edit( - liveControls!, entry, false, this.#bindings.get(entry.key)!.windowBlocks, - )); - const report = await this.#submit(liveControls, edits, epoch); + let liveControls: EngineLiveControls | undefined; + let structureEpoch: bigint | undefined; + if (disarm.length !== 0) { + // The records name the tap by the indices of the plan that renders now, which a rebuild can + // have changed since the subscription was made. + const held = disarm; + const settled = await this.#settle(epoch, (map) => { + this.#remap(map); + return { records: held.length }; + }); + liveControls = settled.liveControls; + structureEpoch = settled.structureEpoch; + disarm = state.entries.filter((entry) => held.some((kept) => kept.key === entry.key)); + } + const edits = disarm.map((entry) => this.#edit(liveControls!, entry, false, entry.windowBlocks)); + const report = await this.#submit(liveControls, edits, epoch, structureEpoch); this.#assertEpoch(epoch); if (report !== undefined && !report.ok) throw commandFailure("observation close was refused", report); disarm.forEach((entry) => this.#bindings.delete(entry.key)); @@ -775,7 +861,87 @@ export class ObservationSubscriptionOwner { this.#spectrumHandles.clear(); this.#spectrumJob = undefined; this.#stopTimer(); - if (disposed) this.#disposed = true; + this.#endedWhileBusy = []; + if (disposed) { + this.#disposed = true; + this.#unsubscribeEnded?.(); + this.#unsubscribeEnded = undefined; + } + } + + /** + * The engine ended taps (`OBSERVATION_ENDED`): the commit removed the effect that owned them, or + * the ID now names an effect without that tap. The core decided it and has already dropped the + * arms, so nothing is sent; the owner forgets the bindings and tells each subscription once what + * it lost (#1562b-K). A subscription left with no selection is closed. + */ + #onEnded(ended: ObservationEnded): void { + if (this.#disposed) return; + if (this.#mutationBusy) { + this.#endedWhileBusy.push(ended); + return; + } + const lost = new Set(ended.keys.map((key) => keyFor({ + trackId: key.stripId, rack: key.rack, effectSlotId: key.effectSlotId, tapId: key.tapId, channels: "both", + }))); + const gone = new Set(); + for (const binding of [...this.#bindings.values()]) { + if (!lost.has(binding.baseKey)) continue; + // An event of a commit at or before this arm was sent ended an earlier arm of the tap (the + // event came late, behind a removal and a re-add); this arm is the core's now. + if (binding.armRevision !== undefined && ended.revision <= binding.armRevision) continue; + this.#bindings.delete(binding.key); + gone.add(binding.baseKey); + } + if (gone.size === 0) return; + for (const state of [...this.#handles.values()]) { + const dropped = state.entries.filter((entry) => gone.has(entry.baseKey)); + if (dropped.length === 0) continue; + const remaining = state.entries.filter((entry) => !gone.has(entry.baseKey)); + for (const entry of dropped) { + state.cursor.delete(entry.key); + state.nativeMissedSeen.delete(entry.key); + } + state.entries = Object.freeze(remaining); + const droppedSelections = Object.freeze(dropped.map((entry) => entry.selection)); + state.ended = Object.freeze([...state.ended, ...droppedSelections]); + if (remaining.length === 0) { + state.closed = true; + this.#handles.delete(state.id); + } + if (state.publicHandle === undefined) continue; + const end: ObservationSubscriptionEnd = Object.freeze({ + handle: state.publicHandle, + owner: this.#owner, + epoch: this.#epoch, + revision: ended.revision, + ended: droppedSelections, + remaining: Object.freeze(remaining.map((entry) => entry.selection)), + }); + try { state.onEnd?.(end); } catch { /* callbacks cannot break the owner */ } + } + this.#stopTimerIfIdle(); + } + + /** + * Take the newest map's address for every held tap. A rebuild can move an effect to another index + * or strip; a tap is held by its stable IDs, and only the records that arm and disarm it name the + * index. A tap the map no longer has keeps its last address: the core ends it with its own event. + */ + #remap(map: ObservationMap): void { + const current = (selection: ObservationSelection): ObservationBinding | undefined => map.bindings.find((candidate) => + candidate.trackId === selection.trackId && candidate.rack === selection.rack + && candidate.effectSlotId === selection.effectSlotId); + for (const binding of this.#bindings.values()) { + const fresh = current(binding.selection); + if (fresh !== undefined) binding.binding = fresh; + } + for (const state of this.#handles.values()) { + state.entries = Object.freeze(state.entries.map((entry) => { + const fresh = current(entry.selection); + return fresh === undefined || fresh === entry.binding ? entry : Object.freeze({ ...entry, binding: fresh }); + })); + } } /** Invalidate only the spectrum lifetime after its analysis transport has failed. */ @@ -804,6 +970,7 @@ export class ObservationSubscriptionOwner { configuration: ObservationSubscriptionConfiguration, entries: readonly ResolvedSelection[], callback: ((notification: ObservationSubscriptionNotification) => void) | undefined, + onEnd: ((end: ObservationSubscriptionEnd) => void) | undefined, appliedAtSample: bigint, ): HandleState { const state: HandleState = { @@ -812,7 +979,9 @@ export class ObservationSubscriptionOwner { epoch: this.#epoch, configuration, entries: Object.freeze([...entries]), + ended: Object.freeze([]), callback, + onEnd, appliedAtSample, cursor: new Map(), nativeMissedSeen: new Map(), @@ -884,12 +1053,87 @@ export class ObservationSubscriptionOwner { .observe(tapName(entry.binding, entry.selection.tapId) as never, armed, windowBlocks); } + /** + * Read the map of the plan that renders and the newest control list until both name one structure + * epoch, so that the effect index of a record (from the map) and its strip index (from the list) + * are of the plan the core admits the record against: the newest (M1 of the batch 12b review). + * + * `resolve` maps the request against a map and says how many records it needs. A request that + * needs none returns at once, with no live controls and no wait. Between a rebuild's commit and + * the block that adopts its replacement the map is older than the list: a browser engine waits + * for a render (within its render-wait bound) and reads again, and an engine that renders only + * when its caller does throws the retryable `backpressure` error at once. Either way, a map that + * does not catch up in {@link MAXIMUM_STRUCTURE_WAITS} renders or the bound refuses with that + * error, and no misaddressed record is sent. + */ + async #settle( + epoch: bigint, + resolve: (map: ObservationMap) => T, + ): Promise<{ + readonly resolved: T; + readonly liveControls: EngineLiveControls | undefined; + /** The epoch the records are addressed at, or `undefined` when nothing could be compared. */ + readonly structureEpoch: bigint | undefined; + }> { + const wait = this.#transport.renderWait; + let deadline: number | undefined; + for (let waits = 0; ; waits += 1) { + // The position is read before the map, so a block rendered after the map was read counts. + const since = wait?.mark(); + since?.catch(() => undefined); + const map = await this.#transport.observationMap(); + this.#assertEpoch(epoch); + const resolved = resolve(map); + if (resolved.records === 0) return { resolved, liveControls: undefined, structureEpoch: undefined }; + const liveControls = await this.#transport.liveControls(); + this.#assertEpoch(epoch); + // A rebuild can have moved the strips: the records name them by the core's list as it is now. + const listEpoch = await liveControls.syncStructure(); + this.#assertEpoch(epoch); + if (map.structureEpoch === undefined || listEpoch === undefined || map.structureEpoch === listEpoch) { + return { resolved, liveControls, structureEpoch: listEpoch }; + } + const behind = structureBehindError( + `the map is of structure epoch ${map.structureEpoch} and the control list is at ${listEpoch}`, + ); + if (wait === undefined || since === undefined || waits >= MAXIMUM_STRUCTURE_WAITS) throw behind; + deadline ??= Date.now() + wait.boundMs; + const remaining = deadline - Date.now(); + if (remaining <= 0) throw behind; + const rendered = await wait.next(remaining, await since); + this.#assertEpoch(epoch); + if (!rendered) throw behind; + } + } + + /** + * The revision the core has committed now, read just before an arm is sent, or `undefined` when + * the transport cannot read it. A commit that lands after the read is later than the arm's + * revision, so its end event is applied to the arm (never ignored by mistake). + */ + async #committedRevision(liveControls: EngineLiveControls | undefined): Promise { + const queue = liveControls?.queue; + if (queue === undefined || typeof queue.readCommittedRevision !== "function") return undefined; + try { + return await queue.readCommittedRevision(); + } catch { + return undefined; + } + } + async #submit( liveControls: EngineLiveControls | undefined, edits: readonly LaneEdit[], epoch: bigint, + structureEpoch?: bigint, ): Promise { if (edits.length === 0) return undefined; + const managedAt = (liveControls as EngineLiveControls & { + readonly submitManagedAt?: ( + structureEpoch: bigint | undefined, + edits: readonly LaneEdit[], + ) => Promise; + } | undefined)?.submitManagedAt; const managedSubmit = (liveControls as EngineLiveControls & { readonly submitManaged?: (...edits: readonly LaneEdit[]) => Promise; } | undefined)?.submitManaged; @@ -901,9 +1145,11 @@ export class ObservationSubscriptionOwner { // answers it first. A block rendered after it and before the refusal returns is then seen. const since = wait?.mark(); since?.catch(() => undefined); - const report = managedSubmit === undefined - ? await liveControls!.submit(...edits) - : await managedSubmit.call(liveControls, ...edits); + const report = managedAt !== undefined + ? await managedAt.call(liveControls, structureEpoch, edits) + : managedSubmit === undefined + ? await liveControls!.submit(...edits) + : await managedSubmit.call(liveControls, ...edits); if (report.ok) return report; // A submission names each tap once, so a refusal for room means a record of an earlier // submission still holds the tap's slot, and a render drains it. That refusal alone is flow @@ -1075,6 +1321,9 @@ export class ObservationSubscriptionOwner { #assertHandle(state: HandleState): void { this.#assertOpen(); if (state.closed || state.epoch !== this.#epoch || this.#handles.get(state.id) !== state) { + if (state.ended.length > 0 && state.entries.length === 0 && state.epoch === this.#epoch) { + throw new MisoUsageError("the observation subscription ended: the engine removed every tap it held"); + } throw new MisoUsageError("the observation subscription handle is closed or stale"); } } @@ -1086,7 +1335,13 @@ export class ObservationSubscriptionOwner { this.#mutationBusy = true; const run = this.#mutation.then(operation, operation); this.#mutation = run.then(() => undefined, () => undefined); - return run.finally(() => { this.#mutationBusy = false; }); + return run.finally(() => { + this.#mutationBusy = false; + // An end that arrived mid-operation applies now, to the state the operation left. + const waiting = this.#endedWhileBusy; + this.#endedWhileBusy = []; + for (const ended of waiting) this.#onEnded(ended); + }); } /** Subscribe to one selected track response while sharing this owner's lifetime and poll. */ diff --git a/sdk/src/core/observation.ts b/sdk/src/core/observation.ts index 35a6f74cb..406fb4ef8 100644 --- a/sdk/src/core/observation.ts +++ b/sdk/src/core/observation.ts @@ -38,6 +38,14 @@ export interface ObservationBinding { /** The current owner map used to resolve stable selections to numeric Wasm addresses. */ export interface ObservationMap { + /** + * The structure epoch of the control list whose strip order `bindings` follow (#1562b): the plan + * that renders. A rebuild changes it at the block that adopts the replacement, so between the + * rebuild's commit and that block it is older than the newest list's, and a binding's effect + * index is not the one the core admits a record against. A transport that cannot say leaves it + * out, and nothing is checked against it. + */ + readonly structureEpoch?: bigint; readonly bindings: readonly ObservationBinding[]; } @@ -201,6 +209,7 @@ export function validateObservationSelections(selections: readonly ObservationSe export function enrichObservationMap( strips: readonly string[], raw: readonly RawObservationBinding[], + structureEpoch?: bigint, ): ObservationMap { const bindings = raw.map((binding, index) => { if (!Number.isSafeInteger(binding.trackIndex) || binding.trackIndex < 0 @@ -228,7 +237,9 @@ export function enrichObservationMap( tapIds: Object.freeze([...binding.tapIds]), }); }); - return Object.freeze({ bindings: Object.freeze(bindings) }); + return Object.freeze(structureEpoch === undefined + ? { bindings: Object.freeze(bindings) } + : { structureEpoch, bindings: Object.freeze(bindings) }); } /** diff --git a/sdk/src/generated/abi.ts b/sdk/src/generated/abi.ts index da213f3ab..a8ed3d392 100644 --- a/sdk/src/generated/abi.ts +++ b/sdk/src/generated/abi.ts @@ -416,7 +416,7 @@ export const ABI_LAYOUT = deepFreeze( ] }, "meterHeader": { - "bytes": 80, + "bytes": 88, "fields": [ { "name": "structSize", @@ -487,6 +487,11 @@ export const ABI_LAYOUT = deepFreeze( "name": "successorPointer", "offset": 76, "type": "u32" + }, + { + "name": "structureEpoch", + "offset": 80, + "type": "u64" } ] }, diff --git a/sdk/src/headless/engine.ts b/sdk/src/headless/engine.ts index 23ab18a6b..6c14e63db 100644 --- a/sdk/src/headless/engine.ts +++ b/sdk/src/headless/engine.ts @@ -42,7 +42,8 @@ import type { ControlListTransport, ControlState, } from "../core/control-list.ts"; -import { EngineLiveControls } from "../core/live-controls.ts"; +import { EngineLiveControls, effectRowResolver, sessionDescribesStrips } from "../core/live-controls.ts"; +import { decodeObservationEnded } from "../core/observation-ended.ts"; import { SessionBuilder } from "../core/session.ts"; import { MisoEngineError, MisoUsageError } from "../core/errors.ts"; import type { ErrorPhase, MisoDiagnostic, MisoErrorCode } from "../core/errors.ts"; @@ -234,7 +235,9 @@ export class OfflineEngine { * An effect's observation queue holds one subscription per tap until a block drains it. A * subscribe, update or close that finds the slot taken by an earlier one is refused at once with * the typed, retryable `backpressure` error. It never waits: this engine renders only when the - * caller does, so the caller calls `render()` and submits again. + * caller does, so the caller calls `render()` and submits again. The same refusal meets a request + * made between a rebuild's commit and the render that adopts its plan: the records would name + * effects of the plan that renders, and the core admits them against the newest plan. */ subscribeObservations(request: ObservationSubscriptionRequest): Promise { return this.#observationOwner().subscribe(request).then((receipt) => receipt.handle); @@ -281,12 +284,21 @@ export class OfflineEngine { * addressable by their stable IDs; otherwise use `withSession()` or address them by index. */ liveControls(): EngineLiveControls { - return new EngineLiveControls(this.sessionMap(), { + const map = this.sessionMap(); + // A rebuild can move the strips away from the session the SDK built; the engine is then no + // longer that session, and the core's control list is the only description of it. + const session = this.#session !== undefined && sessionDescribesStrips(this.#session, map) + ? this.#session + : undefined; + return new EngineLiveControls(map, { queue: this.controlTransactions(), observe: (edits) => this.submitCommands(encodeLaneEdits(edits), edits.length), + // The boundary caches the order by structure epoch: a probe each call, the list when it moved. + stripOrder: () => this.#boundary.stripOrder(), + order: this.#boundary.stripOrder(), }, (edits, managed) => this.#observationSubscriptions?.beforeLiveControlSubmit(edits, managed), - this.#session, + session, this.#booted); } @@ -315,6 +327,8 @@ export class OfflineEngine { requestIds, limits, readRevision: () => this.#boundary.controlRevision(), + // An edit that names an effect's row is checked against the core's list before it is sent. + prepare: effectRowResolver(() => this.#boundary.stripOrder()), }, ); } @@ -479,6 +493,11 @@ export class OfflineEngine { observationMap: () => this.observationMap(), readObservations: (selections) => this.readObservations(selections), liveControls: () => this.liveControls(), + observationEnded: (listener) => this.#boundary.onControlEvent((event) => { + // The reliable lane carries it; the core decides what ended, the SDK only reads the frame. + const ended = event.lane === 0 ? decodeObservationEnded(event.frame) : undefined; + if (ended !== undefined) listener(ended); + }), spectrumPrepared: () => this.#boundary.preparedSpectrumQuery(), spectrumPreparedCollection: () => this.#boundary.preparedSpectrumCollection(), spectrumSelect: (query) => this.#boundary.selectSpectrum(query), diff --git a/sdk/src/headless/index.ts b/sdk/src/headless/index.ts index 0fb8a1382..bcf162048 100644 --- a/sdk/src/headless/index.ts +++ b/sdk/src/headless/index.ts @@ -9,6 +9,7 @@ export type { ObservationSubscriptionBounds, ObservationSubscriptionConfiguration, ObservationSubscriptionLimits, + ObservationSubscriptionEnd, ObservationSubscriptionNotification, ObservationSubscriptionReceipt, ObservationSubscriptionRequest, diff --git a/sdk/src/index.ts b/sdk/src/index.ts index 0a3d8b89a..31c5b6a26 100644 --- a/sdk/src/index.ts +++ b/sdk/src/index.ts @@ -38,6 +38,7 @@ export type { ObservationSubscriptionBounds, ObservationSubscriptionConfiguration, ObservationSubscriptionLimits, + ObservationSubscriptionEnd, ObservationSubscriptionNotification, ObservationSubscriptionReceipt, ObservationSubscriptionRequest, diff --git a/sdk/test/console-evals.mjs b/sdk/test/console-evals.mjs index 1841f3d05..19899a525 100644 --- a/sdk/test/console-evals.mjs +++ b/sdk/test/console-evals.mjs @@ -1016,33 +1016,24 @@ function oneInsert(effectId, bypassed) { } describe("a live bypass edit is a SET_CONTROL on control 0 of the instance (#1563B)", () => { - test("lifting a session bypass is either applied or refused by the core, never acknowledged and ignored", async () => { + test("lifting a session bypass reaches the render, live or by a plan rebuild", async () => { // The SDK holds no list of effects that keep a prepared bypass any more (#1563 D3): it sends - // the lift and the core judges it. The core renders the unbypassed session on most effects, and - // refuses the lift of the two that keep a prepared bypass, the delay and the multiband, because - // that is a plan rebuild this module cannot apply in place yet (`web.apply.rebuild_pending`). - // Red mutation: a core that acknowledges the lift of either and renders the bypassed session. + // the lift and the core judges it. The core lifts it live on most effects, and on the two that + // keep a prepared bypass, the delay and the multiband, as a plan rebuild that the browser module + // now prepares and publishes like the C ABI does (#1562b). Either way the render is the + // unbypassed session's. + // Red mutation: a core that acknowledges the lift of any effect and renders the bypassed session. assert.deepEqual(Object.keys(AUDIBLE).sort(), CATALOG.effects.map((row) => row.id).sort()); - const refused = []; for (const { id } of CATALOG.effects) { const bypassed = await renderBlocks(oneInsert(id, true)); const enabled = await renderBlocks(oneInsert(id, false)); assert.notDeepEqual(bypassed, enabled, `${id}: the fixture must hear the bypass`); - let rendered; - try { - rendered = await renderBlocks( - oneInsert(id, true), - (controls) => controls.edit.track("t").insert("fx", id).bypass(false), - ); - } catch (error) { - assert.ok(error instanceof MisoEngineError, `${id}: ${error}`); - assert.equal(error.diagnostics[0]?.code, "web.apply.rebuild_pending", id); - refused.push(id); - continue; - } - assert.deepEqual(rendered, enabled, `${id}: the engine's live lift`); + const rendered = await renderBlocks( + oneInsert(id, true), + (controls) => controls.edit.track("t").insert("fx", id).bypass(false), + ); + assert.deepEqual(rendered, enabled, `${id}: the engine's lift`); } - assert.deepEqual(refused.sort(), ["miso.delay", "miso.multiband-compressor"]); }); }); diff --git a/sdk/test/control-list-evals.mjs b/sdk/test/control-list-evals.mjs index a1df0bb57..f1a0b5ce4 100644 --- a/sdk/test/control-list-evals.mjs +++ b/sdk/test/control-list-evals.mjs @@ -636,7 +636,7 @@ describe("#1564A gate 4 -- edit.set reaches any row", () => { } }); - test(`${transport}: set on a rebuild-only row passes the core's web.apply.rebuild_pending through`, async () => { + test(`${transport}: set on a rebuild-only row is committed on the rebuild path`, async () => { const engine = await boot(); try { const { controls, list } = await endpoint(engine, transport); @@ -644,12 +644,10 @@ describe("#1564A gate 4 -- edit.set reaches any row", () => { "t1 delay_samples"); assert.equal(delay.class, "rebuildOnly"); const before = engine.controlRevision(); - await assert.rejects(controls.submit(controls.edit.set(delay.address, 64)), (error) => { - assert.ok(error instanceof MisoEngineError, "a refused call, not an SDK refusal"); - assert.equal(error.diagnosticCode, "web.apply.rebuild_pending"); - return true; - }); - assert.equal(engine.controlRevision(), before, "nothing was committed"); + const report = await controls.submit(controls.edit.set(delay.address, 64)); + assert.equal(report.ok, true); + assert.equal(report.path, "rebuild", "the core prepares a replacement plan for a rebuild-only row"); + assert.equal(engine.controlRevision(), before + 1n, "the rebuild is committed"); } finally { engine.dispose(); } @@ -684,10 +682,18 @@ describe("#1564A gate 4 -- edit.set reaches any row", () => { assert.throws(() => edit.set({ ...address, control: "no_such_row" }, 0), MisoUsageError); assert.throws(() => edit.set({ ...address, owner: { kind: "strip", id: 7 } }, 0), MisoUsageError); assert.throws(() => edit.set(address, "loud"), MisoUsageError); - // A named row of an insert depends on the effect it runs, which a document-text boot does not - // tell the SDK: the message says to use the numeric row. + // A named row of an insert depends on the effect it runs. The SDK reads that from the core's + // section header (#1562b-K), so a document-text boot resolves it; an insert the list does not + // have cannot be resolved, and the message says to use the numeric row. + const named = edit.set({ + owner: strip("t1"), section: { kind: "inserts", effect: "comp" }, control: "threshold_db", lane: "both", + }, -20); + const compressorRow = find(await engine.controls({ pageLimit: 50 }), (e) => owned(e, "strip", "t1") + && e.address.section.kind === "inserts" && e.address.section.effect === "comp" && e.name === "threshold_db", + "the comp insert's threshold_db"); + assert.equal(named.edits[0].address.control, compressorRow.address.control); assert.throws( - () => edit.set({ owner: strip("t1"), section: { kind: "inserts", effect: "comp" }, control: "threshold_db", lane: "both" }, -20), + () => edit.set({ owner: strip("t1"), section: { kind: "inserts", effect: "ghost" }, control: "threshold_db", lane: "both" }, -20), /numeric/, ); assert.equal(edit.set(address, 0, { rampSamples: 0 }).edits[0].rampSamples, 0); diff --git a/sdk/test/control-support.mjs b/sdk/test/control-support.mjs index b4b7a8e05..7e9ceb207 100644 --- a/sdk/test/control-support.mjs +++ b/sdk/test/control-support.mjs @@ -142,3 +142,25 @@ export function onEndpointIds(allocator, send) { return reply; }; } + +const STRUCTURAL_FIXTURE = JSON.parse(readFileSync( + new URL("../../hosts/host-web/tests/fixtures/structural-frames.json", import.meta.url), + "utf8", +)); + +/** + * The structural command frames the Rust codec wrote beside the parity session + * (`hosts/host-web/tests/structural_frames_fixture.rs`): `replace_last_strip`, `add_first_strip`, + * `swap_comp_for_soft_clip`, `remove_comp`, `remove_eq` and `swap_t3_multi_and_limiter`, each applied to the booted session. The returned + * function stamps the frame with the endpoint's next request ID and the committed revision, which + * are the only header words that depend on the engine. + */ +export function structuralFrame(name, engine) { + const row = STRUCTURAL_FIXTURE.frames.find((candidate) => candidate.name === name); + if (row === undefined) throw new Error(`no structural frame ${name}`); + const frame = Uint8Array.from(Buffer.from(row.hex, "hex")); + const view = new DataView(frame.buffer); + view.setBigUint64(24, engine.controlTransactions().requestIds.next(), true); + view.setBigUint64(32, engine.controlRevision(), true); + return frame; +} diff --git a/sdk/test/measurement-evals.mjs b/sdk/test/measurement-evals.mjs index 75e94dabd..65baa2ab3 100644 --- a/sdk/test/measurement-evals.mjs +++ b/sdk/test/measurement-evals.mjs @@ -15,6 +15,7 @@ const meterFrame = () => ({ lossCount: 2, windows: 3, trackCount: 2, + structureEpoch: 1, peaks: new Float32Array([0.1, 0.2, 0.3, 0.4, 0.9, 0.8]), trackGrDb: new Float32Array([1.5, 2]), masterGrDb: null, @@ -25,6 +26,15 @@ const meterFrame = () => ({ submixGrDb: new Float32Array(0), }); +/** Strip orders that never change: one list, structure epoch 1. */ +function fixedOrders(tracks, submixes, epoch = 1) { + const order = { tracks, submixes }; + return { + held: (wanted) => (wanted === epoch ? order : undefined), + of: async (wanted) => (wanted === epoch ? order : undefined), + }; +} + const telemetryFrame = { tag: "miso.telemetry.v1", sequence: 8, @@ -101,7 +111,7 @@ function openBrowser( test("SDK measurement feeds use one shared lease and preserve copied projections", async () => { const host = fakeHost(); - const feeds = createMeasurementFeeds(host, ["snare", "kick"], [], true); + const feeds = createMeasurementFeeds(host, fixedOrders(["snare", "kick"], []), true); const firstUpdates = []; const secondUpdates = []; const stopFirst = await feeds.meters((update) => firstUpdates.push(update)); @@ -160,11 +170,11 @@ const busFrame = () => ({ test("a bus frame projects each submix's meter keyed by ID in canonical order", async () => { const host = fakeHost(); - const feeds = createMeasurementFeeds(host, ["snare", "kick"], ["aaa-bus", "zzz-bus"], true); + const feeds = createMeasurementFeeds(host, fixedOrders(["snare", "kick"], ["aaa-bus", "zzz-bus"]), true); const updates = []; const stop = await feeds.meters((update) => updates.push(update)); const frame = busFrame(); - assert.equal(Object.keys(frame).length, 15, "the shipped 15-field miso.meter.v1 message"); + assert.equal(Object.keys(frame).length, 16, "the shipped 16-field miso.meter.v1 message"); host.emitMeter(frame); assert.equal(updates.length, 1); const [update] = updates; @@ -208,9 +218,10 @@ test("the browser engine meters and observes a bus by its submix ID", async () = Object.assign(host, { // The strip order the SDK reads from the core's list: the track, then the bus. control: scriptedControl({ strips: { tracks: ["t"], submixes: ["bus"] } }), + onControlEvent() { return () => {}; }, async observationMap() { return { - tag: "miso.observationmap.v1", result: 0, + tag: "miso.observationmap.v1", result: 0, structureEpoch: 1, bindings: [ { trackIndex: 0, rack: 1, effectIndex: 0, effectSlotId: "comp", nativeEffectId: "miso.compressor", tapIds: [1] }, @@ -249,6 +260,8 @@ test("the browser engine meters and observes a bus by its submix ID", async () = trackGrDb: new Float32Array([1]), submixCount: 1, submixPeaks: new Float32Array([0.3, 0.4]), submixGrDb: new Float32Array([2.5]), }); + // The first frame names its strips by a list the engine reads once, so it arrives a tick later. + await new Promise((resolve) => setTimeout(resolve, 0)); assert.equal(updates.length, 1); assert.deepEqual([...updates[0].submixes.keys()], ["bus"]); assert.equal(updates[0].submixes.get("bus").gainReductionDb, 2.5); @@ -257,9 +270,105 @@ test("the browser engine meters and observes a bus by its submix ID", async () = } }); +// Issue #1562b: a rebuild that adds, removes or reorders a strip changes the structure epoch the +// frames carry. A frame is named by the list of its own epoch. +function epochOrders(lists) { + const reads = []; + return { + reads, + held: (epoch) => (reads.includes(epoch) ? lists[epoch] : undefined), + of: async (epoch) => { + await new Promise((resolve) => setTimeout(resolve, 0)); + reads.push(epoch); + return lists[epoch]; + }, + }; +} + +const frameOf = (epoch, trackCount) => ({ + ...meterFrame(), + structureEpoch: epoch, + trackCount, + peaks: new Float32Array(trackCount * 2 + 2).fill(0.5), + trackGrDb: new Float32Array(trackCount), +}); + +/** Wait until `done()` holds, polling each macrotask, for at most `ms` (a loaded machine must not flake). */ +async function settle(done, ms = 2000) { + const deadline = Date.now() + ms; + while (!done() && Date.now() < deadline) await new Promise((resolve) => setTimeout(resolve, 1)); +} + +test("a meter frame of a new structure epoch is named by the list of that epoch, in order", async () => { + const host = fakeHost(); + const orders = epochOrders({ + 1: { tracks: ["kick", "snare"], submixes: [] }, + 2: { tracks: ["added", "kick", "snare"], submixes: [] }, + }); + const feeds = createMeasurementFeeds(host, orders, true); + const seen = []; + await feeds.meters((update) => seen.push([...update.tracks.keys()])); + host.emitMeter(frameOf(1, 2)); + // The next two arrive while the first is still waiting for its list, and keep their order. + host.emitMeter(frameOf(2, 3)); + host.emitMeter(frameOf(1, 2)); + await settle(() => seen.length >= 3); + assert.deepEqual(seen, [["kick", "snare"], ["added", "kick", "snare"], ["kick", "snare"]]); + assert.deepEqual(orders.reads, [1, 2], "each list is read once, however many frames name it"); + // Held lists are named at once, with no read. + host.emitMeter(frameOf(2, 3)); + assert.equal(seen.length, 4); + assert.deepEqual(orders.reads, [1, 2]); + feeds.close(); +}); + +test("a meter frame whose list the page cannot name is dropped, counted, and the stream goes on", async () => { + const host = fakeHost(); + const orders = epochOrders({ 2: { tracks: ["kick", "snare"], submixes: [] } }); + const feeds = createMeasurementFeeds(host, orders, true); + const seen = []; + await feeds.meters((update) => seen.push(update)); + host.emitMeter({ ...frameOf(2, 2), sequence: 1, lossCount: 3 }); + host.emitMeter({ ...frameOf(1, 2), sequence: 2, windows: 1 }); + host.emitMeter({ ...frameOf(1, 2), sequence: 3, windows: 2 }); + host.emitMeter({ ...frameOf(2, 2), sequence: 4, lossCount: 3 }); + host.emitMeter({ ...frameOf(2, 2), sequence: 5, lossCount: 3 }); + await settle(() => seen.length >= 3); + assert.deepEqual(seen.map((update) => update.sequence), [1n, 4n, 5n], + "epoch 1 has no list: its frames are not named by another's"); + // A gap in the sequence is never silent: the three windows the page dropped join the host's own + // three, and the first update after the drop carries the loss bit once. + assert.deepEqual(seen.map((update) => update.lossCount), [3, 6, 6]); + assert.deepEqual(seen.map((update) => (update.validity & 4) !== 0), [false, true, false]); + feeds.close(); +}); + +test("the browser engine follows a rebuild's new strip order in its meter frames", async () => { + const host = fakeHost(); + const strips = { tracks: ["t"], submixes: [], structureEpoch: 1n }; + Object.assign(host, { control: scriptedControl({ strips }) }); + const engine = await openBrowser(host); + try { + const updates = []; + await engine.subscribeMeters((update) => updates.push([...update.tracks.keys()])); + host.emitMeter(frameOf(1, 1)); + await new Promise((resolve) => setTimeout(resolve, 0)); + // A rebuild adds a track in front: the list moves at the commit, and the plan that still + // renders keeps naming its strips by the list it carries, until the adoption block. + strips.tracks = ["a", "t"]; + strips.structureEpoch = 2n; + host.emitMeter(frameOf(1, 1)); + host.emitMeter(frameOf(2, 2)); + await new Promise((resolve) => setTimeout(resolve, 10)); + assert.deepEqual(updates, [["t"], ["t"], ["a", "t"]]); + } finally { + await engine.close(); + } +}); + test("throwing one SDK listener does not suppress another listener", async () => { const host = fakeHost(); - const feeds = createMeasurementFeeds(host, ["track"], [], true); + const feeds = createMeasurementFeeds(host, fixedOrders(["track"], []), true); let delivered = 0; await feeds.meters(() => { throw new Error("consumer callback"); }); await feeds.meters(() => { delivered += 1; }); @@ -270,7 +379,7 @@ test("throwing one SDK listener does not suppress another listener", async () => test("identical listener functions retain independent subscriptions", async () => { const host = fakeHost(); - const feeds = createMeasurementFeeds(host, ["track"], [], true); + const feeds = createMeasurementFeeds(host, fixedOrders(["track"], []), true); let delivered = 0; const listener = () => { delivered += 1; }; const stopFirst = await feeds.meters(listener); @@ -286,7 +395,7 @@ test("identical listener functions retain independent subscriptions", async () = test("telemetry preserves every host measurement and shares its lease", async () => { const host = fakeHost(); - const feeds = createMeasurementFeeds(host, ["track"], [], true); + const feeds = createMeasurementFeeds(host, fixedOrders(["track"], []), true); const updates = []; const stop = await feeds.telemetry((update) => updates.push(update)); host.emitTelemetry(telemetryFrame); @@ -300,7 +409,7 @@ test("telemetry preserves every host measurement and shares its lease", async () test("a lease refusal is typed and does not reserve a later subscriber", async () => { const host = fakeHost(); host.meterResult = 7; - const feeds = createMeasurementFeeds(host, ["track"], [], true); + const feeds = createMeasurementFeeds(host, fixedOrders(["track"], []), true); await assert.rejects( feeds.meters(() => undefined), (error) => error instanceof MisoEngineError && error.code === "unsupported" && error.result === 7, @@ -324,7 +433,7 @@ test("a rejected numeric lease result is typed and recoverable", async () => { } return originalMeters(request); }; - const feeds = createMeasurementFeeds(host, ["track"], [], true); + const feeds = createMeasurementFeeds(host, fixedOrders(["track"], []), true); await assert.rejects( feeds.meters(() => undefined), (error) => error instanceof MisoEngineError && error.code === "backpressure" && error.result === 6, @@ -337,7 +446,7 @@ test("a rejected numeric lease result is typed and recoverable", async () => { }); test("measurement admission without live controls is a typed SDK refusal", async () => { - const feeds = createMeasurementFeeds(fakeHost(), ["track"], [], false); + const feeds = createMeasurementFeeds(fakeHost(), fixedOrders(["track"], []), false); await assert.rejects(feeds.meters(() => undefined), (error) => error instanceof MisoUsageError); await assert.rejects(feeds.telemetry(() => undefined), (error) => error instanceof MisoUsageError); feeds.close(); @@ -345,7 +454,7 @@ test("measurement admission without live controls is a typed SDK refusal", async test("closing an already-armed feed rejects same-turn admission", async () => { const host = fakeHost(); - const feeds = createMeasurementFeeds(host, ["track"], [], true); + const feeds = createMeasurementFeeds(host, fixedOrders(["track"], []), true); await feeds.meters(() => undefined); const admission = feeds.meters(() => undefined); feeds.close(); @@ -403,9 +512,10 @@ test("browser live controls retain the managed observation conflict hook", async admitted: request.commands.length, appliedAtSample: 0n, }; }, + onControlEvent() { return () => {}; }, async observationMap() { return { - tag: "miso.observationmap.v1", result: 0, + tag: "miso.observationmap.v1", result: 0, structureEpoch: 1, bindings: [{ trackIndex: 0, rack: 1, effectIndex: 0, effectSlotId: "comp", nativeEffectId: "miso.compressor", tapIds: [1] }], }; @@ -442,9 +552,65 @@ test("browser live controls retain the managed observation conflict hook", async }); +test("the browser engine sends an observation record only for the map of the list's epoch (batch 12b M1)", async () => { + // A rebuild is committed: the control list is already epoch 2 (comp moved from effect 1 to 0), + // and the plan that renders still reports epoch 1 and comp at effect 1 until a block adopts the + // replacement. A record built from that map would name effect 1 of the newest plan. + const host = fakeHost(); + const strips = { tracks: ["t"], submixes: [], structureEpoch: 1n }; + let rendered = 0n; + const commands = []; + Object.assign(host, { + control: scriptedControl({ strips }), + async sessionMap() { + return { tag: "miso.sessionmap.v1", result: 0, sources: [], metersAttached: true }; + }, + async status() { rendered += 1n; return { renderedQuanta: rendered }; }, + async command(request) { + commands.push(request.commands); + return { + tag: "miso.ack.v1", result: 0, reason: 0, rejectedIndex: 0, + admitted: request.commands.length, appliedAtSample: 0n, + }; + }, + onControlEvent() { return () => {}; }, + async observationMap() { + const adopted = rendered >= 3n; + return { + tag: "miso.observationmap.v1", result: 0, structureEpoch: adopted ? 2 : 1, + bindings: [{ trackIndex: 0, rack: 1, effectIndex: adopted ? 0 : 1, effectSlotId: "comp", + nativeEffectId: "miso.compressor", tapIds: [1] }], + }; + }, + async readObservations(request) { + return { + tag: "miso.observation.v1", result: 0, + rows: request.selections.map((selection) => ({ ...selection, status: 2, + sampleRateHz: 48_000, firstSample: 0n, endSample: 0n, sequence: 0n, blocks: 0, + leftPresent: 0, rightPresent: 0, left: 0, right: 0 })), + }; + }, + }); + const engine = await openBrowser(host, { liveControls: { meterBlocks: 1, observationTaps: 1 } }); + try { + await engine.liveControls(); + strips.structureEpoch = 2n; + const managed = await engine.subscribeObservations({ + selections: [{ trackId: "t", rack: "inserts", effectSlotId: "comp", tapId: 1, channels: "both" }], + windowBlocks: 1, + }); + assert.ok(rendered >= 3n, "the subscribe waited for the plan of the newest list to render"); + assert.equal(commands.length, 1, "one record batch, sent after the wait"); + assert.equal(commands[0][0].effectIndex, 0, "the record names the effect of the newest plan"); + await managed.close(); + } finally { + await engine.close(); + } +}); + test("telemetry shares one lease until its last subscription and isolates throwing callbacks", async () => { const host = fakeHost(); - const feeds = createMeasurementFeeds(host, [], [], true); + const feeds = createMeasurementFeeds(host, fixedOrders([], []), true); let deliveries = 0; const first = await feeds.telemetry(() => { throw new Error("consumer"); }); const second = await feeds.telemetry(() => { deliveries += 1; }); diff --git a/sdk/test/observation-subscription-evals.mjs b/sdk/test/observation-subscription-evals.mjs index 3ef9e72f0..96842cef5 100644 --- a/sdk/test/observation-subscription-evals.mjs +++ b/sdk/test/observation-subscription-evals.mjs @@ -75,6 +75,7 @@ function injectedOwner() { }], }; const liveControls = { + async syncStructure() {}, edit: { strip: () => ({ effect: () => ({ @@ -508,9 +509,10 @@ describe("issue 1345 -- the browser engine waits for a render, within a bound", slotFull = true; return ack(0, 0, request.commands.length); }, + onControlEvent() { return () => {}; }, async observationMap() { return { - tag: "miso.observationmap.v1", requestId: 1, result: 0, + tag: "miso.observationmap.v1", requestId: 1, result: 0, structureEpoch: 1, bindings: [{ trackIndex: 0, rack: 1, effectIndex: 0, effectSlotId: "comp", nativeEffectId: "miso.compressor", tapIds: [1], diff --git a/sdk/test/response-subscription-evals.mjs b/sdk/test/response-subscription-evals.mjs index 99949b79b..5e85e4e3c 100644 --- a/sdk/test/response-subscription-evals.mjs +++ b/sdk/test/response-subscription-evals.mjs @@ -161,6 +161,7 @@ test("resident and response handles share one owner poll, timer, and epoch", asy }], }; const observationLiveControls = { + async syncStructure() {}, edit: { strip: () => ({ effect: () => ({ observe: (_tap, on) => ({ kind: on ? "observeSubscribe" : "observeUnsubscribe" }) }), diff --git a/sdk/test/spectrum-browser-evals.mjs b/sdk/test/spectrum-browser-evals.mjs index 586188ee5..eb871ee96 100644 --- a/sdk/test/spectrum-browser-evals.mjs +++ b/sdk/test/spectrum-browser-evals.mjs @@ -163,6 +163,7 @@ function encodedSpectrumResult(buffer) { function hostWithCapture(overrides = {}) { return { + onControlEvent() { return () => {}; }, async armSpectrum() { return { result: 0 }; }, async readSpectrum() { return { result: 0, snapshot: new Uint8Array([1]) }; }, async status() { return { result: 0, state: 2, lastResult: 0 }; }, @@ -351,9 +352,11 @@ test("browser spectrum invalidates a dead Worker and permits a fresh managed lif async sessionMap() { return { sources: [], metersAttached: false }; }, + onControlEvent() { return () => {}; }, async observationMap() { return { result: 0, + structureEpoch: 1, bindings: [{ trackIndex: 0, rack: 1, effectIndex: 0, effectSlotId: "comp", nativeEffectId: "miso.compressor", tapIds: [1], diff --git a/sdk/test/spectrum-evals.mjs b/sdk/test/spectrum-evals.mjs index 6b5051d84..030c99bf4 100644 --- a/sdk/test/spectrum-evals.mjs +++ b/sdk/test/spectrum-evals.mjs @@ -629,6 +629,7 @@ test("managed spectrum anchors asynchronous reads to native cadence", async (t) } : {}), })), liveControls: () => ({ + async syncStructure() {}, edit: { strip: () => ({ effect: () => ({ observe: (_tap, armed) => ({ kind: armed ? "observeSubscribe" : "observeUnsubscribe" }) }), diff --git a/sdk/test/structure-evals.mjs b/sdk/test/structure-evals.mjs new file mode 100644 index 000000000..d3f13c093 --- /dev/null +++ b/sdk/test/structure-evals.mjs @@ -0,0 +1,604 @@ +/** + * #1562b-K: the SDK follows a rebuild. + * + * The browser accepts a rebuild like the C ABI does, so the strips, the effect an insert runs and + * the taps a subscription holds can all change under a running engine. The SDK maps and reports; the + * core decides. These evals send the structural frames the Rust codec wrote (`control-support.mjs`) + * to a real core, on the headless engine and through the browser transport over the same core, and + * hold what the SDK names -- the strips of a meter frame, the row of a named effect control, the + * index an observation record carries, the end of a subscription -- to what the core now says. + */ + +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import { before, describe, test } from "node:test"; + +import { createBrowserLiveControls } from "../src/browser/live-controls.ts"; +import { MisoEngineAsset } from "../src/core/asset.ts"; +import { decodeControlReply } from "../src/core/control-frame.ts"; +import { ObservationSubscriptionOwner } from "../src/core/observation-subscriptions.ts"; +import { decodeObservationEnded } from "../src/core/observation-ended.ts"; +import { MisoUsageError } from "../src/core/errors.ts"; +import { createOfflineEngine } from "../src/headless/engine.ts"; +import { structuralFrame } from "./control-support.mjs"; +import { engineHost, moduleBytes, ramp } from "./support.mjs"; + +const FIXTURE = JSON.parse(readFileSync( + new URL("../../hosts/host-web/tests/fixtures/control-list.json", import.meta.url), + "utf8", +)); +const OPTIONS = { liveControls: { meterBlocks: 1, observationTaps: 8 } }; +const STRIP = (id) => ({ kind: "strip", id }); + +let asset; + +before(async () => { + asset = await MisoEngineAsset.load(await moduleBytes()); +}); + +const boot = () => createOfflineEngine(FIXTURE.session, { asset, ...OPTIONS }); + +function feed(engine, block) { + const shape = engine.shape(); + for (const source of shape.sources) { + engine.submitSource({ + sourceId: source.id, + generation: 1n, + startFrame: BigInt(block * shape.quantumFrames), + planes: Array.from({ length: source.channels }, (_unused, channel) => + ramp(shape.quantumFrames, 5 + block * 16 + channel)), + endOfRegion: false, + }); + } +} + +/** Blocks from `from` to `from + count`, each fed and rendered; every meter frame completed on the way. */ +function run(engine, from, count) { + const frames = []; + for (let block = from; block < from + count; block += 1) { + feed(engine, block); + engine.render(); + const frame = engine.pollMeters(); + if (frame !== undefined) frames.push(frame); + } + return frames; +} + +/** Send a structural frame and require the core to take it as a rebuild. */ +function rebuild(engine, name) { + const reply = decodeControlReply(engine.controlFrame(structuralFrame(name, engine))); + assert.equal(reply.status, "ok", `${name} is committed`); + assert.equal(reply.applied.editPath, "rebuild", `${name} is a rebuild`); + return reply.revision; +} + +const row = (list, predicate, what) => { + const found = list.controls.find(predicate); + assert.ok(found !== undefined, `the list has ${what}`); + return found; +}; +const insertRow = (list, strip, insert, name) => row( + list, + (entry) => entry.address.owner.kind === "strip" && entry.address.owner.id === strip + && entry.address.section.kind === "inserts" && entry.address.section.effect === insert + && entry.name === name, + `${strip}/${insert}/${name}`, +); + +const tap = (trackId, rack, effectSlotId, channels = "both") => ({ trackId, rack, effectSlotId, tapId: 1, channels }); + +describe("#1562b-K -- a meter frame names its own strips", () => { + test("a rebuild that adds a strip: frames before name the old strips, frames after the new", async () => { + const engine = await boot(); + try { + assert.equal(engine.meters(true).ok, true); + const before = run(engine, 0, 12); + assert.ok(before.length > 0, "frames complete before the rebuild"); + const epoch = before.at(-1).structureEpoch; + for (const frame of before) { + assert.deepEqual([...frame.trackIds], ["t1", "t2", "t3"]); + assert.equal(frame.structureEpoch, epoch); + } + rebuild(engine, "add_first_strip"); + const after = run(engine, 12, 16); + const last = after.at(-1); + assert.ok(last.structureEpoch > epoch, "the structure epoch moved"); + assert.deepEqual([...last.trackIds], ["a0", "t1", "t2", "t3"]); + assert.equal(last.trackCount, 4); + assert.deepEqual([...last.submixIds], ["b"]); + for (const frame of after) { + // A frame is named by the list of the plan that rendered it, never by a newer list. + assert.deepEqual([...frame.trackIds], frame.structureEpoch === epoch + ? ["t1", "t2", "t3"] + : ["a0", "t1", "t2", "t3"]); + assert.equal(frame.trackIds.length, frame.trackCount); + } + // The muted new strip is silent, so its reading is the first pair, and t1's is the second. + assert.equal(last.peaks[0], 0); + assert.equal(last.peaks[1], 0); + assert.ok(last.peaks[2] > 0 && last.peaks[3] > 0, "t1 is where the list now says"); + assert.equal(last.structureEpoch, (await engine.controls()).structureEpoch); + } finally { + engine.dispose(); + } + }); + + test("a window the old plan completed is named by the old list even after the new list was read", async () => { + const engine = await boot(); + try { + assert.equal(engine.meters(true).ok, true); + const frames = run(engine, 0, 12); + const epoch = frames.at(-1).structureEpoch; + // One block rendered by the plan that is about to be replaced, its window not yet polled. + feed(engine, 12); + engine.render(); + rebuild(engine, "replace_last_strip"); + assert.deepEqual([...engine.shape().tracks], ["t1", "t2", "t4"], "the newest list is read"); + const late = engine.pollMeters(); + assert.equal(late.structureEpoch, epoch); + assert.deepEqual([...late.trackIds], ["t1", "t2", "t3"], "the window is the old plan's, so the old strips'"); + const next = run(engine, 13, 6).at(-1); + assert.deepEqual([...next.trackIds], ["t1", "t2", "t4"]); + } finally { + engine.dispose(); + } + }); + + test("a rebuild that keeps the strip count and changes the strips is named by the new list", async () => { + const engine = await boot(); + try { + assert.equal(engine.meters(true).ok, true); + const before = run(engine, 0, 12); + const epoch = before.at(-1).structureEpoch; + assert.deepEqual([...before.at(-1).trackIds], ["t1", "t2", "t3"]); + rebuild(engine, "replace_last_strip"); + const after = run(engine, 12, 16); + const last = after.at(-1); + assert.equal(last.trackCount, 3, "the count did not change"); + assert.ok(last.structureEpoch > epoch); + assert.deepEqual([...last.trackIds], ["t1", "t2", "t4"]); + for (const frame of after.filter((candidate) => candidate.structureEpoch === epoch)) { + assert.deepEqual([...frame.trackIds], ["t1", "t2", "t3"]); + } + assert.deepEqual([...engine.shape().tracks], ["t1", "t2", "t4"]); + } finally { + engine.dispose(); + } + }); +}); + +describe("#1562b-K -- a named effect control follows the effect the core now lists", () => { + const named = (controls, name, value) => controls.edit.set({ + owner: STRIP("t1"), section: { kind: "inserts", effect: "comp" }, control: name, lane: "both", + }, value); + + for (const path of ["headless", "browser"]) { + test(`${path}: a row named before an insert changed effect lands on the new effect's row`, async () => { + const engine = await boot(); + try { + run(engine, 0, 2); + const controls = path === "headless" + ? engine.liveControls() + : await createBrowserLiveControls(engineHost(engine)); + // The controls are made before the rebuild and keep their view of the strips and effects. + const before = named(controls, "mix", 0.25); + assert.equal(before.edits[0].address.control, 7, "the compressor's mix row"); + assert.equal((await controls.submit(before)).ok, true); + assert.equal(insertRow(await engine.controls(), "t1", "comp", "mix").value, 0.25); + + rebuild(engine, "swap_comp_for_soft_clip"); + // The structural frame did not go through this queue (the SDK has no builder for one yet), + // so the queue takes the committed revision from the core, as after any replaced session. + controls.queue.reset(engine.controlRevision()); + run(engine, 2, 4); + const list = await engine.controls(); + const clipMix = insertRow(list, "t1", "comp", "mix"); + assert.equal(clipMix.address.control, 3, "the soft clip's mix row"); + + // Built before the swap, from the old view: the number is the compressor's, and it is sent + // as the soft clip's. + const stale = named(controls, "mix", 0.5); + assert.equal((await controls.submit(stale)).ok, true); + assert.equal(insertRow(await engine.controls(), "t1", "comp", "mix").value, 0.5); + + // A row the new effect does not have is refused by the SDK, not sent to another row. + await assert.rejects( + () => controls.submit(named(controls, "threshold_db", -20)), + (error) => error instanceof MisoUsageError && /runs miso\.soft-clip now, which has no control 'threshold_db'/.test(error.message), + ); + + // An edit typed for the compressor is not sent to the soft clip. + const typed = controls.edit.track("t1").insert("comp", "miso.compressor").parameter("mix", 0.75); + await assert.rejects( + () => controls.submit(typed), + (error) => error instanceof MisoUsageError && /runs miso\.soft-clip now, and this edit was built for miso\.compressor/.test(error.message), + ); + assert.equal(insertRow(await engine.controls(), "t1", "comp", "mix").value, 0.5, "nothing was sent"); + + // The same insert typed for the effect it runs now goes through. + const typedNow = controls.edit.track("t1").insert("comp", "miso.soft-clip").parameter("mix", 0.75); + assert.equal((await controls.submit(typedNow)).ok, true); + assert.equal(insertRow(await engine.controls(), "t1", "comp", "mix").value, 0.75); + } finally { + engine.dispose(); + } + }); + } + + test("an insert the list never had is the core's to refuse, not the SDK's to resolve", async () => { + const engine = await boot(); + try { + const controls = engine.liveControls(); + assert.throws( + () => controls.edit.set({ owner: STRIP("t1"), section: { kind: "inserts", effect: "ghost" }, control: "mix", lane: "both" }, 1), + /numeric/, + ); + } finally { + engine.dispose(); + } + }); +}); + +/** The OBSERVATION_ENDED events of an engine's reliable lane, decoded. */ +function endedEvents(engine) { + const events = []; + const stop = engine.onControlEvent((event) => { + const ended = event.lane === 0 ? decodeObservationEnded(event.frame) : undefined; + if (ended !== undefined) events.push(ended); + }); + return { events, stop }; +} + +describe("#1562b-K -- a managed observation subscription follows a rebuild", () => { + test("the end of a tap is reported once, with the rest of the subscription kept (headless)", async () => { + const engine = await boot(); + const wire = endedEvents(engine); + try { + const ends = []; + const handle = await engine.subscribeObservations({ + selections: [tap("t1", "inserts", "comp", "left"), tap("t2", "inserts", "gate", "left")], + windowBlocks: 2, + onEnd: (end) => ends.push(end), + }); + run(engine, 0, 4); + await handle.pump(); + assert.deepEqual(handle.readLatest().map((rowRead) => rowRead.status), ["ready", "ready"]); + + const revision = rebuild(engine, "swap_comp_for_soft_clip"); + assert.equal(ends.length, 1, "one report for the commit"); + assert.equal(ends[0].revision, revision); + assert.equal(ends[0].handle, handle); + assert.deepEqual(ends[0].ended.map((selection) => selection.effectSlotId), ["comp"]); + assert.deepEqual(ends[0].remaining.map((selection) => selection.effectSlotId), ["gate"]); + assert.deepEqual(handle.ended.map((selection) => selection.effectSlotId), ["comp"]); + assert.equal(wire.events.length, 1); + assert.equal(wire.events[0].keys.length, 1); + + run(engine, 4, 6); + await handle.pump(); + assert.deepEqual(handle.readLatest().map((rowRead) => [rowRead.effectSlotId, rowRead.status]), [["gate", "ready"]]); + await handle.close(); + assert.equal(engine.readObservations([tap("t2", "inserts", "gate")])[0].status, "unarmed"); + assert.equal(ends.length, 1); + } finally { + wire.stop(); + engine.dispose(); + } + }); + + test("one commit that ends every tap of a subscription closes it and says so once", async () => { + const engine = await boot(); + try { + const ends = []; + const handle = await engine.subscribeObservations({ + selections: [tap("t3", "inserts", "multi"), tap("t3", "inserts", "limiter")], + windowBlocks: 2, + onEnd: (end) => ends.push(end), + }); + run(engine, 0, 4); + const kept = await engine.subscribeObservations({ + selections: [tap("t2", "inserts", "gate")], + windowBlocks: 2, + }); + rebuild(engine, "replace_last_strip"); + assert.equal(ends.length, 1, "the one event of the commit is one report"); + assert.deepEqual(ends[0].ended.map((selection) => selection.effectSlotId), ["multi", "limiter"]); + assert.deepEqual(ends[0].remaining, []); + assert.throws(() => handle.readLatest(), /ended/); + await handle.close(); + assert.equal(kept.ended.length, 0, "a subscription on a surviving tap is untouched"); + run(engine, 4, 6); + await kept.pump(); + assert.equal(kept.readLatest()[0].status, "ready"); + await kept.close(); + } finally { + engine.dispose(); + } + }); + + test("removing the insert ends its tap the same way", async () => { + const engine = await boot(); + try { + const ends = []; + const handle = await engine.subscribeObservations({ + selections: [tap("t1", "inserts", "comp")], + windowBlocks: 2, + onEnd: (end) => ends.push(end), + }); + run(engine, 0, 4); + rebuild(engine, "remove_comp"); + run(engine, 4, 2); + assert.equal(ends.length, 1); + assert.deepEqual(handle.ended.map((selection) => selection.effectSlotId), ["comp"]); + // The subscription is ended; asking for the tap again is the core's unavailable. + await assert.rejects( + () => engine.subscribeObservations({ selections: [tap("t1", "inserts", "comp")], windowBlocks: 2 }), + /unavailable|unsupported/, + ); + } finally { + engine.dispose(); + } + }); + + test("a tap that survives a rebuild keeps publishing and is disarmed at the strip's new index", async () => { + const engine = await boot(); + try { + const handle = await engine.subscribeObservations({ + selections: [tap("t1", "inserts", "comp")], + windowBlocks: 2, + }); + run(engine, 0, 4); + rebuild(engine, "add_first_strip"); + run(engine, 4, 6); + await handle.pump(); + assert.equal(handle.readLatest()[0].status, "ready"); + assert.equal(handle.ended.length, 0); + await handle.close(); + // t1 moved from strip 0 to strip 1: the disarm went to t1, and a0 (strip 0, a copy of t1) + // was never armed. + assert.equal(engine.readObservations([tap("t1", "inserts", "comp")])[0].status, "unarmed"); + assert.equal(engine.readObservations([tap("a0", "inserts", "comp")])[0].status, "unarmed"); + } finally { + engine.dispose(); + } + }); + + test("an effect that moved to another index is disarmed where it is now", async () => { + const engine = await boot(); + try { + const handle = await engine.subscribeObservations({ + selections: [tap("t1", "inserts", "comp")], + windowBlocks: 2, + }); + run(engine, 0, 4); + assert.equal(engine.observationMap().bindings.find((b) => b.effectSlotId === "comp" && b.trackId === "t1").effectIndex, 1); + rebuild(engine, "remove_eq"); + run(engine, 4, 4); + assert.equal(engine.observationMap().bindings.find((b) => b.effectSlotId === "comp" && b.trackId === "t1").effectIndex, 0); + await handle.pump(); + assert.equal(handle.readLatest()[0].status, "ready", "the tap is carried to the moved effect"); + assert.equal(handle.ended.length, 0); + await handle.close(); + run(engine, 8, 2); + assert.equal(engine.readObservations([tap("t1", "inserts", "comp")])[0].status, "unarmed"); + } finally { + engine.dispose(); + } + }); + + test("through the browser transport, whose live controls were made before the rebuild", async () => { + const engine = await boot(); + try { + const host = engineHost(engine); + const controls = await createBrowserLiveControls(host); + const owner = new ObservationSubscriptionOwner({ + observationMap: () => engine.observationMap(), + readObservations: (selections) => engine.readObservations(selections), + liveControls: () => controls, + observationEnded: (listener) => host.onControlEvent((event) => { + const ended = event.lane === 0 ? decodeObservationEnded(event.frame) : undefined; + if (ended !== undefined) listener(ended); + }), + }); + run(engine, 0, 2); + // The strips move before the subscription is made: t1 is strip 1 and the controls still say 0. + rebuild(engine, "add_first_strip"); + run(engine, 2, 4); + const ends = []; + const { handle } = await owner.subscribe({ + selections: [tap("t1", "inserts", "comp"), tap("t2", "inserts", "gate")], + windowBlocks: 2, + onEnd: (end) => ends.push(end), + }); + run(engine, 6, 4); + await handle.pump(); + assert.deepEqual(handle.readLatest().map((rowRead) => rowRead.status), ["ready", "ready"]); + assert.equal(engine.readObservations([tap("a0", "inserts", "comp")])[0].status, "unarmed"); + + rebuild(engine, "swap_comp_for_soft_clip"); + run(engine, 10, 2); + assert.equal(ends.length, 1); + assert.deepEqual(ends[0].ended.map((selection) => selection.effectSlotId), ["comp"]); + await handle.close(); + run(engine, 12, 2); + assert.equal(engine.readObservations([tap("t2", "inserts", "gate")])[0].status, "unarmed"); + owner.invalidate(true); + } finally { + engine.dispose(); + } + }); +}); + +// Batch 12b review, M1. A rebuild's commit moves the control list at once, and the plan that renders +// keeps its own indices until the block that adopts the replacement. An observation record names a +// strip by the newest list and an effect by an index of the map, and the core admits it against the +// newest plan, so a record built in that window can arm another effect's tap and be acked for it. +// `swap_t3_multi_and_limiter` puts two effects with the same tap (`Gain Reduction`, ID 1) at each +// other's index: the record for `multi` built from the rendering plan's map names index 0, which is +// `limiter` in the newest plan. +describe("batch 12b review M1 -- an observation record waits for the map of its epoch", () => { + const armed = (engine, effectSlotId) => engine.readObservations([tap("t3", "inserts", effectSlotId)])[0].status; + const behind = (error) => error.code === "backpressure" && error.diagnosticCode === "sdk.observation.structure_behind"; + + /** An engine whose rebuild is committed and not yet adopted. */ + async function committedNotAdopted() { + const engine = await boot(); + run(engine, 0, 4); + rebuild(engine, "swap_t3_multi_and_limiter"); + return engine; + } + + test("a subscribe in the window is refused, arms nothing, and arms the right tap once a render adopted the plan", async () => { + const engine = await committedNotAdopted(); + try { + await assert.rejects( + () => engine.subscribeObservations({ selections: [tap("t3", "inserts", "multi")], windowBlocks: 2 }), + behind, + "the headless engine renders only when its caller does: the refusal is typed and retryable", + ); + assert.equal(armed(engine, "multi"), "unarmed"); + assert.equal(armed(engine, "limiter"), "unarmed", "the other effect's tap is not armed in its place"); + run(engine, 4, 2); + const handle = await engine.subscribeObservations({ selections: [tap("t3", "inserts", "multi")], windowBlocks: 2 }); + run(engine, 6, 4); + await handle.pump(); + assert.equal(handle.readLatest()[0].status, "ready"); + assert.notEqual(armed(engine, "multi"), "unarmed", "multi's own tap is armed"); + assert.equal(armed(engine, "limiter"), "unarmed", "limiter's tap is not"); + await handle.close(); + } finally { + engine.dispose(); + } + }); + + test("a close in the window is refused and disarms the right tap after the adoption", async () => { + const engine = await boot(); + try { + const handle = await engine.subscribeObservations({ selections: [tap("t3", "inserts", "multi")], windowBlocks: 2 }); + run(engine, 0, 4); + rebuild(engine, "swap_t3_multi_and_limiter"); + await assert.rejects(() => handle.close(), behind); + assert.notEqual(armed(engine, "multi"), "unarmed", "the refused close left the subscription as it was"); + run(engine, 4, 2); + await handle.close(); + run(engine, 6, 2); + assert.equal(armed(engine, "multi"), "unarmed", "multi is disarmed where it is now"); + assert.equal(armed(engine, "limiter"), "unarmed"); + } finally { + engine.dispose(); + } + }); + + test("an update in the window is refused and arms the right tap after the adoption", async () => { + const engine = await boot(); + try { + const handle = await engine.subscribeObservations({ selections: [tap("t3", "inserts", "limiter")], windowBlocks: 2 }); + run(engine, 0, 4); + rebuild(engine, "swap_t3_multi_and_limiter"); + await assert.rejects(() => handle.update({ selections: [tap("t3", "inserts", "multi")], windowBlocks: 2 }), behind); + run(engine, 4, 2); + await handle.update({ selections: [tap("t3", "inserts", "multi")], windowBlocks: 2 }); + run(engine, 6, 4); + await handle.pump(); + assert.equal(handle.readLatest()[0].status, "ready"); + assert.notEqual(armed(engine, "multi"), "unarmed"); + assert.equal(armed(engine, "limiter"), "unarmed", "limiter was released where it is now"); + await handle.close(); + } finally { + engine.dispose(); + } + }); + + /** An owner over the engine whose renders run on a clock of their own: a wait renders one block. */ + function ownerRenderingOnWait(engine, { rendered = true } = {}) { + let block = 20; + const waits = []; + const owner = new ObservationSubscriptionOwner({ + observationMap: () => engine.observationMap(), + readObservations: (selections) => engine.readObservations(selections), + liveControls: () => engine.liveControls(), + renderWait: { + boundMs: 1_000, + mark: async () => engine.status().renderedQuanta, + next: async () => { + waits.push(block); + if (rendered) run(engine, block, 1); + block += 1; + return rendered; + }, + }, + }); + return { owner, waits }; + } + + test("an engine that renders on its own clock waits for a render and sends the record of the adopted plan", async () => { + const engine = await committedNotAdopted(); + const { owner, waits } = ownerRenderingOnWait(engine); + try { + const { handle } = await owner.subscribe({ selections: [tap("t3", "inserts", "multi")], windowBlocks: 2 }); + assert.ok(waits.length >= 1, "it waited for a render"); + assert.notEqual(armed(engine, "multi"), "unarmed", "the tap of multi is armed"); + assert.equal(armed(engine, "limiter"), "unarmed", "the tap of limiter is not"); + await handle.close(); + run(engine, 30, 2); + assert.equal(armed(engine, "multi"), "unarmed"); + } finally { + owner.invalidate(true); + engine.dispose(); + } + }); + + test("a map that does not catch up within the wait is refused, and nothing was sent", async () => { + const engine = await committedNotAdopted(); + const { owner, waits } = ownerRenderingOnWait(engine, { rendered: false }); + try { + await assert.rejects(() => owner.subscribe({ selections: [tap("t3", "inserts", "multi")], windowBlocks: 2 }), behind); + assert.equal(waits.length, 1); + run(engine, 20, 4); + assert.equal(armed(engine, "multi"), "unarmed"); + assert.equal(armed(engine, "limiter"), "unarmed"); + } finally { + owner.invalidate(true); + engine.dispose(); + } + }); +}); + +// Batch 12b review, m3. A deferred `OBSERVATION_ENDED` event of an earlier commit names a tap by its +// stable key only. The tap may have been armed again since (removed at R1, added back, subscribed): +// the event ended the earlier arm, and the new subscription is the core's. +describe("batch 12b review m3 -- an ended event older than the arm is ignored", () => { + test("an event of a commit at or before the arm does not close the subscription; a later one does", async () => { + const engine = await boot(); + let deliver; + const owner = new ObservationSubscriptionOwner({ + observationMap: () => engine.observationMap(), + readObservations: (selections) => engine.readObservations(selections), + liveControls: () => engine.liveControls(), + observationEnded: (listener) => { deliver = listener; return () => {}; }, + }); + try { + run(engine, 0, 2); + const ends = []; + const { handle } = await owner.subscribe({ + selections: [tap("t1", "inserts", "comp")], + windowBlocks: 2, + onEnd: (end) => ends.push(end), + }); + const armedAt = engine.controlRevision(); + const key = { stripId: "t1", rack: "inserts", effectSlotId: "comp", tapId: 1 }; + deliver({ sequence: 1n, revision: armedAt - 1n, keys: [key] }); + deliver({ sequence: 2n, revision: armedAt, keys: [key] }); + assert.equal(ends.length, 0, "the events belong to earlier arms of the tap"); + run(engine, 2, 4); + await handle.pump(); + assert.equal(handle.readLatest()[0].status, "ready", "the tap is still the subscription's"); + deliver({ sequence: 3n, revision: armedAt + 1n, keys: [key] }); + assert.equal(ends.length, 1, "a commit after the arm ended it"); + assert.deepEqual(ends[0].ended.map((selection) => selection.effectSlotId), ["comp"]); + } finally { + owner.invalidate(true); + engine.dispose(); + } + }); +}); diff --git a/sdk/test/support.mjs b/sdk/test/support.mjs index 9dc195f08..15a53942c 100644 --- a/sdk/test/support.mjs +++ b/sdk/test/support.mjs @@ -157,6 +157,7 @@ export function engineHost(engine, { frames = [] } = {}) { // endpoint's one request-ID allocator. const forward = onEndpointIds(engine.controlTransactions().requestIds, (frame) => engine.controlFrame(frame)); return { + onControlEvent(listener) { return engine.onControlEvent(listener); }, async sessionMap() { return { tag: "miso.sessionmap.v1", requestId: (next += 1), result: 0, sources: engine.sessionMap().sources, metersAttached: engine.sessionMap().metersAttached }; }, diff --git a/tools/parameter-metadata/src/abi_layout.rs b/tools/parameter-metadata/src/abi_layout.rs index 66d53d1ae..4ce648186 100644 --- a/tools/parameter-metadata/src/abi_layout.rs +++ b/tools/parameter-metadata/src/abi_layout.rs @@ -468,7 +468,7 @@ fn resource_report_fields() -> [Field; 28] { ] } -fn meter_header_fields() -> [Field; 14] { +fn meter_header_fields() -> [Field; 15] { [ ("structSize", offset_of!(WebMeterHeader, struct_size), "u32"), ("abiVersion", offset_of!(WebMeterHeader, abi_version), "u32"), @@ -508,6 +508,11 @@ fn meter_header_fields() -> [Field; 14] { offset_of!(WebMeterHeader, successor_pointer), "u32", ), + ( + "structureEpoch", + offset_of!(WebMeterHeader, structure_epoch), + "u64", + ), ] } diff --git a/tools/realtime-policy/render-thread-exports.json b/tools/realtime-policy/render-thread-exports.json index e86a726aa..81c2d89d2 100644 --- a/tools/realtime-policy/render-thread-exports.json +++ b/tools/realtime-policy/render-thread-exports.json @@ -11,6 +11,20 @@ {"name": "observation_read", "callers": [{"worklet": "engine", "handler": "receiveObservationRead"}], "wrapped": true}, {"name": "observation_result_ptr", "callers": [{"worklet": "engine", "handler": "receiveObservationRead"}], "wrapped": true}, {"name": "observation_result_bytes", "callers": [{"worklet": "engine", "handler": "receiveObservationRead"}], "wrapped": true}, + {"name": "source_count", "callers": [{"worklet": "engine", "handler": "readSourceRows"}], "wrapped": true}, + {"name": "source_id", "callers": [{"worklet": "engine", "handler": "readSourceRows"}], "wrapped": true}, + {"name": "source_channels", "callers": [{"worklet": "engine", "handler": "readSourceRows"}], "wrapped": true}, + {"name": "source_frames", "callers": [{"worklet": "engine", "handler": "readSourceRows"}], "wrapped": true}, + {"name": "observation_count", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_id_ptr", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_id_capacity", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_effect_slot_id", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_native_effect_id", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_track_index", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_rack", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_effect_index", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_tap_count", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, + {"name": "observation_tap_id", "callers": [{"worklet": "engine", "handler": "readObservationBindings"}], "wrapped": true}, {"name": "track_response_capture", "callers": [{"worklet": "engine", "handler": "receiveTrackResponse"}], "wrapped": true}, {"name": "track_response_result_ptr", "callers": [{"worklet": "engine", "handler": "receiveTrackResponse"}], "wrapped": true}, {"name": "track_response_result_bytes", "callers": [{"worklet": "engine", "handler": "receiveTrackResponse"}], "wrapped": true}, @@ -38,7 +52,7 @@ {"name": "control_response_ptr", "callers": [{"worklet": "engine", "handler": "readControlCommand"}], "wrapped": false, "reason": "single-mode control work (#1562a D1): the address of the last control response, read in the message handler"}, {"name": "control_response_bytes", "callers": [{"worklet": "engine", "handler": "readControlCommand"}], "wrapped": false, "reason": "single-mode control work (#1562a D1): the length of the last control response, read in the message handler"}, {"name": "control_revision", "callers": [{"worklet": "engine", "handler": "readControlCommand"}], "wrapped": false, "reason": "single-mode control work (#1562a, carry-forward 2): the committed revision the SDK's queue seeds from, read in the message handler"}, - {"name": "meter_header_ptr", "callers": [{"worklet": "engine", "handler": "syncMeterViews"}], "wrapped": false, "reason": "single-mode control work (#1290d1): the address of the header the host publishes the meter frame through, read in the message handler to cut the views over a replacement plan's frame, never in process() or a render-locked window"}, + {"name": "meter_header_ptr", "callers": [{"worklet": "engine", "handler": "syncMeterViews"}, {"worklet": "engine", "handler": "renderingStructureEpoch"}], "wrapped": false, "reason": "single-mode control work (#1290d1): the address of the header the host publishes the meter frame through, read in the message handler to cut the views over a replacement plan's frame and (#1562b) to read the rendering plan's structure epoch for the observation map, never in process() or a render-locked window"}, {"name": "control_allocation_count", "callers": [{"worklet": "engine", "handler": "receive"}], "wrapped": false, "reason": "the reader of the control allocation count, which the status reports as singleModeControlAllocations"} ] }