diff --git a/.github/ISSUE_SPECS/1285-keep-every-nodes-latency-from-dropping-during-playback.md b/.github/ISSUE_SPECS/1285-keep-every-nodes-latency-from-dropping-during-playback.md index 1582e5d47..122a57aab 100644 --- a/.github/ISSUE_SPECS/1285-keep-every-nodes-latency-from-dropping-during-playback.md +++ b/.github/ISSUE_SPECS/1285-keep-every-nodes-latency-from-dropping-during-playback.md @@ -196,3 +196,67 @@ Checked against `228fca26c`. browser's parity row on the floored latency. D4 and gate 3 still cover the C ABI only here. - **Anchor.** The `restarted` field of `PlanStateInventory` is `prepare.rs:750` at `228fca26c` (the batch-15 amendment says `:744`). On `main` at `c5b5c88b2` it is `:757`. + +## Decision record: #1285 (batch 15b) + +**What shipped.** +- D1: `GraphBuiltinsCompileRequest::input_floors` (`BTreeMap`, + empty by default). `pdc::timings` takes `max = max(latest incoming arrival, floor)` and + compensates every incoming edge to it. A floor that names no node of the graph is ignored, so the + successor's map can be the predecessor's arrivals as they are ("every node present in both + plans"). `GraphCompileReport::input_arrivals` reports each node's floored `max` where it is + nonzero, sorted by node ID (a missing node has arrival 0, and a 0 floor bounds nothing). +- D2: `PlanStateInventory` holds `input_floors` (from `input_arrivals`, so floored values) and + shows them through `input_floors()`. `retained_bytes` counts them. A preparation with a + `SuccessorBase` passes them as the request's floors. A first plan has none. Across a withdrawn + candidate, `across_withdrawn` clones P0's inventory, so B gets P0's floors (render runs P0). + The discontinuity case (#1323) is not built yet, so every successor floors for now. +- D3: no duck. A restarted strip starts at rest with no duck until #1324 lands, as the batch 15 + amendment says. +- D4: `latency_samples` is the floored output arrival, because PDC's output arrival includes the + floor. No extra code was needed. +- D6: a header paragraph "Latency across a replacement (issue 1285)" and + `docs/C_ABI_V1_QUALIFICATION.md` ยง "Latency floors across a structural transaction (#1285)". + Both say the lower latency comes back with a fresh compile, or at #1323's discontinuity once it + exists. They do not claim a mechanism that is not built. +- Test support: `PlanStateInventory::test_only_floors_only` (feature `test-support`). +- Both hosts run this one core path (control-plane -> host-core). No host adapter changed. + +**Departures.** +- Gates 1 and 2 mute `eq0` in A. A's blocks before the swap then equal B's, so the test can + compare "every block". An audible `eq0` in A differs from B before the swap by design. +- Gate 3's reference is the same `RemoveTrack` transaction committed before the first block. Its + successor is compiled with A's floors and adopts at frame 0 with all state at rest, so it is B + compiled with A's floors and rendered from frame 0. A test-support inventory reference needs + `host-core/test-support` in capi's dev-dependencies, and `crates/capi/Cargo.toml` is not an + authorized path. +- Mechanical neutral field (`input_floors: Default::default()`) in the 8 B15-R9 files and in the + authorized graph-compiler tests. `crates/builtins-compiler` is not touched. +- Fixtures: none regenerated. With no floors the compiled output does not change. + +**Gates.** +- Gate 1: `host-core` `successor_swap::removing_a_limited_track_keeps_the_output_timing_at_every_rate_and_width` + (4 rates, both widths where built). Red when successors get no floors: latency 0 against 447 at + 44.1 kHz. +- Gate 2: `...::removing_a_limited_track_keeps_the_submix_timing_at_every_rate_and_width`. Red + when floors go only to the output: it first differs at the swap block (6). +- Gate 3: `capi` `runtime::tests::removing_a_limited_track_keeps_the_plan_latency`. Red when + successors get no floors: latency 0 against 486. +- Gate 4: `cargo test -p graph-compiler` (fixture corpus) ok; `check-graph-determinism.sh` PASS + 100/100; `cargo test -p console-workload` ok; `check-builtins-fixtures.sh` ok (50 files). +- Gate 5: `...::a_second_removal_keeps_the_first_floors`. Red when the inventory records the + natural `max`: C's latency is 0 against 486. +- Supersession (added): `...::a_replacement_across_a_withdrawn_candidate_keeps_the_running_floors`. + A adds two limiters (latency 972). B across withdrawn A must keep P0's latency (486). Red when + the base takes A's floors (972) or drops them (0). +- Unit: `graph-compiler` `tests::timing_compensates_every_incoming_edge_to_a_floor`. +- Gate 6 command: all suites ok. `check-capi-abi.sh` ok, `check-workspace-policy.sh` ok, + clippy (touched crates) ok, rustdoc ok, `realtime-policy` ok. + +**Measured.** `audit capi`: allocations 0, deallocations 0, pcm_digest `4f52089e4a8cf83c` +(unchanged). No resource ceiling was re-pinned. + +## Amendment (controls coordinator, batch 15b fold, 2026-10-16) + +- The note above that the discontinuity case (#1323) is not built yet is superseded: #1323 shipped + it in the same batch (`SessionState::declare_discontinuity`, both hosts). diff --git a/.github/ISSUE_SPECS/1310-supersede-an-unadopted-candidate-plan-by-compare-and-swap.md b/.github/ISSUE_SPECS/1310-supersede-an-unadopted-candidate-plan-by-compare-and-swap.md index 4a7d50046..cd5c3513e 100644 --- a/.github/ISSUE_SPECS/1310-supersede-an-unadopted-candidate-plan-by-compare-and-swap.md +++ b/.github/ISSUE_SPECS/1310-supersede-an-unadopted-candidate-plan-by-compare-and-swap.md @@ -545,7 +545,7 @@ merged); `plan_swap_race` (gate 5, four tests); `resource_lifecycle` (gate 7); - *Edits outside the original authorized paths.* `hosts/host-web/src/lib.rs` (`promote_spectrum` only), `hosts/host-web/tests/structural_apply_realtime.rs` (the m-5 cost test), `hosts/host-web/tests/control_parity.rs` (the guard's parity row), - `.github/ISSUE_SPECS/1698-carry-an-owner-or-source-a-superseding-candidate-keeps-unchanged-from-p0.md` (one line). No edit under + `.github/ISSUE_SPECS/1698-carry-an-owner-a-superseding-candidate-keeps-unchanged-from-p0.md` (one line). No edit under `crates/engine/src/realtime/`. - *Parity row's silent adoption block (harness, not engine).* A plain run of this session (no edit) first outputs a non-zero sample at frame 459 at 48 kHz and 939 at 96 kHz (the plan's @@ -607,3 +607,14 @@ merged); `plan_swap_race` (gate 5, four tests); `resource_lifecycle` (gate 7); - *Task B-A (doc job).* Five private intra-doc links in `crates/host-core/src/source.rs` (`donate_producers`, `donate_consumers`, `unfed`) are plain code spans; `RUSTDOCFLAGS='-D warnings' cargo doc --locked --workspace --no-deps` passes. + +## Amendment (#1698, batch 15b, coordinator ruling of 2026-10-16 confirmed by root) + +D2's base across a withdrawn candidate is now P0's own inventory less the sources A did not carry +from P0 (`PlanStateInventory::across_withdrawn`), so B carries from P0 an owner it keeps unchanged +from P0, as a direct P0 -> B swap does. `ProviderEpoch::carried_base` and its capi epoch term are +gone (reference `capi_retained_bytes` 168,810 -> 167,514). The fold-3 guard +`drops_acked_fresh_source` stays for all four cases, (a) included: a source A added or redeclared +restarted the host's feed against A (`docs/CONTROL_PROTOCOL_SEMANTICS.md:23`), so B starts it +fresh and cannot read A's chunks into P0's continuing source. The guard's capi and parity rows stay. +See #1698's decision record. diff --git a/.github/ISSUE_SPECS/1318-report-held-source-blocks-apart-from-underruns.md b/.github/ISSUE_SPECS/1318-report-held-source-blocks-apart-from-underruns.md index ecad3f826..8891b1f2c 100644 --- a/.github/ISSUE_SPECS/1318-report-held-source-blocks-apart-from-underruns.md +++ b/.github/ISSUE_SPECS/1318-report-held-source-blocks-apart-from-underruns.md @@ -157,3 +157,43 @@ named command exists. **Realtime-path edits.** None under `crates/engine/src/realtime`. The change is on the source consumer's render path in `crates/source/src/lib.rs`. + +## Decision record: #1318 (batch 15b) + +**Shipped.** D1-D4 and D6 in `crates/source/src/lib.rs` and the one header clause. + +- `begin_block_play`'s missing-PCM branch reads `held_seek` after the acquire step (the batch 15 + precision of D1). Held: the missing frames go to new `held_frames` / `held_blocks` counters and + `underrun_*` stays untouched. Otherwise it is an underrun as before. `next_frame` still advances by + one quantum in both cases, so no rendered bit moves (D2). +- `SourceReadReport` gains `held_frames`, `held`, `cumulative_held_frames`, `cumulative_held_blocks`; + `SourceConsumerTelemetry` gains `held_frames`, `held_blocks` (D3). Held frames use the same + region-bounded count as underrun frames (a held block past the end is neither, unchanged). +- The graph driver needed no change: it folds only `report.underrun_event` and the vacant entry, so a + held block sets nothing (D4). `GraphObservationValidity` and `copy_after_disarm_telemetry` are + untouched (D5). The browser spectrum record and the C ABI read nothing new, so both hosts behave the + same: a held block stops marking spectrum windows as starved on both. The held counters reach hosts + with #1316 (C ABI) and #1537 (browser status), which are the non-goals here. +- Header: "and those blocks count as source underruns" deleted (`miso_engine_v1.h`, lines ~250-253, + rewrapped at 100 columns). #1317 owns the full replacement text. +- D6 facade doc (`crates/host-core/src/source.rs` `seek_at`) left to #1316 as the spec says. + +**Departures.** None. The B15-R/B15-Q rulings the task text names are not in this spec file. + +**Gates -> tests** (all in `crates/source/src/lib.rs`): + +1. `a_wait_for_an_anchor_is_held_and_starvation_after_it_is_an_underrun` (both the delivered and the + undelivered case). +2. `queued_pcm_of_the_playing_generation_plays_while_a_seek_is_held`. +3. `graph_driver_does_not_flag_a_held_block_as_an_underrun` (held source plus a starved second source + in one block). +4. Updated, not deleted: `a_primed_block_waits_for_its_anchor` and + `a_seek_whose_pcm_arrives_inside_the_block_window_keeps_its_pcm` now assert held, not underrun. + +**Red evidence** (temporary mutations, restored): classification forced to never-held turns five tests +red (the three new ones plus the two updated); the driver also OR-ing `report.held` turns only gate 3 +red; deciding played-vs-missing by `held_seek` alone turns gate 2 and +`prepare_seek_holds_an_anchored_seek` red. + +**Measured numbers.** None re-pinned. `TransferBlock` is unchanged; the consumer gains two `u64` +fields, with no resource-report row that counts them. diff --git a/.github/ISSUE_SPECS/1319-test-held-seeks-across-swaps-and-supersession-and-add-a-seek-to-audit-capi.md b/.github/ISSUE_SPECS/1319-test-held-seeks-across-swaps-and-supersession-and-add-a-seek-to-audit-capi.md index 405ce85f7..b5ae34426 100644 --- a/.github/ISSUE_SPECS/1319-test-held-seeks-across-swaps-and-supersession-and-add-a-seek-to-audit-capi.md +++ b/.github/ISSUE_SPECS/1319-test-held-seeks-across-swaps-and-supersession-and-add-a-seek-to-audit-capi.md @@ -207,3 +207,56 @@ audited render, so a defect on that path shows up as an audit violation, not as - **Batch placement.** "#1310 ... is not in the batch-15/16 list" and "#1344, also open and not in the list" (Dependencies on GitHub) are void. #1344 and #1310 are in batch 15a (with #1398), which lands before this slice. + +## Decision record: #1319 (batch 15b) + +**Shipped.** D1-D4 whole, in the two authorized paths. + +- `crates/capi/src/runtime/tests.rs`: `a_held_anchored_seek_crosses_a_plan_swap` (D1), + `a_queued_seek_on_a_persisting_source_survives_supersession` (D2) and + `a_held_seek_carries_from_a_taken_candidate` (D3), with helpers `submit_block_c`, `transact_c` + (returns the response's `EditPath`), `add_source_and_track`, `put_compressor` and + `render_fresh_reference`. +- `tools/audit/src/capi.rs`: right after the structural transaction (outside every render scope) + the audit calls `miso_engine_v1_source_seek_at(session, "fixture-source", 2, 512, 512)` and + submits one generation-2 quantum from frame 512; `submit_source_chunk` takes a generation. The + carried consumer pops the seek at the swap call (call 1), holds it for calls 2-3 and applies it at + call 4. The existing assertions are unchanged. + +**Measured.** `audit capi`: allocations 0, all violation counters 0, `replacements == 1`, +`(carried, carry_mismatches) == (1, 0)`. `pcm_digest` moved `bf7dcedd1d92621f` -> `4f52089e4a8cf83c` +(call 4 now plays the generation-2 quantum instead of an underrun's silence; CI checks the digest +for format only). The JSON schema is unchanged; no resource row exists in this audit. + +**Gates.** `cargo test --locked -p capi`: 151 lib tests and every integration test pass, among them +the three new tests. `cargo test --locked --release -p audit`: 33 pass. Clippy on `capi` and `audit` +(`--all-targets --all-features -D warnings`), `cargo fmt --all -- --check`, `RUSTDOCFLAGS='-D +warnings' cargo doc --locked --workspace --no-deps` and `scripts/check-workspace-policy.sh` pass. +The C header is untouched. + +**Departures from the text.** + +- D1 adds a muted track on the fixture source (`add_muted_track`), not a new track and source: + the spec's "keeps the source" is what the test needs, and the reference then compares with exact + zeros from the extra strip. +- D2 and D3 use inserts and a muted track that leave the reference comparable: D2 compares with + the twin, as the spec says (a fresh session's compressor state would differ from a swapped-in + one's); D3's T2 is a muted track, so its reference is a fresh session of T2's document. +- Both D2/D3 assert `EditPath::Rebuild` in each transaction's response (batch 15 check). +- The held counters (#1318) are consumer telemetry that the C ABI tests cannot reach (no + accessor from a plan or session), so the assertions are bit-identity and nonzero-reference, which + an unapplied or dropped seek breaks (the generation-2 blocks are discarded and the source is + silent from the anchor). +- D2 never feeds fresh sources, so the batch 15a guard (`drops_acked_fresh_source`) does not + arise: the fixture source persists from P0 and is never fresh in A. D3 withdraws nothing. + +**Red evidence (argued, not run: sprint mode forbids mutation runs).** D1: a carry that resets or +rebuilds `held_seek` leaves the successor with no seek at block 8, so the queued generation-2 +block is discarded as unobserved and blocks 8-15 are silent against a playing reference. D2: a +supersession that rebuilds the persisting ring (or clears its command slot) loses the seek or the +queued blocks the same way; the counters pin `(withdrawn, superseded, taken) == (1, 1, 0)`. D3: +preparing B across a taken A makes `s2` fresh in B, so its held seek and queued generation 2 are +lost and `s2` is silent from block 8; the counters pin `(withdrawn, taken) == (0, 0)`. + +**Successor work.** Expose the consumer's held counters through a test accessor (or the counter +snapshot) so a C ABI test can assert `held_blocks` directly. diff --git a/.github/ISSUE_SPECS/1323-reset-latency-floors-at-a-host-declared-discontinuity.md b/.github/ISSUE_SPECS/1323-reset-latency-floors-at-a-host-declared-discontinuity.md index 5feb98b4f..05ed0e263 100644 --- a/.github/ISSUE_SPECS/1323-reset-latency-floors-at-a-host-declared-discontinuity.md +++ b/.github/ISSUE_SPECS/1323-reset-latency-floors-at-a-host-declared-discontinuity.md @@ -332,3 +332,88 @@ through `submit_command` and `control_command`, C13's form with no new export. B - **Batch placement.** "#1310 is open and not in batch 15/16" (Dependencies) is void. #1310, #1344 and #1398 are batch 15a, which lands before this slice (B15-R5). + +## Decision record: #1323 (batch 15b) + +**What shipped.** +- D1: `miso_engine_v1_declare_discontinuity(session)` in `crates/capi/src/ffi.rs`, a forwarder to + `control_plane::SessionState::declare_discontinuity`. `FEATURE_DECLARE_DISCONTINUITY = 1 << 8` + (the next free bit), and the mask ORs it. Header: the define, the prototype, the session thread + list, and a paragraph "Declared discontinuity (issue 1323)" with the host's duty. The #1285 + paragraph now points to it. `scripts/check-capi-abi.sh` lists the symbol (18 frozen). +- B15-R7: `miso_engine_web_v1_declare_discontinuity(handle) -> u32` in `hosts/host-web/src/ffi.rs`, + through `AudioWorkletEngineHost::declare_discontinuity`, which runs the same control-plane + function. The host's `run_command` became `run_control`, so the plan a declaration publishes + keeps its tables as a rebuild's do. The #1293 D7 mirrors are updated and the SDK is regenerated. +- D2/D4 in `crates/control-plane/src/control.rs`. `rebuild` takes `Rebuild::Transaction(token)` + or `Rebuild::Discontinuity(Box)`. The discontinuity arm has no protocol commit + and returns no response. Its copy of the committed model is the kept base of the epoch it + displaces (the direct case), or it is dropped when the arm supersedes a candidate, as a commit's + displaced model is. It is always published `Next`, and the test adoption hook is not taken. + Supersession, the donation, the acked-PCM guard, observation forwarding, the `superseded` word + (`r - 1 - last_running`) and republish-on-refusal are the transaction path's own code. +- D3 in `crates/host-core/src/prepare.rs`. `SuccessorBase::discontinuity`, `SuccessorBase::new` + (with a private static neutral `EditRamps`), and every literal site moved to `new`. At + preparation, a discontinuity base swaps its inventory for `at_discontinuity()`: identity, rate, + quantum and source rows only, with no floors. Every join reads the base's inventory, so that one + substitution is the whole rule. The inventory records `raised_arrivals` (each raised node's + natural arrival, beside its floored one in `input_floors`), and + `runs_behind_natural_timing()` is D2's test. + +**Rule for the supersession case (the spec is silent).** A transaction E that supersedes a +discontinuity successor D before render adopts it is prepared at the same discontinuity (sticky +`ProviderEpoch::discontinuity`). The combined twin P0 -> E is "declare, then edit" as one +transaction against the running plan, and render never played D. So E has no floors and carries +only sources. A declaration while D waits finds nothing to reset (D has no raised floor) and +publishes nothing. D over a withdrawn A carries A's revision. A's revisions complete at D's +adoption, `EXACT`, with `SUPERSEDED` for those A folded in. + +**Departures.** +- `crates/graph-compiler/src/{pdc.rs,compile.rs,lib.rs}` (not authorized): D3 needs natural + arrivals in the inventory, and only PDC knows them. `GraphCompileReport::raised_arrivals` is a + report field only. Program, canonical bytes and fixtures are unchanged, and the unit test is + extended. +- Gate 5 and the supersession gate are in `crates/capi/src/runtime/tests.rs`, not in control-plane + unit tests. Those tests drive `control_plane::SessionState` directly. The control-plane crate has + no render or session-fixture harness, and its `tests/` directory is not authorized. +- Gate 4's error is the replacement-peak cap's `CompileRejected` (set with + `set_replacement_peak_limit`). The C ABI's limits give no per-preparation cap that fails only + the floorless successor. +- Mechanical edits outside the paths: `crates/capi/tests/c/{abi_smoke.c,header_smoke.cpp}` (the + bit inside the mask and the signature, replacing `MASK == 255`), `crates/host-core/tests/strip_meters.rs` + (a `SuccessorBase` site added after the batch check), `ProviderEpoch::test_inventory`, and the capi + test wrappers `test_declare_discontinuity` and `test_plan_latency`. +- `hosts/host-web/tests/control_parity.rs`: a parity row, at the coordinator's request. +- Header: the #1310 paragraph is re-wrapped to 100 columns (one line was 135). +- No browser latency report exists yet (#1688). So the browser gate reads the plan latency through + the host's `queries`, and the parity row compares bits and the C ABI latency. + +**Gates.** +- Gate 1: `host-core` `successor_swap::a_declared_discontinuity_resets_the_floors_at_every_rate_and_width`. + Red when the floors are kept (447 against 0 at 44.1 kHz) and when all state is carried (block 0 + differs). +- Gate 2: `capi` `runtime::tests::a_declared_discontinuity_reports_the_natural_latency_on_the_c_abi`. + Red when nothing is published. Browser: + `host-web` `tests::structural_apply::a_declared_discontinuity_resets_the_floors_on_the_browser`, + red when the host does not forward. Parity: + `control_parity::a_declared_discontinuity_resets_the_floors_alike_on_both_hosts` (48 and + 96 kHz), red at block 8 when the browser does not forward. +- Gate 3: `...::a_declaration_with_nothing_to_reset_publishes_nothing` (revision 0 and after an + added track). Red when the declaration always rebuilds. +- Gate 4: `...::a_refused_declaration_changes_nothing`. Red when the error is swallowed. The retry + under the lifted cap resets the floors. +- Gate 5: `...::a_declared_stop_supersedes_a_pending_warm_successor_alone_and_folded`. Red when + the successor is published `Primed`, and when the pending candidate is left alone. +- Supersession: `...::an_edit_that_supersedes_a_discontinuity_successor_is_prepared_at_it`. Red + without the sticky rule. +- Gate 6: `abi::tests::masks_and_result_codes_are_frozen` (the OR of the symbols) and + `ffi::tests::version_and_capabilities_are_exact`. +- Gate 7 command: all suites pass. `host-web` (test-support) and `graph-compiler` suites pass. + `check-capi-abi.sh` and `--self-test` pass. `check-workspace-policy.sh` passes. `fmt` passes. + Clippy passes on the touched crates. Rustdoc passes. `realtime-policy` passes. + `check-sdk-generated.sh` passes. `check-abi-layout-v1.py` and `--self-test` pass. + `check-web-audioworklet.sh` passes. + +**Measured.** `audit capi`: allocations 0, deallocations 0, syscalls 0, pcm_digest +`4f52089e4a8cf83c` (unchanged; `tools/audit` not touched). C ABI gate 2: latency 486 -> 0 at +48 kHz. No resource ceiling was re-pinned. diff --git a/.github/ISSUE_SPECS/1325-remove-a-strip-in-two-phases-ramp-out-then-a-scheduled-swap.md b/.github/ISSUE_SPECS/1325-remove-a-strip-in-two-phases-ramp-out-then-a-scheduled-swap.md index 68289a699..76992bf64 100644 --- a/.github/ISSUE_SPECS/1325-remove-a-strip-in-two-phases-ramp-out-then-a-scheduled-swap.md +++ b/.github/ISSUE_SPECS/1325-remove-a-strip-in-two-phases-ramp-out-then-a-scheduled-swap.md @@ -268,3 +268,11 @@ two render calls, so the ramp starts at `p`). - *Ramp a route that a plan swap adds to or removes from a surviving strip* (#1363). - *Give every route whose tap precedes its strip's fader a live lane on every plan* (#1391). - *Hold route-lane values in latest-target cells* (#1347). + +## Amendment (controls coordinator, batch 15b fold, 2026-10-16) + +- #1698 (batch 15b) removed `PreparedHost::carried_base`. The text above that reads the candidate's + base inventory as `A.carried_base` (*A restored source keeps its ring*) now reads: B's base across + a withdrawn candidate is the running plan P0's own inventory + (`PlanStateInventory::across_withdrawn`), with its source rows filtered to those that A carried + vacant. The restored source's carry program, the producer check and the adoption are unchanged. diff --git a/.github/ISSUE_SPECS/1344-prepare-a-successor-across-a-withdrawn-candidate-plan.md b/.github/ISSUE_SPECS/1344-prepare-a-successor-across-a-withdrawn-candidate-plan.md index 977765b27..b68a33e89 100644 --- a/.github/ISSUE_SPECS/1344-prepare-a-successor-across-a-withdrawn-candidate-plan.md +++ b/.github/ISSUE_SPECS/1344-prepare-a-successor-across-a-withdrawn-candidate-plan.md @@ -403,3 +403,14 @@ The coordinator added it, as recorded above. all covered; a source the donor carried from P0 is not. Unit test `withdrawn_successor::the_guard_refuses_exactly_a_fed_fresh_ring_the_donation_leaves_behind`; see #1310's fold 3. + +## Amendment (#1698, batch 15b, coordinator ruling of 2026-10-16 confirmed by root) + +D1 and D2 are amended. A replacement B across a withdrawn A is prepared from P0's own inventory +(`PlanStateInventory::across_withdrawn(&A.inventory, A.plan)`), not from A's restricted base, so +an owner B keeps unchanged from P0 carries from P0 even when A restarted or removed it. Source rows +keep only the sources A carries from P0, so the source carry is what this issue shipped. +`PreparedHost::carried_base`, `PlanStateInventory::restricted_to`, `carried_base_retained_bytes`, +the test-support views and `graph::installed_carry_reads` are removed. Gate 2 +(`the_carried_base_holds_exactly_the_rows_the_hand_over_reads`) is deleted with them. D3 (the +donation) is unchanged. See #1698's decision record. diff --git a/.github/ISSUE_SPECS/1397-duck-swap-the-strips-a-latency-growth-restarts-and-fall-back-to-the-transition-w.md b/.github/ISSUE_SPECS/1397-duck-swap-the-strips-a-latency-growth-restarts-and-fall-back-to-the-transition-w.md index 3d3e7c811..c5f9ad440 100644 --- a/.github/ISSUE_SPECS/1397-duck-swap-the-strips-a-latency-growth-restarts-and-fall-back-to-the-transition-w.md +++ b/.github/ISSUE_SPECS/1397-duck-swap-the-strips-a-latency-growth-restarts-and-fall-back-to-the-transition-w.md @@ -323,3 +323,11 @@ warm edit would be `LeadBound`. - `AdmissionPeak::WithReprepare` named above did not ship with #1398 (root ruling: code with no production caller is a placeholder). #1403 adds the reprepare admission peak together with its first caller; see its batch 15a amendment. + +## Amendment (controls coordinator, batch 15b fold, 2026-10-16) + +- #1698 (batch 15b) removed `PreparedHost::carried_base` (the donor's `carried_base`, #1344 D1, + named in the donor base bullet above). With a donor, `inventory` is now P0's own inventory + through `PlanStateInventory::across_withdrawn`, with its source rows filtered to those the + donor carried vacant. `committed` (the running epoch's kept model with #1325 D6's duck overlay) + and the rest of that bullet are unchanged. diff --git a/.github/ISSUE_SPECS/1647-render-moving-stored-fader-automation.md b/.github/ISSUE_SPECS/1647-render-moving-stored-fader-automation.md index 0a937f9fb..b50c86b28 100644 --- a/.github/ISSUE_SPECS/1647-render-moving-stored-fader-automation.md +++ b/.github/ISSUE_SPECS/1647-render-moving-stored-fader-automation.md @@ -251,3 +251,9 @@ three #1638 reviews measured; the mechanism is this slice's design, reviewed wit with `control_smoothing` 0), if D1.2 refreshes `hold_until` inside the hold, or if a grid event cuts the move's ramp in `[E, E + J - 1]`. - **#1648** carries the obligation across a swap (see its amendment). + +## Amendment (controls coordinator, batch 15b fold, 2026-10-16) + +- #1285 landed first (batch 15b). `a(n)` above must therefore be the floored input arrival + (`GraphCompileReport::input_arrivals`, kept in `PlanStateInventory::input_floors()`), not the + arrival the graph compiler computes without floors. diff --git a/.github/ISSUE_SPECS/1688-report-the-plan-latency-on-the-browser.md b/.github/ISSUE_SPECS/1688-report-the-plan-latency-on-the-browser.md index 6aae3f7ff..1ed7dd9ca 100644 --- a/.github/ISSUE_SPECS/1688-report-the-plan-latency-on-the-browser.md +++ b/.github/ISSUE_SPECS/1688-report-the-plan-latency-on-the-browser.md @@ -144,3 +144,56 @@ Half a day. padding. D2's "(56 bytes)" reads "(64 bytes)". The ABI layout mirrors (D3) record 64. - **Decision 17 C13.** The two exports are listed there (root, 2026-10-16) as a transport read of the core's `PlanQueries::resources()` row, so they are not a C13 exception. + +## Decision record: #1688 (batch 15b) + +**Shipped.** + +- D1: `control_plane::tail_words(TailSamples) -> (u64, u64)` with `TAIL_FINITE` (0) and + `TAIL_INFINITE` (1), in `crates/control-plane/src/adapter.rs`, re-exported from `lib.rs`. The + capi adapter's `From` calls it, and capi's `TAIL_FINITE` and `TAIL_INFINITE` are + now defined as the control plane's constants (values unchanged). +- D2: `WebPlanLatency` (64 bytes, per the batch 15a amendment), `PLAN_LATENCY_BYTES`, + `AudioWorkletEngineHost::read_plan_latency` and `plan_latency`, and the exports + `miso_engine_web_v1_plan_latency_read` and `miso_engine_web_v1_plan_latency_ptr`. The read copies + `PlanQueries::resources()` and the `tail_words` encoding; host-web computes nothing. The record + is all zero until the first successful read. +- D3: mirrors in `tools/parameter-metadata/src/abi_layout.rs` (exports 102 -> 104, structure + `planLatency`), `scripts/check-abi-layout-v1.py` (exports, structure, field names, two new + self-test mutations), `scripts/check-web-audioworklet.sh` (export list), + `scripts/fixtures/abi-layout-v1-self-test.json`, `sdk/assets/miso-engine-v1-abi-layout.json` and + `sdk/src/generated/abi.ts` (regenerated by `sdk/codegen/assets.mjs` and `generate.mjs`). No + change to `scripts/check-sdk-generated.sh` or the mock's export list was needed. +- D4: `control_parity.rs` compares the browser record with `miso_engine_v1_plan_resources` before + and after every block of the scripted run, at each launch rate (this covers the boot plan, every + plain successor of the script's rebuilds and the end). + +**Departures.** + +- The spec says a strip's `delay_samples` moves the plan latency. It does not: a track delay + shifts the output and leaves the PDC report alone (`graph-compiler` `track_delay.rs` P2-3). The + moving rebuild is four added true-peak limiter inserts on `t1` (the added latency makes `t1` the + longest path); `a_rebuild_that_adds_latency_moves_the_plan_latency_alike_on_both_hosts` asserts + the value moved. +- "Before boot completes" has no host object: a handle that names no booted host returns + `RESULT_INVALID_ARGUMENT` from the export. The `WRONG_STATE` and zero-record case is a host + whose state is not ready (the native test sets `STATE_FAILED`). +- Mechanical: `crates/capi/src/runtime/tests.rs` calls `tail_words` in two places and spells + `crate::TAIL_INFINITE` in one. + +**Gates.** + +- Gate 1: `control_parity`: `both_hosts_give_the_same_result_for_the_same_control_edits` (the + record equals the C ABI report after every block, every launch rate), + `a_rebuild_that_adds_latency_moves_the_plan_latency_alike_on_both_hosts` (pending values stay + until the adopting block, then both move, equal) and + `removing_a_limited_track_keeps_the_plan_latency_alike_on_both_hosts` (#1285 floors: both hosts + report the same, unchanged latency after the swap, and a fresh compile of the committed model + reports less). Result: 17 of 17 pass. +- Gate 2: `tests::structural_apply::the_plan_latency_record_follows_the_active_plan` (host-web + lib): not ready leaves the record zero, boot copy of the core row, old value while the limiter + rebuild waits, equal to a fresh boot after adoption. Passes. +- Gate 3: `check-abi-layout-v1.py --self-test` (32 mutations caught) and the artifact check in + `check-web-audioworklet.sh` pass against a built module, so the module exports both names. +- `check-browser-expected-resources.py` passes with no re-pin (bridge retained bytes 1549094 of + 1667648). diff --git a/.github/ISSUE_SPECS/1698-carry-an-owner-or-source-a-superseding-candidate-keeps-unchanged-from-p0.md b/.github/ISSUE_SPECS/1698-carry-an-owner-a-superseding-candidate-keeps-unchanged-from-p0.md similarity index 55% rename from .github/ISSUE_SPECS/1698-carry-an-owner-or-source-a-superseding-candidate-keeps-unchanged-from-p0.md rename to .github/ISSUE_SPECS/1698-carry-an-owner-a-superseding-candidate-keeps-unchanged-from-p0.md index aefe50fe1..4843bdafd 100644 --- a/.github/ISSUE_SPECS/1698-carry-an-owner-or-source-a-superseding-candidate-keeps-unchanged-from-p0.md +++ b/.github/ISSUE_SPECS/1698-carry-an-owner-a-superseding-candidate-keeps-unchanged-from-p0.md @@ -1,4 +1,4 @@ -# Carry an owner or source a superseding candidate keeps unchanged from P0 from P0's state +# Carry an owner a superseding candidate keeps unchanged from P0 from P0's state Successor of #1344 (prepare a successor across a withdrawn candidate plan) and #1310 (supersede an unadopted candidate plan by compare and swap), both batch 15a. Found in the batch 15a review; root @@ -11,9 +11,12 @@ review. ## Product outcome An edit followed quickly by its undo is not audible. When the running plan P0 has a committed -candidate A that render has not adopted, and a later transaction commits B that puts an owner or a -source X back to its P0 form, B carries X from P0, as a direct P0 -> B swap would. Today X starts -fresh in B whenever A restarted or removed it. +candidate A that render has not adopted, and a later transaction commits B that puts an owner X (a +strip, an effect, a meter, an observation tap or another owner of plan state) back to its P0 form, +B carries X from P0, as a direct P0 -> B swap would. Before this issue X started fresh in B +whenever A restarted or removed it. Sources stay under the host contract (amended by the +coordinator ruling of 2026-10-16, confirmed by root; see the decision record): a source A added, +redeclared or removed starts fresh in B unless B keeps it as A declares it. ## Context @@ -135,6 +138,79 @@ fresh in B whenever A restarted or removed it. Half a day. -## Decision record: #1698 - -To be filled by the implementer. The base decision goes first. +## Decision record: #1698 (batch 15b) + +**Base decision (amends #1344 D1 and D2).** Across a withdrawn candidate A, B's base inventory is +P0's own inventory, the inventory of the epoch A succeeds (`kept_holder().inventory`), through +`PlanStateInventory::across_withdrawn(&A.inventory, A.plan)` (`crates/host-core/src/prepare.rs`). +Every owner row is P0's, so an owner B keeps unchanged from P0 carries from P0 as a direct P0 -> B +swap carries it, also when A restarted or removed it. `committed` stays P0's kept model. The only +filter is on sources: a P0 source row is kept only when A carries that source from P0 (A's plan +holds it vacant, `can_prepare_source_seek` false on A's index). The filter reads A's ring, not A's +model, so a chain stays right: a source a candidate restarted is fresh in it, whatever its +declaration. `PreparedHost::carried_base`, `PlanStateInventory::restricted_to`, the report row +`carried_base_retained_bytes`, `ProviderEpoch::carried_base`, the capi epoch row term, the +test-support views `test_only_inventory_rows` / `test_only_carry_read_keys`, and +`graph::installed_carry_reads` with `GraphCarryReads` (no production caller left) are removed. +The new base is built once per supersession on the control thread, before the commit; it is not +kept. + +**Why sources do not carry from P0.** The host contract, `docs/CONTROL_PROTOCOL_SEMANTICS.md:23`: +"a source whose declaration the transaction changed restarts the same way, in a new ring at +generation 1, frame 0, with the PCM accepted for it before the commit discarded with its plan, so +the host restarts its feed." Submits and seeks address the newest committed session +(`SessionState::submit_unserviced`, `newest_providers_mut`). After A's commit the host feeds A's +X; B declares X differently from A, so a host that follows the contract restarts X at generation +1, frame 0. P0's X producer requires P0's generation and next frame (`validate_submission_metadata` +in `crates/source/src/lib.rs`), so carrying P0's X would refuse that feed (`StaleGeneration` or +`NonContiguous`). The host could follow only through a new signal, which the spec's non-goals +forbid. B reading A's acked chunks (D1 option i) is also wrong: they are A's declaration's +content. So D1 needs no new mechanism: `host_core::drops_acked_fresh_source` stays unchanged for +all four cases (a)-(d), and the spec's line "Lifts the batch 15a guard, reverted case only" does +not hold. The guard's capi and parity rows stay. The producer passes are unchanged: every source B +prepares vacant is one A carried from P0, whose producer A's commit moved into A's set, so no P0 +producer pass is needed (the order stays: donation, `adopt_persisting` from A, the return pass +after publication). + +**Gates (the new list; gate 2 is dropped by the ruling).** +1. Owners carry from P0, bit-identical to the combined twin P0 -> B: + - `crates/host-core/tests/withdrawn_successor.rs` + `an_owner_a_restarts_or_removes_and_b_reverts_carries_from_p0` (A replaces `t1`'s insert, or + removes track `t3`; B reverts it and replaces `t2`'s; Simd8 and Simd4; `restarted_strips()` + is `[t2]`; 16 blocks equal, zero allocator calls per block). + - capi `supersede_tests::an_owner_the_candidate_restarts_and_the_replacement_reverts_carries_from_p0` + and `..._removes_and_the_replacement_restores_carries_from_p0` (9 blocks equal the combined + transaction; build target's lane width only, as every capi test). + - Red on the base before the fix (recorded here): host-core restart case `restarted_strips` + `["t1", "t2"]` / `["t2", "t3"]`, and bits differ at block 0 in both cases at both widths; + capi both cases differ at block 1. +2. Sources stay as before: `a_source_b_reverts_is_fresh_and_a_source_a_carries_still_carries` (A + redeclares or removes `b`, B reverts it: B's `b` owns a fresh ring and takes generation 1 at + frame 0; `a`, carried by A, is vacant in B and its producer moves from A). Red with the source + filter removed (P0's full inventory): the `b` freshness assert. +3. Tap across a restart: host-web `a_subscription_on_a_candidate_that_restarted_the_tap_publishes_ + on_a_replacement_that_reverts_it` (A changes `comp`'s link mode, a subscription is acked + against A, B puts `comp` back; B's tables hold the subscription and windows publish). Red with + `observation_forwards` emptied (B now carries P0's unarmed tap). No spectrum test: a spectrum + capture carries by target and channels only, so A can restart one only by removing its target, + and the browser refuses that preparation (`RESULT_COMPILE_REJECTED`, probe not kept), so the + pairing B gets is P0's in both the old and the new base. +4. Gate 3 (allocations) runs unchanged: every rendered block in the host-core tests above asserts + zero allocator calls. +5. Deleted, superseded: `the_carried_base_holds_exactly_the_rows_the_hand_over_reads` and its + rich-session helpers. Gate 1's and gate 3's `installed_carry_reads` asserts became + `can_prepare_source_seek` asserts on B's index. + +**Measured.** Reference session `capi_retained_bytes` 168,810 -> 167,514 (x86-64, -1,296 = one +216-byte `Option` in each of six provider epochs); the 175,296 ceiling stays. +No ABI change: the row was a control-plane epoch term, not a header field. +Amendment (batch 15b fold): after #1285's `input_floors` and `raised_arrivals` merged the row +measures 167,754 (+240); the ceiling stays. + +**Edits outside the listed paths (authorized by the ruling or mechanical).** +`crates/graph/src/lib.rs` (deletion), `crates/host-core/src/{lib.rs,render_session.rs}`, +`crates/control-plane/src/compile.rs`, `crates/capi/tests/resource_lifecycle.rs` (destructure, +term, re-pin note), `hosts/host-web/src/tests.rs` (the tap test), +`crates/capi/include/miso_engine_v1.h` (comment only: the "one case is not covered yet" sentence +now states the owner carry and the source rule), `docs/CONTROL_PROTOCOL_SEMANTICS.md`, +`docs/C_ABI_V1_QUALIFICATION.md`. No edit under `crates/engine/src/realtime/`. diff --git a/crates/capi/include/miso_engine_v1.h b/crates/capi/include/miso_engine_v1.h index 8c88b46d5..58de827e8 100644 --- a/crates/capi/include/miso_engine_v1.h +++ b/crates/capi/include/miso_engine_v1.h @@ -18,8 +18,9 @@ extern "C" { * session Control thread. At most one thread at a time calls * miso_engine_v1_source_submit_planar_f32, miso_engine_v1_source_seek, * miso_engine_v1_source_seek_at, miso_engine_v1_submit_command, - * miso_engine_v1_dequeue_event, miso_engine_v1_service, or - * miso_engine_v1_last_error on the session; they are serialized with each other. + * miso_engine_v1_dequeue_event, miso_engine_v1_service, + * miso_engine_v1_declare_discontinuity, or miso_engine_v1_last_error on the + * session; they are serialized with each other. * plan Split ownership. * miso_engine_v1_render_f32_planar: render thread only, never concurrently with * itself, and the exclusive owner of the plan's render state. @@ -82,6 +83,35 @@ extern "C" { * replacement may change it, but an unchanged report does not mean that no replacement * happened; the edit path is the signal.) * + * Latency across a replacement (issue 1285). No node's latency, the output's included, drops + * across a structural transaction: the successor plan is compiled with every surviving node's + * input arrival in the plan it displaces as a lower bound, so removing a latent effect, or the + * track that holds it, moves no other path in time, every compensation line keeps its length, and + * latency_samples does not fall. A path whose own upstream latency dropped gets a new + * compensation line, which starts at rest. An added latency still raises latency_samples. The + * lower latency returns only with a fresh miso_engine_v1_compile_session, or at a host-declared + * discontinuity (below); there is no permanent latency reserve. + * + * Declared discontinuity (issue 1323). miso_engine_v1_declare_discontinuity(session) says that + * the next block the host renders need not continue the previous output, because the host has + * stopped or is about to seek every source. The host's duty: call it at a stop, and before a seek + * of every source. If the newest plan (the waiting replacement, else the plan that renders) runs + * behind its natural timing because a floor above raised some node's arrival, the call prepares + * the committed session again with no floors and publishes it for the next block: from that + * block latency_samples is the session's natural latency. That plan carries only the sources + * (each source's ring, generation and read position), so no frame is repeated or skipped; every + * effect, line and meter starts at rest. A waiting replacement is superseded, as a structural + * transaction supersedes it, and its revisions complete when the new plan is adopted with + * MISO_ENGINE_V1_OUTCOME_EXACT (MISO_ENGINE_V1_OUTCOME_SUPERSEDED for the revisions it had folded + * in), never MISO_ENGINE_V1_OUTCOME_TRANSITION_FALLBACK. A structural transaction that supersedes + * the new plan before render adopts it is prepared the same way. With nothing to reset, or at + * revision 0, the call returns MISO_ENGINE_V1_OK and publishes nothing. The model and the + * revision never change. It returns MISO_ENGINE_V1_OK; a null session or another handle as every + * session call; and MISO_ENGINE_V1_COMPILE_REJECTED, MISO_ENGINE_V1_BACKPRESSURE or + * MISO_ENGINE_V1_INTERNAL with the session diagnostic as miso_engine_v1_submit_command returns + * them for a rebuild; a refusal changes nothing. A host checks + * MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY before calling the symbol. + * * Solo (issue 1559). Solo is a SET_CONTROL like every other value: the address is the strip's * own section, control 2 (named "solo", a toggle, value exactly 0 or 1, lane both), on a track. * A submix has no solo row (a submix is solo-safe), so a solo on a submix is the typed refusal @@ -204,38 +234,41 @@ extern "C" { * A transaction behind an unadopted replacement (issue 1310). A SESSION_TRANSACTION_APPLY that * needs a new plan while an earlier transaction's replacement plan has not been adopted by render * (a paused host never adopts one) commits as any other: it is not refused as backpressure because - * a replacement waits, with the one exception below. The engine takes the waiting replacement - * back, prepares one plan that carries both transactions against the plan that renders, and - * publishes it; nothing of the earlier transaction is lost. A source the new plan keeps from the - * waiting replacement or from the plan that renders keeps playing: the PCM accepted for it and a - * seek or seek_at queued on it move into the new plan, so the host neither refeeds nor reseeks it. - * One case is not covered yet: a source or owner that the earlier transaction restarted or removed - * and the new one puts back in its form in the plan that renders starts fresh in the new plan (a - * source at generation 1, frame 0, as an added one, and the host restarts its feed); that gap - * closes with the successor issue "Carry an owner or source a superseding candidate keeps - * unchanged from P0 from P0's state". No accepted chunk on a ring the waiting replacement created - * is dropped across a supersession: when the waiting replacement created a source's ring (the - * earlier transaction added the source, or redeclared it) and the session has accepted PCM, a seek - * or an end of region for that source since, a new transaction that does not keep the source as - * the waiting replacement declares it (it declares it as the plan that renders does, redeclares it - * another way, or removes it) is refused with MISO_ENGINE_V1_BACKPRESSURE before it commits: - * nothing changes, the waiting replacement stays published, no event is queued and the revision - * does not move. Once render adopts the waiting replacement, a retry commits. A host that does not - * render keeps getting MISO_ENGINE_V1_BACKPRESSURE for such a transaction until it renders. A - * source the waiting replacement carries unchanged from the plan that renders is not refused this - * way. A source the caller's own transaction removes discards its queued audio, whether the plan - * that renders or the waiting replacement (carrying it unchanged) holds it: the caller asked for - * the removal, so this is not an unintended drop. No counter reports the discarded frames yet; a - * counter for the discarded frames is a successor issue (placeholder #1699). A resource cap counts - * every plan the session still holds and the new one: the running plan, the waiting replacement, - * and a plan render has displaced until a later call reclaims it. A refusal is - * MISO_ENGINE_V1_BACKPRESSURE when the new plan fit against the newest held plan alone, so a retry - * after a render normally commits; the split is approximate, and such a retry may still be refused - * within a small band (a plan prepared on the adopted replacement differs by a few hundred bytes - * from one prepared across it). A refusal that does not fit even then is - * MISO_ENGINE_V1_COMPILE_REJECTED with the cap's diagnostic. When render adopts the new plan, the - * watermark advances to its revision with MISO_ENGINE_V1_OUTCOME_SUPERSEDED set, and - * superseded_count grows by the revisions the earlier replacements carried. + * a replacement waits, with the one exception below. The engine takes the waiting replacement back, + * prepares one plan that carries both transactions against the plan that renders, and publishes it; + * nothing of the earlier transaction is lost. A source the new plan keeps from the waiting + * replacement or from the plan that renders keeps playing: the PCM accepted for it and a seek or + * seek_at queued on it move into the new plan, so the host neither refeeds nor reseeks it. An owner + * (a strip, an effect, a meter) that the earlier transaction restarted or removed and the new one + * puts back in its form in the plan that renders carries from the plan that renders, as one + * transaction of both would carry it (issue 1698). A source does not: its declaration changed + * against the newest committed session, so the host restarted its feed, and a source the earlier + * transaction added, redeclared or removed and the new one does not keep as the waiting replacement + * declares it starts fresh in the new plan (generation 1, frame 0, as an added one), also when the + * new one puts it back in its form in the plan that renders. No accepted chunk on a ring the + * waiting replacement created is dropped across a supersession: when the waiting replacement + * created a source's ring (the earlier transaction added the source, or redeclared it) and the + * session has accepted PCM, a seek or an end of region for that source since, a new transaction + * that does not keep the source as the waiting replacement declares it (it declares it as the plan + * that renders does, redeclares it another way, or removes it) is refused with + * MISO_ENGINE_V1_BACKPRESSURE before it commits: nothing changes, the waiting replacement stays + * published, no event is queued and the revision does not move. Once render adopts the waiting + * replacement, a retry commits. A host that does not render keeps getting + * MISO_ENGINE_V1_BACKPRESSURE for such a transaction until it renders. A source the waiting + * replacement carries unchanged from the plan that renders is not refused this way. A source the + * caller's own transaction removes discards its queued audio, whether the plan that renders or the + * waiting replacement (carrying it unchanged) holds it: the caller asked for the removal, so this + * is not an unintended drop. No counter reports the discarded frames yet; a counter for the + * discarded frames is a successor issue (placeholder #1699). A resource cap counts every plan the + * session still holds and the new one: the running plan, the waiting replacement, and a plan render + * has displaced until a later call reclaims it. A refusal is MISO_ENGINE_V1_BACKPRESSURE when the + * new plan fit against the newest held plan alone, so a retry after a render normally commits; the + * split is approximate, and such a retry may still be refused within a small band (a plan prepared + * on the adopted replacement differs by a few hundred bytes from one prepared across it). A refusal + * that does not fit even then is MISO_ENGINE_V1_COMPILE_REJECTED with the cap's diagnostic. When + * render adopts the new plan, the watermark advances to its revision with + * MISO_ENGINE_V1_OUTCOME_SUPERSEDED set, and superseded_count grows by the revisions the earlier + * replacements carried. * * Starting an added stem in time (issue 1275). miso_engine_v1_source_seek_at is * miso_engine_v1_source_seek with an anchor: source_frame enters the graph in the block that starts @@ -247,9 +280,9 @@ extern "C" { * quantum multiple a few quanta past the last rendered block; call * miso_engine_v1_source_seek_at(id, g, F, A), where g is above the source's generation (2 for a * just-added source) and F is the source frame the playing stems read at A; then submit generation - * g from F before the block at A renders. Until A the stem renders silence, and those blocks - * count as source underruns; from A it plays F on. Should the render reach the seek only after A, - * the stem starts at F plus the lateness, so it stays in time. + * g from F before the block at A renders. Until A the stem renders silence; from A it plays F on. + * Should the render reach the seek only after A, the stem starts at F plus the lateness, so it + * stays in time. */ #define MISO_ENGINE_V1_ABI_VERSION UINT32_C(0x00010000) @@ -285,7 +318,8 @@ extern "C" { #define MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_AT UINT64_C(32) #define MISO_ENGINE_V1_FEATURE_PLAN_WATERMARK UINT64_C(64) #define MISO_ENGINE_V1_FEATURE_SERVICE UINT64_C(128) -#define MISO_ENGINE_V1_FEATURE_MASK UINT64_C(255) +#define MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY UINT64_C(256) +#define MISO_ENGINE_V1_FEATURE_MASK UINT64_C(511) #define MISO_ENGINE_V1_OUTCOME_EXACT UINT64_C(1) #define MISO_ENGINE_V1_OUTCOME_TRANSITION_FALLBACK UINT64_C(2) @@ -493,6 +527,7 @@ uint32_t miso_engine_v1_dequeue_event(miso_engine_v1_session *session, uint32_t lane, miso_engine_v1_bytes_out *event); uint32_t miso_engine_v1_service(miso_engine_v1_session *session); +uint32_t miso_engine_v1_declare_discontinuity(miso_engine_v1_session *session); uint32_t miso_engine_v1_render_f32_planar(miso_engine_v1_plan *plan, uint64_t absolute_sample, const miso_engine_v1_planar_output *output); diff --git a/crates/capi/src/abi.rs b/crates/capi/src/abi.rs index 7eda1e635..44a61aafb 100644 --- a/crates/capi/src/abi.rs +++ b/crates/capi/src/abi.rs @@ -34,9 +34,9 @@ pub const EVENT_LANE_RELIABLE: u32 = 0; pub const EVENT_LANE_LOSSY: u32 = 1; /// A plan tail terminates after `tail_samples`. -pub const TAIL_FINITE: u64 = 0; +pub const TAIL_FINITE: u64 = control_plane::TAIL_FINITE; /// A plan tail is unbounded and reports zero `tail_samples`. -pub const TAIL_INFINITE: u64 = 1; +pub const TAIL_INFINITE: u64 = control_plane::TAIL_INFINITE; /// Launch support for 44,100 Hz. pub const RATE_44_100: u64 = 1 << 0; @@ -68,6 +68,9 @@ pub const FEATURE_PLAN_WATERMARK: u64 = 1 << 6; /// Control-service capability: `miso_engine_v1_service` (#1348), an in-place V1 amendment. A host /// checks this bit before calling the symbol. pub const FEATURE_SERVICE: u64 = 1 << 7; +/// Declared-discontinuity capability: `miso_engine_v1_declare_discontinuity` (#1323), an in-place +/// V1 amendment. A host checks this bit before calling the symbol. +pub const FEATURE_DECLARE_DISCONTINUITY: u64 = 1 << 8; /// All ABI V1 feature capability bits. pub const FEATURE_MASK: u64 = FEATURE_IMMUTABLE_SESSION | FEATURE_HOST_PLANAR_SOURCE @@ -76,7 +79,8 @@ pub const FEATURE_MASK: u64 = FEATURE_IMMUTABLE_SESSION | FEATURE_CAPABILITY_COMMAND | FEATURE_SOURCE_SEEK_AT | FEATURE_PLAN_WATERMARK - | FEATURE_SERVICE; + | FEATURE_SERVICE + | FEATURE_DECLARE_DISCONTINUITY; /// Watermark outcome flag: the covered revisions completed exactly as committed. pub const OUTCOME_EXACT: u64 = 1; @@ -551,7 +555,31 @@ mod tests { FEATURE_PLAN_WATERMARK ); assert_eq!(FEATURE_MASK & FEATURE_SERVICE, FEATURE_SERVICE); - assert_eq!(FEATURE_MASK, 0xff); + // #1323 gate 6 (#1317's growth rule): the mask is the OR of every feature symbol, each one + // bit of its own, never a literal. + let features = [ + FEATURE_IMMUTABLE_SESSION, + FEATURE_HOST_PLANAR_SOURCE, + FEATURE_SOURCE_SEEK, + FEATURE_PLANAR_STEREO_RENDER, + FEATURE_CAPABILITY_COMMAND, + FEATURE_SOURCE_SEEK_AT, + FEATURE_PLAN_WATERMARK, + FEATURE_SERVICE, + FEATURE_DECLARE_DISCONTINUITY, + ]; + assert!(features.iter().all(|feature| feature.count_ones() == 1)); + assert_eq!( + features.iter().fold(0, |mask, feature| { + assert_eq!(mask & feature, 0, "every feature has its own bit"); + mask | feature + }), + FEATURE_MASK + ); + assert_eq!( + FEATURE_MASK & FEATURE_DECLARE_DISCONTINUITY, + FEATURE_DECLARE_DISCONTINUITY + ); assert_eq!( [ OUTCOME_EXACT, diff --git a/crates/capi/src/ffi.rs b/crates/capi/src/ffi.rs index c6ae5c5bd..c469eef4e 100644 --- a/crates/capi/src/ffi.rs +++ b/crates/capi/src/ffi.rs @@ -670,6 +670,54 @@ pub unsafe extern "C" fn miso_engine_v1_service(session: *mut Session) -> u32 { }) } +/// Declare a discontinuity (#1323): the next block the host renders need not continue the +/// previous output, because the host has stopped or is about to seek every source. A host calls it +/// at a stop, and before a seek of every source. +/// +/// It forwards to the control plane's `SessionState::declare_discontinuity`, the function the +/// browser's `miso_engine_web_v1_declare_discontinuity` runs. If the newest plan runs behind its +/// natural timing (a latency floor of a structural transaction raised it, #1285), the committed +/// session is prepared again with no floors, only its sources carried, and published for the next +/// block, superseding a waiting replacement; otherwise nothing is published. The model and the +/// revision do not change. +/// +/// Returns `RESULT_OK`; a null or wrong handle as every session call; `RESULT_COMPILE_REJECTED` +/// with the preparation's or the cap's diagnostic, `RESULT_BACKPRESSURE` and `RESULT_INTERNAL` as +/// `miso_engine_v1_submit_command` returns them for a rebuild. A refusal changes nothing. +/// +/// # Safety +/// +/// `session` must be a live session handle. +/// +/// Thread: control, serialized with the other session calls. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn miso_engine_v1_declare_discontinuity(session: *mut Session) -> u32 { + catch_result(|| { + // SAFETY: Nonnull live handle pointers are caller-provided under the handle contract. + let kind = unsafe { session_kind(session) }; + if kind != RESULT_OK { + return kind; + } + // SAFETY: `session` passed the live-kind check and the ABI serializes session calls. + let session = unsafe { &mut *session }; + match session.state.declare_discontinuity() { + Ok(()) => { + session.last_error.borrow_mut().clear(); + RESULT_OK + } + Err(refused) => { + session.last_error.borrow_mut().set(&refused.diagnostic()); + match refused { + CommandError::BufferTooSmall { .. } | CommandError::Internal => RESULT_INTERNAL, + CommandError::Invalid => RESULT_INVALID_ARGUMENT, + CommandError::Backpressure => RESULT_BACKPRESSURE, + CommandError::CompileRejected(_) => RESULT_COMPILE_REJECTED, + } + } + } + }) +} + /// Process one bounded Issue-005 capability command with exact-byte replay. /// /// # Safety @@ -1144,6 +1192,24 @@ pub(crate) fn test_service(session: *mut Session) -> u32 { unsafe { miso_engine_v1_service(session) } } +#[cfg(test)] +pub(crate) fn test_declare_discontinuity(session: *mut Session) -> u32 { + // SAFETY: Test callers pass a live session handle, or a null or wrong one to test refusal. + unsafe { miso_engine_v1_declare_discontinuity(session) } +} + +/// The plan's `latency_samples` through the exported resource query (#1323 gate 2). +#[cfg(test)] +pub(crate) fn test_plan_latency(plan: *const Plan) -> u64 { + // SAFETY: Every field of the report is an integer, for which zero is a valid value. + let mut out: PlanResourceReport = unsafe { core::mem::zeroed() }; + out.struct_size = crate::PLAN_RESOURCE_REPORT_SIZE; + // SAFETY: Test callers pass a live plan; `out` is one complete local report. + let result = unsafe { miso_engine_v1_plan_resources(plan, &mut out) }; + assert_eq!(result, RESULT_OK); + out.latency_samples +} + #[cfg(test)] pub(crate) fn test_dequeue_event(session: *mut Session, lane: u32, output: &mut BytesOut) -> u32 { // SAFETY: Test callers retain the live session and ABI output for this call. @@ -1567,10 +1633,14 @@ mod tests { assert_eq!(query(&mut capabilities), RESULT_OK); assert_eq!(capabilities.abi_version, ABI_VERSION); assert_eq!(capabilities.exact_launch_rate_mask, 0x0f); - assert_eq!(capabilities.feature_mask, 0xff); + assert_eq!(capabilities.feature_mask, crate::FEATURE_MASK); assert_ne!(capabilities.feature_mask & crate::FEATURE_SOURCE_SEEK_AT, 0); assert_ne!(capabilities.feature_mask & crate::FEATURE_PLAN_WATERMARK, 0); assert_ne!(capabilities.feature_mask & crate::FEATURE_SERVICE, 0); + assert_ne!( + capabilities.feature_mask & crate::FEATURE_DECLARE_DISCONTINUITY, + 0 + ); assert_eq!(capabilities.reserved, [0; 4]); } diff --git a/crates/capi/src/runtime/mod.rs b/crates/capi/src/runtime/mod.rs index 8687aac0f..c099920c7 100644 --- a/crates/capi/src/runtime/mod.rs +++ b/crates/capi/src/runtime/mod.rs @@ -21,7 +21,7 @@ pub(crate) use engine::realtime::RenderError; pub(crate) use crate::{ ABI_VERSION, CompileLimits, PlanResourceReport, RESULT_BACKPRESSURE, RESULT_INTERNAL, - RESULT_INVALID_ARGUMENT, TAIL_FINITE, TAIL_INFINITE, + RESULT_INVALID_ARGUMENT, TAIL_FINITE, }; pub(crate) mod error; @@ -218,10 +218,7 @@ impl From for ControlLimits { /// The ABI report row the control plane stores, built once when a row is inserted (#1309 D4). impl From for PlanResourceReport { fn from(resources: PlanResources) -> Self { - let (tail_kind, tail_samples) = match resources.output_tail { - TailSamples::Finite(samples) => (TAIL_FINITE, samples), - TailSamples::Infinite => (TAIL_INFINITE, 0), - }; + let (tail_kind, tail_samples) = control_plane::tail_words(resources.output_tail); Self { struct_size: crate::PLAN_RESOURCE_REPORT_SIZE, abi_version: ABI_VERSION, diff --git a/crates/capi/src/runtime/supersede_tests.rs b/crates/capi/src/runtime/supersede_tests.rs index a4c8b04c2..a6de8fcda 100644 --- a/crates/capi/src/runtime/supersede_tests.rs +++ b/crates/capi/src/runtime/supersede_tests.rs @@ -815,6 +815,49 @@ fn assert_same_blocks(label: &str, actual: &[Vec], expected: &[Vec]) { } } +/// #1698 gate 1 (owners): T1 restarts `t2`'s insert (`restart`: replaced) or removes it, and T2, +/// which supersedes T1, puts `t2`'s compressor back as P0 has it and replaces `t3`'s. `t2` +/// carries from P0, its compressor state included, so the replacement renders bit-identically to +/// the combined transaction P0 -> B (`t3`'s replacement alone). +fn a_reverted_owner_carries_from_p0(restart: bool) { + let mut model = stateless_session(48_000); + model.tracks[1].inserts.effects.push(compressor()); + model.tracks[2].inserts.effects.push(compressor()); + let first = if restart { + put(&model, 1, gate()) + } else { + remove(&model, 1) + }; + let revert = vec![put(&model, 1, compressor()), put(&model, 2, gate())]; + let (superseded, twin) = superseding_and_combined( + &model, + &[(vec![first], true), (revert, true)], + &[put(&model, 2, gate())], + ); + assert_same_blocks( + if restart { "restarted" } else { "removed" }, + &superseded, + &twin, + ); +} + +/// [`a_reverted_owner_carries_from_p0`], T1 replacing `t2`'s insert. +/// +/// Test value: red if an owner the withdrawn candidate restarted starts fresh in the replacement +/// (the batch 15a base, P0's inventory restricted to what the candidate carried). +#[test] +fn an_owner_the_candidate_restarts_and_the_replacement_reverts_carries_from_p0() { + a_reverted_owner_carries_from_p0(true); +} + +/// [`a_reverted_owner_carries_from_p0`], T1 removing `t2`'s insert. +/// +/// Test value: red if an owner the withdrawn candidate removed starts fresh in the replacement. +#[test] +fn an_owner_the_candidate_removes_and_the_replacement_restores_carries_from_p0() { + a_reverted_owner_carries_from_p0(false); +} + /// #1310 (D3 with root ruling B15-Q3, "B wins on a shared row"): T1 moves `t4`'s fader with its own /// ramp length; T2, which supersedes T1, moves it again with no length. The replacement ramps /// `t4` with the session default, as a transaction holding T1's structural edit and T2 does: B diff --git a/crates/capi/src/runtime/tests.rs b/crates/capi/src/runtime/tests.rs index b67c795fc..1494ce962 100644 --- a/crates/capi/src/runtime/tests.rs +++ b/crates/capi/src/runtime/tests.rs @@ -1522,6 +1522,392 @@ fn assert_bit_identical(label: &str, first: u64, actual: &[Vec], expected: } } +/// Submits block `block` of `signal` salt `salt` to `source` as generation `generation`, at the +/// source frame `block * 128`. +fn submit_block_c( + session: *mut crate::Session, + source: &[u8], + generation: u64, + block: u64, + salt: u64, + sample_rate_hz: u32, +) { + let (left, right) = continuity_block(block, salt); + submit_c_to( + session, + source, + generation, + block * 128, + sample_rate_hz, + &left, + &right, + false, + ); +} + +/// Commits `edits` at `base` through the C entry point and returns the path the response reports. +/// Panics unless the transaction commits. +fn transact_c( + session: *mut crate::Session, + request_id: u64, + base: u64, + edits: &[protocol::SessionEdit], +) -> protocol::EditPath { + let request = command_bytes_at_revision( + request_id, + ExpectedRevision::Exact(SessionRevision(base)), + protocol::CommandPayload::SessionTransactionApply(edits), + ); + let (result, response) = command_c(session, &request); + assert_eq!(result, crate::RESULT_OK, "request {request_id}"); + let mut fields = [0_u16; 64]; + match ProtocolCodec::default() + .decode_typed_response(&response, &mut DecodeScratch::new(&mut fields)) + .expect("transaction response") + { + protocol::DecodedTypedResponseFrame::Success { + payload: protocol::DecodedSuccessResponsePayload::SessionTransactionApplied(applied), + header, + } => { + assert_eq!(header.revision, SessionRevision(base + 1)); + applied.edit_path + } + protocol::DecodedTypedResponseFrame::Success { .. } => panic!("not a transaction response"), + protocol::DecodedTypedResponseFrame::NonOk { header, payload } => { + panic!( + "request {request_id}: {:?} {:?}", + header.status, payload.diagnostics + ) + } + } +} + +/// `UpsertSource` of `source_id` (a copy of the first source), `UpsertTrack` of `track_id` on it +/// and `UpsertRoute` of its route. +fn add_source_and_track( + model: &session::SessionModel, + source_id: &str, + track_id: &str, +) -> [protocol::SessionEdit; 3] { + let mut source = model.sources[0].clone(); + source.id = session::StableId::parse(source_id).expect("source ID"); + let mut track = model.tracks[0].clone(); + track.id = session::StableId::parse(track_id).expect("track ID"); + track.source_id = source.id.clone(); + let mut route = model.routes[0].clone(); + route.id = session::StableId::parse(&format!("{track_id}-main")).expect("route ID"); + let session::RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("track route") + }; + *track_id = track.id.clone(); + [ + protocol::SessionEdit::UpsertSource { source }, + protocol::SessionEdit::UpsertTrack { track }, + protocol::SessionEdit::UpsertRoute { route }, + ] +} + +/// A structural insert edit that changes the audio only on `track`: a compressor at its default +/// parameters, put at the head of the track's inserts. +fn put_compressor(model: &session::SessionModel, track: usize) -> protocol::SessionEdit { + use session::{EffectIdentity, EffectQuality, LinkMode, SidechainDeclaration, StableId}; + protocol::SessionEdit::PutTrackEffect { + track_id: model.tracks[track].id.clone(), + rack_name: session::RackName::Inserts, + final_position: 0, + effect: session::Effect { + id: StableId::parse("comp").expect("insert ID"), + identity: EffectIdentity::Native { + effect_id: StableId::parse("miso.compressor").expect("effect ID"), + }, + quality: EffectQuality::Normal, + bypass: false, + link_mode: LinkMode::DualMono, + params: Vec::new(), + sidechain: SidechainDeclaration::None, + }, + } +} + +/// The reference of the held-seek tests: a fresh session of `committed`, fed by `feed(session, +/// block)` one block ahead of each rendered block, rendering `blocks` blocks. +fn render_fresh_reference( + committed: &str, + blocks: u64, + feed: &dyn Fn(*mut crate::Session, u64), +) -> Vec> { + let (session, plan) = boxed_c_children(committed); + feed(session, 0); + let rendered = (0..blocks) + .map(|block| { + feed(session, block + 1); + render_c(plan, block) + }) + .collect::>(); + crate::ffi::test_plan_destroy(plan); + crate::ffi::test_session_destroy(session); + rendered +} + +/// Issue #1319 D1 (the #1275 verifier's probe P5): a seek the predecessor plan already popped, and +/// holds, moves with its source's consumer across a plan swap and applies at its anchor. +/// +/// At 48 kHz the fixture source plays generation 1 for blocks 0-7. After block 3 the host anchors +/// generation 2 at render sample 1024 (block 8) and submits it from frame 0. Blocks 4 and 5 render +/// on the predecessor, whose consumer pops the seek and holds it while the queued generation-1 +/// blocks play. A structural transaction (a muted track on the same source) commits after block 5, +/// block 6 is the swap block, and block 8 applies the seek on the successor. All 16 blocks equal a +/// fresh session of the committed document fed one stream that switches content at block 8, and +/// the swap carried the consumer. +/// +/// Test value: red if the swap resets, rebuilds or drops the carried consumer's held seek (block 8 +/// then finds no seek: the generation-2 blocks are discarded as unobserved and block 8 on is +/// silent), or if the successor applies a held seek before its anchor or after it; no other +/// committed test seeks before a swap (`successor_swap.rs` asserts `seek_before >= SWAP_BLOCK`). +#[test] +fn a_held_anchored_seek_crosses_a_plan_swap() { + const RATE: u32 = 48_000; + const FIXTURE_ID: &[u8] = b"fixture-source"; + const ANCHOR_BLOCK: u64 = 8; + let model = stateless_session(RATE); + let document = session::canonical_session_json(&model).expect("canonical"); + let (session, plan) = boxed_c_children(&document); + let mut swapped = Vec::new(); + submit_block_c(session, FIXTURE_ID, 1, 0, 0, RATE); + swapped.extend(feed_and_render_c(session, plan, RATE, 0, 4)); + // Generation 1 is queued through block 7 (block 4 is queued by the feed above). + for block in 5..8 { + submit_block_c(session, FIXTURE_ID, 1, block, 0, RATE); + } + assert_eq!( + crate::ffi::test_source_seek_at(session, FIXTURE_ID, 2, 0, ANCHOR_BLOCK * 128), + crate::RESULT_OK + ); + for block in 0..4 { + submit_block_c(session, FIXTURE_ID, 2, block, 1, RATE); + } + swapped.push(render_c(plan, 4)); + swapped.push(render_c(plan, 5)); + let edits = add_muted_track(&model, "a-muted"); + assert_eq!( + transact_c(session, 1, 42, &edits), + protocol::EditPath::Rebuild, + "a structural transaction rebuilds" + ); + for block in 6..16 { + // The ring holds eight blocks: generation 2 block `m` fits once `m - 3` blocks played. + match block { + 6 => { + submit_block_c(session, FIXTURE_ID, 2, 4, 1, RATE); + submit_block_c(session, FIXTURE_ID, 2, 5, 1, RATE); + } + 7 => submit_block_c(session, FIXTURE_ID, 2, 6, 1, RATE), + 8 => submit_block_c(session, FIXTURE_ID, 2, 7, 1, RATE), + _ => {} + } + swapped.push(render_c(plan, block)); + } + assert_eq!( + crate::ffi::test_plan_carry_counts(plan), + (1, 0), + "one swap, and it carried the source" + ); + let committed = snapshot_c(session); + crate::ffi::test_plan_destroy(plan); + crate::ffi::test_session_destroy(session); + + let reference = render_fresh_reference(&committed, 16, &|session, block| { + let (salt, content_block) = if block < ANCHOR_BLOCK { + (0, block) + } else { + (1, block - ANCHOR_BLOCK) + }; + let (left, right) = continuity_block(content_block, salt); + submit_c(session, 1, block * 128, RATE, &left, &right, false); + }); + assert_bit_identical("held seek across a swap", 0, &swapped, &reference); +} + +/// Issue #1319 D2: an anchored seek queued on a source that persists from the running plan through +/// a withdrawn candidate into its replacement survives the supersession. +/// +/// P0 plays the fixture source for blocks 0-3 and has generation 1 queued through block 7. With no +/// render between them, T1 (a compressor on the second track) commits and candidate A is +/// published; the seek to generation 2 at block 8 goes to A's producer set, so it is queued in the +/// command slot of the ring P0 owns; T2 (a compressor on the third track) withdraws A and commits +/// B against P0. Generation 2 is then submitted through the producer that is now B's, and blocks +/// 4-11 render. The source switches at the anchor, and every block equals a twin that submits +/// T1 and T2 as one transaction at the same point and seeks the same way. The supersession is one +/// withdrawal (and one drop of A), and each transaction took the rebuild path. +/// +/// Test value: red if the supersession rebuilds a persisting source's ring for B (its queued +/// blocks are lost) or loses the seek in the ring's command slot while A is withdrawn (block 8 on +/// then finds generation 2 unobserved and is silent); the #1310 tests seek only on a source the +/// withdrawn candidate added, whose ring is donated. +#[test] +fn a_queued_seek_on_a_persisting_source_survives_supersession() { + const RATE: u32 = 48_000; + const FIXTURE_ID: &[u8] = b"fixture-source"; + const ANCHOR_BLOCK: u64 = 8; + let model = stateless_session(RATE); + let document = session::canonical_session_json(&model).expect("canonical"); + let first = [put_compressor(&model, 1)]; + let second = [put_compressor(&model, 2)]; + let combined = [put_compressor(&model, 1), put_compressor(&model, 2)]; + + let run = |superseding: bool| { + let (session, plan) = boxed_c_children(&document); + let before = crate::ffi::test_owner_counters(session); + submit_block_c(session, FIXTURE_ID, 1, 0, 0, RATE); + let mut blocks = feed_and_render_c(session, plan, RATE, 0, 4); + for block in 5..8 { + submit_block_c(session, FIXTURE_ID, 1, block, 0, RATE); + } + // The seek goes to the pending candidate's producer set, whichever transaction left it. + let seek = || { + assert_eq!( + crate::ffi::test_source_seek_at(session, FIXTURE_ID, 2, 0, ANCHOR_BLOCK * 128), + crate::RESULT_OK + ); + }; + if superseding { + assert_eq!( + transact_c(session, 1, 42, &first), + protocol::EditPath::Rebuild + ); + seek(); + assert_eq!( + transact_c(session, 2, 43, &second), + protocol::EditPath::Rebuild + ); + } else { + assert_eq!( + transact_c(session, 1, 42, &combined), + protocol::EditPath::Rebuild + ); + seek(); + } + for block in 0..4 { + submit_block_c(session, FIXTURE_ID, 2, block, 1, RATE); + } + for block in 4..12 { + blocks.push(render_c(plan, block)); + } + let counters = crate::ffi::test_owner_counters(session); + let delta = ( + counters.candidate_withdrawn - before.candidate_withdrawn, + counters.candidate_superseded - before.candidate_superseded, + counters.candidate_taken - before.candidate_taken, + ); + let carried = crate::ffi::test_plan_carry_counts(plan); + crate::ffi::test_plan_destroy(plan); + crate::ffi::test_session_destroy(session); + (blocks, delta, carried) + }; + let (superseded, delta, carried) = run(true); + assert_eq!( + delta, + (1, 1, 0), + "one withdrawal, whose candidate A was dropped" + ); + assert_eq!(carried, (1, 0), "one swap, B alone, and it carried"); + let (twin, twin_delta, twin_carried) = run(false); + assert_eq!(twin_delta, (0, 0, 0)); + assert_eq!(twin_carried, (1, 0)); + assert_bit_identical("supersession across a queued seek", 0, &superseded, &twin); +} + +/// Issue #1319 D3: a held seek on a source a candidate added carries on when a later structural +/// transaction finds that candidate already taken by render. +/// +/// T1 adds source `s2` with its track. The host anchors `s2`'s generation 2 at block 8 (frame 0) +/// and submits it, and block 2 renders: render claims T1's candidate A, and A's consumer for `s2` +/// pops the seek and holds it. T2 (a muted track on the fixture source, keeping `s2`) commits: its +/// service step promotes A, so B is prepared against A as the running plan and carries A's `s2` +/// consumer with its held seek, rather than preparing across a withdrawal. Blocks 3-11 render. `s2` +/// is silent until block 8 and plays from there, and every block equals a fresh session of T2's +/// document fed `s2` zeros below the anchor and the same PCM from it. Neither withdrawal outcome +/// is counted: A was neither withdrawn nor found taken by the withdrawal. +/// +/// Test value: red if the control plane treats a candidate render has taken as withdrawn and +/// prepares B across it: `s2`, which A added, is then fresh in B, and A's held, acked seek and its +/// queued generation 2 are lost (block 8 on lacks `s2`). +#[test] +fn a_held_seek_carries_from_a_taken_candidate() { + const RATE: u32 = 48_000; + const FIXTURE_ID: &[u8] = b"fixture-source"; + const ADDED: &[u8] = b"s2-source"; + const ANCHOR_BLOCK: u64 = 8; + let mut model = stateless_session(RATE); + model.tracks.truncate(1); + model.routes.truncate(1); + let document = session::canonical_session_json(&model).expect("canonical"); + let (session, plan) = boxed_c_children(&document); + let before = crate::ffi::test_owner_counters(session); + submit_block_c(session, FIXTURE_ID, 1, 0, 0, RATE); + let mut swapped = feed_and_render_c(session, plan, RATE, 0, 2); + + let first = add_source_and_track(&model, "s2-source", "t2"); + assert_eq!( + transact_c(session, 1, 42, &first), + protocol::EditPath::Rebuild + ); + assert_eq!( + crate::ffi::test_source_seek_at(session, ADDED, 2, 0, ANCHOR_BLOCK * 128), + crate::RESULT_OK + ); + for block in 0..4 { + submit_block_c(session, ADDED, 2, block, 1, RATE); + } + submit_block_c(session, FIXTURE_ID, 1, 3, 0, RATE); + swapped.push(render_c(plan, 2)); + assert_eq!( + crate::ffi::test_plan_carry_counts(plan), + (1, 0), + "render took A" + ); + + let second = add_muted_track(&model, "a-muted"); + assert_eq!( + transact_c(session, 2, 43, &second), + protocol::EditPath::Rebuild + ); + let counters = crate::ffi::test_owner_counters(session); + assert_eq!( + ( + counters.candidate_withdrawn - before.candidate_withdrawn, + counters.candidate_taken - before.candidate_taken, + ), + (0, 0), + "the service step promoted A before the withdrawal ran" + ); + for block in 3..12 { + if block > 3 { + submit_block_c(session, FIXTURE_ID, 1, block, 0, RATE); + } + swapped.push(render_c(plan, block)); + } + assert_eq!( + crate::ffi::test_plan_carry_counts(plan), + (2, 0), + "A, then B, carried" + ); + let committed = snapshot_c(session); + crate::ffi::test_plan_destroy(plan); + crate::ffi::test_session_destroy(session); + + let reference = render_fresh_reference(&committed, 12, &|session, block| { + submit_block_c(session, FIXTURE_ID, 1, block, 0, RATE); + let (left, right) = if block < ANCHOR_BLOCK { + (vec![0.0; 128], vec![0.0; 128]) + } else { + continuity_block(block - ANCHOR_BLOCK, 1) + }; + submit_c_to(session, ADDED, 1, block * 128, RATE, &left, &right, false); + }); + assert_bit_identical("held seek from a taken candidate", 0, &swapped, &reference); +} + /// Issue #1273 gate 2: a transaction that removes a track and its source leaves the remaining /// source playing from its carried ring, bit-identical to the fresh post-edit session, and a /// submit for the removed source is refused as unknown. @@ -1612,6 +1998,654 @@ fn removing_a_track_and_its_source_keeps_the_other_source_playing() { assert_bit_identical("removal", 6, &swapped, &expected[6..]); } +/// Issue #1285 gate 3: three tracks, the first muted behind a true-peak limiter insert. A +/// `RemoveTrack` transaction after block 6 removes it. `miso_engine_v1_plan_resources` reports the +/// same `latency_samples` after the swap, and every block equals the floored reference: the same +/// transaction committed before the first block, so its successor is compiled with A's floors and +/// adopts at frame 0 with every state at rest, which is B compiled with A's floors and rendered +/// from frame 0. B compiled fresh has a lower latency and renders the remaining tracks earlier. +/// +/// Test value: red if the plan's resource row keeps the natural latency while the plan renders +/// the floored one, or if a successor is compiled without floors (the latency drops and the two +/// remaining tracks jump forward at the swap). +#[test] +fn removing_a_limited_track_keeps_the_plan_latency() { + let source: [&[u8]; 1] = [b"fixture-source"]; + let mut model = stateless_session(48_000); + let limited = &mut model.tracks[0]; + limited.fader.left_mute = true; + limited.fader.right_mute = true; + limited.inserts.effects.push(session::Effect { + id: session::StableId::parse("lim").expect("insert ID"), + identity: session::EffectIdentity::Native { + effect_id: session::StableId::parse("miso.true-peak-limiter").expect("limiter id"), + }, + quality: session::EffectQuality::Normal, + bypass: false, + link_mode: session::LinkMode::Maximum, + params: Vec::new(), + sidechain: session::SidechainDeclaration::None, + }); + let document = session::canonical_session_json(&model).expect("canonical"); + let edits = [ + protocol::SessionEdit::RemoveRoute { + route_id: model.routes[0].id.clone(), + }, + protocol::SessionEdit::RemoveTrack { + track_id: model.tracks[0].id.clone(), + }, + ]; + let transaction = command_bytes_at_revision( + 1, + ExpectedRevision::Exact(SessionRevision(42)), + protocol::CommandPayload::SessionTransactionApply(&edits), + ); + let bits = |blocks: &[Vec]| -> Vec> { + blocks + .iter() + .map(|block| block.iter().map(|sample| sample.to_bits()).collect()) + .collect() + }; + + let mut children = compile_children(&document, limits()).expect("children"); + let latency = children.plan.resources().latency_samples; + assert!(latency > 0, "the limiter delays A"); + submit_direct(&mut children, &source, 0); + let mut swapped = feed_and_render_direct(&mut children, &source, 0, 6); + children + .session + .command(&transaction, 4_096) + .expect("removal transaction"); + swapped.extend(feed_and_render_direct(&mut children, &source, 6, 6)); + assert_eq!(children.plan.owner().carried_count(), 1, "the swap carried"); + assert_eq!( + children.plan.resources().latency_samples, + latency, + "the plan reports A's latency after the swap (D4)" + ); + + let mut reference = compile_children(&document, limits()).expect("reference"); + submit_direct(&mut reference, &source, 0); + reference + .session + .command(&transaction, 4_096) + .expect("reference transaction"); + let expected = feed_and_render_direct(&mut reference, &source, 0, 12); + assert_eq!(reference.plan.owner().carried_count(), 1); + assert_eq!(reference.plan.resources().latency_samples, latency); + assert!( + expected[7..].iter().flatten().all(|sample| *sample != 0.0), + "the reference plays after the swap" + ); + assert_eq!( + bits(&swapped), + bits(&expected), + "every block equals B compiled with A's floors" + ); + + let committed = session::canonical_session_json( + children + .session + .test_controller() + .session() + .compiled() + .normalized_model(), + ) + .expect("committed"); + let mut fresh = compile_children(&committed, limits()).expect("fresh B"); + assert!(fresh.plan.resources().latency_samples < latency); + submit_direct(&mut fresh, &source, 0); + let unfloored = feed_and_render_direct(&mut fresh, &source, 0, 12); + assert_ne!( + bits(&unfloored[6..]), + bits(&expected[6..]), + "without floors the remaining tracks move by the limiter's latency" + ); +} + +// Issue #1323: a host-declared discontinuity resets the latency floors of #1285. + +/// The #1285 gate 3 session: three tracks on the fixture source, the first muted behind a +/// true-peak limiter insert, and the `RemoveRoute`/`RemoveTrack` transaction that removes it. +fn limited_session() -> (session::SessionModel, [protocol::SessionEdit; 2]) { + let mut model = stateless_session(48_000); + let limited = &mut model.tracks[0]; + limited.fader.left_mute = true; + limited.fader.right_mute = true; + limited.inserts.effects.push(session::Effect { + id: session::StableId::parse("lim").expect("insert ID"), + identity: session::EffectIdentity::Native { + effect_id: session::StableId::parse("miso.true-peak-limiter").expect("limiter id"), + }, + quality: session::EffectQuality::Normal, + bypass: false, + link_mode: session::LinkMode::Maximum, + params: Vec::new(), + sidechain: session::SidechainDeclaration::None, + }); + let edits = [ + protocol::SessionEdit::RemoveRoute { + route_id: model.routes[0].id.clone(), + }, + protocol::SessionEdit::RemoveTrack { + track_id: model.tracks[0].id.clone(), + }, + ]; + (model, edits) +} + +/// `edits` as one transaction expected at `revision`. +fn transaction_at(request_id: u64, revision: u64, edits: &[protocol::SessionEdit]) -> Vec { + command_bytes_at_revision( + request_id, + ExpectedRevision::Exact(SessionRevision(revision)), + protocol::CommandPayload::SessionTransactionApply(edits), + ) +} + +/// Runs `request` on the session state, which must accept it, and drains the reliable events. +fn command_direct(children: &mut CompiledChildren, request: &[u8]) { + children + .session + .command(request, 4_096) + .unwrap_or_else(|error| panic!("{error:?}")); + while children + .session + .dequeue_event(EventLane::Reliable, 4_096) + .expect("a reliable event") + .is_some() + {} +} + +/// The committed model's canonical JSON. +fn committed_document(children: &CompiledChildren) -> String { + session::canonical_session_json( + children + .session + .test_controller() + .session() + .compiled() + .normalized_model(), + ) + .expect("canonical") +} + +/// `document` compiled fresh, so with no floors, fed the fixture source's signal from source +/// block `from` on, rendered for `blocks` blocks: the reference of a plan that adopts at block +/// `from` with its carried source consumer standing at frame `from * 128`. Returns its latency +/// and its blocks. +fn natural_reference(document: &str, from: u64, blocks: u64) -> (u64, Vec>) { + let mut fresh = compile_children(document, limits()).expect("fresh"); + let latency = fresh.plan.resources().latency_samples; + let submit = |fresh: &mut CompiledChildren, block: u64| { + let (left, right) = continuity_block(from + block, 0); + fresh + .session + .submit( + b"fixture-source", + SourceSubmission { + generation: 1, + start_frame: block * 128, + sample_rate_hz: 48_000, + planes: &[&left, &right], + frames: 128, + end_of_region: false, + }, + ) + .unwrap_or_else(|error| panic!("block {block}: {:?}", error.report())); + }; + submit(&mut fresh, 0); + let rendered = (0..blocks) + .map(|block| { + submit(&mut fresh, block + 1); + render_direct(&mut fresh, block) + }) + .collect(); + (latency, rendered) +} + +fn sample_bits(blocks: &[Vec]) -> Vec> { + blocks + .iter() + .map(|block| block.iter().map(|sample| sample.to_bits()).collect()) + .collect() +} + +/// Issue #1323 gate 2: the #1285 removal through the C ABI, then +/// `miso_engine_v1_declare_discontinuity`. Before the declaration +/// `miso_engine_v1_plan_resources` reports A's latency (the floors), and after one render it +/// reports the committed session's natural latency; from that block the output is the committed +/// session compiled fresh, fed from the frame the carried source stood at. The revision and the +/// watermark do not move. +/// +/// Test value: red if the declaration clears the floors in an inventory but publishes no plan (the +/// report keeps A's latency), if the C entry does not reach the control plane, or if the +/// successor carries any state but the source (the compensation lines differ from a fresh plan's). +#[test] +fn a_declared_discontinuity_reports_the_natural_latency_on_the_c_abi() { + let (model, edits) = limited_session(); + let document = session::canonical_session_json(&model).expect("canonical"); + let (c_session, c_plan) = boxed_c_children(&document); + let latency = crate::ffi::test_plan_latency(c_plan); + assert!(latency > 0, "the limiter delays A"); + let (left, right) = continuity_block(0, 0); + submit_c(c_session, 1, 0, 48_000, &left, &right, false); + feed_and_render_c(c_session, c_plan, 48_000, 0, 6); + assert_eq!( + command_c(c_session, &transaction_at(1, 42, &edits)).0, + crate::RESULT_OK + ); + feed_and_render_c(c_session, c_plan, 48_000, 6, 4); + assert_eq!( + crate::ffi::test_plan_latency(c_plan), + latency, + "the removal keeps A's latency (#1285)" + ); + let summary = crate::ffi::test_session_state_summary(c_session); + let (_, watermark) = crate::ffi::test_plan_watermark(c_plan); + assert_eq!(watermark.revision, 43); + + assert_eq!( + crate::ffi::test_declare_discontinuity(c_session), + crate::RESULT_OK + ); + assert_eq!( + crate::ffi::test_plan_latency(c_plan), + latency, + "the running plan reports until render adopts the successor" + ); + let reset = feed_and_render_c(c_session, c_plan, 48_000, 10, 6); + let (natural, expected) = natural_reference(&snapshot_c(c_session), 10, 6); + assert!(natural < latency, "B's natural latency is lower"); + assert_eq!(crate::ffi::test_plan_latency(c_plan), natural); + assert!( + expected.iter().flatten().any(|sample| *sample != 0.0), + "the reference plays" + ); + assert_eq!(sample_bits(&reset), sample_bits(&expected)); + let after = crate::ffi::test_session_state_summary(c_session); + assert_eq!(after.0, summary.0, "the revision does not move"); + let (_, adopted) = crate::ffi::test_plan_watermark(c_plan); + assert_eq!(adopted, watermark, "the watermark does not move"); + crate::ffi::test_plan_destroy(c_plan); + crate::ffi::test_session_destroy(c_session); +} + +/// Issue #1323 gate 3: a declaration with nothing to reset returns `OK`, publishes no plan and +/// leaves the output bit-identical to a run without it: on a session at revision 0 (B15-R6: no +/// successor, so no floor), and after a structural transaction that adds a muted track, whose +/// successor's floors raise no node. +/// +/// Test value: red if the declaration always rebuilds (every effect tail and line would restart +/// at each stop: here the compensation lines of the added track's path and the limiter restart, +/// and the output differs from the twin), or if it refuses or moves the revision at revision 0. +#[test] +fn a_declaration_with_nothing_to_reset_publishes_nothing() { + for (label, revision, structural) in [("revision 0", 0, false), ("added track", 42, true)] { + let (mut model, _) = limited_session(); + model.revision = revision; + let document = session::canonical_session_json(&model).expect("canonical"); + let run = |declare: bool| { + let mut children = compile_children(&document, limits()).expect("children"); + let latency = children.plan.resources().latency_samples; + submit_direct(&mut children, &[b"fixture-source"], 0); + if declare { + let epoch = children.session.newest_plan_epoch(); + children + .session + .declare_discontinuity() + .expect("nothing to reset"); + assert_eq!(children.session.newest_plan_epoch(), epoch, "{label}"); + } + let mut blocks = feed_and_render_direct(&mut children, &[b"fixture-source"], 0, 4); + if structural { + command_direct( + &mut children, + &transaction_at( + 1, + revision, + &add_muted_track(&stateless_session(48_000), "a-muted"), + ), + ); + } + blocks.extend(feed_and_render_direct( + &mut children, + &[b"fixture-source"], + 4, + 2, + )); + assert!( + !children + .session + .test_providers() + .test_inventory() + .runs_behind_natural_timing(), + "{label}: no floor is raised" + ); + if declare { + let epoch = children.session.newest_plan_epoch(); + let summary = children.session.test_state_summary(); + children + .session + .declare_discontinuity() + .expect("nothing to reset"); + assert_eq!(children.session.newest_plan_epoch(), epoch, "{label}"); + assert_eq!(children.session.test_state_summary(), summary, "{label}"); + } + blocks.extend(feed_and_render_direct( + &mut children, + &[b"fixture-source"], + 6, + 6, + )); + assert_eq!( + children.plan.resources().latency_samples, + latency, + "{label}" + ); + blocks + }; + let declared = run(true); + let twin = run(false); + assert!( + twin[8..].iter().flatten().any(|sample| *sample != 0.0), + "{label}: the twin plays" + ); + assert_eq!(sample_bits(&declared), sample_bits(&twin), "{label}"); + } +} + +/// Issue #1323 gate 4: with the replacement peak cap below any successor, the declaration after +/// the #1285 removal returns the cap's compile rejection, and the plan render runs, its floors, +/// the committed model and the revision are unchanged: the next blocks equal a run without the +/// call. With the cap lifted, the same declaration then resets the floors. +/// +/// Test value: red if the declaration drops the floors before its preparation succeeds (the +/// running epoch's inventory loses them and the retry finds nothing to reset, so the latency +/// never comes down), or if a refusal publishes or moves anything. +#[test] +fn a_refused_declaration_changes_nothing() { + let (model, edits) = limited_session(); + let document = session::canonical_session_json(&model).expect("canonical"); + let run = |declare: bool| { + let mut children = compile_children(&document, limits()).expect("children"); + let latency = children.plan.resources().latency_samples; + submit_direct(&mut children, &[b"fixture-source"], 0); + let mut blocks = feed_and_render_direct(&mut children, &[b"fixture-source"], 0, 6); + command_direct(&mut children, &transaction_at(1, 42, &edits)); + blocks.extend(feed_and_render_direct( + &mut children, + &[b"fixture-source"], + 6, + 4, + )); + if declare { + let floors = children.session.test_providers().test_inventory().clone(); + assert!(floors.runs_behind_natural_timing()); + let epoch = children.session.newest_plan_epoch(); + let summary = children.session.test_state_summary(); + let committed = committed_document(&children); + children.session.set_replacement_peak_limit(1); + let refused = children.session.declare_discontinuity(); + assert!( + matches!(refused, Err(CommandError::CompileRejected(_))), + "{refused:?}" + ); + assert_eq!(children.session.newest_plan_epoch(), epoch); + assert_eq!(children.session.test_state_summary(), summary); + assert_eq!(committed_document(&children), committed); + assert_eq!( + children + .session + .test_providers() + .test_inventory() + .input_floors(), + floors.input_floors(), + "the running plan keeps its floors" + ); + children.session.set_replacement_peak_limit(u64::MAX); + } + blocks.extend(feed_and_render_direct( + &mut children, + &[b"fixture-source"], + 10, + 2, + )); + assert_eq!(children.plan.resources().latency_samples, latency); + (children, blocks, latency) + }; + let (mut refused, blocks, latency) = run(true); + let (_, twin, _) = run(false); + assert_eq!(sample_bits(&blocks), sample_bits(&twin)); + + refused + .session + .declare_discontinuity() + .expect("the retry under the lifted cap"); + feed_and_render_direct(&mut refused, &[b"fixture-source"], 12, 1); + let (natural, _) = natural_reference(&committed_document(&refused), 12, 1); + assert!(natural < latency); + assert_eq!(refused.plan.resources().latency_samples, natural); +} + +/// The lifecycle counters of every owner and reservation, balanced once both handles are gone. +fn assert_owners_balanced(counters: TestOwnerCounters) { + assert_eq!( + counters.current_provider_constructed + counters.candidate_provider_constructed, + counters.current_provider_disposed + counters.candidate_provider_disposed, + "providers" + ); + assert_eq!( + counters.current_plan_constructed + counters.candidate_plan_constructed, + counters.current_plan_disposed + counters.candidate_plan_disposed, + "plans" + ); + assert_eq!( + counters.token_constructed, counters.token_disposed, + "tokens" + ); + assert_eq!( + counters.reservation_constructed, + counters.reservation_canceled + counters.reservation_committed, + "reservations" + ); +} + +/// Issue #1323 gate 5: a declared stop supersedes a pending warm successor. The #1285 removal B +/// is committed and published `Primed` 64 blocks ahead (floored at A's arrivals, so above its +/// natural ones). After 4 blocks B is still pending; the declaration withdraws it, and the block +/// after the declaration is the discontinuity successor's: from it the output is the committed +/// session compiled fresh, fed from the frame the carried source stood at, at the natural +/// latency. B is never adopted. The watermark advances to the committed revision at that block +/// with `EXACT` and never `TRANSITION_FALLBACK`; with an earlier structural edit B folded in, with +/// `EXACT | SUPERSEDED`, and `superseded` grows by 1. The source's producer moves into the +/// successor, and every owner and reservation balances. +/// +/// Test value: red if the successor is published with B's `Primed` kind (it waits for +/// `not_before` and readiness, and the floored B plays past the stop), if B is left to its own +/// adoption, or if the revision is reported as a fallback or loses B's folded revision. +fn a_declared_stop_supersedes_a_pending_warm_successor(fold: bool) { + const DECLARED_AT: u64 = 8; + test_reset_lifecycle_observer(); + { + let (model, edits) = limited_session(); + let document = session::canonical_session_json(&model).expect("canonical"); + let mut children = compile_children(&document, limits()).expect("children"); + let latency = children.plan.resources().latency_samples; + submit_direct(&mut children, &[b"fixture-source"], 0); + feed_and_render_direct(&mut children, &[b"fixture-source"], 0, 4); + let mut revision = 42; + if fold { + // An edit committed and left pending: B supersedes it, so B folds in one revision. + command_direct( + &mut children, + &transaction_at( + 1, + revision, + &add_muted_track(&stateless_session(48_000), "a-muted"), + ), + ); + revision += 1; + } + children + .session + .test_set_next_adoption(engine::realtime::PlanAdoption::Primed { + not_before: (4 + 64) * 128, + lead_blocks: 1, + }); + command_direct(&mut children, &transaction_at(2, revision, &edits)); + revision += 1; + assert!( + children.session.test_pending_providers()[0] + .test_inventory() + .runs_behind_natural_timing(), + "B is floored above its natural arrivals" + ); + feed_and_render_direct(&mut children, &[b"fixture-source"], 4, 4); + let before = children.plan.queries().watermark().expect("watermark"); + assert_eq!(before.revision, 42, "B waits, unadopted"); + assert_eq!(children.session.test_pending_providers().len(), 1); + let counters = children.session.test_owner_counters(); + + children + .session + .declare_discontinuity() + .expect("the declaration"); + let after = children.session.test_owner_counters(); + assert_eq!(after.candidate_withdrawn, counters.candidate_withdrawn + 1); + assert_eq!( + after.candidate_superseded, + counters.candidate_superseded + 1 + ); + assert_eq!(after.candidate_republished, counters.candidate_republished); + assert_eq!( + after.candidate_provider_published, + counters.candidate_provider_published + 1 + ); + let newest = children.session.newest_plan_epoch(); + let successor = children + .session + .test_epoch_mut(newest) + .expect("the successor's epoch"); + assert_eq!( + successor.test_source_vacant(b"fixture-source"), + Some(false), + "the source's producer moved into the successor" + ); + + let reset = feed_and_render_direct(&mut children, &[b"fixture-source"], DECLARED_AT, 8); + let watermark = children.plan.queries().watermark().expect("watermark"); + assert_eq!( + (watermark.revision, watermark.first_sample), + (revision, DECLARED_AT * 128), + "the successor is adopted at the block after the declaration" + ); + let superseded = u64::from(fold); + assert_eq!( + watermark.outcome_flags, + engine::realtime::OUTCOME_EXACT + | if fold { + engine::realtime::OUTCOME_SUPERSEDED + } else { + 0 + } + ); + assert_eq!(watermark.superseded, before.superseded + superseded); + assert_eq!(watermark.exact, before.exact + 1); + assert_eq!(watermark.transition_fallback, before.transition_fallback); + let (natural, expected) = natural_reference(&committed_document(&children), DECLARED_AT, 8); + assert!(natural < latency); + assert_eq!(children.plan.resources().latency_samples, natural); + assert!(expected.iter().flatten().any(|sample| *sample != 0.0)); + assert_eq!(sample_bits(&reset), sample_bits(&expected)); + } + assert_owners_balanced(test_lifecycle_counters()); +} + +#[test] +fn a_declared_stop_supersedes_a_pending_warm_successor_alone_and_folded() { + a_declared_stop_supersedes_a_pending_warm_successor(false); + a_declared_stop_supersedes_a_pending_warm_successor(true); +} + +/// Issue #1323, the supersession the spec leaves open: a structural edit E committed while the +/// discontinuity successor D waits unadopted supersedes it, and E is prepared at the same +/// discontinuity. Render never played D, so the pair is the one transaction "declare, then edit" +/// against the running plan, whose combined twin (P0 -> E) has no floors: E runs at its natural +/// latency, carries only its source, and renders as its model compiled fresh from the block that +/// adopts it. A second declaration while D waits finds nothing to reset (D has no raised floor) +/// and publishes nothing. +/// +/// Test value: red if E is prepared as an ordinary successor across the withdrawn D: its base is +/// the running plan's inventory, so E takes the running plan's floors (A's latency) and the +/// declared stop is lost. +#[test] +fn an_edit_that_supersedes_a_discontinuity_successor_is_prepared_at_it() { + const ADOPTED_AT: u64 = 10; + let (model, edits) = limited_session(); + let document = session::canonical_session_json(&model).expect("canonical"); + let mut children = compile_children(&document, limits()).expect("children"); + let latency = children.plan.resources().latency_samples; + submit_direct(&mut children, &[b"fixture-source"], 0); + feed_and_render_direct(&mut children, &[b"fixture-source"], 0, 6); + command_direct(&mut children, &transaction_at(1, 42, &edits)); + feed_and_render_direct(&mut children, &[b"fixture-source"], 6, 4); + assert_eq!(children.plan.resources().latency_samples, latency); + let before = children.plan.queries().watermark().expect("watermark"); + + children + .session + .declare_discontinuity() + .expect("the declaration"); + let declared = children.session.newest_plan_epoch(); + children + .session + .declare_discontinuity() + .expect("a second declaration"); + assert_eq!( + children.session.newest_plan_epoch(), + declared, + "the waiting discontinuity successor has nothing to reset" + ); + let counters = children.session.test_owner_counters(); + let edit = add_muted_track( + children + .session + .test_controller() + .session() + .compiled() + .normalized_model(), + "a-muted", + ); + command_direct(&mut children, &transaction_at(2, 43, &edit)); + let after = children.session.test_owner_counters(); + assert_eq!(after.candidate_withdrawn, counters.candidate_withdrawn + 1); + assert_eq!( + after.candidate_superseded, + counters.candidate_superseded + 1 + ); + assert!( + !children.session.test_pending_providers()[0] + .test_inventory() + .runs_behind_natural_timing(), + "E has no raised floor" + ); + + let rendered = feed_and_render_direct(&mut children, &[b"fixture-source"], ADOPTED_AT, 6); + let watermark = children.plan.queries().watermark().expect("watermark"); + assert_eq!( + (watermark.revision, watermark.first_sample), + (44, ADOPTED_AT * 128) + ); + assert_eq!(watermark.outcome_flags, engine::realtime::OUTCOME_EXACT); + assert_eq!(watermark.superseded, before.superseded); + let (natural, expected) = natural_reference(&committed_document(&children), ADOPTED_AT, 6); + assert!(natural < latency); + assert_eq!(children.plan.resources().latency_samples, natural); + assert!(expected.iter().flatten().any(|sample| *sample != 0.0)); + assert_eq!(sample_bits(&rendered), sample_bits(&expected)); +} + /// Issue #1273 gate 3: a structural transaction refused at any phase, including a protocol /// commit that returns `Err`, moves no producer. The source still accepts PCM and the running /// plan renders it, bit-identical to a session that never saw the transaction. @@ -3061,10 +4095,7 @@ fn assert_live_lanes_render_like_lanes_free(document: &str, label: &str, expect_ // Same latency and tail as the lanes-free plan. let resources = c.plan.resources(); - let (tail_kind, tail_samples) = match host.output_tail { - TailSamples::Finite(samples) => (TAIL_FINITE, samples), - TailSamples::Infinite => (TAIL_INFINITE, 0), - }; + let (tail_kind, tail_samples) = control_plane::tail_words(host.output_tail); if expect_finite { assert_eq!( tail_kind, TAIL_FINITE, @@ -3087,7 +4118,8 @@ fn assert_live_lanes_render_like_lanes_free(document: &str, label: &str, expect_ ); } else { assert_eq!( - resources.tail_kind, TAIL_INFINITE, + resources.tail_kind, + crate::TAIL_INFINITE, "{label}: an infinite-tail effect keeps the plan's tail infinite" ); } @@ -3110,10 +4142,7 @@ fn assert_live_lanes_render_like_lanes_free(document: &str, label: &str, expect_ ); assert_eq!( (resources.tail_kind, resources.tail_samples), - match without_effects.output_tail { - TailSamples::Finite(samples) => (TAIL_FINITE, samples), - TailSamples::Infinite => (TAIL_INFINITE, 0), - }, + control_plane::tail_words(without_effects.output_tail), "{label}: tail without effect lanes" ); diff --git a/crates/capi/tests/c/abi_smoke.c b/crates/capi/tests/c/abi_smoke.c index cfe1e0ece..0b148147c 100644 --- a/crates/capi/tests/c/abi_smoke.c +++ b/crates/capi/tests/c/abi_smoke.c @@ -20,7 +20,8 @@ ABI_ASSERT(MISO_ENGINE_V1_TAIL_FINITE == 0); ABI_ASSERT(MISO_ENGINE_V1_TAIL_INFINITE == 1); ABI_ASSERT(MISO_ENGINE_V1_EXACT_LAUNCH_RATE_MASK == 15); ABI_ASSERT(MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_AT == 32); -ABI_ASSERT(MISO_ENGINE_V1_FEATURE_MASK == 255); +ABI_ASSERT((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY) == + MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY); ABI_ASSERT((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_SERVICE) == MISO_ENGINE_V1_FEATURE_SERVICE); ABI_ASSERT((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_PLAN_WATERMARK) == @@ -121,6 +122,8 @@ static uint32_t (*const watermark_signature)(const miso_engine_v1_plan *, miso_engine_v1_watermark *) = miso_engine_v1_plan_watermark; static uint32_t (*const service_signature)(miso_engine_v1_session *) = miso_engine_v1_service; +static uint32_t (*const declare_discontinuity_signature)(miso_engine_v1_session *) = + miso_engine_v1_declare_discontinuity; static uint32_t (*const last_error_signature)(const void *, miso_engine_v1_bytes_out *) = miso_engine_v1_last_error; static void (*const session_destroy_signature)(miso_engine_v1_session *) = @@ -250,11 +253,19 @@ int main(void) { return 6; } + /* A host checks the declared-discontinuity bit before calling the symbol (issue 1323); a null + * session is refused. */ + if ((capabilities.feature_mask & MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY) == 0 || + declare_discontinuity_signature(NULL) != MISO_ENGINE_V1_INVALID_ARGUMENT) { + return 7; + } + return compile_session_signature == NULL || source_submit_signature == NULL || source_seek_signature == NULL || submit_command_signature == NULL || dequeue_event_signature == NULL || render_signature == NULL || resources_signature == NULL || watermark_signature == NULL || service_signature == NULL || + declare_discontinuity_signature == NULL || last_error_signature == NULL || session_destroy_signature == NULL || plan_destroy_signature == NULL ? 3 diff --git a/crates/capi/tests/c/header_smoke.cpp b/crates/capi/tests/c/header_smoke.cpp index 936c0b92a..5eb2d35b3 100644 --- a/crates/capi/tests/c/header_smoke.cpp +++ b/crates/capi/tests/c/header_smoke.cpp @@ -25,5 +25,9 @@ static_assert((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_PLAN_WATERMA MISO_ENGINE_V1_FEATURE_PLAN_WATERMARK); static_assert((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_SERVICE) == MISO_ENGINE_V1_FEATURE_SERVICE); +static_assert((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY) == + MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY); +static_assert(std::is_same::value); static_assert(std::is_same::value); diff --git a/crates/capi/tests/resource_lifecycle.rs b/crates/capi/tests/resource_lifecycle.rs index abd49d748..ba5d22e55 100644 --- a/crates/capi/tests/resource_lifecycle.rs +++ b/crates/capi/tests/resource_lifecycle.rs @@ -797,10 +797,8 @@ fn observe_compile(document: &str, compile_limits: &CompileLimits) -> CompileObs sources, report: host_report, inventory, - carried_base, control_catalog, } = prepared; - assert!(carried_base.is_none(), "a boot plan has no carried base"); let owners = HostOwners { routes: freed_by_drop(routes), effects: freed_by_drop(effects), @@ -1233,6 +1231,17 @@ impl Budget { /// 168,810 after (x86-64, +2,888: one more 1,648-byte provider-epoch slot and 248 bytes in each of /// the five slots; the row does not depend on the lane width), within the 175,296 ceiling, so the /// ceiling stays. +/// +/// #1698 (batch 15b) removes `ProviderEpoch::carried_base` (a replacement across a withdrawn +/// candidate is now prepared from the running plan's own inventory). `capi_retained_bytes` +/// measured 168,810 before and 167,514 after (x86-64, -1,296: one 216-byte +/// `Option` in each of the six provider epochs, the current one, two pending +/// and three retired), within the 175,296 ceiling, so the ceiling stays. +/// +/// #1285 (batch 15b) adds the plan state inventory's `input_floors` and `raised_arrivals` +/// tables. `capi_retained_bytes` measured 167,514 before and 167,754 after (x86-64, +240: the +/// two tables in the inventory the provider epochs keep), within the 175,296 ceiling, so the +/// ceiling stays. const REFERENCE_BUDGETS: [Budget; 19] = [ Budget { row: "graph_session_plus_plan_bytes", @@ -1781,14 +1790,11 @@ fn double_live_oracle_drives_exact_and_one_below_c_caps() { effects: true, routes: true, }, - host_core::SuccessorBase { - inventory: ¤t_host.prepared.inventory, - committed: current_host.store.compiled().normalized_model(), - revision: 1, - edit_ramps: &session::EditRamps::new(), - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }, + host_core::SuccessorBase::new( + ¤t_host.prepared.inventory, + current_host.store.compiled().normalized_model(), + 1, + ), &host_core::LiveValuesSuperseded::default(), ) .unwrap_or_else(|_| panic!("the prospective session prepares as a successor")) @@ -1826,12 +1832,9 @@ fn double_live_oracle_drives_exact_and_one_below_c_caps() { // strips the swap restarts (#1277 D6, #1279 D1). This session's console slots have live // effect lanes, which the effect carry does not move (#1280), so every strip restarts and the // inventory is that much larger than `capi_epoch_terms` read off the fresh preparation. - // It also keeps the base a replacement would be prepared from were it withdrawn unadopted - // (#1310 D2, #1344 D1), which a fresh preparation has none of. let prospective_epoch = prospective_epoch - prospective_host.prepared.report.inventory_retained_bytes - + successor.inventory_retained_bytes - + successor.carried_base_retained_bytes; + + successor.inventory_retained_bytes; let rows: [(&str, u64); 8] = [ ( "graph", diff --git a/crates/control-plane/src/adapter.rs b/crates/control-plane/src/adapter.rs index 552911d2a..dd887f457 100644 --- a/crates/control-plane/src/adapter.rs +++ b/crates/control-plane/src/adapter.rs @@ -89,6 +89,22 @@ pub struct ControlLimits { pub maximum_replacement_peak_bytes: u64, } +/// The tail kind word of a tail that terminates after its sample count (#1688 D1). +pub const TAIL_FINITE: u64 = 0; +/// The tail kind word of an unbounded tail, which reports zero samples (#1688 D1). +pub const TAIL_INFINITE: u64 = 1; + +/// The two-word encoding of an output tail every host reports: `(kind, samples)` with +/// [`TAIL_FINITE`] and the sample count, or [`TAIL_INFINITE`] and zero. The C ABI's report and +/// the browser's plan-latency record both call it, so they cannot encode a tail differently. +#[must_use] +pub const fn tail_words(tail: TailSamples) -> (u64, u64) { + match tail { + TailSamples::Finite(samples) => (TAIL_FINITE, samples), + TailSamples::Infinite => (TAIL_INFINITE, 0), + } +} + /// One plan's frozen resource accounting (#1309 D4): every field of the C ABI's /// `PlanResourceReport` except its ABI header and reserved words, with the output tail as /// [`TailSamples`] and the control plane's own retained bytes as `control_retained_bytes`. diff --git a/crates/control-plane/src/compile.rs b/crates/control-plane/src/compile.rs index 70bdbcdec..cbecca672 100644 --- a/crates/control-plane/src/compile.rs +++ b/crates/control-plane/src/compile.rs @@ -23,10 +23,6 @@ pub(crate) struct PreparedRuntime { pub(crate) carried: CarriedSourceBytes, /// What the plan holds, for preparing its successor (issue #1273 D1). pub(crate) inventory: PlanStateInventory, - /// What the plan carries from the plan it was prepared to displace (#1344 D1), the base its - /// replacement is prepared from should it be withdrawn before render adopts it (#1310 D2). - /// `None` for a plan prepared without a successor base. - pub(crate) carried_base: Option, pub(crate) control_catalog: ControlCatalog, pub(crate) capi: CapiResources, /// The designated master strip the plan was prepared with, as a strip index (#1683): a first @@ -254,7 +250,6 @@ pub(crate) fn capi_resources( strip_table_bytes: u64, provider: host_core::SessionControlProviderResources, inventory_bytes: u64, - carried_base_bytes: u64, effect_controls: host_core::EffectControlResources, route_controls: host_core::RouteControlResources, observation: ObservationResources, @@ -323,9 +318,6 @@ pub(crate) fn capi_resources( source_id_row, // Issue #1273 D5: each epoch keeps its plan's state inventory for the successor. inventory_bytes, - // #1310 D2 (#1344 D1): and the base a successor is prepared from should this epoch's plan - // be withdrawn before render adopts it; zero for a plan prepared without a base. - carried_base_bytes, strip_table_bytes, effect_row, route_row, @@ -454,7 +446,6 @@ pub(crate) fn prepared_capi_resources( compiled: &CompiledSession, catalog: &ControlCatalog, inventory: &PlanStateInventory, - carried_base_bytes: u64, effect_controls: host_core::EffectControlResources, route_controls: host_core::RouteControlResources, observation: ObservationResources, @@ -494,7 +485,6 @@ pub(crate) fn prepared_capi_resources( .ok_or_else(|| failure(ResourceFault::Arithmetic))?, provider, inventory.retained_bytes(), - carried_base_bytes, effect_controls, route_controls, observation, @@ -1209,7 +1199,6 @@ pub(crate) fn prepare_runtime( compiled, &prepared.control_catalog, &prepared.inventory, - prepared.report.carried_base_retained_bytes, prepared.report.effect_control_resources, prepared.report.route_control_resources, observation_resources(&effects, observation_slots_high_water)?, @@ -1283,7 +1272,6 @@ pub(crate) fn prepare_runtime( }, carried, inventory: prepared.inventory, - carried_base: prepared.carried_base, control_catalog: prepared.control_catalog, capi, master_track, @@ -1374,8 +1362,6 @@ where resources, carried: _, inventory, - // A boot plan is prepared without a successor base, so it has none. - carried_base: _, control_catalog, capi, master_track: _, diff --git a/crates/control-plane/src/control.rs b/crates/control-plane/src/control.rs index 8f7ce18b9..b226abbfc 100644 --- a/crates/control-plane/src/control.rs +++ b/crates/control-plane/src/control.rs @@ -56,10 +56,6 @@ pub struct ProviderEpoch { /// the epoch's life and moves only with the epoch, which holds no open stamp between /// commands. pub(crate) revision_gate: PlanRevisionGate, - /// What this epoch's plan carries from the plan it was prepared to displace (#1344 D1): the - /// base inventory of a replacement prepared across this plan should it be withdrawn before - /// render adopts it (#1310 D2). `None` for the boot plan. - pub(crate) carried_base: Option, /// The part of this epoch's plan's source rows it carries from the plan it was prepared to /// displace (#1273 D5), kept for the replacement admission while the epoch is a held /// candidate (#1398 D4, batch 15a review fold M2). Zero for the boot plan. @@ -78,6 +74,10 @@ pub struct ProviderEpoch { /// reclaims it, so both admissions charge it from here until then. `None` while the epoch is /// running or pending. pub(crate) parked: Option, + /// This epoch's plan was prepared at a host-declared discontinuity (#1323 D2): no floors, only + /// its source consumers carried. A successor that supersedes it unadopted is prepared at the + /// same discontinuity, since render never played this plan. + pub(crate) discontinuity: bool, } /// The committed base an epoch keeps while a successor prepared from it is pending (#1310 D3, @@ -100,6 +100,20 @@ pub(crate) struct KeptBase { pub(crate) monitor_ramp: Option>, } +/// What a rebuild publishes a successor for (#1323). +enum Rebuild { + /// A structural transaction: the successor is compiled from the token's prospective model and + /// the protocol commit acknowledges it. + Transaction(ObservedPreparedToken), + /// A host-declared discontinuity (#1323 D2): the successor is the committed model, of which + /// this is the control plane's own copy, prepared with no floors and only its source + /// consumers carried. Nothing is committed. + Discontinuity(Box), +} + +/// The edit ramps of a discontinuity successor: no edit gave a row a length (#1323). +static NO_EDIT_RAMPS: session::EditRamps = session::EditRamps::new(); + /// `earlier`'s ramp lengths with `later`'s laid over them: `later` wins on every row it wrote, with /// or without a length (`EditRamps::overlay`). The /// merge of the kept base's lengths with a newer transaction's (#1310, root ruling B15-Q3). @@ -123,6 +137,12 @@ impl ProviderEpoch { (self.carried.total, self.carried.overhead) } + /// What this epoch's plan holds, for preparing its successor (#1323's gates read its floors). + #[cfg(feature = "test-support")] + pub fn test_inventory(&self) -> &PlanStateInventory { + &self.inventory + } + /// This epoch's source producers. #[cfg(feature = "test-support")] pub fn test_sources(&self) -> &SourceControlSet { @@ -186,10 +206,10 @@ impl ProviderEpoch { effects, capi, revision_gate, - carried_base: None, carried: CarriedSourceBytes::default(), kept: None, parked: None, + discontinuity: false, }; #[cfg(feature = "test-support")] update_test_owners(|owners| owners.current_provider_constructed += 1); @@ -200,7 +220,6 @@ impl ProviderEpoch { pub(crate) fn candidate( sources: SourceControlSet, inventory: PlanStateInventory, - carried_base: Option, carried: CarriedSourceBytes, strips: StripLanes, effects: Box<[host_core::EffectControlProducer]>, @@ -215,10 +234,10 @@ impl ProviderEpoch { effects, capi, revision_gate, - carried_base, carried, kept: None, parked: None, + discontinuity: false, }; #[cfg(feature = "test-support")] update_test_owners(|owners| owners.candidate_provider_constructed += 1); @@ -2045,7 +2064,7 @@ where // move while the control thread owns them. let mut withdrawn = self.withdraw_pending_candidate()?; let result = self.rebuild( - prepared, + Rebuild::Transaction(prepared), revision, monitor_change, observation_change, @@ -2064,6 +2083,84 @@ where } } + /// Declares a discontinuity (#1323 D1, D2): the next block the host renders need not continue + /// the previous output, because the host has stopped or is about to seek every source. Both + /// hosts call this one function: `miso_engine_v1_declare_discontinuity` and + /// `miso_engine_web_v1_declare_discontinuity` only forward to it. + /// + /// After the service step, it rebuilds only if the newest plan (the pending candidate, else + /// the running plan) runs behind its natural timing + /// ([`PlanStateInventory::runs_behind_natural_timing`]). It then prepares the committed model + /// as a successor at a discontinuity (`SuccessorBase::discontinuity`): no floors, only the + /// source consumers carried, every other owner at rest. It is published + /// [`PlanAdoption::Next`] with the committed revision and its retirement credit reserved, as a + /// structural transaction's plan is. A pending candidate is superseded first (#1310): withdrawn, + /// the donor of the rings it created, its persisting producers returned to the running epoch, + /// then dropped; its revisions complete when the successor is adopted, `EXACT`, with + /// `SUPERSEDED` for the revisions it had itself folded in (D4), never as a fallback. A `Primed` + /// warm candidate always runs behind its natural timing, so a declaration always supersedes + /// one. Otherwise it publishes nothing and returns `Ok`; so does a session at revision 0, + /// which has no successor and so no floor (root ruling B15-R6). + /// + /// **Acked-batch question (D4): can an ack ever precede a drop? No.** The declaration + /// acknowledges no edit and changes neither the committed model nor the revision. A refusal + /// (a preparation or admission error, backpressure) changes nothing: a withdrawn candidate is + /// published again whole, and the running plan keeps its floors. A superseded candidate's + /// edits are in the committed model the successor is compiled from, and its fed rings move + /// by the checked donation or the supersession is refused (`drops_acked_fresh_source`). + /// + /// # Errors + /// + /// [`CommandError::CompileRejected`] when the preparation or the replacement admission + /// refuses; [`CommandError::Backpressure`] when the plan is gone, when a retry can fit, or + /// when a superseded candidate's fed fresh ring could not move; [`CommandError::Internal`] + /// for an accounting fault. + pub fn declare_discontinuity(&mut self) -> Result<(), CommandError> { + self.service()?; + if !self + .newest_providers() + .inventory + .runs_behind_natural_timing() + { + return Ok(()); + } + if !self.shared.plan_alive.load(Ordering::Acquire) { + return Err(CommandError::Backpressure); + } + // D4: the successor carries the committed revision. A plan behind its natural timing is a + // successor, so the committed revision is at least 1 (B15-R6). + let revision = self.controller.session().revision().0; + if !(1..=engine::realtime::GATE_REVISION_MAX).contains(&revision) { + return Err(CommandError::Internal); + } + let model = Box::new(self.controller.session().compiled().clone()); + // Nothing changes the overlay or the subscriptions: the successor renders under the ones + // the newest plan holds, and every binding keeps its target. + let monitor_change = self.monitor.after(&[], model.normalized_model()); + let observation_change = self + .observation + .after(model.normalized_model()) + .map_err(|_| CommandError::CompileRejected(failure(ResourceFault::Allocation)))?; + if !observation_change.ended.is_empty() { + return Err(CommandError::Internal); + } + let mut withdrawn = self.withdraw_pending_candidate()?; + let result = self.rebuild( + Rebuild::Discontinuity(model), + revision, + monitor_change, + observation_change, + &mut withdrawn, + ); + // #1310 D4: as for a transaction, a refusal republishes the withdrawn candidate whole. + if let Some(candidate) = withdrawn.take() { + self.publisher.republish(candidate); + #[cfg(feature = "test-support")] + update_test_owners(|owners| owners.candidate_republished += 1); + } + result.map(|_| ()) + } + /// Takes back the newest pending candidate if render has not adopted it (#1310 D1). `None` /// while nothing is pending, or when render adopted it first (`Taken`), which is the path a /// rebuild on the newest plan has always taken. @@ -2125,10 +2222,18 @@ where /// back to P0 before A drops, and every observation record queued on A is queued on B before /// B is published. B is acked only after every fallible step; A's revisions complete as /// `superseded` when B is adopted (D6). + /// + /// **A declared discontinuity (#1323).** With [`Rebuild::Discontinuity`] the successor is the + /// committed model prepared at a discontinuity: no floors, only its source consumers carried, + /// published [`PlanAdoption::Next`]. Nothing is committed: the model and the revision do not + /// change, and the successor carries the committed revision. A successor that supersedes a + /// discontinuity successor unadopted is prepared at the same discontinuity: render never + /// played the withdrawn plan, so the pair is the one transaction "declare, then edit" against + /// the running plan. #[allow(clippy::too_many_lines)] fn rebuild( &mut self, - prepared: ObservedPreparedToken, + kind: Rebuild, revision: u64, monitor_change: controls::MonitorChange, observation_change: controls::ObservationChange, @@ -2141,9 +2246,23 @@ where // The mailbox's last publication is the newest pending candidate by construction. return Err(CommandError::Internal); } - let own_ramps = prepared.get().prospective_session().edit_ramps(); + let declared = matches!(kind, Rebuild::Discontinuity(_)); + // #1323: a successor of a withdrawn discontinuity successor is prepared at it too. + let discontinuity = declared + || (superseding + && self + .pending_providers + .last() + .is_some_and(|newest| newest.discontinuity)); + let own_ramps = match &kind { + Rebuild::Transaction(prepared) => prepared.get().prospective_session().edit_ramps(), + Rebuild::Discontinuity(_) => &NO_EDIT_RAMPS, + }; // Whether B writes a monitor row: then its overlay change's ramp is the latest. - let writes_monitor = !prepared.get().monitor_edits().is_empty(); + let writes_monitor = match &kind { + Rebuild::Transaction(prepared) => !prepared.get().monitor_edits().is_empty(), + Rebuild::Discontinuity(_) => false, + }; // The ramp lengths the epoch B succeeds keeps from now on: B's own, or, across a // withdrawn candidate, the kept base's with B's laid over them (B wins on a shared row). let kept_ramps = if superseding { @@ -2155,26 +2274,39 @@ where } else { own_ramps.clone() }; - let prospective = prepared - .get() - .prospective_session() - .compiled() - .ok_or(CommandError::Internal)?; - let base = if superseding { - // #1310 D2: A's carried base (every row a P0 row) and P0's kept base. + let prospective = match &kind { + Rebuild::Transaction(prepared) => prepared + .get() + .prospective_session() + .compiled() + .ok_or(CommandError::Internal)?, + Rebuild::Discontinuity(model) => &**model, + }; + // #1698 (amending #1310 D2 and #1344 D1): across a withdrawn candidate A, P0's own + // inventory, with only the source rows A carries from P0, and P0's kept base. An owner B + // keeps unchanged from P0 carries from P0 as a direct P0 -> B swap carries it; a source A + // added or redeclared restarted the host's feed against A, so B starts it fresh. + let withdrawn_base = match withdrawn.as_ref() { + Some(candidate) => { + let holder = self.kept_holder().ok_or(CommandError::Internal)?; + let donor = self + .pending_providers + .last() + .ok_or(CommandError::Internal)?; + Some( + holder + .inventory + .across_withdrawn(&donor.inventory, candidate.plan()), + ) + } + None => None, + }; + let base = if let Some(inventory) = withdrawn_base.as_ref() { let kept = self .kept_holder() .and_then(|holder| holder.kept.as_ref()) .ok_or(CommandError::Internal)?; - let inventory = self - .pending_providers - .last() - .and_then(|candidate| candidate.carried_base.as_ref()) - .ok_or(CommandError::Internal)?; SuccessorBase { - inventory, - committed: kept.model.normalized_model(), - revision, edit_ramps: &kept_ramps, monitor: host_core::MonitorBase { committed: &kept.monitor, @@ -2186,14 +2318,13 @@ where }, }, observation: &observation_change.bindings, + discontinuity, + ..SuccessorBase::new(inventory, kept.model.normalized_model(), revision) } } else { // Issue #1273 D1: the candidate succeeds the newest plan, so every source the // transaction leaves unchanged is prepared to carry that plan's ring. SuccessorBase { - inventory: &self.newest_providers().inventory, - committed: self.controller.session().compiled().normalized_model(), - revision, // #1394 D6: a carry retarget is the record of the transaction's own edit, so it // takes that edit's ramp length. edit_ramps: own_ramps, @@ -2203,6 +2334,12 @@ where ramp: monitor_change.ramp, }, observation: &observation_change.bindings, + discontinuity, + ..SuccessorBase::new( + &self.newest_providers().inventory, + self.controller.session().compiled().normalized_model(), + revision, + ) } }; let prepared_runtime = prepare_runtime::( @@ -2222,7 +2359,6 @@ where resources, carried, inventory, - carried_base, control_catalog: candidate_catalog, capi: prospective_capi, revision_gate, @@ -2231,20 +2367,20 @@ where let mut candidate_provider = ProviderEpoch::candidate( sources, inventory, - carried_base, carried, strips, effects, prospective_capi, revision_gate, ); + candidate_provider.discontinuity = discontinuity; let candidate_plan = ObservedCandidatePlan::new(candidate_plan); #[cfg(feature = "test-support")] { if self.take_test_fault(TestStructuralFaultPhase::AfterRuntimePrepare) { drop(candidate_plan); drop(candidate_provider); - drop(prepared); + drop(kind); return Err(CommandError::Backpressure); } } @@ -2292,14 +2428,20 @@ where if self.take_test_fault(TestStructuralFaultPhase::AfterAdmission) { drop(candidate_plan); drop(candidate_provider); - drop(prepared); + drop(kind); return Err(CommandError::Backpressure); } - // #1310 D1: B is published with the adoption of its own classification, never A's. + // #1310 D1: B is published with the adoption of its own classification, never A's. A + // discontinuity successor is always `Next` (#1323 D2): no lead, readiness, deadline or + // transition. #[cfg(feature = "test-support")] - let adoption = TEST_NEXT_ADOPTION - .with(core::cell::Cell::take) - .unwrap_or(PlanAdoption::Next); + let adoption = if declared { + PlanAdoption::Next + } else { + TEST_NEXT_ADOPTION + .with(core::cell::Cell::take) + .unwrap_or(PlanAdoption::Next) + }; #[cfg(not(feature = "test-support"))] let adoption = PlanAdoption::Next; let reservation = match self @@ -2333,7 +2475,7 @@ where if take_test_fault_state(TestStructuralFaultPhase::AfterPlanReservation) { drop(reservation); drop(candidate_provider); - drop(prepared); + drop(kind); return Err(CommandError::Backpressure); } } @@ -2369,11 +2511,12 @@ where // #1310, root rulings of 2026-10-16 (no ack before a drop; M-A of fold 3): a ring A // created (A added or redeclared the source) that the host fed and the donation above does // not move to B would drop with A, and with it PCM the host was acked for. Until an issue - // lets B read it (#1698 for a source B reverts to its P0 form), such a supersession is - // refused here as backpressure. The guard reads the donation plan itself, so it sits after - // the check, still before the commit: the reservation it follows is cancelled by the - // return (it returns its credit and consumes no epoch), nothing else has changed, the - // caller republishes A, and once render adopts A the same edit commits on it. + // lets B read it, such a supersession is refused here as backpressure (#1698 keeps the + // reverted case refused: B starts a reverted source fresh, as the host contract says). The + // guard reads the donation plan itself, so it sits after the check, still before the + // commit: the reservation it follows is cancelled by the return (it returns its credit and + // consumes no epoch), nothing else has changed, the caller republishes A, and once render + // adopts A the same edit commits on it. if let (Some(plan), Some(candidate)) = (donation.as_ref(), withdrawn.as_ref()) { let donor = self .pending_providers @@ -2431,7 +2574,7 @@ where drop(reports); drop(reservation); drop(candidate_provider); - drop(prepared); + drop(kind); return Err(CommandError::Backpressure); } @@ -2446,7 +2589,7 @@ where drop(reports); drop(reservation); drop(candidate_provider); - drop(prepared); + drop(kind); return Err(CommandError::Internal); } // #1584 D6a: the newest plan's pending-subscription storage is built here, with the other @@ -2460,26 +2603,36 @@ where drop(reports); drop(reservation); drop(candidate_provider); - drop(prepared); + drop(kind); return Err(CommandError::CompileRejected(failure( ResourceFault::Allocation, ))); }; // #1310 D4: the protocol's own commit predicate, the last check. - if self - .controller - .check_prepared_structural(prepared.get()) - .is_err() + if let Rebuild::Transaction(prepared) = &kind + && self + .controller + .check_prepared_structural(prepared.get()) + .is_err() { return Err(CommandError::Internal); } - // #1313 D2: a transaction that takes this arm is a rebuild, whatever else it holds. - let mut committed = prepared - .commit(&mut self.controller, protocol::EditPath::Rebuild) - .map_err(|_| CommandError::Internal)?; - // #1310 D3: the model the commit displaced, never one committed in place (#1613). - let displaced = committed.take_displaced(); - debug_assert!(displaced.is_some(), "a rebuild displaces a compiled model"); + let (committed, displaced) = match kind { + Rebuild::Transaction(prepared) => { + // #1313 D2: a transaction that takes this arm is a rebuild, whatever else it holds. + let mut committed = prepared + .commit(&mut self.controller, protocol::EditPath::Rebuild) + .map_err(|_| CommandError::Internal)?; + // #1310 D3: the model the commit displaced, never one committed in place (#1613). + let displaced = committed.take_displaced(); + debug_assert!(displaced.is_some(), "a rebuild displaces a compiled model"); + (Some(committed), displaced) + } + // #1323 D4: nothing is committed. The copy of the committed model is the base the + // epoch this successor displaces keeps (the model that epoch renders), or, across a + // withdrawn candidate, drops below as a commit's displaced model does. + Rebuild::Discontinuity(model) => (None, Some(*model)), + }; // #1559 D8: the overlay is written after the last fallible step, with the commit. let monitor_ramp = writes_monitor.then_some(monitor_change.ramp); let previous_monitor = core::mem::replace(&mut self.monitor, monitor_change.overlay); @@ -2632,9 +2785,11 @@ where #[cfg(feature = "test-support")] update_test_owners(|owners| owners.candidate_superseded += 1); } - Ok(committed - .write_into(&mut self.response_scratch) - .expect("prepared response capacity was admitted before protocol commit")) + Ok(committed.map_or(0, |committed| { + committed + .write_into(&mut self.response_scratch) + .expect("prepared response capacity was admitted before protocol commit") + })) } /// The live arm of a structural command (#1257 D2, #1053 D6). diff --git a/crates/control-plane/src/lib.rs b/crates/control-plane/src/lib.rs index e3e10ccc0..bf22eadf3 100644 --- a/crates/control-plane/src/lib.rs +++ b/crates/control-plane/src/lib.rs @@ -58,7 +58,9 @@ mod control; mod error; mod plan; -pub use adapter::{ControlAdapter, ControlLimits, PlanResources}; +pub use adapter::{ + ControlAdapter, ControlLimits, PlanResources, TAIL_FINITE, TAIL_INFINITE, tail_words, +}; pub use compile::{CompiledChildren, RuntimePreparer, compile_children}; pub use control::{ CommandError, EventError, EventLane, ObservationChange, ObservationForward, diff --git a/crates/graph-compiler/src/bin/graph_fixture.rs b/crates/graph-compiler/src/bin/graph_fixture.rs index f1b547169..109245c94 100644 --- a/crates/graph-compiler/src/bin/graph_fixture.rs +++ b/crates/graph-compiler/src/bin/graph_fixture.rs @@ -132,6 +132,7 @@ fn compile_fixture() -> PreparedGraphBuiltinsArtifact { ) .unwrap_or_else(|diagnostics| panic!("builtin diagnostics: {diagnostics:?}")); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: FIXTURE_DISPATCH.expect(NO_FIXTURE_WIDTH), plan_id: 0, effects: EffectPreparedSession { diff --git a/crates/graph-compiler/src/compile.rs b/crates/graph-compiler/src/compile.rs index 3921cbc36..27a159c72 100644 --- a/crates/graph-compiler/src/compile.rs +++ b/crates/graph-compiler/src/compile.rs @@ -84,6 +84,7 @@ impl GraphCompiler { builtins, caps, dispatch, + input_floors, } = request; let builtin_diagnostics = builtins.validate_for_session(&effects.session); if !builtin_diagnostics.0.is_empty() { @@ -99,16 +100,17 @@ impl GraphCompiler { ), }); } - let compiled = match Self::compile_graph(plan_id, effects, caps, dispatch, &builtins) { - Ok(value) => value, - Err(failure) => { - return Err(GraphBuiltinsCompileFailure { - effects: failure.effects, - builtins, - diagnostics: failure.diagnostics, - }); - } - }; + let compiled = + match Self::compile_graph(plan_id, effects, caps, dispatch, &builtins, &input_floors) { + Ok(value) => value, + Err(failure) => { + return Err(GraphBuiltinsCompileFailure { + effects: failure.effects, + builtins, + diagnostics: failure.diagnostics, + }); + } + }; let levels = compiled.graph.dependency_levels.clone(); Ok(builtins.into_graph_artifact_with_banks( compiled.graph, @@ -134,6 +136,7 @@ impl GraphCompiler { caps: GraphCompileCaps, dispatch: Backend, builtins: &PreparedBuiltinsSession, + input_floors: &BTreeMap, ) -> Result { // `validate_for_session` has checked the tail set against the session's tracks, so every // track has exactly one entry here. @@ -419,7 +422,14 @@ impl GraphCompiler { vec![diag("graph.resource.limit", "$.graph_compile_caps")], )); } - let timing = match timings(&schedule, &edges, &node_latency, &node_tail, &caps) { + let timing = match timings( + &schedule, + &edges, + &node_latency, + &node_tail, + input_floors, + &caps, + ) { Ok(value) => value, Err(diagnostic) => return Err(failure(effects, vec![diagnostic])), }; @@ -863,6 +873,8 @@ impl GraphCompiler { report: GraphCompileReport { output_latency: timing.output_latency, output_tail: timing.output_tail, + input_arrivals: timing.input_arrivals.into_boxed_slice(), + raised_arrivals: timing.raised_arrivals.into_boxed_slice(), semantic_estimate, estimate, rack_cohorts, diff --git a/crates/graph-compiler/src/lib.rs b/crates/graph-compiler/src/lib.rs index 7667b9092..1753516b2 100644 --- a/crates/graph-compiler/src/lib.rs +++ b/crates/graph-compiler/src/lib.rs @@ -54,6 +54,12 @@ pub struct GraphBuiltinsCompileRequest { /// schedule, levels, PDC, reductions, canonical bytes -- is deliberately independent of this /// value; only the bank overlay and the bank half of the estimate depend on it. pub dispatch: Backend, + /// A lower bound on the input arrival of a node, in samples (#1285 D1). PDC takes a node's + /// `max` as the larger of its latest incoming arrival and its floor, and compensates every + /// incoming edge to it, so a successor keeps every surviving node's input timing. A floor that + /// names no node of this graph is ignored. Empty for a plan with no predecessor: with no + /// floors, neither the program nor its canonical bytes change. + pub input_floors: BTreeMap, } /// The one-way, sealed builtin attachment result. /// @@ -182,6 +188,15 @@ pub struct GraphCompileReport { pub output_latency: LatencySamples, /// Propagated extent of the sole session output after latency and declared tails. pub output_tail: TailSamples, + /// The input arrival (the floored `max` of PDC) of every node at which it is nonzero, sorted + /// by node ID (#1285 D2). A node absent here has input arrival 0. A successor's preparation + /// passes this back as [`GraphBuiltinsCompileRequest::input_floors`]. + pub input_arrivals: Box<[(GraphNodeId, LatencySamples)]>, + /// The natural input arrival (the latest incoming arrival, before the floor) of every node + /// whose [`GraphBuiltinsCompileRequest::input_floors`] floor raised it, sorted by node ID + /// (#1323 D3). Its floored arrival is the node's row in [`Self::input_arrivals`]. Empty when no + /// floor raised any node: the plan then runs at its natural timing. + pub raised_arrivals: Box<[(GraphNodeId, LatencySamples)]>, /// The retained estimate, including the bank overlay and capped against the session limits. pub estimate: GraphResourceEstimate, /// The pre-bank estimate that participates in the semantic hash. Dispatch-independent by @@ -2148,6 +2163,7 @@ mod tests { let builtins = prepare_session_builtins(&effects.session, &[], unbounded_builtin_caps()) .expect("the session's builtins prepare"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), plan_id, effects, builtins, @@ -2971,6 +2987,7 @@ mod tests { } .expect("prepared scalar builtins"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::Scalar, plan_id: if controlled { 443_001 } else { 443_000 }, effects: EffectPreparedSession { @@ -3648,6 +3665,7 @@ mod tests { // scalar dispatch is the one production compile that has no slot owner to charge, so the // host compile below must publish exactly its estimate plus the literal reservation. let semantic_baseline = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::Scalar, plan_id: 5_115, effects: EffectPreparedSession { @@ -3703,6 +3721,7 @@ mod tests { expected.session_plus_plan_bytes += slot_total; expected.largest_allocation_bytes = expected.largest_allocation_bytes.max(slot_largest); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: 5_112, effects: EffectPreparedSession { @@ -3731,6 +3750,7 @@ mod tests { // reservation and the attached builtin payload. Until #964 an effect-only, builtins-less // compile at the host dispatch supplied the prior owners and the effect-bank term together. let scalar_mixed = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::Scalar, plan_id: 5_117, effects: { @@ -3802,6 +3822,7 @@ mod tests { .max(combined_largest); let mixed = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: 5_116, effects: { @@ -3868,6 +3889,7 @@ mod tests { let prepared = prepare_session_builtins(&effects.session, &[], builtin_caps) .expect("prepared for cap"); let exact_result = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: 5_113, effects, @@ -3886,6 +3908,7 @@ mod tests { let prepared = prepare_session_builtins(&effects.session, &[], builtin_caps) .expect("returned prepared"); let failure = match GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: 5_114, effects, @@ -4156,6 +4179,7 @@ mod tests { }; let (baseline_effects, baseline_builtins) = prepare(); let baseline = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), plan_id: 162, effects: baseline_effects, builtins: baseline_builtins, @@ -4178,6 +4202,7 @@ mod tests { let mut constrained = integration_caps(); constrained.maximum_nodes = maximum_nodes; let failure = match GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), plan_id: 162, effects, builtins, @@ -4228,6 +4253,7 @@ mod tests { builtins = failure.builtins; } let recovered = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), plan_id: 162, effects, builtins, @@ -11469,6 +11495,7 @@ mod tests { .expect("prepared console builtins"); let registry = launch_native_effect_registry().expect("launch registry"); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch, plan_id, effects: prepare_native_session_effects( @@ -11681,6 +11708,7 @@ mod tests { ) .expect("prepared console builtins"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch, plan_id, effects: prepare_native_session_effects( @@ -11772,6 +11800,7 @@ mod tests { } .expect("prepared selected console builtins"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch, plan_id, effects: prepare_native_session_effects( @@ -11930,6 +11959,7 @@ mod tests { .expect("prepared between-render-call console builtins"); let registry = scalar_console_registry(); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::Scalar, plan_id, effects: prepare_native_session_effects( @@ -14250,6 +14280,7 @@ mod tests { .expect("vocal tail"); assert!(matches!(prepared_tail, TailSamples::Finite(samples) if samples > 0)); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: 77, effects: EffectPreparedSession { @@ -14368,6 +14399,7 @@ mod tests { ) .expect("builtins"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id, effects: EffectPreparedSession { @@ -14585,6 +14617,7 @@ mod tests { ) .expect("baseline builtins"); let baseline = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: 80, effects: EffectPreparedSession { @@ -14621,6 +14654,7 @@ mod tests { ) .expect("returned builtins"); let failure = match GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: 81, effects: EffectPreparedSession { @@ -14731,6 +14765,7 @@ mod tests { ) .expect("prepared seeded builtins"); let artifact = match GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: u64::from(layout) + 50_000, effects: EffectPreparedSession { @@ -14761,6 +14796,7 @@ mod tests { .expect("independently prepared native seeded builtins"); let repeat_artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: u64::from(layout) + 60_000, effects: EffectPreparedSession { @@ -15077,6 +15113,7 @@ mod tests { .expect("builtins"); builtins.test_only_corrupt_for_compiler_test(corruption); let Err(failure) = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: host_dispatch(), plan_id: 78, effects: EffectPreparedSession { @@ -15180,9 +15217,16 @@ mod tests { .iter() .map(|node| (node.id.clone(), node.tail)) .collect(); - let error = timings(&schedule, &edges, &latencies, &tails, &caps(7)) - .err() - .expect("latency plus tail exceeds cap"); + let error = timings( + &schedule, + &edges, + &latencies, + &tails, + &BTreeMap::new(), + &caps(7), + ) + .err() + .expect("latency plus tail exceeds cap"); assert_eq!(error.code, "graph.tail.limit"); } @@ -15241,20 +15285,138 @@ mod tests { .iter() .map(|node| (node.id.clone(), node.tail)) .collect(); - let timing = timings(&schedule, &edges, &latencies, &tails, &caps(100)) - .expect("checked output timing"); + let timing = timings( + &schedule, + &edges, + &latencies, + &tails, + &BTreeMap::new(), + &caps(100), + ) + .expect("checked output timing"); assert_eq!(timing.output_latency, LatencySamples(7)); assert_eq!(timing.output_tail, TailSamples::Finite(12)); assert_eq!(timing.delays.len(), 1); let mut infinite_tails = tails; infinite_tails.insert(node("late"), TailSamples::Infinite); - let infinite = timings(&schedule, &edges, &latencies, &infinite_tails, &caps(100)) - .expect("infinite output tail"); + let infinite = timings( + &schedule, + &edges, + &latencies, + &infinite_tails, + &BTreeMap::new(), + &caps(100), + ) + .expect("infinite output tail"); assert_eq!(infinite.output_latency, LatencySamples(7)); assert_eq!(infinite.output_tail, TailSamples::Infinite); } + /// #1285 D1: a floor above a node's latest incoming arrival raises its `max`, and every + /// incoming edge is compensated to it; a floor at or below the natural arrival, or one that + /// names no node of the graph, changes nothing. Red if the floor moves only the reported + /// arrival and not the edges (the early and late routes keep the natural delays), or if it is + /// added to the natural arrival instead of bounding it. + #[test] + fn timing_compensates_every_incoming_edge_to_a_floor() { + let early = node("early"); + let late = node("late"); + let output = GraphNodeId::Output { + output_id: gid("main"), + }; + let nodes = [ + GraphNode { + id: early.clone(), + latency: LatencySamples(3), + tail: TailSamples::Finite(0), + }, + GraphNode { + id: late.clone(), + latency: LatencySamples(7), + tail: TailSamples::Finite(0), + }, + GraphNode { + id: output.clone(), + latency: LatencySamples(0), + tail: TailSamples::Finite(0), + }, + ]; + let edges = [ + GraphEdge { + id: GraphEdgeId::RouteDestination { + route_id: gid("early-main"), + }, + source: port(early, GraphPortKind::MainOutput), + destination: port(output.clone(), GraphPortKind::MainInput), + path: "$.routes[id=early-main]".to_owned(), + }, + GraphEdge { + id: GraphEdgeId::RouteDestination { + route_id: gid("late-main"), + }, + source: port(late, GraphPortKind::MainOutput), + destination: port(output.clone(), GraphPortKind::MainInput), + path: "$.routes[id=late-main]".to_owned(), + }, + ]; + let levels = topo(&nodes, &edges).expect("acyclic output graph"); + let schedule: Vec<_> = levels + .iter() + .flat_map(|level| level.nodes.iter().cloned()) + .collect(); + let latencies = nodes + .iter() + .map(|node| (node.id.clone(), node.latency)) + .collect(); + let tails = nodes + .iter() + .map(|node| (node.id.clone(), node.tail)) + .collect(); + let run = |floors: &[(GraphNodeId, u64)]| { + let floors = floors + .iter() + .map(|(node, floor)| (node.clone(), LatencySamples(*floor))) + .collect(); + let timing = timings(&schedule, &edges, &latencies, &tails, &floors, &caps(100)) + .expect("checked timing"); + let delays: Vec = timing + .routes + .iter() + .map(|route| route.compensation_delay.0) + .collect(); + ( + timing.output_latency.0, + delays, + timing.input_arrivals, + timing.raised_arrivals, + ) + }; + let natural = run(&[]); + assert_eq!((natural.0, &natural.1[..]), (7, &[4, 0][..])); + assert_eq!(natural.2.as_slice(), [(output.clone(), LatencySamples(7))]); + assert!(natural.3.is_empty(), "no floor raised a node"); + let floored = run(&[(output.clone(), 10)]); + assert_eq!((floored.0, &floored.1[..]), (10, &[7, 3][..])); + assert_eq!(floored.2.as_slice(), [(output.clone(), LatencySamples(10))]); + // #1323 D3: the raised node's natural arrival, beside its floored one. + assert_eq!(floored.3.as_slice(), [(output.clone(), LatencySamples(7))]); + assert!( + run(&[(output.clone(), 7)]).3.is_empty(), + "a floor equal to the natural arrival raises nothing" + ); + assert_eq!( + run(&[(output.clone(), 5)]), + natural, + "a lower floor is no bound" + ); + assert_eq!( + run(&[(node("absent"), 50)]), + natural, + "an absent node's floor" + ); + } + #[test] fn buffer_coloring_aliases_identity_and_preserves_fanout_liveness() { let source = track_node("track", TrackStage::Input); @@ -15707,6 +15869,7 @@ mod tests { &edges, &latencies, &tails, + &BTreeMap::new(), &caps(1_000_000), ); let second = timings( @@ -15714,6 +15877,7 @@ mod tests { &edges, &latencies, &tails, + &BTreeMap::new(), &caps(1_000_000), ); assert_eq!( diff --git a/crates/graph-compiler/src/pdc.rs b/crates/graph-compiler/src/pdc.rs index be9c63097..c8ca7e187 100644 --- a/crates/graph-compiler/src/pdc.rs +++ b/crates/graph-compiler/src/pdc.rs @@ -32,14 +32,26 @@ pub(crate) struct TimingResult { pub(crate) delay_count: u64, pub(crate) output_latency: LatencySamples, pub(crate) output_tail: TailSamples, + /// Every node whose input arrival (its floored `max`) is nonzero, sorted by node ID (#1285 D2). + pub(crate) input_arrivals: Vec<(GraphNodeId, LatencySamples)>, + /// Every node whose floor raised its input arrival, with its latest incoming arrival, sorted + /// by node ID (#1323 D3). + pub(crate) raised_arrivals: Vec<(GraphNodeId, LatencySamples)>, } +/// Arrivals, compensation and tails for `schedule`. +/// +/// `floors` holds a lower bound on a node's input arrival (#1285 D1): a node's `max` is the larger +/// of its latest incoming arrival and its floor, and every incoming edge is compensated to that +/// `max`. A floor that names no node of `schedule` is ignored. With no floors the result is the +/// natural one. #[allow(clippy::too_many_arguments)] pub(crate) fn timings( schedule: &[GraphNodeId], edges: &[GraphEdge], latencies: &BTreeMap, tails: &BTreeMap, + floors: &BTreeMap, caps: &GraphCompileCaps, ) -> Result { let mut incoming_by_node: BTreeMap<&GraphNodeId, Vec<&GraphEdge>> = @@ -56,13 +68,22 @@ pub(crate) fn timings( let mut delay_count: u64 = 0; let mut routes = Vec::new(); let mut delays = Vec::new(); + let mut input_arrivals = Vec::new(); + let mut raised_arrivals = Vec::new(); for node in schedule { let incoming = &incoming_by_node[node]; - let max = incoming + let latest = incoming .iter() .filter_map(|edge| arrivals.get(&edge.source.node).copied()) .max() .unwrap_or(0); + let max = latest.max(floors.get(node).map_or(0, |floor| floor.0)); + if max > latest { + raised_arrivals.push((node.clone(), LatencySamples(latest))); + } + if max > 0 { + input_arrivals.push((node.clone(), LatencySamples(max))); + } for edge in incoming { let source = arrivals.get(&edge.source.node).copied().unwrap_or(0); let delay = max @@ -138,6 +159,8 @@ pub(crate) fn timings( } routes.sort_by(|a, b| a.route_id.cmp(&b.route_id)); delays.sort_by(|left, right| left.edge_id.cmp(&right.edge_id)); + input_arrivals.sort_by(|left, right| left.0.cmp(&right.0)); + raised_arrivals.sort_by(|left, right| left.0.cmp(&right.0)); let output = schedule .iter() .find(|node| matches!(node, GraphNodeId::Output { .. })) @@ -157,6 +180,8 @@ pub(crate) fn timings( delay_count, output_latency, output_tail, + input_arrivals, + raised_arrivals, }) } pub(crate) fn shifted_tail(value: TailSamples, add: u64) -> Result { diff --git a/crates/graph-compiler/tests/bank_levels.rs b/crates/graph-compiler/tests/bank_levels.rs index 4681dada1..fb4a9b2f1 100644 --- a/crates/graph-compiler/tests/bank_levels.rs +++ b/crates/graph-compiler/tests/bank_levels.rs @@ -277,6 +277,7 @@ pub fn compile_bind_render( let builtins = prepare_session_builtins(&session, &[], builtin_caps()).expect("the session's builtins"); let artifact = match GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), plan_id: 966, effects, builtins, diff --git a/crates/graph-compiler/tests/bypass_cohorts.rs b/crates/graph-compiler/tests/bypass_cohorts.rs index 4484115a5..4073c8f29 100644 --- a/crates/graph-compiler/tests/bypass_cohorts.rs +++ b/crates/graph-compiler/tests/bypass_cohorts.rs @@ -300,6 +300,7 @@ fn render_with( } let builtins = prepare_session_builtins(&session, &[], builtin_caps()).expect("builtins"); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), plan_id: 1087, effects, builtins, @@ -491,6 +492,7 @@ fn a_mixed_bypass_cohort_binds_one_bank_per_slot() { .expect("prepared effects"); let builtins = prepare_session_builtins(&session, &[], builtin_caps()).expect("builtins"); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), plan_id: 1087, effects, builtins, diff --git a/crates/graph-compiler/tests/bypass_resources.rs b/crates/graph-compiler/tests/bypass_resources.rs index cc1ae1ba3..295af5965 100644 --- a/crates/graph-compiler/tests/bypass_resources.rs +++ b/crates/graph-compiler/tests/bypass_resources.rs @@ -247,6 +247,7 @@ fn bind( }); let builtins = prepare_session_builtins(&session, &[], builtin_caps()).expect("builtins"); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), plan_id: 1100, effects, builtins, diff --git a/crates/graph-compiler/tests/compile_shapes.rs b/crates/graph-compiler/tests/compile_shapes.rs index 39eec6567..b1b33ccf3 100644 --- a/crates/graph-compiler/tests/compile_shapes.rs +++ b/crates/graph-compiler/tests/compile_shapes.rs @@ -131,6 +131,7 @@ fn compile( let builtins = prepare_session_builtins(&effects.session, requests, builtin_caps()).expect("builtins"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch, plan_id: 1026, effects, @@ -735,6 +736,7 @@ fn a_loop_through_a_bus_tap_is_a_graph_cycle() { let builtins = prepare_session_builtins(&effects.session, &[], builtin_caps()).expect("builtins"); let failure = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: 1203, effects, @@ -817,6 +819,7 @@ fn a_bus_insert_keyed_from_its_own_fader_names_its_sidechain() { let builtins = prepare_session_builtins(&effects.session, &[], builtin_caps()).expect("builtins"); let failure = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: 1205, effects, diff --git a/crates/graph-compiler/tests/live_routes.rs b/crates/graph-compiler/tests/live_routes.rs index c3b3686eb..9e2e2e00e 100644 --- a/crates/graph-compiler/tests/live_routes.rs +++ b/crates/graph-compiler/tests/live_routes.rs @@ -346,6 +346,7 @@ fn compile(model: &SessionModel) -> PreparedGraphBuiltinsArtifact { ) .expect("the builtins prepare"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: 1_220, effects, diff --git a/crates/graph-compiler/tests/route_activity.rs b/crates/graph-compiler/tests/route_activity.rs index 0ace30ee3..c751d282d 100644 --- a/crates/graph-compiler/tests/route_activity.rs +++ b/crates/graph-compiler/tests/route_activity.rs @@ -84,6 +84,7 @@ fn bind(model: &SessionModel) -> Bound { ) .expect("the builtins prepare"); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: 1_217, effects, diff --git a/crates/graph-compiler/tests/route_coefficients.rs b/crates/graph-compiler/tests/route_coefficients.rs index 411b2f330..c08e55f32 100644 --- a/crates/graph-compiler/tests/route_coefficients.rs +++ b/crates/graph-compiler/tests/route_coefficients.rs @@ -67,6 +67,7 @@ fn compile(model: &SessionModel) -> Result Result Vec { ) .expect("the native effects prepare"); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: 1, effects, diff --git a/crates/graph/src/lib.rs b/crates/graph/src/lib.rs index e1cf6d6a2..f6b850fff 100644 --- a/crates/graph/src/lib.rs +++ b/crates/graph/src/lib.rs @@ -4396,111 +4396,6 @@ pub fn carry_program_retained_bytes(plan: &mut PreparedRenderPlan) -> u64 { }) } -/// Every predecessor key the hand-over installed in a successor graph plan reads (#1344 D1), one -/// list per executor carry section, each in its section's order. What the successor carried is -/// derived from this, never recorded family by family. -#[derive(Clone, Debug, Default, Eq, PartialEq)] -pub struct GraphCarryReads { - /// [`GraphCarryProgram::predecessor`]: the plan every key below belongs to. - pub predecessor: u64, - /// Source indices the program moves consumers from. - pub sources: Vec, - /// Lanes the builtin-input section copies from. - pub input_lanes: Vec, - /// `(owner, lane)` pairs the strip section copies from. - pub strip_lanes: Vec<(BuiltinStripOwner, GraphLaneLocation)>, - /// Effect instances the effect section moves from. - pub effect_sites: Vec, - /// `(kind, observer)` pairs the observer section takes: meters and spectrum captures. - pub observers: Vec<(ObserverKind, GraphObserverLocation)>, - /// Effect instances whose observation taps the observer section takes. - pub observation_sites: Vec, - /// Compensation lines the delay section continues. - pub delay_lines: Vec, - /// Strip delay lines the strip-delay section continues. - pub strip_delay_lines: Vec, - /// Live route units the live-send section takes. - pub live_routes: Vec, -} - -/// The predecessor keys the hand-over installed in this graph plan reads (#1344 D1); `None` for any -/// other plan and for a graph plan with no [`GraphCarryProgram`] (no section installs without one). -/// -/// Read off the render thread, before publication. It names every executor carry section without a -/// rest pattern, so a carry section added to the executor does not compile until this reports its -/// predecessor keys: the installer of a new family owns its row here. -#[must_use] -pub fn installed_carry_reads(plan: &PreparedRenderPlan) -> Option { - let executor = plan.executor_any()?.downcast_ref::()?; - let GraphExecutor { - runtime: _, - sample_rate_hz: _, - source_set: _, - source_input_buffers: _, - active_units: _, - identity: _, - carry, - carry_inputs, - carry_strips, - carry_effects, - // The payload scratch reads no predecessor key. - carry_scratch: _, - next_block_sample: _, - node_carry_refused: _, - observer_carry, - carry_delays, - compensation_lines: _, - carry_strip_delays, - carry_routes, - bind_identities: _, - live_snapshot: _, - jump_lengths: _, - } = executor; - let program = carry.as_ref()?; - let (observers, observation_sites) = observer_carry.as_deref().map_or_else( - || (Vec::new(), Vec::new()), - |section| { - let ObserverCarryProgram { - moves, - // A join names two successor observers: it reads no predecessor key. - joins: _, - taps, - record: _, - carried: _, - } = section; - ( - moves - .iter() - .map(|step| (step.kind, step.predecessor)) - .collect(), - taps.iter().map(|tap| tap.step.predecessor).collect(), - ) - }, - ); - Some(GraphCarryReads { - predecessor: program.predecessor, - sources: program - .sources - .iter() - .map(|&(_, predecessor)| predecessor) - .collect(), - input_lanes: carry_inputs.iter().map(|step| step.predecessor).collect(), - strip_lanes: carry_strips - .iter() - .map(|step| (step.owner, step.predecessor)) - .collect(), - effect_sites: carry_effects.iter().map(|step| step.predecessor).collect(), - observers, - observation_sites, - delay_lines: carry_delays.iter().map(|step| step.predecessor).collect(), - strip_delay_lines: carry_strip_delays - .iter() - .map(|step| step.predecessor) - .collect(), - live_routes: carry_routes.iter().map(|step| step.predecessor).collect(), - }) -} - /// Why a source donation between two graph plans is refused (#1344 D4, D5). #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub enum GraphDonationRefusal { diff --git a/crates/host-core/src/lib.rs b/crates/host-core/src/lib.rs index 74151ff4a..dc0bce084 100644 --- a/crates/host-core/src/lib.rs +++ b/crates/host-core/src/lib.rs @@ -227,8 +227,7 @@ pub use effect_compiler::{ }; #[cfg(feature = "test-support")] pub use prepare::{ - test_only_carry_read_keys, test_only_inventory_rows, test_only_prepare_host_runtime_on, - test_only_prepare_host_runtime_successor_on, + test_only_prepare_host_runtime_on, test_only_prepare_host_runtime_successor_on, test_only_prepare_host_runtime_with_live_controls_on, test_only_prepare_host_runtime_with_live_controls_successor_on, test_only_prepare_host_runtime_with_selected_meters_on, diff --git a/crates/host-core/src/prepare.rs b/crates/host-core/src/prepare.rs index 529261b5b..314067a07 100644 --- a/crates/host-core/src/prepare.rs +++ b/crates/host-core/src/prepare.rs @@ -312,10 +312,6 @@ pub struct HostPrepareReport { pub carry_program_retained_bytes: u64, /// Bytes [`PreparedHost::inventory`] retains ([`PlanStateInventory::retained_bytes`]). pub inventory_retained_bytes: u64, - /// Bytes [`PreparedHost::carried_base`] retains ([`PlanStateInventory::retained_bytes`]); - /// zero for a plan prepared without a successor base (#1344 D1). A host that keeps the base - /// to prepare across this plan once it is withdrawn charges it; one that drops it does not. - pub carried_base_retained_bytes: u64, /// Summed effect state bytes. pub effect_scalar_state_bytes: u64, /// Summed effect scratch bytes. @@ -677,16 +673,6 @@ pub struct PreparedHost { /// What this plan holds, for preparing its successor ([`SuccessorBase`]). Control-side plain /// data; never read on the render thread. pub inventory: PlanStateInventory, - /// What this plan carries from the plan it was prepared to displace (#1344 D1): that plan's - /// inventory restricted to the rows the installed hand-over reads, with its identity, so - /// every row is a predecessor row. `None` for a plan prepared without a [`SuccessorBase`]. - /// - /// Derived once, at the end of preparation, from what is installed in the plan - /// (`graph::installed_carry_reads`), never recorded family by family. Should this plan be - /// committed and then withdrawn before render adopts it, a successor of the plan render runs - /// is prepared with this as its base inventory and the running plan's committed model - /// ([`SuccessorBase`], D2). Control-side plain data; never read on the render thread. - pub carried_base: Option, /// The control list of the compiled session (decision 17, C4), built before graph lowering /// consumes the prepared entries. #[cfg(feature = "control-provider")] @@ -769,6 +755,16 @@ pub struct PlanStateInventory { /// present in both, with at least one owner that does not carry. Sorted by strip ID. Empty /// for a plan prepared without a predecessor. restarted: Box<[Box]>, + /// The input arrival this plan renders at every node where it is nonzero (#1285 D2): the + /// floored `max` of PDC, in samples, sorted by node ID. A successor is compiled with these as + /// its input floors, so no surviving node's latency drops across a structural transaction. + /// They are the floored values, not the natural ones, so a chain of successors keeps the + /// first plan's timing until a declared discontinuity (#1323). + input_floors: Box<[(graph::GraphNodeId, u64)]>, + /// The natural input arrival of every node whose floor raised it (#1323 D3), sorted by node + /// ID: its latest incoming arrival, below its floored arrival in `input_floors`. Empty when the + /// plan runs at its natural timing. + raised_arrivals: Box<[(graph::GraphNodeId, u64)]>, } /// One strip stage of a prepared plan, owner key `(strip ID, PostFader)` or `(strip ID, @@ -984,6 +980,84 @@ impl PlanStateInventory { self.live_routes.len() } + /// The input floors a successor of this plan is compiled with (#1285 D2): every node whose + /// input arrival in this plan is nonzero, with that arrival in samples, sorted by node ID. + #[must_use] + pub fn input_floors(&self) -> &[(graph::GraphNodeId, u64)] { + &self.input_floors + } + + /// The natural input arrival of every node a floor raised in this plan (#1323 D3), sorted by + /// node ID; each node's floored arrival is its row in [`Self::input_floors`]. + #[must_use] + pub fn raised_arrivals(&self) -> &[(graph::GraphNodeId, u64)] { + &self.raised_arrivals + } + + /// Whether this plan runs behind its natural timing (#1323 D2): a floor raised some node's + /// input arrival above its natural one. A host-declared discontinuity rebuilds only such a + /// plan. (*Give a plan a source-read clock that leads its render clock*, #1396 D2, adds a + /// source-read offset above 0 to this test.) + #[must_use] + pub fn runs_behind_natural_timing(&self) -> bool { + !self.raised_arrivals.is_empty() + } + + /// The base inventory of a successor at a host-declared discontinuity (#1323 D3): this + /// inventory's identity, rate, quantum and source rows, and nothing else. A successor prepared + /// against it has no floors and carries only its source consumers (ring, generation and read + /// position); every other owner starts at rest. + fn at_discontinuity(&self) -> Self { + Self { + plan_identity: self.plan_identity, + sources: self.sources.clone(), + input_sections: Box::default(), + strip_stages: Box::default(), + effect_lanes: Box::default(), + meters: Box::default(), + observations: Box::default(), + sample_rate_hz: self.sample_rate_hz, + quantum_frames: self.quantum_frames, + spectrum: Box::default(), + spectrum_collection: false, + spectrum_pairing: Box::default(), + delay_lines: Box::default(), + strip_delays: Box::default(), + live_routes: Box::default(), + restarted: Box::default(), + input_floors: Box::default(), + raised_arrivals: Box::default(), + } + } + + /// An inventory that holds only this inventory's input floors (#1285, deliverable 2): a + /// successor prepared against it carries no state, so a test renders it from frame 0 as the + /// fresh reference of a plan compiled with the same floors. + #[cfg(feature = "test-support")] + #[must_use] + pub fn test_only_floors_only(&self) -> Self { + Self { + plan_identity: self.plan_identity, + sources: Box::default(), + input_sections: Box::default(), + strip_stages: Box::default(), + effect_lanes: Box::default(), + meters: Box::default(), + observations: Box::default(), + sample_rate_hz: self.sample_rate_hz, + quantum_frames: self.quantum_frames, + spectrum: Box::default(), + spectrum_collection: false, + spectrum_pairing: Box::default(), + delay_lines: Box::default(), + strip_delays: Box::default(), + live_routes: Box::default(), + restarted: Box::default(), + input_floors: self.input_floors.clone(), + raised_arrivals: self.raised_arrivals.clone(), + } + } + /// Strip stage rows, one per banked fader or matrix lane (issue #1277). #[must_use] pub fn strip_stage_count(&self) -> usize { @@ -1057,7 +1131,13 @@ impl PlanStateInventory { .iter() .map(graph::GraphLiveRoute::heap_bytes) .sum::() - + self.restarted.iter().map(|id| id.len()).sum::(); + + self.restarted.iter().map(|id| id.len()).sum::() + + self + .input_floors + .iter() + .chain(self.raised_arrivals.iter()) + .map(|(node, _)| graph_node_heap_bytes(node)) + .sum::(); u64::try_from( core::mem::size_of_val::<[SourceInventoryRow]>(&self.sources) + core::mem::size_of_val::<[InputSectionInventoryRow]>(&self.input_sections) @@ -1071,274 +1151,72 @@ impl PlanStateInventory { + core::mem::size_of_val::<[graph::GraphStripDelayLine]>(&self.strip_delays) + core::mem::size_of_val::<[graph::GraphLiveRoute]>(&self.live_routes) + core::mem::size_of_val::<[Box]>(&self.restarted) + + core::mem::size_of_val::<[(graph::GraphNodeId, u64)]>(&self.input_floors) + + core::mem::size_of_val::<[(graph::GraphNodeId, u64)]>(&self.raised_arrivals) + ids, ) .unwrap_or(u64::MAX) } } -impl PlanStateInventory { - /// This inventory restricted to the rows `reads` names (#1344 D1): the rows of each family a - /// successor's installed hand-over reads, keyed as that section keys its predecessor. Keeps - /// this plan's identity, rate, quantum and capture kind; the pairing and the restart set - /// describe a successor against this plan, not this plan, so they are empty. Every row kept is - /// this plan's own row, unchanged and in its order. - fn restricted_to(&self, reads: &graph::GraphCarryReads) -> Self { - // Named without a rest pattern: a family added to the inventory does not compile until - // its restriction is written here. - let Self { - plan_identity, - sources, - input_sections, - strip_stages, - effect_lanes, - meters, - observations, - sample_rate_hz, - quantum_frames, - spectrum, - spectrum_collection, - spectrum_pairing: _, - delay_lines, - strip_delays, - live_routes, - restarted: _, - } = self; - fn keep( - rows: &[Row], - keys: &BTreeSet, - key: impl Fn(&Row) -> Key, - ) -> Box<[Row]> { - rows.iter() - .filter(|row| keys.contains(&key(row))) - .cloned() - .collect() +/// Bytes of ID text a graph node ID owns on the heap. +fn graph_node_heap_bytes(node: &graph::GraphNodeId) -> usize { + fn edge(id: &graph::GraphEdgeId) -> usize { + match id { + graph::GraphEdgeId::TrackMain { target } => graph_node_heap_bytes(target), + graph::GraphEdgeId::RouteSource { route_id } + | graph::GraphEdgeId::RouteDestination { route_id } => route_id.as_str().len(), + graph::GraphEdgeId::EffectSidechain { effect, port } => { + effect.track_id.as_str().len() + effect.effect_id.as_str().len() + port.len() + } } - let observers = |wanted: engine::realtime::ObserverKind| -> BTreeSet<_> { - reads - .observers - .iter() - .filter(|(kind, _)| *kind == wanted) - .map(|(_, location)| *location) - .collect() - }; - let set = |keys: &[u32]| keys.iter().copied().collect::>(); - Self { - plan_identity: *plan_identity, - sources: keep(sources, &set(&reads.sources), |row| row.index), - input_sections: keep( - input_sections, - &reads.input_lanes.iter().copied().collect(), - |row| row.location, - ), - strip_stages: keep( - strip_stages, - &reads.strip_lanes.iter().copied().collect(), - |row| (row.owner, row.location), - ), - effect_lanes: keep( - effect_lanes, - &reads.effect_sites.iter().copied().collect(), - |row| row.location, - ), - meters: keep( - meters, - &observers(engine::realtime::ObserverKind::Meter), - |row| row.location, - ), - observations: keep( - observations, - &reads.observation_sites.iter().copied().collect(), - |row| row.location, - ), - sample_rate_hz: *sample_rate_hz, - quantum_frames: *quantum_frames, - spectrum: keep( - spectrum, - &observers(engine::realtime::ObserverKind::Spectrum), - |row| row.location, - ), - spectrum_collection: *spectrum_collection, - spectrum_pairing: Box::default(), - delay_lines: keep(delay_lines, &set(&reads.delay_lines), |row| row.line), - strip_delays: keep(strip_delays, &set(&reads.strip_delay_lines), |row| row.line), - live_routes: keep(live_routes, &set(&reads.live_routes), |row| row.unit), - restarted: Box::default(), + } + match node { + graph::GraphNodeId::TrackStage { track_id, .. } => track_id.as_str().len(), + graph::GraphNodeId::Effect(effect) => { + effect.track_id.as_str().len() + effect.effect_id.as_str().len() + } + graph::GraphNodeId::Route { route_id } => route_id.as_str().len(), + graph::GraphNodeId::Submix { submix_id } => submix_id.as_str().len(), + graph::GraphNodeId::Output { output_id } => output_id.as_str().len(), + graph::GraphNodeId::CompensationDelay { edge_id } => { + core::mem::size_of::() + edge(edge_id) } } } -/// Test-support view of an inventory (#1344 fold M3): every row as `(family, owner, key)`, where -/// `owner` names what the row belongs to by session identity (a strip, an effect, a route, a -/// source) and `key` is the predecessor key a successor's hand-over names it by, formatted as -/// [`test_only_carry_read_keys`] formats the same key. A gate compares the two to prove a -/// restricted base holds exactly the rows the hand-over reads, and that those rows belong to the -/// owners the test expects. -#[cfg(feature = "test-support")] -#[doc(hidden)] -#[must_use] -pub fn test_only_inventory_rows( - inventory: &PlanStateInventory, -) -> BTreeSet<(&'static str, String, String)> { - let PlanStateInventory { - plan_identity: _, - sources, - input_sections, - strip_stages, - effect_lanes, - meters, - observations, - sample_rate_hz: _, - quantum_frames: _, - spectrum, - spectrum_collection: _, - spectrum_pairing: _, - delay_lines, - strip_delays, - live_routes, - restarted: _, - } = inventory; - let mut rows = BTreeSet::new(); - let mut add = |family, owner: String, key: String| { - rows.insert((family, owner, key)); - }; - for row in sources.iter() { - add("sources", row.id.to_string(), format!("{:?}", row.index)); - } - for row in input_sections.iter() { - add( - "input sections", - row.id.to_string(), - format!("{:?}", row.location), - ); - } - for row in strip_stages.iter() { - add( - "strip stages", - format!("{} {:?}", row.id, row.owner), - format!("{:?}", (row.owner, row.location)), - ); - } - let effect_owner = |location: &graph::GraphEffectSite| { - effect_lanes +impl PlanStateInventory { + /// The base inventory a replacement B of the withdrawn successor A is prepared from, this + /// inventory being P0's, the plan render runs and A was prepared to displace (#1698, amending + /// #1344 D1 and D2). Control-side plain data, built once per supersession. + /// + /// Every owner row is P0's own, so an owner B keeps unchanged from P0 carries from P0 as a + /// direct P0 -> B swap carries it, whatever A did to it. A source row is kept only when A + /// carries that source from P0 (A's plan holds it vacant, `can_prepare_source_seek` false on + /// its index): the host feeds the newest committed session, and a source A added or + /// redeclared restarted the host's feed at generation 1, frame 0 against A, so B, which + /// declares it differently from A, restarts it the same way (`CONTROL_PROTOCOL_SEMANTICS.md`, + /// "Sources across a structural transaction"). A source A created fresh and B keeps as A + /// declares it moves by the donation ([`crate::check_donation`]), never through this base. + #[must_use] + pub fn across_withdrawn( + &self, + withdrawn: &PlanStateInventory, + withdrawn_plan: &PreparedRenderPlan, + ) -> Self { + let mut base = self.clone(); + base.sources = self + .sources .iter() - .find(|lane| lane.location == *location) - .map_or_else( - || "unknown effect".to_owned(), - |lane| format!("{} {:?} {}", lane.id, lane.rack, lane.effect_id), - ) - }; - for row in effect_lanes.iter() { - add( - "effect lanes", - effect_owner(&row.location), - format!("{:?}", row.location), - ); - } - for row in meters.iter() { - add( - "meters", - format!("{} {:?} {:?}", row.id, row.tap, row.metrics), - format!("{:?}", row.location), - ); - } - for row in observations.iter() { - add( - "observations", - effect_owner(&row.location), - format!("{:?}", row.location), - ); - } - for row in spectrum.iter() { - add( - "spectrum", - format!("{:?} {:?}", row.target, row.channels), - format!("{:?}", row.location), - ); - } - for row in delay_lines.iter() { - add( - "delay lines", - format!("{:?}", row.edge), - format!("{:?}", row.line), - ); - } - for row in strip_delays.iter() { - add( - "strip delays", - row.strip.to_string(), - format!("{:?}", row.line), - ); - } - for row in live_routes.iter() { - add( - "live routes", - row.route_id.to_string(), - format!("{:?}", row.unit), - ); - } - rows -} - -/// Test-support view of a hand-over's reads (#1344 fold M3): every predecessor key as `(family, -/// key)`, formatted as [`test_only_inventory_rows`] formats it. -#[cfg(feature = "test-support")] -#[doc(hidden)] -#[must_use] -pub fn test_only_carry_read_keys( - reads: &graph::GraphCarryReads, -) -> BTreeSet<(&'static str, String)> { - let graph::GraphCarryReads { - predecessor: _, - sources, - input_lanes, - strip_lanes, - effect_sites, - observers, - observation_sites, - delay_lines, - strip_delay_lines, - live_routes, - } = reads; - let mut keys = BTreeSet::new(); - let mut add = |family, key: String| { - keys.insert((family, key)); - }; - sources - .iter() - .for_each(|key| add("sources", format!("{key:?}"))); - input_lanes - .iter() - .for_each(|key| add("input sections", format!("{key:?}"))); - strip_lanes - .iter() - .for_each(|key| add("strip stages", format!("{key:?}"))); - effect_sites - .iter() - .for_each(|key| add("effect lanes", format!("{key:?}"))); - for (kind, location) in observers { - let family = match kind { - engine::realtime::ObserverKind::Meter => "meters", - engine::realtime::ObserverKind::Spectrum => "spectrum", - // The observation taps' readers move with their effect instance; the restriction - // keys observation rows by `observation_sites`, below. - engine::realtime::ObserverKind::Observation => continue, - }; - add(family, format!("{location:?}")); + .filter(|row| { + withdrawn.source(&row.id).is_some_and(|carried| { + !withdrawn_plan.can_prepare_source_seek(carried.index as usize) + }) + }) + .cloned() + .collect(); + base } - observation_sites - .iter() - .for_each(|key| add("observations", format!("{key:?}"))); - delay_lines - .iter() - .for_each(|key| add("delay lines", format!("{key:?}"))); - strip_delay_lines - .iter() - .for_each(|key| add("strip delays", format!("{key:?}"))); - live_routes - .iter() - .for_each(|key| add("live routes", format!("{key:?}"))); - keys } /// The plan a successor is prepared to displace (issue #1272 D2, umbrella #1269 P1). @@ -1358,15 +1236,15 @@ pub fn test_only_carry_read_keys( /// the successor's cells as a retarget stamped with `revision`, as a strip's is. A strip whose /// effect chain changed restarts even if its remaining slots carry. /// -/// **Across a withdrawn plan (#1344 D2).** When the plan render runs (P0) has a committed -/// successor A that was taken back before render adopted it, A's replacement B is prepared -/// against P0 with this same entry: `inventory` is A's [`PreparedHost::carried_base`] and -/// `committed` is P0's committed model. Every row of that inventory is a P0 row, so the joins run -/// unchanged and an owner carries from P0 only if A carried it and B leaves it unchanged against -/// P0's model; everything A restarted, added or removed is fresh in B. A strip A restarted has no -/// row there, so B restarts it too and lists it in [`PreparedHost::restarted_strips`]. The rings -/// A created fresh move to B by a checked donation ([`crate::check_donation`]), never through this -/// base. +/// **Across a withdrawn plan (#1344 D2, amended by #1698).** When the plan render runs (P0) has a +/// committed successor A that was taken back before render adopted it, A's replacement B is +/// prepared against P0 with this same entry: `inventory` is +/// [`PlanStateInventory::across_withdrawn`] of P0's inventory and A, and `committed` is P0's +/// committed model. Every row of that inventory is a P0 row, so the joins run unchanged and B +/// carries from P0 every owner it leaves unchanged against P0's model, as a direct P0 -> B swap +/// does, including an owner A restarted or removed. Sources carry only as A carried them: a +/// source A added, redeclared or removed is fresh in B. The rings A created fresh move to B by a +/// checked donation ([`crate::check_donation`]), never through this base. #[derive(Clone, Copy, Debug)] pub struct SuccessorBase<'a> { /// The predecessor's [`PreparedHost::inventory`]. @@ -1395,11 +1273,45 @@ pub struct SuccessorBase<'a> { /// takes the predecessor's own state instead, and is never restarted by this. /// [`NO_OBSERVATION`] for a host without subscriptions. pub observation: &'a controls::ObservationBindings, + /// The host declared a discontinuity (#1323 D3): the successor need not continue the + /// predecessor's output. It is compiled with no input floors, so it runs at its natural + /// timing, and its join carries only the source consumers (ring, generation and read + /// position), so no frame is repeated or skipped; every other owner starts at rest, with no + /// continuity constraint. `false` is the neutral value: a successor continues its + /// predecessor. + pub discontinuity: bool, +} + +impl<'a> SuccessorBase<'a> { + /// The base of a successor of the plan whose inventory is `inventory`, prepared from + /// `committed` and committing as `revision` (#1323 D3, #1277 D10). Every other field takes + /// its neutral value: no edit ramps, no monitor overlay ([`MonitorBase::NONE`]), no armed + /// tap ([`NO_OBSERVATION`]) and no discontinuity. A field added later extends this with its + /// own neutral value; a base that needs another value sets it with struct update syntax. + #[must_use] + pub fn new( + inventory: &'a PlanStateInventory, + committed: &'a SessionModel, + revision: u64, + ) -> Self { + Self { + inventory, + committed, + revision, + edit_ramps: &NO_EDIT_RAMPS, + monitor: MonitorBase::NONE, + observation: &NO_OBSERVATION, + discontinuity: false, + } + } } /// The bindings that arm nothing, for a [`SuccessorBase`] of a host without subscriptions. pub static NO_OBSERVATION: controls::ObservationBindings = controls::ObservationBindings::new(); +/// The edit ramps of a successor whose transaction gave no row a length ([`SuccessorBase::new`]). +static NO_EDIT_RAMPS: EditRamps = EditRamps::new(); + /// The host-local monitor values (solo) a successor is prepared under (decision 17, C8, #1559). /// /// The successor bakes `controls::effective(model, next)`, so it renders the solo from its first @@ -2215,6 +2127,17 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( if successor.is_some_and(|base| !(1..=GATE_REVISION_MAX).contains(&base.revision)) { return Err(successor_revision_refusal()); } + // #1323 D3: at a host-declared discontinuity the successor joins against its predecessor's + // sources only, so it has no input floors and every owner but a source consumer starts at + // rest. Every join below reads the base's inventory, so this one substitution is the whole + // rule. + let discontinuity_inventory = successor + .filter(|base| base.discontinuity) + .map(|base| base.inventory.at_discontinuity()); + let successor = successor.map(|base| match discontinuity_inventory.as_ref() { + Some(inventory) => SuccessorBase { inventory, ..base }, + None => base, + }); // #1312 D2: the session's count when the host passed one, else this preparation's own, which // the handles hand back. Made before any observed allocation of the preparation. let live_superseded = superseded.cloned().unwrap_or_default(); @@ -2833,6 +2756,17 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( plan_id: 1, effects, builtins, + // #1285 D2: a successor keeps every surviving node's input arrival; a first plan has no + // floors. Across a withdrawn candidate the base is P0's inventory, so these are P0's. + input_floors: successor + .map(|base| { + base.inventory + .input_floors + .iter() + .map(|(node, floor)| (node.clone(), effect_contract::LatencySamples(*floor))) + .collect() + }) + .unwrap_or_default(), caps: GraphCompileCaps { maximum_nodes: u64::MAX, maximum_edges: u64::MAX, @@ -2881,8 +2815,21 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( .into_iter() .map(RouteControlProducer::new) .collect(); + // #1285 D4: the floored output arrival, the one the plan renders. let output_latency = artifact.report().output_latency; let output_tail = artifact.report().output_tail; + let input_floors: Box<[(graph::GraphNodeId, u64)]> = artifact + .report() + .input_arrivals + .iter() + .map(|(node, arrival)| (node.clone(), arrival.0)) + .collect(); + let raised_arrivals: Box<[(graph::GraphNodeId, u64)]> = artifact + .report() + .raised_arrivals + .iter() + .map(|(node, arrival)| (node.clone(), arrival.0)) + .collect(); let mut graph_resources = artifact.graph_resource_estimate().clone(); // The plan's own allocations that the graph row does not count: its revision gate (#1502 D1), // its jump-length cell (#1639 D1, attached after bind below) and, when this preparation made @@ -3478,33 +3425,9 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( strip_delays: strip_delays.into_boxed_slice(), live_routes: live_routes.into_boxed_slice(), restarted: restarted.into_boxed_slice(), + input_floors, + raised_arrivals, }; - // #1344 D1: what this plan carries, read back from the hand-over just installed. A successor - // with nothing installed carries nothing, so its base is the predecessor's identity alone. - let carried_base = successor.map(|base| { - let reads = graph::installed_carry_reads(&plan).unwrap_or_default(); - debug_assert!( - reads == graph::GraphCarryReads::default() - || reads.predecessor == base.inventory.plan_identity, - "the hand-over names the base it was prepared against" - ); - let restricted = base.inventory.restricted_to(&reads); - // #1395 D3: the observer section's spectrum moves are the pairs the control side applies. - debug_assert_eq!( - restricted - .spectrum - .iter() - .map(|row| row.entry) - .collect::>(), - inventory - .spectrum_pairing - .iter() - .map(|pair| pair.predecessor) - .collect::>(), - "the spectrum rows the hand-over reads are the ones the pairing names" - ); - restricted - }); let report = HostPrepareReport { sample_rate_hz: compiled.sample_rate().0, quantum_frames: compiled.quantum().0, @@ -3532,9 +3455,6 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( carried_source_overhead_bytes: carried_overhead_bytes, carry_program_retained_bytes, inventory_retained_bytes: inventory.retained_bytes(), - carried_base_retained_bytes: carried_base - .as_ref() - .map_or(0, PlanStateInventory::retained_bytes), effect_scalar_state_bytes: effect_state_bytes, effect_scalar_scratch_bytes: effect_scratch_bytes, builtin_processor_payload_bytes: builtin_resources.engine_owned_processor_payload_bytes, @@ -3651,7 +3571,6 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( sources, report, inventory, - carried_base, #[cfg(feature = "control-provider")] control_catalog, }, @@ -6160,6 +6079,8 @@ mod tests { strip_delays: Box::default(), live_routes: Box::default(), restarted: Box::default(), + input_floors: Box::default(), + raised_arrivals: Box::default(), } } diff --git a/crates/host-core/src/render_session.rs b/crates/host-core/src/render_session.rs index a51e08896..2b55aac08 100644 --- a/crates/host-core/src/render_session.rs +++ b/crates/host-core/src/render_session.rs @@ -192,7 +192,6 @@ impl PreparedHost { sources, report, inventory, - carried_base, #[cfg(feature = "control-provider")] control_catalog, } = self; @@ -204,7 +203,6 @@ impl PreparedHost { sources, report, inventory, - carried_base, #[cfg(feature = "control-provider")] control_catalog, }, diff --git a/crates/host-core/src/source.rs b/crates/host-core/src/source.rs index ce0acaf9c..2c739eb4c 100644 --- a/crates/host-core/src/source.rs +++ b/crates/host-core/src/source.rs @@ -590,9 +590,11 @@ pub fn check_donation( /// /// The caller refuses the replacement before its commit as retryable backpressure: once render /// adopts the donor, X's ring is the running plan's and the same edit commits on it. Read-only; -/// it allocates nothing. Issue #1698 lifts the reverted case only (it carries X from the running -/// plan); the others stay refused until an issue gives the replacement a way to read those chunks -/// or the owner rules that a removed source's chunks may drop. +/// it allocates nothing. Every case stays refused until an issue gives the replacement a way to +/// read those chunks or the owner rules that a removed source's chunks may drop. #1698 keeps the +/// reverted case too: the host restarted its feed of X against the donor, so the replacement +/// starts X fresh ([`crate::PlanStateInventory::across_withdrawn`]) and cannot read the donor's +/// chunks into the running plan's continuing X. #[must_use] pub fn drops_acked_fresh_source(donor: DonationSide<'_>, donation: &DonationPlan) -> bool { donor.inventory.sources().iter().any(|row| { diff --git a/crates/host-core/tests/browser_successor.rs b/crates/host-core/tests/browser_successor.rs index d97f09f01..09dbb11a5 100644 --- a/crates/host-core/tests/browser_successor.rs +++ b/crates/host-core/tests/browser_successor.rs @@ -282,12 +282,8 @@ fn base<'a>( edit_ramps: &'a EditRamps, ) -> SuccessorBase<'a> { SuccessorBase { - inventory, - committed, - revision: SUCCESSOR_REVISION, edit_ramps, - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, + ..SuccessorBase::new(inventory, committed, SUCCESSOR_REVISION) } } diff --git a/crates/host-core/tests/effect_observation.rs b/crates/host-core/tests/effect_observation.rs index 221334065..605c12a2e 100644 --- a/crates/host-core/tests/effect_observation.rs +++ b/crates/host-core/tests/effect_observation.rs @@ -547,12 +547,8 @@ fn prepare_successor( &caps(), &successor_request, host_core::SuccessorBase { - inventory: &first.prepared.inventory, - committed, - revision: 2, - edit_ramps: &session::EditRamps::new(), - monitor: host_core::MonitorBase::NONE, observation: bindings, + ..host_core::SuccessorBase::new(&first.prepared.inventory, committed, 2) }, ), None => host_core::prepare_host_runtime_with_live_controls(compiled, &caps(), &request), diff --git a/crates/host-core/tests/live_routes.rs b/crates/host-core/tests/live_routes.rs index 12c309844..54d88e004 100644 --- a/crates/host-core/tests/live_routes.rs +++ b/crates/host-core/tests/live_routes.rs @@ -959,6 +959,7 @@ fn graph_route_resources(document: &str) -> RouteControlResources { ) .expect("builtins"); let mut artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: 1_221, effects, diff --git a/crates/host-core/tests/route_mute.rs b/crates/host-core/tests/route_mute.rs index 25a1c5699..2311e5b03 100644 --- a/crates/host-core/tests/route_mute.rs +++ b/crates/host-core/tests/route_mute.rs @@ -823,6 +823,7 @@ fn graph_artifact(document: &str) -> PreparedGraphBuiltinsArtifact { ) .expect("builtins"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: 1_216, effects, diff --git a/crates/host-core/tests/strip_meters.rs b/crates/host-core/tests/strip_meters.rs index fa73ece26..120562379 100644 --- a/crates/host-core/tests/strip_meters.rs +++ b/crates/host-core/tests/strip_meters.rs @@ -715,16 +715,12 @@ fn a_successor_takes_its_master_from_the_overlay_by_strip_id() { &caps(), live, host_core::SuccessorBase { - inventory: &prepared.inventory, - committed: compiled.normalized_model(), - revision: 2, - edit_ramps: &session::EditRamps::new(), monitor: host_core::MonitorBase { committed: overlay, next: overlay, ramp: None, }, - observation: &host_core::NO_OBSERVATION, + ..host_core::SuccessorBase::new(&prepared.inventory, compiled.normalized_model(), 2) }, ) }; diff --git a/crates/host-core/tests/submix_strip.rs b/crates/host-core/tests/submix_strip.rs index 430342a46..a6abef311 100644 --- a/crates/host-core/tests/submix_strip.rs +++ b/crates/host-core/tests/submix_strip.rs @@ -972,6 +972,7 @@ fn graph_artifact(document: &str, plan_id: u64) -> PreparedGraphBuiltinsArtifact ) .expect("builtins"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id, effects, diff --git a/crates/host-core/tests/successor_swap.rs b/crates/host-core/tests/successor_swap.rs index 47c173db8..e6e93541b 100644 --- a/crates/host-core/tests/successor_swap.rs +++ b/crates/host-core/tests/successor_swap.rs @@ -1055,12 +1055,13 @@ impl LivePlan { &caps(), request, host_core::SuccessorBase { - inventory: predecessor, - committed, - revision: successor::SUCCESSOR_REVISION, edit_ramps, - monitor: host_core::MonitorBase::NONE, observation, + ..host_core::SuccessorBase::new( + predecessor, + committed, + successor::SUCCESSOR_REVISION, + ) }, backend, ) @@ -3000,12 +3001,12 @@ impl CellPlan { Self::successor_at( session, host_core::SuccessorBase { - inventory: predecessor, - committed, - revision: successor::SUCCESSOR_REVISION, edit_ramps, - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, + ..host_core::SuccessorBase::new( + predecessor, + committed, + successor::SUCCESSOR_REVISION, + ) }, backend, form, @@ -4378,14 +4379,7 @@ fn a_successor_revision_its_gate_cannot_stamp_is_refused() { for revision in [0, engine::realtime::GATE_REVISION_MAX + 1, u64::MAX] { let refused = CellPlan::successor_at( &b, - host_core::SuccessorBase { - inventory: &predecessor.inventory, - committed: &a.model, - revision, - edit_ramps: &EditRamps::new(), - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }, + host_core::SuccessorBase::new(&predecessor.inventory, &a.model, revision), backend, Form::Split, ) @@ -4402,14 +4396,7 @@ fn a_successor_revision_its_gate_cannot_stamp_is_refused() { assert!( CellPlan::successor_at( &b, - host_core::SuccessorBase { - inventory: &predecessor.inventory, - committed: &a.model, - revision: accepted, - edit_ramps: &EditRamps::new(), - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }, + host_core::SuccessorBase::new(&predecessor.inventory, &a.model, accepted), backend, Form::Split, ) @@ -4432,14 +4419,11 @@ fn successor_handle( &b.compiled, &caps(), &LivePlan::request(), - host_core::SuccessorBase { - inventory: &predecessor.live.prepared.inventory, + host_core::SuccessorBase::new( + &predecessor.live.prepared.inventory, committed, - revision: successor::SUCCESSOR_REVISION, - edit_ramps: &EditRamps::new(), - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }, + successor::SUCCESSOR_REVISION, + ), backend, ) .unwrap_or_else(|failure| panic!("successor: {failure:?}")); @@ -7427,8 +7411,8 @@ mod meter_carry { use builtins_compiler::MeterConsumer; use engine::realtime::{HostAttachment, ObserverCarry, ObserverKind, PlanAdoption}; use host_core::{ - HostLiveControlRequest, HostMeterRequest, MonitorBase, PreparedHost, SuccessorBase, - pair_carried_meters, test_only_prepare_host_runtime_with_selected_meters_on, + HostLiveControlRequest, HostMeterRequest, PreparedHost, SuccessorBase, pair_carried_meters, + test_only_prepare_host_runtime_with_selected_meters_on, }; use super::successor::{self, SUCCESSOR_REVISION}; @@ -7611,14 +7595,7 @@ mod meter_carry { b, b_live, b_meters, - Some(SuccessorBase { - inventory: &inventory, - committed: &a.model, - revision: SUCCESSOR_REVISION, - edit_ramps: &session::EditRamps::new(), - monitor: MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }), + Some(SuccessorBase::new(&inventory, &a.model, SUCCESSOR_REVISION)), backend, ); b_prepared.sources.adopt_persisting(&mut a_sources); @@ -9033,6 +9010,411 @@ fn a_repointed_route_does_not_carry_its_line_at_every_rate_and_width() { } } +// Issue #1285 (slice 15 of #1269): latency floors. A successor is compiled with every node's +// input arrival in its predecessor as a floor, so removing a latent effect, or the track that +// holds it, moves no surviving path in time: every compensation line keeps its length and carries. + +/// B prepared against an inventory that holds only `floors`' input floors, rendered from frame 0 +/// and fed `feed`: the fresh reference of B compiled with those floors (#1285 gate 1). +fn floored_reference( + b: &Session, + floors: &host_core::PlanStateInventory, + feed: &Feed, + backend: Backend, +) -> Vec> { + let quantum = b.model.quantum_frames as usize; + let floors = floors.test_only_floors_only(); + let mut plan = LivePlan::successor(b, &floors, &b.model, backend); + plan.submit(feed, quantum, 0); + (0..BLOCKS) + .map(|block| { + if block + 1 < BLOCKS { + plan.submit(feed, quantum, block + 1); + } + plan.render(quantum) + }) + .collect() +} + +/// `model` with one more true-peak limiter insert, `insert`, on `track`, after its others. +fn add_limiter(model: &mut SessionModel, track: &str, insert: &str) { + let (index, strip) = model + .tracks + .iter_mut() + .enumerate() + .find(|(_, strip)| strip.id.as_str() == track) + .expect("the track"); + strip.inserts.effects.push(Effect { + id: id(insert), + identity: EffectIdentity::Native { + effect_id: id("miso.true-peak-limiter"), + }, + quality: EffectQuality::Normal, + bypass: false, + link_mode: LinkMode::Maximum, + params: console_params("limiter", index, false), + sidechain: SidechainDeclaration::None, + }); +} + +/// The checks gates 1 and 2 share: B removes the limited track `eq0` from `a_model`. The swapped +/// run equals B compiled with A's floors from frame 0 in every block, its latency is A's, nothing +/// restarts, and B without floors renders the remaining tracks earlier. `eq0` is muted in A, so it +/// adds exact `+0.0` and A's blocks before the swap are B's too. +fn a_removed_latency_moves_no_surviving_path(a_model: SessionModel, backend: Backend, what: &str) { + let a_model = with_mute(a_model, "eq0", true); + let mut b_model = a_model.clone(); + remove_track(&mut b_model, "eq0"); + let (a, b) = (Session::compile(a_model), Session::compile(b_model)); + let predecessor = LivePlan::fresh(&a, backend).prepared; + let latency = predecessor.report.latency_samples; + assert!(latency > 0, "{what}: the limiter delays A"); + let successor = LivePlan::successor(&b, &predecessor.inventory, &a.model, backend).prepared; + assert_eq!( + successor.report.latency_samples, latency, + "{what}: the successor reports A's latency (D4)" + ); + assert!( + b.prepare(backend).report.latency_samples < latency, + "{what}: B's natural latency is lower" + ); + let feed = Feed::new(SOURCE, 1); + let (run, unfloored) = pdc_runs(&a, &b, &feed, backend, what); + assert_eq!(run.restarted, Vec::>::new(), "{what}"); + let reference = floored_reference(&b, &predecessor.inventory, &feed, backend); + assert_eq!( + run.blocks + .iter() + .zip(&reference) + .position(|(swapped, fresh)| swapped != fresh), + None, + "{what}: every block equals B compiled with A's floors from frame 0" + ); + assert!( + reference[SWAP_BLOCK + 2..] + .iter() + .flatten() + .any(|bits| *bits != 0), + "{what}: the reference plays after the swap block" + ); + assert_ne!( + unfloored.blocks[SWAP_BLOCK + 1..], + reference[SWAP_BLOCK + 1..], + "{what}: without floors the remaining tracks move by the limiter's latency" + ); +} + +/// Gate 1: three tracks, the first behind a limiter; B removes the first track. Red if a +/// successor is compiled without floors: the output's two compensation lines vanish and the +/// remaining tracks jump forward by the limiter's latency at the swap. +#[test] +fn removing_a_limited_track_keeps_the_output_timing_at_every_rate_and_width() { + for backend in backends() { + for rate in PDC_RATES { + let mut a_model = pdc_session(rate, &["eq0"]); + remove_track(&mut a_model, "eq3"); + a_removed_latency_moves_no_surviving_path( + a_model, + backend, + &format!("{backend:?} at {rate} Hz"), + ); + } + } +} + +/// Gate 2: `eq0` (behind a limiter) and `eq1` feed the submix `bus1`; the output takes `bus1` and +/// `eq2`. B removes `eq0`. Red if floors are applied only at the output: the bus input's natural +/// arrival drops to 0, `eq1`'s line into the bus vanishes and the bus's own route into the output +/// gets a new line that starts at rest. +#[test] +fn removing_a_limited_track_keeps_the_submix_timing_at_every_rate_and_width() { + for backend in backends() { + for rate in PDC_RATES { + a_removed_latency_moves_no_surviving_path( + bus_session(rate), + backend, + &format!("{backend:?} at {rate} Hz, submix"), + ); + } + } +} + +/// Gate 5: after gate 1's swap, C removes `eq1` from B. C keeps A's output latency, because B's +/// inventory records the floored arrivals. Red if the inventory records the natural `max`: B's +/// output arrival is naturally 0, so C's latency drops to it. +#[test] +fn a_second_removal_keeps_the_first_floors() { + for backend in backends() { + let mut a_model = pdc_session(48_000, &["eq0"]); + remove_track(&mut a_model, "eq3"); + let mut b_model = a_model.clone(); + remove_track(&mut b_model, "eq0"); + let mut c_model = b_model.clone(); + remove_track(&mut c_model, "eq1"); + let (a, b, c) = ( + Session::compile(a_model), + Session::compile(b_model), + Session::compile(c_model), + ); + let first = a.prepare(backend); + let latency = first.report.latency_samples; + assert!(latency > 0, "{backend:?}"); + let second = b + .prepare_successor(&first.inventory, &a.model, &caps(), backend) + .expect("B"); + let third = c + .prepare_successor(&second.inventory, &b.model, &caps(), backend) + .expect("C"); + assert_eq!(second.report.latency_samples, latency, "{backend:?}: B"); + assert_eq!(third.report.latency_samples, latency, "{backend:?}: C"); + assert_eq!( + third.inventory.input_floors(), + second.inventory.input_floors(), + "{backend:?}: C renders at B's arrivals" + ); + } +} + +/// #1344 with floors: P0 has `eq0` behind a limiter. The candidate A adds a second limiter to +/// `eq1`, so its own floors are higher than P0's, and is withdrawn. B removes `eq0` from P0 and is +/// prepared against P0 across A. Render runs P0, so B keeps P0's timing: its latency is P0's and +/// its floors are P0's arrivals. Red if the base across a withdrawn plan drops the floors (B's +/// latency falls to its natural value) or takes A's (B's latency is A's). +#[test] +fn a_replacement_across_a_withdrawn_candidate_keeps_the_running_floors() { + for backend in backends() { + let mut p0_model = pdc_session(48_000, &["eq0"]); + remove_track(&mut p0_model, "eq3"); + let mut a_model = p0_model.clone(); + add_limiter(&mut a_model, "eq1", "lim"); + add_limiter(&mut a_model, "eq1", "lim2"); + let mut b_model = p0_model.clone(); + remove_track(&mut b_model, "eq0"); + let (p0, a, b) = ( + Session::compile(p0_model), + Session::compile(a_model), + Session::compile(b_model), + ); + let running = p0.prepare(backend); + let latency = running.report.latency_samples; + let withdrawn = a + .prepare_successor(&running.inventory, &p0.model, &caps(), backend) + .expect("A"); + assert!( + withdrawn.report.latency_samples > latency, + "{backend:?}: A's two limiters delay it past P0" + ); + let base = running + .inventory + .across_withdrawn(&withdrawn.inventory, &withdrawn.plan); + let replacement = b + .prepare_successor(&base, &p0.model, &caps(), backend) + .expect("B"); + assert_eq!( + replacement.report.latency_samples, latency, + "{backend:?}: B keeps the running plan's latency" + ); + let direct = b + .prepare_successor(&running.inventory, &p0.model, &caps(), backend) + .expect("P0 -> B"); + assert_eq!( + replacement.inventory.input_floors(), + direct.inventory.input_floors(), + "{backend:?}: B renders at the arrivals of a direct P0 -> B swap" + ); + } +} + +// Issue #1323: a host-declared discontinuity. Its successor is compiled with no floors and +// carries only its source consumers, so it renders as the session compiled fresh, fed from the +// frames the carried consumers stood at. + +/// The block the discontinuity successor renders first. +const DISCONTINUITY_BLOCK: usize = 10; +/// Blocks in a discontinuity run. +const DISCONTINUITY_BLOCKS: usize = 16; + +/// `session` prepared with live controls as the successor of `predecessor` at a declared +/// discontinuity, its gate stamped and published with the committed revision, as the control +/// plane's publication does. +fn discontinuity_successor( + session: &Session, + predecessor: &host_core::PlanStateInventory, + committed: &SessionModel, + backend: Backend, +) -> LivePlan { + let (prepared, handles) = + host_core::test_only_prepare_host_runtime_with_live_controls_successor_on( + &session.compiled, + &caps(), + &LivePlan::request(), + host_core::SuccessorBase { + discontinuity: true, + ..host_core::SuccessorBase::new( + predecessor, + committed, + successor::SUCCESSOR_REVISION, + ) + }, + backend, + ) + .unwrap_or_else(|failure| panic!("discontinuity successor: {failure:?}")); + let mut successor = LivePlan::new(prepared, handles); + let _ = successor + .gate + .stamp(successor::SUCCESSOR_REVISION) + .expect("a fresh gate stamps the committed revision"); + successor.gate.publish(successor::SUCCESSOR_REVISION); + successor +} + +/// `successor` adopts `plan` and renders one block; the run goes on with it. +fn adopt_and_render( + plan: &mut LivePlan, + mut successor: LivePlan, + feed: &Feed, + quantum: usize, + block: usize, +) -> (Vec, CarryOutcome) { + assert_eq!( + successor + .prepared + .sources + .adopt_persisting(&mut plan.prepared.sources), + 1, + "the source's producer moves" + ); + successor.submit(feed, quantum, block + 1); + let carry = successor + .prepared + .plan + .adopt_predecessor_plan(&mut plan.prepared.plan); + let bits = successor.render(quantum); + *plan = successor; + (bits, carry) +} + +/// Gate 1: A has three tracks, the first behind a limiter; B removes it and is swapped in at +/// [`SWAP_BLOCK`] with A's floors (#1285). B is then prepared again against itself with +/// `discontinuity: true` and swapped in at [`DISCONTINUITY_BLOCK`]. From that block on, every +/// block equals B compiled fresh, with no floors, fed the PCM from the frame its carried source +/// consumer stood at; its latency is B's natural latency, and its inventory has no raised floor. +/// +/// Test value: red if the discontinuity successor keeps the floors (its latency stays A's and the +/// remaining tracks stay late), or if it carries any DSP state but the source consumer (the EQs' +/// filter state and the compensation lines differ from a fresh plan's at rest), or if it carries +/// no source (it reads its ring from frame 0, a repeat). +fn a_declared_discontinuity_resets_the_floors(backend: Backend, rate: u32) { + let what = format!("{backend:?} at {rate} Hz"); + let mut a_model = pdc_session(rate, &["eq0"]); + remove_track(&mut a_model, "eq3"); + let a_model = with_mute(a_model, "eq0", true); + let mut b_model = a_model.clone(); + remove_track(&mut b_model, "eq0"); + let (a, b) = (Session::compile(a_model), Session::compile(b_model)); + let quantum = a.model.quantum_frames as usize; + let feed = Feed::with_blocks(SOURCE, 1, DISCONTINUITY_BLOCKS + DISCONTINUITY_BLOCK + 1); + + let mut plan = LivePlan::fresh(&a, backend); + let latency = plan.prepared.report.latency_samples; + let natural = LivePlan::fresh(&b, backend); + let natural_latency = natural.prepared.report.latency_samples; + assert!( + natural_latency < latency, + "{what}: B's natural latency is lower" + ); + assert!( + !plan.prepared.inventory.runs_behind_natural_timing(), + "{what}: a first plan has no floor" + ); + let mut blocks = Vec::with_capacity(DISCONTINUITY_BLOCKS); + plan.submit(&feed, quantum, 0); + for block in 0..DISCONTINUITY_BLOCKS { + let bits = match block { + SWAP_BLOCK => { + let floored = LivePlan::successor(&b, &plan.prepared.inventory, &a.model, backend); + assert_eq!(floored.prepared.report.latency_samples, latency, "{what}"); + assert!( + floored.prepared.inventory.runs_behind_natural_timing(), + "{what}: B with A's floors runs behind its natural timing" + ); + adopt_and_render(&mut plan, floored, &feed, quantum, block).0 + } + DISCONTINUITY_BLOCK => { + let reset = + discontinuity_successor(&b, &plan.prepared.inventory, &b.model, backend); + assert_eq!( + reset.prepared.report.latency_samples, natural_latency, + "{what}: the discontinuity successor runs at B's natural latency" + ); + assert!( + !reset.prepared.inventory.runs_behind_natural_timing(), + "{what}: and has no raised floor" + ); + assert_eq!( + reset.prepared.inventory.input_floors(), + natural.prepared.inventory.input_floors(), + "{what}: its arrivals are a fresh B's" + ); + let (bits, carry) = adopt_and_render(&mut plan, reset, &feed, quantum, block); + assert!( + carry == CarryOutcome::Carried, + "{what}: the source consumer carries: {carry:?}" + ); + bits + } + _ => { + if block + 1 < DISCONTINUITY_BLOCKS { + plan.submit(&feed, quantum, block + 1); + } + plan.render(quantum) + } + }; + blocks.push(bits); + } + + // The fresh reference: B with no floors, rendered from frame 0, fed the PCM from the frame the + // carried consumer stood at when the discontinuity successor adopted. + let offset = DISCONTINUITY_BLOCK * quantum; + let shifted = Feed { + id: SOURCE, + planes: [ + feed.planes[0][offset..].to_vec(), + feed.planes[1][offset..].to_vec(), + ], + }; + let mut reference = natural; + let tail = DISCONTINUITY_BLOCKS - DISCONTINUITY_BLOCK; + reference.submit(&shifted, quantum, 0); + let expected: Vec> = (0..tail) + .map(|block| { + reference.submit(&shifted, quantum, block + 1); + reference.render(quantum) + }) + .collect(); + assert!( + expected.iter().flatten().any(|bits| *bits != 0), + "{what}: the reference plays" + ); + assert_eq!( + blocks[DISCONTINUITY_BLOCK..] + .iter() + .zip(&expected) + .position(|(reset, fresh)| reset != fresh), + None, + "{what}: every block from the discontinuity equals a fresh B" + ); +} + +#[test] +fn a_declared_discontinuity_resets_the_floors_at_every_rate_and_width() { + for backend in backends() { + for rate in PDC_RATES { + a_declared_discontinuity_resets_the_floors(backend, rate); + } + } +} + // Issue #1284 (slice 14 of #1269): the strip input alignment lines (`delay_samples` of a track and // of a submix) and the live sends. A line is keyed by its strip and lane, a send by its route ID; // a lane whose delay changed starts at rest and its strip goes into the restart set, and a @@ -9438,7 +9820,7 @@ fn a_carried_send_is_retargeted_to_the_successors_gain_at_four_lanes() { mod spectrum_carry { use engine::realtime::{HostAttachment, ObserverCarry, ObserverKind, PlanAdoption}; use host_core::{ - HostLiveControlRequest, HostLiveLanes, LiveValuesSuperseded, MonitorBase, PreparedHost, + HostLiveControlRequest, HostLiveLanes, LiveValuesSuperseded, PreparedHost, PreparedSpectrumCaptures, SpectrumCaptureCollectionEntry, SpectrumCaptureCollectionRequest, SpectrumCaptureRequest, SpectrumChannels, SpectrumContinuousWindow, SpectrumHop, SpectrumPair, SpectrumService, SpectrumServiceError, SpectrumSmoothingConfig, @@ -9784,14 +10166,7 @@ mod spectrum_carry { } let (mut b_prepared, b_capture) = b_request.prepare( b, - Some(SuccessorBase { - inventory: &inventory, - committed: &a.model, - revision: SUCCESSOR_REVISION, - edit_ramps: &session::EditRamps::new(), - monitor: MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }), + Some(SuccessorBase::new(&inventory, &a.model, SUCCESSOR_REVISION)), ); b_prepared.sources.adopt_persisting(&mut a_sources); let pairing = b_prepared.inventory.spectrum_pairing().to_vec(); diff --git a/crates/host-core/tests/support/successor.rs b/crates/host-core/tests/support/successor.rs index cad230f53..2a230b9cc 100644 --- a/crates/host-core/tests/support/successor.rs +++ b/crates/host-core/tests/support/successor.rs @@ -95,14 +95,7 @@ impl Session { test_only_prepare_host_runtime_successor_on( &self.compiled, caps, - SuccessorBase { - inventory, - committed, - revision: SUCCESSOR_REVISION, - edit_ramps: &session::EditRamps::new(), - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }, + SuccessorBase::new(inventory, committed, SUCCESSOR_REVISION), backend, ) } diff --git a/crates/host-core/tests/withdrawn_successor.rs b/crates/host-core/tests/withdrawn_successor.rs index 9b71b4663..500d40cbf 100644 --- a/crates/host-core/tests/withdrawn_successor.rs +++ b/crates/host-core/tests/withdrawn_successor.rs @@ -1,8 +1,9 @@ //! Issue #1344: prepare a successor across a withdrawn candidate plan. //! //! P0 is the plan render runs. A was prepared against P0, committed and taken back before render -//! adopted it. B replaces A: it is prepared against P0 with A's `carried_base` and P0's committed -//! model (D2), and takes A's fresh rings by a checked donation (D3-D5). +//! adopted it. B replaces A: it is prepared against P0 with P0's inventory less the sources A did +//! not carry from P0 (`PlanStateInventory::across_withdrawn`, #1698) and P0's committed model +//! (D2), and takes A's fresh rings by a checked donation (D3-D5). //! //! The twin of every case is the path render would have taken had A not been withdrawn: A' //! (prepared against a second copy P0' of P0, rendered identically) adopts P0' render-free, B' is @@ -14,14 +15,10 @@ //! A replaces `t1`'s insert, removes `b`, adds `c`; B replaces `t2`'s (the spec's //! insert-quality edit does not prepare: no launch effect has a second quality). B renders //! bit-identically to B' over 16 blocks, with `a` and `t3` carried, `t1` and `t2` restarted and -//! `b` absent; B prepared with no base does not. (An owner A restarts and B reverts to its P0 -//! form starts fresh here, as in the sequential twin; root ruled that a defect and its fix is a -//! successor issue, so no assertion pins it either way.) -//! * **Gate 2, `carried_base` follows the installed hand-over.** Over a session with a row in -//! every inventory family, where A drops a strip and a source that sort first (so A's keys are -//! not P0's) and restarts what a session edit can restart, A's base holds exactly the P0 rows -//! whose keys A's hand-over reads (set equality of keys), and they belong to exactly the owners -//! both plans hold less those A restarted; a plan with no base has none. +//! `b` absent; B prepared with no base does not. +//! * **#1698 gate 1, an owner B reverts carries from P0.** A restarts or removes an owner and B +//! reverts it: B renders as the combined twin P0 -> B. A source B reverts is fresh in B, and +//! one A carries still carries. //! * **Gate 3, a source absent from P0 is never carried from A.** A adds `d` with 4 seconds, B with //! 5: B's `d` is a fresh ring at generation 1, frame 0, and no donation applies. //! * **Gate 4, donation.** `c` fed through A's producers and seeked (anchored) moves to B whole: @@ -38,18 +35,14 @@ use bench_support::alloc as bench_alloc; use engine::realtime::{CarryOutcome, PlanarBufferMut, PreparedRenderPlan, RenderIo}; use graph_compiler::Backend; use host_core::{ - DonationError, DonationSide, DonationSideMut, HostLiveControlRequest, HostLiveLanes, - LiveValuesSuperseded, PlanStateInventory, PreparedHost, SourceControlSet, SourceSubmission, - SpectrumCaptureRequest, SpectrumChannels, SpectrumTarget, SuccessorBase, apply_donation, - check_donation, drops_acked_fresh_source, prepare_host_runtime_with_live_controls_and_spectrum, - prepare_host_runtime_with_live_controls_and_spectrum_successor, spectrum_capture_resources_for, - test_only_carry_read_keys, test_only_inventory_rows, + DonationError, DonationSide, DonationSideMut, HostLiveControlRequest, PlanStateInventory, + PreparedHost, SourceControlSet, SourceSubmission, SuccessorBase, apply_donation, + check_donation, drops_acked_fresh_source, }; use session::{ - DualMonoFader, Effect, EffectIdentity, EffectQuality, LinkMode, RouteDestination, RouteSource, - SendTap, SessionModel, SidechainDeclaration, Source, StableId, Submix, parse_session_json, + Effect, EffectIdentity, EffectQuality, LinkMode, RouteSource, SendTap, SessionModel, + SidechainDeclaration, Source, StableId, parse_session_json, }; -use std::collections::BTreeSet; use successor::{QUANTUM, SUCCESSOR_REVISION, Session, caps}; const FIXTURE: &str = include_str!("../../../fixtures/session/v1/parametric-eq-nine-track.json"); @@ -277,14 +270,7 @@ fn successor_of( &session.compiled, &caps(), &HostLiveControlRequest::default(), - SuccessorBase { - inventory, - committed, - revision: SUCCESSOR_REVISION, - edit_ramps: &session::EditRamps::new(), - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }, + SuccessorBase::new(inventory, committed, SUCCESSOR_REVISION), backend, ) .map(|(host, _)| host) @@ -408,20 +394,17 @@ fn base_across_a_withdrawn_plan(backend: Backend) { let mut running = running_p0(p0, backend, &signal); // A: prepared against P0, committed and taken back; render never adopts it. let withdrawn = successor_of(a, &running.inventory, &p0.model, backend); - let base = withdrawn.carried_base.as_ref().expect("A has a base"); - let mut replacement = successor_of(b, base, &p0.model, backend); + let base = base_across(&running, &withdrawn); + let mut replacement = successor_of(b, &base, &p0.model, backend); commit(&mut replacement); // `a` carries from P0: its producer moves, and its ring moves at adoption. assert_eq!( replacement.sources.adopt_persisting(&mut running.sources), 1 ); - let a_index = running.inventory.source("a").expect("P0 has a").index; - assert_eq!( - graph::installed_carry_reads(&replacement.plan) - .expect("B has a hand-over") - .sources, - vec![a_index], + let a_index = replacement.inventory.source("a").expect("B has a").index; + assert!( + !replacement.plan.can_prepare_source_seek(a_index as usize), "B carries a's ring from P0" ); assert!(replacement.inventory.source("b").is_none(), "b is absent"); @@ -460,248 +443,6 @@ fn a_successor_prepared_across_a_withdrawn_plan_renders_as_its_twin() { } } -/// Gate 2's session: three sources (`a`, `b`, `c`), sends into a submix (live routes), a true-peak limiter on -/// `t0001`, whose latency the other paths compensate (compensation lines), strip delays on `t` and -/// `t0002`, a compressor on `t`, `t0000` and `t0001` (effect lanes and observation taps), a console -/// slot on every strip, meters, and a spectrum capture on `t0000`. Track `t` and its source `a` -/// sort first, so a plan without them has other keys for every row after them. -fn rich_model() -> SessionModel { - let template = parse_session_json(FIXTURE).expect("fixture parses"); - let mut model = template.clone(); - model.tracks.clear(); - model.routes.clear(); - model.sources = vec![ - source(&template.sources[0], "a", 48_000), - source(&template.sources[0], "b", 48_000), - source(&template.sources[0], "c", 48_000), - ]; - for (track, source) in [ - ("t", "a"), - ("t0000", "b"), - ("t0001", "b"), - ("t0002", "c"), - ("t0003", "b"), - ] { - add_track(&mut model, &template, track, source, true); - } - for track in &mut model.tracks { - if track.id.as_str() == "t0001" { - track.inserts.effects.push(Effect { - id: id("limiter"), - identity: EffectIdentity::Native { - effect_id: id("miso.true-peak-limiter"), - }, - ..compressor("limiter") - }); - } - if matches!(track.id.as_str(), "t0002" | "t0003") { - track.inserts.effects.clear(); - } - if matches!(track.id.as_str(), "t" | "t0002") { - track.builtins.left.delay_samples = 64; - track.builtins.right.delay_samples = 32; - } - } - let bus = id("bus"); - model - .submixes - .push(Submix::unity(bus.clone(), &model.console)); - for track in ["t", "t0000", "t0001", "t0003"] { - let mut send = model.routes[0].clone(); - send.id = id(&format!("{track}-bus")); - send.source = RouteSource::Track { - track_id: id(track), - tap: SendTap::PreFader, - }; - send.destination = RouteDestination::SubmixInput { - submix_id: bus.clone(), - }; - model.routes.push(send); - } - let mut out = model.routes[0].clone(); - out.id = id("bus-main"); - out.source = RouteSource::Submix { - submix_id: bus, - tap: SendTap::PostPan, - }; - model.routes.push(out); - // A fader of its own on `t0003`, so the strips' values are not all equal. - model.tracks[4].fader = DualMonoFader { - left_db: -6.0, - right_db: -6.0, - left_mute: false, - right_mute: false, - }; - model -} - -/// Gate 2's A: P0 without track `t` (and its routes) and source `a`, which sort first, so every -/// family loses `t`'s (or `a`'s) row; and a restart where a session edit restarts an owner that -/// stays: `c` redeclared (sources) and `t0000`'s insert replaced (its effect lane and observation -/// taps). Under live input lanes an input-section or strip-stage value edit carries and -/// retargets, so those families restart only by removal; the plan's one spectrum capture carries, -/// since restarting it would leave the family unread. -fn rich_a_model() -> SessionModel { - let mut model = rich_model(); - remove_track(&mut model, "t"); - model.routes.retain(|route| route.id.as_str() != "t-bus"); - model.sources.retain(|source| source.id.as_str() != "a"); - for source in &mut model.sources { - if source.id.as_str() == "c" { - source.frames = 2 * 48_000; - } - } - replace_insert(&mut model, "t0000"); - model -} - -fn live_request() -> HostLiveControlRequest { - HostLiveControlRequest { - meter_period_frames: core::num::NonZeroU32::new(QUANTUM as u32), - observation_taps: 1, - ..HostLiveControlRequest::default() - } -} - -fn spectrum_request() -> SpectrumCaptureRequest { - let target = SpectrumTarget::TrackPostMatrix("t0000".into()); - SpectrumCaptureRequest { - maximum_capture_bytes: spectrum_capture_resources_for(&target).retained_bytes, - target, - channels: SpectrumChannels::Stereo, - } -} - -fn prepare_rich(session: &Session, base: Option>) -> PreparedHost { - let live = live_request(); - let superseded = LiveValuesSuperseded::default(); - let spectrum = spectrum_request(); - match base { - None => prepare_host_runtime_with_live_controls_and_spectrum( - &session.compiled, - &caps(), - &live, - &spectrum, - HostLiveLanes::ALL, - &superseded, - ), - Some(base) => prepare_host_runtime_with_live_controls_and_spectrum_successor( - &session.compiled, - &caps(), - &live, - &spectrum, - HostLiveLanes::ALL, - &superseded, - base, - ), - } - .map(|(host, _, _)| host) - .unwrap_or_else(|failure| panic!("prepare: {failure:?}")) -} - -/// `(family, owner)` of every row of `rows`. -fn owners(rows: &BTreeSet<(&'static str, String, String)>) -> BTreeSet<(&'static str, String)> { - rows.iter() - .map(|(family, owner, _)| (*family, owner.clone())) - .collect() -} - -/// `(family, key)` of every row of `rows`. -fn keys(rows: &BTreeSet<(&'static str, String, String)>) -> BTreeSet<(&'static str, String)> { - rows.iter() - .map(|(family, _, key)| (*family, key.clone())) - .collect() -} - -/// Gate 2, tightened by the batch 15a fold (M3). Red if the restriction keeps a row the hand-over -/// does not read (an unfiltered `restricted_to`: the removed `t`, `a` and every restarted -/// owner would stay), or if `installed_carry_reads` reports successor keys for predecessor keys -/// (A lacks `t` and `a`, which sort first, so A's keys name other P0 rows: the owners differ). -#[test] -fn the_carried_base_holds_exactly_the_rows_the_hand_over_reads() { - let p0 = Session::compile(rich_model()); - let a = Session::compile(rich_a_model()); - let running = prepare_rich(&p0, None); - assert!( - running.carried_base.is_none(), - "a plan with no base has none" - ); - assert_eq!(running.report.carried_base_retained_bytes, 0); - let withdrawn = prepare_rich( - &a, - Some(SuccessorBase { - inventory: &running.inventory, - committed: &p0.model, - revision: SUCCESSOR_REVISION, - edit_ramps: &session::EditRamps::new(), - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, - }), - ); - let base = withdrawn.carried_base.as_ref().expect("A has a base"); - assert_eq!(base.plan_identity(), running.inventory.plan_identity()); - assert_eq!( - withdrawn.report.carried_base_retained_bytes, - base.retained_bytes() - ); - assert!(base.spectrum_pairing().is_empty()); - let reads = graph::installed_carry_reads(&withdrawn.plan).expect("A has a hand-over"); - assert_eq!(reads.predecessor, running.inventory.plan_identity()); - - let p0_rows = test_only_inventory_rows(&running.inventory); - let a_rows = test_only_inventory_rows(&withdrawn.inventory); - let base_rows = test_only_inventory_rows(base); - let read_keys = test_only_carry_read_keys(&reads); - // Every family is exercised by the hand-over. - for family in [ - "sources", - "input sections", - "strip stages", - "effect lanes", - "meters", - "observations", - "spectrum", - "delay lines", - "strip delays", - "live routes", - ] { - assert!( - read_keys.iter().any(|(read, _)| *read == family), - "{family}: the session exercises the family" - ); - } - // Every base row is P0's row, unchanged. - assert!(base_rows.is_subset(&p0_rows), "base rows are P0 rows"); - // The base holds exactly the keys the hand-over reads: no more, no fewer. - assert_eq!(keys(&base_rows), read_keys, "base keys equal read keys"); - // Not vacuous: A's own keys differ from P0's, so a successor-key read names other rows. - assert_ne!(keys(&a_rows), keys(&p0_rows), "A's keys are not P0's"); - // The owners: those both plans hold, less those A restarted. Independent of keys. - let restarted = |(family, owner): &(&'static str, String)| match *family { - "sources" => owner == "c", - "effect lanes" | "observations" => owner.starts_with("t0000 ") && owner.ends_with(" comp"), - _ => false, - }; - let expected: BTreeSet<_> = owners(&p0_rows) - .intersection(&owners(&a_rows)) - .filter(|owner| !restarted(owner)) - .cloned() - .collect(); - assert_eq!( - owners(&base_rows), - expected, - "base owners are A's carried owners" - ); - for owner in owners(&p0_rows) { - if restarted(&owner) || !owners(&a_rows).contains(&owner) { - assert!( - !owners(&base_rows).contains(&owner), - "{owner:?}: removed or restarted, so not in the base" - ); - } - } -} - /// Gate 3: P0 lacks `d`; A adds it with 4 seconds of frames, B with 5. #[test] fn a_source_absent_from_the_running_plan_is_never_carried_from_the_withdrawn_one() { @@ -725,19 +466,17 @@ fn a_source_absent_from_the_running_plan_is_never_carried_from_the_withdrawn_one for block in 0..2 { submit(&mut withdrawn.sources, "d", &signal[0], 1, block).expect("feed A's d"); } - let base = withdrawn.carried_base.as_ref().expect("A has a base"); + let base = base_across(&running, &withdrawn); assert!( base.source("d").is_none(), - "P0 has no d, so A's base has none" + "P0 has no d, so B's base has none" ); - let mut replacement = successor_of(&b, base, &p0.model, backend); + let mut replacement = successor_of(&b, &base, &p0.model, backend); let d_index = replacement.inventory.source("d").expect("B has d").index; - assert_eq!( - graph::installed_carry_reads(&replacement.plan) - .expect("B carries a") - .sources, - vec![running.inventory.source("a").expect("P0 has a").index], - "B carries a from P0 and nothing for d" + let a_index = replacement.inventory.source("a").expect("B has a").index; + assert!( + !replacement.plan.can_prepare_source_seek(a_index as usize), + "B carries a from P0" ); assert!( replacement.plan.can_prepare_source_seek(d_index as usize), @@ -774,7 +513,7 @@ fn donation_plays_the_fed_ring(backend: Backend) { let mut running = running_p0(p0, backend, &signal); let mut withdrawn = successor_of(a, &running.inventory, &p0.model, backend); feed_c(&mut withdrawn.sources, &c_signal); - let base = withdrawn.carried_base.clone().expect("A has a base"); + let base = base_across(&running, &withdrawn); let mut replacement = successor_of(b, &base, &p0.model, backend); commit(&mut replacement); if donate { @@ -891,7 +630,7 @@ fn a_refused_or_mismatched_donation_moves_nothing() { let mut withdrawn = successor_of(a, &running.inventory, &p0.model, backend); feed_four(&mut withdrawn.sources); render(&mut withdrawn.plan); - let base = withdrawn.carried_base.clone().expect("A has a base"); + let base = base_across(&running, &withdrawn); let mut replacement = successor_of(b, &base, &p0.model, backend); let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); assert_eq!(check_donation(into, from), Err(DonationError::Consumed)); @@ -915,7 +654,7 @@ fn a_refused_or_mismatched_donation_moves_nothing() { let running = running_p0(p0, backend, &signal); let mut withdrawn = successor_of(a, &running.inventory, &p0.model, backend); feed_four(&mut withdrawn.sources); - let base = withdrawn.carried_base.clone().expect("A has a base"); + let base = base_across(&running, &withdrawn); let mut replacement = session .prepare_successor(&base, &p0.model, &caps, backend) .unwrap_or_else(|failure| panic!("{what}: successor: {failure:?}")); @@ -957,7 +696,7 @@ fn a_donation_never_separates_an_acked_ring_from_its_producer() { for block in 0..4 { submit(&mut withdrawn.sources, "c", &signal[0], 1, block).expect("feed A's c"); } - let base = withdrawn.carried_base.clone().expect("A has a base"); + let base = base_across(&running, &withdrawn); let replacement = successor_of(b, &base, &p0.model, backend); (withdrawn, replacement) }; @@ -1018,7 +757,7 @@ fn fed_withdrawn_and_replacement( for block in 0..4 { submit(&mut withdrawn.sources, "c", &signal[0], 1, block).expect("feed A's c"); } - let base = withdrawn.carried_base.clone().expect("A has a base"); + let base = base_across(&running, &withdrawn); let replacement = successor_of(b, &base, &p0.model, backend); (withdrawn, replacement) } @@ -1174,7 +913,7 @@ fn guard_refuses(a: SessionModel, b: SessionModel, feed: &str, carry: bool) -> b submit(&mut withdrawn.sources, feed, &signal[0], 1, block).expect("feed A"); } } - let base = withdrawn.carried_base.clone().expect("A has a base"); + let base = base_across(&running, &withdrawn); let replacement = successor_of(&b, &base, &p0.model, backend); let (into, from) = sides(&replacement, &b.model, &withdrawn, &a.model); let plan = check_donation(into, from).expect("the donation checks"); @@ -1277,3 +1016,139 @@ fn the_guard_refuses_exactly_a_fed_fresh_ring_the_donation_leaves_behind() { assert_eq!(guard_refuses(a, b, feed, carry), refused, "{label}"); } } + +/// The base inventory a replacement of the withdrawn A is prepared from, as the control plane +/// builds it (#1698). +fn base_across(running: &PreparedHost, withdrawn: &PreparedHost) -> PlanStateInventory { + running + .inventory + .across_withdrawn(&withdrawn.inventory, &withdrawn.plan) +} + +/// The combined twin of #1698 gate 1: B prepared against P0 directly (one transaction P0 -> B) +/// and adopted, then rendered [`AFTER`] blocks. +fn direct_twin( + p0: &Session, + b: &Session, + backend: Backend, + signal: &[[Vec; 2]; 2], +) -> Vec> { + let mut running = running_p0(p0, backend, signal); + let mut twin = successor_of(b, &running.inventory, &p0.model, backend); + commit(&mut twin); + twin.sources.adopt_persisting(&mut running.sources); + assert_eq!( + twin.plan.adopt_predecessor_plan(&mut running.plan), + CarryOutcome::Carried + ); + play(&mut twin, signal) +} + +/// #1698 gate 1 at one width, for one A: B is P0 with `t2`'s insert replaced, so it reverts to +/// its P0 form the owner A changed. The producers move as the control plane moves them: A takes +/// those of the sources it carries at its commit, B takes them from A, and P0 takes back what B +/// left. +fn reverted_owner_carries_from_p0(a_model: SessionModel, backend: Backend) { + let p0 = Session::compile(p0_model()); + let a = Session::compile(a_model); + let mut b_model = p0_model(); + replace_insert(&mut b_model, "t2"); + let b = Session::compile(b_model); + let signal = signal(); + let twin = direct_twin(&p0, &b, backend, &signal); + + let mut running = running_p0(&p0, backend, &signal); + let mut withdrawn = successor_of(&a, &running.inventory, &p0.model, backend); + withdrawn.sources.adopt_persisting(&mut running.sources); + let base = base_across(&running, &withdrawn); + let mut replacement = successor_of(&b, &base, &p0.model, backend); + commit(&mut replacement); + replacement.sources.adopt_persisting(&mut withdrawn.sources); + running.sources.adopt_persisting(&mut withdrawn.sources); + assert_eq!( + replacement.restarted_strips(), + [Box::::from("t2")], + "{backend:?}: only t2 restarts; the reverted owner carries from P0" + ); + assert_eq!( + replacement.plan.adopt_predecessor_plan(&mut running.plan), + CarryOutcome::Carried + ); + let blocks = play(&mut replacement, &signal); + for (block, (left, right)) in blocks.iter().zip(&twin).enumerate() { + assert_eq!(left, right, "{backend:?} block {block}: B equals P0 -> B"); + } +} + +/// #1698 gate 1 (owners): A restarts `t1` (its compressor insert replaced, so the strip restarts) +/// or removes `t3` (strip, input section, transient shaper and route), and B reverts it to its P0 +/// form. The owner carries from P0, so B renders bit-identically to the combined twin P0 -> B at +/// both lane widths, and only `t2`, which B itself edits, restarts. +/// +/// Test value: red if an owner A restarted or removed starts fresh in B (the batch 15a base, +/// P0's inventory restricted to the rows A's hand-over reads). +#[test] +fn an_owner_a_restarts_or_removes_and_b_reverts_carries_from_p0() { + let restarted = || { + let mut model = p0_model(); + replace_insert(&mut model, "t1"); + model + }; + let removed = || { + let mut model = p0_model(); + remove_track(&mut model, "t3"); + model + }; + for backend in backends() { + reverted_owner_carries_from_p0(restarted(), backend); + reverted_owner_carries_from_p0(removed(), backend); + } +} + +/// #1698, sources stay under the host contract: a source declared differently in the newest +/// committed session restarts at generation 1, frame 0 (`CONTROL_PROTOCOL_SEMANTICS.md`). A +/// redeclares or removes P0's `b`, and B declares it as P0 does: `b` is fresh in B, a ring of its +/// own that takes generation 1 from frame 0, while `a`, which A carries from P0, still carries. +/// +/// Test value: red if B's base carries a source A did not carry from P0 (P0's full inventory +/// without the source filter), which would refuse the host's restarted feed of `b`. +#[test] +fn a_source_b_reverts_is_fresh_and_a_source_a_carries_still_carries() { + let backend = Backend::current(); + let p0 = Session::compile(p0_model()); + let signal = signal(); + for (label, a_model) in [ + ("b redeclared", with_b_frames(40_000)), + ("b removed", without(p0_model(), "b", "tb")), + ] { + let a = Session::compile(a_model); + let mut running = running_p0(&p0, backend, &signal); + let mut withdrawn = successor_of(&a, &running.inventory, &p0.model, backend); + withdrawn.sources.adopt_persisting(&mut running.sources); + let base = base_across(&running, &withdrawn); + let mut replacement = successor_of(&p0, &base, &p0.model, backend); + let index = |source: &str| { + replacement + .inventory + .source(source) + .expect("B declares it") + .index as usize + }; + assert!( + replacement.plan.can_prepare_source_seek(index("b")), + "{label}: B's b owns a fresh ring" + ); + assert!( + !replacement.plan.can_prepare_source_seek(index("a")), + "{label}: B's a is vacant, carried from P0" + ); + submit(&mut replacement.sources, "b", &signal[1], 1, 0).unwrap_or_else(|error| { + panic!("{label}: B's b takes generation 1 at frame 0: {error:?}") + }); + assert_eq!( + replacement.sources.adopt_persisting(&mut withdrawn.sources), + 1, + "{label}: a's producer moves from A" + ); + } +} diff --git a/crates/source/src/lib.rs b/crates/source/src/lib.rs index 94f868806..5f1887362 100644 --- a/crates/source/src/lib.rs +++ b/crates/source/src/lib.rs @@ -408,6 +408,11 @@ pub struct SourceReadReport { pub underrun_frames: u32, /// Whether this call contained one maximal missing in-region run. pub underrun_event: bool, + /// In-region frames emitted as positive zero because an anchored seek is held: the host asked + /// the source to wait, so the frames are not an underrun. + pub held_frames: u32, + /// Whether this call was a held block (`held_frames != 0`). + pub held: bool, /// Whether the source is at a declared end-of-region after this call. pub end_of_region: bool, /// Active generation used for this render quantum. @@ -420,6 +425,10 @@ pub struct SourceReadReport { pub cumulative_underrun_frames: u64, /// Saturating cumulative missing-run count. pub cumulative_underrun_events: u64, + /// Saturating cumulative held in-region frame count. + pub cumulative_held_frames: u64, + /// Saturating cumulative held block count. + pub cumulative_held_blocks: u64, } /// Owner-local producer telemetry copied only outside render. @@ -442,6 +451,10 @@ pub struct SourceConsumerTelemetry { pub stale_generation_discard_count: u64, pub underrun_frames: u64, pub underrun_events: u64, + /// Frames rendered as silence while an anchored seek was held (not underruns). + pub held_frames: u64, + /// Blocks rendered as silence while an anchored seek was held (not underruns). + pub held_blocks: u64, pub end_of_region: bool, /// Native-decoder replacements, or zero for a host-supplied consumer. pub native_decoder_sanitized_samples: u64, @@ -612,6 +625,8 @@ impl PcmSourceRing { stale_generation_discard_count: 0, underrun_frames: 0, underrun_events: 0, + held_frames: 0, + held_blocks: 0, native_decoder_sanitized_samples: 0, generation_changed: false, held_seek: None, @@ -1013,6 +1028,8 @@ pub struct PcmSourceConsumer { stale_generation_discard_count: u64, underrun_frames: u64, underrun_events: u64, + held_frames: u64, + held_blocks: u64, native_decoder_sanitized_samples: u64, generation_changed: bool, /// An observed [`SourceCommand::SeekAt`] whose anchor block has not begun yet (issue #1274). @@ -1114,8 +1131,9 @@ impl PcmSourceConsumer { /// anchor; the block that starts at or past it applies the seek at `frame + (first_sample - /// anchor_sample)` (saturating). While it is held, queued PCM of the playing generation still /// plays, the new generation's first block is kept as the pending block, and with nothing - /// playable the block underruns. A newer command replaces a held one. This render-path - /// operation allocates nothing and never blocks. + /// playable the block is **held**, not an underrun (the host asked the source to wait; underrun + /// means the host owed PCM and did not deliver it). A newer command replaces a held one. This + /// render-path operation allocates nothing and never blocks. pub fn begin_block_at(&mut self, first_sample: u64) -> SourceReadReport { self.begin_block_with(SeekClock::At(first_sample)) } @@ -1141,6 +1159,7 @@ impl PcmSourceConsumer { self.acquire_current_block(clock); let mut copied_frames = 0_u32; let mut underrun_frames = 0_u32; + let mut held_frames = 0_u32; if self.end_reached() { self.end_of_region = true; } else if self.current_matches_next_frame() { @@ -1161,7 +1180,7 @@ impl PcmSourceConsumer { .end_frame .map(|end| end.0.saturating_sub(self.next_frame.0)) .unwrap_or(u64::from(self.quantum_frames)); - underrun_frames = u32::try_from(cmp::min( + let missing_frames = u32::try_from(cmp::min( u64::from(self.quantum_frames), available_until_end, )) @@ -1171,11 +1190,22 @@ impl PcmSourceConsumer { .0 .saturating_add(u64::from(self.quantum_frames)), ); - if underrun_frames != 0 { - self.underrun_frames = self - .underrun_frames - .saturating_add(u64::from(underrun_frames)); - self.underrun_events = self.underrun_events.saturating_add(1); + // After the acquire step, which can set or apply a held seek (`acquire_current_block`): + // missing PCM while an anchored seek is still held is a planned wait, not starvation. + if self.held_seek.is_some() { + held_frames = missing_frames; + if held_frames != 0 { + self.held_frames = self.held_frames.saturating_add(u64::from(held_frames)); + self.held_blocks = self.held_blocks.saturating_add(1); + } + } else { + underrun_frames = missing_frames; + if underrun_frames != 0 { + self.underrun_frames = self + .underrun_frames + .saturating_add(u64::from(underrun_frames)); + self.underrun_events = self.underrun_events.saturating_add(1); + } } if self.end_reached() { self.end_of_region = true; @@ -1185,12 +1215,16 @@ impl PcmSourceConsumer { copied_frames, underrun_frames, underrun_event: underrun_frames != 0, + held_frames, + held: held_frames != 0, end_of_region: self.end_of_region, active_generation: self.active_generation, generation_changed: self.generation_changed, cumulative_read_frames: self.cumulative_read_frames, cumulative_underrun_frames: self.underrun_frames, cumulative_underrun_events: self.underrun_events, + cumulative_held_frames: self.held_frames, + cumulative_held_blocks: self.held_blocks, } } @@ -1308,6 +1342,8 @@ impl PcmSourceConsumer { stale_generation_discard_count: self.stale_generation_discard_count, underrun_frames: self.underrun_frames, underrun_events: self.underrun_events, + held_frames: self.held_frames, + held_blocks: self.held_blocks, end_of_region: self.end_of_region, native_decoder_sanitized_samples: self.native_decoder_sanitized_samples, } @@ -2330,10 +2366,16 @@ mod tests { host.try_seek(seek_at(2, 4, 8)).unwrap(); host.submit(chunk(2, 4, &[&[2.0; 4]], 4, false)).unwrap(); let (output, report) = render_at(&mut consumer, 0); - assert_eq!((output, report.underrun_frames), ([0.0; 4], 4)); - // The old generation's underrun has reached frame 4, the primed block's start. + assert_eq!( + (output, report.held_frames, report.underrun_frames), + ([0.0; 4], 4, 0) + ); + // The old generation's held wait has reached frame 4, the primed block's start. let (output, report) = render_at(&mut consumer, 4); - assert_eq!((output, report.underrun_frames), ([0.0; 4], 4)); + assert_eq!( + (output, report.held_frames, report.underrun_frames), + ([0.0; 4], 4, 0) + ); assert_eq!(report.active_generation, SourceGeneration(1)); let (output, report) = render_at(&mut consumer, 8); assert_eq!((output, report.copied_frames), ([2.0; 4], 4)); @@ -2462,7 +2504,9 @@ mod tests { } assert_eq!(consumer.active_generation, SourceGeneration(2)); assert_eq!(consumer.stale_generation_discard_count, 0); - assert_eq!(consumer.underrun_events, u64::from(anchored)); + // The anchored run waited one block for its anchor: held, never an underrun. + assert_eq!(consumer.underrun_events, 0); + assert_eq!(consumer.held_blocks, u64::from(anchored)); } // An anchored seek whose anchor has already passed when the window's re-observe takes @@ -2878,6 +2922,134 @@ mod tests { ); } + /// #1318 gate 3. Red if the graph source driver still folds a held block into the graph-wide + /// `source_underrun` flag (a planned wait would mark every spectrum window as starved), or if + /// the fix swallows a true underrun on another source in the same block. + #[test] + fn graph_driver_does_not_flag_a_held_block_as_an_underrun() { + let (producer, held, _) = PcmSourceRing::prepare(config(1, 4, 8)).expect("held ring"); + let mut host = producer.into_host_chunk_provider(RATE); + let (_other_producer, starved, _) = + PcmSourceRing::prepare(config(1, 4, 8)).expect("starved ring"); + host.try_seek(seek_at(2, 100, 16)).expect("anchored seek"); + let mut driver = test_driver(held, vec![test_mapping(0)]); + driver.begin_block(0, 4).expect("held block"); + assert!( + !driver.observation_validity().source_underrun, + "a held block is a planned wait" + ); + assert!( + !driver.sources[0] + .consumer + .as_ref() + .unwrap() + .held_seek + .is_none() + ); + let telemetry = driver.sources[0].consumer.as_ref().unwrap().telemetry(); + assert_eq!((telemetry.held_blocks, telemetry.underrun_events), (1, 0)); + + // A second source with nothing queued and no held seek is a true underrun in the same + // block, and sets the flag. + driver.sources = vec![ + GraphSourceEntry { + channel_count: 1, + consumer: driver.sources[0].consumer.take(), + }, + GraphSourceEntry { + channel_count: 1, + consumer: Some(starved), + }, + ] + .into_boxed_slice(); + driver.mappings = vec![ + test_mapping(0), + SourceGraphTrackMapping { + source_index: 1, + ..test_mapping(1) + }, + ] + .into_boxed_slice(); + driver.begin_block(4, 4).expect("held and underrun block"); + assert!(driver.observation_validity().source_underrun); + let telemetry = driver.sources[0].consumer.as_ref().unwrap().telemetry(); + assert_eq!((telemetry.held_blocks, telemetry.underrun_events), (2, 0)); + let telemetry = driver.sources[1].consumer.as_ref().unwrap().telemetry(); + assert_eq!((telemetry.held_blocks, telemetry.underrun_events), (0, 1)); + } + + /// #1318 gate 1. Red if a block waiting for its anchor still counts as an underrun, or if + /// "held" outlives the anchor (the block at the anchor applied the seek: a host that owes PCM + /// then and delivers none is starved, not waiting). + #[test] + fn a_wait_for_an_anchor_is_held_and_starvation_after_it_is_an_underrun() { + for delivered in [true, false] { + let (producer, mut consumer, _) = PcmSourceRing::prepare(config(1, 4, 8)).unwrap(); + let mut host = producer.into_host_chunk_provider(RATE); + host.try_seek(seek_at(2, 100, 16)).unwrap(); + if delivered { + host.submit(chunk(2, 100, &[&[2.0; 4]], 4, false)).unwrap(); + } + for block in 0..4_u64 { + let (output, report) = render_at(&mut consumer, 4 * block); + assert_eq!(output, [0.0; 4]); + assert!(report.held, "block {block}"); + assert_eq!(report.held_frames, 4); + assert!(!report.underrun_event, "block {block}"); + assert_eq!(report.underrun_frames, 0); + assert_eq!(report.cumulative_held_blocks, block + 1); + assert_eq!(report.cumulative_held_frames, 4 * (block + 1)); + assert_eq!(report.cumulative_underrun_events, 0); + assert_eq!(report.cumulative_underrun_frames, 0); + } + let (output, report) = render_at(&mut consumer, 16); + assert_eq!(report.active_generation, SourceGeneration(2)); + assert!(report.generation_changed); + assert!(!report.held, "the seek applied at its anchor"); + assert_eq!(report.held_frames, 0); + assert_eq!(report.cumulative_held_blocks, 4); + if delivered { + assert_eq!(output, [2.0; 4]); + assert_eq!(report.copied_frames, 4); + assert!(!report.underrun_event); + assert_eq!(report.cumulative_underrun_events, 0); + } else { + assert_eq!(output, [0.0; 4]); + assert!(report.underrun_event, "the host owed PCM at the anchor"); + assert_eq!(report.underrun_frames, 4); + assert_eq!(report.cumulative_underrun_events, 1); + } + let telemetry = consumer.telemetry(); + assert_eq!((telemetry.held_blocks, telemetry.held_frames), (4, 16)); + } + } + + /// #1318 gate 2. Red if held is decided by `held_seek` alone: the playing generation's + /// queued PCM still plays while a seek is held, and those blocks are neither held nor + /// underruns. + #[test] + fn queued_pcm_of_the_playing_generation_plays_while_a_seek_is_held() { + let (producer, mut consumer, _) = PcmSourceRing::prepare(config(1, 4, 8)).unwrap(); + let mut host = producer.into_host_chunk_provider(RATE); + host.submit(chunk(1, 0, &[&[1.0; 4]], 4, false)).unwrap(); + host.submit(chunk(1, 4, &[&[1.5; 4]], 4, false)).unwrap(); + host.try_seek(seek_at(2, 100, 16)).unwrap(); + for (block, expected) in [(0_u64, 1.0), (1, 1.5)] { + let (output, report) = render_at(&mut consumer, 4 * block); + assert_eq!(output, [expected; 4]); + assert_eq!(report.copied_frames, 4); + assert!(!report.held && !report.underrun_event, "block {block}"); + assert_eq!(report.cumulative_held_blocks, 0); + } + for block in 2..4_u64 { + let (output, report) = render_at(&mut consumer, 4 * block); + assert_eq!(output, [0.0; 4]); + assert!(report.held && !report.underrun_event, "block {block}"); + assert_eq!(report.cumulative_held_blocks, block - 1); + } + assert_eq!(consumer.telemetry().underrun_events, 0); + } + /// #917 restates the pre-change `..._last_claim_recycles_in_call_...` test for the hold rule: /// no claim path releases the played block, the next `begin_block` does, and the producer /// keeps its configured one-block depth while the render holds its block. The old incomplete diff --git a/docs/CONTROL_PROTOCOL_SEMANTICS.md b/docs/CONTROL_PROTOCOL_SEMANTICS.md index b890f79f7..67f86dca7 100644 --- a/docs/CONTROL_PROTOCOL_SEMANTICS.md +++ b/docs/CONTROL_PROTOCOL_SEMANTICS.md @@ -6,7 +6,7 @@ This defines controller behavior for the [BTLV bytes](CONTROL_BTLV_V1.md) and [r The committed `SessionModel` owns its revision, and the revision lies in `0..=9223372036854775807` (2^63 - 1, the session schema's domain; the wire field stays `u64`); command `revision-any` is therefore the header flag plus a zero carrier, never a revision sentinel. Structural edits require an exact revision and at least one edit; a configured zero edit limit disables the transaction command and its paired session events in capabilities and dispatch. Transactions clone/apply/compile a candidate typed session and replace `(SessionModel, CompiledSession, SessionRevision)` only when all validation, capacity reservation, and compilation steps succeed. One success increments once; malformed input, empty input, validation/compile/resource failure, backpressure, and replay leave model and revision untouched. A mutation at revision 2^63 - 1 returns `REVISION_EXHAUSTED` before the candidate is built; the controller does not turn a compiled session into a `PreparedRenderPlan`. -A transaction that needs a new plan while an earlier one's replacement plan is still unadopted commits as any other (#1310): the host takes the waiting replacement back, prepares one successor of the running plan that carries both, and publishes it, or, should the new transaction be refused, publishes the waiting replacement again unchanged. Host-fed sources the earlier transaction added keep their accepted PCM and queued seeks. No accepted chunk is dropped across a supersession: when the waiting replacement created a source's ring (the earlier transaction added the source, or redeclared it) and the host has fed that ring (a chunk, a seek or an end of region), a new transaction that does not keep the source as the replacement declares it (it declares it as the running plan does, redeclares it another way, or removes it) is refused with backpressure before it commits. Nothing changes (the replacement stays published, no event, no revision), and a retry after render adopts the replacement commits. A host that does not render keeps getting backpressure for such a transaction until it renders. A source the replacement carries unchanged from the running plan is not refused this way. The successor issue #1698 lifts the refusal only where the new transaction declares the source as the running plan does; the other cases stay refused until an issue lets the successor read those chunks or the owner rules that a removed source's chunks may drop. A source the caller's own transaction removes discards its queued audio, whether the running plan or the waiting replacement (carrying it unchanged from the running plan) holds it: the caller asked for the removal, so this is not an unintended drop. No counter or telemetry field reports those frames (source telemetry leaves with the source; the watermark outcome flags and the session counters name none); a counter for the discarded frames is a successor issue (placeholder #1699). The applied-revision watermark completes the displaced revisions as `superseded` when the successor is adopted. The resource admission counts every plan the session still holds (the running plan, the waiting replacement, and a plan render has displaced until the service step reclaims it) and the successor. A resource refusal is backpressure when the successor fit against the newest held plan alone, so a retry after a render normally commits; the split is approximate, and such a retry may still be refused within a small band (a successor prepared on the adopted replacement differs by a few hundred bytes from one prepared across it). It is a compile rejection when the successor does not fit even then. +A transaction that needs a new plan while an earlier one's replacement plan is still unadopted commits as any other (#1310): the host takes the waiting replacement back, prepares one successor of the running plan that carries both, and publishes it, or, should the new transaction be refused, publishes the waiting replacement again unchanged. Host-fed sources the earlier transaction added keep their accepted PCM and queued seeks. No accepted chunk is dropped across a supersession: when the waiting replacement created a source's ring (the earlier transaction added the source, or redeclared it) and the host has fed that ring (a chunk, a seek or an end of region), a new transaction that does not keep the source as the replacement declares it (it declares it as the running plan does, redeclares it another way, or removes it) is refused with backpressure before it commits. Nothing changes (the replacement stays published, no event, no revision), and a retry after render adopts the replacement commits. A host that does not render keeps getting backpressure for such a transaction until it renders. A source the replacement carries unchanged from the running plan is not refused this way. Every case stays refused, the one where the new transaction declares the source as the running plan does included (#1698): the host restarted its feed of the source against the waiting replacement, so the successor starts the source fresh and cannot play that replacement's chunks; the cases stay refused until an issue lets the successor read those chunks or the owner rules that a removed source's chunks may drop. Everything else the new transaction leaves as the running plan has it (a strip, an effect, a meter) carries from the running plan as one direct transaction would carry it, also when the waiting replacement had restarted or removed it (#1698). A source the caller's own transaction removes discards its queued audio, whether the running plan or the waiting replacement (carrying it unchanged from the running plan) holds it: the caller asked for the removal, so this is not an unintended drop. No counter or telemetry field reports those frames (source telemetry leaves with the source; the watermark outcome flags and the session counters name none); a counter for the discarded frames is a successor issue (placeholder #1699). The applied-revision watermark completes the displaced revisions as `superseded` when the successor is adopted. The resource admission counts every plan the session still holds (the running plan, the waiting replacement, and a plan render has displaced until the service step reclaims it) and the successor. A resource refusal is backpressure when the successor fit against the newest held plan alone, so a retry after a render normally commits; the split is approximate, and such a retry may still be refused within a small band (a successor prepared on the adopted replacement differs by a few hundred bytes from one prepared across it). It is a compile rejection when the successor does not fit even then. All v1 operations are absolute and idempotent. A new logical command has a nonzero, strictly increasing endpoint-local request ID. The bounded replay cache stores exact canonical request and response bytes. Identical repeated bytes return the cached response without execution; same ID with changed bytes returns `REQUEST_ID_REUSE`; evicted/retired IDs return `REPLAY_EXPIRED`. Completed backpressure is replayed too, so retrying it uses a new ID. Cache capacity is reserved before execution; an unretainable response is rejected before state changes. The guarantee ends with the endpoint lifetime; reconnect recovery belongs to an adapter. diff --git a/docs/C_ABI_V1_QUALIFICATION.md b/docs/C_ABI_V1_QUALIFICATION.md index a8c4ed2e2..778950a48 100644 --- a/docs/C_ABI_V1_QUALIFICATION.md +++ b/docs/C_ABI_V1_QUALIFICATION.md @@ -826,15 +826,16 @@ that window is admitted against the plan that renders. compiled models. The repository's reference limits (`tools/audit/src/capi.rs`, `crates/capi/tests/plan_swap_race.rs`, `crates/capi/tests/resource_lifecycle.rs`) admit three plans of the nine-track EQ reference session with room to spare (graph 316,350 bytes against - 100,000,000; capi 168,810 against 10,000,000), so none was raised. The rule is measured from the - row of a running successor plan, which carries its sources and owns their rings. The browser's caps are its - one memory budget, which charges the same tables through the same control-plane code; its + 100,000,000; capi 167,754 against 10,000,000), so none was raised. The rule is measured from the + row of a running successor plan, which carries its sources and owns their rings. The browser's + caps are its one memory budget, which charges the same tables through the same control-plane code; its replacement-peak term must cover the three-plan peak when a warm candidate is admitted (#1403). - **Resource movement.** The larger tables add fixed bytes to every session's `capi_retained_bytes` (+1,904 bytes, #1398 D1's tables). On the nine-track EQ reference session - the row measures 165,842 bytes at the batch 15a head, 165,922 after its review fold and 168,810 - after fold 2 (x86-64; the ceiling in `crates/capi/tests/resource_lifecycle.rs` is 175,296). A caller whose - `maximum_capi_retained_bytes` was exact must raise it. No symbol, result code or event is added. + the row measures 165,842 bytes at the batch 15a head, 165,922 after its review fold, 168,810 + after fold 2, 167,514 after #1698 and 167,754 after #1285's floors (+240: the `input_floors` and + `raised_arrivals` tables; x86-64; the ceiling in `crates/capi/tests/resource_lifecycle.rs` is + 175,296). A caller whose `maximum_capi_retained_bytes` was exact must raise it. No symbol, result code or event is added. ## Supersession of an unadopted candidate (#1310) @@ -868,7 +869,9 @@ that window is admitted against the plan that renders. getting `RESULT_BACKPRESSURE` until it renders. A source the candidate carries unchanged from the running plan is not refused. Both hosts answer alike (`control_parity::a_supersession_that_would_drop_acked_pcm_is_backpressure_alike_on_both_ - hosts`, cases reverted, redeclared a third way and removed). #1698 lifts only the reverted case. + hosts`, cases reverted, redeclared a third way and removed). #1698 keeps the reverted case + refused too: the host restarted its feed of the source against the candidate, so the successor + starts it fresh and cannot play the candidate's chunks in the running plan's source. - **Displaced plans are charged until reclaimed (batch 15a fold 2).** When a transaction finds the candidate taken by render and builds on it, the plan render displaced may not yet be reclaimed (a render preempted between its claim and its retirement). The session keeps that plan's @@ -880,10 +883,87 @@ that window is admitted against the plan that renders. - **Admission.** While a successor of the running plan is pending, the running plan's model is kept, so the replacement and the live admissions count it: three plans and three models when a candidate is superseded, three models when a live edit lands while one waits. -- **Resource movement.** Each candidate epoch keeps its carried base (#1344), charged in its capi - epoch row (`carried_base_retained_bytes`); the boot plan has none. The provider epoch itself grew +- **Resource movement.** Each candidate epoch kept its carried base (#1344), charged in its capi + epoch row (`carried_base_retained_bytes`), until #1698 removed it (below); the boot plan had + none. The provider epoch itself grew (its kept base and carried base, and the batch 15a review fold's carried source bytes), and the pending and retired provider lists charge that size, so the reference session's boot `capi_retained_bytes` moves: 165,842 bytes at the batch 15a head, 165,922 after its review fold, 168,810 after fold 2 (x86-64; a third retired slot and a 248-byte parked row in each of the five slots, +2,888), within the 175,296 ceiling. No symbol, result code or event is added. +- **An owner the successor reverts carries from the running plan (#1698).** The successor is + prepared from the running plan's own inventory, with only the source rows the candidate carried + from it, so a strip, effect or other owner it keeps unchanged from the running plan carries as + a direct swap would carry it, also when the candidate restarted or removed it. A source the + candidate added, redeclared or removed, and the successor does not keep as the candidate + declares it (that one moves by the donation), starts fresh in the successor (generation 1, + frame 0, as the host contract for a declaration change against the newest committed session + says). The candidate epoch no longer keeps a carried base, and `carried_base_retained_bytes` is gone: the + reference session's `capi_retained_bytes` moves 168,810 -> 167,514 (x86-64, -1,296: 216 bytes + in each of the six provider epochs). #1285's floors (`input_floors`, `raised_arrivals`) then add + 240 bytes: 167,754 (ceiling unchanged). + +## Latency floors across a structural transaction (#1285) + +- **Behavior.** No node's latency, the output's included, drops across a structural transaction. + The core (`host_core`, for the C ABI and the browser alike) compiles a successor with every node's + input arrival in the plan it displaces as a lower bound (an input floor), and PDC compensates + every incoming edge of the node to it. Removing a latent effect, or the track that holds it, + therefore moves no other path in time: every surviving compensation line keeps its length and + carries (#1283), at the output and inside a submix. `latency_samples` reports the floored output + arrival, the one the plan renders. A path whose own upstream latency dropped gets a new + compensation line, which starts at rest; its strip lost a latent effect, so it is in the restart + set, and it starts at rest with no duck until #1324 adds the duck-swap. An added latency still + grows the node (#1287 owns growth). +- **How long.** The inventory records the floored arrivals, so a chain of successors keeps the + first plan's timing. A successor prepared across a withdrawn candidate takes the running plan's + floors (#1698's base is the running plan's inventory). A fresh `miso_engine_v1_compile_session` + and a browser boot compile with no floors; the lower latency returns there, or at a + host-declared discontinuity (#1323, below). There is no permanent latency reserve. +- **Evidence.** `capi` `runtime::tests::removing_a_limited_track_keeps_the_plan_latency` + (`RemoveTrack` of a limited track: an unchanged `latency_samples` after the swap, and every + block equal to the floored reference). With no floors neither the program nor its canonical bytes + change: the graph fixture corpus, `scripts/check-graph-determinism.sh` and `console-workload` + keep their digests. + +## A host-declared discontinuity resets the floors (#1323) + +- **Surface.** `miso_engine_v1_declare_discontinuity(session)`, an in-place V1 amendment: one new + exported symbol, announced by `MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY` (the next free bit, + 256) in the capability report, whose feature mask grows to include it; no struct, size or + `ABI_VERSION` changes. The mask is the OR of every feature symbol and is never compared with a + literal (`capi` `abi::tests::masks_and_result_codes_are_frozen` folds the symbols; `abi_smoke.c` + and `header_smoke.cpp` pin the bit inside the mask and the signature, and `abi_smoke.c` calls + the symbol with a null session for `INVALID_ARGUMENT`). The frozen exported set is now 18 + `miso_engine_v1_*` definitions. It is a session call, serialized with the others; it runs the + service step first. The browser's `miso_engine_web_v1_declare_discontinuity(handle)` forwards to + the same control-plane function, `control_plane::SessionState::declare_discontinuity` (root + ruling B15-R7); the browser SDK surface is #1296's. +- **Duty.** A host calls it at a stop, and before a seek of every source: the next block it renders + need not continue the previous output. +- **Behavior.** If the newest plan (the waiting replacement, else the running plan) runs behind its + natural timing (a floor raised some node's input arrival above its natural one; the inventory + records each raised node's natural arrival beside its floored one), the committed model is + prepared again as a successor at a discontinuity (`SuccessorBase::discontinuity`): no floors, and + only the source consumers (ring, generation, read position) carry, so no frame is repeated or + skipped and every other owner starts at rest. It is published `Next` with the committed + revision and its retirement credit reserved. A waiting replacement, a `Primed` warm successor + included (it always runs behind its natural timing), is superseded through #1310: withdrawn, the + donor of the rings it created, its persisting producers returned to the running epoch, then + dropped. Its revisions complete when the successor is adopted: `EXACT`, with `SUPERSEDED` for + the revisions it had folded in, never `TRANSITION_FALLBACK`. With nothing to reset, or at + revision 0 (root ruling B15-R6), the call returns `OK` and publishes nothing. The model and the + revision never change; a refusal (a preparation or cap rejection, backpressure) changes nothing, + and a withdrawn replacement is published again whole. +- **Supersession of the successor.** A structural transaction that supersedes the discontinuity + successor before render adopts it is prepared at the same discontinuity: render never played the + successor, so the pair is one transaction "declare, then edit" against the running plan, whose + combined twin has no floors. +- **Evidence.** `capi` `runtime::tests::` gates + `a_declared_discontinuity_reports_the_natural_latency_on_the_c_abi`, + `a_declaration_with_nothing_to_reset_publishes_nothing`, `a_refused_declaration_changes_nothing`, + `a_declared_stop_supersedes_a_pending_warm_successor_alone_and_folded` and + `an_edit_that_supersedes_a_discontinuity_successor_is_prepared_at_it`; `host-core` + `successor_swap::a_declared_discontinuity_resets_the_floors_at_every_rate_and_width`; `host-web` + `tests::structural_apply::a_declared_discontinuity_resets_the_floors_on_the_browser` and the + parity row `control_parity::a_declared_discontinuity_resets_the_floors_alike_on_both_hosts`. diff --git a/hosts/host-web/src/ffi.rs b/hosts/host-web/src/ffi.rs index 76c23a944..21fda2868 100644 --- a/hosts/host-web/src/ffi.rs +++ b/hosts/host-web/src/ffi.rs @@ -3339,6 +3339,24 @@ pub extern "C" fn miso_engine_web_v1_service(handle: u32) -> u32 { }) } +/// Declare a discontinuity (issue #1323): the next block need not continue the previous output, +/// because the page has stopped or is about to seek every source. The page calls it at a stop and +/// before a seek of every source. It runs the control plane's `SessionState::declare_discontinuity`, +/// the function the C ABI's `miso_engine_v1_declare_discontinuity` runs, so both hosts reset the +/// latency floors alike. Control side only: the control Worker, or in `single` mode the worklet's +/// message handler, never `process()`; a plan it publishes is counted as a blocking rebuild. +/// +/// Results: [`RESULT_OK`] (also when nothing needed a reset); [`RESULT_INVALID_ARGUMENT`] for a +/// wrong handle; [`RESULT_WRONG_STATE`] for a host that is not ready; and the refusals of +/// [`miso_engine_web_v1_control_command`] for a rebuild, with the core's diagnostic at +/// [`miso_engine_web_v1_control_response_ptr`]. A refusal changes nothing. +#[unsafe(no_mangle)] +pub extern "C" fn miso_engine_web_v1_declare_discontinuity(handle: u32) -> u32 { + with_host_mut(handle, RESULT_INVALID_ARGUMENT, |host| { + with_promotion_analysis(host, AudioWorkletEngineHost::declare_discontinuity) + }) +} + /// Return the control request staging address for a frame of `len` bytes (issue #1562a D1), or /// zero for an invalid handle, a host that is not ready, or a `len` that is zero or over the boot's /// `maximum_control_frame_bytes`. The page writes one command frame there, then calls @@ -3445,6 +3463,28 @@ pub extern "C" fn miso_engine_web_v1_watermark_ptr(handle: u32) -> u32 { }) } +/// Copy the active plan's output latency and tail into their fixed record (issue #1688 D2), +/// found at [`miso_engine_web_v1_plan_latency_ptr`]. Control side only. +/// +/// Results: [`RESULT_OK`]; [`RESULT_INVALID_ARGUMENT`] for a wrong handle, which writes nothing; +/// [`RESULT_WRONG_STATE`] for a host that is not ready, which leaves the record alone. The +/// record's words are the control plane's row for the plan that renders now; a rebuild's +/// successor shows from the first read after the block that adopts it. +#[unsafe(no_mangle)] +pub extern "C" fn miso_engine_web_v1_plan_latency_read(handle: u32) -> u32 { + with_host_mut(handle, RESULT_INVALID_ARGUMENT, |host| { + host.read_plan_latency() + }) +} + +/// Return the stable address of the 64-byte plan latency record or zero for an invalid handle. +#[unsafe(no_mangle)] +pub extern "C" fn miso_engine_web_v1_plan_latency_ptr(handle: u32) -> u32 { + with_host(handle, 0, |host| { + pointer_u32(ptr::from_ref(host.plan_latency())) + }) +} + /// Return the stable prepared resource-report address or zero for an invalid handle. #[unsafe(no_mangle)] pub extern "C" fn miso_engine_web_v1_resource_ptr(handle: u32) -> u32 { @@ -4119,6 +4159,12 @@ pub mod native_staging { super::with_host(handle, None, |host| Some(*host.watermark())) } + /// The plan latency record the host `handle` names last copied (#1688), as the worklet reads + /// it through `miso_engine_web_v1_plan_latency_ptr`; `None` for an unknown handle. + pub fn plan_latency(handle: u32) -> Option { + super::with_host(handle, None, |host| Some(*host.plan_latency())) + } + /// The meter header and frame the host `handle` names last published, as the worklet reads /// them through `miso_engine_web_v1_meter_header_ptr` and the header's frame pointer (#1290d1); /// `None` for an unknown handle. diff --git a/hosts/host-web/src/lib.rs b/hosts/host-web/src/lib.rs index c7b2fe732..9348d23b0 100644 --- a/hosts/host-web/src/lib.rs +++ b/hosts/host-web/src/lib.rs @@ -1217,6 +1217,34 @@ pub struct WebPlanWatermark { /// Exact byte size of [`WebPlanWatermark`], equal to the C ABI watermark's. pub const WATERMARK_BYTES: u32 = size_of::() as u32; +/// The active plan's output latency and tail (#1688 D2), in the order and sizes of the C ABI +/// report's `latency_samples`, `tail_kind` and `tail_samples` words. +/// +/// The host keeps one at a stable address, written by +/// [`AudioWorkletEngineHost::read_plan_latency`] and found through +/// `miso_engine_web_v1_plan_latency_ptr`. The words are the control plane's row for the active +/// plan epoch, copied: the browser computes nothing. +#[repr(C)] +#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] +pub struct WebPlanLatency { + /// Exact structure byte size ([`PLAN_LATENCY_BYTES`]); zero until the first successful read. + pub struct_size: u32, + /// Required zero. + pub reserved0: u32, + /// The plan's output latency in samples. + pub latency_samples: u64, + /// `0` for a tail that ends after `tail_samples`, `1` for an unbounded one: the C ABI's + /// `TAIL_FINITE` and `TAIL_INFINITE`. + pub tail_kind: u64, + /// The finite tail in samples; zero for an unbounded tail. + pub tail_samples: u64, + /// Required-zero expansion words. + pub reserved: [u64; 4], +} + +/// Exact byte size of [`WebPlanLatency`]. +pub const PLAN_LATENCY_BYTES: u32 = size_of::() as u32; + /// Exact retained-resource projection for the browser bridge and production artifacts. #[repr(C)] #[derive(Clone, Copy, Debug, Eq, PartialEq)] @@ -1462,7 +1490,7 @@ struct PlanShape { /// host drops a superseded candidate's tables as soon as the core no longer holds its epoch /// ([`ReadyOwnership::drop_superseded_shapes`]); the tables of plans adopted since the last /// service step wait for it. So a command meets at most one entry after the reclaim and adds one: -/// four leaves room to spare, and the room check in [`ReadyOwnership::run_command`] never refuses +/// four leaves room to spare, and the room check in [`ReadyOwnership::run_control`] never refuses /// a structural edit, paused or not. const PENDING_SHAPES: usize = 4; @@ -1619,13 +1647,41 @@ impl ReadyOwnership { output_capacity: u64, analysis: Option<&mut SpectrumAnalysis>, applied_at_sample: u64, + ) -> Result { + self.run_control(analysis, applied_at_sample, |control| { + control.command(request, output_capacity) + }) + } + + /// Declare a discontinuity through the control plane (#1323), the function the C ABI's + /// `miso_engine_v1_declare_discontinuity` runs. A plan it publishes is kept and counted as + /// [`Self::run_command`] keeps a rebuild's. + fn declare_discontinuity( + &mut self, + analysis: Option<&mut SpectrumAnalysis>, + applied_at_sample: u64, + ) -> Result<(), control_plane::CommandError> { + self.run_control(analysis, applied_at_sample, |control| { + control.declare_discontinuity().map(|()| 0) + }) + .map(|_| ()) + } + + /// Run one control-plane call that may publish a replacement plan (#1290a, #1323): reclaim + /// first, check the room for the plan's tables, run `call`, and keep the tables of a plan it + /// published for its adoption block. + fn run_control( + &mut self, + analysis: Option<&mut SpectrumAnalysis>, + applied_at_sample: u64, + call: impl FnOnce(&mut WebSession) -> Result, ) -> Result { self.reclaim_shapes(analysis); if self.pending_shapes.len() == self.pending_shapes.capacity() { return Err(control_plane::CommandError::Backpressure); } let before = self.control.newest_plan_epoch(); - let result = self.control.command(request, output_capacity); + let result = call(&mut self.control); // Whatever the preparer left is taken: a command refused after its preparation leaves // tables nothing will read. let (prepared, handover) = self.record.lock().map_or((None, None), |mut record| { @@ -2091,6 +2147,9 @@ pub struct AudioWorkletEngineHost { /// The watermark `read_watermark` last copied (#1381a D5), at a stable address in the /// control half: all zero until the first successful read. watermark: WebPlanWatermark, + /// The plan latency `read_plan_latency` last copied (#1688 D2), at a stable address in the + /// control half: all zero until the first successful read. + plan_latency: WebPlanLatency, /// First sample eligible for the current lease. A snapshot that opens before this boundary /// is an old/partial interval and is dropped rather than presented as a clean measurement. meter_activation_sample: u64, @@ -2290,6 +2349,7 @@ impl AudioWorkletEngineHost { struct_size: WATERMARK_BYTES, ..WebPlanWatermark::default() }, + plan_latency: WebPlanLatency::default(), meter_lease: false, meter_activation_sample: 0, buffers: Some(buffers), @@ -2443,28 +2503,63 @@ impl AudioWorkletEngineHost { self.control_response = ControlResponse::Frame(length); self.record(RESULT_OK) } - Err(error) => { - let diagnostic = error.diagnostic(); - let kept = diagnostic.len().min(buffers.diagnostic.len()); - buffers.diagnostic[..kept].copy_from_slice(&diagnostic[..kept]); - self.diagnostic_len = kept; - self.control_response = ControlResponse::Diagnostic; - self.record(match error { - control_plane::CommandError::Invalid => RESULT_INVALID_ARGUMENT, - control_plane::CommandError::BufferTooSmall { .. } => RESULT_BUFFER_TOO_SMALL, - control_plane::CommandError::Backpressure => RESULT_BACKPRESSURE, - // #1290a: a transaction that changes the rate or the quantum cannot be applied - // to a running plan; the page re-prepares the session from its document. - control_plane::CommandError::CompileRejected(_) - if diagnostic.starts_with(control_plane::REPREPARE_REQUIRED.as_bytes()) => - { - RESULT_REPREPARE_REQUIRED - } - control_plane::CommandError::CompileRejected(_) => RESULT_COMPILE_REJECTED, - control_plane::CommandError::Internal => RESULT_INTERNAL, - }) - } + Err(error) => self.refuse_control(&error), + } + } + + /// Declare a discontinuity (#1323): the next block this host renders need not continue the + /// previous output, because the page has stopped or is about to seek every source. The page + /// calls it at a stop and before a seek of every source. It runs the control plane's + /// `SessionState::declare_discontinuity`, the function the C ABI's + /// `miso_engine_v1_declare_discontinuity` runs: if the newest plan runs behind its natural + /// timing (a latency floor of a structural transaction raised it, #1285), the committed + /// session is prepared again with no floors and only its sources carried, and published for + /// the next block; otherwise nothing is published. The model and the revision do not change. + /// Control work, as [`Self::control_command`]: never `process()`. + /// + /// Results: [`RESULT_OK`]; [`RESULT_WRONG_STATE`] for a host that is not ready; and the + /// refusals of [`Self::control_command`] for a rebuild, with the core's diagnostic at + /// [`Self::control_response`]. A refusal changes nothing. + pub fn declare_discontinuity(&mut self, analysis: Option<&mut SpectrumAnalysis>) -> u32 { + self.control_response = ControlResponse::None; + if self.status.state != STATE_READY { + return self.record(RESULT_WRONG_STATE); } + let applied_at_sample = self.status.next_absolute_sample; + let Some(ready) = self.ready.as_mut() else { + return self.fail(RESULT_INTERNAL, b"web.internal.ready\t$\n"); + }; + match ready.declare_discontinuity(analysis, applied_at_sample) { + Ok(()) => self.record(RESULT_OK), + Err(error) => self.refuse_control(&error), + } + } + + /// A refused control call (#1562a D1, #1323): the core's diagnostic into the session + /// diagnostic and the response, and the result code for `error`. + fn refuse_control(&mut self, error: &control_plane::CommandError) -> u32 { + let Some(buffers) = self.buffers.as_mut() else { + return self.fail(RESULT_INTERNAL, b"web.internal.ready\t$\n"); + }; + let diagnostic = error.diagnostic(); + let kept = diagnostic.len().min(buffers.diagnostic.len()); + buffers.diagnostic[..kept].copy_from_slice(&diagnostic[..kept]); + self.diagnostic_len = kept; + self.control_response = ControlResponse::Diagnostic; + self.record(match error { + control_plane::CommandError::Invalid => RESULT_INVALID_ARGUMENT, + control_plane::CommandError::BufferTooSmall { .. } => RESULT_BUFFER_TOO_SMALL, + control_plane::CommandError::Backpressure => RESULT_BACKPRESSURE, + // #1290a: a transaction that changes the rate or the quantum cannot be applied + // to a running plan; the page re-prepares the session from its document. + control_plane::CommandError::CompileRejected(_) + if diagnostic.starts_with(control_plane::REPREPARE_REQUIRED.as_bytes()) => + { + RESULT_REPREPARE_REQUIRED + } + control_plane::CommandError::CompileRejected(_) => RESULT_COMPILE_REJECTED, + control_plane::CommandError::Internal => RESULT_INTERNAL, + }) } /// Dequeue the next event of `lane` (issue #1570b D1): `0` is the reliable lane, `1` the lossy @@ -2627,6 +2722,38 @@ impl AudioWorkletEngineHost { RESULT_OK } + /// Copy the active plan's output latency and tail into the host's record (#1688 D2). + /// + /// The words are the control plane's resource row for the active plan epoch, the same + /// `PlanQueries::resources()` row the C ABI reports, with the tail in the control plane's + /// two-word encoding: a successor's values appear from the first read after the block that + /// adopts it. Like [`Self::service`] it leaves the status's last result alone. + pub fn read_plan_latency(&mut self) -> u32 { + if self.status.state != STATE_READY { + return RESULT_WRONG_STATE; + } + let Some(ready) = self.ready.as_ref() else { + return RESULT_WRONG_STATE; + }; + let resources = ready.queries.resources(); + let (tail_kind, tail_samples) = control_plane::tail_words(resources.output_tail); + self.plan_latency = WebPlanLatency { + struct_size: PLAN_LATENCY_BYTES, + reserved0: 0, + latency_samples: resources.latency_samples, + tail_kind, + tail_samples, + reserved: [0; 4], + }; + RESULT_OK + } + + /// The record [`Self::read_plan_latency`] fills. + #[must_use] + pub const fn plan_latency(&self) -> &WebPlanLatency { + &self.plan_latency + } + /// The record [`Self::read_watermark`] fills. #[must_use] pub const fn watermark(&self) -> &WebPlanWatermark { diff --git a/hosts/host-web/src/tests.rs b/hosts/host-web/src/tests.rs index f45643aff..d321bdfd6 100644 --- a/hosts/host-web/src/tests.rs +++ b/hosts/host-web/src/tests.rs @@ -5936,12 +5936,8 @@ fn web_runtime_preparer_attaches_the_companions_on_every_branch() { let edit_ramps = session::EditRamps::new(); let base = host_core::SuccessorBase { - inventory: &boot.inventory, - committed: model, - revision: 1, edit_ramps: &edit_ramps, - monitor: host_core::MonitorBase::NONE, - observation: &host_core::NO_OBSERVATION, + ..host_core::SuccessorBase::new(&boot.inventory, model, 1) }; let (mut successor, successor_handles) = preparer .prepare(&compiled, &caps, lanes, Some(base), &superseded) @@ -7065,6 +7061,149 @@ mod structural_apply { assert_eq!(rig.host.service(None), RESULT_OK); } + /// #1323 gate 2 on the browser: the #1285 removal of a muted track behind a true-peak limiter + /// keeps A's latency; `declare_discontinuity`, the host method + /// `miso_engine_web_v1_declare_discontinuity` runs, then publishes the committed session with no + /// floors. After one render the plan's latency is the natural one, and from that block every + /// block equals a fresh boot of the committed model fed from the frame the carried source stood + /// at. The revision does not move. A wrong handle is refused. + /// + /// Test value: red if the browser entry does not reach the control plane's declaration (the + /// floors survive, no plan is pending and the remaining tracks stay late), or if the successor + /// carries any state but the source (the filters differ from a fresh boot's). + #[test] + fn a_declared_discontinuity_resets_the_floors_on_the_browser() { + const DECLARED_AT: u64 = 10; + let mut model = gate_model(); + let limited = &mut model.tracks[0]; + limited.fader.left_mute = true; + limited.fader.right_mute = true; + limited.inserts.effects.push(session::Effect { + id: StableId::parse("lim").expect("insert ID"), + identity: session::EffectIdentity::Native { + effect_id: StableId::parse("miso.true-peak-limiter").expect("limiter id"), + }, + quality: session::EffectQuality::Normal, + bypass: false, + link_mode: session::LinkMode::Maximum, + params: Vec::new(), + sidechain: session::SidechainDeclaration::None, + }); + let edits = [ + SessionEdit::RemoveRoute { + route_id: model.routes[0].id.clone(), + }, + SessionEdit::RemoveTrack { + track_id: model.tracks[0].id.clone(), + }, + ]; + let latency = |rig: &Rig| { + rig.host + .ready + .as_ref() + .expect("ready") + .queries + .resources() + .latency_samples + }; + assert_eq!( + crate::ffi::miso_engine_web_v1_declare_discontinuity(u32::MAX), + RESULT_INVALID_ARGUMENT + ); + let mut rig = Rig::boot(&model, options(), None); + let floored = latency(&rig); + assert!(floored > 0, "the limiter delays A"); + for _ in 0..APPLY_BEFORE { + rig.step(); + } + assert!(matches!( + rig.apply(&edits, 1), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + )); + while rig.block < DECLARED_AT { + rig.step(); + } + assert_eq!( + latency(&rig), + floored, + "the removal keeps A's latency (#1285)" + ); + let revision = rig.host.committed_revision(); + assert_eq!(rig.host.declare_discontinuity(None), RESULT_OK); + assert!(rig.pending(), "the declaration published a plan"); + while rig.block < BLOCKS { + rig.step(); + } + assert_eq!(rig.host.committed_revision(), revision); + + let document = canonical_session_json( + rig.host + .ready + .as_ref() + .expect("ready") + .control + .committed_session() + .normalized_model(), + ) + .expect("canonical session"); + let committed = parse_session_json(&document).expect("the committed model parses"); + let mut fresh = Rig::boot(&committed, options(), None); + let natural = latency(&fresh); + assert!(natural < floored, "B's natural latency is lower"); + assert_eq!( + latency(&rig), + natural, + "the plan runs at its natural latency" + ); + // The fresh boot reads the PCM from the frame the carried source stood at. + let quantum = u64::from(QUANTUM); + let offset = DECLARED_AT * quantum; + let expected: Vec> = (0..BLOCKS - DECLARED_AT) + .map(|block| { + let lane = |lane: u64| { + (block * quantum..(block + 1) * quantum) + .map(|frame| strip_sample(0, lane, frame + offset)) + .collect::>() + }; + let planes = [lane(0), lane(1)]; + let views: [&[f32]; 2] = [&planes[0], &planes[1]]; + assert_eq!( + fresh.host.submit_source( + SOURCE, + 1, + block * quantum, + 48_000, + &views, + QUANTUM, + false + ), + RESULT_OK + ); + assert_eq!(fresh.host.render_next(), RESULT_OK); + fresh + .host + .output_pcm() + .expect("output") + .iter() + .map(|sample| sample.to_bits()) + .collect() + }) + .collect(); + assert!( + expected.iter().flatten().any(|bits| *bits != 0), + "the fresh boot plays" + ); + assert_eq!( + first_difference(&rig.bits[DECLARED_AT as usize..], &expected), + None, + "from the declaration on, the output is a fresh boot's" + ); + assert_eq!(rig.host.service(None), RESULT_OK); + } + /// #1310 on the browser path, which runs the same control plane: a structural edit while a /// candidate waits unadopted commits and supersedes it. The first candidate never renders; the /// replacement carries both transactions and is adopted at the next block. @@ -7392,6 +7531,84 @@ mod structural_apply { ); } + /// #1698 on the browser: a subscription acked against a candidate that restarted "track"'s + /// compressor (its link mode changed), which a later structural edit supersedes by putting the + /// compressor back as the plan that renders has it. The replacement now carries the + /// compressor from the plan that renders, whose tap never saw the record, so the core queues + /// the record again on the replacement and the host copies it onto the replacement's tables, + /// as for a compressor the candidate itself carried. + /// + /// Test value: red if the record forwarding or the host's table hand-over serves only a tap + /// the candidate carried, and not one the replacement carries from the plan that renders + /// across a candidate that restarted it: the carried tap keeps the running plan's unarmed + /// state, the tables report it unarmed and no window publishes. + #[test] + fn a_subscription_on_a_candidate_that_restarted_the_tap_publishes_on_a_replacement_that_reverts_it() + { + let mut model = + parse_session_json(include_str!("../tests/browser-v1/command-session.json")) + .expect("accepted command fixture"); + model.tracks[0].inserts.effects[0] = observed_compressor_insert(); + let original = model.tracks[0].inserts.effects[0].clone(); + let mut restarted = original.clone(); + restarted.link_mode = session::LinkMode::Maximum; + let put = |effect: session::Effect| { + [SessionEdit::PutTrackEffect { + track_id: model.tracks[0].id.clone(), + rack_name: session::RackName::Inserts, + final_position: 0, + effect, + }] + }; + let mut rig = Rig::boot(&model, options(), None); + rig.step(); + assert!(matches!( + rig.apply(&put(restarted), 1), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + )); + assert_eq!( + observe_key(&mut rig.host, "track", RACK_INSERTS, "comp", 1, 2, true), + RESULT_OK, + "the subscription is acked against the pending candidate" + ); + assert!(matches!( + rig.apply(&put(original), 2), + Outcome::Applied { + path: EditPath::Rebuild, + .. + } + )); + rig.step(); + assert_eq!(rig.epoch(), 2, "the replacement is adopted"); + assert_eq!( + armed_mask(&rig.host, "track", "comp"), + Some(1), + "the replacement's tables hold the subscription" + ); + for _ in 0..6 { + rig.step(); + } + let binding = (0..rig.host.observation_binding_count() as u32) + .filter_map(|index| rig.host.observation_binding(index)) + .next() + .expect("the compressor is bound"); + let mut address = binding.address; + address.tap_id = 1; + let read = rig + .host + .read_observation_addresses(&[address]) + .expect("a read"); + assert_eq!(&*read[0].track_id, "track"); + assert!( + read[0].window.is_some(), + "the subscription publishes windows on the replacement: {:?}", + read[0].status + ); + } + /// Drive `blocks` more blocks with the lease on, polling after each; the windows folded. fn poll_blocks(rig: &mut Rig, blocks: u64) -> u32 { let mut windows = 0; @@ -8730,6 +8947,109 @@ mod structural_apply { "the adopted plan has no master" ); } + /// The latency and tail words a fresh boot of `model` reports, the oracle a rebuilt plan's + /// record is compared with. + fn plan_latency_of_boot(model: &SessionModel) -> WebPlanLatency { + let mut fresh = Rig::boot(model, options(), None); + assert_eq!(fresh.host.read_plan_latency(), RESULT_OK); + *fresh.host.plan_latency() + } + + /// Issue #1688 gate 2: the plan latency record is the control plane's row for the plan that + /// renders now. It is refused (and left zero) while the host is not ready, holds the boot + /// plan's words after boot, keeps them while a rebuild that adds a true-peak limiter insert + /// waits for its adoption block, and holds the successor's words from the first read after it, + /// equal to a fresh boot of the committed model. + /// + /// Test value: red if the read reports the candidate's latency before render adopts it, keeps + /// the boot value after a rebuild (a boot-time copy like `WebResourceReport`), or answers a + /// host that is not ready. + #[test] + fn the_plan_latency_record_follows_the_active_plan() { + const LIMITER_ID: &str = "added-limiter"; + let model = gate_model(); + let mut rig = Rig::boot(&model, options(), None); + assert_eq!(PLAN_LATENCY_BYTES, 64); + + // A host that is not ready refuses and leaves the record alone. + rig.host.status.state = STATE_FAILED; + assert_eq!(rig.host.read_plan_latency(), RESULT_WRONG_STATE); + assert_eq!(*rig.host.plan_latency(), WebPlanLatency::default()); + rig.host.status.state = STATE_READY; + + // After boot the record holds the boot plan's row, copied. + assert_eq!(rig.host.read_plan_latency(), RESULT_OK); + let boot = *rig.host.plan_latency(); + let row = rig.host.ready.as_ref().expect("ready").queries.resources(); + assert_eq!(boot.struct_size, PLAN_LATENCY_BYTES); + assert_eq!(boot.reserved0, 0); + assert_eq!(boot.reserved, [0; 4]); + assert_eq!(boot.latency_samples, row.latency_samples); + assert_eq!( + (boot.tail_kind, boot.tail_samples), + control_plane::tail_words(row.output_tail) + ); + + for _ in 0..APPLY_BEFORE { + rig.step(); + } + let (_, _, _, mut limiter, _) = strip_base(); + limiter.id = strip_id(LIMITER_ID); + limiter.identity = session::EffectIdentity::Native { + effect_id: strip_id("miso.true-peak-limiter"), + }; + limiter.params = vec![session::EffectParam { + parameter_id: 1, + channel: session::ParameterChannel::Both, + unit: session::ParameterUnit::Db, + value: -6.0, + }]; + let mut committed = model.clone(); + committed.tracks[0].inserts.effects.push(limiter.clone()); + committed.revision += 1; + let base = rig.host.committed_revision(); + assert_eq!( + rig.apply( + &[SessionEdit::PutTrackEffect { + track_id: model.tracks[0].id.clone(), + rack_name: session::RackName::Inserts, + final_position: 0, + effect: limiter, + }], + 1 + ), + Outcome::Applied { + path: EditPath::Rebuild, + revision: base + 1 + } + ); + let successor = plan_latency_of_boot(&committed); + assert!( + successor.latency_samples > boot.latency_samples, + "the limiter's lookahead moves the plan latency ({} -> {})", + boot.latency_samples, + successor.latency_samples + ); + + // Committed but not adopted: the plan that renders is the boot plan. + assert!(rig.pending()); + assert_eq!(rig.host.read_plan_latency(), RESULT_OK); + assert_eq!( + *rig.host.plan_latency(), + boot, + "the record is the active plan's" + ); + + // The first read after the adopting block holds the successor's words. + rig.step(); + assert_eq!(rig.epoch(), 1); + assert_eq!(rig.host.read_plan_latency(), RESULT_OK); + assert_eq!( + *rig.host.plan_latency(), + successor, + "the adopted plan's row equals a fresh boot of the committed model" + ); + } } /// The encoded bytes of the one `OBSERVATION_ENDED` frame that ends the insert taps keyed by diff --git a/hosts/host-web/tests/control_parity.rs b/hosts/host-web/tests/control_parity.rs index 5278f290e..de2e8e4bb 100644 --- a/hosts/host-web/tests/control_parity.rs +++ b/hosts/host-web/tests/control_parity.rs @@ -42,9 +42,9 @@ use core::ptr; use capi::{ ABI_VERSION as C_ABI_VERSION, BYTES_OUT_SIZE, BytesOut, COMPILE_LIMITS_SIZE, CompileLimits, - ENGINE_CONFIG_SIZE, EVENT_LANE_RELIABLE, EngineConfig, PLANAR_OUTPUT_SIZE, Plan, PlanarOutput, - RESULT_OK as C_RESULT_OK, SOURCE_CHUNK_SIZE, SUBMIT_REPORT_SIZE, Session, SourceChunk, - SubmitReport, + ENGINE_CONFIG_SIZE, EVENT_LANE_RELIABLE, EngineConfig, PLAN_RESOURCE_REPORT_SIZE, + PLANAR_OUTPUT_SIZE, Plan, PlanResourceReport, PlanarOutput, RESULT_OK as C_RESULT_OK, + SOURCE_CHUNK_SIZE, SUBMIT_REPORT_SIZE, Session, SourceChunk, SubmitReport, }; use controls::{ControlAddress, Lane, Owner, Section}; use protocol::{ @@ -1368,6 +1368,18 @@ impl CAbi { (out.revision, out.first_sample, out.outcome_flags) } + /// The active plan's `(latency_samples, tail_kind, tail_samples)` through + /// `miso_engine_v1_plan_resources`. + fn plan_latency(&self) -> (u64, u64, u64) { + // SAFETY: `PlanResourceReport` is plain integers, for which all-zero bytes are a value. + let mut out: PlanResourceReport = unsafe { core::mem::zeroed() }; + out.struct_size = PLAN_RESOURCE_REPORT_SIZE; + // SAFETY: The plan is live; the query is pure and `out` is owned storage of the exact size. + let code = unsafe { capi::miso_engine_v1_plan_resources(self.plan, &mut out) }; + assert_eq!(code, C_RESULT_OK, "C ABI plan resources"); + (out.latency_samples, out.tail_kind, out.tail_samples) + } + fn superseded(&mut self) -> u64 { let response = self.send( CommandPayload::CountersGet(&CountersRequest { @@ -1654,7 +1666,20 @@ fn run(rate: u32, steps: &[Step]) -> (Vec>, u64) { } } } + // #1688 D4: the browser's latency and tail words are the C ABI report's, after every + // scripted edit (the plan that renders has not changed yet) and after every block. + let latency = c_abi.plan_latency(); + assert_eq!( + browser_plan_latency(&mut browser), + latency, + "{rate} Hz: before block {block}: the browser plan latency is the C ABI's" + ); let (b, c) = (browser.step(block), c_abi.step(block)); + assert_eq!( + browser_plan_latency(&mut browser), + c_abi.plan_latency(), + "{rate} Hz: after block {block}: the browser plan latency is the C ABI's" + ); // #1349 gate 1: the status words are the C ABI's watermark after every block. assert_eq!( status_watermark(&browser), @@ -1722,6 +1747,21 @@ fn both_hosts_give_the_same_result_for_the_same_control_edits() { } } +/// The browser record's `(latency_samples, tail_kind, tail_samples)` after a fresh read (#1688). +fn browser_plan_latency(browser: &mut Rig) -> (u64, u64, u64) { + assert_eq!( + browser.host.read_plan_latency(), + host_web::RESULT_OK, + "the browser plan latency read" + ); + let record = browser.host.plan_latency(); + ( + record.latency_samples, + record.tail_kind, + record.tail_samples, + ) +} + /// The three watermark words of the browser's status: `(revision, first sample, outcome flags)`. fn status_watermark(browser: &Rig) -> (u64, u64, u64) { let status = browser.host.status(); @@ -3359,3 +3399,259 @@ fn a_supersession_that_would_drop_acked_pcm_is_backpressure_alike_on_both_hosts( } } } + +/// The C ABI plan's `latency_samples`, through the exported resource query. +fn c_latency(c_abi: &CAbi) -> u64 { + // SAFETY: Every field of the report is an integer, for which zero is a valid value. + let mut report: capi::PlanResourceReport = unsafe { core::mem::zeroed() }; + report.struct_size = capi::PLAN_RESOURCE_REPORT_SIZE; + // SAFETY: The plan is live; `report` is one complete local report. + let code = unsafe { capi::miso_engine_v1_plan_resources(c_abi.plan, &mut report) }; + assert_eq!(code, C_RESULT_OK); + report.latency_samples +} + +/// #1323 on both hosts: removing the true-peak limiter insert keeps every node's latency (#1285), +/// and a declared discontinuity, `miso_engine_v1_declare_discontinuity` on the C ABI and the +/// browser host's `declare_discontinuity`, publishes the committed session with no floors on both. +/// From the block after it the two hosts render the same bits, and the C ABI reports a lower +/// latency than before the declaration. +/// +/// Test value: red if one host's entry does not reach the control plane's declaration, or +/// reaches another rule (that host keeps the floors, and its remaining tracks play later than the +/// other host's), or if the declaration is a no-op on both (the C ABI latency stays floored). +#[test] +fn a_declared_discontinuity_resets_the_floors_alike_on_both_hosts() { + for rate in [48_000, 96_000] { + let model = model_at(rate); + let mut browser = Rig::new(&model); + let mut c_abi = CAbi::new(&model); + same_blocks(&mut browser, &mut c_abi, 0..4, &format!("{rate} Hz before")); + let edits = [SessionEdit::RemoveTrackEffect { + track_id: id(LIMITER.track.id), + rack_name: session::RackName::Inserts, + effect_id: id(LIMITER.id), + }]; + let (b_verdict, _) = browser.transaction_reply(&edits); + let (c_verdict, _) = c_abi.transaction(&edits); + c_abi.drain_reliable(); + assert!( + matches!( + b_verdict, + Verdict::Applied { + path: EditPath::Rebuild, + .. + } + ), + "{rate} Hz: {b_verdict:?}" + ); + assert_eq!(b_verdict, c_verdict, "{rate} Hz"); + same_blocks( + &mut browser, + &mut c_abi, + 4..8, + &format!("{rate} Hz floored"), + ); + let floored = c_latency(&c_abi); + assert_eq!( + browser_plan_latency(&mut browser).0, + floored, + "{rate} Hz: both hosts report the floored latency" + ); + assert_eq!( + browser.host.declare_discontinuity(None), + host_web::RESULT_OK, + "{rate} Hz" + ); + // SAFETY: The session is live and used by this thread alone. + let code = unsafe { capi::miso_engine_v1_declare_discontinuity(c_abi.session) }; + assert_eq!(code, C_RESULT_OK, "{rate} Hz"); + same_blocks( + &mut browser, + &mut c_abi, + 8..16, + &format!("{rate} Hz declared"), + ); + assert!( + c_latency(&c_abi) < floored, + "{rate} Hz: the declaration resets the floors ({floored})" + ); + assert_eq!( + browser_plan_latency(&mut browser).0, + c_latency(&c_abi), + "{rate} Hz: both hosts report the reset latency" + ); + } +} + +/// #1688 gate 1 with #1285's floors: a muted track behind a true-peak limiter (and the delay and +/// multiband beside it) sets the plan's latency. A transaction that removes the track and its +/// routes rebuilds the plan, and a fresh compile of the committed session would report a lower +/// latency. The successor is compiled with the predecessor's floors, so both hosts report the same, +/// unchanged latency (and tail) after the swap, and the browser record equals the C ABI's before +/// the swap, at it and after it. +/// +/// Test value: red if the browser reports the unfloored latency of a fresh compile, reads a stale +/// plan's row, or encodes the tail differently from the C ABI; and red if the row is vacuous (the +/// latency must be positive and the rebuild must be adopted). +#[test] +fn removing_a_limited_track_keeps_the_plan_latency_alike_on_both_hosts() { + for rate in RATES { + let model = model_at(rate); + let limited = id(LIMITER.track.id); + let mut removal: Vec = model + .routes + .iter() + .filter(|route| { + matches!( + &route.source, + session::RouteSource::Track { track_id, .. } if *track_id == limited + ) + }) + .map(|route| SessionEdit::RemoveRoute { + route_id: route.id.clone(), + }) + .collect(); + removal.push(SessionEdit::RemoveTrack { track_id: limited }); + + let mut browser = Rig::new(&model); + let mut c_abi = CAbi::new(&model); + let boot = c_abi.plan_latency(); + assert!(boot.0 > 0, "{rate} Hz: the limiter delays the plan"); + assert_eq!(browser_plan_latency(&mut browser), boot, "{rate} Hz: boot"); + for block in 0..4 { + browser.step(block); + c_abi.step(block); + } + let (b, c) = (browser.transaction(&removal), c_abi.transaction(&removal).0); + assert_eq!(b, c, "{rate} Hz: one verdict"); + assert!( + matches!( + c, + Verdict::Applied { + path: EditPath::Rebuild, + .. + } + ), + "{rate} Hz: the removal rebuilds, got {c:?}" + ); + // Committed, not adopted: both still report the running plan's words. + assert_eq!( + browser_plan_latency(&mut browser), + boot, + "{rate} Hz: pending" + ); + assert_eq!(c_abi.plan_latency(), boot, "{rate} Hz: pending"); + for block in 4..8 { + browser.step(block); + c_abi.step(block); + } + let (watermark, _, _) = c_abi.watermark(); + assert_eq!( + watermark, c_abi.revision, + "{rate} Hz: the rebuild is adopted" + ); + assert_eq!( + c_abi.plan_latency(), + boot, + "{rate} Hz: the floored successor keeps the latency" + ); + assert_eq!( + browser_plan_latency(&mut browser), + boot, + "{rate} Hz: the browser reports the same, unchanged latency after the swap" + ); + + // Without floors the committed session would report less: the row is not vacuous. + let mut committed = model.clone(); + committed + .routes + .retain(|route| !matches!( + &route.source, + session::RouteSource::Track { track_id, .. } if track_id.as_str() == LIMITER.track.id + )); + committed + .tracks + .retain(|track| track.id.as_str() != LIMITER.track.id); + let mut fresh = Rig::new(&committed); + assert!( + browser_plan_latency(&mut fresh).0 < boot.0, + "{rate} Hz: a fresh compile of the committed session reports a lower latency" + ); + } +} + +/// #1688 gate 1, the moving row: inserts that add latency to `t1` until its path is the longest +/// make a rebuild that raises the plan's latency. (A strip's `delay_samples` does not: a track +/// delay shifts the output and leaves the PDC report alone, `graph-compiler`'s `track_delay.rs` +/// P2-3.) Both hosts report the boot latency until the adopting block, then the same raised one; +/// the value moved, so the comparison is not vacuous. +/// +/// Test value: red if the browser keeps the boot value after a rebuild (a boot-time copy) or +/// reports the candidate's before the adoption block. +#[test] +fn a_rebuild_that_adds_latency_moves_the_plan_latency_alike_on_both_hosts() { + const LIMITERS: usize = 4; + for rate in RATES { + let model = model_at(rate); + let template = model + .tracks + .iter() + .flat_map(|track| &track.inserts.effects) + .find(|effect| effect.id.as_str() == LIMITER.id) + .expect("the session's limiter insert") + .clone(); + let edits: Vec = (0..LIMITERS) + .map(|index| { + let mut effect = template.clone(); + effect.id = id(&format!("added-limiter-{index}")); + SessionEdit::PutTrackEffect { + track_id: id(T1.id), + rack_name: session::RackName::Inserts, + final_position: 2 + index as u32, + effect, + } + }) + .collect(); + let mut browser = Rig::new(&model); + let mut c_abi = CAbi::new(&model); + let boot = c_abi.plan_latency(); + assert_eq!(browser_plan_latency(&mut browser), boot, "{rate} Hz: boot"); + for block in 0..4 { + browser.step(block); + c_abi.step(block); + } + let (b, c) = (browser.transaction(&edits), c_abi.transaction(&edits).0); + assert_eq!(b, c, "{rate} Hz: one verdict"); + assert!( + matches!( + c, + Verdict::Applied { + path: EditPath::Rebuild, + .. + } + ), + "{rate} Hz: the inserts rebuild, got {c:?}" + ); + assert_eq!( + browser_plan_latency(&mut browser), + boot, + "{rate} Hz: pending" + ); + assert_eq!(c_abi.plan_latency(), boot, "{rate} Hz: pending"); + browser.step(4); + c_abi.step(4); + let moved = c_abi.plan_latency(); + assert!( + moved.0 > boot.0, + "{rate} Hz: the added inserts raise the plan latency ({} -> {})", + boot.0, + moved.0 + ); + assert_eq!( + browser_plan_latency(&mut browser), + moved, + "{rate} Hz: the browser reports the adopted plan's latency" + ); + } +} diff --git a/scripts/check-abi-layout-v1.py b/scripts/check-abi-layout-v1.py index 2d0eab483..ec1c3ff38 100644 --- a/scripts/check-abi-layout-v1.py +++ b/scripts/check-abi-layout-v1.py @@ -135,6 +135,7 @@ "miso_engine_web_v1_control_response_bytes", "miso_engine_web_v1_control_response_ptr", "miso_engine_web_v1_control_revision", + "miso_engine_web_v1_declare_discontinuity", "miso_engine_web_v1_dispose", "miso_engine_web_v1_document_ptr", "miso_engine_web_v1_meter_header_ptr", @@ -156,6 +157,8 @@ "miso_engine_web_v1_observation_tap_count", "miso_engine_web_v1_observation_tap_id", "miso_engine_web_v1_observation_track_index", + "miso_engine_web_v1_plan_latency_ptr", + "miso_engine_web_v1_plan_latency_read", "miso_engine_web_v1_render", "miso_engine_web_v1_render_allocation_count", "miso_engine_web_v1_resource_ptr", @@ -233,6 +236,7 @@ "meterHeader": 88, "commandReport": 48, "planWatermark": 96, + "planLatency": 64, "observationSelection": 32, "observationResult": 96, "spectrumRequest": 40, @@ -270,6 +274,10 @@ "structSize", "reserved0", "revision", "firstSample", "outcomeFlags", "exactCount", "transitionFallbackCount", "supersededCount", "reserved", ] +# The plan latency record (#1688 D2): the C ABI report's latency and tail words. +PLAN_LATENCY_FIELDS = [ + "structSize", "reserved0", "latencySamples", "tailKind", "tailSamples", "reserved", +] # The status block (#1349 D2): offset 48 `liveValuesSuperseded` (#1399), then the applied-revision # watermark words at 56, 64 and 72, and no reserved word. STATUS_FIELDS = [ @@ -465,6 +473,7 @@ def validate(document: object) -> None: f"status.{row['name']} is a u64 at {STATUS_WATERMARK_OFFSETS[row['name']]}") for name, expected_fields in ( ("planWatermark", PLAN_WATERMARK_FIELDS), + ("planLatency", PLAN_LATENCY_FIELDS), ("observationSelection", OBSERVATION_SELECTION_FIELDS), ("observationResult", OBSERVATION_RESULT_FIELDS), ): @@ -613,6 +622,16 @@ def drop_service_export(document: dict) -> None: def drop_watermark_field(document: dict) -> None: document["structures"]["planWatermark"]["fields"].pop(2) + def drop_plan_latency_export(document: dict) -> None: + export = "miso_engine_web_v1_plan_latency_read" + try: + document["exports"].remove(export) + except ValueError as error: + raise AssertionError("the plan latency export exists in the valid fixture") from error + + def drop_plan_latency_field(document: dict) -> None: + document["structures"]["planLatency"]["fields"].pop(2) + def drop_status_watermark_word(document: dict) -> None: document["structures"]["status"]["fields"].pop() @@ -695,6 +714,8 @@ def boolean_named_value(document: dict) -> None: ("an export is dropped", drop_export), ("the service export is dropped", drop_service_export), ("the watermark record omits a field", drop_watermark_field), + ("the plan latency record omits a field", drop_plan_latency_field), + ("the plan latency read export is dropped", drop_plan_latency_export), ("the status omits a watermark word", drop_status_watermark_word), ("the spectrum-hop capability export is dropped", drop_spectrum_hop_capability_export), ("the export set is unsorted", unsorted_exports), diff --git a/scripts/check-capi-abi.sh b/scripts/check-capi-abi.sh index 7903c5586..bd14df6f4 100755 --- a/scripts/check-capi-abi.sh +++ b/scripts/check-capi-abi.sh @@ -248,6 +248,7 @@ fi printf '%s\n' \ miso_engine_v1_abi_version \ miso_engine_v1_compile_session \ + miso_engine_v1_declare_discontinuity \ miso_engine_v1_dequeue_event \ miso_engine_v1_engine_create \ miso_engine_v1_engine_destroy \ diff --git a/scripts/check-web-audioworklet.sh b/scripts/check-web-audioworklet.sh index 653a176ea..a5bb979e5 100755 --- a/scripts/check-web-audioworklet.sh +++ b/scripts/check-web-audioworklet.sh @@ -290,6 +290,7 @@ expected_exports=$(printf '%s\n' \ miso_engine_web_v1_control_response_bytes \ miso_engine_web_v1_control_response_ptr \ miso_engine_web_v1_control_revision \ + miso_engine_web_v1_declare_discontinuity \ miso_engine_web_v1_document_ptr \ miso_engine_web_v1_dispose \ miso_engine_web_v1_meter_header_ptr \ @@ -311,6 +312,8 @@ expected_exports=$(printf '%s\n' \ miso_engine_web_v1_observation_tap_count \ miso_engine_web_v1_observation_tap_id \ miso_engine_web_v1_observation_track_index \ + miso_engine_web_v1_plan_latency_ptr \ + miso_engine_web_v1_plan_latency_read \ miso_engine_web_v1_render \ miso_engine_web_v1_render_allocation_count \ miso_engine_web_v1_response_frame_close \ diff --git a/scripts/fixtures/abi-layout-v1-self-test.json b/scripts/fixtures/abi-layout-v1-self-test.json index 24cd35f74..c13164449 100644 --- a/scripts/fixtures/abi-layout-v1-self-test.json +++ b/scripts/fixtures/abi-layout-v1-self-test.json @@ -21,6 +21,7 @@ "miso_engine_web_v1_control_response_bytes", "miso_engine_web_v1_control_response_ptr", "miso_engine_web_v1_control_revision", + "miso_engine_web_v1_declare_discontinuity", "miso_engine_web_v1_dispose", "miso_engine_web_v1_document_ptr", "miso_engine_web_v1_meter_header_ptr", @@ -42,6 +43,8 @@ "miso_engine_web_v1_observation_tap_count", "miso_engine_web_v1_observation_tap_id", "miso_engine_web_v1_observation_track_index", + "miso_engine_web_v1_plan_latency_ptr", + "miso_engine_web_v1_plan_latency_read", "miso_engine_web_v1_render", "miso_engine_web_v1_render_allocation_count", "miso_engine_web_v1_resource_ptr", @@ -223,6 +226,17 @@ { "name": "reserved", "offset": 56, "type": "u64[5]" } ] }, + "planLatency": { + "bytes": 64, + "fields": [ + { "name": "structSize", "offset": 0, "type": "u32" }, + { "name": "reserved0", "offset": 4, "type": "u32" }, + { "name": "latencySamples", "offset": 8, "type": "u64" }, + { "name": "tailKind", "offset": 16, "type": "u64" }, + { "name": "tailSamples", "offset": 24, "type": "u64" }, + { "name": "reserved", "offset": 32, "type": "u64[4]" } + ] + }, "observationSelection": { "bytes": 32, "fields": [ diff --git a/sdk/assets/miso-engine-v1-abi-layout.json b/sdk/assets/miso-engine-v1-abi-layout.json index 24cd35f74..c13164449 100644 --- a/sdk/assets/miso-engine-v1-abi-layout.json +++ b/sdk/assets/miso-engine-v1-abi-layout.json @@ -21,6 +21,7 @@ "miso_engine_web_v1_control_response_bytes", "miso_engine_web_v1_control_response_ptr", "miso_engine_web_v1_control_revision", + "miso_engine_web_v1_declare_discontinuity", "miso_engine_web_v1_dispose", "miso_engine_web_v1_document_ptr", "miso_engine_web_v1_meter_header_ptr", @@ -42,6 +43,8 @@ "miso_engine_web_v1_observation_tap_count", "miso_engine_web_v1_observation_tap_id", "miso_engine_web_v1_observation_track_index", + "miso_engine_web_v1_plan_latency_ptr", + "miso_engine_web_v1_plan_latency_read", "miso_engine_web_v1_render", "miso_engine_web_v1_render_allocation_count", "miso_engine_web_v1_resource_ptr", @@ -223,6 +226,17 @@ { "name": "reserved", "offset": 56, "type": "u64[5]" } ] }, + "planLatency": { + "bytes": 64, + "fields": [ + { "name": "structSize", "offset": 0, "type": "u32" }, + { "name": "reserved0", "offset": 4, "type": "u32" }, + { "name": "latencySamples", "offset": 8, "type": "u64" }, + { "name": "tailKind", "offset": 16, "type": "u64" }, + { "name": "tailSamples", "offset": 24, "type": "u64" }, + { "name": "reserved", "offset": 32, "type": "u64[4]" } + ] + }, "observationSelection": { "bytes": 32, "fields": [ diff --git a/sdk/src/generated/abi.ts b/sdk/src/generated/abi.ts index af9c887dd..3b68aa71c 100644 --- a/sdk/src/generated/abi.ts +++ b/sdk/src/generated/abi.ts @@ -48,6 +48,7 @@ export const ABI_LAYOUT = deepFreeze( "miso_engine_web_v1_control_response_bytes", "miso_engine_web_v1_control_response_ptr", "miso_engine_web_v1_control_revision", + "miso_engine_web_v1_declare_discontinuity", "miso_engine_web_v1_dispose", "miso_engine_web_v1_document_ptr", "miso_engine_web_v1_meter_header_ptr", @@ -69,6 +70,8 @@ export const ABI_LAYOUT = deepFreeze( "miso_engine_web_v1_observation_tap_count", "miso_engine_web_v1_observation_tap_id", "miso_engine_web_v1_observation_track_index", + "miso_engine_web_v1_plan_latency_ptr", + "miso_engine_web_v1_plan_latency_read", "miso_engine_web_v1_render", "miso_engine_web_v1_render_allocation_count", "miso_engine_web_v1_resource_ptr", @@ -590,6 +593,41 @@ export const ABI_LAYOUT = deepFreeze( } ] }, + "planLatency": { + "bytes": 64, + "fields": [ + { + "name": "structSize", + "offset": 0, + "type": "u32" + }, + { + "name": "reserved0", + "offset": 4, + "type": "u32" + }, + { + "name": "latencySamples", + "offset": 8, + "type": "u64" + }, + { + "name": "tailKind", + "offset": 16, + "type": "u64" + }, + { + "name": "tailSamples", + "offset": 24, + "type": "u64" + }, + { + "name": "reserved", + "offset": 32, + "type": "u64[4]" + } + ] + }, "observationSelection": { "bytes": 32, "fields": [ diff --git a/tools/audit/src/builtins_graph.rs b/tools/audit/src/builtins_graph.rs index e3c70ab72..727acabcd 100644 --- a/tools/audit/src/builtins_graph.rs +++ b/tools/audit/src/builtins_graph.rs @@ -683,6 +683,7 @@ fn prepare_graph_plan( ) .expect("sealed builtins"); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id, effects, diff --git a/tools/audit/src/capi.rs b/tools/audit/src/capi.rs index 59195ef3d..d35dfeede 100644 --- a/tools/audit/src/capi.rs +++ b/tools/audit/src/capi.rs @@ -11,6 +11,12 @@ //! After the first call, one structural transaction (#1273 deliverable 3) adds a muted track, so //! the next audited call is a swap block that carries the unchanged source's ring into the //! successor plan, and the live edits after it ride the successor's lanes. +//! +//! Right after that transaction, one anchored seek (#1319 D4) moves the source to generation 2 at +//! render sample [`SEEK_ANCHOR_SAMPLE`], and the generation-2 quantum for that frame is submitted. +//! The swap block's carried consumer pops the seek and holds it for the two calls before the +//! anchor, and the call at the anchor applies it and plays the generation-2 quantum, so the +//! held-seek paths of the render run inside the audited scope. #![allow(unsafe_code)] @@ -24,8 +30,8 @@ use capi::{ SubmitReport, miso_engine_v1_compile_session, miso_engine_v1_dequeue_event, miso_engine_v1_engine_create, miso_engine_v1_engine_destroy, miso_engine_v1_plan_destroy, miso_engine_v1_render_f32_planar, miso_engine_v1_session_destroy, - miso_engine_v1_source_submit_planar_f32, miso_engine_v1_submit_command, plan_carry_counts, - plan_replacement_count, + miso_engine_v1_source_seek_at, miso_engine_v1_source_submit_planar_f32, + miso_engine_v1_submit_command, plan_carry_counts, plan_replacement_count, }; use engine::realtime::audit::{self, AuditSnapshot}; use protocol::{ @@ -50,6 +56,11 @@ const COMPRESSOR: &str = "comp"; /// Render calls before the structural transaction (issue #1273 deliverable 3): the next call is /// the carrying swap block. const CALLS_BEFORE_TRANSACTION: u64 = 1; +/// The render sample at which the audit's anchored seek applies (#1319 D4): the call index +/// `SEEK_ANCHOR_SAMPLE / QUANTUM_FRAMES`, three calls after the swap block. +const SEEK_ANCHOR_SAMPLE: u64 = 4 * QUANTUM_FRAMES as u64; +/// The generation the anchored seek starts. +const SEEK_GENERATION: u64 = 2; /// The nine-track EQ fixture with an enabled compressor insert (`comp`) on `eq2`, so the live /// editor also reaches a non-EQ effect lane. Every track keeps its console EQ slot, whose band 1 @@ -161,7 +172,7 @@ impl AuditHandles { return Err(compiled); } - let (submitted, accepted_frames) = submit_source_chunk(handles.session, 0); + let (submitted, accepted_frames) = submit_source_chunk(handles.session, 1, 0); if submitted != RESULT_OK || accepted_frames != QUANTUM_FRAMES as u64 { return Err(if submitted != RESULT_OK { submitted @@ -174,7 +185,11 @@ impl AuditHandles { /// Commits [`structural_transaction`] through `editor`'s request and revision sequence, and /// feeds the source's next block through the producer it moved, so the swap block renders PCM - /// from the carried ring. Control work, run outside every render scope. + /// from the carried ring. Then it anchors a seek of the source to generation 2 at + /// [`SEEK_ANCHOR_SAMPLE`] and submits the generation-2 quantum for that frame (#1319 D4): the + /// seek and the chunk go to the moved producer, in the order the host owes them, and the + /// carried consumer holds the seek until its anchor. Control work, run outside every render + /// scope. fn apply_structural_transaction(&mut self, editor: &mut LiveEditor) -> Result<(), u32> { editor.request_id += 1; let request = structural_transaction(editor.request_id, editor.revision); @@ -200,7 +215,27 @@ impl AuditHandles { return Err(committed); } editor.revision += 1; - match submit_source_chunk(self.session, QUANTUM_FRAMES as u64) { + match submit_source_chunk(self.session, 1, QUANTUM_FRAMES as u64) { + (RESULT_OK, accepted) if accepted == QUANTUM_FRAMES as u64 => {} + (RESULT_OK, _) => return Err(RESULT_INTERNAL), + (refused, _) => return Err(refused), + } + // SAFETY: The session is live and used by this thread alone; the source ID bytes outlive + // the synchronous call. + let seeked = unsafe { + miso_engine_v1_source_seek_at( + self.session, + SOURCE_ID.as_ptr(), + SOURCE_ID.len() as u64, + SEEK_GENERATION, + SEEK_ANCHOR_SAMPLE, + SEEK_ANCHOR_SAMPLE, + ) + }; + if seeked != RESULT_OK { + return Err(seeked); + } + match submit_source_chunk(self.session, SEEK_GENERATION, SEEK_ANCHOR_SAMPLE) { (RESULT_OK, accepted) if accepted == QUANTUM_FRAMES as u64 => Ok(()), (RESULT_OK, _) => Err(RESULT_INTERNAL), (refused, _) => Err(refused), @@ -208,16 +243,19 @@ impl AuditHandles { } } -/// Submits one generation-1 quantum of constant source at `start_frame`, returning the result +/// The fixture's one source. +const SOURCE_ID: &[u8] = b"fixture-source"; + +/// Submits one quantum of constant source of `generation` at `start_frame`, returning the result /// and the accepted frame count. -fn submit_source_chunk(session: *mut Session, start_frame: u64) -> (u32, u64) { +fn submit_source_chunk(session: *mut Session, generation: u64, start_frame: u64) -> (u32, u64) { let left = [0.25_f32; QUANTUM_FRAMES]; let right = [-0.5_f32; QUANTUM_FRAMES]; let planes = [left.as_ptr(), right.as_ptr()]; let chunk = SourceChunk { struct_size: SOURCE_CHUNK_SIZE, sample_rate_hz: SAMPLE_RATE_HZ, - generation: 1, + generation, start_frame, planes: planes.as_ptr(), plane_count: 2, @@ -237,8 +275,8 @@ fn submit_source_chunk(session: *mut Session, start_frame: u64) -> (u32, u64) { let submitted = unsafe { miso_engine_v1_source_submit_planar_f32( session, - b"fixture-source".as_ptr(), - 14, + SOURCE_ID.as_ptr(), + SOURCE_ID.len() as u64, &chunk, &mut report, ) diff --git a/tools/audit/src/fixture_builtins.rs b/tools/audit/src/fixture_builtins.rs index 92caeea9a..58811a755 100644 --- a/tools/audit/src/fixture_builtins.rs +++ b/tools/audit/src/fixture_builtins.rs @@ -709,6 +709,7 @@ fn graph_tap_artifact() -> graph_compiler::PreparedGraphBuiltinsArtifact { ) .expect("prepare builtins"); GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: 7, effects, diff --git a/tools/audit/src/graph.rs b/tools/audit/src/graph.rs index 944d13946..43929c2e4 100644 --- a/tools/audit/src/graph.rs +++ b/tools/audit/src/graph.rs @@ -308,6 +308,7 @@ fn prepared_graph( .expect("sealed builtins"); let dispatch = Backend::current(); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch, plan_id, effects: EffectPreparedSession { diff --git a/tools/console-workload/src/lib.rs b/tools/console-workload/src/lib.rs index 0f95a863b..6a99db19d 100644 --- a/tools/console-workload/src/lib.rs +++ b/tools/console-workload/src/lib.rs @@ -1399,6 +1399,7 @@ impl SessionRuntime { } .expect("prepared console builtins"); let artifact = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch, plan_id: PLAN_ID, effects, @@ -2605,6 +2606,7 @@ mod tests { let builtins = builtins_compiler::prepare_session_builtins(&session, &[], builtin_caps()) .expect("prepared console builtins"); let Ok(artifact) = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: PLAN_ID, effects, @@ -3227,6 +3229,7 @@ mod tests { let builtins = builtins_compiler::prepare_session_builtins(&session, &[], builtin_caps()) .expect("prepared console builtins"); let Ok(artifact) = GraphCompiler::compile_with_builtins(GraphBuiltinsCompileRequest { + input_floors: Default::default(), dispatch: Backend::current(), plan_id: PLAN_ID, effects, diff --git a/tools/parameter-metadata/src/abi_layout.rs b/tools/parameter-metadata/src/abi_layout.rs index c52236167..4bbaf6b19 100644 --- a/tools/parameter-metadata/src/abi_layout.rs +++ b/tools/parameter-metadata/src/abi_layout.rs @@ -68,25 +68,26 @@ use host_web::{ MAXIMUM_COMMAND_RECORDS, MAXIMUM_DOCUMENT_BYTES, MAXIMUM_OBSERVATION_TAPS, METER_HEADER_BYTES, OBSERVATION_CHANNEL_BOTH, OBSERVATION_CHANNEL_LEFT, OBSERVATION_CHANNEL_RIGHT, OBSERVATION_RESULT_BYTES, OBSERVATION_SELECTION_BYTES, OBSERVATION_STATUS_PENDING, - OBSERVATION_STATUS_READY, OBSERVATION_STATUS_UNARMED, RACK_CONSOLE, RACK_INSERTS, - RACK_NOT_APPLICABLE, RESOURCE_REPORT_BYTES, RESPONSE_CHANNEL_BOTH, RESPONSE_CHANNEL_LEFT, - RESPONSE_CHANNEL_RIGHT, RESPONSE_GRID_LINEAR, RESPONSE_GRID_LOGARITHMIC, RESULT_ABI_MISMATCH, - RESULT_BACKPRESSURE, RESULT_BUFFER_TOO_SMALL, RESULT_COMPILE_REJECTED, RESULT_INTERNAL, - RESULT_INVALID_ARGUMENT, RESULT_OK, RESULT_REFUSED_BUDGET, RESULT_REFUSED_DOCUMENT, - RESULT_REFUSED_LIFECYCLE, RESULT_REFUSED_OPTIONS, RESULT_RENDER_REJECTED, - RESULT_REPREPARE_REQUIRED, RESULT_UNSUPPORTED, RESULT_WRONG_STATE, SOURCE_STALL_TOLERANCE_MS, - SPECTRUM_BIN_COUNT, SPECTRUM_CAPTURE_BYTES, SPECTRUM_CHANNEL_BOTH, SPECTRUM_CHANNEL_LEFT, - SPECTRUM_CHANNEL_RIGHT, SPECTRUM_COLLECTION_ENTRY_BYTES, SPECTRUM_COLLECTION_ENTRY_CAPACITY, - SPECTRUM_COLLECTION_REQUEST_BYTES, SPECTRUM_COLLECTION_TARGET_IDS_BYTES, - SPECTRUM_MAXIMUM_ID_BYTES, SPECTRUM_MAXIMUM_PREPARED_TARGETS, SPECTRUM_REQUEST_BYTES, - SPECTRUM_RESULT_HEADER_BYTES, SPECTRUM_STREAM_METADATA_BYTES, SPECTRUM_STREAM_STATUS_FAILED, - SPECTRUM_STREAM_STATUS_GAP, SPECTRUM_STREAM_STATUS_INACTIVE, SPECTRUM_STREAM_STATUS_PENDING, - SPECTRUM_STREAM_STATUS_READY, SPECTRUM_STREAM_STATUS_STOPPED, SPECTRUM_STREAM_STATUS_WARMING, - SPECTRUM_TARGET_OUTPUT, SPECTRUM_TARGET_TRACK_POST_INPUT, SPECTRUM_TARGET_TRACK_POST_PAN, - SPECTRUM_WINDOW_FRAMES, SPECTRUM_WINDOW_HEADER_BYTES, STATE_DISPOSED, STATE_FAILED, - STATE_READY, STATUS_BYTES, WATERMARK_BYTES, WebBootOptions, WebCommandReport, - WebLiveResponseOwner, WebLiveResponseRequest, WebLiveResponseResult, WebLiveResponseSection, - WebMeterHeader, WebObservationResult, WebObservationSelection, WebPlanWatermark, + OBSERVATION_STATUS_READY, OBSERVATION_STATUS_UNARMED, PLAN_LATENCY_BYTES, RACK_CONSOLE, + RACK_INSERTS, RACK_NOT_APPLICABLE, RESOURCE_REPORT_BYTES, RESPONSE_CHANNEL_BOTH, + RESPONSE_CHANNEL_LEFT, RESPONSE_CHANNEL_RIGHT, RESPONSE_GRID_LINEAR, RESPONSE_GRID_LOGARITHMIC, + RESULT_ABI_MISMATCH, RESULT_BACKPRESSURE, RESULT_BUFFER_TOO_SMALL, RESULT_COMPILE_REJECTED, + RESULT_INTERNAL, RESULT_INVALID_ARGUMENT, RESULT_OK, RESULT_REFUSED_BUDGET, + RESULT_REFUSED_DOCUMENT, RESULT_REFUSED_LIFECYCLE, RESULT_REFUSED_OPTIONS, + RESULT_RENDER_REJECTED, RESULT_REPREPARE_REQUIRED, RESULT_UNSUPPORTED, RESULT_WRONG_STATE, + SOURCE_STALL_TOLERANCE_MS, SPECTRUM_BIN_COUNT, SPECTRUM_CAPTURE_BYTES, SPECTRUM_CHANNEL_BOTH, + SPECTRUM_CHANNEL_LEFT, SPECTRUM_CHANNEL_RIGHT, SPECTRUM_COLLECTION_ENTRY_BYTES, + SPECTRUM_COLLECTION_ENTRY_CAPACITY, SPECTRUM_COLLECTION_REQUEST_BYTES, + SPECTRUM_COLLECTION_TARGET_IDS_BYTES, SPECTRUM_MAXIMUM_ID_BYTES, + SPECTRUM_MAXIMUM_PREPARED_TARGETS, SPECTRUM_REQUEST_BYTES, SPECTRUM_RESULT_HEADER_BYTES, + SPECTRUM_STREAM_METADATA_BYTES, SPECTRUM_STREAM_STATUS_FAILED, SPECTRUM_STREAM_STATUS_GAP, + SPECTRUM_STREAM_STATUS_INACTIVE, SPECTRUM_STREAM_STATUS_PENDING, SPECTRUM_STREAM_STATUS_READY, + SPECTRUM_STREAM_STATUS_STOPPED, SPECTRUM_STREAM_STATUS_WARMING, SPECTRUM_TARGET_OUTPUT, + SPECTRUM_TARGET_TRACK_POST_INPUT, SPECTRUM_TARGET_TRACK_POST_PAN, SPECTRUM_WINDOW_FRAMES, + SPECTRUM_WINDOW_HEADER_BYTES, STATE_DISPOSED, STATE_FAILED, STATE_READY, STATUS_BYTES, + WATERMARK_BYTES, WebBootOptions, WebCommandReport, WebLiveResponseOwner, + WebLiveResponseRequest, WebLiveResponseResult, WebLiveResponseSection, WebMeterHeader, + WebObservationResult, WebObservationSelection, WebPlanLatency, WebPlanWatermark, WebResourceReport, WebSpectrumCollectionEntry, WebSpectrumCollectionRequest, WebSpectrumRequest, WebSpectrumResult, WebSpectrumStreamMetadata, WebSpectrumWindow, WebStatus, }; @@ -122,7 +123,7 @@ pub const ERROR_PHASES: [&str; 6] = ["asset", "boot", "source", "render", "outpu /// Publishing the whole surface -- not just the four boot calls -- is what lets a JavaScript /// consumer name an export without typing a string. `memory` is deliberately absent: it is the /// module's linear memory, not a call, and a consumer reaches it as `instance.exports.memory`. -pub const EXPORTS: [&str; 102] = [ +pub const EXPORTS: [&str; 105] = [ "miso_engine_web_v1_abi_version", "miso_engine_web_v1_boot", "miso_engine_web_v1_boot_diagnostic_bytes", @@ -140,6 +141,7 @@ pub const EXPORTS: [&str; 102] = [ "miso_engine_web_v1_control_response_bytes", "miso_engine_web_v1_control_response_ptr", "miso_engine_web_v1_control_revision", + "miso_engine_web_v1_declare_discontinuity", "miso_engine_web_v1_dispose", "miso_engine_web_v1_document_ptr", "miso_engine_web_v1_meter_header_ptr", @@ -161,6 +163,8 @@ pub const EXPORTS: [&str; 102] = [ "miso_engine_web_v1_observation_tap_count", "miso_engine_web_v1_observation_tap_id", "miso_engine_web_v1_observation_track_index", + "miso_engine_web_v1_plan_latency_ptr", + "miso_engine_web_v1_plan_latency_read", "miso_engine_web_v1_render", "miso_engine_web_v1_render_allocation_count", "miso_engine_web_v1_resource_ptr", @@ -556,6 +560,26 @@ fn plan_watermark_fields() -> [Field; 9] { ] } +/// The plan latency record (issue #1688 D2): the C ABI report's latency and tail words. +fn plan_latency_fields() -> [Field; 6] { + [ + ("structSize", offset_of!(WebPlanLatency, struct_size), "u32"), + ("reserved0", offset_of!(WebPlanLatency, reserved0), "u32"), + ( + "latencySamples", + offset_of!(WebPlanLatency, latency_samples), + "u64", + ), + ("tailKind", offset_of!(WebPlanLatency, tail_kind), "u64"), + ( + "tailSamples", + offset_of!(WebPlanLatency, tail_samples), + "u64", + ), + ("reserved", offset_of!(WebPlanLatency, reserved), "u64[4]"), + ] +} + fn command_report_fields() -> [Field; 8] { [ ( @@ -1423,6 +1447,13 @@ pub fn render() -> String { &plan_watermark_fields(), true, ); + render_structure( + &mut out, + "planLatency", + PLAN_LATENCY_BYTES, + &plan_latency_fields(), + true, + ); render_structure( &mut out, "observationSelection",