diff --git a/.github/ISSUE_SPECS/1726-finish-the-test-policy-cut.md b/.github/ISSUE_SPECS/1726-finish-the-test-policy-cut.md new file mode 100644 index 000000000..5dd9e17d5 --- /dev/null +++ b/.github/ISSUE_SPECS/1726-finish-the-test-policy-cut.md @@ -0,0 +1,181 @@ +# Finish the test policy cut: the five owner answers to TA1's asks (test policy cut 2) + +Owner answers: the five open questions of `.github/ISSUE_SPECS/1722-delete-digest-type-size-and-unrun-tests.md` +("ASK"), answered by the owner and relayed by root on 2026-10-19. Branch: +`codex/batch-test-policy-2`, from `main` at `ce714017c` (TA1 #1722 / PR #1723), with `origin/main` +`fdb1c44c2` (TA2 #1725) merged in before the gates. Phase B PR 2 (TA3) of the 2026-10 test audit. + +## The owner answers + +1. **ASK 1, the last three scalar-oracle pairing tests: option A.** Delete them and the test-only + initial-matrix trace. The oracle's render path stays. +2. **ASK 2, `math` m2 `EXP2_DIGEST`/`LOG2_DIGEST`: keep.** It stays until #1019 is fixed; the PR that + fixes #1019 deletes it together with its `scripts/lib/aarch64-known-defects.py` row. +3. **ASK 3, the host-core `limiter_linked_session` `PIN`:** if G5 already covers the case, delete + the pin and cite the covering entry; if not, add the case to G5 in the corpus's own way, then + delete the pin, and point true-peak-limiter `MUTATIONS.md` row 996-K2 at the new owner. +4. **ASK 4, `gain_pan_profile` `phase_profile`: delete.** Then delete the graph phase-profile probes + too if only tests or test-support use them. +5. **ASK 5, the one-line `size_of` / byte-count re-derivations inside larger tests: trim (class (b), + approved).** Keep real resource ceilings, allocation counts and ABI layouts. A line that guards a + relation the product enforces becomes that relation without `size_of` of an internal type. + +## Non-goals + +- No change to product render paths, ceilings the product enforces, C ABI or browser layouts, wire + goldens, fuzz targets, randomized differentials, allocation counts, or open-defect reproducers. +- `crates/math/tests/m2_lane_identity.rs` is not touched (answer 2). + +## Objective gates + +1. `cargo fmt --all --check`. +2. `cargo clippy --locked -p --all-targets --all-features -- -D warnings`. +3. `cargo test --locked --all-features -p `. +4. `bash scripts/check-workspace-policy.sh`. +5. `scripts/run-wasm-gates.sh --without-v8-spill` (the G5 wasm leg, for the new corpus case). + +Touched packages: builtins-compiler, true-peak-limiter, wasm-gate-corpus, wasm-gates (its G5 test +reads the new case), host-core, graph, rack, console-workload, graph-compiler. + +## Evidence + +Line numbers are the base's (`ce714017c` for items 1-4, the merged tree `be08944b3` for item 5). + +### Item 1: the scalar-oracle pairing tests (answer 1) + +`crates/builtins-compiler/src/lib.rs` (1,100 lines removed, 12 added), commit "TA3 item 1": + +- Deleted tests: `actual_scalar_nonadjacent_intervening_observer_error_completes_and_retries`, + `actual_scalar_nonadjacent_failed_render_materializes_post_fader_before_error`, + `actual_scalar_nonadjacent_ramp_retarget_and_failed_retry_stay_at_original_boundaries` (12466-13178). +- Deleted helpers they alone used: `render_failed_buffer_result`, `first_fader_state`, + `last_fader_state`, `first_matrix_state`, `last_matrix_state`, `assert_owner_states_match` + (12224-12338). +- Deleted instrumentation (5252-5528): `TestOnlyInitialMatrixTrace`, `EMPTY_INITIAL_MATRIX_TRACE`, + `SCALAR_INITIAL_MATRIX_TRACE`, `record_initial_matrix_state`, `initial_matrix_state`, and with them + the scalar state trace only those tests read: `TestOnlyScalarStateTrace`, + `SCALAR_STATE_TRACE_CAPACITY`, `EMPTY_SCALAR_STATE_TRACE`, `SCALAR_STATE_TRACE`, + `test_only_scalar_state_trace`, `SCALAR_FADER_STATE_WITNESS`, `SCALAR_MATRIX_STATE_WITNESS`, the + witness's `scalar_fader_words`/`scalar_matrix_words` fields (unread since TA1), the writers + `record_scalar_fader_state`, `record_scalar_matrix_state`, `record_scalar_state`, `scalar_owner_id`. +- Deleted plumbing in the oracle: the `test_only_owner` fields of `LiveControlFaderProcessor` and + `LiveControlMatrixProcessor` and their initializers (7036-7097, 9418-9513), and the fifteen + `record_scalar_*` calls of `ScalarPairProcessor` and `ScalarSplitPairProcessor` (9632-9910). Each + `.inspect_err(|_| record…)?` became `?`. The render arithmetic and the remaining witnesses + (`FADER_MATRIX_LIVE_WITNESS`, `FADER_MATRIX_OUTPUT_WITNESS`, owner drops, outer lifetime) are + unchanged. +- `BoundaryVariant::Nonadjacent` and `NonadjacentOutputConflict` (constructed only by the deleted + tests) and the output-track branch for the latter (11198-11295). +- `crates/graph/tests/MUTATIONS.md`: a TA3 note after the TA1 note (rows 916-16, 916-17). + +### Item 2: the m2 digests (answer 2) + +No change. The PR that fixes #1019 deletes `EXP2_DIGEST`/`LOG2_DIGEST` and the LANE-3 row of +`scripts/lib/aarch64-known-defects.py`. + +### Item 3: the linked-limiter session pin (answer 3) + +**Did G5 cover it? No.** G5's owner is `tools/wasm-gates/tests/g5_native_corpus.rs` +(`g5_native_digests_match_pins`) and the wasm leg of `scripts/run-wasm-gates.sh`, over +`tools/wasm-gate-corpus`. The limiter's delegated cases (`crates/true-peak-limiter/src/corpus.rs`, +`D90_DIGESTS`) were five, each one `limiter_block` call over 1,024 frames with fixed parameters: +`dual_mono/noise`, `maximum/noise`, `dual_mono/near_nyquist_w_min`, +`dual_mono/impulse_train_long_lookahead`, `dual_mono/subnormal_release_flush`. None retargets a +parameter, unlinks a pair or re-equals one; `limiter_block` passes `linked = false`, and the #990 +decision and its `gain_linked` record live in `LimiterCore::process_block`, which no case reached. +So a relink on designed agreement (996-K2) could not move any G5 digest. + +**Added, in the corpus's own way:** case 5, `maximum/link_unlink_relink` +(`crates/true-peak-limiter/src/corpus.rs`, appended; `CASE_COUNT` 5 -> 6; `run_linked`, +`LINKED_RETARGETS`, `fill_linked`). It builds the prepared `LimiterCore` at each width (eight +uniform lanes at a 5 ms lookahead, `LinkMode::Maximum`), renders eight 128-frame blocks of noise at ++6 to +18 dBFS, and retargets: both channels of every lane at block 3 (a linked ramp), lane 0 left +and lane 7 right at block 4 (unlinks), the other channels equal again at block 5. Pin, generated as +the corpus documents (the scalar-oracle digest `g5_native_digests_match_pins` prints): `31b09672…`. +It was identical at `Scalar`, `Simd4` and `Simd8` natively and in the wasm `simd128` leg +(`run-wasm-gates.sh`: 153 cases, no mismatch). AArch64 runs it in CI's `aarch64-release` G5 step; +it was not run on this box. Appending it moves the compressor, builtin-tail and automation blocks up +by one global index; their pins are indexed in their own crates, and the layout test passed. +`tools/wasm-gate-corpus/src/lib.rs` (`LIMITER_CASE_COUNT` doc) says so. + +Mutants (true-peak-limiter `src/lib.rs`, one at a time, restored with `git checkout`), on +`g5_native_digests_match_pins`: + +| mutant | Scalar | Simd4 | Simd8 | +|---|---|---|---| +| control: the decision always `false` (never links, the pre-#990 behaviour) | green | green | green | +| 996-K2 / 997-K2: the decision not written back to `gain_linked` | RED | RED | RED | +| K1: the decision drops `designed_gain_agree` | RED | RED | RED | +| #990 M1, M2, M6 (mirrored stores into the right rings) | RED | RED | RED | +| #990 M9: right ramps not copied at block end | green | green | green | +| E0: `designed_gain_agree` skips lane 0 | RED | RED | green | + +The control shows the pin is the words of a kernel that never links. M9 and E0 at `Simd8` stay +owned by the session test's "equal" and "parts nothing" checks. + +**Deleted:** `crates/host-core/src/limiter_linked_session.rs` `PIN`, the SHA-256 over master words +and readings (`Sha256Sink`, `digest`), the digest print and the `MISO_ENGINE_REPIN_LIMITER_LINKED_SESSION` +switch (lines 156, 193-194, 532-695, 763-771). The three tests keep every session check and are +renamed `the_hot_console_links_unlinks_and_limits_at_{simd8,simd4,scalar}`; the module note says +where the words are owned now. `crates/true-peak-limiter/tests/MUTATIONS.md`: row 996-K2 points at +the G5 case, and a TA3 section records the table above. + +### Item 4: the phase profile and its probes (answer 4) + +**Probe decision: delete.** No product code uses them: every probe was `cfg(feature = +"test-support")` (rack) or `cfg(any(test, feature = "test-support"))` (graph), and their only +consumers were `phase_profile`, the `bus_send_profile` example (an ignored-style measurement +instrument that gates nothing), and the probes' own tests. + +- `tools/console-workload/tests/gain_pan_profile.rs`: deleted (504 lines; `phase_profile` and + `probes_split_the_bank_units_and_move_no_bit`, the probes' own gate). +- `tools/console-workload/examples/bus_send_profile.rs`: deleted (397 lines). +- `crates/graph/src/lib.rs`: `test_only_phase_profile` (38-210) and the four probe sites in + `GraphExecutor::render` (5879-6004). `crates/graph/src/runtime.rs`: `test_only_unit_phase` + (4582-4610). +- `crates/rack/src/lib.rs`: `test_only_bank_phase_profile` (21-188), `BankStage::test_only_stage_name` + (575-581), the probe sites in `BankChain::run` and the scatter paths (3215-3978), and + `ProbedMembers` with `bank_phase_probes_split_every_scatter_path_and_move_no_bit` (7926-8081). +- `crates/rack/Cargo.toml`: the now empty `test-support` feature; `crates/graph/Cargo.toml`: + `test-support = []`; `scripts/test-test-support-ci.py` line 161: `rack` leaves the expected set. + +### Item 5: size_of re-derivations (answer 5) + +| file | lines | change | relation kept, and its mutant | +|---|---|---|---| +| `crates/host-core/tests/successor_swap.rs` | 48-52, 214-218, 314-315, 967-970, 1680-1681, 1732-1756, 1805-1810, 3250-3254, 3293-3294, 4393-4395, 4696-4780, 4980-4982, 5175-5176, 5410-5411 | the fourteen `size_of::` formulas (and the literal `8`s beside them) become exact table counts, `[sources, input lanes, strip stages, effect lanes]`, through a new test-support accessor `graph::test_only_carry_counts` (`crates/graph/src/lib.rs`, beside `carry_program_retained_bytes`) and a `carry_counts` field in `tests/support/successor.rs` and `PlainRun` | every exact move count. Mutant: drop `row.live != live_input` from the input-section join (`host-core/src/prepare.rs`) -> `a_changed_control_kind_carries_no_input_section` RED | +| `crates/host-core/tests/prepare.rs` | 407-475 | `effect_control_report_uses_actual_native_capacity_strings_and_owners` (a field-by-field `size_of` formula of the effect compiler's charge) becomes `effect_control_report_charges_the_producers_the_host_hands_out`: the report equals `effect_compiler::effect_control_resources` of the producers handed out; the owners share one factory; largest = max(table, owned). It now checks the argument host-core passes to the charge; the charge itself is tested in `crates/effect-compiler/src/prepare.rs` by `effect_control_resources_charge_actual_tables_and_shared_owner_once` (:1334) and `effect_control_resources_charge_no_registry_owned_factory` (:1371) | Mutant: host-core charges `&Vec::new()` -> RED | +| `crates/host-core/tests/submix_strip.rs` | 836-848 | `size_of::() + params * size_of::()` goes | "a bus insert is charged once": the model delta equals the same insert's on a track, the strings are retained once, and the delta exceeds the canonical part. Mutant: charge submix insert vectors twice (`session/src/estimate.rs`) -> RED | +| `crates/host-core/tests/route_mute.rs` | 2460, 2514-2517 | the `LatestCell<7>` + `DirtyWord` layout line goes | the audited-allocator ceiling (`total_bytes >= added`) stays; no product equality depended on the line | +| `crates/graph-compiler/src/lib.rs` | 385-391, 12414, 12824, 13181, 13549 | `literal_bank_slot_reservation_bytes` (a `size_of` of `rack::BankSlot` and `Box`) becomes `GraphBankSlotResourceEstimate::checked_for(bank_count, width).total_bytes`, as TA1 did in `runtime_bank_slot_reservation_is_published_and_capped_transactionally` | the reports fold the reservation exactly once into four estimate rows. Mutant: skip the `checked_add_bank_slot_owners` fold on `estimate` (`graph-compiler/src/compile.rs`) -> six tests RED, the four launch-fixture tests among them | +| `crates/capi/tests/resource_lifecycle.rs` | 670, 693, 863 | **not changed** | each line feeds an exact-cap peak the product enforces (at the peak admits, one byte below refuses): the strips' slice (`size_of_val` of the real slice), the affine token's `PreparedStructuralCommand` layout, and the `u16` decode field. No ABI report exposes the prepared-protocol term, so a relation without `size_of` needs a product change; out of scope here | + +## Residual risk + +The new G5 case runs the linked limiter body (`linked_frame_uniform` and the mirrored stores) on +NEON under G5 for the first time, and its pin was not checked on aarch64 here (x86-64-v3 native and +wasm `simd128` only). If the required `aarch64-release` G5 step goes red on +`effect/true_peak_limiter/maximum/link_unlink_relink`, the first step is to check the arm64 digest +against the scalar oracle; it is not by itself a regression of this change. + +## Test value + +No test is new. The rewritten assertions and the defect each turns red on: + +- `limiter` G5 case 5: a pair that relinks on designed agreement without reset, restore or + `desymmetrize` (996-K2), at every width and on every target G5 runs. +- `successor_swap` counts: a carry clause that moves the wrong number of input, strip or effect + entries (as the byte formulas did, without the layout). +- `effect_control_report_charges_the_producers_the_host_hands_out`: a host that admits a charge of + anything but the producers it hands out. +- `a_bus_insert_is_charged_once_in_the_session_estimate`: a bus insert charged differently from + the same track insert. +- the launch-fixture estimate tests: a slot reservation folded twice, or not at all, into the + estimate rows. + +## Docs updated + +`crates/graph/tests/MUTATIONS.md` and `crates/true-peak-limiter/tests/MUTATIONS.md` (TA3 sections), +the module notes of `limiter_linked_session.rs` and `corpus.rs`, and the G5 limiter block doc in +`tools/wasm-gate-corpus/src/lib.rs`. `docs/handoffs/`, `docs/audits/` and `artifacts/` are dated +records and were not changed. diff --git a/crates/builtins-compiler/src/lib.rs b/crates/builtins-compiler/src/lib.rs index 3e2039b5a..27ab82913 100644 --- a/crates/builtins-compiler/src/lib.rs +++ b/crates/builtins-compiler/src/lib.rs @@ -5249,34 +5249,6 @@ static FADER_MATRIX_FUSED_CALLS: core::sync::atomic::AtomicUsize = static FADER_MATRIX_FALLBACK_CALLS: core::sync::atomic::AtomicUsize = core::sync::atomic::AtomicUsize::new(0); -// Render-local witnesses for the serialized-pair proof. These are deliberately thread-local: -// a host test can reset and read the counters around one real render without a process-global -// diagnostic surface or interference from another test thread. -#[cfg(any(test, feature = "test-support"))] -const SCALAR_STATE_TRACE_CAPACITY: usize = 8; - -#[cfg(any(test, feature = "test-support"))] -#[derive(Clone, Copy, Debug, Eq, PartialEq)] -#[doc(hidden)] -pub struct TestOnlyScalarStateTrace { - pub fader_len: usize, - pub fader_overflow: bool, - pub fader_owners: [u16; SCALAR_STATE_TRACE_CAPACITY], - pub fader_words: [[u32; 14]; SCALAR_STATE_TRACE_CAPACITY], - pub matrix_len: usize, - pub matrix_overflow: bool, - pub matrix_owners: [u16; SCALAR_STATE_TRACE_CAPACITY], - pub matrix_words: [[u32; 15]; SCALAR_STATE_TRACE_CAPACITY], -} - -#[cfg(all(test, feature = "test-support"))] -#[derive(Clone, Copy, Debug, Eq, PartialEq)] -struct TestOnlyInitialMatrixTrace { - len: usize, - owners: [u16; SCALAR_STATE_TRACE_CAPACITY], - words: [[u32; 15]; SCALAR_STATE_TRACE_CAPACITY], -} - /// Test-only source of stamps: a gate that stamps the next revision for a write and publishes it /// when the write is done, as a transaction's commit does (#1312 D7). #[cfg(any(test, feature = "test-support"))] @@ -5340,36 +5312,13 @@ impl TestOnlyWriteGate for TestOnlyGate { } } -#[cfg(any(test, feature = "test-support"))] -const EMPTY_SCALAR_STATE_TRACE: TestOnlyScalarStateTrace = TestOnlyScalarStateTrace { - fader_len: 0, - fader_overflow: false, - fader_owners: [0; SCALAR_STATE_TRACE_CAPACITY], - fader_words: [[0; 14]; SCALAR_STATE_TRACE_CAPACITY], - matrix_len: 0, - matrix_overflow: false, - matrix_owners: [0; SCALAR_STATE_TRACE_CAPACITY], - matrix_words: [[0; 15]; SCALAR_STATE_TRACE_CAPACITY], -}; - -#[cfg(all(test, feature = "test-support"))] -const EMPTY_INITIAL_MATRIX_TRACE: TestOnlyInitialMatrixTrace = TestOnlyInitialMatrixTrace { - len: 0, - owners: [0; SCALAR_STATE_TRACE_CAPACITY], - words: [[0; 15]; SCALAR_STATE_TRACE_CAPACITY], -}; - +// Render-local witnesses for the serialized-pair proof. These are deliberately thread-local: +// a host test can reset and read the counters around one real render without a process-global +// diagnostic surface or interference from another test thread. #[cfg(any(test, feature = "test-support"))] thread_local! { static FADER_MATRIX_LIVE_WITNESS: std::cell::Cell<[u64; 8]> = const { std::cell::Cell::new([0; 8]) }; static FADER_MATRIX_OUTPUT_WITNESS: std::cell::Cell<[u32; 2]> = const { std::cell::Cell::new([0; 2]) }; - static SCALAR_FADER_STATE_WITNESS: std::cell::Cell<[u32; 14]> = const { std::cell::Cell::new([0; 14]) }; - static SCALAR_MATRIX_STATE_WITNESS: std::cell::Cell<[u32; 15]> = const { std::cell::Cell::new([0; 15]) }; - static SCALAR_STATE_TRACE: std::cell::Cell = - const { std::cell::Cell::new(EMPTY_SCALAR_STATE_TRACE) }; - #[cfg(all(test, feature = "test-support"))] - static SCALAR_INITIAL_MATRIX_TRACE: std::cell::Cell = - const { std::cell::Cell::new(EMPTY_INITIAL_MATRIX_TRACE) }; static SCALAR_OWNER_DROPS: std::cell::Cell<[u64; 3]> = const { std::cell::Cell::new([0; 3]) }; static SCALAR_OUTER_LIFETIME: std::cell::Cell<[u64; 3]> = const { std::cell::Cell::new([0; 3]) }; } @@ -5388,8 +5337,6 @@ pub struct TestOnlyFaderMatrixWitness { pub matrix_values_applied: u64, pub first_left_bits: u32, pub first_right_bits: u32, - pub scalar_fader_words: [u32; 14], - pub scalar_matrix_words: [u32; 15], } #[cfg(any(test, feature = "test-support"))] @@ -5397,9 +5344,6 @@ pub struct TestOnlyFaderMatrixWitness { pub fn test_only_reset_fader_matrix_witness() { FADER_MATRIX_LIVE_WITNESS.with(|value| value.set([0; 8])); FADER_MATRIX_OUTPUT_WITNESS.with(|value| value.set([0; 2])); - SCALAR_FADER_STATE_WITNESS.with(|value| value.set([0; 14])); - SCALAR_MATRIX_STATE_WITNESS.with(|value| value.set([0; 15])); - SCALAR_STATE_TRACE.with(|value| value.set(EMPTY_SCALAR_STATE_TRACE)); SCALAR_OWNER_DROPS.with(|value| value.set([0; 3])); } @@ -5431,101 +5375,9 @@ pub fn test_only_fader_matrix_witness() -> TestOnlyFaderMatrixWitness { matrix_values_applied: values[7], first_left_bits: FADER_MATRIX_OUTPUT_WITNESS.with(std::cell::Cell::get)[0], first_right_bits: FADER_MATRIX_OUTPUT_WITNESS.with(std::cell::Cell::get)[1], - scalar_fader_words: SCALAR_FADER_STATE_WITNESS.with(std::cell::Cell::get), - scalar_matrix_words: SCALAR_MATRIX_STATE_WITNESS.with(std::cell::Cell::get), } } -#[cfg(any(test, feature = "test-support"))] -#[must_use] -#[doc(hidden)] -pub fn test_only_scalar_state_trace() -> TestOnlyScalarStateTrace { - SCALAR_STATE_TRACE.with(std::cell::Cell::get) -} - -#[cfg(all(test, feature = "test-support"))] -fn record_initial_matrix_state(owner: u16, matrix: &MatrixBuiltins) { - let words = builtins::test_support::scalar_matrix_words(matrix); - SCALAR_INITIAL_MATRIX_TRACE.with(|value| { - let mut trace = value.get(); - let index = trace.owners[..trace.len] - .iter() - .position(|candidate| *candidate == owner) - .unwrap_or_else(|| { - let index = trace.len; - if index < SCALAR_STATE_TRACE_CAPACITY { - trace.len += 1; - } - index - }); - if index < SCALAR_STATE_TRACE_CAPACITY { - trace.owners[index] = owner; - trace.words[index] = words; - value.set(trace); - } - }); -} - -#[cfg(all(test, feature = "test-support"))] -fn initial_matrix_state(owner: u16) -> Option<[u32; 15]> { - SCALAR_INITIAL_MATRIX_TRACE.with(|value| { - let trace = value.get(); - trace.owners[..trace.len] - .iter() - .position(|candidate| *candidate == owner) - .map(|index| trace.words[index]) - }) -} - -#[cfg(any(test, feature = "test-support"))] -fn record_scalar_fader_state(owner: u16, fader: &FaderMuteRampBuiltins) { - let words = builtins::test_support::scalar_fader_words(fader); - SCALAR_FADER_STATE_WITNESS.with(|value| value.set(words)); - SCALAR_STATE_TRACE.with(|value| { - let mut trace = value.get(); - if trace.fader_len < SCALAR_STATE_TRACE_CAPACITY { - trace.fader_owners[trace.fader_len] = owner; - trace.fader_words[trace.fader_len] = words; - trace.fader_len += 1; - } else { - trace.fader_overflow = true; - } - value.set(trace); - }); -} - -#[cfg(any(test, feature = "test-support"))] -fn record_scalar_matrix_state(owner: u16, matrix: &MatrixBuiltins) { - let words = builtins::test_support::scalar_matrix_words(matrix); - SCALAR_MATRIX_STATE_WITNESS.with(|value| value.set(words)); - SCALAR_STATE_TRACE.with(|value| { - let mut trace = value.get(); - if trace.matrix_len < SCALAR_STATE_TRACE_CAPACITY { - trace.matrix_owners[trace.matrix_len] = owner; - trace.matrix_words[trace.matrix_len] = words; - trace.matrix_len += 1; - } else { - trace.matrix_overflow = true; - } - value.set(trace); - }); -} - -#[cfg(any(test, feature = "test-support"))] -fn record_scalar_state(owner: u16, fader: &FaderMuteRampBuiltins, matrix: &MatrixBuiltins) { - record_scalar_fader_state(owner, fader); - record_scalar_matrix_state(owner, matrix); -} - -#[cfg(any(test, feature = "test-support"))] -fn scalar_owner_id(track_id: &StableGraphId) -> u16 { - track_id - .as_str() - .strip_prefix('t') - .and_then(|value| value.parse().ok()) - .unwrap_or(u16::MAX) -} - // REALTIME_POLICY_BEGIN /// Drain the fader and mute cells of the lanes in `lanes` into `bank` under the block's `snapshot` /// (#1312 D4, D12): every lane for the block drain, the carried lanes for the plan-swap drain. @@ -7033,12 +6885,6 @@ impl PreparedBuiltinsSession { control, .. } = strip; - #[cfg(any(test, feature = "test-support"))] - let test_only_owner = scalar_owner_id(&graph_id); - #[cfg(all(test, feature = "test-support"))] - if control.is_some() { - record_initial_matrix_state(test_only_owner, &matrix); - } let (input_processor, fader_processor, matrix_processor): ( Box, Box, @@ -7082,8 +6928,6 @@ impl PreparedBuiltinsSession { .expect("a strip is banked on all three stages or on none"), control_delivery, automation, - #[cfg(any(test, feature = "test-support"))] - test_only_owner, }), Box::new(LiveControlMatrixProcessor { matrix, @@ -7091,8 +6935,6 @@ impl PreparedBuiltinsSession { .matrix .expect("a strip is banked on all three stages or on none"), control_delivery, - #[cfg(any(test, feature = "test-support"))] - test_only_owner, }), ) } @@ -9415,8 +9257,6 @@ struct LiveControlMatrixProcessor { matrix: MatrixBuiltins, control: MatrixReader, control_delivery: BuiltinControlDelivery, - #[cfg(any(test, feature = "test-support"))] - test_only_owner: u16, } #[cfg(any(test, feature = "test-support"))] impl Drop for LiveControlMatrixProcessor { @@ -9431,15 +9271,10 @@ impl Drop for LiveControlMatrixProcessor { #[cfg(any(test, feature = "test-support"))] impl GraphRuntimeProcessor for LiveControlMatrixProcessor { fn process(&mut self, block: GraphBindingBlock<'_>) -> Result<(), RenderError> { - self.drain_controls(block.live_snapshot).inspect_err(|_| { - #[cfg(any(test, feature = "test-support"))] - record_scalar_matrix_state(self.test_only_owner, &self.matrix); - })?; + self.drain_controls(block.live_snapshot)?; let block = DualMonoBlock::new(block.left, block.right, block.first_sample) .map_err(render_error)?; self.matrix.process(block); - #[cfg(any(test, feature = "test-support"))] - record_scalar_matrix_state(self.test_only_owner, &self.matrix); Ok(()) } } @@ -9482,8 +9317,6 @@ struct LiveControlFaderProcessor { control_delivery: BuiltinControlDelivery, /// The strip's stored fader automation (#1646 D1, R17-4): the bank's event driver, per strip. automation: Option>, - #[cfg(any(test, feature = "test-support"))] - test_only_owner: u16, } #[cfg(any(test, feature = "test-support"))] impl Drop for LiveControlFaderProcessor { @@ -9498,10 +9331,7 @@ impl Drop for LiveControlFaderProcessor { #[cfg(any(test, feature = "test-support"))] impl GraphRuntimeProcessor for LiveControlFaderProcessor { fn process(&mut self, block: GraphBindingBlock<'_>) -> Result<(), RenderError> { - self.drain_controls(block.live_snapshot).inspect_err(|_| { - #[cfg(any(test, feature = "test-support"))] - record_scalar_fader_state(self.test_only_owner, &self.fader); - })?; + self.drain_controls(block.live_snapshot)?; render_oracle_fader( &mut self.fader, self.automation.as_deref_mut(), @@ -9509,8 +9339,6 @@ impl GraphRuntimeProcessor for LiveControlFaderProcessor { block.right, block.first_sample, )?; - #[cfg(any(test, feature = "test-support"))] - record_scalar_fader_state(self.test_only_owner, &self.fader); Ok(()) } fn set_input_arrival(&mut self, arrival: u64) { @@ -9629,14 +9457,7 @@ impl GraphRuntimeProcessor for ScalarPairProcessor { first_sample, live_snapshot, } = block; - self.fader.drain_controls(live_snapshot).inspect_err(|_| { - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); - })?; + self.fader.drain_controls(live_snapshot)?; // Preserve the original fader boundary: its queue is drained before the envelope is // checked, and an invalid envelope stops before the later matrix owner consumes anything. let automated = self.fader.automation.is_some(); @@ -9658,12 +9479,6 @@ impl GraphRuntimeProcessor for ScalarPairProcessor { } else { self.fader.fader.process(fused_block); } - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); return Err(error); } // #1647 D1.5: an automated fader renders its events on the two-stage path. @@ -9674,12 +9489,6 @@ impl GraphRuntimeProcessor for ScalarPairProcessor { .process_fader_matrix(&mut self.matrix.matrix, &mut fused_block) }; if fused { - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); #[cfg(any(test, feature = "test-support"))] FADER_MATRIX_LIVE_WITNESS.with(|value| { let mut counters = value.get(); @@ -9712,12 +9521,6 @@ impl GraphRuntimeProcessor for ScalarPairProcessor { DualMonoBlock::new(&mut *left, &mut *right, first_sample).map_err(render_error)?, ); #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); - #[cfg(any(test, feature = "test-support"))] FADER_MATRIX_OUTPUT_WITNESS.with(|value| { value.set([ left.first().copied().unwrap_or(0.0).to_bits(), @@ -9775,16 +9578,7 @@ impl GraphRuntimeSplitPairProcessor for ScalarSplitPairProcessor { counters[4] = counters[4].saturating_add(1); value.set(counters); }); - self.fader - .drain_controls(block.live_snapshot) - .inspect_err(|_| { - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); - })?; + self.fader.drain_controls(block.live_snapshot)?; let GraphBindingBlock { left, right, @@ -9807,23 +9601,11 @@ impl GraphRuntimeSplitPairProcessor for ScalarSplitPairProcessor { let block = DualMonoBlock::new(left, right, first_sample).map_err(render_error)?; if self.fader.fader.is_settled() { self.pending_fader = true; - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); return Ok(()); } self.fader.fader.process(block); } self.pending_fader = false; - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); Ok(()) } @@ -9839,12 +9621,6 @@ impl GraphRuntimeSplitPairProcessor for ScalarSplitPairProcessor { self.fader.fader.process_pending(left, right); self.pending_fader = false; } - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); return Err(error); } let mut block = DualMonoBlock::new(left, right, first_sample).map_err(render_error)?; @@ -9863,12 +9639,6 @@ impl GraphRuntimeSplitPairProcessor for ScalarSplitPairProcessor { counters[1] = counters[1].saturating_add(1); value.set(counters); }); - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); return Ok(()); } if self.pending_fader { @@ -9884,12 +9654,6 @@ impl GraphRuntimeSplitPairProcessor for ScalarSplitPairProcessor { value.set(counters); }); self.matrix.matrix.process(block); - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); Ok(()) } @@ -9899,12 +9663,6 @@ impl GraphRuntimeSplitPairProcessor for ScalarSplitPairProcessor { } self.fader.fader.process_pending(block.left, block.right); self.pending_fader = false; - #[cfg(any(test, feature = "test-support"))] - record_scalar_state( - self.fader.test_only_owner, - &self.fader.fader, - &self.matrix.matrix, - ); } } @@ -11195,9 +10953,8 @@ mod tests { /// Builds the five-level graph the harness renders: one level per track stage. /// - /// `Output` is fed by one track's `PostMatrix` only (track 0's, or the last track's for - /// `NonadjacentOutputConflict`): `GraphEdgeId::TrackMain { target }` is not unique for fan-in, - /// and a reduction is not what this harness measures. + /// `Output` is fed by track 0's `PostMatrix` only: `GraphEdgeId::TrackMain { target }` is not + /// unique for fan-in, and a reduction is not what this harness measures. #[derive(Clone, Copy, Debug, Eq, PartialEq)] enum BoundaryVariant { Plain, @@ -11205,11 +10962,7 @@ mod tests { SelectedSend, Alias, AliasObserved, - #[cfg(all(test, feature = "test-support"))] - Nonadjacent, NonadjacentTrackA, - #[cfg(all(test, feature = "test-support"))] - NonadjacentOutputConflict, } fn track_graph(n: usize) -> (PreparedGraphPlan, Vec) { @@ -11292,16 +11045,8 @@ mod tests { } } let main_target = output.clone(); - // Issue #1420: only `NonadjacentOutputConflict` moves the output track, and the #916 test - // reads that edge. `NonadjacentTrackA` used to give `n - 1` too, but since #916 the - // `Output` is dedicated storage and no outcome read it, so it takes track 0. - #[cfg(all(test, feature = "test-support"))] - let output_track = if variant == BoundaryVariant::NonadjacentOutputConflict { - n - 1 - } else { - 0 - }; - #[cfg(not(all(test, feature = "test-support")))] + // Issue #1420: `NonadjacentTrackA` used to give `n - 1`, but since #916 the `Output` is + // dedicated storage and no outcome read it, so every variant takes track 0. let output_track = 0; edges.push(GraphEdge { id: GraphEdgeId::TrackMain { @@ -11985,12 +11730,7 @@ mod tests { ) } .expect("prepared builtins"); - let nonadjacent = match variant { - BoundaryVariant::NonadjacentTrackA => true, - #[cfg(all(test, feature = "test-support"))] - BoundaryVariant::Nonadjacent | BoundaryVariant::NonadjacentOutputConflict => true, - _ => false, - }; + let nonadjacent = variant == BoundaryVariant::NonadjacentTrackA; let (mut graph, mut levels) = track_graph_variant(n, variant); if backend == Backend::Scalar && n >= 2 { let stage = |index: usize, stage: TrackStage| GraphNodeId::TrackStage { @@ -12221,121 +11961,6 @@ mod tests { .map(|_| ()) } - #[cfg(feature = "test-support")] - fn render_failed_buffer_result( - bound: &mut PreparedBuiltinsGraphBound, - sample: u64, - completion_disabled: bool, - target: u32, - ) -> (RenderError, graph::TestOnlyFailedBufferCapture) { - graph::test_only_arm_failed_buffer_capture(target); - graph::test_only_set_completion_disabled(completion_disabled); - let mut pcm = vec![0.0_f32; HARNESS_QUANTUM as usize * 2]; - let error = render_bound_result(bound, sample, &mut pcm) - .expect_err("the failed-render fixture must return its injected error"); - graph::test_only_set_completion_disabled(false); - (error, graph::test_only_failed_buffer_capture()) - } - - fn first_fader_state(trace: TestOnlyScalarStateTrace, owner: u16) -> [u32; 14] { - trace.fader_owners[..trace.fader_len] - .iter() - .position(|candidate| *candidate == owner) - .map(|index| trace.fader_words[index]) - .expect("owner fader state trace") - } - - fn last_fader_state(trace: TestOnlyScalarStateTrace, owner: u16) -> [u32; 14] { - trace.fader_owners[..trace.fader_len] - .iter() - .rposition(|candidate| *candidate == owner) - .map(|index| trace.fader_words[index]) - .expect("owner fader state trace") - } - - fn first_matrix_state(trace: TestOnlyScalarStateTrace, owner: u16) -> [u32; 15] { - trace.matrix_owners[..trace.matrix_len] - .iter() - .position(|candidate| *candidate == owner) - .map(|index| trace.matrix_words[index]) - .expect("owner matrix state trace") - } - - fn last_matrix_state(trace: TestOnlyScalarStateTrace, owner: u16) -> [u32; 15] { - trace.matrix_owners[..trace.matrix_len] - .iter() - .rposition(|candidate| *candidate == owner) - .map(|index| trace.matrix_words[index]) - .expect("owner matrix state trace") - } - - /// Every owner that recorded a scalar state in either run recorded one in both, and its last - /// recorded fader and matrix words are equal. - fn assert_owner_states_match( - paired: TestOnlyScalarStateTrace, - separate: TestOnlyScalarStateTrace, - ) { - assert!( - !paired.fader_overflow - && !paired.matrix_overflow - && !separate.fader_overflow - && !separate.matrix_overflow, - "the scalar state trace overflowed" - ); - let owners = |trace: &[u16]| { - let mut owners = trace.to_vec(); - owners.sort_unstable(); - owners.dedup(); - owners - }; - let last = |owners: &[u16], words: &[[u32; 15]], owner: u16| { - owners - .iter() - .rposition(|candidate| *candidate == owner) - .map(|index| words[index]) - }; - let fader_owners = owners(&paired.fader_owners[..paired.fader_len]); - assert!( - !fader_owners.is_empty(), - "a scalar fader recorded its state" - ); - assert_eq!( - fader_owners, - owners(&separate.fader_owners[..separate.fader_len]), - "the same fader owners recorded" - ); - let matrix_owners = owners(&paired.matrix_owners[..paired.matrix_len]); - assert_eq!( - matrix_owners, - owners(&separate.matrix_owners[..separate.matrix_len]), - "the same matrix owners recorded" - ); - for owner in fader_owners { - let fader = |trace: &TestOnlyScalarStateTrace| { - trace.fader_owners[..trace.fader_len] - .iter() - .rposition(|candidate| *candidate == owner) - .map(|index| trace.fader_words[index]) - }; - assert_eq!(fader(&paired), fader(&separate), "fader owner {owner}"); - } - for owner in matrix_owners { - assert_eq!( - last( - &paired.matrix_owners[..paired.matrix_len], - &paired.matrix_words, - owner - ), - last( - &separate.matrix_owners[..separate.matrix_len], - &separate.matrix_words, - owner - ), - "matrix owner {owner}" - ); - } - } - /// A two-track graph whose strips carry fader-and-matrix channels only (`input_lane: false`, /// #1254 D1), prepared, lowered and bound at `backend`. fn input_free_pair_graph(backend: Backend) -> PreparedBuiltinsGraphBound { @@ -12463,719 +12088,6 @@ mod tests { ); } - #[cfg(all(test, feature = "test-support"))] - #[test] - fn actual_scalar_nonadjacent_intervening_observer_error_completes_and_retries() { - let _guard = PAIR_WITNESS_LOCK - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner); - let paired_capture = Arc::new(std::sync::Mutex::new(Vec::new())); - let separate_capture = Arc::new(std::sync::Mutex::new(Vec::new())); - test_only_reset_fader_matrix_witness(); - let mut paired = prepared_pair_graph_variant_observed( - false, - false, - false, - true, - Some(Arc::clone(&paired_capture)), - Backend::Scalar, - 2, - BoundaryVariant::Nonadjacent, - 1, - None, - false, - true, - ) - .0; - let paired_prepared = test_only_fader_matrix_witness(); - assert_eq!( - ( - paired_prepared.factory_calls, - paired_prepared.factory_members - ), - (1, 1) - ); - test_only_reset_fader_matrix_witness(); - let mut separate = prepared_pair_graph_variant_observed( - false, - false, - false, - false, - Some(Arc::clone(&separate_capture)), - Backend::Scalar, - 2, - BoundaryVariant::Nonadjacent, - 1, - None, - false, - true, - ) - .0; - let separate_prepared = test_only_fader_matrix_witness(); - assert_eq!( - ( - separate_prepared.factory_calls, - separate_prepared.factory_members - ), - (0, 0) - ); - let fader = TrackFaderRecord::FaderDb { - lanes: BuiltinLaneSelector::Both, - db: -6.0, - smoothing_samples: 0, - }; - let matrix = TrackControlRecord { - matrix: Matrix2x2 { - ll: 0.5, - lr: 0.25, - rl: -0.25, - rr: 0.75, - }, - smoothing_samples: 0, - }; - // Issue #916: the selected split pair is t00's, so its controls are the ones the failing - // observer must interrupt between `F0` and `M0`. - assert!(matches!( - paired.test_only_post_fader_node.as_ref(), - Some(GraphNodeId::TrackStage { - track_id, - stage: TrackStage::PostFader, - }) if track_id.as_str() == "t00" - )); - for bound in [&mut paired, &mut separate] { - bound.test_only_write(|controls, stamp| controls[0].fader.write(stamp, fader)); - bound.test_only_write(|controls, stamp| controls[0].producer.write(stamp, matrix)); - } - - INTERVENING_OBSERVER_ERRORS.store(0, Ordering::Relaxed); - test_only_reset_fader_matrix_witness(); - let mut paired_pcm = vec![0.0_f32; HARNESS_QUANTUM as usize * 2]; - let paired_error = render_bound_result(&mut paired, 0, &mut paired_pcm).unwrap_err(); - let paired_failed = test_only_fader_matrix_witness(); - let paired_failed_trace = test_only_scalar_state_trace(); - assert_eq!(paired_error, RenderError::InvalidEnvelope); - assert_eq!(INTERVENING_OBSERVER_ERRORS.load(Ordering::Relaxed), 1); - assert_eq!(paired_failed.fader_values_applied, 1); - assert_eq!(paired_failed.matrix_values_applied, 0); - - INTERVENING_OBSERVER_ERRORS.store(0, Ordering::Relaxed); - test_only_reset_fader_matrix_witness(); - let mut separate_pcm = vec![0.0_f32; HARNESS_QUANTUM as usize * 2]; - let separate_error = render_bound_result(&mut separate, 0, &mut separate_pcm).unwrap_err(); - let separate_failed = test_only_fader_matrix_witness(); - let separate_failed_trace = test_only_scalar_state_trace(); - assert_eq!(separate_error, paired_error); - assert_eq!(separate_failed.fader_values_applied, 1); - assert_eq!(separate_failed.matrix_values_applied, 0); - // Per owner, because the pending fader is completed after t01's separate fader ran, so - // the globally last-recorded word names a different owner in the two runs. - assert!(!paired_failed_trace.fader_overflow && !paired_failed_trace.matrix_overflow); - assert!(!separate_failed_trace.fader_overflow && !separate_failed_trace.matrix_overflow); - for owner in [0, 1] { - assert_eq!( - last_fader_state(paired_failed_trace, owner), - last_fader_state(separate_failed_trace, owner), - "observer error preserves fader {owner}'s state before retry" - ); - } - let initial_matrix = initial_matrix_state(0).expect("t00 initial matrix state"); - assert_eq!( - last_matrix_state(paired_failed_trace, 0), - initial_matrix, - "observer error leaves the later matrix command untouched" - ); - assert!( - !separate_failed_trace.matrix_owners[..separate_failed_trace.matrix_len].contains(&0), - "the separate twin never reached M0 either" - ); - - INTERVENING_OBSERVER_ERRORS.store(0, Ordering::Relaxed); - test_only_reset_fader_matrix_witness(); - paired_pcm.fill(0.0); - render_bound_result(&mut paired, HARNESS_QUANTUM as u64, &mut paired_pcm) - .expect("paired retry after observer error"); - let paired_retry = test_only_fader_matrix_witness(); - let paired_retry_trace = test_only_scalar_state_trace(); - INTERVENING_OBSERVER_ERRORS.store(0, Ordering::Relaxed); - test_only_reset_fader_matrix_witness(); - separate_pcm.fill(0.0); - render_bound_result(&mut separate, HARNESS_QUANTUM as u64, &mut separate_pcm) - .expect("separate retry after observer error"); - let separate_retry = test_only_fader_matrix_witness(); - let separate_retry_trace = test_only_scalar_state_trace(); - assert_eq!( - paired_pcm - .iter() - .map(|sample| sample.to_bits()) - .collect::>(), - separate_pcm - .iter() - .map(|sample| sample.to_bits()) - .collect::>(), - "retry renders the same PCM after the observer error" - ); - assert_eq!(paired_retry.matrix_values_applied, 1); - assert_eq!( - separate_retry.matrix_values_applied, 1, - "the separate twin drains the same queued matrix command on retry" - ); - assert_owner_states_match(paired_retry_trace, separate_retry_trace); - assert_eq!( - (paired_retry.fused_calls, paired_retry.fallback_calls), - (1, 0) - ); - assert_eq!( - std::mem::take(&mut *paired_capture.lock().unwrap()), - std::mem::take(&mut *separate_capture.lock().unwrap()) - ); - } - - #[cfg(all(test, feature = "test-support"))] - #[test] - fn actual_scalar_nonadjacent_failed_render_materializes_post_fader_before_error() { - let _guard = PAIR_WITNESS_LOCK - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner); - let fader = TrackFaderRecord::FaderDb { - lanes: BuiltinLaneSelector::Both, - db: -6.0, - smoothing_samples: 0, - }; - let matrix = Matrix2x2 { - ll: 0.5, - lr: 0.25, - rl: -0.25, - rr: 0.75, - }; - let make_observer_case = |between_render_calls| { - let mut bound = prepared_pair_graph_variant_observed( - false, - false, - false, - between_render_calls, - None, - Backend::Scalar, - 2, - BoundaryVariant::NonadjacentTrackA, - 1, - None, - false, - true, - ) - .0; - bound.test_only_write(|controls, stamp| controls[0].fader.write(stamp, fader)); - bound.test_only_write(|controls, stamp| { - controls[0].producer.write( - stamp, - TrackControlRecord { - matrix, - smoothing_samples: 0, - }, - ) - }); - bound - }; - - let mut paired = make_observer_case(true); - let mut separate = make_observer_case(false); - assert!(matches!( - paired.test_only_post_fader_node.as_ref(), - Some(GraphNodeId::TrackStage { - track_id, - stage: TrackStage::PostFader, - }) if track_id.as_str() == "t00" - )); - assert!(separate.test_only_post_fader_node.is_none()); - let selected_post_fader_buffer = paired - .test_only_post_fader_buffer - .expect("paired fixture has a selected post-fader buffer"); - INTERVENING_OBSERVER_ERRORS.store(0, Ordering::Relaxed); - test_only_reset_fader_matrix_witness(); - let (paired_error, paired_capture) = - render_failed_buffer_result(&mut paired, 0, false, selected_post_fader_buffer); - let paired_failed = test_only_fader_matrix_witness(); - let paired_trace = test_only_scalar_state_trace(); - INTERVENING_OBSERVER_ERRORS.store(0, Ordering::Relaxed); - test_only_reset_fader_matrix_witness(); - let (separate_error, separate_capture) = - render_failed_buffer_result(&mut separate, 0, false, selected_post_fader_buffer); - let separate_failed = test_only_fader_matrix_witness(); - let separate_trace = test_only_scalar_state_trace(); - assert!(!paired_trace.fader_overflow && !paired_trace.matrix_overflow); - assert!(!separate_trace.fader_overflow && !separate_trace.matrix_overflow); - assert_eq!(paired_error, separate_error); - assert!(paired_capture.captured && separate_capture.captured); - assert!(!paired_capture.overflow && !separate_capture.overflow); - assert_eq!(paired_capture, separate_capture); - assert_eq!( - first_fader_state(paired_trace, 0), - first_fader_state(separate_trace, 0), - "observer failure preserves t00 fader state at F_A" - ); - assert_eq!( - last_fader_state(paired_trace, 0), - first_fader_state(separate_trace, 0), - "observer failure completes t00 fader state before returning" - ); - let initial_matrix = initial_matrix_state(0).expect("t00 initial matrix state"); - assert_eq!( - first_matrix_state(paired_trace, 0), - initial_matrix, - "observer failure preserves t00 matrix state at M_A" - ); - assert_eq!( - last_matrix_state(paired_trace, 0), - initial_matrix, - "observer failure preserves t00 matrix state at completion" - ); - assert_eq!(paired_failed.matrix_values_applied, 0); - assert_eq!(separate_failed.matrix_values_applied, 0); - - // The same post-fader assertion must discriminate the executor's completion step. With - // completion disabled, the split owner leaves its deferred fader output behind while the - // separate owner has already materialized it. - let mut disabled = make_observer_case(true); - INTERVENING_OBSERVER_ERRORS.store(0, Ordering::Relaxed); - let (_, disabled_capture) = - render_failed_buffer_result(&mut disabled, 0, true, selected_post_fader_buffer); - assert_ne!( - disabled_capture, separate_capture, - "omitting pending completion must fail the post-fader buffer equality" - ); - - let make_matrix_case = |between_render_calls| { - let mut bound = prepared_pair_graph_variant( - false, - false, - false, - between_render_calls, - None, - Backend::Scalar, - 2, - BoundaryVariant::NonadjacentTrackA, - 1, - None, - ); - // One transaction: a valid fader move and a matrix target render refuses. The cell - // holds the refused target alone, since a later write replaces an earlier one. - bound.test_only_write(|controls, stamp| { - controls[0].fader.write(stamp, fader); - controls[0].producer.write( - stamp, - TrackControlRecord { - matrix: Matrix2x2 { - ll: f32::NAN, - ..matrix - }, - smoothing_samples: 0, - }, - ) - }); - bound - }; - let mut paired = make_matrix_case(true); - let mut separate = make_matrix_case(false); - assert!(matches!( - paired.test_only_post_fader_node.as_ref(), - Some(GraphNodeId::TrackStage { - track_id, - stage: TrackStage::PostFader, - }) if track_id.as_str() == "t00" - )); - assert!(separate.test_only_post_fader_node.is_none()); - let selected_post_fader_buffer = paired - .test_only_post_fader_buffer - .expect("paired fixture has a selected post-fader buffer"); - test_only_reset_fader_matrix_witness(); - let (paired_error, paired_capture) = - render_failed_buffer_result(&mut paired, 0, false, selected_post_fader_buffer); - let paired_failed = test_only_fader_matrix_witness(); - let paired_trace = test_only_scalar_state_trace(); - test_only_reset_fader_matrix_witness(); - let (separate_error, separate_capture) = - render_failed_buffer_result(&mut separate, 0, false, selected_post_fader_buffer); - let separate_failed = test_only_fader_matrix_witness(); - let separate_trace = test_only_scalar_state_trace(); - assert!(!paired_trace.fader_overflow && !paired_trace.matrix_overflow); - assert!(!separate_trace.fader_overflow && !separate_trace.matrix_overflow); - assert_eq!(paired_error, separate_error); - assert!(paired_capture.captured && separate_capture.captured); - assert_eq!(paired_capture, separate_capture); - assert_eq!( - first_fader_state(paired_trace, 0), - first_fader_state(separate_trace, 0), - "matrix failure preserves t00 fader state at F_A" - ); - assert_eq!( - last_fader_state(paired_trace, 0), - first_fader_state(separate_trace, 0), - "matrix failure completes t00 fader state before returning" - ); - let initial_matrix = initial_matrix_state(0).expect("t00 initial matrix state"); - assert_eq!( - first_matrix_state(paired_trace, 0), - initial_matrix, - "matrix failure preserves t00 matrix state at F_A" - ); - assert_eq!( - last_matrix_state(paired_trace, 0), - first_matrix_state(separate_trace, 0), - "matrix failure preserves t00 matrix state at M_A/error completion" - ); - assert_eq!(paired_failed.fader_values_applied, 1); - assert_eq!(paired_failed.matrix_values_applied, 1); - assert_eq!( - paired_failed.fader_values_applied, - separate_failed.fader_values_applied - ); - assert_eq!( - paired_failed.matrix_values_applied, - separate_failed.matrix_values_applied - ); - } - - #[cfg(all(test, feature = "test-support"))] - #[test] - fn actual_scalar_nonadjacent_ramp_retarget_and_failed_retry_stay_at_original_boundaries() { - let _guard = PAIR_WITNESS_LOCK - .lock() - .unwrap_or_else(std::sync::PoisonError::into_inner); - let matrix_a = Matrix2x2 { - ll: 0.5, - lr: 0.25, - rl: -0.25, - rr: 0.75, - }; - let matrix_b = Matrix2x2 { - ll: 0.75, - lr: -0.5, - rl: 0.125, - rr: 0.625, - }; - - // The selected production fixture is t00's genuinely nonadjacent F_A -> F_B -> M_A -> - // M_B interval. t00's post-matrix capture observes the paired owner itself, beside the - // graph's final PCM. - let make_pair = || { - let paired_capture = Arc::new(std::sync::Mutex::new(Vec::new())); - let separate_capture = Arc::new(std::sync::Mutex::new(Vec::new())); - let paired = prepared_pair_graph_variant( - false, - false, - false, - true, - Some(Arc::clone(&paired_capture)), - Backend::Scalar, - 2, - BoundaryVariant::NonadjacentTrackA, - 1, - Some((0, Arc::clone(&paired_capture))), - ); - let separate = prepared_pair_graph_variant( - false, - false, - false, - false, - Some(Arc::clone(&separate_capture)), - Backend::Scalar, - 2, - BoundaryVariant::NonadjacentTrackA, - 1, - Some((0, Arc::clone(&separate_capture))), - ); - assert_eq!( - ( - test_only_fader_matrix_witness().factory_calls, - test_only_fader_matrix_witness().factory_members - ), - (1, 1), - "paired fixture selects the t00 split owner" - ); - test_only_reset_fader_matrix_witness(); - (paired, separate, paired_capture, separate_capture) - }; - - let render_and_compare = |paired: &mut PreparedBuiltinsGraphBound, - separate: &mut PreparedBuiltinsGraphBound, - paired_capture: &Arc>>, - separate_capture: &Arc>>, - sample: u64| { - let mut paired_pcm = vec![0.0_f32; HARNESS_QUANTUM as usize * 2]; - let mut separate_pcm = vec![0.0_f32; HARNESS_QUANTUM as usize * 2]; - test_only_reset_fader_matrix_witness(); - render_bound_result(paired, sample, &mut paired_pcm).expect("paired render"); - let paired_witness = test_only_fader_matrix_witness(); - let paired_trace = test_only_scalar_state_trace(); - test_only_reset_fader_matrix_witness(); - render_bound_result(separate, sample, &mut separate_pcm).expect("separate render"); - let separate_witness = test_only_fader_matrix_witness(); - let separate_trace = test_only_scalar_state_trace(); - assert!(!paired_trace.fader_overflow && !paired_trace.matrix_overflow); - assert!(!separate_trace.fader_overflow && !separate_trace.matrix_overflow); - assert_eq!( - std::mem::take(&mut *paired_capture.lock().unwrap()), - std::mem::take(&mut *separate_capture.lock().unwrap()), - "paired and separate t00/t01 post-matrix PCM" - ); - assert_eq!( - paired_pcm - .iter() - .map(|sample| sample.to_bits()) - .collect::>(), - separate_pcm - .iter() - .map(|sample| sample.to_bits()) - .collect::>(), - "paired and separate output PCM" - ); - assert_eq!( - last_fader_state(paired_trace, 0), - last_fader_state(separate_trace, 0), - "t00 fader state at its original boundary" - ); - assert_eq!( - last_matrix_state(paired_trace, 0), - last_matrix_state(separate_trace, 0), - "t00 matrix state at its original boundary" - ); - ( - paired_witness, - separate_witness, - paired_trace, - separate_trace, - ) - }; - - // A ramping fader must execute at F_A. The split owner cannot defer this block, so the - // matrix boundary uses the original two-stage fallback and leaves no pending fader. - let (mut paired, mut separate, paired_capture, separate_capture) = make_pair(); - for bound in [&mut paired, &mut separate] { - bound.test_only_write(|controls, stamp| { - controls[0].fader.write( - stamp, - TrackFaderRecord::FaderDb { - lanes: BuiltinLaneSelector::Both, - db: -6.0, - smoothing_samples: 3, - }, - ) - }); - } - let (paired_ramp, separate_ramp, paired_trace, separate_trace) = render_and_compare( - &mut paired, - &mut separate, - &paired_capture, - &separate_capture, - 0, - ); - assert_eq!( - (paired_ramp.process_calls, paired_ramp.process_members), - (1, 1) - ); - assert_eq!( - (paired_ramp.fused_calls, paired_ramp.fallback_calls), - (0, 1) - ); - assert_eq!( - ( - paired_ramp.fader_values_applied, - paired_ramp.matrix_values_applied - ), - (1, 0) - ); - assert_eq!( - first_fader_state(paired_trace, 0), - last_fader_state(paired_trace, 0), - "ramping fader state is materialized at F_A before M_A" - ); - assert_eq!( - first_fader_state(paired_trace, 0), - first_fader_state(separate_trace, 0) - ); - assert_eq!( - paired_ramp.fader_values_applied, - separate_ramp.fader_values_applied - ); - - // A settled fader plus a ramping matrix must defer only the fader and fall back at M_A. - let (mut paired, mut separate, paired_capture, separate_capture) = make_pair(); - for bound in [&mut paired, &mut separate] { - bound.test_only_write(|controls, stamp| { - controls[0].fader.write( - stamp, - TrackFaderRecord::FaderDb { - lanes: BuiltinLaneSelector::Both, - db: -6.0, - smoothing_samples: 0, - }, - ) - }); - bound.test_only_write(|controls, stamp| { - controls[0].producer.write( - stamp, - TrackControlRecord { - matrix: matrix_a, - smoothing_samples: 96, - }, - ) - }); - } - let (paired_matrix_ramp, separate_matrix_ramp, _, _) = render_and_compare( - &mut paired, - &mut separate, - &paired_capture, - &separate_capture, - 0, - ); - assert_eq!( - ( - paired_matrix_ramp.fused_calls, - paired_matrix_ramp.fallback_calls - ), - (0, 1) - ); - assert_eq!( - ( - paired_matrix_ramp.fader_values_applied, - paired_matrix_ramp.matrix_values_applied - ), - (1, 1) - ); - assert_eq!( - paired_matrix_ramp.matrix_values_applied, - separate_matrix_ramp.matrix_values_applied - ); - - // Retargeting the matrix while its prior ramp is in flight must still run at M_A. The - // second block also proves that the split owner clears its pending flag on success. - for bound in [&mut paired, &mut separate] { - bound.test_only_write(|controls, stamp| { - controls[0].producer.write( - stamp, - TrackControlRecord { - matrix: matrix_b, - smoothing_samples: 3, - }, - ) - }); - } - let (paired_retarget, separate_retarget, _, _) = render_and_compare( - &mut paired, - &mut separate, - &paired_capture, - &separate_capture, - HARNESS_QUANTUM as u64, - ); - assert_eq!( - (paired_retarget.fused_calls, paired_retarget.fallback_calls), - (0, 1) - ); - assert_eq!( - paired_retarget.matrix_values_applied, - separate_retarget.matrix_values_applied - ); - - // A failed M_A render must complete the pending fader before returning and retry exactly - // once with the next write. A cell holds the newest value only, so the malformed target - // is the one the failed block reads, and the tail is written after it. - let (mut paired, mut separate, paired_capture, separate_capture) = make_pair(); - let invalid_matrix = TrackControlRecord { - matrix: Matrix2x2 { - ll: f32::NAN, - ..matrix_a - }, - smoothing_samples: 0, - }; - let matrix_tail = TrackControlRecord { - matrix: Matrix2x2::IDENTITY, - smoothing_samples: 0, - }; - for bound in [&mut paired, &mut separate] { - bound.test_only_write(|controls, stamp| { - controls[0].fader.write( - stamp, - TrackFaderRecord::FaderDb { - lanes: BuiltinLaneSelector::Both, - db: -6.0, - smoothing_samples: 0, - }, - ) - }); - bound.test_only_write(|controls, stamp| { - controls[0].producer.write(stamp, invalid_matrix) - }); - } - let selected_buffer = paired - .test_only_post_fader_buffer - .expect("paired fixture has t00 post-fader buffer"); - test_only_reset_fader_matrix_witness(); - let (paired_error, paired_failed_capture) = - render_failed_buffer_result(&mut paired, 0, false, selected_buffer); - let paired_failed = test_only_fader_matrix_witness(); - let paired_failed_trace = test_only_scalar_state_trace(); - test_only_reset_fader_matrix_witness(); - let (separate_error, separate_failed_capture) = - render_failed_buffer_result(&mut separate, 0, false, selected_buffer); - let separate_failed = test_only_fader_matrix_witness(); - let separate_failed_trace = test_only_scalar_state_trace(); - assert_eq!(paired_error, separate_error); - assert_eq!(paired_failed_capture, separate_failed_capture); - assert_eq!( - ( - paired_failed.fader_values_applied, - paired_failed.matrix_values_applied - ), - (1, 1) - ); - assert_eq!( - ( - paired_failed.fader_values_applied, - paired_failed.matrix_values_applied - ), - ( - separate_failed.fader_values_applied, - separate_failed.matrix_values_applied - ) - ); - assert_eq!( - last_fader_state(paired_failed_trace, 0), - first_fader_state(separate_failed_trace, 0), - "failed M_A render completes t00 fader before returning" - ); - assert_eq!( - last_matrix_state(paired_failed_trace, 0), - last_matrix_state(separate_failed_trace, 0), - "failed M_A render leaves the matrix state alone" - ); - - for bound in [&mut paired, &mut separate] { - bound.test_only_write(|controls, stamp| controls[0].producer.write(stamp, matrix_tail)); - } - let (paired_retry, separate_retry, _, _) = render_and_compare( - &mut paired, - &mut separate, - &paired_capture, - &separate_capture, - HARNESS_QUANTUM as u64, - ); - assert_eq!( - (paired_retry.fused_calls, paired_retry.fallback_calls), - (1, 0) - ); - assert_eq!(paired_retry.matrix_values_applied, 1); - assert_eq!( - ( - paired_retry.fader_values_applied, - paired_retry.matrix_values_applied - ), - ( - separate_retry.fader_values_applied, - separate_retry.matrix_values_applied - ), - "retry applies only the write after the error, once" - ); - } - #[test] fn actual_graph_mono_collapse_disengages_on_input_command_and_recovers_nonfinite_input() { let _guard = PAIR_WITNESS_LOCK diff --git a/crates/graph-compiler/src/lib.rs b/crates/graph-compiler/src/lib.rs index b0fa600cf..3460cd9f0 100644 --- a/crates/graph-compiler/src/lib.rs +++ b/crates/graph-compiler/src/lib.rs @@ -382,13 +382,6 @@ mod tests { (index, nodes) } - fn literal_bank_slot_reservation_bytes(bank_count: u64, lanes: u64) -> u64 { - let stage = core::mem::size_of::>() as u64; - let slot = core::mem::size_of::() as u64; - let mask = lanes * core::mem::size_of::() as u64; - bank_count * (stage + 3 * slot + 3 * mask) - } - /// Track stages the builtins compiler banks, one bank each per cohort. /// /// Three since issue #212: the post-input builtin stage, the fader, and the pan matrix. It was @@ -12411,7 +12404,12 @@ mod tests { assert_eq!(scalar_artifact.graph().prepared_bank_count(), 0); let lanes = width.map_or(0_u64, |width| u64::from(width.lanes())); let bank_count = u64::try_from(expected_banks).expect("bank count"); - let expected_bank_slot_bytes = literal_bank_slot_reservation_bytes(bank_count, lanes); + // The product's own slot reservation for these banks at this width (TA3): the + // deltas below hold it to the reports, without re-deriving its layout. + let expected_bank_slot_bytes = + graph::GraphBankSlotResourceEstimate::checked_for(bank_count, width) + .expect("bank slot reservation") + .total_bytes; let quantum = u64::from(session.quantum().0); let expected_bank_scratch_bytes = bank_count * lanes * quantum * 2 * 4; let expected_bank_runtime_buffer_bytes = bank_count * lanes * quantum * 2 * 4; @@ -12821,7 +12819,12 @@ mod tests { let lanes = width.map_or(0_u64, |width| u64::from(width.lanes())); let bank_count = u64::try_from(expected_banks).expect("bank count"); - let expected_bank_slot_bytes = literal_bank_slot_reservation_bytes(bank_count, lanes); + // The product's own slot reservation for these banks at this width (TA3): the + // deltas below hold it to the reports, without re-deriving its layout. + let expected_bank_slot_bytes = + graph::GraphBankSlotResourceEstimate::checked_for(bank_count, width) + .expect("bank slot reservation") + .total_bytes; let quantum = u64::from(session.quantum().0); let expected_bank_scratch_bytes = bank_count * lanes * quantum * 2 * 4; let expected_bank_runtime_buffer_bytes = bank_count * lanes * quantum * 2 * 4; @@ -13178,7 +13181,12 @@ mod tests { let lanes = width.map_or(0_u64, |width| u64::from(width.lanes())); let bank_count = u64::try_from(expected_banks).expect("bank count"); - let expected_bank_slot_bytes = literal_bank_slot_reservation_bytes(bank_count, lanes); + // The product's own slot reservation for these banks at this width (TA3): the + // deltas below hold it to the reports, without re-deriving its layout. + let expected_bank_slot_bytes = + graph::GraphBankSlotResourceEstimate::checked_for(bank_count, width) + .expect("bank slot reservation") + .total_bytes; let quantum = u64::from(session.quantum().0); let expected_bank_scratch_bytes = bank_count * lanes * quantum * 2 * 4; let expected_bank_runtime_buffer_bytes = bank_count * lanes * quantum * 2 * 4; @@ -13546,7 +13554,12 @@ mod tests { let lanes = width.map_or(0_u64, |width| u64::from(width.lanes())); let bank_count = u64::try_from(expected_banks).expect("bank count"); - let expected_bank_slot_bytes = literal_bank_slot_reservation_bytes(bank_count, lanes); + // The product's own slot reservation for these banks at this width (TA3): the + // deltas below hold it to the reports, without re-deriving its layout. + let expected_bank_slot_bytes = + graph::GraphBankSlotResourceEstimate::checked_for(bank_count, width) + .expect("bank slot reservation") + .total_bytes; let quantum = u64::from(session.quantum().0); let expected_bank_scratch_bytes = bank_count * lanes * quantum * 2 * 4; let expected_bank_runtime_buffer_bytes = bank_count * lanes * quantum * 2 * 4; diff --git a/crates/graph/Cargo.toml b/crates/graph/Cargo.toml index 3d9255b82..9849dc861 100644 --- a/crates/graph/Cargo.toml +++ b/crates/graph/Cargo.toml @@ -10,7 +10,7 @@ publish.workspace = true name = "graph" [features] -test-support = ["rack/test-support"] +test-support = [] [dependencies] # The route op of a following send evaluates its source strip's stored mute curves with the diff --git a/crates/graph/src/lib.rs b/crates/graph/src/lib.rs index b320b40ba..c2f665239 100644 --- a/crates/graph/src/lib.rs +++ b/crates/graph/src/lib.rs @@ -35,179 +35,6 @@ pub use runtime::{ test_only_source_plane_reset, test_only_split_pair_table_witness, }; -/// Phase-level timing of `GraphExecutor::render`: the instrument of the gain/pan phase profile -/// (issue #960, `tools/console-workload/tests/gain_pan_profile.rs`), first built for the -/// plumbing-floor diagnosis. Test builds only: nothing here exists in a production build, and a -/// `test-support` build that never calls [`test_only_phase_profile::enable`] pays one -/// thread-local read per block and nothing per unit (plus, with the feature, one per bank-chain -/// probe site). -/// -/// The clock is `std::time::Instant` (the vDSO monotonic clock), because `crates/graph` may not -/// carry the `unsafe` block `_rdtsc` needs. A probe is taken only where the *kind* of unit -/// changes, so a block whose units are grouped by kind costs a handful of probes rather than one -/// per unit; the harness measures the probe cost and states it beside the numbers. -/// -/// A bank unit is one [`BANK`](test_only_phase_profile::BANK) phase here. With the -/// `test-support` feature, [`bank`](test_only_phase_profile::bank) splits the chain inside it -/// into the gather, each slot, the collapse seam, the scatter and the route/master fold (issue -/// #960); [`enable`](test_only_phase_profile::enable) and -/// [`reset`](test_only_phase_profile::reset) drive both profiles together. What `BANK` holds -/// beyond the chain's sub-phases is the unit's dispatch, member setup and observation. -#[cfg(any(test, feature = "test-support"))] -#[doc(hidden)] -pub mod test_only_phase_profile { - use core::cell::Cell; - use std::time::Instant; - - /// Render entry to the first source work: the host planes' shape check. - pub const ENTER: usize = 0; - /// The source set's `begin_block` and the `copy_track_input` loop. Empty when the plan binds - /// no source set, as the bound-feed console rows do; on `sixty_four_track_gain_pan_ring` it is - /// the driver's `begin_block` alone, because every claim is read in place (issue #918). - pub const SOURCE: usize = 1; - /// Plain units whose op is a host-bound processor (`NodeKind::Bound`): on the bound-feed - /// console rows, the sixty-four `FrozenGraphSource` copies. - pub const BOUND: usize = 2; - /// Plain units whose op is a route (`NodeKind::Route`). - pub const ROUTE: usize = 3; - /// The session Output op's unit. - pub const OUTPUT: usize = 4; - /// A plain identity unit with one input that is not in place: `reduce_plane`'s copy arm. - pub const IDENTITY_COPY: usize = 5; - /// A plain identity unit in place over its one input: dispatch and nothing else. - pub const IDENTITY_ALIAS: usize = 6; - /// Any other plain unit. - pub const OTHER_OP: usize = 7; - /// A bank unit: its dispatch, member setup and observation, and the whole chain run that - /// [`bank`] splits. - pub const BANK: usize = 8; - /// The unit loop's end to the return (on the pre-#916 tree, the end-of-block master copy). - pub const EXIT: usize = 9; - pub const COUNT: usize = 10; - - thread_local! { - static ENABLED: Cell = const { Cell::new(false) }; - static NANOS: Cell<[u64; COUNT]> = const { Cell::new([0; COUNT]) }; - static BLOCKS: Cell = const { Cell::new(0) }; - static PROBES: Cell = const { Cell::new(0) }; - static RUNS: Cell<[(usize, u64); RUNS_CAPACITY]> = - const { Cell::new([(COUNT, 0); RUNS_CAPACITY]) }; - } - - /// Runs of same-kind units recorded from the first profiled block, `(kind, units)`. - pub const RUNS_CAPACITY: usize = 16; - - /// The unit-kind runs of the first block profiled since the last reset, in schedule order; - /// `(COUNT, 0)` marks unused entries. - #[must_use] - pub fn runs() -> [(usize, u64); RUNS_CAPACITY] { - RUNS.with(Cell::get) - } - - /// The bank chain's sub-phase profile (issue #960): the gather, each slot, the collapse seam, - /// the scatter and the route/master fold inside every [`BANK`] unit. - #[cfg(feature = "test-support")] - pub use rack::test_only_bank_phase_profile as bank; - - /// Switch the probes on or off for this thread, the bank chain's sub-phase probes included. - /// Off, `render` reads one thread-local per block and nothing else, and a bank chain one per - /// probe site. - pub fn enable(enabled: bool) { - ENABLED.with(|value| value.set(enabled)); - #[cfg(feature = "test-support")] - bank::enable(enabled); - } - - /// Zero the accumulators, the bank chain's sub-phase totals included. - pub fn reset() { - NANOS.with(|value| value.set([0; COUNT])); - BLOCKS.with(|value| value.set(0)); - PROBES.with(|value| value.set(0)); - RUNS.with(|value| value.set([(COUNT, 0); RUNS_CAPACITY])); - #[cfg(feature = "test-support")] - bank::reset(); - } - - /// `(nanoseconds per phase, blocks profiled, probes taken)` since the last reset. - #[must_use] - pub fn snapshot() -> ([u64; COUNT], u64, u64) { - ( - NANOS.with(Cell::get), - BLOCKS.with(Cell::get), - PROBES.with(Cell::get), - ) - } - - /// One block's running probe. `None` when disabled, so the hot path is one `Option` test. - pub(crate) struct Probe { - last: Instant, - phase: usize, - nanos: [u64; COUNT], - probes: u64, - runs: [(usize, u64); RUNS_CAPACITY], - run: usize, - } - - impl Probe { - #[inline] - pub(crate) fn start() -> Option { - ENABLED.with(Cell::get).then(|| Self { - last: Instant::now(), - phase: ENTER, - nanos: [0; COUNT], - probes: 1, - runs: [(COUNT, 0); RUNS_CAPACITY], - run: 0, - }) - } - - /// Charge the time since the last probe to the current phase and make `next` current. - #[inline] - pub(crate) fn enter(&mut self, next: usize) { - let now = Instant::now(); - self.nanos[self.phase] += - u64::try_from(now.duration_since(self.last).as_nanos()).unwrap_or(u64::MAX); - self.last = now; - self.phase = next; - self.probes += 1; - } - - /// Charge a unit of kind `kind`; a probe is taken only when the kind changes. - #[inline] - pub(crate) fn unit(&mut self, kind: usize) { - if kind != self.phase { - self.enter(kind); - if self.run < RUNS_CAPACITY && self.runs[self.run].1 != 0 { - self.run += 1; - } - if self.run < RUNS_CAPACITY { - self.runs[self.run].0 = kind; - } - } - if self.run < RUNS_CAPACITY { - self.runs[self.run].1 += 1; - } - } - - #[inline] - pub(crate) fn finish(mut self) { - self.enter(COUNT - 1); - NANOS.with(|value| { - let mut total = value.get(); - for (slot, nanos) in total.iter_mut().zip(self.nanos) { - *slot += nanos; - } - value.set(total); - }); - if BLOCKS.with(Cell::get) == 0 { - RUNS.with(|value| value.set(self.runs)); - } - BLOCKS.with(|value| value.set(value.get() + 1)); - PROBES.with(|value| value.set(value.get() + self.probes)); - } - } -} - use core::cell::Cell; use std::any::Any; use std::collections::{BTreeMap, BTreeSet}; @@ -5225,6 +5052,26 @@ pub fn test_only_timeline_at( graph_source_driver_mut(plan)?.timeline_at(source_read_sample) } +/// The entries of the plan's carry tables that [`carry_program_retained_bytes`] charges, as +/// counts: `[source moves, input-section lane moves, strip-stage moves, effect-lane moves]`. All +/// zero for a plan without a carry program or for any other plan. +#[cfg(any(test, feature = "test-support"))] +pub fn test_only_carry_counts(plan: &mut PreparedRenderPlan) -> [usize; 4] { + plan.executor_any_mut() + .and_then(|any| any.downcast_mut::()) + .map_or([0; 4], |executor| { + [ + executor + .carry + .as_ref() + .map_or(0, |program| program.sources.len()), + executor.carry_inputs.len(), + executor.carry_strips.len(), + executor.carry_effects.len(), + ] + }) +} + /// Bytes the plan's installed [`GraphCarryProgram`] retains; zero for a plan without one or for /// any other plan. pub fn carry_program_retained_bytes(plan: &mut PreparedRenderPlan) -> u64 { @@ -5876,16 +5723,10 @@ impl PreparedPlanExecutor for GraphExecutor { *next_block_sample = time .absolute_sample .saturating_add(runtime.arena.frames() as u64); - #[cfg(any(test, feature = "test-support"))] - let mut probe = test_only_phase_profile::Probe::start(); let (left, right) = output.stereo_planes_mut()?; let Some(mut host) = runtime::HostMaster::new(left, right, runtime.arena.frames()) else { return Err(RenderError::InvalidEnvelope); }; - #[cfg(any(test, feature = "test-support"))] - if let Some(probe) = probe.as_mut() { - probe.enter(test_only_phase_profile::SOURCE); - } // #1396 D1: the source-read sample, `ΣP` ahead of the render clock. let block_source_read_sample = time.absolute_sample.checked_add(*source_read_offset); let source_validity = if let Some(source_set) = source_set.as_mut() { @@ -5953,10 +5794,6 @@ impl PreparedPlanExecutor for GraphExecutor { { dispatched += 1; } - #[cfg(any(test, feature = "test-support"))] - if let Some(probe) = probe.as_mut() { - probe.unit(runtime.test_only_unit_phase(unit)); - } if let Err(error) = runtime.execute( unit, time.absolute_sample, @@ -5998,10 +5835,6 @@ impl PreparedPlanExecutor for GraphExecutor { // Issue #1609: the block rendered without failure, so a result an observer held for it // may now be published. runtime.block_end_observers(); - #[cfg(any(test, feature = "test-support"))] - if let Some(probe) = probe { - probe.finish(); - } Ok(()) } // REALTIME_POLICY_END diff --git a/crates/graph/src/runtime.rs b/crates/graph/src/runtime.rs index 099d210d3..da6507dbe 100644 --- a/crates/graph/src/runtime.rs +++ b/crates/graph/src/runtime.rs @@ -4579,35 +4579,6 @@ impl Runtime { TEST_ONLY_FAILED_BUFFER.with(|value| value.set(capture)); } - /// The phase a unit's time is charged to by `crate::test_only_phase_profile`. - #[cfg(any(test, feature = "test-support"))] - #[inline] - pub(crate) fn test_only_unit_phase(&self, index: usize) -> usize { - use crate::test_only_phase_profile::{ - BANK, BOUND, IDENTITY_ALIAS, IDENTITY_COPY, OTHER_OP, OUTPUT, ROUTE, - }; - if self.output_unit == Some(index) { - return OUTPUT; - } - match &self.units[index] { - RuntimeUnit::Op(op) => match (&op.kind, &*op.inputs) { - (NodeKind::Bound(_), _) => BOUND, - (NodeKind::Route(_) | NodeKind::LiveRoute(_) | NodeKind::LiveFollowRoute(_), _) => { - ROUTE - } - (NodeKind::Identity, [single]) if op.split_pair.is_none() => { - if *single == op.output { - IDENTITY_ALIAS - } else { - IDENTITY_COPY - } - } - _ => OTHER_OP, - }, - RuntimeUnit::Bank { .. } => BANK, - } - } - /// Runs unit `index`. Every producer this unit reads precedes it in `units`, or was written /// by a strictly earlier wave. /// diff --git a/crates/graph/tests/MUTATIONS.md b/crates/graph/tests/MUTATIONS.md index be1b11569..b249ae64f 100644 --- a/crates/graph/tests/MUTATIONS.md +++ b/crates/graph/tests/MUTATIONS.md @@ -750,3 +750,12 @@ deleted or trimmed some tests that rows above name. The rows stay as history; th no longer re-derives the charge from `size_of`; it keeps the layout-witness, fold and overflow assertions. console-workload "gate 5" (`the_driver_fed_rows_metadata_charge_grows_by_exactly_the_executor_tables`) is deleted. + +## TA3 policy cut (2026-10-19): further tests these rows name that no longer exist + +The second owner-approved policy cut (`.github/ISSUE_SPECS/1726-finish-the-test-policy-cut.md`) +deleted the last three builtins-compiler scalar-oracle pairing tests. The rows stay as history. + +- 916-16 and 916-17: `actual_scalar_nonadjacent_failed_render_materializes_post_fader_before_error` + and `actual_scalar_nonadjacent_ramp_retarget_and_failed_retry_stay_at_original_boundaries` are + deleted. The allocation_tracker tests these rows name remain. diff --git a/crates/host-core/src/limiter_linked_session.rs b/crates/host-core/src/limiter_linked_session.rs index 70e265da8..f05628bc1 100644 --- a/crates/host-core/src/limiter_linked_session.rs +++ b/crates/host-core/src/limiter_linked_session.rs @@ -1,9 +1,8 @@ -//! The single cross-target owner of the 64-track linked-limiter session corpus (#996/#997). -//! Non-repeating hot noise exercises linked ramps, gain-ring mirroring, one-sided unlinks and -//! later equal designed words at every available width (`Simd8` with `avx2`, `Simd4`, `Scalar`). -//! Its shared master/observation pin detects common scalar/vector mirroring and premature-relink -//! defects that comparing the current widths alone cannot detect. The pin's pre-#990 provenance -//! is recorded below; this is a current state-transition contract, not a timing witness. +//! The 64-track linked-limiter session (#996/#997). Non-repeating hot noise exercises linked +//! ramps, gain-ring mirroring, one-sided unlinks and later equal designed words at every available +//! width (`Simd8` with `avx2`, `Simd4`, `Scalar`), through the host's own preparation and live +//! control channels. It checks the session-visible state transitions; the rendered words are +//! owned by gate G5's `true-peak-limiter` case `maximum/link_unlink_relink` (see "What is pinned"). //! //! # Why this test exists //! @@ -100,16 +99,16 @@ //! the right channel's window is still answered from the previous van Herk block, which is where //! a stale right ring gives a wrong minimum. //! -//! # What is pinned, and how +//! # What is pinned, and where //! -//! One SHA-256 over, after every block, the master's 256 output words and every limiter's -//! published window (its sequence number and both channels' reduction words). It was recorded on -//! the **pre-#990 kernel**, which never links: this tree with `crates/true-peak-limiter` put back -//! to its text at `bbcf8ce1`. `Simd8`, `Simd4` and `Scalar` rendered the same digest there, in -//! dev and in release, and so does the batch head. One pin serves every width because banking -//! regroups lanes and never changes a lane's arithmetic. +//! Nothing here. This test used to pin one SHA-256 over every block's master words and limiter +//! readings, recorded on the pre-#990 kernel. TA3 (2026-10-19) moved that claim to gate G5: +//! `crates/true-peak-limiter/src/corpus.rs` case 5, `maximum/link_unlink_relink`, drives the +//! prepared core through a linked ramp, one-sided unlinks on a bank's edge lanes and re-equalled +//! pairs, and its pin, from the scalar oracle, equals the words a never-linking kernel renders. +//! G5 compares it at every width, natively and under wasmtime. //! -//! Four checks run ahead of the pin, and all four held on the pre-#990 kernel: +//! Four checks run here, and all four held on the pre-#990 kernel: //! //! * every lane of a bank carries the retarget that unlinks some bank (above); //! * every limiter is limiting: from block 3, more than 6 dB of reduction on both channels at @@ -135,11 +134,10 @@ //! rest of its plan. A cheaper proof (for example at silent rest, the #990 verification's L2) //! would be class A and is out of scope here. //! -//! What this test asserts about it: a session cannot see *whether* a pair relinked, only the -//! words it rendered. So it asserts that every block after a pair is equal again renders the -//! pre-#990 words. A kernel that relinks on designed agreement alone (row 996-K2 of -//! `crates/true-peak-limiter/tests/MUTATIONS.md`) moves the digest. The relink decision itself is -//! asserted inside the crate, where the kernel's choice is visible +//! A session cannot see *whether* a pair relinked, only the words it rendered, and this test no +//! longer pins words. A kernel that relinks on designed agreement alone (row 996-K2 of +//! `crates/true-peak-limiter/tests/MUTATIONS.md`) moves the G5 case `maximum/link_unlink_relink`. +//! The relink decision itself is asserted inside the crate, where the kernel's choice is visible //! (`the_linked_body_engages_exactly_where_the_record_allows`, #990 gate 2). //! //! # Width @@ -148,12 +146,11 @@ //! the `#[cfg(test)]` seam that is `prepare_host_runtime_with_live_controls` at a named backend. It //! renders W8/W4/Scalar in CI's x86-64-v3 debug and release package runs, and W4/Scalar in the //! AArch64 product debug run. AArch64 release's package subset does not run host-core. The Wasm -//! guest does not contain this corpus: it plays one frozen block per track, and a per-block source -//! there is tooling (the #996 verdict's owner ruling). +//! guest does not run this session (a per-block source there is tooling, the #996 verdict's owner +//! ruling); the G5 case named above carries the linked pair's words to every target instead. use core::ops::RangeInclusive; -use bench_support::digest::Sha256Sink; use effect_contract::{EffectControlRecord, ParameterChannel}; use engine::realtime::{PlanarBufferMut, RenderIo, RenderTime}; use graph_compiler::Backend; @@ -190,9 +187,6 @@ const REDUCTION_FLOOR: f32 = 0.5; /// Blocks, from [`LIMITING_FROM`], at whose end each limiter's reduction must have risen. const MINIMUM_RISES: usize = 4; -/// SHA-256 of the scenario, recorded on the pre-#990 kernel at every width (see the module note). -const PIN: &str = "6d87267b7502a4b4cb663315629d777350ce6ecc62a2d14e0eaebca9b88d9ff9"; - const LEFT: &[ParameterChannel] = &[ParameterChannel::Left]; const RIGHT: &[ParameterChannel] = &[ParameterChannel::Right]; const BOTH: &[ParameterChannel] = &[ParameterChannel::Left, ParameterChannel::Right]; @@ -529,9 +523,8 @@ fn push(console: &mut Console, channel: usize, record: EffectControlRecord) { console.gate.publish(revision); } -/// What the run saw, besides its digest. +/// What the run saw. struct Run { - digest: String, /// `[block][track]` published reduction words, `(left, right)`. readings: Vec>, /// Track -> `(bank, lane)` at the run's width (see [`placement`]). @@ -541,7 +534,6 @@ struct Run { fn run(backend: Backend) -> Run { let mut console = prepare(backend); let placement = placement(&console, backend); - let mut hash = Sha256Sink::new(); let mut readings = Vec::with_capacity(BLOCKS); // Before block 0, so both land at its first sample. for track in 0..TRACKS { @@ -621,9 +613,6 @@ fn run(backend: Backend) -> Run { samples.iter().all(|word| word.is_finite()), "block {block}: the master is finite" ); - for word in samples { - hash.update(word.to_bits().to_le_bytes()); - } let mut row = Vec::with_capacity(TRACKS); for track in 0..TRACKS { let handle = &console.handles.effect_observations[console.observers[track]]; @@ -635,15 +624,11 @@ fn run(backend: Backend) -> Run { block as u64 + 1, "block {block}: track {track} publishes one window per block" ); - hash.update(window.sequence.to_le_bytes()); - hash.update(window.left.to_bits().to_le_bytes()); - hash.update(window.right.to_bits().to_le_bytes()); row.push((window.left, window.right)); } readings.push(row); } Run { - digest: hash.finish_hex(), readings, placement, } @@ -677,28 +662,23 @@ fn unlinking_lanes(placement: &[(usize, usize)]) -> Vec { // The 8-lane (AVX2) twin of the `_at_simd4` and `_at_scalar` tests below (issue #1112). #[test] #[cfg(target_feature = "avx2")] -fn the_hot_console_renders_the_pre_990_words_at_simd8() { +fn the_hot_console_links_unlinks_and_limits_at_simd8() { check(Backend::Simd8); } #[test] -fn the_hot_console_renders_the_pre_990_words_at_simd4() { +fn the_hot_console_links_unlinks_and_limits_at_simd4() { check(Backend::Simd4); } #[test] -fn the_hot_console_renders_the_pre_990_words_at_scalar() { +fn the_hot_console_links_unlinks_and_limits_at_scalar() { check(Backend::Scalar); } fn check(backend: Backend) { let run = run(backend); - println!( - "limiter linked session digest at {backend:?}: {}", - run.digest - ); - // Every lane of a bank, the two edge lanes included, carries the retarget that unlinks some // bank. let lanes = unlinking_lanes(&run.placement); @@ -760,13 +740,4 @@ fn check(backend: Backend) { } } } - - if std::env::var_os("MISO_ENGINE_REPIN_LIMITER_LINKED_SESSION").is_some() { - println!("limiter linked session checks pass at {backend:?}"); - return; - } - assert_eq!( - run.digest, PIN, - "{backend:?}: the hot console's words moved from the pre-#990 kernel's" - ); } diff --git a/crates/host-core/tests/prepare.rs b/crates/host-core/tests/prepare.rs index c93ce24e6..8c8e9b3c0 100644 --- a/crates/host-core/tests/prepare.rs +++ b/crates/host-core/tests/prepare.rs @@ -5,14 +5,12 @@ //! checked against the row the facade itself reports, and every source-control rejection is typed. use builtins::{MeterMetricSet, MeterTap}; -use core::mem::{size_of, size_of_val}; use host_core::{ - EffectControlProducer, HostLiveControlRequest, HostLiveLanes, HostMeterRequest, - HostPrepareCaps, HostPrepareReport, HostShapePolicy, LAUNCH_SAMPLE_RATES, LiveValuesSuperseded, - PrepareRejection, SOURCE_STALL_TOLERANCE_MS, SourceControlError, SourceSubmission, - compile_host_session, control_table_bytes, default_source_ring_frames, diagnostic_lines, - prepare_host_runtime, prepare_host_runtime_with_live_controls, - prepare_host_runtime_with_live_lanes, + HostLiveControlRequest, HostLiveLanes, HostMeterRequest, HostPrepareCaps, HostPrepareReport, + HostShapePolicy, LAUNCH_SAMPLE_RATES, LiveValuesSuperseded, PrepareRejection, + SOURCE_STALL_TOLERANCE_MS, SourceControlError, SourceSubmission, compile_host_session, + control_table_bytes, default_source_ring_frames, diagnostic_lines, prepare_host_runtime, + prepare_host_runtime_with_live_controls, prepare_host_runtime_with_live_lanes, prepare_host_runtime_with_selected_meters_between_render_calls, prepare_host_session, source_id_arena_bytes, }; @@ -386,8 +384,12 @@ fn retained_bytes_projection_matches_the_live_set() { ); } +/// Red if the host reports (and so admits) effect-control bytes for anything but the producers it +/// hands out: a copy with another capacity, a subset, or a figure of its own. The charge itself +/// (table by capacity, a shared factory once, no registry-owned factory) is the effect compiler's, +/// and its relations are tested there (`effect_control_resources_charge_*`). #[test] -fn effect_control_report_uses_actual_native_capacity_strings_and_owners() { +fn effect_control_report_charges_the_producers_the_host_hands_out() { let compiled = compile_host_session(SESSION, &caps()).expect("compiled fixture"); let live_controls = HostLiveControlRequest::default(); let (prepared, handles) = prepare_host_runtime_with_selected_meters_between_render_calls( @@ -404,73 +406,25 @@ fn effect_control_report_uses_actual_native_capacity_strings_and_owners() { 9, "launch EQ fixture has one effect per track" ); - let table = (handles.effect_controls.capacity() * size_of::()) as u64; - let mut payload = handles - .effect_controls - .iter() - .flat_map(|producer| [producer.track_id.len(), producer.effect_id.len()]) - .map(|bytes| bytes as u64) - .sum::(); - let mut largest_owned = handles - .effect_controls - .iter() - .flat_map(|producer| [producer.track_id.len(), producer.effect_id.len()]) - .map(|bytes| bytes as u64) - .max() - .unwrap_or(0); + // All nine owners share one factory. let shared_factory = handles.effect_controls[0].owner().unwrap().factory(); - // Each EQ lane's writer holds one cell writer per `(slot, channel)` of its target cells, and - // none for parameters (its parameters ride its targets): the EQ has six target slots. - let writer_array = (2 - * 6 - * size_of::>( - )) as u64; for producer in &handles.effect_controls { - payload += writer_array; - largest_owned = largest_owned.max(writer_array); let owner = producer.owner().expect("live EQ owner"); assert!(std::sync::Arc::ptr_eq(shared_factory, owner.factory())); - for bytes in [size_of_val(owner), size_of_val(owner.committed())] { - payload += bytes as u64; - largest_owned = largest_owned.max(bytes as u64); - } } - // All nine owners share one factory Arc: its actual header/layout is charged at most once. - let factory_bytes = core::alloc::Layout::new::<[core::sync::atomic::AtomicUsize; 2]>() - .extend(core::alloc::Layout::for_value(shared_factory.as_ref())) - .expect("factory layout") - .0 - .pad_to_align() - .size() as u64; - // #1469 Amendment 1: a factory the launch registry owns is charged to no plan. - if !host_core::launch_registry_owns_factory(shared_factory) { - payload += factory_bytes; - largest_owned = largest_owned.max(factory_bytes); - } - assert_eq!( - prepared - .report - .effect_control_resources - .producer_table_bytes, - table - ); - assert_eq!( - prepared.report.effect_control_resources.owned_payload_bytes, - payload - ); + let report = prepared.report.effect_control_resources; assert_eq!( - prepared - .report - .effect_control_resources - .largest_owned_allocation_bytes, - largest_owned + report, + effect_compiler::effect_control_resources(&handles.effect_controls) + .expect("resource facts"), + "the report is the charge of the producers handed out" ); + assert!(report.producer_table_bytes > 0 && report.owned_payload_bytes > 0); assert_eq!( - prepared - .report - .effect_control_resources - .largest_allocation_bytes(), - table.max(largest_owned) + report.largest_allocation_bytes(), + report + .producer_table_bytes + .max(report.largest_owned_allocation_bytes) ); } diff --git a/crates/host-core/tests/route_mute.rs b/crates/host-core/tests/route_mute.rs index 4c77dd245..4a358527a 100644 --- a/crates/host-core/tests/route_mute.rs +++ b/crates/host-core/tests/route_mute.rs @@ -2457,7 +2457,7 @@ fn a_rebuild_that_hand_mutes_the_source_ramps_the_send_over_the_edits_length() { /// no six-word lane is attached: a mute entry on `t` gives `t-b` its seven-word lane, the same /// entry on `u` gives no route a lane. The plans differ only in that lane (and the route-activity /// table its plan builds), and the bytes the first plan retains beyond the second are within the -/// report's `route_control_resources`, whose one route is charged a seven-word cell. +/// report's `route_control_resources`, which charges one route. /// /// Test value: red if the seven-word cell, its owner, its binding, its producer or the activity /// table it makes the plan build is left out of the charge a host admits against its caps. @@ -2511,10 +2511,6 @@ fn the_following_route_charge_covers_its_attach_and_bind() { let (without, none) = retained(&elsewhere); assert_eq!(none, host_core::RouteControlResources::default()); assert_eq!(resources.routes, 1); - let follow_cell = (2 * core::mem::size_of::() - + core::mem::size_of::>() - + core::mem::size_of::()) as u64; - assert_eq!(resources.cell_bytes, follow_cell, "a seven-word cell"); let added = with .checked_sub(without) .filter(|bytes| *bytes > 0) diff --git a/crates/host-core/tests/submix_strip.rs b/crates/host-core/tests/submix_strip.rs index a6abef311..c96a4dab8 100644 --- a/crates/host-core/tests/submix_strip.rs +++ b/crates/host-core/tests/submix_strip.rs @@ -827,25 +827,31 @@ fn a_bus_insert_is_charged_once_in_the_session_estimate() { LinkMode::Maximum, ); processed.submixes[0].inserts.effects.push(insert.clone()); + let mut on_track = bare.clone(); + on_track.tracks[0].inserts.effects.push(insert.clone()); let bare = estimate_session_resources(&bare).expect("bare estimate"); let processed = estimate_session_resources(&processed).expect("processed estimate"); + let on_track = estimate_session_resources(&on_track).expect("track-insert estimate"); let params = insert.params.len() as u64; assert_eq!(processed.effect_count - bare.effect_count, 1); assert_eq!(processed.parameter_count - bare.parameter_count, params); - // By hand: the insert's vector element and its parameter vector, its two retained strings - // (instance ID and native ID) once in the model and ten times in the canonical bound, and one - // 1 KiB canonical item for the instance and for each parameter. + // The insert's two retained strings (instance ID and native ID) are retained once. let strings = (insert.id.as_str().len() + "miso.compressor".len()) as u64; - let vectors = (size_of::() as u64) + params * size_of::() as u64; assert_eq!( processed.retained_string_bytes - bare.retained_string_bytes, strings ); + // The model charge is the track insert's: once, not once per place that names it. + let added = processed.compiled_model_bytes - bare.compiled_model_bytes; assert_eq!( - processed.compiled_model_bytes - bare.compiled_model_bytes, - vectors + strings + 10 * strings + 1_024 * (1 + params) + added, + on_track.compiled_model_bytes - bare.compiled_model_bytes, + "a bus insert is charged as the same track insert is" ); + // Beyond the vectors, the strings once in the model and ten times in the canonical bound, and + // one 1 KiB canonical item for the instance and for each parameter. + assert!(added > strings + 10 * strings + 1_024 * (1 + params)); } /// Heavy inserts: eight effects of eight parameters each, and one effect of 100 parameters. diff --git a/crates/host-core/tests/successor_swap.rs b/crates/host-core/tests/successor_swap.rs index 899cba243..ba83bd3c3 100644 --- a/crates/host-core/tests/successor_swap.rs +++ b/crates/host-core/tests/successor_swap.rs @@ -45,10 +45,10 @@ const SOURCE: &str = "fixture-source"; const AUX_SOURCE: &str = "aux-source"; const MUTED_TRACK: &str = "a-muted"; -/// Bytes of `moves` strip-stage moves in a carry program (#1277 D8): one per carried fader or -/// matrix stage. -fn strip_moves(moves: u64) -> u64 { - moves * core::mem::size_of::() as u64 +/// The carry program's table entries, `[source moves, input-section lane moves, strip-stage +/// moves, effect-lane moves]` (#1276, #1277 D8, #1280): `graph::test_only_carry_counts`. +fn carry_counts(prepared: &mut host_core::PreparedHost) -> [usize; 4] { + graph::test_only_carry_counts(&mut prepared.plan) } /// The command queue a successor's carried timeline brings (#1641 D7): every successor's carried @@ -211,11 +211,9 @@ fn added_muted_track_keeps_playing(backend: Backend) { fresh.source_overhead_bytes ); assert_eq!( - report.carry_program_retained_bytes, - core::mem::size_of::<(u32, u32)>() as u64 - + 2 * core::mem::size_of::() as u64 - + strip_moves(4), - "one (u32, u32) source move, the two unchanged input sections (#1276) and the fader and \ + run.carry_counts, + [1, 2, 4, 0], + "one source move, the two unchanged input sections (#1276) and the fader and \ matrix stage of both kept strips (#1277)" ); @@ -311,8 +309,8 @@ fn a_changed_source_gets_its_own_ring() { timeline_queue_bytes() ); assert_eq!( - prepared.report.carry_program_retained_bytes, - 2 * core::mem::size_of::() as u64 + strip_moves(4), + carry_counts(&mut prepared), + [0, 2, 4, 0], "no source move; the two unchanged input sections (#1276) and the fader and matrix \ stage of both strips (#1277)" ); @@ -964,10 +962,7 @@ fn filtered_strips_keep_their_state(backend: Backend) { // Nine input sections recorded, and all nine carry: one source move and nine lane moves, and // (#1277) the fader and matrix stage of each of the nine strips. assert_eq!(a.prepare(backend).inventory.input_section_count(), 9); - assert_eq!( - run.successor.carry_program_retained_bytes, - 8 + 9 * core::mem::size_of::() as u64 + strip_moves(18) - ); + assert_eq!(run.carry_counts, [1, 9, 18, 0]); } /// Gate 1 at eight lanes: banks of eight and one become eight and two, so every kept lane moves @@ -1677,8 +1672,8 @@ fn a_changed_section_starts_at_rest(name: &str, edit: fn(&mut session::DualMonoB let run = swapped_run(&a, &b, &[&feed], &[&feed], backend, Successor::Carry); assert_eq!(run.blocks[SWAP_BLOCK].carry, CarryOutcome::Carried); assert_eq!( - run.successor.carry_program_retained_bytes, - core::mem::size_of::<(u32, u32)>() as u64 + strip_moves(2), + run.carry_counts, + [1, 0, 2, 0], "{name}, {backend:?}: the source moves, the changed section does not; the strip's \ fader and matrix stage still do (#1277)" ); @@ -1729,31 +1724,26 @@ fn a_changed_control_kind_carries_no_input_section() { } fn a_changed_control_kind_carries_no_input_section_at(a: &Session, b: &Session, backend: Backend) { - let source_only = core::mem::size_of::<(u32, u32)>() as u64; + let source_only = [1, 0, 0, 0]; // Predecessor without live controls, successor with them. let plain = a.prepare(backend); - let live = LivePlan::successor(b, &plain.inventory, &a.model, backend); + let mut live = LivePlan::successor(b, &plain.inventory, &a.model, backend); assert_eq!( - live.prepared.report.carry_program_retained_bytes, source_only, + carry_counts(&mut live.prepared), + source_only, "plain -> live" ); // Predecessor with live controls, successor without them. let live = LivePlan::fresh(a, backend); - let plain = b + let mut plain = b .prepare_successor(&live.prepared.inventory, &a.model, &caps(), backend) .unwrap_or_else(|failure| panic!("successor: {failure:?}")); - assert_eq!( - plain.report.carry_program_retained_bytes, source_only, - "live -> plain" - ); + assert_eq!(carry_counts(&mut plain), source_only, "live -> plain"); // The control: the same kind on both sides carries all nine sections. - let plain = b + let mut plain = b .prepare_successor(&a.prepare(backend).inventory, &a.model, &caps(), backend) .unwrap_or_else(|failure| panic!("successor: {failure:?}")); - assert_eq!( - plain.report.carry_program_retained_bytes, - source_only + 9 * core::mem::size_of::() as u64 + strip_moves(18) - ); + assert_eq!(carry_counts(&mut plain), [1, 9, 18, 0]); } /// D1's strip-kind clause: B removes track `eq0` and adds a submix named `eq0`, with `eq0`'s input @@ -1802,12 +1792,12 @@ fn a_track_replaced_by_a_submix_of_its_name_does_not_carry() { b_model.routes.push(out); let b = Session::compile(b_model); for backend in backends() { - let prepared = b + let mut prepared = b .prepare_successor(&a.prepare(backend).inventory, &a.model, &caps(), backend) .unwrap_or_else(|failure| panic!("successor: {failure:?}")); assert_eq!( - prepared.report.carry_program_retained_bytes, - 8 + core::mem::size_of::() as u64 + strip_moves(2), + carry_counts(&mut prepared), + [1, 1, 2, 0], "{backend:?}: the source and eq1's section move, the submix's section does not; eq1's \ fader and matrix stage move, the submix's do not (#1277)" ); @@ -3247,11 +3237,7 @@ fn strip_state_survives_a_swap(backend: Backend) { assert!(run.restarted.is_empty(), "{backend:?}: {:?}", run.restarted); // Nine strips: one source move, nine input lanes, and a fader and a matrix move each. assert_eq!(a.prepare(backend).inventory.strip_stage_count(), 18); - assert_eq!( - run.successor.carry_program_retained_bytes, - 8 + 9 * core::mem::size_of::() as u64 - + 18 * core::mem::size_of::() as u64 - ); + assert_eq!(run.carry_counts, [1, 9, 18, 0]); } /// Red if a stage is keyed by lane or index, not strip ID (the added track sorts first, so every @@ -3290,8 +3276,8 @@ fn a_strip_behind_an_effect_still_carries(backend: Backend) { let run = swapped_run(&a, &b, &[&feed], &[&feed], backend, Successor::Carry); assert_eq!(run.blocks[SWAP_BLOCK].carry, CarryOutcome::Carried); assert!(run.restarted.is_empty(), "{backend:?}: {:?}", run.restarted); - assert!( - run.successor.carry_program_retained_bytes >= strip_moves(18), + assert_eq!( + run.carry_counts[2], 18, "{backend:?}: all eighteen strip stages move" ); } @@ -4380,7 +4366,7 @@ fn a_changed_control_kind_carries_no_strip_stage() { let b = Session::compile(with_muted_track(strip_state_session())); for backend in backends() { let predecessor = a.prepare(backend); - let successor = CellPlan::successor( + let mut successor = CellPlan::successor( &b, &predecessor.inventory, &a.model, @@ -4390,9 +4376,9 @@ fn a_changed_control_kind_carries_no_strip_stage() { ); assert_eq!(successor.live.prepared.restarted_strips().len(), 9); assert_eq!( - successor.live.prepared.report.carry_program_retained_bytes, + carry_counts(&mut successor.live.prepared), // The source only: the control kind differs, so no input lane and no strip stage moves. - 8, + [1, 0, 0, 0], "{backend:?}" ); } @@ -4693,8 +4679,8 @@ struct PlainRun { transitions: [u64; 3], /// The successor's `carry_refusal_count()` after the last block (D6). refusals: u64, - /// The successor's `carry_program_retained_bytes`. - carry_bytes: u64, + /// The successor's carry-table entries ([`carry_counts`]). + carry_counts: [usize; 4], /// Effect lanes the predecessor's inventory records. effect_lanes: usize, } @@ -4731,7 +4717,7 @@ fn plain_run( let mut restarted = Vec::new(); let mut swap_allocator = ((0, 0, 0), (0, 0)); let mut collapses = [[0; 2]; 2]; - let mut carry_bytes = 0; + let mut carry_counts = [0; 4]; feed.submit(&mut plan.sources, 48_000, 0); for block in 0..BLOCKS { if let (SWAP_BLOCK, Some((session, committed))) = (block, b) { @@ -4741,7 +4727,7 @@ fn plain_run( .unwrap_or_else(|failure| panic!("successor: {failure:?}")); successor.plan.force_mono_collapse_off(successor_dual); restarted = successor.restarted_strips().to_vec(); - carry_bytes = successor.report.carry_program_retained_bytes; + carry_counts = self::carry_counts(&mut successor); assert_eq!(successor.sources.adopt_persisting(&mut plan.sources), 1); if block + 1 < BLOCKS { feed.submit(&mut successor.sources, 48_000, block + 1); @@ -4777,7 +4763,7 @@ fn plain_run( collapses, transitions: plan.plan.bank_collapse_transitions(), refusals: graph::carry_refusal_count(&mut plan.plan).expect("a graph plan"), - carry_bytes, + carry_counts, effect_lanes, } } @@ -4977,8 +4963,8 @@ fn console_chains_keep_collapsing(backend: Backend, successor_dual: bool, inputs let kept = Session::compile(with_muted_track(base())); let carried = plain_run(&a, Some((&kept, &a.model)), &feed, backend, false); assert_eq!( - carried.carry_bytes - run.carry_bytes, - 9 * core::mem::size_of::() as u64, + carried.carry_counts[1] - run.carry_counts[1], + 9, "{backend:?}" ); } @@ -5172,8 +5158,8 @@ fn a_prepared_change_to_one_slot_restarts_only_its_strip(backend: Backend, slots assert_eq!(run.restarted, vec![Box::::from("eq3")], "{backend:?}"); assert_eq!(carried_all.restarted, Vec::>::new(), "{backend:?}"); assert_eq!( - carried_all.carry_bytes - run.carry_bytes, - core::mem::size_of::() as u64, + carried_all.carry_counts[3] - run.carry_counts[3], + 1, "{backend:?}: one move fewer" ); assert_eq!( @@ -5407,8 +5393,8 @@ fn a_removed_console_slot_restarts_every_strip(backend: Backend) { assert_eq!(run.restarted, every_strip, "{backend:?}"); assert_eq!(unchanged.restarted, Vec::>::new(), "{backend:?}"); assert_eq!( - unchanged.carry_bytes - run.carry_bytes, - 9 * core::mem::size_of::() as u64, + unchanged.carry_counts[3] - run.carry_counts[3], + 9, "{backend:?}: the removed slot's nine lanes do not move, the other forty-five do" ); assert_eq!(run.refusals, 0, "{backend:?}"); diff --git a/crates/host-core/tests/support/successor.rs b/crates/host-core/tests/support/successor.rs index 2a230b9cc..6313fb616 100644 --- a/crates/host-core/tests/support/successor.rs +++ b/crates/host-core/tests/support/successor.rs @@ -218,6 +218,10 @@ pub(crate) struct SwappedRun { pub(crate) blocks: Vec, /// The successor's report. pub(crate) successor: host_core::HostPrepareReport, + /// The successor's carry-table entries, `graph::test_only_carry_counts`. + // Read by the successor_swap cases; every test binary shares this file. + #[allow(dead_code)] + pub(crate) carry_counts: [usize; 4], /// The producers each set held after the hand-over: the predecessor's, then the successor's. pub(crate) predecessor_sources: SourceControlSet, pub(crate) successor_sources: SourceControlSet, @@ -297,6 +301,7 @@ pub(crate) fn swapped_run_with( Successor::Fresh => 0, }; let restarted = b_prepared.restarted_strips().to_vec(); + let carry_counts = graph::test_only_carry_counts(&mut b_prepared.plan); let PreparedHost { mut plan, sources: mut b_sources, @@ -326,6 +331,7 @@ pub(crate) fn swapped_run_with( SwappedRun { blocks, successor: report, + carry_counts, predecessor_sources: a_sources, successor_sources: b_sources, adopted, diff --git a/crates/rack/Cargo.toml b/crates/rack/Cargo.toml index d9c0a549a..8a73889e4 100644 --- a/crates/rack/Cargo.toml +++ b/crates/rack/Cargo.toml @@ -9,11 +9,6 @@ publish.workspace = true [lib] name = "rack" -[features] -# Issue #960: the bank chain's sub-phase probes (`test_only_bank_phase_profile`). Test builds -# only; `graph/test-support` forwards it, and no production build enables it. -test-support = [] - [dependencies] engine.workspace = true effect-contract.workspace = true diff --git a/crates/rack/src/lib.rs b/crates/rack/src/lib.rs index 3f76f5d92..d404f7601 100644 --- a/crates/rack/src/lib.rs +++ b/crates/rack/src/lib.rs @@ -18,174 +18,6 @@ use effect_contract::{ }; use engine::realtime::{Consumer, LiveSnapshot, RenderError}; -/// Sub-phase timing inside [`BankChain::run`] for the gain/pan phase profile (issue #960). -/// -/// The graph crate's phase profile charges a whole bank unit to one `BANK` phase, which hides -/// everything that matters on a row whose every track renders on a bank chain. This module splits -/// that time at the chain's own boundaries: the prologue (slot queue drains and the mono-collapse -/// decision), the gather, each slot by its position in the chain, the collapse seam, the scatter, -/// the route/master fold the members perform, and the auxiliary epilogue. -/// -/// Test builds only. It exists under the `test-support` feature and nowhere else, so a production -/// build carries neither the module nor a single probe statement. A `test-support` build that never -/// calls [`enable`](test_only_bank_phase_profile::enable) pays one thread-local read per probe site -/// a chain reaches, and nothing else. -/// -/// The clock is `std::time::Instant`, as in the graph crate's profile, because this crate may not -/// carry the block a cycle-counter intrinsic needs. Each probe is one clock read and a few -/// thread-local updates, and its cost lands in the sub-phases on either side of it, about one probe -/// per entry; the harness derives that cost in situ and reports the sub-phases net of it. Only the -/// thread that renders is profiled. -#[cfg(feature = "test-support")] -#[doc(hidden)] -pub mod test_only_bank_phase_profile { - use core::cell::Cell; - use std::time::Instant; - - /// Chain entry to the gather: every slot's `begin_block` drain, the channel-symmetry witness - /// and the mono-collapse decision. - pub const PROLOGUE: usize = 0; - /// Planar to AoSoA: the gather, the collapsed gather, or the resident-input copy. - pub const GATHER: usize = 1; - /// The first slot position. Slot `i` of a chain is charged to `SLOT + min(i, SLOTS - 1)`. - pub const SLOT: usize = 2; - /// Slot positions profiled separately; a deeper slot shares the last one. - pub const SLOTS: usize = 8; - /// The mono-collapse seam: the one copy of the resident left plane into the right. - pub const SEAM: usize = SLOT + SLOTS; - /// AoSoA to planar: the scatter's transposes and copies, excluding the fold calls. - pub const SCATTER: usize = SEAM + 1; - /// The members' route and master accumulation (`fold_resident`, `fold_cohort`, `fold_plane`), - /// including the resident-block offer's shape checks. - pub const FOLD: usize = SCATTER + 1; - /// The auxiliary-destination epilogue (`accumulate_aux`). - pub const AUX: usize = FOLD + 1; - /// Sub-phase count. - pub const COUNT: usize = AUX + 1; - - /// One profile's totals since the last [`reset`]. - #[derive(Clone, Copy, Debug, Eq, PartialEq)] - pub struct Snapshot { - /// Nanoseconds charged to each sub-phase. - pub nanos: [u64; COUNT], - /// Times each sub-phase was entered; [`PROLOGUE`] once per profiled chain run. - pub entries: [u64; COUNT], - /// Chain runs that completed a profiled run. - pub chain_runs: u64, - /// Clock reads taken, the unit of probe overhead. - pub probes: u64, - } - - thread_local! { - static ENABLED: Cell = const { Cell::new(false) }; - /// The running chain's clock and current sub-phase; `None` outside a profiled run. - static CURRENT: Cell> = const { Cell::new(None) }; - static NANOS: [Cell; COUNT] = const { [const { Cell::new(0) }; COUNT] }; - static ENTRIES: [Cell; COUNT] = const { [const { Cell::new(0) }; COUNT] }; - static CHAIN_RUNS: Cell = const { Cell::new(0) }; - static PROBES: Cell = const { Cell::new(0) }; - static SLOT_NAMES: [Cell<&'static str>; SLOTS] = const { [const { Cell::new("") }; SLOTS] }; - } - - /// The label a slot position shows when two chains put different stage types there. - pub const MIXED: &str = "(mixed)"; - - /// Switch the probes on or off for this thread. - pub fn enable(enabled: bool) { - ENABLED.set(enabled); - if !enabled { - CURRENT.set(None); - } - } - - /// Zero the accumulators and forget the slot names. - pub fn reset() { - CURRENT.set(None); - NANOS.with(|cells| cells.iter().for_each(|cell| cell.set(0))); - ENTRIES.with(|cells| cells.iter().for_each(|cell| cell.set(0))); - CHAIN_RUNS.set(0); - PROBES.set(0); - SLOT_NAMES.with(|cells| cells.iter().for_each(|cell| cell.set(""))); - } - - /// Totals since the last [`reset`]. - #[must_use] - pub fn snapshot() -> Snapshot { - Snapshot { - nanos: NANOS.with(|cells| cells.each_ref().map(Cell::get)), - entries: ENTRIES.with(|cells| cells.each_ref().map(Cell::get)), - chain_runs: CHAIN_RUNS.get(), - probes: PROBES.get(), - } - } - - /// The stage type each slot position ran since the last [`reset`]: `""` for a position no - /// profiled chain reached, [`MIXED`] for one whose chains disagreed. - #[must_use] - pub fn slot_names() -> [&'static str; SLOTS] { - SLOT_NAMES.with(|cells| cells.each_ref().map(Cell::get)) - } - - /// Open a chain run's profile in [`PROLOGUE`] when the probes are on. - #[inline] - pub(crate) fn start() { - if ENABLED.get() { - CURRENT.set(Some((Instant::now(), PROLOGUE))); - PROBES.set(PROBES.get() + 1); - ENTRIES.with(|cells| cells[PROLOGUE].set(cells[PROLOGUE].get() + 1)); - } - } - - /// Charge the time since the last probe to the current sub-phase and make `next` current. - /// Outside a profiled run this is one thread-local read. - #[inline] - pub(crate) fn enter(next: usize) { - let Some((last, phase)) = CURRENT.get() else { - return; - }; - let now = Instant::now(); - charge(phase, now, last); - CURRENT.set(Some((now, next))); - ENTRIES.with(|cells| cells[next].set(cells[next].get() + 1)); - } - - /// [`enter`] the slot at chain position `position`, and remember the stage type it runs. - #[inline] - pub(crate) fn enter_slot(position: usize, stage: &dyn super::BankStage) { - if CURRENT.get().is_none() { - return; - } - let position = position.min(SLOTS - 1); - enter(SLOT + position); - let name = stage.test_only_stage_name(); - SLOT_NAMES.with(|cells| { - let cell = &cells[position]; - if cell.get().is_empty() { - cell.set(name); - } else if cell.get() != name { - cell.set(MIXED); - } - }); - } - - /// Close a chain run's profile: charge the rest to the current sub-phase. - #[inline] - pub(crate) fn finish() { - let Some((last, phase)) = CURRENT.get() else { - return; - }; - charge(phase, Instant::now(), last); - CURRENT.set(None); - CHAIN_RUNS.set(CHAIN_RUNS.get() + 1); - } - - fn charge(phase: usize, now: Instant, last: Instant) { - let elapsed = u64::try_from(now.duration_since(last).as_nanos()).unwrap_or(u64::MAX); - NANOS.with(|cells| cells[phase].set(cells[phase].get().saturating_add(elapsed))); - PROBES.set(PROBES.get() + 1); - } -} - #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub enum RackError { ZeroQuantum, @@ -572,13 +404,6 @@ impl<'a> BankPlaneViews<'a> { /// validated once at prepare - so `Err` is the stage's own render failure. pub trait BankStage: Send { fn process(&mut self, block: BankBlock<'_>) -> Result<(), RenderError>; - /// The stage's type name: the slot label of the bank sub-phase profile (issue #960). Test - /// builds only, and never called unless the profile is on. - #[cfg(feature = "test-support")] - #[doc(hidden)] - fn test_only_stage_name(&self) -> &'static str { - core::any::type_name::() - } /// Copy one prepared lane's response sections without advancing render state. fn copy_response_snapshot_lane( &self, @@ -3212,8 +3037,6 @@ impl BankChain { snapshot: LiveSnapshot, predecessor: Option<&BankChain>, ) -> Result<(), RenderError> { - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::start(); debug_assert!(frames >= 1 && frames <= self.scratch.quantum); let len = frames as usize * self.lanes; // Validate stored shape before any successor drain or destination write. Graph owns @@ -3314,8 +3137,6 @@ impl BankChain { if collapse { self.collapses = self.collapses.saturating_add(1); } - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::GATHER); if let Some(source) = predecessor { self.acquire_resident_input(source, frames, collapse); } else if collapse { @@ -3329,14 +3150,8 @@ impl BankChain { } else { self.slots.len() }; - #[cfg(feature = "test-support")] - let mut position = 0_usize; - // The probe's slot position is the only counter here, and it exists only with the feature. - #[cfg_attr(feature = "test-support", allow(clippy::explicit_counter_loop))] for slot in &mut self.slots[..prefix] { if slot.has_active_lanes() { - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter_slot(position, &*slot.stage); let block = BankBlock { left: &mut self.scratch.left[..len], right: &mut self.scratch.right[..len], @@ -3350,10 +3165,6 @@ impl BankChain { slot.stage.process(block)?; } } - #[cfg(feature = "test-support")] - { - position += 1; - } } if collapse { // The seam. One extra copy of the resident block per collapsed chain per block, and it @@ -3362,17 +3173,10 @@ impl BankChain { // exactly the arithmetic it runs on a dual block, in exactly its operation order. The // fold's `ll*l + lr*r` is *not* rewritten as `(ll + lr) * l` on the strength of // `l == r`: the two round differently, and the second is not what a dual run computes. - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::SEAM); let AoSoaScratch { left, right, .. } = &mut self.scratch; right[..len].copy_from_slice(&left[..len]); - #[cfg(feature = "test-support")] - let mut position = self.collapse_prefix; - #[cfg_attr(feature = "test-support", allow(clippy::explicit_counter_loop))] for slot in &mut self.slots[self.collapse_prefix..] { if slot.has_active_lanes() { - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter_slot(position, &*slot.stage); slot.stage.process(BankBlock { left: &mut self.scratch.left[..len], right: &mut self.scratch.right[..len], @@ -3381,24 +3185,14 @@ impl BankChain { lanes: self.lanes, })?; } - #[cfg(feature = "test-support")] - { - position += 1; - } } } - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::SCATTER); self.scatter(members, frames); // The epilogue. Empty on every chain the engine builds today, so this is one branch and // no auxiliary state; see `BankMembers::aux_plane_mut` for what arming it is for. if !self.aux.is_empty() { - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::AUX); self.accumulate_aux(members, frames); } - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::finish(); Ok(()) } // REALTIME_POLICY_END @@ -3791,11 +3585,7 @@ impl BankChain { folded_lanes[folded_count] = lane; folded_count += 1; } else { - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::FOLD); members.fold_plane(lane, left, right); - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::SCATTER); } } else { let (left, right) = members.plane_mut(lane); @@ -3811,8 +3601,6 @@ impl BankChain { used, ) { - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::FOLD); members.fold_cohort(cohort); } } @@ -3880,10 +3668,6 @@ impl BankChain { // would name -- and that no lane is left needing a `plane_mut` copy. An accepted offer // finishes the fold; the staging block and every plane stay unwritten. A declined one // wrote nothing, and the staged path below runs as it always has. - #[cfg(feature = "test-support")] - if !self.fold.is_empty() { - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::FOLD); - } if self.fold.len() == W && self.fold.iter().all(|folded| *folded) && let Ok(cohort) = ResidentFoldCohort::new( @@ -3896,10 +3680,6 @@ impl BankChain { { return; } - #[cfg(feature = "test-support")] - if !self.fold.is_empty() { - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::SCATTER); - } let tiled = (frames_used / W) * W; tile_scatter( &self.scratch.left[..tiled * W], @@ -3952,11 +3732,7 @@ impl BankChain { folded_lanes[folded_count] = lane; folded_count += 1; } else { - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::FOLD); members.fold_plane(lane, left, right); - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::SCATTER); } } else { let (plane_left, plane_right) = members.plane_mut(lane); @@ -3974,8 +3750,6 @@ impl BankChain { frames_used, ) { - #[cfg(feature = "test-support")] - test_only_bank_phase_profile::enter(test_only_bank_phase_profile::FOLD); members.fold_cohort(cohort); } } @@ -7923,162 +7697,6 @@ mod tests { "the disengage copy re-establishes agreement; a witness that still holds keeps it" ); } - - /// Issue #960: members with a fold, an auxiliary destination and direct planes, so one type - /// reaches every scatter path the bank sub-phase probes split. - #[cfg(feature = "test-support")] - struct ProbedMembers { - planes: Planes, - bus_left: Vec, - bus_right: Vec, - aux: Vec, - aux_left: Vec, - aux_right: Vec, - } - #[cfg(feature = "test-support")] - impl ProbedMembers { - fn words(&self) -> Vec { - let planes = self - .planes - .left - .iter() - .chain(self.planes.right.iter()) - .flatten(); - planes - .chain(&self.bus_left) - .chain(&self.bus_right) - .chain(&self.aux_left) - .chain(&self.aux_right) - .map(|word| word.to_bits()) - .collect() - } - } - #[cfg(feature = "test-support")] - impl BankMembers for ProbedMembers { - fn plane(&self, lane: usize) -> (&[f32], &[f32]) { - self.planes.plane(lane) - } - fn plane_mut(&mut self, lane: usize) -> (&mut [f32], &mut [f32]) { - self.planes.plane_mut(lane) - } - fn distinct_planes_mut( - &mut self, - lanes: usize, - frames: usize, - ) -> Option> { - self.planes.distinct_planes_mut(lanes, frames) - } - fn fold_plane(&mut self, lane: usize, left: &mut [f32], right: &mut [f32]) { - let gain = 0.5 + lane as f32; - for (bus, sample) in self.bus_left.iter_mut().zip(left.iter()) { - *bus += *sample * gain; - } - for (bus, sample) in self.bus_right.iter_mut().zip(right.iter()) { - *bus += *sample * gain; - } - } - fn aux_plane_mut(&mut self, lane: usize) -> Option<(&mut [f32], &mut [f32])> { - self.aux[lane].then_some((&mut self.aux_left[..], &mut self.aux_right[..])) - } - } - - /// Issue #960: the bank sub-phase probes enter each sub-phase where they claim to, on every - /// scatter path -- direct, a declined resident offer then one staged cohort, per-lane folds, - /// the partial bank's per-lane scatter, and the auxiliary epilogue -- name each slot by its - /// stage type, and leave every word a run writes exactly as an unprofiled run writes it. - #[cfg(feature = "test-support")] - #[test] - fn bank_phase_probes_split_every_scatter_path_and_move_no_bit() { - use test_only_bank_phase_profile as profile; - const FRAMES: u32 = 16; - let mut state = 0x5eed_0960_0000_0001_u64; - for &width in BankWidth::ALL { - let lanes = width.lanes() as usize; - let all = vec![true; lanes]; - let mut partial = all.clone(); - partial[lanes - 1] = false; - let mut some = vec![false; lanes]; - some[0] = true; - some[2] = true; - // (active lanes, fold mask, aux mask, expected [SCATTER, FOLD, AUX] entries) - type Case<'a> = (&'a [bool], Option<&'a [bool]>, Option<&'a [bool]>, [u64; 3]); - let cases: [Case<'_>; 5] = [ - (&all, None, None, [1, 0, 0]), - (&all, Some(&all), None, [2, 2, 0]), - (&all, Some(&some), None, [4, 3, 0]), - (&partial, Some(&partial), None, [1, 1, 0]), - (&all, None, Some(&some), [1, 0, 1]), - ]; - for (case, (active, fold, aux, expected)) in cases.into_iter().enumerate() { - let what = format!("{lanes} lanes, case {case}"); - let source = hostile_planes(lanes, FRAMES, &mut state); - let run = |profiled: bool| { - let mut chain = BankChain::new( - AoSoaScratch::new(width, 128).expect("scratch"), - active.to_vec().into_boxed_slice(), - vec![ - slot(active.to_vec(), Box::new(ScaleByLane)), - slot(active.to_vec(), Box::new(PassThrough)), - ], - ) - .expect("chain"); - if let Some(fold) = fold { - chain - .arm_fold(fold.to_vec().into_boxed_slice()) - .expect("fold"); - } - if let Some(aux) = aux { - chain.arm_aux(aux.to_vec().into_boxed_slice()).expect("aux"); - } - let mut members = ProbedMembers { - planes: source.clone(), - bus_left: vec![0.0; FRAMES as usize], - bus_right: vec![0.0; FRAMES as usize], - aux: aux.map_or_else(|| vec![false; lanes], <[bool]>::to_vec), - aux_left: vec![0.0; FRAMES as usize], - aux_right: vec![0.0; FRAMES as usize], - }; - profile::reset(); - profile::enable(profiled); - chain - .run(&mut members, FRAMES, 0, LiveSnapshot::ALL) - .expect("run"); - profile::enable(false); - (members.words(), profile::snapshot(), profile::slot_names()) - }; - let (plain, unprofiled, _) = run(false); - let (words, snapshot, names) = run(true); - assert_eq!(words, plain, "{what}: the probes moved a bit"); - assert_eq!(unprofiled.chain_runs, 0, "{what}: off, nothing is profiled"); - assert_eq!(snapshot.chain_runs, 1, "{what}"); - assert!(snapshot.nanos.iter().sum::() > 0, "{what}"); - let entries = snapshot.entries; - assert_eq!( - [ - entries[profile::PROLOGUE], - entries[profile::GATHER], - entries[profile::SLOT], - entries[profile::SLOT + 1], - entries[profile::SEAM], - ], - [1, 1, 1, 1, 0], - "{what}" - ); - assert_eq!( - [ - entries[profile::SCATTER], - entries[profile::FOLD], - entries[profile::AUX], - ], - expected, - "{what}: scatter, fold, aux" - ); - assert!(names[0].ends_with("ScaleByLane"), "{what}: {}", names[0]); - assert!(names[1].ends_with("PassThrough"), "{what}: {}", names[1]); - assert!(names[2..].iter().all(|name| name.is_empty()), "{what}"); - } - } - } } /// The plan-swap carry of an [`EffectBankStage`] lane (#1279 D3, D5, D6), against a bank whose diff --git a/crates/true-peak-limiter/src/corpus.rs b/crates/true-peak-limiter/src/corpus.rs index 0c390e6f0..9eb27e264 100644 --- a/crates/true-peak-limiter/src/corpus.rs +++ b/crates/true-peak-limiter/src/corpus.rs @@ -19,15 +19,39 @@ //! same signal at every width. Nothing crosses lanes except the `Maximum` link, which links the two //! channels of one lane and not two lanes of one channel. //! +//! # The linked pair (#990, #996, #997) +//! +//! Case 5, `maximum/link_unlink_relink`, is the only case that drives the whole prepared core +//! (`LimiterCore::process_block`) block by block instead of one `limiter_block` call, because the +//! linked-pair decision and its record live there. Eight 128-frame blocks, every lane at the same +//! 5 ms lookahead so the bank is uniform and the linked body runs, `LinkMode::Maximum`, and loud +//! noise whose level changes every 96 frames so every limiter limits and its window minimum moves. +//! The retargets (`LINKED_RETARGETS`) land while the output carries delayed audio (from frame +//! 486): a linked ramp of both channels of every lane at block 3, a one-sided retarget of lane 0 +//! and of lane 7 at block 4 (each unlinks its bank at every width: lane 0 and the last lane of a +//! four-lane bank, the edge lanes of the eight-lane one), and the other channel made equal again at +//! block 5, after which the pairs must stay dual. Its pin equals the words of a kernel that never +//! links, so it renders what a mirrored right ring leaves behind after an unlink and a pair that +//! relinks on designed agreement alone (`tests/MUTATIONS.md`, section "TA3"). It replaced the +//! host-core 64-track linked-limiter session pin (TA3, 2026-10-19), whose session checks stay in +//! `crates/host-core/src/limiter_linked_session.rs`. +//! //! # No NaN //! //! The determinism claim excludes NaN payloads (wasm canonicalises them). Every case is driven with //! finite input inside the ranges the boundary check accepts, and the crate's determinism test //! asserts the outputs are finite rather than assuming it. +use effect_contract::{ + EffectQuality, LinkMode, PreparedEffectMetadata, PreparedPorts, PreparedSidechainPort, +}; use lane::Lane; -use crate::{ChannelState, Cursors, LimiterCoef, PARAMETER_COUNT, Shape, limiter_block}; +use crate::{ + ChannelState, Cursors, LimiterCoef, LimiterCore, PARAMETER_COUNT, QUALITIES, RAMP_UPDATES, + Shape, TRUE_PEAK_LIMITER_DESCRIPTOR, limit_coefficient, limiter_block, release_coefficient, + tail_and_rest, +}; /// Independent tracks in every case; a multiple of the widest backend. pub const LANES: usize = 8; @@ -42,7 +66,7 @@ pub const POINTS: usize = 2 * LANES * FRAMES; const RATE: u32 = 48_000; /// Number of corpus cases. -pub const CASE_COUNT: usize = 5; +pub const CASE_COUNT: usize = 6; /// Human-readable name of each case, indexed by case number. pub const CASE_NAMES: [&str; CASE_COUNT] = [ @@ -51,6 +75,7 @@ pub const CASE_NAMES: [&str; CASE_COUNT] = [ "dual_mono/near_nyquist_w_min", "dual_mono/impulse_train_long_lookahead", "dual_mono/subnormal_release_flush", + "maximum/link_unlink_relink", ]; /// The pinned digest of each case, generated from the scalar `Lane` instantiation. @@ -94,6 +119,12 @@ pub const D90_DIGESTS: [[u8; 32]; CASE_COUNT] = [ 0x42, 0x93, 0xca, 0x05, 0x80, 0x2e, 0x25, 0x30, 0x46, 0xe2, 0x79, 0xb5, 0x58, 0xba, 0x6a, 0xe0, 0x26, ], + // maximum/link_unlink_relink + [ + 0x31, 0xb0, 0x96, 0x72, 0xde, 0x3d, 0xfa, 0x2d, 0x04, 0x2d, 0x87, 0x0e, 0x8a, 0x1e, 0xca, + 0x84, 0x9e, 0xf8, 0x79, 0x50, 0x9f, 0x1c, 0xc1, 0xbd, 0x8d, 0xa7, 0xcf, 0x71, 0x6a, 0xb0, + 0x69, 0xcd, + ], ]; /// `xorshift64*`. Integer-only, so every target builds the same sequence. @@ -206,6 +237,10 @@ pub fn run_case(case: usize, out: &mut [u32]) { assert_eq!(out.len(), POINTS, "corpus output must be POINTS long"); let width = L::WIDTH; assert!(LANES.is_multiple_of(width), "LANES must divide by a width"); + if case == LINKED_CASE { + run_linked::(out); + return; + } let shape = Shape::new(RATE).expect("launch rate"); let coefficients = LimiterCoef::::new(case == 1, false); @@ -255,3 +290,204 @@ pub fn run_case(case: usize, out: &mut [u32]) { } } } + +/// The case that drives the prepared core block by block (see the module note). +const LINKED_CASE: usize = 5; + +/// Frames per block of [`LINKED_CASE`]: eight blocks make [`FRAMES`]. +const LINKED_BLOCK: usize = 128; + +/// Frames per noise segment of [`LINKED_CASE`]: divides neither the block nor the 241-frame window. +const LINKED_SEGMENT: usize = 96; + +/// Per-lane ceiling (dB) and release (ms) of [`LINKED_CASE`], equal on both channels; every lane +/// at the 5 ms lookahead, so the bank is uniform. +const LINKED_CEILINGS: [f32; LANES] = [-1.0, -3.0, -6.0, -2.0, -4.0, -1.5, -5.0, -2.5]; +const LINKED_RELEASES: [f32; LANES] = [50.0, 100.0, 200.0, 30.0, 400.0, 75.0, 150.0, 250.0]; +const LINKED_LOOKAHEAD: f32 = 5.0; + +/// The parameter a [`LinkedRetarget`] moves. +#[derive(Clone, Copy)] +enum LinkedParameter { + Ceiling, + Release, +} + +/// Before `block`, lane `lane` (every lane when `None`) moves `parameter` to `value` on the left +/// channel when `left` and on the right channel when `right`. +struct LinkedRetarget { + block: usize, + lane: Option, + parameter: LinkedParameter, + left: bool, + right: bool, + value: f32, +} + +/// The schedule of [`LINKED_CASE`] (see the module note). +const LINKED_RETARGETS: [LinkedRetarget; 5] = [ + // A linked ramp on both channels of every lane. + LinkedRetarget { + block: 3, + lane: None, + parameter: LinkedParameter::Ceiling, + left: true, + right: true, + value: -7.0, + }, + // Lane 0 unlinks on its left channel. + LinkedRetarget { + block: 4, + lane: Some(0), + parameter: LinkedParameter::Ceiling, + left: true, + right: false, + value: -9.0, + }, + // The last lane unlinks on its right channel. + LinkedRetarget { + block: 4, + lane: Some(LANES - 1), + parameter: LinkedParameter::Release, + left: false, + right: true, + value: 20.0, + }, + // Both made equal again: the pairs stay dual. + LinkedRetarget { + block: 5, + lane: Some(0), + parameter: LinkedParameter::Ceiling, + left: false, + right: true, + value: -9.0, + }, + LinkedRetarget { + block: 5, + lane: Some(LANES - 1), + parameter: LinkedParameter::Release, + left: true, + right: false, + value: 20.0, + }, +]; + +/// One lane's stereo pair for [`LINKED_CASE`]: uniform noise, each channel at its own level +/// between +6 and +18 dBFS peak, redrawn every [`LINKED_SEGMENT`] frames. +fn fill_linked(lane: usize, left: &mut [f32], right: &mut [f32]) { + let mut rng = Rng::new(0x0996_0997_0990_0001 ^ (lane as u64).wrapping_mul(0x9E37_79B9)); + let unit = |rng: &mut Rng| f32::from((rng.next() >> 48) as u16) * (1.0 / 65_536.0); + let (mut left_level, mut right_level) = (0.0, 0.0); + for frame in 0..FRAMES { + if frame.is_multiple_of(LINKED_SEGMENT) { + left_level = 2.0 + 6.0 * unit(&mut rng); + right_level = 2.0 + 6.0 * unit(&mut rng); + } + left[frame] = left_level * (unit(&mut rng) * 2.0 - 1.0); + right[frame] = right_level * (unit(&mut rng) * 2.0 - 1.0); + } +} + +/// Runs [`LINKED_CASE`] at width `L::WIDTH` through the prepared core. +fn run_linked(out: &mut [u32]) { + let width = L::WIDTH; + let mut left_lanes = vec![0.0_f32; LANES * FRAMES]; + let mut right_lanes = vec![0.0_f32; LANES * FRAMES]; + for lane in 0..LANES { + fill_linked( + lane, + &mut left_lanes[lane * FRAMES..(lane + 1) * FRAMES], + &mut right_lanes[lane * FRAMES..(lane + 1) * FRAMES], + ); + } + let quality = QUALITIES + .iter() + .find(|quality| quality.sample_rate == RATE) + .expect("launch rate"); + let mut left_block = vec![0.0_f32; LINKED_BLOCK * width]; + let mut right_block = vec![0.0_f32; LINKED_BLOCK * width]; + for group in 0..LANES / width { + // Both channels of a lane start equal: the pair is linked from block 0. + let defaults: Vec<[f32; PARAMETER_COUNT]> = (0..width) + .map(|offset| { + let lane = group * width + offset; + [ + LINKED_CEILINGS[lane], + LINKED_RELEASES[lane], + LINKED_LOOKAHEAD, + ] + }) + .collect(); + let bound = tail_and_rest(RATE, EffectQuality::Normal); + let metadata = PreparedEffectMetadata { + descriptor: &TRUE_PEAK_LIMITER_DESCRIPTOR, + sample_rate: RATE, + quantum: LINKED_BLOCK as u32, + quality: EffectQuality::Normal, + bypass: false, + link_mode: LinkMode::Maximum, + ports: PreparedPorts { + sidechain: PreparedSidechainPort::None, + }, + latency: quality.latency, + tail: bound.tail, + tail_every_peak: bound.tail_every_peak, + rest: bound.rest, + composition: bound.composition, + state_sizes: quality.maximum_state, + scratch_bytes: quality.scratch_fixed_bytes, + automation_capacity: 16, + }; + let mut core = LimiterCore::::new( + metadata, + defaults.clone().into_boxed_slice(), + defaults.into_boxed_slice(), + ) + .expect("corpus core"); + for block in 0..FRAMES / LINKED_BLOCK { + for retarget in LINKED_RETARGETS.iter().filter(|r| r.block == block) { + for offset in 0..width { + let lane = group * width + offset; + if retarget.lane.is_some_and(|target| target != lane) { + continue; + } + for (moves, state) in [ + (retarget.left, &mut core.left), + (retarget.right, &mut core.right), + ] { + if !moves { + continue; + } + match retarget.parameter { + LinkedParameter::Ceiling => state.limit[offset] + .set_target(limit_coefficient(retarget.value), RAMP_UPDATES), + LinkedParameter::Release => state.release[offset].set_target( + release_coefficient(retarget.value, RATE), + RAMP_UPDATES, + ), + } + } + } + } + let first = block * LINKED_BLOCK; + for frame in 0..LINKED_BLOCK { + for offset in 0..width { + let lane = group * width + offset; + left_block[frame * width + offset] = left_lanes[lane * FRAMES + first + frame]; + right_block[frame * width + offset] = + right_lanes[lane * FRAMES + first + frame]; + } + } + core.process_block(&mut left_block, &mut right_block, LINKED_BLOCK); + for frame in 0..LINKED_BLOCK { + for offset in 0..width { + let lane = group * width + offset; + out[lane * FRAMES + first + frame] = + left_block[frame * width + offset].to_bits(); + out[(LANES + lane) * FRAMES + first + frame] = + right_block[frame * width + offset].to_bits(); + } + } + } + } +} diff --git a/crates/true-peak-limiter/tests/MUTATIONS.md b/crates/true-peak-limiter/tests/MUTATIONS.md index e57aba4d9..55e8269f4 100644 --- a/crates/true-peak-limiter/tests/MUTATIONS.md +++ b/crates/true-peak-limiter/tests/MUTATIONS.md @@ -418,7 +418,7 @@ run printed the pin, `b22ef17c…`. |---|---|---|---| | 996-M1 | #990 M1: the mirrored backward pass skips its store into the right ring (`sliding_minimum_uniform_mirrored`) | channels stay equal: `ch00` parts at block 17 | `881bf62b…` | | 996-K1 | the pair keeps the shared path after it unlinks: the decision in `LimiterCore::process_block` drops `designed_gain_agree`, so a one-sided retarget renders linked | one-sided retarget parts nothing: `ch03` at block 17 | `4c664808…` | -| 996-K2 | #990 M12: `LimiterCore::process_block` no longer writes the decision back to `gain_linked`, so a pair relinks as soon as its designed words agree again, without reset, restore or `desymmetrize` | the pin | `198af660…` | +| 996-K2 | #990 M12: `LimiterCore::process_block` no longer writes the decision back to `gain_linked`, so a pair relinks as soon as its designed words agree again, without reset, restore or `desymmetrize` | the pin (since TA3: G5 case `maximum/link_unlink_relink`, see the TA3 section) | `198af660…` (historical: the digest of the deleted session pin) | | 996-M2 | #990 M2: the box term is not stored into the right box ring (`linked_frame_uniform`) | channels stay equal: `ch00` at block 17 | `a6cf49f7…` | | 996-M6 | #990 M6: the required gain is not stored into the right ring at the cursor (`linked_frame_uniform`) | channels stay equal: `ch00` at block 17 | `2abf0380…` | | 996-M7 | #990 M7: the right box sum is not set from the left at block end (`limiter_block_uniform`) | reduction floor | `c25993b2…` | @@ -609,3 +609,35 @@ What the rows say: * **M6 and M10 are payload-only defects**: the right channel's ring words go stale while every output word of a linked block is right, until the pair unlinks. The payload and complete-state comparisons after every block are what see them. + +## TA3 (2026-10-19) — the linked pair's words moved to gate G5 + +The test policy cut TA3 (`.github/ISSUE_SPECS/1726-finish-the-test-policy-cut.md`) deleted the +host-core session pin (`PIN`, `6d87267b…`) of `crates/host-core/src/limiter_linked_session.rs`. +Its session checks stay, renamed `the_hot_console_links_unlinks_and_limits_at_simd8`, `_at_simd4` +and `_at_scalar`; they still own the "equal", "parts nothing" and "floor" cells above. The cells +marked "pin" (996-K2, and 997-K2, M1, M2, M6, S0, SL at `Scalar`) now belong to G5 case 5 of +`src/corpus.rs`, `maximum/link_unlink_relink`: the prepared core, `LinkMode::Maximum`, eight +uniform lanes, a linked ramp at block 3, one-sided retargets of lane 0 and lane 7 at block 4, both +made equal again at block 5. Its pin is the scalar oracle's (`31b09672…`), and it is identical at +`Scalar`, `Simd4` and `Simd8`. + +Driver: one mutation at a time in `src/lib.rs`, then +`cargo test -p wasm-gates --test g5_native_corpus g5_native_digests_match_pins` (dev profile, an +`x86-64-v3` build, so all three widths), then the file restored with `git checkout`. + +| # | mutation | `Scalar` | `Simd4` | `Simd8` | +|---|---|---|---|---| +| TA3-NL | control: the decision is always `false` (the kernel never links, as before #990) | green | green | green | +| TA3-K2 | 996-K2 / 997-K2: the decision is not written back to `gain_linked` | RED | RED | RED | +| TA3-K1 | 996-K1: the decision drops `designed_gain_agree` (both sites) | RED | RED | RED | +| TA3-M1 | #990 M1: the mirrored backward pass skips its store into the right ring | RED | RED | RED | +| TA3-M2 | #990 M2: `linked_frame_uniform` does not store the box term into the right box ring | RED | RED | RED | +| TA3-M6 | #990 M6: `linked_frame_uniform` does not store the required gain into the right ring | RED | RED | RED | +| TA3-M9 | #990 M9: the right ramps are not set from the left at block end | green | green | green | +| TA3-E0 | `designed_gain_agree` skips lane 0 | RED | RED | green | + +NL green is the case's reference: its pin is the words of the pre-#990 behaviour. M9 is green +here; the session test's "equal" check (`ch16`) still owns it at every width. E0 is green at +`Simd8` because lanes 0 and 7 unlink in the same block, so the eight-lane bank unlinks through +lane 7; the session test's `ch48` row still owns E0 and EL at `Simd8`. diff --git a/scripts/test-test-support-ci.py b/scripts/test-test-support-ci.py index 626b2727a..f6abc54fa 100644 --- a/scripts/test-test-support-ci.py +++ b/scripts/test-test-support-ci.py @@ -158,7 +158,7 @@ def main() -> int: feature_b(f"{package}/test-support")) fails("every test-support feature removed from test-debug-a", {"builtins-compiler", "control-plane", "effect-compiler", "effect-contract", "engine", - "graph", "host-core", "host-web", "protocol", "rack"}, + "graph", "host-core", "host-web", "protocol"}, in_a(DEBUG_A_FEATURES, "--features engine/realtime-audit\n")) fails("every test-support feature removed from test-debug-b", {"builtins", "lane", "parametric-eq"}, diff --git a/tools/console-workload/examples/bus_send_profile.rs b/tools/console-workload/examples/bus_send_profile.rs deleted file mode 100644 index 2a84d276c..000000000 --- a/tools/console-workload/examples/bus_send_profile.rs +++ /dev/null @@ -1,397 +0,0 @@ -//! Issue #1229 D2: the untimed phase profile of the native bus-and-send row, -//! `sixty_four_track_console_sends`, at [`PlanConfig::BASELINE`] (the static plan: no meters, no -//! live controls, `Concurrent` builtins). -//! -//! An instrument, not a test: it gates nothing and asserts nothing about time. It drives -//! `graph::test_only_phase_profile` (test-support only, absent from every production build; this -//! crate's dev-dependency on `graph` enables it for examples as well as tests) the way -//! `tests/gain_pan_profile.rs`'s `phase_profile` does: 512 warmup blocks, then three repeats of -//! 4000 blocks with the probes off and on, the in-situ probe cost derived from the two, and every -//! graph phase and bank sub-phase reported net of the probes as the per-line median of the -//! repeats. Unlike that harness it prints every line, zero rows included, assumes no strip shape -//! (the slot positions are printed as `bank::slot_names()` reports them), derives no cycles, and -//! prints the plan census #1229 D5 reads: the bank shape, the route folds and, per bank chain, its -//! lanes, the submix lanes among them and the pad lanes of the bank width. -//! -//! Build it with the native console runner's profile overrides and run it once, pinned, under the -//! operator's timing lock (#1229 D2): -//! -//! ```text -//! CARGO_PROFILE_RELEASE_OPT_LEVEL=3 CARGO_PROFILE_RELEASE_LTO=false \ -//! CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 CARGO_INCREMENTAL=0 \ -//! cargo build --locked --release -p console-workload --example bus_send_profile -//! flock -w 7200 "$LOCK" taskset -c target/release/examples/bus_send_profile -//! ``` - -use std::collections::BTreeSet; -use std::time::Instant; - -use console_workload::{PlanConfig, QUANTUM, SessionRuntime, Workload}; -use graph::test_only_phase_profile as profile; -use profile::bank; - -/// The row under profile. -const ROW: Workload = Workload::SixtyFourTrackConsoleSends; -const WARMUP_BLOCKS: u64 = 512; -const PROFILED_BLOCKS: u64 = 4000; -const REPEATS: usize = 3; - -/// Graph phase labels, in `test_only_phase_profile` index order. -const PHASES: [&str; profile::COUNT] = [ - "enter", - "source", - "bound", - "route", - "output", - "identity-copy", - "identity-alias", - "other", - "bank", - "exit", -]; - -/// Median cost of one `Instant::now()` pair, the unit of probe overhead. -fn clock_read_ns() -> f64 { - let mut deltas: Vec = (0..20_000) - .map(|_| { - let a = Instant::now(); - let b = Instant::now(); - u64::try_from(b.duration_since(a).as_nanos()).unwrap_or(u64::MAX) - }) - .collect(); - deltas.sort_unstable(); - deltas[deltas.len() / 2] as f64 -} - -/// Renders one block and drains any meter it published, outside any clock. -fn render(runtime: &mut SessionRuntime, observation: &mut u64) { - runtime.render(*observation).expect("console render"); - runtime.drain_meters(); - *observation += 1; -} - -/// Every chain sub-phase row of the report, zero rows included: `(label, sub-phase index)`. -fn sub_phase_rows() -> Vec<(String, usize)> { - let mut rows = vec![ - ("prologue".to_owned(), bank::PROLOGUE), - ("gather".to_owned(), bank::GATHER), - ]; - rows.extend( - (0..bank::SLOTS).map(|position| (format!("slot {position}"), bank::SLOT + position)), - ); - rows.extend([ - ("seam".to_owned(), bank::SEAM), - ("scatter".to_owned(), bank::SCATTER), - ("fold".to_owned(), bank::FOLD), - ("aux".to_owned(), bank::AUX), - ]); - rows -} - -/// One repeat, per block. -struct Repeat { - /// Graph phases, probed nanoseconds. - phases: [f64; profile::COUNT], - /// Graph phase entries (probes that began each phase), derived from `profile::runs()`. - phase_entries: [f64; profile::COUNT], - /// Chain sub-phases, probed nanoseconds. - sub_phases: [f64; bank::COUNT], - /// Chain sub-phase entries. - sub_entries: [f64; bank::COUNT], - /// Clock reads the chain probes took. - chain_reads: f64, - /// Clock reads every probe took, the graph's and the chain's. - reads: f64, - /// In-situ cost of one probe: the probed mean less the unprobed mean, over the clock reads. - probe_ns: f64, -} - -impl Repeat { - fn net_phase(&self, index: usize) -> f64 { - let nested = if index == profile::BANK { - self.chain_reads - } else { - 0.0 - }; - self.phases[index] - (self.phase_entries[index] + nested) * self.probe_ns - } - - fn net_sub_phase(&self, index: usize) -> f64 { - self.sub_phases[index] - self.sub_entries[index] * self.probe_ns - } - - /// What the bank phase holds outside the chain runs: the unit's dispatch, member setup and - /// observation (and the chain probes' closing reads). - fn outside_chain(&self, net: bool) -> f64 { - if net { - self.net_phase(profile::BANK) - - (0..bank::COUNT) - .map(|index| self.net_sub_phase(index)) - .sum::() - } else { - self.phases[profile::BANK] - self.sub_phases.iter().sum::() - } - } -} - -fn median(mut values: Vec) -> f64 { - values.sort_by(f64::total_cmp); - values[values.len() / 2] -} - -/// Renders `PROFILED_BLOCKS` blocks and returns the mean nanoseconds per block. -fn timed_loop(runtime: &mut SessionRuntime, observation: &mut u64) -> f64 { - let start = Instant::now(); - for _ in 0..PROFILED_BLOCKS { - render(runtime, observation); - } - start.elapsed().as_nanos() as f64 / PROFILED_BLOCKS as f64 -} - -/// The plan census (#1229 D5), printed before anything is timed. -fn print_census(runtime: &SessionRuntime) { - let [chains, slots] = runtime.bank_shape(); - let units = runtime.unit_eligibility(); - let banked = units.iter().filter(|unit| unit.banked).count(); - let width = lane::Backend::current().width(); - println!( - "census: bank_shape [chains, slots] = [{chains}, {slots}], units = {} \ - ({banked} bank chains, {} single ops), bank_route_folds = {}, bank width = {width}", - units.len(), - units.len() - banked, - runtime.bank_route_folds() - ); - // The fixture's tracks are exactly the structural-symmetry rows (keyed by track id, tracks - // only); a non-empty lane name outside them is a submix strip's lane (since #1200). - let tracks: BTreeSet<&str> = runtime - .structural_symmetry() - .iter() - .map(|(track, _)| &**track) - .collect(); - println!("census: {} tracks", tracks.len()); - for unit in units.iter().filter(|unit| unit.banked) { - let active = unit.lanes() as usize; - let submix_lanes: Vec<&str> = unit - .lane_tracks - .iter() - .map(|lane| &**lane) - .filter(|lane| !lane.is_empty() && !tracks.contains(lane)) - .collect(); - let kind = match submix_lanes.len() { - 0 => "track", - count if count == active => "submix", - _ => "mixed", - }; - println!( - "census chain: unit {} {kind} stages {} active lanes {active} pad lanes {} lanes [{}]", - unit.unit, - unit.stages, - active.div_ceil(width) * width - active, - unit.lane_tracks - .iter() - .map(|lane| &**lane) - .collect::>() - .join(" ") - ); - } -} - -fn main() { - let clock_ns = clock_read_ns(); - let tracks = ROW.tracks(); - println!( - "row {}, backend {:?}, quantum {QUANTUM}, {tracks} tracks, PlanConfig::BASELINE", - ROW.kind(), - lane::Backend::current() - ); - println!("clock read {clock_ns:.1} ns (median Instant::now() pair)"); - - let mut runtime = SessionRuntime::build(ROW, PlanConfig::BASELINE); - print_census(&runtime); - let [chains, _] = runtime.bank_shape(); - let mut observation = 0_u64; - profile::enable(false); - for _ in 0..WARMUP_BLOCKS { - render(&mut runtime, &mut observation); - } - - let per_block = |total: u64| total as f64 / PROFILED_BLOCKS as f64; - let mut repeats = Vec::with_capacity(REPEATS); - let mut off_p50 = Vec::with_capacity(REPEATS); - let mut off_mean = Vec::with_capacity(REPEATS); - let mut graph_probes_per_block = Vec::with_capacity(REPEATS); - for repeat in 0..REPEATS { - // Probes compiled in but off: the row's own cost, block by block, then as one loop. - let mut samples = Vec::with_capacity(PROFILED_BLOCKS as usize); - for _ in 0..PROFILED_BLOCKS { - let start = Instant::now(); - let result = runtime.render(observation); - let elapsed = u64::try_from(start.elapsed().as_nanos()).unwrap_or(u64::MAX); - result.expect("console render"); - runtime.drain_meters(); - observation += 1; - samples.push(elapsed); - } - samples.sort_unstable(); - let p50 = bench_support::stats::per_mille(&samples, 500); - off_p50.push(p50 as f64); - let mean_off = timed_loop(&mut runtime, &mut observation); - off_mean.push(mean_off); - println!( - "repeat {repeat}: probes off min {} ns p50 {p50} ns p95 {} ns mean {mean_off:.0} ns", - samples[0], - bench_support::stats::per_mille(&samples, 950), - ); - - // Probes on: the same loop, charged to phases and sub-phases. - profile::reset(); - profile::enable(true); - let mean_on = timed_loop(&mut runtime, &mut observation); - profile::enable(false); - let (nanos, blocks, graph_reads) = profile::snapshot(); - let chain = bank::snapshot(); - // Printed, not asserted: a measurement has already been printed, so a failure here is a - // finding to record, never a reason to run again. - if blocks != PROFILED_BLOCKS || chain.chain_runs != PROFILED_BLOCKS * chains { - println!( - "repeat {repeat}: NOTE profiled blocks {blocks} (expected {PROFILED_BLOCKS}), \ - chain runs {} (expected {})", - chain.chain_runs, - PROFILED_BLOCKS * chains - ); - } - // The graph's phases are entered once per block each (entry, source), and a unit kind - // once per run of it in the schedule, which is fixed at bind. - let runs = profile::runs(); - let used: Vec<(usize, u64)> = runs - .iter() - .copied() - .filter(|&(kind, units)| kind < profile::COUNT && units > 0) - .collect(); - let mut phase_entries = [0.0; profile::COUNT]; - phase_entries[profile::ENTER] = 1.0; - phase_entries[profile::SOURCE] = 1.0; - for &(kind, _) in &used { - phase_entries[kind] += 1.0; - } - let graph_probes = per_block(graph_reads); - graph_probes_per_block.push(graph_probes); - if repeat == 0 { - let listed: Vec = used - .iter() - .map(|&(kind, units)| format!("{units} x {}", PHASES[kind])) - .collect(); - println!( - "unit runs in schedule order ({} of {} entries used): {}", - used.len(), - profile::RUNS_CAPACITY, - listed.join(", ") - ); - // A block's graph probes are its start, the source entry, one per unit-kind run and - // the finish, so a runs list that holds every run accounts for all of them. - println!( - "graph probes per block {graph_probes:.1}; start + source + runs + finish = {}{}", - used.len() + 3, - if used.len() == profile::RUNS_CAPACITY { - "; NOTE every runs() entry is used, so the entry counts and the net figures \ - may be truncated: read the probed figures" - } else { - "" - } - ); - for (position, name) in bank::slot_names().iter().enumerate() { - println!( - "slot {position}: {}", - if name.is_empty() { - "(not reached)" - } else { - name - } - ); - } - } - let reads = per_block(graph_reads + chain.probes); - let probe_ns = (mean_on - mean_off) / reads; - println!( - "repeat {repeat}: probes on mean {mean_on:.0} ns/block, \ - {reads:.1} clock reads/block ({graph_probes:.1} graph, {:.1} chain), \ - {probe_ns:.1} ns per probe in situ (clock read alone {clock_ns:.1} ns)", - per_block(chain.probes) - ); - repeats.push(Repeat { - phases: nanos.map(per_block), - phase_entries, - sub_phases: chain.nanos.map(per_block), - sub_entries: chain.entries.map(per_block), - chain_reads: per_block(chain.probes), - reads, - probe_ns, - }); - } - - // The report: per line, the median over the repeats. "Probed" is what the probes charged; - // "net" subtracts one in-situ probe cost per entry (and, for the bank phase, every chain probe - // nested in it), an estimate of the unprobed cost, not a measurement of it. Every share has the - // one denominator `net_total`, the sum of every graph phase's net time: the profiled block time. - let over = |value: &dyn Fn(&Repeat) -> f64| median(repeats.iter().map(value).collect()); - let net_total = over(&|r| (0..profile::COUNT).map(|i| r.net_phase(i)).sum()); - let row = |label: &str, entries: f64, probed: f64, net: f64| { - println!( - "| {label:<30} | {entries:>8.1} | {probed:>10.1} | {net:>10.1} | {:>6.2}% |", - 100.0 * net / net_total - ); - }; - println!(); - println!( - "median of {REPEATS} repeats of {PROFILED_BLOCKS} blocks; probes off: p50 {:.0} ns/block, \ - loop mean {:.0} ns/block; probe cost {:.1} ns in situ; {:.1} clock reads/block \ - ({:.1} graph probes/block); profiled block time (net_total) {net_total:.1} ns", - median(off_p50), - median(off_mean), - over(&|r| r.probe_ns), - over(&|r| r.reads), - median(graph_probes_per_block) - ); - println!( - "| {:<30} | {:>8} | {:>10} | {:>10} | {:>7} |", - "phase", "entries", "probed ns", "net ns", "share" - ); - println!( - "|{}|{}|{}|{}|{}|", - "-".repeat(32), - "-".repeat(10), - "-".repeat(12), - "-".repeat(12), - "-".repeat(9) - ); - for (index, name) in PHASES.iter().enumerate() { - let entries = over(&|r| r.phase_entries[index]); - let probed = over(&|r| r.phases[index]); - let net = over(&|r| r.net_phase(index)); - if index != profile::BANK { - row(name, entries, probed, net); - continue; - } - row("bank (whole unit)", entries, probed, net); - for (label, sub) in sub_phase_rows() { - row( - &format!(" {label}"), - over(&|r| r.sub_entries[sub]), - over(&|r| r.sub_phases[sub]), - over(&|r| r.net_sub_phase(sub)), - ); - } - row( - " outside chain", - entries, - over(&|r| r.outside_chain(false)), - over(&|r| r.outside_chain(true)), - ); - } - row( - "total (entries: clock reads)", - over(&|r| r.reads), - over(&|r| r.phases.iter().sum()), - net_total, - ); -} diff --git a/tools/console-workload/tests/gain_pan_profile.rs b/tools/console-workload/tests/gain_pan_profile.rs deleted file mode 100644 index b72a222f4..000000000 --- a/tools/console-workload/tests/gain_pan_profile.rs +++ /dev/null @@ -1,504 +0,0 @@ -//! Issue #960: the phase profile of the native pure-audio-path target, -//! `sixty_four_track_gain_pan_ring`. -//! -//! It replaces the deleted `plumbing_profile.rs`, whose builtins-less rows no host compiles, with a -//! profile of the row a host actually feeds: sixty-four gain/pan strips compiled with builtins and -//! fed through a prepared source set whose claims every bank gather reads in place. -//! -//! Two instruments, both test-support only and both absent from every production build: -//! -//! * `graph::test_only_phase_profile` charges each block to the executor's phases (entry, source, -//! the unit kinds, exit), taking a probe only where the unit kind changes; -//! * its `bank` module splits every bank unit's chain run at the chain's own boundaries: the -//! prologue (queue drains and the collapse decision), the gather, each slot by its chain -//! position, the collapse seam, the scatter, and the route/master fold the members perform. -//! What the `BANK` phase holds beyond those is the unit's dispatch, member setup and -//! observation, reported as "outside chain". -//! -//! `phase_profile` is ignored by default. It prints measurements and asserts nothing about time: -//! the numbers are descriptive, taken on whatever host runs it, and are not the console benchmark. -//! Run it pinned, on an otherwise quiet core: -//! -//! ```text -//! CARGO_INCREMENTAL=0 cargo test --release -p console-workload --test gain_pan_profile --no-run -//! taskset -c 31 target/release/deps/gain_pan_profile- --ignored --nocapture \ -//! --test-threads 1 phase_profile -//! ``` -//! -//! A builtin slot shows the graph's `BuiltinStage` wrapper as its type. On this row every chain -//! is one cohort's builtin strip in its fixed order (AGENTS.md: input section, fader, pan -//! matrix), so slot 0 is the input section, slot 1 the fader and slot 2 the pan matrix. -//! -//! `probes_split_the_bank_units_and_move_no_bit` is the standing, untimed gate: the probes fire -//! where they claim to and render the row's bits. -//! -//! Cycles are derived, not counted: the core clock is calibrated from a dependent chain of scalar -//! `f32` adds at [`FADD_LATENCY_CYCLES`] cycles each (Zen 3), timed by the same monotonic clock the -//! probes read. - -use std::hint::black_box; -use std::time::Instant; - -use bench_support::digest::Sha256Sink; -use console_workload::{PlanConfig, QUANTUM, SessionRuntime, Workload}; -use graph::test_only_phase_profile as profile; -use profile::bank; - -/// The row under profile. -const ROW: Workload = Workload::SixtyFourTrackGainPanRing; -const WARMUP_BLOCKS: u64 = 512; -const PROFILED_BLOCKS: u64 = 4000; -const REPEATS: usize = 3; -/// Dependent `addss` latency on the Zen 3 core this harness was first run on. -const FADD_LATENCY_CYCLES: f64 = 3.0; - -/// Graph phase labels, in `test_only_phase_profile` index order. -const PHASES: [&str; profile::COUNT] = [ - "enter", - "source", - "bound", - "route", - "output", - "identity-copy", - "identity-alias", - "other", - "bank", - "exit", -]; - -/// Median cost of one `Instant::now()` pair, the unit of probe overhead. -fn clock_read_ns() -> f64 { - let mut deltas: Vec = (0..20_000) - .map(|_| { - let a = Instant::now(); - let b = Instant::now(); - u64::try_from(b.duration_since(a).as_nanos()).unwrap_or(u64::MAX) - }) - .collect(); - deltas.sort_unstable(); - deltas[deltas.len() / 2] as f64 -} - -/// Core clock from a dependent `f32` add chain: [`FADD_LATENCY_CYCLES`] cycles per add. -fn core_hz() -> f64 { - const ADDS: u64 = 300_000_000; - let mut best = f64::MAX; - for _ in 0..3 { - let mut x = black_box(0.0_f32); - let d = black_box(1.0e-3_f32); - let start = Instant::now(); - for _ in 0..ADDS { - x += d; - } - let elapsed = start.elapsed().as_secs_f64(); - black_box(x); - best = best.min(elapsed); - } - FADD_LATENCY_CYCLES * ADDS as f64 / best -} - -/// Renders one block and drains any meter it published, outside any clock. -fn render(runtime: &mut SessionRuntime, observation: &mut u64) { - runtime.render(*observation).expect("console render"); - runtime.drain_meters(); - *observation += 1; -} - -/// The chain sub-phase rows of the report: `(label, sub-phase index)`, slots named by the stage -/// type the first profiled chain ran there. -fn sub_phase_rows(names: &[&'static str; bank::SLOTS]) -> Vec<(String, usize)> { - let mut rows = vec![ - ("prologue".to_owned(), bank::PROLOGUE), - ("gather".to_owned(), bank::GATHER), - ]; - for (position, name) in names.iter().enumerate() { - if !name.is_empty() { - rows.push(( - format!("slot {position} ({})", short_type_name(name)), - bank::SLOT + position, - )); - } - } - rows.extend([ - ("seam".to_owned(), bank::SEAM), - ("scatter".to_owned(), bank::SCATTER), - ("fold".to_owned(), bank::FOLD), - ("aux".to_owned(), bank::AUX), - ]); - rows -} - -/// `a::b::Stage` as `Stage`: the last path segment of every type in the name. -fn short_type_name(name: &str) -> String { - let mut short = String::new(); - let mut segment = String::new(); - for character in name.chars() { - if character.is_alphanumeric() || character == '_' || character == ':' { - segment.push(character); - } else { - short.push_str(segment.rsplit("::").next().unwrap_or("")); - segment.clear(); - short.push(character); - } - } - short.push_str(segment.rsplit("::").next().unwrap_or("")); - short -} - -/// One repeat, per block. -struct Repeat { - /// Graph phases, probed nanoseconds. - phases: [f64; profile::COUNT], - /// Graph phase entries (probes that began each phase). - phase_entries: [f64; profile::COUNT], - /// Chain sub-phases, probed nanoseconds. - sub_phases: [f64; bank::COUNT], - /// Chain sub-phase entries. - sub_entries: [f64; bank::COUNT], - /// Clock reads the chain probes took. - chain_reads: f64, - /// Clock reads every probe took, the graph's and the chain's. - reads: f64, - /// In-situ cost of one probe: the probed mean less the unprobed mean, over the clock reads. - probe_ns: f64, -} - -impl Repeat { - fn net_phase(&self, index: usize) -> f64 { - let nested = if index == profile::BANK { - self.chain_reads - } else { - 0.0 - }; - self.phases[index] - (self.phase_entries[index] + nested) * self.probe_ns - } - - fn net_sub_phase(&self, index: usize) -> f64 { - self.sub_phases[index] - self.sub_entries[index] * self.probe_ns - } - - /// What the bank phase holds outside the chain runs: the unit's dispatch, member setup and - /// observation (and the chain probes' closing reads). - fn outside_chain(&self, net: bool) -> f64 { - if net { - self.net_phase(profile::BANK) - - (0..bank::COUNT) - .map(|index| self.net_sub_phase(index)) - .sum::() - } else { - self.phases[profile::BANK] - self.sub_phases.iter().sum::() - } - } -} - -fn median(mut values: Vec) -> f64 { - values.sort_by(f64::total_cmp); - values[values.len() / 2] -} - -/// Renders `PROFILED_BLOCKS` blocks and returns the mean nanoseconds per block. -fn timed_loop(runtime: &mut SessionRuntime, observation: &mut u64) -> f64 { - let start = Instant::now(); - for _ in 0..PROFILED_BLOCKS { - render(runtime, observation); - } - start.elapsed().as_nanos() as f64 / PROFILED_BLOCKS as f64 -} - -#[test] -#[ignore = "measurement harness; run pinned with --ignored --nocapture"] -fn phase_profile() { - let hz = core_hz(); - let clock_ns = clock_read_ns(); - let tracks = u64::from(ROW.tracks()); - let lane_samples = (tracks * QUANTUM as u64 * 2) as f64; - let cycles = |ns: f64| ns * hz / 1e9; - println!( - "row {}, backend {:?}, quantum {QUANTUM}, {tracks} tracks", - ROW.kind(), - lane::Backend::current() - ); - println!( - "core clock {:.3} GHz (derived: dependent f32 add chain at {FADD_LATENCY_CYCLES} cycles), \ - clock read {clock_ns:.1} ns", - hz / 1e9 - ); - - let mut runtime = SessionRuntime::build(ROW, PlanConfig::BASELINE); - let [chains, slots] = runtime.bank_shape(); - assert_eq!( - slots, - 3 * chains, - "the slot labels assume three builtin stages per chain: input section, fader, pan matrix" - ); - println!( - "bank_shape [chains, slots] = [{chains}, {slots}], units = {}, route folds = {}", - runtime.unit_eligibility().len(), - runtime.bank_route_folds() - ); - let mut observation = 0_u64; - profile::enable(false); - for _ in 0..WARMUP_BLOCKS { - render(&mut runtime, &mut observation); - } - - let per_block = |total: u64| total as f64 / PROFILED_BLOCKS as f64; - let mut repeats = Vec::with_capacity(REPEATS); - let mut off_p50 = Vec::with_capacity(REPEATS); - let mut off_mean = Vec::with_capacity(REPEATS); - let mut names = [""; bank::SLOTS]; - for repeat in 0..REPEATS { - // Probes compiled in but off: the row's own cost, block by block, then as one loop. - let mut samples = Vec::with_capacity(PROFILED_BLOCKS as usize); - for _ in 0..PROFILED_BLOCKS { - let start = Instant::now(); - let result = runtime.render(observation); - let elapsed = u64::try_from(start.elapsed().as_nanos()).unwrap_or(u64::MAX); - result.expect("console render"); - runtime.drain_meters(); - observation += 1; - samples.push(elapsed); - } - samples.sort_unstable(); - let p50 = bench_support::stats::per_mille(&samples, 500); - off_p50.push(p50 as f64); - let mean_off = timed_loop(&mut runtime, &mut observation); - off_mean.push(mean_off); - println!( - "repeat {repeat}: probes off min {} ns p50 {p50} ns p95 {} ns \ - mean {mean_off:.0} ns \ - (p50 = {:.0} cycles/block, {:.3} cycles/lane-sample)", - samples[0], - bench_support::stats::per_mille(&samples, 950), - cycles(p50 as f64), - cycles(p50 as f64) / lane_samples - ); - - // Probes on: the same loop, charged to phases and sub-phases. - profile::reset(); - profile::enable(true); - let mean_on = timed_loop(&mut runtime, &mut observation); - profile::enable(false); - let (nanos, blocks, graph_reads) = profile::snapshot(); - let chain = bank::snapshot(); - assert_eq!( - blocks, PROFILED_BLOCKS, - "every profiled block closed its probe" - ); - assert_eq!( - chain.chain_runs, - PROFILED_BLOCKS * chains, - "every chain run closed its probe" - ); - // The graph's phases are entered once per block each (entry, source), and a unit kind - // once per run of it in the schedule, which is fixed at bind. - let mut phase_entries = [0.0; profile::COUNT]; - phase_entries[profile::ENTER] = 1.0; - phase_entries[profile::SOURCE] = 1.0; - for (kind, units) in profile::runs() { - if kind < profile::COUNT && units > 0 { - phase_entries[kind] += 1.0; - } - } - if repeat == 0 { - let runs: Vec = profile::runs() - .iter() - .filter(|(kind, units)| *kind < profile::COUNT && *units > 0) - .map(|(kind, units)| format!("{units} x {}", PHASES[*kind])) - .collect(); - println!("unit runs in schedule order: {}", runs.join(", ")); - names = bank::slot_names(); - for (position, name) in names.iter().enumerate() { - if !name.is_empty() { - println!("slot {position}: {name}"); - } - } - } - let reads = per_block(graph_reads + chain.probes); - let probe_ns = (mean_on - mean_off) / reads; - println!( - "repeat {repeat}: probes on mean {mean_on:.0} ns/block, \ - {reads:.1} clock reads/block, \ - {probe_ns:.1} ns per probe in situ (clock read alone {clock_ns:.1} ns)" - ); - repeats.push(Repeat { - phases: nanos.map(per_block), - phase_entries, - sub_phases: chain.nanos.map(per_block), - sub_entries: chain.entries.map(per_block), - chain_reads: per_block(chain.probes), - reads, - probe_ns, - }); - } - - // The report: per row, the median over the repeats. "Probed" is what the probes charged; "net" - // subtracts one in-situ probe cost per entry (and, for the bank phase, every chain probe nested - // in it), which is an estimate of the unprobed cost, not a measurement of it. - let over = |value: &dyn Fn(&Repeat) -> f64| median(repeats.iter().map(value).collect()); - let net_total = over(&|r| (0..profile::COUNT).map(|i| r.net_phase(i)).sum()); - let row = |label: &str, entries: f64, probed: f64, net: f64| { - println!( - "| {label:<30} | {entries:>7.1} | {probed:>9.1} | {net:>9.1} \ - | {:>9.0} | {:>7.3} | {:>5.1}% |", - cycles(net), - cycles(net) / lane_samples, - 100.0 * net / net_total - ); - }; - println!(); - println!( - "median of {REPEATS} repeats, per block of {lane_samples} lane-samples; \ - probes off: p50 {:.0} ns \ - = {:.0} cycles, loop mean {:.0} ns; probe cost {:.1} ns in situ", - median(off_p50.clone()), - cycles(median(off_p50)), - median(off_mean), - over(&|r| r.probe_ns) - ); - println!( - "| {:<30} | {:>7} | {:>9} | {:>9} | {:>9} | {:>7} | {:>6} |", - "phase", "entries", "probed ns", "net ns", "net cyc", "net c/ls", "share" - ); - println!( - "|{}|{}|{}|{}|{}|{}|{}|", - "-".repeat(32), - "-".repeat(9), - "-".repeat(11), - "-".repeat(11), - "-".repeat(11), - "-".repeat(9), - "-".repeat(8) - ); - for (index, name) in PHASES.iter().enumerate() { - let probed = over(&|r| r.phases[index]); - if probed == 0.0 { - continue; - } - let entries = over(&|r| r.phase_entries[index]); - let net = over(&|r| r.net_phase(index)); - if index != profile::BANK { - row(name, entries, probed, net); - continue; - } - row("bank (whole unit)", entries, probed, net); - for (label, sub) in sub_phase_rows(&names) { - let probed = over(&|r| r.sub_phases[sub]); - if probed > 0.0 { - row( - &format!(" {label}"), - over(&|r| r.sub_entries[sub]), - probed, - over(&|r| r.net_sub_phase(sub)), - ); - } - } - row( - " outside chain", - entries, - over(&|r| r.outside_chain(false)), - over(&|r| r.outside_chain(true)), - ); - } - row( - "total (entries: clock reads)", - over(&|r| r.reads), - over(&|r| r.phases.iter().sum()), - net_total, - ); -} - -/// Renders `blocks` blocks of `workload` and returns the digest of everything it emitted. -fn digest(workload: Workload, blocks: u64) -> String { - let mut runtime = SessionRuntime::build(workload, PlanConfig::BASELINE); - let mut hash = Sha256Sink::new(); - let mut observation = 0; - for _ in 0..blocks { - render(&mut runtime, &mut observation); - runtime.hash_output(&mut hash); - } - hash.finish_hex() -} - -/// The probes fire where they claim to, and a profiled render is the row's own bits. -/// -/// Untimed. Every chain run opens and closes one sub-phase profile, gathers once and scatters -/// once; the row folds every route, so every chain run enters the fold; the chain sub-phases nest -/// inside the graph's `BANK` phase; and the digest with every probe on is the digest with every -/// probe off. -#[test] -fn probes_split_the_bank_units_and_move_no_bit() { - const BLOCKS: u64 = 16; - profile::enable(false); - let off = digest(ROW, BLOCKS); - - let mut runtime = SessionRuntime::build(ROW, PlanConfig::BASELINE); - let [chains, slots] = runtime.bank_shape(); - assert!(chains > 0, "the row binds bank chains"); - assert_eq!( - slots, - 3 * chains, - "the slot labels assume three builtin stages per chain: input section, fader, pan matrix" - ); - assert_eq!( - runtime.bank_route_folds(), - u64::from(ROW.tracks()), - "the row folds every route" - ); - let mut hash = Sha256Sink::new(); - let mut observation = 0; - profile::reset(); - profile::enable(true); - for _ in 0..BLOCKS { - render(&mut runtime, &mut observation); - runtime.hash_output(&mut hash); - } - profile::enable(false); - assert_eq!(hash.finish_hex(), off, "the probes moved a bit"); - - let (nanos, blocks, _) = profile::snapshot(); - let chain = bank::snapshot(); - let runs = BLOCKS * chains; - assert_eq!(blocks, BLOCKS); - assert_eq!(chain.chain_runs, runs, "one closed profile per chain run"); - assert_eq!( - chain.entries[bank::PROLOGUE], - runs, - "one prologue per chain run" - ); - assert_eq!( - chain.entries[bank::GATHER], - runs, - "one gather per chain run" - ); - assert_eq!( - chain.entries[bank::SCATTER], - runs, - "one scatter per chain run" - ); - assert_eq!( - chain.entries[bank::FOLD], - runs, - "one fold per folded chain run" - ); - assert_eq!(chain.entries[bank::SEAM], 0, "the row does not collapse"); - let slot_entries: u64 = chain.entries[bank::SLOT..bank::SLOT + bank::SLOTS] - .iter() - .sum(); - assert!( - slot_entries > 0 && slot_entries <= BLOCKS * slots, - "each slot with an active lane is entered once per run ({slot_entries} of at most {})", - BLOCKS * slots - ); - for (position, name) in bank::slot_names().iter().enumerate() { - assert_eq!( - name.is_empty(), - chain.entries[bank::SLOT + position] == 0, - "slot {position} is named exactly when it ran" - ); - } - assert!( - chain.nanos.iter().sum::() <= nanos[profile::BANK], - "the chain sub-phases nest inside the bank phase" - ); -} diff --git a/tools/wasm-gate-corpus/src/lib.rs b/tools/wasm-gate-corpus/src/lib.rs index ac0a9dd1b..26c6daee5 100644 --- a/tools/wasm-gate-corpus/src/lib.rs +++ b/tools/wasm-gate-corpus/src/lib.rs @@ -35,8 +35,10 @@ //! //! An effect crate's cases are appended, never inserted: [`LANE_DIGESTS`] is indexed by case //! number, so a new block of cases has to go on the end for the existing pins to keep describing -//! the same computations. Issue #943 appended one lane case, the banked sample-peak meter kernel, -//! after the element-wise block: it extends [`LANE_DIGESTS`] at its end, so every existing lane +//! the same computations. A delegated crate may append a case inside its own block; the blocks +//! after it then move up by one, as described next for #943 (TA3 did this for the limiter). +//! Issue #943 appended one lane case, the banked sample-peak meter kernel, after the element-wise +//! block: it extends [`LANE_DIGESTS`] at its end, so every existing lane //! pin keeps its index, while the delegated blocks after it -- whose pins live in their own crates //! and are indexed there -- move up by one. The original layout ends with the compressor family of issue #88. //! Issue #213 appends two builtin cases after that family, preserving the original eight-case @@ -221,8 +223,11 @@ pub const BUILTINS_ORIGINAL_CASE_COUNT: usize = 8; /// summed box ramp on its `2^-14` grid, and the two-rounding multiply/add one-pole on the reduction /// word with its D7 `flush`. One case per link mode, one at the `W_MIN` window with near-Nyquist /// input, one at the longest lookahead, and one of subnormal input between bursts, which is where -/// the flush has to remove the same bits on every target. Lane generic, and their pins live in that -/// crate. +/// the flush has to remove the same bits on every target. A sixth case (TA3) drives the prepared +/// core block by block through the #990 linked pair: a linked ramp, one-sided unlinks on the bank's +/// edge lanes and pairs made equal again. Appending it moved the compressor, builtin-tail and +/// automation blocks up by one index; their pins are indexed in their own crates. Lane generic, and +/// their pins live in that crate. pub const LIMITER_CASE_COUNT: usize = limiter_corpus::CASE_COUNT; /// Cases delegated to [`compressor::corpus`] (issue #88 E4), replayed under wasm.