From 564a315ef5421363d8b302454713065438d92240 Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 04:16:12 +0000 Subject: [PATCH 01/10] #1316: anchor every seek on the plan's source-read clock and report where it landed - begin_block_at's anchor is the source-read sample (equal to the render clock in this version). - Render publishes one seek report per ring through the engine's one single-writer seqlock (new crates/engine/src/realtime/seqlock.rs, B15-R1; watermark.rs moved onto it): generation in effect, source-read sample and source frame of its first block, cumulative underrun and held frames. - C ABI: miso_engine_v1_source_seek_report (struct miso_engine_v1_seek_report, 64 bytes; feature bit 1 << 8, mask 0x1ff); frozen symbol list 18. Browser export is out of scope per the spec (#1332, #1293). - seek_report_bytes resource row (64 B per ring on 64-bit, 56 on wasm32); no ceiling moved. realtime policy floors 32/150 -> 33/151. audit capi pcm_digest unchanged (4f52089e4a8cf83c). Co-Authored-By: Claude Opus 5.5 --- ...ery-seek-on-the-plans-source-read-clock.md | 72 ++++ crates/capi/include/miso_engine_v1.h | 33 +- crates/capi/src/abi.rs | 48 ++- crates/capi/src/ffi.rs | 224 ++++++++++- crates/capi/src/runtime/tests.rs | 205 ++++++++++ crates/capi/tests/c/abi_smoke.c | 46 ++- crates/capi/tests/c/header_smoke.cpp | 16 + crates/control-plane/src/control.rs | 11 + crates/control-plane/src/lib.rs | 1 + crates/engine/src/realtime/mod.rs | 5 + crates/engine/src/realtime/seqlock.rs | 202 ++++++++++ crates/engine/src/realtime/watermark.rs | 216 +++-------- crates/host-core/src/source.rs | 41 +- crates/host-core/tests/source_diagnostics.rs | 7 + crates/source/src/lib.rs | 352 ++++++++++++++++-- docs/C_ABI_V1_QUALIFICATION.md | 28 ++ scripts/check-capi-abi.sh | 1 + tools/realtime-policy/src/policy.rs | 11 +- 18 files changed, 1295 insertions(+), 224 deletions(-) create mode 100644 crates/engine/src/realtime/seqlock.rs diff --git a/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md b/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md index cdffb0a52..1d4b2d326 100644 --- a/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md +++ b/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md @@ -238,3 +238,75 @@ Re-pin from the measurement, with the reason. **Realtime-path edits.** D2 edits `crates/engine/src/realtime/watermark.rs` and `mod.rs` and adds the new `seqlock.rs` (B15-R1). D3 adds a render-side publication in the source consumer. That slice gets one attempt. + +## Decision record: #1316 (batch 15c) + +**Shipped.** D1-D8 and B15-R1/R2 on `460fca68b`. + +- `crates/engine/src/realtime/seqlock.rs` (new, B15-R1): `PublishedWords`, a + non-`Clone` `PublishedWordsWriter::publish(&mut self, &[u64; N])` (marked realtime region, + wait-free) and a `Clone` `PublishedWordsReader::read` bounded at `MAXIMUM_READ_ATTEMPTS` (64), + then `PublishedWordsBusy`. No `unsafe`. `watermark.rs` now stores its six words through it; its + record size is unchanged (72 bytes). #1314's torn-read test moved to `seqlock.rs` with the + protocol and the watermark copy was deleted. +- `crates/source`: `begin_block_at(source_read_sample)` and `SeekClock::At { source_read_sample }` + with D1's text; `SourceCommand::SeekAt` and `HeldSeek` anchors documented on the source-read + clock. `SourceSeekReport` (five words) and `SourceSeekReportBusy`; one record per ring made in + `prepare_at_source_frame`, writer in the consumer, reader in the producer + (`PcmSourceProducer::seek_report`, `HostChunkProvider::seek_report`). D3's publication is + `publish_seek_report` at the end of `begin_block_play`, inside the existing marked region. + `SourceResourceReport::seek_report_bytes` is in overhead, total and the largest-allocation list. +- host-core: `SourceControlSet::seek_report`, the `seek_at` doc (D7), and a new + `SourceControlError::ReportBusy` (`source.report.busy`, backpressure). +- control-plane: `SessionState::seek_report` runs `service()` first (amendment), reads the newest + providers; `pub use source::SourceSeekReport`. +- capi: `miso_engine_v1_source_seek_report`, `SOURCE_SEEK_REPORT_SIZE` 64, + `FEATURE_SOURCE_SEEK_REPORT` = `1 << 8` (next free bit; mask 511), header thread table, service + paragraph and a two-sentence note, both smoke programs, the frozen list (18 symbols) and + `docs/C_ABI_V1_QUALIFICATION.md`. + +**Departures.** + +- The C struct is `miso_engine_v1_seek_report`, not D6's `miso_engine_v1_source_seek_report`: C + and C++ give a typedef and a function one name space, so D6's two names do not compile + (`redeclared as different kind of entity`). The size macro and feature bit keep D6's names. +- D3 read: a block that changes only a counter rewrites the two counters and keeps the + generation's `first_sample` and `source_frame`, so `(g, s, f)` always names the generation's + first block (the host workflow of gate 3). +- A clockless `begin_block` (tests and `read_block` only) continues its own clock: the block after + a block at `t` is at `t + quantum`, from 0. +- No re-export in `crates/capi/src/runtime/mod.rs`: the export reads the report's fields without + naming the type. +- Mechanical edits outside Authorized paths: `crates/host-core/tests/source_diagnostics.rs` gains + the `ReportBusy` row (its exhaustive table does not compile without it), and + `tools/realtime-policy/src/policy.rs` raises the floors to the tree's counts (33 files, 151 + regions), as its comment requires. + +**Gates.** + +1. `capi` `runtime::tests::a_plain_seek_is_reported_at_the_block_that_applied_it`: `(2, 768, + 4096, 0, 0)` after block 6, and `(1, 0, 0, 0, 0)` between the seek call and block 6. +2. `a_late_anchored_seek_reports_its_frame_plus_the_lateness`: `(2, 1280, 768, 0, 0)`, and block + 10 is bit-identical to block 6 (frames 768 on). +3. and 4. `a_scrubbed_stem_and_a_stem_started_from_its_report_stay_aligned`: Y anchored at + `f + (A - s)`; blocks 10-15 bit-identical to a fresh session of the committed document fed both + stems from the aligned frames; X unchanged across the carrying swap and its underrun counter + moves after it; Y `(0, ..)` before the swap block, `(1, 896, 0, 0, held)` while held, then + `(2, 1280, 4608, 0, 384)`. Source-level: `a_carried_consumer_continues_the_predecessor_audio_gap_free` + now also checks the report across the swap. +5. `source` `the_seek_report_carries_the_cumulative_held_and_underrun_frames`; also + `the_seek_report_names_the_block_and_frame_each_generation_started_at` and + `a_clockless_consumer_reports_consecutive_quanta`. +6. `report_separates_session_pcm_from_source_overhead` adds the row. `./target/release/audit capi`: + 0 allocations, 0 deallocations, 0 syscalls, `"total_violations":0`, `pcm_digest` + `4f52089e4a8cf83c` (unchanged). +7. `ffi::tests::seek_report_refusals_leave_out_untouched_and_name_their_rule`; `cargo test -p + source`, `-p engine --features engine/realtime-audit`, `-p host-core --features + host-core/test-support`, `-p capi`; `check-capi-abi.sh` and `--self-test`; `realtime-policy`; + `check-cross-targets.sh`; `check-workspace-policy.sh`; clippy of the touched crates; fmt; doc. + +**Measured.** `seek_report_bytes` is 64 per ring on 64-bit targets, 56 on wasm32 (`Arc`'s two +counters plus six `u64` words). host-web's `retained_ceilings` stays inside its budgets. + +**Successors.** The browser export is #1332 and #1293 (Non-goals). `observe.rs` onto +`seqlock.rs` is #1482 D3. `source.id.invalid` on this entry point is #1350. diff --git a/crates/capi/include/miso_engine_v1.h b/crates/capi/include/miso_engine_v1.h index 4fcbd0681..aef98ec00 100644 --- a/crates/capi/include/miso_engine_v1.h +++ b/crates/capi/include/miso_engine_v1.h @@ -17,9 +17,10 @@ extern "C" { * miso_engine_v1_last_error on the engine). * session Control thread. At most one thread at a time calls * miso_engine_v1_source_submit_planar_f32, miso_engine_v1_source_seek, - * miso_engine_v1_source_seek_at, miso_engine_v1_submit_command, - * miso_engine_v1_dequeue_event, miso_engine_v1_service, or - * miso_engine_v1_last_error on the session; they are serialized with each other. + * miso_engine_v1_source_seek_at, miso_engine_v1_source_seek_report, + * miso_engine_v1_submit_command, miso_engine_v1_dequeue_event, + * miso_engine_v1_service, or miso_engine_v1_last_error on the session; they are + * serialized with each other. * plan Split ownership. * miso_engine_v1_render_f32_planar: render thread only, never concurrently with * itself, and the exclusive owner of the plan's render state. @@ -149,7 +150,8 @@ extern "C" { * between edits. miso_engine_v1_service(session) runs one bounded step of it: it reclaims plans * render has retired, brings the session's view of the running plan up to render, refreshes the * session counters, and stages render telemetry. Every other session call (source submit, seek, - * seek_at, submit_command, dequeue_event) that passes its argument checks runs the same step + * seek_at, seek_report, submit_command, dequeue_event) that passes its argument checks runs the + * same step * first; a call refused for its arguments returns before the step. So a host that makes those * calls anyway need not also call service. The step never waits for render, never commits or * acknowledges anything, and never takes or drops a pending edit; its cost does not grow with the @@ -250,6 +252,10 @@ extern "C" { * g from F before the block at A renders. Until A the stem renders silence; from A it plays F on. * Should the render reach the seek only after A, the stem starts at F plus the lateness, so it * stays in time. + * + * Where a seek landed (issue 1316). miso_engine_v1_source_seek_report copies the source's + * (generation, first_sample, source_frame) and its cumulative underrun and held frames; a host + * checks MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT before calling it. */ #define MISO_ENGINE_V1_ABI_VERSION UINT32_C(0x00010000) @@ -285,7 +291,8 @@ extern "C" { #define MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_AT UINT64_C(32) #define MISO_ENGINE_V1_FEATURE_PLAN_WATERMARK UINT64_C(64) #define MISO_ENGINE_V1_FEATURE_SERVICE UINT64_C(128) -#define MISO_ENGINE_V1_FEATURE_MASK UINT64_C(255) +#define MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT UINT64_C(256) +#define MISO_ENGINE_V1_FEATURE_MASK UINT64_C(511) #define MISO_ENGINE_V1_OUTCOME_EXACT UINT64_C(1) #define MISO_ENGINE_V1_OUTCOME_TRANSITION_FALLBACK UINT64_C(2) @@ -300,6 +307,7 @@ extern "C" { #define MISO_ENGINE_V1_CAPABILITIES_SIZE UINT32_C(56) #define MISO_ENGINE_V1_PLAN_RESOURCE_REPORT_SIZE UINT32_C(240) #define MISO_ENGINE_V1_WATERMARK_SIZE UINT32_C(96) +#define MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE UINT32_C(64) typedef struct miso_engine_v1_engine miso_engine_v1_engine; typedef struct miso_engine_v1_session miso_engine_v1_session; @@ -457,6 +465,17 @@ typedef struct miso_engine_v1_watermark { uint64_t reserved[5]; /* Must be zero in ABI V1. */ } miso_engine_v1_watermark; +typedef struct miso_engine_v1_seek_report { + uint32_t struct_size; + uint32_t reserved0; /* Must be zero in ABI V1. */ + uint64_t generation; + uint64_t first_sample; + uint64_t source_frame; + uint64_t cumulative_underrun_frames; + uint64_t cumulative_held_frames; + uint64_t reserved[2]; /* Must be zero in ABI V1. */ +} miso_engine_v1_seek_report; + uint32_t miso_engine_v1_abi_version(void); uint32_t miso_engine_v1_query_capabilities(miso_engine_v1_capabilities *out); uint32_t miso_engine_v1_engine_create(const miso_engine_v1_engine_config *config, @@ -485,6 +504,10 @@ uint32_t miso_engine_v1_source_seek_at(miso_engine_v1_session *session, uint64_t generation, uint64_t source_frame, uint64_t anchor_sample); +uint32_t miso_engine_v1_source_seek_report(miso_engine_v1_session *session, + const uint8_t *source_id, + uint64_t source_id_bytes, + miso_engine_v1_seek_report *out); uint32_t miso_engine_v1_submit_command(miso_engine_v1_session *session, const uint8_t *request, uint64_t request_bytes, diff --git a/crates/capi/src/abi.rs b/crates/capi/src/abi.rs index 7eda1e635..1acb376e0 100644 --- a/crates/capi/src/abi.rs +++ b/crates/capi/src/abi.rs @@ -68,6 +68,9 @@ pub const FEATURE_PLAN_WATERMARK: u64 = 1 << 6; /// Control-service capability: `miso_engine_v1_service` (#1348), an in-place V1 amendment. A host /// checks this bit before calling the symbol. pub const FEATURE_SERVICE: u64 = 1 << 7; +/// Source seek-report capability: `miso_engine_v1_source_seek_report` (#1316), an in-place V1 +/// amendment. A host checks this bit before calling the symbol. +pub const FEATURE_SOURCE_SEEK_REPORT: u64 = 1 << 8; /// All ABI V1 feature capability bits. pub const FEATURE_MASK: u64 = FEATURE_IMMUTABLE_SESSION | FEATURE_HOST_PLANAR_SOURCE @@ -76,7 +79,8 @@ pub const FEATURE_MASK: u64 = FEATURE_IMMUTABLE_SESSION | FEATURE_CAPABILITY_COMMAND | FEATURE_SOURCE_SEEK_AT | FEATURE_PLAN_WATERMARK - | FEATURE_SERVICE; + | FEATURE_SERVICE + | FEATURE_SOURCE_SEEK_REPORT; /// Watermark outcome flag: the covered revisions completed exactly as committed. pub const OUTCOME_EXACT: u64 = 1; @@ -355,6 +359,29 @@ pub struct Watermark { pub reserved: [u64; 5], } +/// Where a source's newest generation landed, as render last published it (#1316 D6). +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +#[repr(C)] +pub struct SourceSeekReport { + /// Must equal [`SOURCE_SEEK_REPORT_SIZE`]. + pub struct_size: u32, + /// Must be zero in ABI V1. + pub reserved0: u32, + /// The generation in effect; `0` until the source's ring has rendered a block. + pub generation: u64, + /// The source-read sample (the render clock in this version) of the first block `generation` + /// played in: the clock `miso_engine_v1_source_seek_at`'s anchor is on. + pub first_sample: u64, + /// The source frame that block started at; a late anchored seek's includes its lateness. + pub source_frame: u64, + /// Saturating cumulative missing in-region frames (true underruns). + pub cumulative_underrun_frames: u64, + /// Saturating cumulative in-region frames held while an anchored seek waited. + pub cumulative_held_frames: u64, + /// Must be zero in ABI V1. + pub reserved: [u64; 2], +} + /// Frozen size of [`EngineConfig`] on the pinned 64-bit ABI. pub const ENGINE_CONFIG_SIZE: u32 = 40; /// Frozen size of [`CompileLimits`] on the pinned 64-bit ABI. @@ -373,6 +400,8 @@ pub const CAPABILITIES_SIZE: u32 = 56; pub const PLAN_RESOURCE_REPORT_SIZE: u32 = 240; /// Frozen size of [`Watermark`] on the pinned 64-bit ABI. pub const WATERMARK_SIZE: u32 = 96; +/// Frozen size of [`SourceSeekReport`] on the pinned 64-bit ABI. +pub const SOURCE_SEEK_REPORT_SIZE: u32 = 64; const HANDLE_COOKIE: u64 = 0x4d49_534f_5632_4142; const HANDLE_KIND_ENGINE: u32 = 1; @@ -507,6 +536,10 @@ mod tests { PLAN_RESOURCE_REPORT_SIZE as usize ); assert_eq!(size_of::(), WATERMARK_SIZE as usize); + assert_eq!( + size_of::(), + SOURCE_SEEK_REPORT_SIZE as usize + ); assert_eq!(align_of::(), 8); assert_eq!(align_of::(), 8); @@ -517,6 +550,7 @@ mod tests { assert_eq!(align_of::(), 8); assert_eq!(align_of::(), 8); assert_eq!(align_of::(), 8); + assert_eq!(align_of::(), 8); assert_eq!(offset_of!(CompileLimits, maximum_document_bytes), 16); assert_eq!(offset_of!(CompileLimits, maximum_replay_entries), 168); @@ -539,6 +573,12 @@ mod tests { assert_eq!(offset_of!(Watermark, transition_fallback_count), 40); assert_eq!(offset_of!(Watermark, superseded_count), 48); assert_eq!(offset_of!(Watermark, reserved), 56); + assert_eq!(offset_of!(SourceSeekReport, generation), 8); + assert_eq!(offset_of!(SourceSeekReport, first_sample), 16); + assert_eq!(offset_of!(SourceSeekReport, source_frame), 24); + assert_eq!(offset_of!(SourceSeekReport, cumulative_underrun_frames), 32); + assert_eq!(offset_of!(SourceSeekReport, cumulative_held_frames), 40); + assert_eq!(offset_of!(SourceSeekReport, reserved), 48); } #[test] @@ -551,7 +591,11 @@ mod tests { FEATURE_PLAN_WATERMARK ); assert_eq!(FEATURE_MASK & FEATURE_SERVICE, FEATURE_SERVICE); - assert_eq!(FEATURE_MASK, 0xff); + assert_eq!( + FEATURE_MASK & FEATURE_SOURCE_SEEK_REPORT, + FEATURE_SOURCE_SEEK_REPORT + ); + assert_eq!(FEATURE_MASK, 0x1ff); assert_eq!( [ OUTCOME_EXACT, diff --git a/crates/capi/src/ffi.rs b/crates/capi/src/ffi.rs index c6ae5c5bd..678b41ff3 100644 --- a/crates/capi/src/ffi.rs +++ b/crates/capi/src/ffi.rs @@ -7,8 +7,9 @@ use crate::{ EXACT_LAUNCH_RATE_MASK, Engine, EngineConfig, FEATURE_MASK, HandleHeader, Plan, PlanResourceReport, PlanarOutput, RESULT_ABI_MISMATCH, RESULT_BACKPRESSURE, RESULT_BUFFER_TOO_SMALL, RESULT_COMPILE_REJECTED, RESULT_INTERNAL, RESULT_INVALID_ARGUMENT, - RESULT_OK, RESULT_RENDER_REJECTED, RESULT_UNSUPPORTED, RESULT_WRONG_HANDLE, Session, - SourceChunk, SubmitReport, WATERMARK_SIZE, Watermark, + RESULT_OK, RESULT_RENDER_REJECTED, RESULT_UNSUPPORTED, RESULT_WRONG_HANDLE, + SOURCE_SEEK_REPORT_SIZE, Session, SourceChunk, SourceSeekReport, SubmitReport, WATERMARK_SIZE, + Watermark, }; use core::ffi::c_void; use core::ptr; @@ -626,6 +627,85 @@ unsafe fn source_seek_entry( }) } +/// Copy where a source of the newest committed session landed, as render last published it +/// (#1316): the generation in effect, the source-read sample (the render clock in this version) of +/// the first block it played in, the source frame that block started at, and the source's +/// cumulative underrun and held frames. A plain seek lands at the first block that begins after the +/// call; an anchored seek at its anchor block, at its frame plus the lateness if render reached it +/// late. Generation `0` means the source has not rendered a block yet. The report belongs to the +/// source's ring, so a plan swap that carries the source keeps it; a source a transaction adds +/// starts at generation `0`. +/// +/// Runs the service step first, as every session call does. Returns `RESULT_OK` and fills `out`; +/// `RESULT_INVALID_ARGUMENT` for a wrong `struct_size`, a nonzero reserved word, or a null, empty, +/// non-UTF-8 or over-long source ID, and, with `source.id.unknown`, for a source the session does +/// not declare; `RESULT_BACKPRESSURE` with `source.report.busy` when every bounded read landed +/// inside render's publication (`out` untouched; read again). +/// +/// # Safety +/// +/// `session` must be live, `source_id` must reference the declared borrowed byte count and `out` +/// must satisfy the writable ABI V1 seek-report contract. +/// +/// Thread: control. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn miso_engine_v1_source_seek_report( + session: *mut Session, + source_id: *const u8, + source_id_bytes: u64, + out: *mut SourceSeekReport, +) -> u32 { + catch_result(|| { + // SAFETY: Nonnull live handle pointers are caller-provided under the handle contract. + let kind = unsafe { session_kind(session) }; + if kind != RESULT_OK { + return kind; + } + if source_id.is_null() || out.is_null() { + return RESULT_INVALID_ARGUMENT; + } + // SAFETY: `out` is nonnull and the caller promises readable seek-report storage for these + // input-validation fields before the complete fixed-size write. + let (struct_size, reserved0, reserved) = + unsafe { ((*out).struct_size, (*out).reserved0, (*out).reserved) }; + if struct_size != SOURCE_SEEK_REPORT_SIZE || reserved0 != 0 || reserved != [0; 2] { + return RESULT_INVALID_ARGUMENT; + } + // SAFETY: The caller promises the source-ID storage is readable for this call. + let source_id = + match unsafe { borrowed_bytes(source_id, source_id_bytes, MAX_SOURCE_ID_BYTES) } { + Ok(value) if !value.is_empty() && core::str::from_utf8(value).is_ok() => value, + _ => return RESULT_INVALID_ARGUMENT, + }; + // SAFETY: `session` passed the live-kind check and the ABI serializes session calls. + let session = unsafe { &mut *session }; + match session.state.seek_report(source_id) { + Ok(report) => { + // SAFETY: Exact struct size establishes writable ABI V1 seek-report storage. + unsafe { + out.write(SourceSeekReport { + struct_size: SOURCE_SEEK_REPORT_SIZE, + reserved0: 0, + generation: report.generation, + first_sample: report.first_sample, + source_frame: report.source_frame, + cumulative_underrun_frames: report.cumulative_underrun_frames, + cumulative_held_frames: report.cumulative_held_frames, + reserved: [0; 2], + }); + } + session.last_error.borrow_mut().clear(); + RESULT_OK + } + Err(error) => { + let (code, diagnostic) = error.report(); + session.last_error.borrow_mut().set(diagnostic); + code + } + } + }) +} + /// Run one bounded step of control work between edits (#1348): reclaim retired plans, bring the /// provider epochs up to render, refresh the session counters and stage render telemetry. Every /// other session control call runs the same step first; this entry point runs only the step. @@ -1181,6 +1261,35 @@ pub(crate) fn test_plan_snapshot(plan: *mut Plan) -> (u64, PlanResourceReport) { (plan.owner().next_absolute_sample(), plan.resources()) } +/// A source's seek report through the exported entry point, with a valid zeroed struct. +#[cfg(test)] +pub(crate) fn test_source_seek_report( + session: *mut Session, + source_id: &[u8], +) -> (u32, SourceSeekReport) { + let mut out = SourceSeekReport { + struct_size: SOURCE_SEEK_REPORT_SIZE, + reserved0: 0, + generation: 0, + first_sample: 0, + source_frame: 0, + cumulative_underrun_frames: 0, + cumulative_held_frames: 0, + reserved: [0; 2], + }; + // SAFETY: Test callers pass a live session and a borrowed source ID; `out` is one complete + // local report. + let result = unsafe { + miso_engine_v1_source_seek_report( + session, + source_id.as_ptr(), + source_id.len() as u64, + &mut out, + ) + }; + (result, out) +} + /// The plan's watermark through the exported entry point, with a valid zeroed struct. #[cfg(test)] pub(crate) fn test_plan_watermark(plan: *const Plan) -> (u32, Watermark) { @@ -1567,10 +1676,14 @@ mod tests { assert_eq!(query(&mut capabilities), RESULT_OK); assert_eq!(capabilities.abi_version, ABI_VERSION); assert_eq!(capabilities.exact_launch_rate_mask, 0x0f); - assert_eq!(capabilities.feature_mask, 0xff); + assert_eq!(capabilities.feature_mask, 0x1ff); assert_ne!(capabilities.feature_mask & crate::FEATURE_SOURCE_SEEK_AT, 0); assert_ne!(capabilities.feature_mask & crate::FEATURE_PLAN_WATERMARK, 0); assert_ne!(capabilities.feature_mask & crate::FEATURE_SERVICE, 0); + assert_ne!( + capabilities.feature_mask & crate::FEATURE_SOURCE_SEEK_REPORT, + 0 + ); assert_eq!(capabilities.reserved, [0; 4]); } @@ -2460,6 +2573,111 @@ mod tests { destroy_fixture(engine, session, plan); } + /// #1316 D6: `miso_engine_v1_source_seek_report` refuses a bad handle, a bad struct and an + /// unknown or malformed source ID without touching `out`, and a valid call fills it. + /// + /// Test value: red if the export skips the struct checks (a wrong size or a nonzero reserved + /// word is then written through), if an unknown source is not `source.id.unknown`, or if the + /// fresh ring does not read generation 0, "not yet rendered". + #[test] + fn seek_report_refusals_leave_out_untouched_and_name_their_rule() { + let (engine, session, plan) = compiled_fixture(); + let report = |session: *mut Session, id: &[u8], out: &mut SourceSeekReport| { + // SAFETY: Test callers pass a borrowed ID and one complete local report. + unsafe { miso_engine_v1_source_seek_report(session, id.as_ptr(), id.len() as u64, out) } + }; + let valid = SourceSeekReport { + struct_size: SOURCE_SEEK_REPORT_SIZE, + reserved0: 0, + generation: 77, + first_sample: 77, + source_frame: 77, + cumulative_underrun_frames: 77, + cumulative_held_frames: 77, + reserved: [0; 2], + }; + let refusals: [(&str, *mut Session, &[u8], SourceSeekReport, u32); 7] = [ + ( + "null session", + ptr::null_mut(), + b"fixture-source", + valid, + RESULT_INVALID_ARGUMENT, + ), + ( + "engine handle", + engine.cast(), + b"fixture-source", + valid, + RESULT_WRONG_HANDLE, + ), + ( + "short struct", + session, + b"fixture-source", + SourceSeekReport { + struct_size: SOURCE_SEEK_REPORT_SIZE - 8, + ..valid + }, + RESULT_INVALID_ARGUMENT, + ), + ( + "nonzero reserved0", + session, + b"fixture-source", + SourceSeekReport { + reserved0: 1, + ..valid + }, + RESULT_INVALID_ARGUMENT, + ), + ( + "nonzero reserved", + session, + b"fixture-source", + SourceSeekReport { + reserved: [0, 1], + ..valid + }, + RESULT_INVALID_ARGUMENT, + ), + ("empty ID", session, b"", valid, RESULT_INVALID_ARGUMENT), + ( + "non-UTF-8 ID", + session, + b"\xff", + valid, + RESULT_INVALID_ARGUMENT, + ), + ]; + for (why, handle, id, input, code) in refusals { + let mut out = input; + assert_eq!(report(handle, id, &mut out), code, "{why}"); + assert_eq!(out, input, "{why}: out untouched"); + } + let mut out = valid; + assert_eq!( + report(session, b"absent-source", &mut out), + RESULT_INVALID_ARGUMENT + ); + assert_eq!(out, valid); + assert_eq!(read_last_error(session.cast()), b"source.id.unknown"); + assert_eq!(report(session, b"fixture-source", &mut out), RESULT_OK); + assert_eq!(read_last_error(session.cast()), b""); + assert_eq!( + out, + SourceSeekReport { + generation: 0, + first_sample: 0, + source_frame: 0, + cumulative_underrun_frames: 0, + cumulative_held_frames: 0, + ..valid + } + ); + destroy_fixture(engine, session, plan); + } + /// F4: one validation pass, one diagnostic per rule. Each rejection names the single check it /// failed -- five of these used to share the string `render.contract.rejected` -- and none of /// them writes a sample, so the host buffer still holds its pre-call NaN fill. diff --git a/crates/capi/src/runtime/tests.rs b/crates/capi/src/runtime/tests.rs index 82fab6583..0b872e0a6 100644 --- a/crates/capi/src/runtime/tests.rs +++ b/crates/capi/src/runtime/tests.rs @@ -1450,6 +1450,211 @@ fn an_added_c_abi_source_starts_at_its_anchored_render_sample() { an_added_source_starts_at_its_anchor_at(48_000, 0); } +/// The seek report of `id` through the exported entry point, as `(generation, first_sample, +/// source_frame, cumulative_underrun_frames, cumulative_held_frames)`. +fn seek_report_c(session: *mut crate::Session, id: &[u8]) -> (u64, u64, u64, u64, u64) { + let (result, report) = crate::ffi::test_source_seek_report(session, id); + assert_eq!(result, crate::RESULT_OK); + ( + report.generation, + report.first_sample, + report.source_frame, + report.cumulative_underrun_frames, + report.cumulative_held_frames, + ) +} + +/// Issue #1316 gate 1: a plain seek during playback is reported at the block that applied it. +/// Blocks 0-5 play generation 1; the host seeks to generation 2 at frame 4096 and submits it from +/// there, and block 6 (render sample 768) reports `(2, 768, 4096, 0, 0)`. +/// +/// Test value: red if the report follows the producer's generation at the call (it reads 2 +/// before block 6 renders), or names the block after the one that applied the seek. +#[test] +fn a_plain_seek_is_reported_at_the_block_that_applied_it() { + let model = stateless_session(48_000); + let document = session::canonical_session_json(&model).expect("canonical"); + let (session, plan) = boxed_c_children(&document); + const SOURCE: &[u8] = b"fixture-source"; + assert_eq!(seek_report_c(session, SOURCE), (0, 0, 0, 0, 0)); + let (left, right) = continuity_block(0, 0); + submit_c(session, 1, 0, 48_000, &left, &right, false); + let _ = feed_and_render_c(session, plan, 48_000, 0, 6); + assert_eq!(seek_report_c(session, SOURCE), (1, 0, 0, 0, 0)); + assert_eq!( + crate::ffi::test_source_seek(session, SOURCE, 2, 4_096), + crate::RESULT_OK + ); + submit_block_c(session, SOURCE, 2, 32, 0, 48_000); + assert_eq!( + seek_report_c(session, SOURCE), + (1, 0, 0, 0, 0), + "nothing has played generation 2 yet" + ); + let _ = render_c(plan, 6); + assert_eq!(seek_report_c(session, SOURCE), (2, 768, 4_096, 0, 0)); + submit_block_c(session, SOURCE, 2, 33, 0, 48_000); + let _ = render_c(plan, 7); + assert_eq!(seek_report_c(session, SOURCE), (2, 768, 4_096, 0, 0)); + crate::ffi::test_plan_destroy(plan); + crate::ffi::test_session_destroy(session); +} + +/// Issue #1316 gate 2: an anchored seek render reaches after its anchor starts at its frame plus +/// the lateness, and the report says so. After block 9, `seek_at(2, F = 384, A = 896)`; block 10 +/// (render sample 1280) starts at `384 + (1280 - 896) = 768` and reports `(2, 1280, 768)`. The +/// session holds no DSP state, so block 10 is bit-identical to block 6, which read frames 768 on. +/// +/// Test value: red if the report omits the lateness (reports 384), or the block plays from it. +#[test] +fn a_late_anchored_seek_reports_its_frame_plus_the_lateness() { + let model = stateless_session(48_000); + let document = session::canonical_session_json(&model).expect("canonical"); + let (session, plan) = boxed_c_children(&document); + const SOURCE: &[u8] = b"fixture-source"; + let (left, right) = continuity_block(0, 0); + submit_c(session, 1, 0, 48_000, &left, &right, false); + let played = feed_and_render_c(session, plan, 48_000, 0, 10); + assert_eq!( + crate::ffi::test_source_seek_at(session, SOURCE, 2, 384, 896), + crate::RESULT_OK + ); + // The producer takes generation 2 from frame 384; render discards what precedes 768. + for block in 3..=6 { + submit_block_c(session, SOURCE, 2, block, 0, 48_000); + } + let late = render_c(plan, 10); + assert_eq!(seek_report_c(session, SOURCE), (2, 1_280, 768, 0, 0)); + assert_bit_identical("late anchored seek", 10, &[late], &played[6..7]); + crate::ffi::test_plan_destroy(plan); + crate::ffi::test_session_destroy(session); +} + +/// Issue #1316 gates 3 and 4: scrub one stem, then start another aligned to it. +/// +/// One track plays stem X. After block 5 the host plain-seeks X to generation 2 at frame 4096 and +/// reads its report `(g, s, f)` after block 6. A transaction then adds stem Y on its own track; +/// before the swap block renders, Y reads generation 0. The host anchors Y at `A` = 1280 (block +/// 10) on frame `f + (A - s)` and submits it from there. From A on, every block is bit-identical +/// to the committed session compiled fresh and fed both stems from the aligned frames. X's report +/// is unchanged by the swap that carries it; Y reads `(1, 896, 0)` after the swap block (held +/// while its seek waits) and `(2, A, f + (A - s))` from A. One starved block after the run shows +/// the carried ring still publishes to the host's reader. +/// +/// Test value: red if the report's sample and frame are not on the anchor's clock (Y is then out of +/// time with X from A), if the record lives on the plan instead of the ring (X's report is lost at +/// the swap, or no longer moves after it), or if an added ring inherits a predecessor's record. +#[test] +fn a_scrubbed_stem_and_a_stem_started_from_its_report_stay_aligned() { + const X: &[u8] = b"fixture-source"; + const Y: &[u8] = b"s2-source"; + const ANCHOR_BLOCK: u64 = 10; + const BLOCKS: u64 = 16; + let anchor = ANCHOR_BLOCK * 128; + let mut model = stateless_session(48_000); + model.tracks.truncate(1); + model.routes.truncate(1); + let document = session::canonical_session_json(&model).expect("canonical"); + let (session, plan) = boxed_c_children(&document); + let (left, right) = continuity_block(0, 0); + submit_c(session, 1, 0, 48_000, &left, &right, false); + let _ = feed_and_render_c(session, plan, 48_000, 0, 6); + + // Scrub X during playback. + assert_eq!( + crate::ffi::test_source_seek(session, X, 2, 4_096), + crate::RESULT_OK + ); + submit_block_c(session, X, 2, 32, 0, 48_000); + let _ = render_c(plan, 6); + let (g, s, f, _, _) = seek_report_c(session, X); + assert_eq!((g, s, f), (2, 768, 4_096)); + // X's frame at render sample `t` is `f + (t - s)`, in blocks of 128. + let x_block = |block: u64| (f + block * 128 - s) / 128; + + transact_c( + session, + 1, + 42, + &add_source_and_track(&model, "s2-source", "t2"), + ); + assert_eq!( + seek_report_c(session, Y), + (0, 0, 0, 0, 0), + "an added ring has not rendered" + ); + let y_frame = f + (anchor - s); + assert_eq!( + crate::ffi::test_source_seek_at(session, Y, 2, y_frame, anchor), + crate::RESULT_OK + ); + let mut rendered = Vec::new(); + for block in 7..BLOCKS { + submit_block_c(session, X, 2, x_block(block), 0, 48_000); + if block >= ANCHOR_BLOCK { + submit_block_c(session, Y, 2, x_block(block), 1, 48_000); + } + rendered.push(render_c(plan, block)); + assert_eq!( + seek_report_c(session, X), + (2, 768, 4_096, 0, 0), + "block {block}" + ); + let held = 128 * (block.min(ANCHOR_BLOCK - 1) - 6); + let expected_y = if block < ANCHOR_BLOCK { + (1, 896, 0, 0, held) + } else { + (2, anchor, y_frame, 0, held) + }; + assert_eq!(seek_report_c(session, Y), expected_y, "block {block}"); + } + assert_eq!( + crate::ffi::test_plan_carry_counts(plan), + (1, 0), + "one swap, and it carried X" + ); + // A starved block after the swap: the carried ring still publishes to the host's reader. + let _ = render_c(plan, BLOCKS); + assert_eq!(seek_report_c(session, X), (2, 768, 4_096, 128, 0)); + let committed = snapshot_c(session); + crate::ffi::test_plan_destroy(plan); + crate::ffi::test_session_destroy(session); + + // The reference reads both stems from the aligned frames, from its own block 0. + let reference = render_fresh_reference(&committed, BLOCKS - ANCHOR_BLOCK, &|session, block| { + submit_c_to_block(session, X, x_block(ANCHOR_BLOCK + block), block, 0); + submit_c_to_block(session, Y, x_block(ANCHOR_BLOCK + block), block, 1); + }); + assert_bit_identical( + "scrubbed X with Y aligned from its report", + ANCHOR_BLOCK, + &rendered[(ANCHOR_BLOCK - 7) as usize..], + &reference, + ); +} + +/// Submits the content of block `content` (salt `salt`) as generation 1 at source frame +/// `block * 128` of `source`. +fn submit_c_to_block( + session: *mut crate::Session, + source: &[u8], + content: u64, + block: u64, + salt: u64, +) { + let (left, right) = continuity_block(content, salt); + submit_c_to( + session, + source, + 1, + block * 128, + 48_000, + &left, + &right, + false, + ); +} + /// Submits block `block` of each of `ids`' signals (salted by position) through the session state. pub(super) fn submit_direct(children: &mut CompiledChildren, ids: &[&[u8]], block: u64) { for (salt, id) in ids.iter().enumerate() { diff --git a/crates/capi/tests/c/abi_smoke.c b/crates/capi/tests/c/abi_smoke.c index cfe1e0ece..92530678d 100644 --- a/crates/capi/tests/c/abi_smoke.c +++ b/crates/capi/tests/c/abi_smoke.c @@ -20,7 +20,9 @@ ABI_ASSERT(MISO_ENGINE_V1_TAIL_FINITE == 0); ABI_ASSERT(MISO_ENGINE_V1_TAIL_INFINITE == 1); ABI_ASSERT(MISO_ENGINE_V1_EXACT_LAUNCH_RATE_MASK == 15); ABI_ASSERT(MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_AT == 32); -ABI_ASSERT(MISO_ENGINE_V1_FEATURE_MASK == 255); +ABI_ASSERT(MISO_ENGINE_V1_FEATURE_MASK == 511); +ABI_ASSERT((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT) == + MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT); ABI_ASSERT((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_SERVICE) == MISO_ENGINE_V1_FEATURE_SERVICE); ABI_ASSERT((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_PLAN_WATERMARK) == @@ -40,6 +42,9 @@ ABI_ASSERT(sizeof(miso_engine_v1_plan_resource_report) == MISO_ENGINE_V1_PLAN_RESOURCE_REPORT_SIZE); ABI_ASSERT(sizeof(miso_engine_v1_watermark) == MISO_ENGINE_V1_WATERMARK_SIZE); ABI_ASSERT(MISO_ENGINE_V1_WATERMARK_SIZE == 96); +ABI_ASSERT(sizeof(miso_engine_v1_seek_report) == + MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE); +ABI_ASSERT(MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE == 64); ABI_ASSERT(_Alignof(miso_engine_v1_engine_config) == 8); ABI_ASSERT(_Alignof(miso_engine_v1_compile_limits) == 8); @@ -50,6 +55,7 @@ ABI_ASSERT(_Alignof(miso_engine_v1_planar_output) == 8); ABI_ASSERT(_Alignof(miso_engine_v1_capabilities) == 8); ABI_ASSERT(_Alignof(miso_engine_v1_plan_resource_report) == 8); ABI_ASSERT(_Alignof(miso_engine_v1_watermark) == 8); +ABI_ASSERT(_Alignof(miso_engine_v1_seek_report) == 8); ABI_ASSERT(offsetof(miso_engine_v1_compile_limits, maximum_document_bytes) == 16); ABI_ASSERT(offsetof(miso_engine_v1_compile_limits, maximum_replay_entries) == 168); @@ -67,6 +73,9 @@ ABI_ASSERT(offsetof(miso_engine_v1_plan_resource_report, source_count) == 16); ABI_ASSERT(offsetof(miso_engine_v1_plan_resource_report, reserved) == 208); ABI_ASSERT(offsetof(miso_engine_v1_watermark, revision) == 8); ABI_ASSERT(offsetof(miso_engine_v1_watermark, reserved) == 56); +ABI_ASSERT(offsetof(miso_engine_v1_seek_report, generation) == 8); +ABI_ASSERT(offsetof(miso_engine_v1_seek_report, cumulative_held_frames) == 40); +ABI_ASSERT(offsetof(miso_engine_v1_seek_report, reserved) == 48); static uint32_t (*const abi_version_signature)(void) = miso_engine_v1_abi_version; static uint32_t (*const query_capabilities_signature)(miso_engine_v1_capabilities *) = @@ -101,6 +110,11 @@ static uint32_t (*const source_seek_at_signature)(miso_engine_v1_session *, uint64_t, uint64_t, uint64_t) = miso_engine_v1_source_seek_at; +static uint32_t (*const source_seek_report_signature)(miso_engine_v1_session *, + const uint8_t *, + uint64_t, + miso_engine_v1_seek_report *) = + miso_engine_v1_source_seek_report; static uint32_t (*const submit_command_signature)(miso_engine_v1_session *, const uint8_t *, uint64_t, @@ -154,6 +168,7 @@ static int check_plan_watermark(miso_engine_v1_engine *engine) { miso_engine_v1_session *session = NULL; miso_engine_v1_plan *plan = NULL; miso_engine_v1_watermark watermark = {0}; + miso_engine_v1_seek_report seek_report = {0}; int failed = 0; limits.struct_size = MISO_ENGINE_V1_COMPILE_LIMITS_SIZE; @@ -203,6 +218,28 @@ static int check_plan_watermark(miso_engine_v1_engine *engine) { failed = 3; } + /* Issue 1316: before its first block the source reports generation 0, "not yet rendered"; an + * unknown source and a wrong struct size are refused. */ + seek_report.struct_size = MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE; + if (failed == 0 && + (source_seek_report_signature(session, (const uint8_t *)"voice", 5, &seek_report) != + MISO_ENGINE_V1_OK || + seek_report.struct_size != MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE || + seek_report.generation != 0)) { + failed = 4; + } + if (failed == 0 && + source_seek_report_signature(session, (const uint8_t *)"none", 4, &seek_report) != + MISO_ENGINE_V1_INVALID_ARGUMENT) { + failed = 5; + } + seek_report.struct_size = MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE - 8; + if (failed == 0 && + source_seek_report_signature(session, (const uint8_t *)"voice", 5, &seek_report) != + MISO_ENGINE_V1_INVALID_ARGUMENT) { + failed = 6; + } + plan_destroy_signature(plan); session_destroy_signature(session); return failed; @@ -227,8 +264,10 @@ int main(void) { if (engine_create_signature(&config, &engine) != MISO_ENGINE_V1_OK || engine == NULL) { return 2; } - /* A host checks the watermark bit before calling the symbol (issue 1314). */ + /* A host checks the watermark and seek-report bits before calling the symbols (issues 1314, + * 1316). */ if ((capabilities.feature_mask & MISO_ENGINE_V1_FEATURE_PLAN_WATERMARK) == 0 || + (capabilities.feature_mask & MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT) == 0 || check_plan_watermark(engine) != 0) { engine_destroy_signature(engine); return 5; @@ -251,7 +290,8 @@ int main(void) { } return compile_session_signature == NULL || source_submit_signature == NULL || - source_seek_signature == NULL || submit_command_signature == NULL || + source_seek_signature == NULL || source_seek_report_signature == NULL || + submit_command_signature == NULL || dequeue_event_signature == NULL || render_signature == NULL || resources_signature == NULL || watermark_signature == NULL || service_signature == NULL || diff --git a/crates/capi/tests/c/header_smoke.cpp b/crates/capi/tests/c/header_smoke.cpp index 936c0b92a..da3a4ca05 100644 --- a/crates/capi/tests/c/header_smoke.cpp +++ b/crates/capi/tests/c/header_smoke.cpp @@ -25,5 +25,21 @@ static_assert((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_PLAN_WATERMA MISO_ENGINE_V1_FEATURE_PLAN_WATERMARK); static_assert((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_SERVICE) == MISO_ENGINE_V1_FEATURE_SERVICE); +static_assert(sizeof(miso_engine_v1_seek_report) == MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE); +static_assert(MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE == 64); +static_assert(std::is_standard_layout::value); +static_assert(offsetof(miso_engine_v1_seek_report, struct_size) == 0); +static_assert(offsetof(miso_engine_v1_seek_report, reserved0) == 4); +static_assert(offsetof(miso_engine_v1_seek_report, generation) == 8); +static_assert(offsetof(miso_engine_v1_seek_report, first_sample) == 16); +static_assert(offsetof(miso_engine_v1_seek_report, source_frame) == 24); +static_assert(offsetof(miso_engine_v1_seek_report, cumulative_underrun_frames) == 32); +static_assert(offsetof(miso_engine_v1_seek_report, cumulative_held_frames) == 40); +static_assert(offsetof(miso_engine_v1_seek_report, reserved) == 48); +static_assert((MISO_ENGINE_V1_FEATURE_MASK & MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT) == + MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT); +static_assert(std::is_same::value); static_assert(std::is_same::value); diff --git a/crates/control-plane/src/control.rs b/crates/control-plane/src/control.rs index 8f7ce18b9..4bcfb720d 100644 --- a/crates/control-plane/src/control.rs +++ b/crates/control-plane/src/control.rs @@ -3892,6 +3892,17 @@ where .seek_at(id, generation, source_frame, anchor_sample) .map_err(SourceFailure::Control) } + + /// Where a source of the newest committed session landed, as render last published it, after + /// the service step (#1316 D6, D7): [`host_core::SourceControlSet::seek_report`] of the ring + /// the newest session's producer feeds, which is the one `seek` queues on. + pub fn seek_report(&mut self, id: &[u8]) -> Result { + self.service().map_err(|_| SourceFailure::Internal)?; + self.newest_providers() + .sources + .seek_report(id) + .map_err(SourceFailure::Control) + } } /// A source submission or seek that the adapter's boundary must report. diff --git a/crates/control-plane/src/lib.rs b/crates/control-plane/src/lib.rs index e3e10ccc0..844844702 100644 --- a/crates/control-plane/src/lib.rs +++ b/crates/control-plane/src/lib.rs @@ -70,6 +70,7 @@ pub use error::{CompileFailure, ResourceFault}; /// One planar chunk a host submits to a source of the session ([`SessionState::submit`]). pub use host_core::SourceSubmission; pub use plan::{PlanQueries, PlanState}; +pub use source::SourceSeekReport; #[cfg(feature = "test-support")] pub use compile::{ diff --git a/crates/engine/src/realtime/mod.rs b/crates/engine/src/realtime/mod.rs index 2b48fc18f..4e09a061e 100644 --- a/crates/engine/src/realtime/mod.rs +++ b/crates/engine/src/realtime/mod.rs @@ -11,6 +11,7 @@ mod latest_cell; mod observe; mod plan; mod plan_exchange; +mod seqlock; mod spsc; mod watermark; @@ -42,6 +43,10 @@ pub use plan_exchange::{ UnadoptedCandidate, Withdrawal, plan_exchange, plan_exchange_at_revision, plan_exchange_resource_report, }; +pub use seqlock::{ + MAXIMUM_READ_ATTEMPTS, PublishedWords, PublishedWordsBusy, PublishedWordsReader, + PublishedWordsWriter, published_words, published_words_retained_bytes, +}; pub use spsc::{ Consumer, PlanAdoption, Producer, QueueEmpty, QueueFull, QueueGeneration, SpscError, SpscRetainedPayload, bounded_spsc, bounded_spsc_move, bounded_spsc_retained_payload, diff --git a/crates/engine/src/realtime/seqlock.rs b/crates/engine/src/realtime/seqlock.rs new file mode 100644 index 000000000..3e01e151a --- /dev/null +++ b/crates/engine/src/realtime/seqlock.rs @@ -0,0 +1,202 @@ +//! The engine's one single-writer seqlock: a fixed-width record of `u64` words (B15-R1). +//! +//! # What it is for +//! +//! A *level* that render publishes and any thread reads: the applied-revision watermark (#1314) +//! and a source ring's seek report (#1316) are both instances. Render overwrites the record in +//! bounded time and never waits; a reader takes the newest whole record, or reports +//! [`PublishedWordsBusy`] after a bounded number of torn attempts. Nothing is queued, so nothing +//! can back up or be dropped. +//! +//! The record is generic over its width and knows nothing of its words' meaning, so any owner, +//! a ring or not, can publish its own words through it. +//! +//! # The protocol, spelled in safe Rust +//! +//! Every published word is its own atomic, and an odd/even sequence counter fences the window. +//! The writer is the only one: [`PublishedWordsWriter`] is not `Clone`, and publishing takes +//! `&mut self`. A reader that sees an odd counter, or a different counter before and after its +//! loads, retries, at most [`MAXIMUM_READ_ATTEMPTS`] times. There is no `unsafe` here: this module +//! is inside the realtime root that `tools/realtime-policy` holds to its approved-unsafe list. + +use core::sync::atomic::{AtomicU64, AtomicUsize, Ordering, fence}; +use std::sync::Arc; + +/// The bound on one read's attempts. It keeps a read provably finite while a writer publishing in +/// a tight loop still lets the reader through. A production writer publishes at most once per +/// render block, so a read retries only when it lands inside one record store. +pub const MAXIMUM_READ_ATTEMPTS: usize = 64; + +/// A read gave up after [`MAXIMUM_READ_ATTEMPTS`] torn attempts; the caller retries later. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct PublishedWordsBusy; + +/// The shared record: a sequence counter and `N` words. One allocation, made off render. +#[derive(Debug)] +pub struct PublishedWords { + /// Odd while a publication is in flight, even between them. + sequence_lock: AtomicU64, + words: [AtomicU64; N], +} + +/// The writer's half: the record's only writer. Not `Clone`. +#[derive(Debug)] +pub struct PublishedWordsWriter { + record: Arc>, +} + +/// Any-thread reader handle. Cloning it shares the one record. +#[derive(Clone, Debug)] +pub struct PublishedWordsReader { + record: Arc>, +} + +/// Make one record holding `initial`, and split it into its writer and a reader. Allocates the +/// record once; call it on a control thread. +#[must_use] +pub fn published_words( + initial: [u64; N], +) -> (PublishedWordsWriter, PublishedWordsReader) { + let record = Arc::new(PublishedWords { + sequence_lock: AtomicU64::new(0), + words: initial.map(AtomicU64::new), + }); + let reader = PublishedWordsReader { + record: Arc::clone(&record), + }; + (PublishedWordsWriter { record }, reader) +} + +/// Exact engine-owned bytes of one shared record of `N` words, with its shared-owner header. +#[must_use] +pub fn published_words_retained_bytes() -> usize { + // `Arc`'s allocation is `{ strong, weak, data }`, as the mailbox's and the ring's mirrors spell. + #[repr(C)] + struct SharedAllocation { + strong: AtomicUsize, + weak: AtomicUsize, + record: PublishedWords, + } + core::alloc::Layout::new::>().size() +} + +// REALTIME_POLICY_BEGIN +impl PublishedWordsWriter { + /// Overwrite the record with `words`. Wait-free: two counter stores and `N` word stores. + #[inline] + pub fn publish(&mut self, words: &[u64; N]) { + let record = &*self.record; + let opening = record.sequence_lock.load(Ordering::Relaxed).wrapping_add(1); + record.sequence_lock.store(opening, Ordering::Relaxed); + // Keeps the word stores below from being observed before the counter went odd. + fence(Ordering::Release); + for (cell, word) in record.words.iter().zip(words) { + cell.store(*word, Ordering::Relaxed); + } + record + .sequence_lock + .store(opening.wrapping_add(1), Ordering::Release); + } +} +// REALTIME_POLICY_END + +impl PublishedWordsReader { + /// The newest whole record, or [`PublishedWordsBusy`] if every one of the bounded attempts + /// landed inside a publication. Never a partial record: the words are taken between two equal + /// even counter reads, so what comes back is exactly one publication. + pub fn read(&self) -> Result<[u64; N], PublishedWordsBusy> { + let record = &*self.record; + for _ in 0..MAXIMUM_READ_ATTEMPTS { + let before = record.sequence_lock.load(Ordering::Acquire); + if !before.is_multiple_of(2) { + continue; + } + let mut words = [0_u64; N]; + for (word, cell) in words.iter_mut().zip(&record.words) { + *word = cell.load(Ordering::Relaxed); + } + fence(Ordering::Acquire); + if record.sequence_lock.load(Ordering::Relaxed) == before { + return Ok(words); + } + } + Err(PublishedWordsBusy) + } +} + +#[cfg(test)] +mod tests { + use super::{PublishedWordsBusy, published_words}; + use std::sync::atomic::{AtomicBool, Ordering}; + use std::sync::{Arc, Barrier}; + use std::thread; + use std::time::{Duration, Instant}; + + /// Every word derived from one counter, so a mixture of two publications is detectable. + fn record(counter: u64) -> [u64; 6] { + [ + counter, + counter.wrapping_mul(128), + counter % 7, + counter.wrapping_mul(3), + counter ^ 0x5555_5555_5555_5555, + !counter, + ] + } + + struct DoneOnDrop(Arc); + + impl Drop for DoneOnDrop { + fn drop(&mut self) { + self.0.store(true, Ordering::Release); + } + } + + /// #1314 gate 5, moved here with the protocol (B15-R1): a writer publishing in a tight loop + /// never lets a reader see two publications' words in one record; a busy result is the only + /// alternative to a whole one. + #[test] + fn a_read_is_one_whole_publication_or_busy() { + const PUBLICATIONS: u64 = 1_000_000; + const STALL_DEADLINE: Duration = Duration::from_secs(10); + let (mut writer, reader) = published_words(record(0)); + let barrier = Arc::new(Barrier::new(2)); + let writer_barrier = Arc::clone(&barrier); + let done = Arc::new(AtomicBool::new(false)); + let writer_done = Arc::clone(&done); + let publisher = thread::spawn(move || { + let _done = DoneOnDrop(writer_done); + writer_barrier.wait(); + for counter in 1..=PUBLICATIONS { + writer.publish(&record(counter)); + } + }); + barrier.wait(); + let (mut whole, mut busy, mut torn, mut newest) = (0_u64, 0_u64, 0_u64, 0_u64); + let started = Instant::now(); + while !done.load(Ordering::Acquire) { + match reader.read() { + Ok(observed) => { + whole += 1; + if observed != record(observed[0]) { + torn += 1; + } + newest = newest.max(observed[0]); + } + Err(PublishedWordsBusy) => busy += 1, + } + assert!(started.elapsed() < STALL_DEADLINE, "the writer stalled"); + } + publisher.join().expect("writer"); + assert_eq!(torn, 0, "{torn} of {whole} reads mixed two publications"); + assert!( + whole > 0, + "the reader never got a whole record ({busy} busy)" + ); + assert_eq!( + reader.read().expect("a quiescent read is never busy"), + record(PUBLICATIONS) + ); + assert!(newest <= PUBLICATIONS); + } +} diff --git a/crates/engine/src/realtime/watermark.rs b/crates/engine/src/realtime/watermark.rs index 4f0f6d73f..4b714bdbd 100644 --- a/crates/engine/src/realtime/watermark.rs +++ b/crates/engine/src/realtime/watermark.rs @@ -23,16 +23,15 @@ //! grows. That is the acked-batch answer: no ack can precede a drop, because there is no queue to //! drop from (#1314 D8). //! -//! # The seqlock, spelled in safe Rust +//! # The record //! -//! The pattern of `observe.rs`: every published word is its own atomic, and an odd/even sequence -//! counter fences the window. Render is the only writer. A reader that sees an odd counter, or a -//! different counter before and after its loads, retries, at most [`MAXIMUM_READ_ATTEMPTS`] times, -//! then reports [`WatermarkBusy`]. There is no `unsafe` here: this module is inside the realtime -//! root that `tools/realtime-policy` holds to its approved-unsafe list. +//! The words travel through the engine's one single-writer seqlock (`seqlock.rs`, B15-R1): six +//! words, render the only writer, a reader bounded to `seqlock::MAXIMUM_READ_ATTEMPTS` attempts +//! before it reports [`WatermarkBusy`]. -use core::sync::atomic::{AtomicU64, AtomicUsize, Ordering, fence}; -use std::sync::Arc; +use super::seqlock::{ + PublishedWordsReader, PublishedWordsWriter, published_words, published_words_retained_bytes, +}; /// Outcome flag: the covered revisions completed exactly as committed. pub const OUTCOME_EXACT: u64 = 1; @@ -43,11 +42,6 @@ pub const OUTCOME_TRANSITION_FALLBACK: u64 = 2; /// replaced before render adopted them (#1310). pub const OUTCOME_SUPERSEDED: u64 = 4; -/// The same bound as `observe.rs`'s, for the same reason: it keeps a read provably finite while a -/// writer publishing in a tight loop still lets the reader through. Render publishes at most once -/// per block, so a production read retries only when it lands inside one seven-word store. -const MAXIMUM_READ_ATTEMPTS: usize = 64; - /// How a candidate's own revisions complete when render adopts it (#1314 D1, D5). /// /// Written only while control owns the candidate, before publication. In this slice every @@ -86,35 +80,50 @@ pub struct PlanWatermark { #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct WatermarkBusy; -/// The shared record. One allocation, made when the plan exchange is prepared. -#[derive(Debug)] -pub(crate) struct WatermarkRecord { - /// Odd while a publication is in flight, even between them. - sequence_lock: AtomicU64, - revision: AtomicU64, - first_sample: AtomicU64, - flags: AtomicU64, - exact: AtomicU64, - transition_fallback: AtomicU64, - superseded: AtomicU64, +/// The words of one watermark in the shared record, in [`PlanWatermark`]'s field order. +const WATERMARK_WORDS: usize = 6; + +impl PlanWatermark { + const fn to_words(self) -> [u64; WATERMARK_WORDS] { + [ + self.revision, + self.first_sample, + self.outcome_flags, + self.exact, + self.transition_fallback, + self.superseded, + ] + } + + const fn from_words(words: [u64; WATERMARK_WORDS]) -> Self { + let [ + revision, + first_sample, + outcome_flags, + exact, + transition_fallback, + superseded, + ] = words; + Self { + revision, + first_sample, + outcome_flags, + exact, + transition_fallback, + superseded, + } + } } /// Exact engine-owned bytes of the shared watermark record, with its shared-owner header. pub(crate) fn watermark_retained_bytes() -> usize { - // `Arc`'s allocation is `{ strong, weak, data }`, as the mailbox's and the ring's mirrors spell. - #[repr(C)] - struct SharedWatermarkAllocation { - strong: AtomicUsize, - weak: AtomicUsize, - record: WatermarkRecord, - } - core::alloc::Layout::new::().size() + published_words_retained_bytes::() } /// Render's half: the only writer, with the render-local state an advance needs. #[derive(Debug)] pub(crate) struct WatermarkWriter { - record: Arc, + record: PublishedWordsWriter, /// What the record holds; render is its only writer, so it never loads it back. current: PlanWatermark, /// `superseded` of the candidates claimed since the last advance (#1314 D5). @@ -126,7 +135,7 @@ pub(crate) struct WatermarkWriter { /// Any-thread reader handle. Cloning it shares the one record. #[derive(Clone, Debug)] pub struct PlanWatermarkReader { - record: Arc, + record: PublishedWordsReader, } /// Make the record with the initial watermark `(initial_revision, 0, EXACT)` and every counter 0. @@ -139,18 +148,8 @@ pub(crate) fn plan_watermark(initial_revision: u64) -> (WatermarkWriter, PlanWat transition_fallback: 0, superseded: 0, }; - let record = Arc::new(WatermarkRecord { - sequence_lock: AtomicU64::new(0), - revision: AtomicU64::new(current.revision), - first_sample: AtomicU64::new(current.first_sample), - flags: AtomicU64::new(current.outcome_flags), - exact: AtomicU64::new(current.exact), - transition_fallback: AtomicU64::new(current.transition_fallback), - superseded: AtomicU64::new(current.superseded), - }); - let reader = PlanWatermarkReader { - record: Arc::clone(&record), - }; + let (record, reader) = published_words(current.to_words()); + let reader = PlanWatermarkReader { record: reader }; ( WatermarkWriter { record, @@ -219,29 +218,8 @@ impl WatermarkWriter { /// Overwrite the record with `watermark`. Wait-free: two counter stores and six word stores. #[inline] - fn store(&self, watermark: PlanWatermark) { - let record = &*self.record; - let opening = record.sequence_lock.load(Ordering::Relaxed).wrapping_add(1); - record.sequence_lock.store(opening, Ordering::Relaxed); - // Keeps the word stores below from being observed before the counter went odd. - fence(Ordering::Release); - record.revision.store(watermark.revision, Ordering::Relaxed); - record - .first_sample - .store(watermark.first_sample, Ordering::Relaxed); - record - .flags - .store(watermark.outcome_flags, Ordering::Relaxed); - record.exact.store(watermark.exact, Ordering::Relaxed); - record - .transition_fallback - .store(watermark.transition_fallback, Ordering::Relaxed); - record - .superseded - .store(watermark.superseded, Ordering::Relaxed); - record - .sequence_lock - .store(opening.wrapping_add(1), Ordering::Release); + fn store(&mut self, watermark: PlanWatermark) { + self.record.publish(&watermark.to_words()); } } // REALTIME_POLICY_END @@ -251,101 +229,9 @@ impl PlanWatermarkReader { /// landed inside a publication. Never a partial record: the words are taken between two equal /// even counter reads, so what comes back is exactly one publication. pub fn read(&self) -> Result { - let record = &*self.record; - for _ in 0..MAXIMUM_READ_ATTEMPTS { - let before = record.sequence_lock.load(Ordering::Acquire); - if !before.is_multiple_of(2) { - continue; - } - let watermark = PlanWatermark { - revision: record.revision.load(Ordering::Relaxed), - first_sample: record.first_sample.load(Ordering::Relaxed), - outcome_flags: record.flags.load(Ordering::Relaxed), - exact: record.exact.load(Ordering::Relaxed), - transition_fallback: record.transition_fallback.load(Ordering::Relaxed), - superseded: record.superseded.load(Ordering::Relaxed), - }; - fence(Ordering::Acquire); - if record.sequence_lock.load(Ordering::Relaxed) == before { - return Ok(watermark); - } - } - Err(WatermarkBusy) - } -} - -#[cfg(test)] -mod tests { - use super::{PlanWatermark, WatermarkBusy, plan_watermark}; - use std::sync::atomic::{AtomicBool, Ordering}; - use std::sync::{Arc, Barrier}; - use std::thread; - use std::time::{Duration, Instant}; - - /// Every word derived from one counter, so a mixture of two publications is detectable. - fn record(counter: u64) -> PlanWatermark { - PlanWatermark { - revision: counter, - first_sample: counter.wrapping_mul(128), - outcome_flags: counter % 7, - exact: counter.wrapping_mul(3), - transition_fallback: counter ^ 0x5555_5555_5555_5555, - superseded: !counter, - } - } - - struct DoneOnDrop(Arc); - - impl Drop for DoneOnDrop { - fn drop(&mut self) { - self.0.store(true, Ordering::Release); - } - } - - /// #1314 gate 5: a writer publishing in a tight loop never lets a reader see two - /// publications' words in one record; a busy result is the only alternative to a whole one. - #[test] - fn a_watermark_read_is_one_whole_publication_or_busy() { - const PUBLICATIONS: u64 = 1_000_000; - const STALL_DEADLINE: Duration = Duration::from_secs(10); - let (writer, reader) = plan_watermark(0); - let barrier = Arc::new(Barrier::new(2)); - let writer_barrier = Arc::clone(&barrier); - let done = Arc::new(AtomicBool::new(false)); - let writer_done = Arc::clone(&done); - let publisher = thread::spawn(move || { - let _done = DoneOnDrop(writer_done); - writer_barrier.wait(); - for counter in 1..=PUBLICATIONS { - writer.store(record(counter)); - } - }); - barrier.wait(); - let (mut whole, mut busy, mut torn, mut newest) = (0_u64, 0_u64, 0_u64, 0_u64); - let started = Instant::now(); - while !done.load(Ordering::Acquire) { - match reader.read() { - Ok(observed) => { - whole += 1; - if observed.revision != 0 && observed != record(observed.revision) { - torn += 1; - } - newest = newest.max(observed.revision); - } - Err(WatermarkBusy) => busy += 1, - } - assert!(started.elapsed() < STALL_DEADLINE, "the writer stalled"); - } - publisher.join().expect("writer"); - assert_eq!(torn, 0, "{torn} of {whole} reads mixed two publications"); - assert!( - whole > 0, - "the reader never got a whole record ({busy} busy)" - ); - assert_eq!( - reader.read().expect("a quiescent read is never busy"), - record(PUBLICATIONS) - ); - assert!(newest <= PUBLICATIONS); + self.record + .read() + .map(PlanWatermark::from_words) + .map_err(|_| WatermarkBusy) } } diff --git a/crates/host-core/src/source.rs b/crates/host-core/src/source.rs index ce0acaf9c..df65be46d 100644 --- a/crates/host-core/src/source.rs +++ b/crates/host-core/src/source.rs @@ -12,7 +12,7 @@ use engine::realtime::PreparedRenderPlan; use session::SessionModel; use source::{ HostChunkError, HostChunkProvider, HostPlanarChunk, SourceCommand, SourceFrame, - SourceGeneration, SourceSeekError, SubmitReport, + SourceGeneration, SourceSeekError, SourceSeekReport, SubmitReport, }; use crate::PlanStateInventory; @@ -75,6 +75,9 @@ pub enum SourceControlError { /// or to a predecessor whose producer has already moved to its successor. Neither /// backpressure nor an engine invariant: the caller feeds the newest set. Vacated, + /// Every bounded attempt to read the source's seek report landed inside render's publication + /// of it (#1316 D6): backpressure, the caller reads again. + ReportBusy, } impl SourceControlError { @@ -104,6 +107,7 @@ impl SourceControlError { Self::Seek(SourceSeekError::Backpressure { .. }) => "source.seek.backpressure", Self::Vacated => "source.ring.vacated", Self::Seek(SourceSeekError::AnchorUnaligned) => "source.seek.anchor_unaligned", + Self::ReportBusy => "source.report.busy", } } @@ -114,6 +118,7 @@ impl SourceControlError { self, Self::Chunk(HostChunkError::Full { .. }) | Self::Seek(SourceSeekError::Backpressure { .. }) + | Self::ReportBusy ) } @@ -212,11 +217,11 @@ impl SourceControlSet { } /// Queue one strictly increasing, generation-tagged seek whose source `frame` enters the graph - /// in the block that starts at absolute render sample `anchor_sample` (issue #1274). + /// in the block that reads source time from `anchor_sample` (issue #1274). /// - /// `anchor_sample` is on the plan's absolute render clock, the one a host renders with and - /// every plan swap continues; it names when `frame` enters the graph, and the output hears it - /// the plan's latency later. It must be a multiple of the quantum, or the seek is refused with + /// `anchor_sample` is on the plan's source-read clock, which every plan swap continues and + /// which equals the absolute render clock a host renders with in this version (#1316 D1); it + /// names when `frame` enters the graph, and the output hears it the plan's latency later. It must be a multiple of the quantum, or the seek is refused with /// `source.seek.anchor_unaligned`. The region and generation rules are [`Self::seek`]'s. /// /// On success the source accepts the new generation's PCM from `frame` at once; submit it @@ -224,7 +229,8 @@ impl SourceControlSet { /// the render pops that PCM before it pops the seek. Should the render reach the seek only /// after the anchor block has passed, it starts at `frame` plus the lateness, so it stays in /// time. Until the anchor block, PCM already queued for the playing generation keeps playing; - /// when that runs out, the source renders silence and reports an underrun. + /// when that runs out, the source renders silence and holds: the blocks count as held, not as + /// underruns (#1318). [`Self::seek_report`] tells where the seek landed. pub fn seek_at( &mut self, id: &[u8], @@ -235,6 +241,29 @@ impl SourceControlSet { self.queue_seek(id, generation, frame, Some(anchor_sample)) } + /// Where the named source's newest generation landed, as render last published it (#1316 D7): + /// the generation in effect, the source-read sample of the first block it played in and the + /// source frame that block started at, plus the ring's cumulative underrun and held frames. + /// Generation `0` means the ring has not rendered a block yet. + /// + /// A plain seek lands at the first block that begins after it is queued; an anchored one at + /// its anchor block, at `frame` plus the lateness if render reached it late. So a host that + /// reads `(g, s, f)` here knows the source plays frame `f + (t - s)` at source-read sample `t`. + /// + /// Bounded: a read that keeps landing inside render's publication is + /// [`SourceControlError::ReportBusy`]. The record belongs to the ring, so it survives a plan + /// swap that carries the source; a vacant entry is [`SourceControlError::Vacated`], as for + /// [`Self::seek`]. + pub fn seek_report(&self, id: &[u8]) -> Result { + let index = self.index_of(id).ok_or(SourceControlError::UnknownSource)?; + self.sources[index] + .provider + .as_ref() + .ok_or(SourceControlError::Vacated)? + .seek_report() + .map_err(|_| SourceControlError::ReportBusy) + } + fn queue_seek( &mut self, id: &[u8], diff --git a/crates/host-core/tests/source_diagnostics.rs b/crates/host-core/tests/source_diagnostics.rs index e3ec885ee..744a54689 100644 --- a/crates/host-core/tests/source_diagnostics.rs +++ b/crates/host-core/tests/source_diagnostics.rs @@ -160,6 +160,12 @@ const TABLE: &[(SourceControlError, &str, bool, bool)] = &[ false, false, ), + ( + SourceControlError::ReportBusy, + "source.report.busy", + true, + false, + ), ]; /// The two codes a host must see from more than one variant, and why. Anything else sharing a @@ -194,6 +200,7 @@ const fn variant_index(error: SourceControlError) -> usize { SourceControlError::Seek(SourceSeekError::Backpressure { .. }) => 16, SourceControlError::Vacated => 17, SourceControlError::Seek(SourceSeekError::AnchorUnaligned) => 18, + SourceControlError::ReportBusy => 19, } } diff --git a/crates/source/src/lib.rs b/crates/source/src/lib.rs index 6bb10c848..cfc1be345 100644 --- a/crates/source/src/lib.rs +++ b/crates/source/src/lib.rs @@ -13,8 +13,9 @@ use std::fmt; use engine::{ QuantumFrames, SampleRateHz, realtime::{ - Consumer, Producer, QueueEmpty, QueueFull, QueueGeneration, SpscError, bounded_spsc, - bounded_spsc_move, bounded_spsc_retained_payload, + Consumer, Producer, PublishedWordsReader, PublishedWordsWriter, QueueEmpty, QueueFull, + QueueGeneration, SpscError, bounded_spsc, bounded_spsc_move, bounded_spsc_retained_payload, + published_words, published_words_retained_bytes, }, }; use graph::{ @@ -57,11 +58,12 @@ pub enum SourceCommand { frame: SourceFrame, }, /// Switch to a strictly newer generation so that `frame` enters the graph in the block that - /// starts at absolute render sample `anchor_sample` (issue #1274). + /// reads source time from `anchor_sample` (issue #1274). /// - /// `anchor_sample` is on the plan's absolute render clock -- the first sample the graph hands - /// its source set each block, which every plan swap continues -- and must be a multiple of the - /// quantum. The output hears `frame` the plan's latency later. The render consumer holds the + /// `anchor_sample` is on the plan's source-read clock -- the absolute sample of source time a + /// block reads, which the graph hands its source set each block and every plan swap + /// continues; equal to the render clock in this version (#1316 D1) -- and must be a multiple + /// of the quantum. The output hears `frame` the plan's latency later. The render consumer holds the /// seek until the first block starting at or after the anchor; observed late, it starts at /// `frame + (block start - anchor_sample)`, so the source is in time either way. While the /// seek is held, already queued PCM of the playing generation keeps playing and the new @@ -72,7 +74,7 @@ pub enum SourceCommand { generation: SourceGeneration, /// Decoded source frame that enters the graph at `anchor_sample`. frame: SourceFrame, - /// Absolute render sample of the block `frame` enters in; a multiple of the quantum. + /// Source-read sample of the block `frame` enters in; a multiple of the quantum. anchor_sample: u64, }, } @@ -220,6 +222,8 @@ pub struct SourceResourceReport { pub recycle_queue_bytes: u64, /// One-slot seek-command SPSC header plus slots. pub command_queue_bytes: u64, + /// The ring's shared seek report (#1316 D2): one record, with its shared-owner header. + pub seek_report_bytes: u64, /// One `TransferBlock` allocation per allocated block (configured plus retained), excluding /// its PCM allocation. pub transfer_block_metadata_bytes: u64, @@ -460,6 +464,63 @@ pub struct SourceConsumerTelemetry { pub native_decoder_sanitized_samples: u64, } +/// Where the ring's newest generation landed, as render published it (#1316 D2, D3). +/// +/// A level, not a queue: render overwrites it at the block a seek applies at, and whenever a +/// cumulative counter moves. Before the ring's first block it reads generation `0`, meaning "not +/// yet rendered". +#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] +pub struct SourceSeekReport { + /// The generation in effect; `0` until the ring's first block. + pub generation: u64, + /// The plan's source-read sample of the first block `generation` played in: the clock an + /// anchored seek's `anchor_sample` is on. + pub first_sample: u64, + /// The source frame that block started reading at; a late anchored seek's includes its + /// lateness. + pub source_frame: u64, + /// Saturating cumulative missing in-region frames (true underruns). + pub cumulative_underrun_frames: u64, + /// Saturating cumulative held in-region frames (an anchored seek waiting, #1318). + pub cumulative_held_frames: u64, +} + +/// The words of a [`SourceSeekReport`] in the ring's shared record. +const SEEK_REPORT_WORDS: usize = 5; + +impl SourceSeekReport { + const fn to_words(self) -> [u64; SEEK_REPORT_WORDS] { + [ + self.generation, + self.first_sample, + self.source_frame, + self.cumulative_underrun_frames, + self.cumulative_held_frames, + ] + } + + const fn from_words(words: [u64; SEEK_REPORT_WORDS]) -> Self { + let [ + generation, + first_sample, + source_frame, + cumulative_underrun_frames, + cumulative_held_frames, + ] = words; + Self { + generation, + first_sample, + source_frame, + cumulative_underrun_frames, + cumulative_held_frames, + } + } +} + +/// A seek-report read landed inside render's publication on every bounded attempt; retry later. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct SourceSeekReportBusy; + struct TransferBlock { generation: SourceGeneration, start_frame: SourceFrame, @@ -519,6 +580,8 @@ impl PcmSourceRing { let retained_block_count = u64::try_from(RETAINED_TRANSFER_BLOCKS) .map_err(|_| PcmSourceRingError::PlatformSizeLimit)?; let retained_pcm = checked_u64_mul(block_pcm, retained_block_count)?; + let seek_report = u64::try_from(published_words_retained_bytes::()) + .map_err(|_| PcmSourceRingError::PlatformSizeLimit)?; let metadata = checked_u64_mul( shape.allocated_block_count_u64, u64::try_from(size_of::()) @@ -529,7 +592,7 @@ impl PcmSourceRing { checked_u64_add(data_queue, recycle_queue)?, checked_u64_add(command_queue, metadata)?, )?, - retained_pcm, + checked_u64_add(retained_pcm, seek_report)?, )?; let total = checked_u64_add(pcm, overhead)?; let largest = [ @@ -538,6 +601,7 @@ impl PcmSourceRing { .map_err(|_| PcmSourceRingError::PlatformSizeLimit)?, largest_queue_allocation::>(shape.allocated_block_count)?, largest_queue_allocation::(command_capacity)?, + seek_report, ] .into_iter() .max() @@ -549,6 +613,7 @@ impl PcmSourceRing { data_queue_bytes: data_queue, recycle_queue_bytes: recycle_queue, command_queue_bytes: command_queue, + seek_report_bytes: seek_report, transfer_block_metadata_bytes: metadata, transfer_block_pcm_bytes: pcm, retained_block_pcm_bytes: retained_pcm, @@ -599,6 +664,9 @@ impl PcmSourceRing { queue_generation, ) .map_err(map_spsc_error)?; + // Generation 0 until the first block: "not yet rendered" (#1316 D3). + let (seek_report_writer, seek_report_reader) = + published_words(SourceSeekReport::default().to_words()); let mut consumer = PcmSourceConsumer { data_consumer, recycle_producer, @@ -631,6 +699,9 @@ impl PcmSourceRing { generation_changed: false, held_seek: None, begun_block: false, + seek_report: seek_report_writer, + published_seek_report: SourceSeekReport::default(), + next_source_read_sample: 0, }; // The producer starts with exactly the configured blocks; the retained block above never // enters the recycle queue until the consumer has a played block to hold instead. @@ -655,6 +726,7 @@ impl PcmSourceRing { cumulative_written_frames: 0, deferred_block: None, native_decoder_sanitized_samples: 0, + seek_report: seek_report_reader, }, consumer, report, @@ -740,6 +812,8 @@ pub struct PcmSourceProducer { cumulative_written_frames: u64, deferred_block: Option>, native_decoder_sanitized_samples: u64, + /// The reader of the ring's seek report, which the consumer publishes (#1316 D2). + seek_report: PublishedWordsReader, } /// Name for the prepared producer when it is used solely to control source seeks. @@ -800,6 +874,15 @@ impl PcmSourceProducer { } } + /// The ring's seek report as render last published it (#1316 D2): bounded retries, then + /// [`SourceSeekReportBusy`]. Any thread; it never waits for render. + pub fn seek_report(&self) -> Result { + self.seek_report + .read() + .map(SourceSeekReport::from_words) + .map_err(|_| SourceSeekReportBusy) + } + /// Copy producer-local telemetry outside render. #[must_use] pub fn telemetry(&self) -> SourceProducerTelemetry { @@ -911,6 +994,11 @@ impl HostChunkProvider { pub fn telemetry(&self) -> SourceProducerTelemetry { self.producer.telemetry() } + + /// The ring's seek report; see [`PcmSourceProducer::seek_report`]. + pub fn seek_report(&self) -> Result { + self.producer.seek_report() + } } fn validate_host_chunk( @@ -1037,6 +1125,14 @@ pub struct PcmSourceConsumer { /// Whether any block has begun on this consumer (#1344 D5): render-owned, set at block entry /// and never cleared. begun_block: bool, + /// The ring's seek report: this consumer is its only writer (#1316 D2). It belongs to the + /// ring, so it moves with the consumer across a plan swap (D5). + seek_report: PublishedWordsWriter, + /// What the seek report holds; render is its only writer, so it never loads it back. + published_seek_report: SourceSeekReport, + /// The source-read sample of the block after the last one begun: the clock a clockless + /// [`PcmSourceConsumer::begin_block`] continues, so its blocks are consecutive quanta. + next_source_read_sample: u64, } /// A [`SourceCommand::SeekAt`] the consumer has popped and holds until its anchor block. @@ -1044,6 +1140,7 @@ pub struct PcmSourceConsumer { struct HeldSeek { generation: SourceGeneration, frame: SourceFrame, + /// On the plan's source-read clock, as [`SourceCommand::SeekAt::anchor_sample`]. anchor_sample: u64, } @@ -1052,8 +1149,13 @@ struct HeldSeek { enum SeekClock { /// At once, as a plain seek to its frame: a caller that supplies no block time. Now, - /// In the block starting at this absolute render sample, if it is at or past the anchor. - At(u64), + /// In the block that reads source time from `source_read_sample`, if it is at or past the + /// anchor. + At { + /// The plan's source-read clock: the absolute sample of source time this block reads; + /// equal to the render clock in this version (#1316 D1). + source_read_sample: u64, + }, /// Not now: preparation between blocks, which has no render time. Hold, } @@ -1125,17 +1227,23 @@ impl PcmSourceConsumer { self.begin_block_with(SeekClock::Now) } - /// [`Self::begin_block`] for the block that starts at absolute render sample `first_sample`. + /// [`Self::begin_block`] for the block that reads source time from `source_read_sample`: the + /// plan's source-read clock, the absolute sample of source time this block reads; equal to + /// the render clock in this version (#1316 D1). *Give a plan a source-read clock that leads + /// its render clock* (#1396) is the issue that makes the two differ. /// - /// An anchored seek ([`SourceCommand::SeekAt`]) is held while `first_sample` is before its - /// anchor; the block that starts at or past it applies the seek at `frame + (first_sample - - /// anchor_sample)` (saturating). While it is held, queued PCM of the playing generation still + /// An anchored seek ([`SourceCommand::SeekAt`]) is held while `source_read_sample` is before + /// its anchor; the block that starts at or past it applies the seek at `frame + + /// (source_read_sample - anchor_sample)` (saturating). While it is held, queued PCM of the playing generation still /// plays, the new generation's first block is kept as the pending block, and with nothing /// playable the block is **held**, not an underrun (the host asked the source to wait; underrun /// means the host owed PCM and did not deliver it). A newer command replaces a held one. This render-path /// operation allocates nothing and never blocks. - pub fn begin_block_at(&mut self, first_sample: u64) -> SourceReadReport { - self.begin_block_with(SeekClock::At(first_sample)) + /// + /// The block publishes the ring's seek report when a seek applies at it, at the ring's first + /// block and when a cumulative counter moves ([`SourceSeekReport`], #1316 D3). + pub fn begin_block_at(&mut self, source_read_sample: u64) -> SourceReadReport { + self.begin_block_with(SeekClock::At { source_read_sample }) } fn begin_block_with(&mut self, clock: SeekClock) -> SourceReadReport { @@ -1157,6 +1265,15 @@ impl PcmSourceConsumer { /// The second half of [`Self::begin_block_with`]: acquire and play this block. fn begin_block_play(&mut self, clock: SeekClock) -> SourceReadReport { self.acquire_current_block(clock); + // Every seek this block can apply has applied by now (the observe step or the acquire's + // re-observe), and nothing below has advanced: this is the frame the block reads from. + let source_frame = self.next_frame; + let source_read_sample = match clock { + SeekClock::At { source_read_sample } => source_read_sample, + SeekClock::Now | SeekClock::Hold => self.next_source_read_sample, + }; + self.next_source_read_sample = + source_read_sample.saturating_add(u64::from(self.quantum_frames)); let mut copied_frames = 0_u32; let mut underrun_frames = 0_u32; let mut held_frames = 0_u32; @@ -1211,6 +1328,7 @@ impl PcmSourceConsumer { self.end_of_region = true; } } + self.publish_seek_report(source_read_sample, source_frame); SourceReadReport { copied_frames, underrun_frames, @@ -1369,6 +1487,25 @@ impl PcmSourceConsumer { Ok(()) } // REALTIME_POLICY_BEGIN + /// Publish the ring's seek report if this block changed it (#1316 D3): a new generation in + /// effect (the ring's first block, or one a seek applied at) takes this block's + /// `source_read_sample` and `source_frame`; a moved cumulative counter rewrites the counters + /// alone. Otherwise nothing is written. Wait-free, allocation-free. + fn publish_seek_report(&mut self, source_read_sample: u64, source_frame: SourceFrame) { + let mut report = self.published_seek_report; + if report.generation != self.active_generation.0 { + report.generation = self.active_generation.0; + report.first_sample = source_read_sample; + report.source_frame = source_frame.0; + } + report.cumulative_underrun_frames = self.underrun_frames; + report.cumulative_held_frames = self.held_frames; + if report != self.published_seek_report { + self.published_seek_report = report; + self.seek_report.publish(&report.to_words()); + } + } + /// Pop the admitted command, if any, then apply a held anchored seek whose block has come. fn observe_seek_at_block_boundary(&mut self, clock: SeekClock) { match self.command_consumer.try_pop() { @@ -1402,10 +1539,10 @@ impl PcmSourceConsumer { }; let late_by = match clock { SeekClock::Now => 0, - SeekClock::At(first_sample) if first_sample >= held.anchor_sample => { - first_sample - held.anchor_sample + SeekClock::At { source_read_sample } if source_read_sample >= held.anchor_sample => { + source_read_sample - held.anchor_sample } - SeekClock::At(_) | SeekClock::Hold => return, + SeekClock::At { .. } | SeekClock::Hold => return, }; self.held_seek = None; self.apply_seek( @@ -1818,7 +1955,8 @@ impl GraphPreparedSourceSetDriver for SourceGraphSourceSetDriver { self.block_validity.source_underrun = true; continue; }; - // The block's absolute render sample is the clock an anchored seek waits on. + // The plan's source-read clock, which an anchored seek waits on, equals the render + // clock in this version (#1316 D1); #1396 passes the source-read sample here. let report = consumer.begin_block_at(first_sample); self.block_validity.source_underrun |= report.underrun_event; self.block_validity.source_generation_changed |= report.generation_changed; @@ -2211,6 +2349,13 @@ mod tests { + report.command_queue_bytes + report.transfer_block_metadata_bytes + report.retained_block_pcm_bytes + + report.seek_report_bytes + ); + // #1316 D2: the seek report is one shared record of a sequence word and five report + // words, behind `Arc`'s two counters, on every target. + assert_eq!( + report.seek_report_bytes, + u64::try_from(2 * size_of::() + 6 * size_of::()).expect("record size") ); // The memory delta over the pre-#917 two-block ring, which sized both queues and the // metadata at two blocks: one block's PCM, one block's metadata and one pointer slot in @@ -2218,6 +2363,7 @@ mod tests { let two = NonZeroUsize::new(2).expect("configured blocks"); let pre_change_overhead = 2 * queue_bytes::>(two).expect("queue") + report.command_queue_bytes + + report.seek_report_bytes + 2 * metadata; let slot = u64::try_from(size_of::>()).expect("slot size"); assert_eq!( @@ -2232,7 +2378,10 @@ mod tests { largest_queue_allocation::>(three).expect("largest queue"); assert_eq!( report.largest_allocation_bytes, - largest_queue.max(32).max(metadata) + largest_queue + .max(32) + .max(metadata) + .max(report.seek_report_bytes) ); } @@ -2486,12 +2635,17 @@ mod tests { frame: SourceFrame(100), } }; - let (output, report) = - render_with_producer_in_window(&mut consumer, SeekClock::At(4), || { + let (output, report) = render_with_producer_in_window( + &mut consumer, + SeekClock::At { + source_read_sample: 4, + }, + || { host.try_seek(command).unwrap(); host.submit(chunk(2, 100, &[&[2.0; 4]], 4, false)) .expect("accepted"); - }); + }, + ); if anchored { assert_eq!(output, [0.0; 4], "held until its anchor"); assert_eq!(report.active_generation, SourceGeneration(1)); @@ -2514,14 +2668,19 @@ mod tests { // plays frame 104, and only the skipped block at frame 100 is discarded. let (producer, mut consumer, _) = PcmSourceRing::prepare(config(1, 4, 16)).unwrap(); let mut host = producer.into_host_chunk_provider(RATE); - let (output, report) = - render_with_producer_in_window(&mut consumer, SeekClock::At(8), || { + let (output, report) = render_with_producer_in_window( + &mut consumer, + SeekClock::At { + source_read_sample: 8, + }, + || { host.try_seek(seek_at(2, 100, 4)).unwrap(); host.submit(chunk(2, 100, &[&[2.0; 4]], 4, false)) .expect("accepted"); host.submit(chunk(2, 104, &[&[2.5; 4]], 4, false)) .expect("accepted"); - }); + }, + ); assert_eq!( output, [2.5; 4], "late by one block: frame 104 plays at block 8" @@ -3027,6 +3186,108 @@ mod tests { /// #1318 gate 2. Red if held is decided by `held_seek` alone: the playing generation's /// queued PCM still plays while a seek is held, and those blocks are neither held nor /// underruns. + fn seek_report( + generation: u64, + first_sample: u64, + source_frame: u64, + cumulative_underrun_frames: u64, + cumulative_held_frames: u64, + ) -> SourceSeekReport { + SourceSeekReport { + generation, + first_sample, + source_frame, + cumulative_underrun_frames, + cumulative_held_frames, + } + } + + /// #1316 D3. Red if the report moves at the seek call (the producer's generation) rather than + /// at the block that applies the seek, if it names the block after that one, if a late + /// anchored seek reports its frame without the lateness, or if a block that applied nothing + /// rewrites the sample and frame. + #[test] + fn the_seek_report_names_the_block_and_frame_each_generation_started_at() { + let (producer, mut consumer, _) = PcmSourceRing::prepare(config(1, 4, 16)).unwrap(); + let mut host = producer.into_host_chunk_provider(RATE); + assert_eq!(host.seek_report(), Ok(SourceSeekReport::default())); + for block in 0..2_u64 { + host.submit(chunk(1, 4 * block, &[&[1.0; 4]], 4, false)) + .unwrap(); + } + let _ = render_at(&mut consumer, 0); + assert_eq!(host.seek_report(), Ok(seek_report(1, 0, 0, 0, 0))); + let _ = render_at(&mut consumer, 4); + assert_eq!(host.seek_report(), Ok(seek_report(1, 0, 0, 0, 0))); + + // A plain seek lands at the next block. + host.try_seek(SourceCommand::Seek { + generation: SourceGeneration(2), + frame: SourceFrame(400), + }) + .unwrap(); + host.submit(chunk(2, 400, &[&[2.0; 4]], 4, false)).unwrap(); + assert_eq!( + host.seek_report(), + Ok(seek_report(1, 0, 0, 0, 0)), + "not at the call" + ); + let (output, _) = render_at(&mut consumer, 8); + assert_eq!(output, [2.0; 4]); + assert_eq!(host.seek_report(), Ok(seek_report(2, 8, 400, 0, 0))); + + // An anchored seek render reaches one block late starts at its frame plus the lateness. + host.try_seek(seek_at(3, 96, 12)).unwrap(); + host.submit(chunk(3, 96, &[&[-3.0; 4]], 4, false)).unwrap(); + host.submit(chunk(3, 100, &[&[3.0; 4]], 4, false)).unwrap(); + let (output, _) = render_at(&mut consumer, 16); + assert_eq!(output, [3.0; 4]); + assert_eq!(host.seek_report(), Ok(seek_report(3, 16, 100, 0, 0))); + } + + /// #1316 D3 for a clockless consumer: its blocks are consecutive quanta from 0. Red if + /// `begin_block` publishes no sample, or does not continue its own clock. + #[test] + fn a_clockless_consumer_reports_consecutive_quanta() { + let (producer, mut consumer, _) = PcmSourceRing::prepare(config(1, 4, 8)).unwrap(); + let mut host = producer.into_host_chunk_provider(RATE); + let mut output = [0.0; 4]; + host.submit(chunk(1, 0, &[&[1.0; 4]], 4, false)).unwrap(); + consumer.read_block(&mut [&mut output]).unwrap(); + host.try_seek(SourceCommand::Seek { + generation: SourceGeneration(2), + frame: SourceFrame(40), + }) + .unwrap(); + host.submit(chunk(2, 40, &[&[2.0; 4]], 4, false)).unwrap(); + consumer.read_block(&mut [&mut output]).unwrap(); + assert_eq!(output, [2.0; 4]); + assert_eq!(host.seek_report(), Ok(seek_report(2, 4, 40, 0, 0))); + } + + /// #1316 gate 5. Red if the report carries a block's own held or underrun frames instead of + /// the cumulative counters, or if a counter-only publication rewrites the generation's sample + /// and frame. + #[test] + fn the_seek_report_carries_the_cumulative_held_and_underrun_frames() { + let (producer, mut consumer, _) = PcmSourceRing::prepare(config(1, 4, 8)).unwrap(); + let mut host = producer.into_host_chunk_provider(RATE); + host.try_seek(seek_at(2, 100, 16)).unwrap(); + for block in 0..4_u64 { + let _ = render_at(&mut consumer, 4 * block); + assert_eq!( + host.seek_report(), + Ok(seek_report(1, 0, 0, 0, 4 * (block + 1))), + "held block {block}" + ); + } + // The anchor block applies the seek; the host owed PCM and delivered none. + let _ = render_at(&mut consumer, 16); + assert_eq!(host.seek_report(), Ok(seek_report(2, 16, 100, 4, 16))); + let _ = render_at(&mut consumer, 20); + assert_eq!(host.seek_report(), Ok(seek_report(2, 16, 100, 8, 16))); + } + #[test] fn queued_pcm_of_the_playing_generation_plays_while_a_seek_is_held() { let (producer, mut consumer, _) = PcmSourceRing::prepare(config(1, 4, 8)).unwrap(); @@ -4090,17 +4351,26 @@ mod tests { /// Plan A over the filled ring for `SWAP_BLOCK` blocks through a plan exchange, then the /// vacant plan B, given the program `program(identity of A)`, for the rest. Returns every /// block, the swap block's carry outcome, and each block's `(underrun, generation)` facts. - fn swap( - program: impl FnOnce(u64) -> Option, - ) -> (Vec, CarryOutcome, Vec<(bool, bool)>) { + /// What [`swap`] returns: each block's bits, the carry outcome, each block's validity + /// facts and the source's seek report after each block. + type SwapRun = ( + Vec, + CarryOutcome, + Vec<(bool, bool)>, + Vec, + ); + + /// Renders `BLOCKS` blocks, swapping at `SWAP_BLOCK`. + fn swap(program: impl FnOnce(u64) -> Option) -> SwapRun { let validity = Arc::new(AtomicU8::new(0)); - let (_host, source) = filled_source(); + let (host, source) = filled_source(); let mut predecessor = bind(source, &validity); let identity = graph::plan_identity(&mut predecessor).expect("graph plan"); let (mut publisher, mut owner, _retirer) = plan_exchange(predecessor, exchange_config()).expect("exchange"); let mut blocks = Vec::new(); let mut facts = Vec::new(); + let mut reports = Vec::new(); let mut carry = CarryOutcome::NotRequested; let mut successor = Some(bind(SourceGraphSource::vacant(2), &validity)); let mut program = Some(program); @@ -4127,8 +4397,9 @@ mod tests { assert_eq!(report.swap, expected_swap, "block {block}"); blocks.push(bits(&output)); facts.push(take_validity(&validity)); + reports.push(host.seek_report().expect("a quiescent read")); } - (blocks, carry, facts) + (blocks, carry, facts, reports) } fn carry_source_zero(predecessor: u64) -> GraphCarryProgram { @@ -4143,9 +4414,20 @@ mod tests { #[test] fn a_carried_consumer_continues_the_predecessor_audio_gap_free() { let reference = reference(); - let (blocks, carry, facts) = swap(|identity| Some(carry_source_zero(identity))); + let (blocks, carry, facts, reports) = + swap(|identity| Some(carry_source_zero(identity))); assert_eq!(blocks, reference); assert_eq!(carry, CarryOutcome::Carried); + // #1316 gate 4: the seek report is the ring's, so it moves with the consumer and + // reads the same before, at and after the swap block. + assert!( + reports.iter().all(|report| *report + == SourceSeekReport { + generation: 1, + ..SourceSeekReport::default() + }), + "{reports:?}" + ); assert!( facts.iter().all(|&fact| fact == (false, false)), "{facts:?}" @@ -4164,7 +4446,7 @@ mod tests { (CarryOutcome::PredecessorMismatch, Some(1_u64)), ]; for (expected, offset) in outcomes { - let (blocks, carry, facts) = swap(|identity| { + let (blocks, carry, facts, _) = swap(|identity| { offset.map(|offset| carry_source_zero(identity.wrapping_add(offset))) }); assert_eq!(carry, expected); @@ -4182,7 +4464,7 @@ mod tests { ); } // A predecessor index the predecessor does not have refuses the whole move. - let (blocks, carry, _) = swap(|identity| { + let (blocks, carry, _, _) = swap(|identity| { Some(GraphCarryProgram { predecessor: identity, sources: vec![(0, 1)].into_boxed_slice(), diff --git a/docs/C_ABI_V1_QUALIFICATION.md b/docs/C_ABI_V1_QUALIFICATION.md index a8c4ed2e2..56581d15c 100644 --- a/docs/C_ABI_V1_QUALIFICATION.md +++ b/docs/C_ABI_V1_QUALIFICATION.md @@ -188,6 +188,34 @@ mask and the signature, and the capi runtime tests cover reclaim by `service` al inside a source submit, the counter refresh, a scheduled candidate kept across 1,000 calls, and idle calls that change nothing. +A host learns where a seek landed through `miso_engine_v1_source_seek_report` (#1316, decision 15 +D15-12), an in-place V1 amendment: one new exported symbol and one new 64-byte struct, +`miso_engine_v1_seek_report` (`MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE`; not D6's +`miso_engine_v1_source_seek_report`, which is the function's name, and C gives a typedef and a +function one name space), announced by +`MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT` (the next free bit, 256) in the capability report, +whose feature mask grows to include it; no other struct, size or `ABI_VERSION` changes. It is a +session call, serialized with the others, and runs the service step first. It copies the source +ring's `(generation, first_sample, source_frame)` and its cumulative underrun and held frames: +the generation in effect, the source-read sample (the render clock in this version) of the first +block it played in, and the source frame that block started at, a late anchored seek's frame +including its lateness. Render publishes the record from the source consumer at the end of a +block's acquire step, only when a new generation took effect or a cumulative counter moved, +through the engine's one single-writer seqlock (`crates/engine/src/realtime/seqlock.rs`), so it is +a level, not a queue: no ack can precede a drop. Before the ring's first block it reads generation +0. The record belongs to the ring (one shared record charged as `seek_report_bytes` in the +source's overhead), so a plan swap that carries the source keeps it and an added or redeclared +source starts at 0. A host that plain-seeks one stem and reads `(g, s, f)` then starts another +with `seek_at(Y, g', f + (A - s), A)` and the two align exactly. Results: `OK`; +`INVALID_ARGUMENT` for a wrong `struct_size`, a nonzero reserved word or a malformed source ID, +and with `source.id.unknown` for an undeclared source; `BACKPRESSURE` with `source.report.busy` +when every bounded read (64 attempts) landed inside render's publication, `out` untouched. The +frozen exported set is now 18 `miso_engine_v1_*` definitions; `abi_smoke.c` checks the bit +against the queried mask and reads a compiled session's source (generation 0), an unknown source +and a wrong size, `header_smoke.cpp` pins the size, every field offset and the signature, and the +capi runtime tests cover a plain seek, a late anchored seek, a scrub-then-align across a +transaction, a carried and an added ring, and the two counters. + The first C11-static launch found one qualification-fixture error: it attempted generation-1 seek before the initial generation-1 submission and exited 13. No product byte or staged library was changed or rebuilt. The new consumer was corrected to submit generation 1 first, then seek and diff --git a/scripts/check-capi-abi.sh b/scripts/check-capi-abi.sh index 7903c5586..df7625828 100755 --- a/scripts/check-capi-abi.sh +++ b/scripts/check-capi-abi.sh @@ -261,6 +261,7 @@ printf '%s\n' \ miso_engine_v1_session_destroy \ miso_engine_v1_source_seek \ miso_engine_v1_source_seek_at \ + miso_engine_v1_source_seek_report \ miso_engine_v1_source_submit_planar_f32 \ miso_engine_v1_submit_command \ | sort >"$expected_symbols" diff --git a/tools/realtime-policy/src/policy.rs b/tools/realtime-policy/src/policy.rs index 48bbcff04..0469f6ec9 100644 --- a/tools/realtime-policy/src/policy.rs +++ b/tools/realtime-policy/src/policy.rs @@ -222,11 +222,12 @@ impl Policy { // (`crates/builtins`: the fader operation sink, and the bank's and the scalar track's // `process_with`) and #1639's two (`crates/graph`: the jump-length cell's word decode // and its block-entry read) and #1630's one (`crates/builtins-compiler`: the cell - // readers' carry marks, D15-2). The floors had fallen behind the tree before #1637; they - // are set to its counts again. A change that adds a marker raises them in the same - // change. - marked_files_floor: 32, - marked_regions_floor: 150, + // readers' carry marks, D15-2) and #1316's (`seqlock.rs`, the engine's one seqlock + // writer, B15-R1, and `crates/source`'s seek-report publication). The floors had fallen + // behind the tree before #1637; they are set to its counts again. A change that adds a + // marker raises them in the same change. + marked_files_floor: 33, + marked_regions_floor: 151, render_thread_exports: table::parse(RENDER_THREAD_EXPORTS) .unwrap_or_else(|error| panic!("render-thread-exports.json: {error}")), unwrapped_exports: UNWRAPPED_EXPORTS From 69d53a7a07fee48e3fa631e6c853e34f0623013b Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 04:17:05 +0000 Subject: [PATCH 02/10] #1316/#1293: the browser seek report is #1293 D6 with a parity row, independent of #1332 (root ruling) Co-Authored-By: Claude Opus 5.5 --- ...ored-seek-from-the-browser-engine-module.md | 18 ++++++++++++++++++ ...very-seek-on-the-plans-source-read-clock.md | 5 +++++ 2 files changed, 23 insertions(+) diff --git a/.github/ISSUE_SPECS/1293-export-transaction-apply-and-anchored-seek-from-the-browser-engine-module.md b/.github/ISSUE_SPECS/1293-export-transaction-apply-and-anchored-seek-from-the-browser-engine-module.md index 3431253d2..6ee8c7b0b 100644 --- a/.github/ISSUE_SPECS/1293-export-transaction-apply-and-anchored-seek-from-the-browser-engine-module.md +++ b/.github/ISSUE_SPECS/1293-export-transaction-apply-and-anchored-seek-from-the-browser-engine-module.md @@ -264,3 +264,21 @@ Done under B15-R8: #1386 and #1643 each carry an amendment of 2026-10-15. **Gates and commands.** Every named script and file exists. `build-web-audioworklet.sh --named-twin`, `check-sdk-generated.sh [DIR]`, the parameter-metadata `--check` (`tools/parameter-metadata/src/lib.rs:1143`) and `check-abi-layout-v1.py --self-test` are valid. + +## Amendment (root ruling, batch 15c, 2026-10-16): D6 owns the browser seek report + +- #1316 (batch 15c) shipped the core seek report and its C ABI export, + `miso_engine_v1_source_seek_report` (struct `miso_engine_v1_seek_report`, 64 bytes; the core + reads it through `SessionState::seek_report`, which runs the service step first). The browser + export is this issue's D6, in batch 16. It does not wait for #1332 or #1387, because #1332 waits + on the owner's iOS test and a parity gap there could wait indefinitely. In `single` mode D6 + ships on its own (B15-R8). +- D6 reads the same core call as the C ABI (thick core, thin clients): no seek-report logic in + host-web. Its record carries the same fields as the C struct, in the same order and with the + same meanings (generation in effect, source-read sample and source frame of its first block, + cumulative underrun and held frames). A busy read is `RESULT_BACKPRESSURE` and an unknown source + `RESULT_INVALID_ARGUMENT`, as the C ABI's `source.report.busy` and `source.id.unknown`. +- **New gate (parity row).** In `hosts/host-web/tests/control_parity.rs`: a plain seek and a late + `seek_at` on both hosts at 48 and 96 kHz; after each block, the browser record and the C ABI + struct are equal field by field, from the seek call to the block that applies it and after. + Red if either host reports at the call, names a different block, or drops the lateness. diff --git a/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md b/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md index 1d4b2d326..0be6bad31 100644 --- a/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md +++ b/.github/ISSUE_SPECS/1316-anchor-every-seek-on-the-plans-source-read-clock.md @@ -310,3 +310,8 @@ counters plus six `u64` words). host-web's `retained_ceilings` stays inside its **Successors.** The browser export is #1332 and #1293 (Non-goals). `observe.rs` onto `seqlock.rs` is #1482 D3. `source.id.invalid` on this entry point is #1350. + +**Successor owner (root ruling, 2026-10-16).** The browser export of the seek report is #1293 D6 +(batch 16) alone, with a parity row against the C ABI struct, independent of #1332 (which waits +on the owner's iOS test). Until #1293 lands, the browser cannot seek through the core, so it has +no seek to report; the C ABI reads the report today. #1293's amendment of 2026-10-16 records it. From dc4bd13acd97345ddae084f49acb24f07fe1baa9 Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 04:34:07 +0000 Subject: [PATCH 03/10] #1320: let a source consumer check and replay its next blocks for a prime - spsc Consumer::peek (shared read, changes nothing; unit and loom tests). - source: prime_ready (read-only readiness walk) and prime_block_at (plays the blocks the consumer would have played, observing no command), PcmSourcePrimeFlag shared by the handle and the consumer, prime_required_bytes report row (24 B per ring on 64-bit, 12 on wasm32). - 0 allocations for readiness and sixteen primes; realtime policy unchanged (151/33); audit capi unchanged (4f52089e4a8cf83c). Callers are #1355's. Co-Authored-By: Claude Opus 5.5 --- ...-and-replay-its-next-blocks-for-a-prime.md | 57 ++ crates/engine/src/realtime/spsc.rs | 134 ++++ crates/source/src/lib.rs | 570 +++++++++++++++++- crates/source/tests/prime.rs | 332 ++++++++++ 4 files changed, 1083 insertions(+), 10 deletions(-) create mode 100644 crates/source/tests/prime.rs diff --git a/.github/ISSUE_SPECS/1320-let-a-source-consumer-check-and-replay-its-next-blocks-for-a-prime.md b/.github/ISSUE_SPECS/1320-let-a-source-consumer-check-and-replay-its-next-blocks-for-a-prime.md index b49ca8b47..b1bcb9768 100644 --- a/.github/ISSUE_SPECS/1320-let-a-source-consumer-check-and-replay-its-next-blocks-for-a-prime.md +++ b/.github/ISSUE_SPECS/1320-let-a-source-consumer-check-and-replay-its-next-blocks-for-a-prime.md @@ -256,3 +256,60 @@ is not a control. the timeline consumer is for #1355 or #1641 to decide. **Dependencies.** #1316, #1318 and #1319 are open and in the batch. Nothing outside the batch. + +## Decision record: #1320 (batch 15c) + +**What shipped.** +- D1: `Consumer::peek(&self, i) -> Option<&T>` in `crates/engine/src/realtime/spsc.rs`, inside the + consumer's existing realtime-policy region. One `Acquire` load of the producer cursor (through + `available_at_entry`), no cursor, cache or counter change. A shared-access `sync::with` helper + (loom: `UnsafeCell::with`) sits beside `with_mut` in both `sync` shims; it is peek's read path. +- D2: `PcmSourceConsumer::prime_ready(&self, first_sample, lead_blocks)`. One command-queue load + and at most `lead_blocks + 1` peeks; the walk takes `current` first, then the queue; it returns + `true` as soon as the region's end is reached, because later blocks move no frame. +- D3: `PcmSourceConsumer::prime_block_at(&mut self, sample)`: `end_block`, + `flush_deferred_recycle`, `generation_changed = false`, then `begin_block_play(SeekClock::Hold)`. + Debug assertions check no underrun, no held frames and no command pop. +- D4: `prime_required` is one `Arc` per ring, made at preparation. + `PcmSourceProducer::prime_flag()` and `HostChunkProvider::prime_flag()` return a + `PcmSourcePrimeFlag` with `set(bool)` (`Relaxed`). `PcmSourceConsumer::prime_required()` loads it + (`Relaxed`). New resource row `SourceResourceReport::prime_required_bytes`. It is in + `overhead_bytes` and in the largest-allocation list. + +**Departures.** +- The prime's acquire stops before a block of an unobserved generation + (`CommandObservation::Withheld`, one `peek(0)` per pop in a prime only). D3 says the acquire + never pops past block `j`. That is not true at the region's end: there `begin_block_play` still + runs the acquire, and the acquire pops everything queued. A `Seek` pushed after a true + readiness, with its PCM behind the end block, would then make the prime observe it at once + (`:1408-1431`), and a plain seek would apply inside the window. The guard leaves that block + and its command queued. Ordinary blocks and `prepare_seek` pass `Allowed` and do not change. + Gate 4's end-at-`j = 1` case is red without the guard. +- The prime sets `next_source_read_sample = sample` before `begin_block_play(Hold)`, because + `Hold` reports on the consumer's own clock. Without this, a prime after a source-read clock + jump (#1639) would leave a later clockless block, and its seek report, on the old clock. + `the_prime_continues_the_callers_clock_across_a_jump` defends this. +- No new marked region. `peek`, `prime_ready` and `prime_block_at` are in existing regions, so + the realtime-policy floors do not change (151 regions in 33 files). +- No browser or C ABI surface: #1355 owns the callers. The browser's `source_overhead_bytes` + grows by the flag's bytes per ring. The host-web ceilings test still passes, so no ceiling moves. + +**Gates.** +1. `realtime::spsc::tests::peek_returns_each_queued_item_in_order_across_wraps_and_changes_nothing` + and the loom test `realtime::spsc::loom_tests::spsc_loom_peek_returns_the_item_later_popped` + (`spsc_loom` CI command: 15 passed). +2. `crates/source/src/lib.rs` `tests::prime_readiness::*` (11 tests). Each readiness call + asserts that telemetry, `next_frame`, `end_frame`, the data, recycle and command queue counts, + the pop counts, the held seek, `current`, the clock and the seek report did not change. Held + anchors inside or late in the window are set directly, because `try_seek` refuses unaligned + anchors. Gaps and stale blocks are pushed raw, because the host path refuses them. +3. `crates/source/tests/prime.rs::the_prime_equals_the_blocks_the_consumer_would_have_played` + (steady, region end at `j = 1`, a seek 2 blocks earlier). +4. `the_prime_observes_no_command` (steady and end at `j = 1`, with the seek's PCM queued). +5. `the_prime_flag_is_shared_between_the_handle_and_the_consumer`. +6. `readiness_and_sixteen_primes_allocate_nothing`: `k = 16`, 16 primes, 0 allocations and + 0 frees (current-thread counters). + +**Measured.** `prime_required_bytes` = 24 B per ring on 64-bit targets and 12 B on wasm32 +(three words). `audit capi` reports allocations 0 and pcm_digest `4f52089e4a8cf83c`, which is +unchanged. diff --git a/crates/engine/src/realtime/spsc.rs b/crates/engine/src/realtime/spsc.rs index 5776feaa6..61fa9720a 100644 --- a/crates/engine/src/realtime/spsc.rs +++ b/crates/engine/src/realtime/spsc.rs @@ -28,6 +28,12 @@ mod sync { body(cell.get()) } + /// Shared access to a slot's storage, spelled the way loom's `UnsafeCell` spells it. + #[inline(always)] + pub(super) fn with(cell: &UnsafeCell, body: impl FnOnce(*const T) -> R) -> R { + body(cell.get()) + } + /// Read a cursor with exclusive access, without an atomic operation. #[inline(always)] pub(super) fn load_exclusive(cursor: &mut AtomicUsize) -> usize { @@ -44,6 +50,10 @@ mod sync { cell.with_mut(body) } + pub(super) fn with(cell: &UnsafeCell, body: impl FnOnce(*const T) -> R) -> R { + cell.with(body) + } + /// Loom's `AtomicUsize` has no `get_mut`; `unsync_load` is its exclusive-access read. pub(super) fn load_exclusive(cursor: &mut AtomicUsize) -> usize { // SAFETY: `&mut` proves no other thread can be touching this cursor. @@ -427,6 +437,36 @@ impl Consumer { self.ring().slots_len - consumer + producer } } + /// Borrow the `index`-th queued item, counted from this consumer's cursor, without popping it + /// (#1320 D1). `None` when `index` is not below [`Self::available_at_entry`]. + /// + /// The producer cursor is loaded once with `Acquire`, as [`Self::available_at_entry`] loads + /// it; no cursor, cache or counter changes. The read is sound because the producer writes only + /// the slots from its own cursor up to (excluding) this consumer's: every slot in + /// `[local, producer)` was initialized by a `Release`-published push and nothing rewrites it + /// until this consumer pops it. The borrow is tied to `&self`, so no pop can happen while it + /// is held. Wait-free: one atomic load. + #[must_use] + pub fn peek(&self, index: usize) -> Option<&T> { + if index >= self.available_at_entry() { + return None; + } + let ring = self.ring(); + // `index < available <= logical_capacity < slots_len`, so one subtraction wraps it, and + // the comparison form cannot overflow. + let to_end = ring.slots_len - self.local; + let slot = if index >= to_end { + index - to_end + } else { + self.local + index + }; + // SAFETY: `slot` is in `[local, producer)` of the producer cursor loaded with `Acquire` + // above, so its push's `Release` publication happened-before this read and the producer + // cannot write it again until this consumer pops it, which needs `&mut self`. + Some(sync::with(&ring.slots[slot], |slot| unsafe { + (*slot).assume_init_ref() + })) + } /// Whether the queue is empty, reloading the shared line only if it looks so. /// /// See [`Consumer::cached_producer`] for why one reload settles it. @@ -1003,6 +1043,46 @@ mod loom_tests { }); } + /// #1320 gate 1: the consumer peeks while the producer pushes three items through a two-slot + /// queue. Under every interleaving each peeked item is the one the consumer pops next (or, + /// at index 1, the one after it), and loom's cells fail the model if a peek reads a slot the + /// producer is writing. + #[test] + fn spsc_loom_peek_returns_the_item_later_popped() { + loom::model(|| { + let (mut producer, mut consumer) = + bounded_spsc::(NonZeroUsize::new(2).expect("two"), QueueGeneration(1)) + .expect("ring"); + let writer = loom::thread::spawn(move || { + for value in 0..3 { + while producer.try_push(value).is_err() { + loom::thread::yield_now(); + } + } + }); + let reader = loom::thread::spawn(move || { + for expected in 0..3 { + loop { + let Some(&peeked) = consumer.peek(0) else { + loom::thread::yield_now(); + continue; + }; + let behind = consumer.peek(1).copied(); + assert_eq!(peeked, expected); + assert_eq!(consumer.try_pop(), Ok(peeked)); + if let Some(behind) = behind { + assert_eq!(consumer.peek(0).copied(), Some(behind)); + assert_eq!(behind, expected + 1); + } + break; + } + } + }); + writer.join().expect("producer"); + reader.join().expect("consumer"); + }); + } + /// A mailbox payload that counts its drops, so every model can prove that each published /// value ends exactly once: adopted by render, held by control, or dropped with the mailbox. /// Its drop stands for the return of the candidate's retirement credit. @@ -1617,6 +1697,60 @@ mod tests { assert_eq!(consumer.available_at_entry(), 1); } + /// #1320 gate 1: `peek(i)` is the `i`-th queued item for every `i` below the queued count and + /// `None` at it, across several wraps of the cursor space, and no number of peeks moves a + /// cursor, the producer cache or a counter. + #[test] + fn peek_returns_each_queued_item_in_order_across_wraps_and_changes_nothing() { + let (mut producer, mut consumer) = + bounded_spsc(NonZeroUsize::new(3).expect("capacity"), QueueGeneration(17)) + .expect("queue"); + assert_eq!(consumer.peek(0), None); + let mut next_push = 0_u64; + let mut next_pop = 0_u64; + // Push 1, 2, 3, 1, 2, 3 ... items and pop them again, cycling the four-slot ring more than + // ten times so every cursor position is peeked from, with the queue full at some. + for round in 0..24_u64 { + let pushes = 1 + round % 3; + for _ in 0..pushes { + if producer.try_push(next_push).is_ok() { + next_push += 1; + } + } + let queued = consumer.available_at_entry(); + let before = ( + consumer.local, + consumer.cached_producer, + consumer.success_count(), + consumer.empty_count(), + ); + for _ in 0..2 { + for index in 0..queued { + let expected = next_pop + u64::try_from(index).expect("index"); + assert_eq!(consumer.peek(index), Some(&expected), "round {round}"); + } + assert_eq!(consumer.peek(queued), None, "round {round}"); + assert_eq!(consumer.peek(usize::MAX), None, "round {round}"); + } + assert_eq!( + ( + consumer.local, + consumer.cached_producer, + consumer.success_count(), + consumer.empty_count(), + ), + before, + "round {round}" + ); + assert_eq!(consumer.available_at_entry(), queued); + for _ in 0..pushes.min(u64::try_from(queued).expect("queued")) { + assert_eq!(consumer.try_pop(), Ok(next_pop)); + next_pop += 1; + } + } + assert!(next_pop > 8, "the cursors wrapped the four-slot ring"); + } + #[test] fn producer_available_capacity_is_acquired_once_and_wrap_safe() { let (mut producer, mut consumer) = diff --git a/crates/source/src/lib.rs b/crates/source/src/lib.rs index cfc1be345..6f7e8c661 100644 --- a/crates/source/src/lib.rs +++ b/crates/source/src/lib.rs @@ -7,8 +7,14 @@ #![allow(missing_docs)] -use core::{alloc::Layout, cmp, mem::size_of, num::NonZeroUsize}; -use std::fmt; +use core::{ + alloc::Layout, + cmp, + mem::size_of, + num::NonZeroUsize, + sync::atomic::{AtomicBool, AtomicUsize, Ordering}, +}; +use std::{fmt, sync::Arc}; use engine::{ QuantumFrames, SampleRateHz, @@ -224,6 +230,9 @@ pub struct SourceResourceReport { pub command_queue_bytes: u64, /// The ring's shared seek report (#1316 D2): one record, with its shared-owner header. pub seek_report_bytes: u64, + /// The ring's shared `prime_required` flag (#1320 D4): one `AtomicBool`, with its + /// shared-owner header. + pub prime_required_bytes: u64, /// One `TransferBlock` allocation per allocated block (configured plus retained), excluding /// its PCM allocation. pub transfer_block_metadata_bytes: u64, @@ -582,6 +591,8 @@ impl PcmSourceRing { let retained_pcm = checked_u64_mul(block_pcm, retained_block_count)?; let seek_report = u64::try_from(published_words_retained_bytes::()) .map_err(|_| PcmSourceRingError::PlatformSizeLimit)?; + let prime_required = u64::try_from(prime_flag_retained_bytes()) + .map_err(|_| PcmSourceRingError::PlatformSizeLimit)?; let metadata = checked_u64_mul( shape.allocated_block_count_u64, u64::try_from(size_of::()) @@ -592,7 +603,7 @@ impl PcmSourceRing { checked_u64_add(data_queue, recycle_queue)?, checked_u64_add(command_queue, metadata)?, )?, - checked_u64_add(retained_pcm, seek_report)?, + checked_u64_add(checked_u64_add(retained_pcm, seek_report)?, prime_required)?, )?; let total = checked_u64_add(pcm, overhead)?; let largest = [ @@ -602,6 +613,7 @@ impl PcmSourceRing { largest_queue_allocation::>(shape.allocated_block_count)?, largest_queue_allocation::(command_capacity)?, seek_report, + prime_required, ] .into_iter() .max() @@ -614,6 +626,7 @@ impl PcmSourceRing { recycle_queue_bytes: recycle_queue, command_queue_bytes: command_queue, seek_report_bytes: seek_report, + prime_required_bytes: prime_required, transfer_block_metadata_bytes: metadata, transfer_block_pcm_bytes: pcm, retained_block_pcm_bytes: retained_pcm, @@ -667,6 +680,8 @@ impl PcmSourceRing { // Generation 0 until the first block: "not yet rendered" (#1316 D3). let (seek_report_writer, seek_report_reader) = published_words(SourceSeekReport::default().to_words()); + // #1320 D4: no warm successor carries a new ring. + let prime_required = Arc::new(AtomicBool::new(false)); let mut consumer = PcmSourceConsumer { data_consumer, recycle_producer, @@ -702,6 +717,7 @@ impl PcmSourceRing { seek_report: seek_report_writer, published_seek_report: SourceSeekReport::default(), next_source_read_sample: 0, + prime_required: Arc::clone(&prime_required), }; // The producer starts with exactly the configured blocks; the retained block above never // enters the recycle queue until the consumer has a played block to hold instead. @@ -727,6 +743,7 @@ impl PcmSourceRing { deferred_block: None, native_decoder_sanitized_samples: 0, seek_report: seek_report_reader, + prime_required, }, consumer, report, @@ -814,6 +831,34 @@ pub struct PcmSourceProducer { native_decoder_sanitized_samples: u64, /// The reader of the ring's seek report, which the consumer publishes (#1316 D2). seek_report: PublishedWordsReader, + /// The ring's `prime_required` flag, shared with the consumer (#1320 D4). + prime_required: Arc, +} + +/// The control thread's handle on one ring's `prime_required` flag (#1320 D4). +/// +/// Control sets it for the rings a warm successor carries, before the `Release` that publishes +/// the candidate; render loads it after its `Acquire` of the plan mailbox, so the flag needs no +/// ordering of its own. A handle is an `Arc` clone taken off render. +#[derive(Clone, Debug)] +pub struct PcmSourcePrimeFlag(Arc); + +impl PcmSourcePrimeFlag { + /// Mark the ring as one a warm successor carries (`true`) or not (`false`). + pub fn set(&self, required: bool) { + self.0.store(required, Ordering::Relaxed); + } +} + +/// Exact bytes of the ring's shared `prime_required` allocation: `Arc`'s `{ strong, weak, data }`. +fn prime_flag_retained_bytes() -> usize { + #[repr(C)] + struct SharedAllocation { + strong: AtomicUsize, + weak: AtomicUsize, + flag: AtomicBool, + } + Layout::new::().size() } /// Name for the prepared producer when it is used solely to control source seeks. @@ -883,6 +928,12 @@ impl PcmSourceProducer { .map_err(|_| SourceSeekReportBusy) } + /// A handle on the ring's `prime_required` flag (#1320 D4). Control only; never on render. + #[must_use] + pub fn prime_flag(&self) -> PcmSourcePrimeFlag { + PcmSourcePrimeFlag(Arc::clone(&self.prime_required)) + } + /// Copy producer-local telemetry outside render. #[must_use] pub fn telemetry(&self) -> SourceProducerTelemetry { @@ -999,6 +1050,12 @@ impl HostChunkProvider { pub fn seek_report(&self) -> Result { self.producer.seek_report() } + + /// The ring's `prime_required` flag; see [`PcmSourceProducer::prime_flag`]. + #[must_use] + pub fn prime_flag(&self) -> PcmSourcePrimeFlag { + self.producer.prime_flag() + } } fn validate_host_chunk( @@ -1133,6 +1190,8 @@ pub struct PcmSourceConsumer { /// The source-read sample of the block after the last one begun: the clock a clockless /// [`PcmSourceConsumer::begin_block`] continues, so its blocks are consecutive quanta. next_source_read_sample: u64, + /// The ring's `prime_required` flag, which control sets (#1320 D4). + prime_required: Arc, } /// A [`SourceCommand::SeekAt`] the consumer has popped and holds until its anchor block. @@ -1160,6 +1219,16 @@ enum SeekClock { Hold, } +/// Whether an acquire may observe the admitted command when it meets a block of an unobserved +/// generation (#1320 D3). +#[derive(Clone, Copy, Eq, PartialEq)] +enum CommandObservation { + /// Every ordinary block and `prepare_seek`: pop the block, then observe its seek. + Allowed, + /// A prime: stop before such a block and leave it and its command queued. + Withheld, +} + impl PcmSourceConsumer { /// Apply an already-admitted seek on the exclusive consumer owner between blocks. /// Recycles stale storage and retains current-generation PCM without consuming a frame. @@ -1183,7 +1252,7 @@ impl PcmSourceConsumer { { return false; } - self.acquire_current_block(SeekClock::Hold); + self.acquire_current_block(SeekClock::Hold, CommandObservation::Allowed); // A producer running concurrently may push a newer seek and its block between the command // pop above and the data pop in the acquire; the acquire's re-observe takes that seek. // Readiness is granted for `generation` alone, so check again. @@ -1249,7 +1318,7 @@ impl PcmSourceConsumer { fn begin_block_with(&mut self, clock: SeekClock) -> SourceReadReport { self.begun_block = true; self.begin_block_observe(clock); - self.begin_block_play(clock) + self.begin_block_play(clock, CommandObservation::Allowed) } /// The first half of [`Self::begin_block_with`]: release the played block and observe the @@ -1263,8 +1332,12 @@ impl PcmSourceConsumer { } /// The second half of [`Self::begin_block_with`]: acquire and play this block. - fn begin_block_play(&mut self, clock: SeekClock) -> SourceReadReport { - self.acquire_current_block(clock); + fn begin_block_play( + &mut self, + clock: SeekClock, + observation: CommandObservation, + ) -> SourceReadReport { + self.acquire_current_block(clock, observation); // Every seek this block can apply has applied by now (the observe step or the acquire's // re-observe), and nothing below has advanced: this is the frame the block reads from. let source_frame = self.next_frame; @@ -1430,6 +1503,14 @@ impl PcmSourceConsumer { } // REALTIME_POLICY_END + /// Whether a warm successor carries this ring and so needs it primed (#1320 D4). Control sets + /// it through [`PcmSourcePrimeFlag`] before it publishes that successor; render reads it + /// after its `Acquire` of the plan mailbox, so the `Relaxed` load is ordered by that. + #[must_use] + pub fn prime_required(&self) -> bool { + self.prime_required.load(Ordering::Relaxed) + } + /// Exact source channel count. #[must_use] pub const fn channel_count(&self) -> u32 { @@ -1572,13 +1653,123 @@ impl PcmSourceConsumer { } } // REALTIME_POLICY_BEGIN + /// Whether this consumer would play each of its next `lead_blocks + 1` blocks from queued PCM, + /// with no underrun, no discard and no command (#1320 D2): the readiness check render makes + /// in the block before it adopts a warm successor whose sources read `lead_blocks` quanta + /// further ahead. + /// + /// `first_sample` is the source-read sample of the block render is about to run on the active + /// plan. `true` exactly when the command queue is empty, no held anchored seek is anchored + /// before `first_sample + (lead_blocks + 1) * quantum`, and for each of those blocks either + /// the region's end is reached or the next block in queue order (the pending block first, then + /// the data queue) is of the active generation and starts exactly where the consumer reads. + /// A short block must carry `end_of_region`. + /// + /// `false` (a gap, another generation, a block the consumer would discard, a held seek's + /// pending block, too few blocks queued) only delays adoption: each clears as render keeps + /// playing. Read-only: it changes no state, and costs one command-queue load and at most + /// `lead_blocks + 1` data-queue loads, all `Acquire`, with no allocation, lock or syscall. + /// Only render changes the consumer and the producer can only add blocks or a command, so the + /// answer holds for the rest of the render callback for [`Self::prime_block_at`], which + /// observes no command. + #[must_use] + pub fn prime_ready(&self, first_sample: u64, lead_blocks: u32) -> bool { + if self.command_consumer.available_at_entry() != 0 { + return false; + } + let quantum = u64::from(self.quantum_frames); + let Some(window_end) = u64::from(lead_blocks) + .checked_add(1) + .and_then(|blocks| blocks.checked_mul(quantum)) + .and_then(|window| first_sample.checked_add(window)) + else { + return false; + }; + // A held anchor is always past the last block's sample, so this is every held seek that + // would apply at one of the blocks, a late one at `first_sample` included. + if self + .held_seek + .is_some_and(|held| held.anchor_sample < window_end) + { + return false; + } + let mut frame = self.next_frame.0; + let mut end = self.end_frame.map(|end| end.0); + let mut pending = self.current.as_deref(); + let mut queued = 0_usize; + for _ in 0..=lead_blocks { + if end.is_some_and(|end| frame >= end) { + // At the region's end a block reads nothing and moves no frame: so are the rest. + return true; + } + let block = if let Some(block) = pending.take() { + block + } else { + let Some(block) = self.data_consumer.peek(queued) else { + return false; + }; + queued += 1; + &**block + }; + if block.generation != self.active_generation + || block.start_frame.0 != frame + || (block.frames < self.quantum_frames && !block.end_of_region) + { + return false; + } + frame = frame.saturating_add(u64::from(block.frames)); + if block.end_of_region { + end = Some(frame); + } + } + true + } + + /// Replay the next block exactly as [`Self::begin_block_at`]`(source_read_sample)` would play + /// it, without observing any command (#1320 D3): a warm successor's prime, in its adoption + /// block. + /// + /// It releases the played block and plays, discards and recycles as an ordinary block does, + /// but neither pops the command queue nor applies a held seek, and its acquire stops before a + /// block of a generation it has not observed, so the command queue and any held seek are + /// unchanged. The played planes are lent by [`Self::played_plane`] until the next `&mut self` + /// call, as for any block. + /// + /// Precondition: [`Self::prime_ready`]`(first_sample, k)` was `true` in this render callback, + /// and the calls are `prime_block_at(first_sample + j * quantum)` for `j = 0..k`, in order. + /// Then no call underruns: each block matches the frame the consumer reads, or the region has + /// ended. Allocation-free and wait-free. + pub fn prime_block_at(&mut self, source_read_sample: u64) -> SourceReadReport { + let commands_popped = self.command_consumer.success_count(); + self.begun_block = true; + self.end_block(); + self.flush_deferred_recycle(); + self.generation_changed = false; + // `Hold` reports on the consumer's own clock; this block's sample is the caller's. + self.next_source_read_sample = source_read_sample; + let report = self.begin_block_play(SeekClock::Hold, CommandObservation::Withheld); + debug_assert!( + report.underrun_frames == 0 && report.held_frames == 0, + "a prime after a true readiness neither underruns nor holds" + ); + debug_assert_eq!( + self.command_consumer.success_count(), + commands_popped, + "a prime observes no command" + ); + report + } + /// Pop toward the block to play. `clock` is the block's, for a seek observed here. /// /// The call pops at most the blocks queued when it begins (`available`), never the blocks the /// host publishes while it runs: a count taken at entry, not the ring's capacity, bounds it, so /// a host that keeps the ring fed cannot hold the render here. Two counted phases, with the /// one seek re-observe between them and in neither loop. - fn acquire_current_block(&mut self, clock: SeekClock) { + /// + /// With [`CommandObservation::Withheld`] (a prime) it stops before a block of an unobserved + /// generation, which stays queued with its command, so it never observes a command. + fn acquire_current_block(&mut self, clock: SeekClock, observation: CommandObservation) { if self.current.is_some() { return; } @@ -1589,6 +1780,14 @@ impl PcmSourceConsumer { let mut popped = 0_usize; let mut unobserved = None; for _ in 0..available { + if observation == CommandObservation::Withheld + && self + .data_consumer + .peek(0) + .is_some_and(|block| self.is_unobserved_generation(block.generation)) + { + return; + } let Ok(block) = self.data_consumer.try_pop() else { break; }; @@ -2350,6 +2549,7 @@ mod tests { + report.transfer_block_metadata_bytes + report.retained_block_pcm_bytes + report.seek_report_bytes + + report.prime_required_bytes ); // #1316 D2: the seek report is one shared record of a sequence word and five report // words, behind `Arc`'s two counters, on every target. @@ -2357,6 +2557,12 @@ mod tests { report.seek_report_bytes, u64::try_from(2 * size_of::() + 6 * size_of::()).expect("record size") ); + // #1320 D4: the `prime_required` flag is one `AtomicBool` behind `Arc`'s two counters, + // padded to the counters' alignment: three words on every target. + assert_eq!( + report.prime_required_bytes, + u64::try_from(3 * size_of::()).expect("flag size") + ); // The memory delta over the pre-#917 two-block ring, which sized both queues and the // metadata at two blocks: one block's PCM, one block's metadata and one pointer slot in // each queue. Nothing moves into the session-charged PCM row. @@ -2364,6 +2570,7 @@ mod tests { let pre_change_overhead = 2 * queue_bytes::>(two).expect("queue") + report.command_queue_bytes + report.seek_report_bytes + + report.prime_required_bytes + 2 * metadata; let slot = u64::try_from(size_of::>()).expect("slot size"); assert_eq!( @@ -2382,6 +2589,7 @@ mod tests { .max(32) .max(metadata) .max(report.seek_report_bytes) + .max(report.prime_required_bytes) ); } @@ -2394,7 +2602,7 @@ mod tests { host.submit(chunk(1, 0, &[&old], 4, false)).unwrap(); host.submit(chunk(1, 4, &[&old], 4, false)).unwrap(); if retained { - consumer.acquire_current_block(SeekClock::Now); + consumer.acquire_current_block(SeekClock::Now, CommandObservation::Allowed); } host.try_seek(SourceCommand::Seek { generation: SourceGeneration(2), @@ -2611,7 +2819,7 @@ mod tests { ) -> ([f32; 4], SourceReadReport) { consumer.begin_block_observe(clock); producer(); - let report = consumer.begin_block_play(clock); + let report = consumer.begin_block_play(clock, CommandObservation::Allowed); let mut output = [f32::NAN; 4]; consumer.copy_channel(0, &mut output).unwrap(); consumer.end_block(); @@ -5066,4 +5274,346 @@ mod tests { assert!(!swapped(&[]), "an empty donation takes nothing"); } } + + /// #1320 gate 2: readiness at its exact boundaries, on a two-channel ring of quantum 128 + /// with `k = 3`. Every readiness call is wrapped by [`prime_ready_checked`], which asserts the + /// call changed nothing a twin that made no call would show. + mod prime_readiness { + use super::*; + + const Q: u32 = 128; + const K: u32 = 3; + + fn ring() -> (HostChunkProvider, PcmSourceConsumer) { + let (producer, consumer, _) = + PcmSourceRing::prepare(config(2, Q, 16 * u64::from(Q))).expect("ring"); + (producer.into_host_chunk_provider(RATE), consumer) + } + + fn sample(frame: u64, channel: u64) -> f32 { + #[allow(clippy::cast_precision_loss)] + let value = (frame * 2 + channel) as f32; + value + } + + /// Submit one block of `frames` encoded frames through the validating host path. + fn submit( + host: &mut HostChunkProvider, + generation: u64, + start: u64, + frames: u32, + end: bool, + ) { + let planes: [Vec; 2] = core::array::from_fn(|channel| { + (0..u64::from(frames)) + .map(|frame| sample(start + frame, channel as u64)) + .collect() + }); + host.submit(chunk( + generation, + start, + &[&planes[0], &planes[1]], + frames, + end, + )) + .expect("valid block"); + } + + /// Publish one block with no submission check: a gap or a stale block the host path + /// refuses, as a ring the consumer advanced past would hold it. + fn push_raw(host: &mut HostChunkProvider, generation: u64, start: u64) { + let producer = &mut host.producer; + let mut block = producer.take_recycled_block().expect("free block"); + block.generation = SourceGeneration(generation); + block.start_frame = SourceFrame(start); + block.frames = Q; + block.end_of_region = false; + assert!( + producer.data_producer.try_push(block).is_ok(), + "queue space" + ); + } + + #[derive(Debug, PartialEq)] + struct Twin { + telemetry: SourceConsumerTelemetry, + next_frame: SourceFrame, + end_frame: Option, + data_queued: usize, + recycle_free: usize, + commands_queued: usize, + commands_popped: u64, + data_popped: u64, + held: Option<(SourceGeneration, SourceFrame, u64)>, + current: Option<(SourceGeneration, SourceFrame)>, + next_source_read_sample: u64, + seek_report: SourceSeekReport, + } + + fn twin(consumer: &PcmSourceConsumer) -> Twin { + Twin { + telemetry: consumer.telemetry(), + next_frame: consumer.next_frame, + end_frame: consumer.end_frame, + data_queued: consumer.data_consumer.available_at_entry(), + recycle_free: consumer.recycle_producer.available_capacity(), + commands_queued: consumer.command_consumer.available_at_entry(), + commands_popped: consumer.command_consumer.success_count(), + data_popped: consumer.data_consumer.success_count(), + held: consumer + .held_seek + .map(|held| (held.generation, held.frame, held.anchor_sample)), + current: consumer + .current + .as_ref() + .map(|block| (block.generation, block.start_frame)), + next_source_read_sample: consumer.next_source_read_sample, + seek_report: consumer.published_seek_report, + } + } + + fn prime_ready_checked(consumer: &PcmSourceConsumer, first_sample: u64, k: u32) -> bool { + let before = twin(consumer); + let ready = consumer.prime_ready(first_sample, k); + assert_eq!(twin(consumer), before, "readiness changed the consumer"); + ready + } + + /// One ordinary block at `first_sample`. + fn ordinary(consumer: &mut PcmSourceConsumer, first_sample: u64) -> SourceReadReport { + let report = consumer.begin_block_at(first_sample); + consumer.end_block(); + report + } + + #[test] + fn k_plus_one_contiguous_blocks_are_ready_and_k_are_not() { + let (mut host, consumer) = ring(); + for block in 0..3 { + submit(&mut host, 1, block * u64::from(Q), Q, false); + } + assert!(!prime_ready_checked(&consumer, 0, K), "only k blocks"); + assert!( + prime_ready_checked(&consumer, 0, K - 1), + "k blocks for k - 1" + ); + submit(&mut host, 1, 3 * u64::from(Q), Q, false); + assert!(prime_ready_checked(&consumer, 0, K), "k + 1 blocks"); + assert!(!prime_ready_checked(&consumer, 0, K + 1), "one block past"); + } + + #[test] + fn a_held_current_block_is_block_zero() { + let (mut host, mut consumer) = ring(); + for block in 0..3 { + submit(&mut host, 1, block * u64::from(Q), Q, false); + } + // `prepare_seek` of the playing generation pops block 0 into `current`. + assert!(consumer.prepare_seek(SourceGeneration(1), SourceFrame(0))); + assert!(consumer.current.is_some()); + assert!( + !prime_ready_checked(&consumer, 0, K), + "current plus two queued" + ); + assert!(prime_ready_checked(&consumer, 0, K - 1)); + submit(&mut host, 1, 3 * u64::from(Q), Q, false); + assert!( + prime_ready_checked(&consumer, 0, K), + "current plus three queued" + ); + } + + #[test] + fn a_region_ending_inside_the_window_is_ready_only_with_its_end_block() { + let (mut host, consumer) = ring(); + submit(&mut host, 1, 0, Q, false); + assert!( + !prime_ready_checked(&consumer, 0, K), + "the end block is missing" + ); + submit(&mut host, 1, u64::from(Q), Q / 2, true); + assert!( + prime_ready_checked(&consumer, 0, K), + "a short end block at j = 1" + ); + assert!( + prime_ready_checked(&consumer, 0, K + 8), + "the end holds past it" + ); + } + + #[test] + fn a_short_block_without_its_end_is_not_ready() { + let (mut host, consumer) = ring(); + submit(&mut host, 1, 0, Q, false); + let producer = &mut host.producer; + let mut block = producer.take_recycled_block().expect("free block"); + block.generation = SourceGeneration(1); + block.start_frame = SourceFrame(u64::from(Q)); + block.frames = Q / 2; + block.end_of_region = false; + assert!( + producer.data_producer.try_push(block).is_ok(), + "queue space" + ); + assert!(!prime_ready_checked(&consumer, 0, K)); + assert!( + prime_ready_checked(&consumer, 0, 0), + "block 0 alone is whole" + ); + } + + #[test] + fn a_one_quantum_gap_at_j_two_is_not_ready() { + let (mut host, consumer) = ring(); + push_raw(&mut host, 1, 0); + push_raw(&mut host, 1, u64::from(Q)); + push_raw(&mut host, 1, 3 * u64::from(Q)); + push_raw(&mut host, 1, 4 * u64::from(Q)); + assert!(!prime_ready_checked(&consumer, 0, K), "a gap at j = 2"); + assert!( + prime_ready_checked(&consumer, 0, 1), + "the window before the gap" + ); + } + + #[test] + fn a_queued_plain_seek_is_not_ready_until_an_ordinary_block_applies_it() { + let (mut host, mut consumer) = ring(); + for block in 0..4 { + submit(&mut host, 1, block * u64::from(Q), Q, false); + } + assert!(prime_ready_checked(&consumer, 0, K)); + host.try_seek(SourceCommand::Seek { + generation: SourceGeneration(2), + frame: SourceFrame(5_000), + }) + .expect("seek"); + assert!(!prime_ready_checked(&consumer, 0, K), "a queued seek"); + for block in 0..5 { + submit(&mut host, 2, 5_000 + block * u64::from(Q), Q, false); + } + assert!(!prime_ready_checked(&consumer, 0, K), "still queued"); + let report = ordinary(&mut consumer, 0); + assert!(report.generation_changed && report.copied_frames == Q); + assert!( + prime_ready_checked(&consumer, u64::from(Q), K), + "k + 1 new blocks" + ); + assert!(!prime_ready_checked(&consumer, u64::from(Q), K + 1)); + } + + #[test] + fn a_queued_anchored_seek_is_not_ready_until_it_applies() { + let (mut host, mut consumer) = ring(); + for block in 0..4 { + submit(&mut host, 1, block * u64::from(Q), Q, false); + } + // Anchored at the second block: the first block observes and holds it, the second + // applies it. + host.try_seek(seek_at(2, 5_000, u64::from(Q))) + .expect("seek"); + assert!( + !prime_ready_checked(&consumer, 0, K), + "a queued anchored seek" + ); + for block in 0..5 { + submit(&mut host, 2, 5_000 + block * u64::from(Q), Q, false); + } + let first = ordinary(&mut consumer, 0); + assert!(!first.generation_changed && first.copied_frames == Q); + assert!(consumer.held_seek.is_some()); + assert!( + !prime_ready_checked(&consumer, u64::from(Q), K), + "held, anchored at the next block" + ); + let second = ordinary(&mut consumer, u64::from(Q)); + assert!(second.generation_changed && second.copied_frames == Q); + assert!(prime_ready_checked(&consumer, 2 * u64::from(Q), K)); + } + + #[test] + fn a_far_anchored_seek_once_held_leaves_the_window_ready() { + let (mut host, mut consumer) = ring(); + for block in 0..5 { + submit(&mut host, 1, block * u64::from(Q), Q, false); + } + host.try_seek(seek_at(2, 5_000, 64 * u64::from(Q))) + .expect("seek"); + assert!(!prime_ready_checked(&consumer, 0, K), "queued"); + ordinary(&mut consumer, 0); + assert!(consumer.held_seek.is_some()); + assert!( + prime_ready_checked(&consumer, u64::from(Q), K), + "held past the window" + ); + } + + #[test] + fn a_held_seek_anchored_inside_the_window_or_late_is_not_ready() { + let (mut host, mut consumer) = ring(); + for block in 0..6 { + submit(&mut host, 1, block * u64::from(Q), Q, false); + } + ordinary(&mut consumer, 0); + let first_sample = u64::from(Q); + let window_end = first_sample + u64::from(K + 1) * u64::from(Q); + assert!(prime_ready_checked(&consumer, first_sample, K)); + let hold = |consumer: &mut PcmSourceConsumer, anchor_sample| { + consumer.held_seek = Some(HeldSeek { + generation: SourceGeneration(2), + frame: SourceFrame(9_000), + anchor_sample, + }); + }; + hold(&mut consumer, window_end - 1); + assert!( + !prime_ready_checked(&consumer, first_sample, K), + "last frame inside" + ); + hold(&mut consumer, window_end); + assert!( + prime_ready_checked(&consumer, first_sample, K), + "first frame past" + ); + // Held because the last block began at `first_sample - q`, before the anchor; it + // applies late at `first_sample`. + hold(&mut consumer, first_sample - 1); + assert!(!prime_ready_checked(&consumer, first_sample, K), "late"); + } + + #[test] + fn stale_blocks_ahead_are_not_ready_until_ordinary_blocks_discard_them() { + let (mut host, mut consumer) = ring(); + // The consumer reads generation 2 from frame 0; two generation-1 blocks are queued + // ahead of its PCM. + consumer.active_generation = SourceGeneration(2); + push_raw(&mut host, 1, 0); + push_raw(&mut host, 1, u64::from(Q)); + for block in 0..5 { + push_raw(&mut host, 2, block * u64::from(Q)); + } + assert!(!prime_ready_checked(&consumer, 0, K), "stale blocks ahead"); + let report = ordinary(&mut consumer, 0); + assert_eq!(report.copied_frames, Q); + assert_eq!(consumer.telemetry().stale_generation_discard_count, 2); + assert!(prime_ready_checked(&consumer, u64::from(Q), K), "discarded"); + } + + #[test] + fn a_block_behind_the_read_frame_is_not_ready() { + let (mut host, mut consumer) = ring(); + // An underrun moved the read frame a quantum past the host's next block. + ordinary(&mut consumer, 0); + for block in 0..5 { + submit(&mut host, 1, block * u64::from(Q), Q, false); + } + assert!( + !prime_ready_checked(&consumer, u64::from(Q), K), + "block 0 is behind" + ); + ordinary(&mut consumer, u64::from(Q)); + assert!(prime_ready_checked(&consumer, 2 * u64::from(Q), K - 1)); + } + } } diff --git a/crates/source/tests/prime.rs b/crates/source/tests/prime.rs new file mode 100644 index 000000000..0a810ce5c --- /dev/null +++ b/crates/source/tests/prime.rs @@ -0,0 +1,332 @@ +//! Issue #1320 gates 3-6: a primed source consumer replays exactly the blocks it would have +//! played, observes no command, carries a shared `prime_required` flag, and allocates nothing. +//! +//! Every ring has two channels and a quantum of 128 frames, and its samples encode +//! `(frame, channel)`, so a block played at the wrong frame or from the wrong channel differs by +//! value. The lead is `k = 3` blocks unless a gate says otherwise. + +use bench_support::alloc::{assert_installed, current_thread_counters, current_thread_delta_since}; +use engine::{QuantumFrames, SampleRateHz}; +use source::{ + HostChunkProvider, HostPlanarChunk, PcmSourceConsumer, PcmSourceRing, PcmSourceRingConfig, + SourceCommand, SourceConsumerTelemetry, SourceFrame, SourceGeneration, SourceReadReport, + SourceSeekError, SourceSeekReport, +}; + +const RATE: SampleRateHz = SampleRateHz(48_000); +const Q: u32 = 128; +const QUANTUM: u64 = Q as u64; +const K: u32 = 3; +const RING_BLOCKS: u64 = 24; + +fn ring() -> (HostChunkProvider, PcmSourceConsumer) { + let (producer, consumer, _) = PcmSourceRing::prepare(PcmSourceRingConfig { + channel_count: 2, + quantum_frames: QuantumFrames(Q), + frame_capacity: RING_BLOCKS * QUANTUM, + initial_generation: SourceGeneration(1), + }) + .expect("ring"); + (producer.into_host_chunk_provider(RATE), consumer) +} + +#[allow(clippy::cast_precision_loss)] +fn sample(generation: u64, frame: u64, channel: u64) -> f32 { + ((generation << 20) + frame * 2 + channel) as f32 +} + +fn submit(host: &mut HostChunkProvider, generation: u64, start: u64, frames: u32, end: bool) { + let planes: [Vec; 2] = core::array::from_fn(|channel| { + (0..u64::from(frames)) + .map(|frame| sample(generation, start + frame, channel as u64)) + .collect() + }); + host.submit(HostPlanarChunk { + sample_rate_hz: RATE, + generation: SourceGeneration(generation), + start_frame: SourceFrame(start), + planes: &[&planes[0], &planes[1]], + frames, + end_of_region: end, + }) + .expect("valid block"); +} + +/// What one block made observable: both played planes as bits (or `None`), its read report and +/// the consumer's telemetry after it. +#[derive(Debug, PartialEq)] +struct Played { + planes: [Option>; 2], + report: SourceReadReport, + telemetry: SourceConsumerTelemetry, +} + +fn observe(consumer: &PcmSourceConsumer, report: SourceReadReport) -> Played { + Played { + planes: core::array::from_fn(|channel| { + consumer + .played_plane(u32::try_from(channel).expect("channel")) + .map(|plane| plane.iter().map(|value| value.to_bits()).collect()) + }), + report, + telemetry: consumer.telemetry(), + } +} + +fn ordinary(consumer: &mut PcmSourceConsumer, first_sample: u64) -> Played { + let report = consumer.begin_block_at(first_sample); + observe(consumer, report) +} + +fn primed(consumer: &mut PcmSourceConsumer, first_sample: u64) -> Played { + let report = consumer.prime_block_at(first_sample); + observe(consumer, report) +} + +fn seek_report(host: &HostChunkProvider) -> SourceSeekReport { + host.seek_report().expect("no publication in flight") +} + +/// How a pair of identically fed rings is set up before the window. +#[derive(Clone, Copy, Debug)] +enum Feed { + /// Generation 1 from frame 0, enough for the window and 8 more blocks. + Steady, + /// The region ends with a short block at window block `j = 1`. + EndsAtJOne, + /// A plain seek to generation 2 applied 2 blocks before the window. + SeekTwoBlocksEarlier, +} + +const WARM_UP_BLOCKS: u64 = 2; +const SEEK_FRAME: u64 = 10_000; + +/// Feed `host` and play the warm-up blocks; returns the first sample of the window. +fn prepare(feed: Feed, host: &mut HostChunkProvider, consumer: &mut PcmSourceConsumer) -> u64 { + match feed { + Feed::Steady => { + for block in 0..WARM_UP_BLOCKS + u64::from(K) + 8 { + submit(host, 1, block * QUANTUM, Q, false); + } + } + Feed::EndsAtJOne => { + for block in 0..=WARM_UP_BLOCKS { + submit(host, 1, block * QUANTUM, Q, false); + } + submit(host, 1, (WARM_UP_BLOCKS + 1) * QUANTUM, Q / 2, true); + } + Feed::SeekTwoBlocksEarlier => { + for block in 0..4 { + submit(host, 1, block * QUANTUM, Q, false); + } + host.try_seek(SourceCommand::Seek { + generation: SourceGeneration(2), + frame: SourceFrame(SEEK_FRAME), + }) + .expect("seek"); + for block in 0..WARM_UP_BLOCKS + u64::from(K) + 8 { + submit(host, 2, SEEK_FRAME + block * QUANTUM, Q, false); + } + } + } + let mut warm_up = Vec::new(); + for block in 0..WARM_UP_BLOCKS { + warm_up.push(ordinary(consumer, block * QUANTUM)); + } + if matches!(feed, Feed::SeekTwoBlocksEarlier) { + assert!( + warm_up[0].report.generation_changed, + "the seek began 2 blocks earlier" + ); + } + WARM_UP_BLOCKS * QUANTUM +} + +const FEEDS: [Feed; 3] = [Feed::Steady, Feed::EndsAtJOne, Feed::SeekTwoBlocksEarlier]; + +/// The reference: `k` ordinary blocks over the window, then 8 more. +fn reference(feed: Feed) -> (Vec, Vec, SourceSeekReport) { + let (mut host, mut consumer) = ring(); + let first_sample = prepare(feed, &mut host, &mut consumer); + let window = (0..u64::from(K)) + .map(|j| ordinary(&mut consumer, first_sample + j * QUANTUM)) + .collect(); + let report = seek_report(&host); + let after = (u64::from(K)..u64::from(K) + 8) + .map(|j| ordinary(&mut consumer, first_sample + j * QUANTUM)) + .collect(); + (window, after, report) +} + +/// #1320 gate 3: on its twin `k` ordinary blocks, on this ring `k` primes after a true +/// readiness. Every played plane, read report and telemetry are bit-identical, as is the seek +/// report, and so are the next 8 ordinary blocks on both. Steady, with the region ending at +/// `j = 1`, and with a generation a seek began 2 blocks earlier. +#[test] +fn the_prime_equals_the_blocks_the_consumer_would_have_played() { + for feed in FEEDS { + let (window, after, reference_report) = reference(feed); + let (mut host, mut consumer) = ring(); + let first_sample = prepare(feed, &mut host, &mut consumer); + assert!(consumer.prime_ready(first_sample, K), "{feed:?}"); + let primed_window: Vec = (0..u64::from(K)) + .map(|j| primed(&mut consumer, first_sample + j * QUANTUM)) + .collect(); + assert_eq!(primed_window, window, "{feed:?}: the primed window"); + assert_eq!( + seek_report(&host), + reference_report, + "{feed:?}: the seek report" + ); + let primed_after: Vec = (u64::from(K)..u64::from(K) + 8) + .map(|j| ordinary(&mut consumer, first_sample + j * QUANTUM)) + .collect(); + assert_eq!(primed_after, after, "{feed:?}: the blocks after the prime"); + let ended = matches!(feed, Feed::EndsAtJOne); + assert_eq!(window[1].report.end_of_region, ended, "{feed:?}"); + assert!( + window + .iter() + .all(|played| played.report.underrun_frames == 0), + "{feed:?}: no window block underruns" + ); + assert_eq!( + window + .iter() + .filter(|played| played.planes[0].is_some()) + .count(), + if ended { 2 } else { K as usize }, + "{feed:?}: the window plays PCM until the region's end" + ); + } +} + +/// #1320 gate 4: a seek pushed after a true readiness, with its generation's PCM behind it, is +/// not observed by the prime: the primed planes equal gate 3's, the command is still queued (a +/// second seek meets backpressure), and the next ordinary block applies it. +#[test] +fn the_prime_observes_no_command() { + for feed in [Feed::Steady, Feed::EndsAtJOne] { + let (window, _, reference_report) = reference(feed); + let (mut host, mut consumer) = ring(); + let first_sample = prepare(feed, &mut host, &mut consumer); + assert!(consumer.prime_ready(first_sample, K), "{feed:?}"); + host.try_seek(SourceCommand::Seek { + generation: SourceGeneration(7), + frame: SourceFrame(SEEK_FRAME), + }) + .expect("seek"); + // At the region's end the prime's acquire would reach these blocks: it must stop before + // them rather than observe their seek. + for block in 0..2 { + submit(&mut host, 7, SEEK_FRAME + block * QUANTUM, Q, false); + } + let primed_window: Vec = (0..u64::from(K)) + .map(|j| primed(&mut consumer, first_sample + j * QUANTUM)) + .collect(); + assert_eq!(primed_window, window, "{feed:?}: the primed window"); + assert_eq!(seek_report(&host), reference_report, "{feed:?}"); + assert!( + matches!( + host.try_seek(SourceCommand::Seek { + generation: SourceGeneration(8), + frame: SourceFrame(0), + }), + Err(SourceSeekError::Backpressure { .. }) + ), + "{feed:?}: the seek is still queued" + ); + let next = ordinary(&mut consumer, first_sample + u64::from(K) * QUANTUM); + assert!(next.report.generation_changed, "{feed:?}"); + assert_eq!(next.report.active_generation, SourceGeneration(7)); + assert_eq!(next.report.copied_frames, Q, "{feed:?}"); + assert_eq!( + next.planes[1].as_deref().map(|plane| plane[0]), + Some(sample(7, SEEK_FRAME, 1).to_bits()), + "{feed:?}: the sought PCM plays" + ); + } +} + +/// #1320 gate 5: the control thread's handle and the render consumer share one flag. +#[test] +fn the_prime_flag_is_shared_between_the_handle_and_the_consumer() { + let (producer, consumer, _) = PcmSourceRing::prepare(PcmSourceRingConfig { + channel_count: 2, + quantum_frames: QuantumFrames(Q), + frame_capacity: 4 * QUANTUM, + initial_generation: SourceGeneration(1), + }) + .expect("ring"); + assert!(!consumer.prime_required()); + let from_producer = producer.prime_flag(); + let host = producer.into_host_chunk_provider(RATE); + let from_host = host.prime_flag(); + from_producer.set(true); + assert!(consumer.prime_required()); + from_host.set(false); + assert!(!consumer.prime_required()); + from_host.set(true); + assert!(consumer.prime_required()); +} + +/// #1320 gate 6: readiness with `k = 16` and 16 primes allocate and free nothing on the render +/// thread, measured after a warm-up window on the same ring. +#[test] +fn readiness_and_sixteen_primes_allocate_nothing() { + assert_installed(); + const LEAD: u32 = 16; + let (mut host, mut consumer) = ring(); + for block in 0..u64::from(LEAD) + 1 { + submit(&mut host, 1, block * QUANTUM, Q, false); + } + // Warm-up: one readiness and one prime, so every lazily touched static is warmed. + assert!(consumer.prime_ready(0, LEAD)); + let _ = consumer.prime_block_at(0); + let first_sample = QUANTUM; + submit(&mut host, 1, (u64::from(LEAD) + 1) * QUANTUM, Q, false); + let mark = current_thread_counters(); + let ready = consumer.prime_ready(first_sample, LEAD); + let mut copied = 0_u32; + for j in 0..u64::from(LEAD) { + copied += consumer + .prime_block_at(first_sample + j * QUANTUM) + .copied_frames; + } + let delta = current_thread_delta_since(mark); + assert!(ready); + assert_eq!(copied, LEAD * Q); + assert_eq!((delta.allocations, delta.deallocations), (0, 0)); +} + +/// #1320 D3: a prime reads on the caller's source-read clock, as `begin_block_at` does, even when +/// that clock jumped since the last block: a clockless block after the window continues it, so a +/// seek applied there reports the sample after the window on both rings. +#[test] +fn the_prime_continues_the_callers_clock_across_a_jump() { + const JUMPED: u64 = 64 * QUANTUM; + let mut reports = Vec::new(); + for prime in [false, true] { + let (mut host, mut consumer) = ring(); + let first_sample = prepare(Feed::Steady, &mut host, &mut consumer); + assert!(JUMPED > first_sample); + assert!(consumer.prime_ready(JUMPED, K)); + for j in 0..u64::from(K) { + let sample = JUMPED + j * QUANTUM; + let _ = if prime { + consumer.prime_block_at(sample) + } else { + consumer.begin_block_at(sample) + }; + } + host.try_seek(SourceCommand::Seek { + generation: SourceGeneration(2), + frame: SourceFrame(SEEK_FRAME), + }) + .expect("seek"); + assert!(consumer.begin_block().generation_changed); + reports.push(seek_report(&host)); + } + assert_eq!(reports[0].first_sample, JUMPED + u64::from(K) * QUANTUM); + assert_eq!(reports[1], reports[0]); +} From 9cb7c6cdf9fd697e07f2775d49725f422b674990 Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 04:42:57 +0000 Subject: [PATCH 04/10] #1317: document the seek contract and the C ABI growth rule in the header Eleven seek paragraphs on the source-read clock (equal to the render clock in this version), the ABI growth comment (B15-R3), the qualification record, and abi_smoke.c tests each defined feature bit instead of the exact mask. #1355 note: amend the clock sentence and call prime_ready on the source-read sample when the offset becomes nonzero. Co-Authored-By: Claude Opus 5.5 --- ...and-the-c-abi-growth-rule-in-the-header.md | 61 +++++++ ...aw-frame-prime-at-the-first-ready-block.md | 7 + crates/capi/include/miso_engine_v1.h | 157 +++++++++++++++--- crates/capi/tests/c/abi_smoke.c | 19 ++- docs/C_ABI_V1_QUALIFICATION.md | 37 ++++- 5 files changed, 249 insertions(+), 32 deletions(-) diff --git a/.github/ISSUE_SPECS/1317-document-the-seek-contract-and-the-c-abi-growth-rule-in-the-header.md b/.github/ISSUE_SPECS/1317-document-the-seek-contract-and-the-c-abi-growth-rule-in-the-header.md index 9e37f6ac5..7973b0a41 100644 --- a/.github/ISSUE_SPECS/1317-document-the-seek-contract-and-the-c-abi-growth-rule-in-the-header.md +++ b/.github/ISSUE_SPECS/1317-document-the-seek-contract-and-the-c-abi-growth-rule-in-the-header.md @@ -190,3 +190,64 @@ the lines moved. exact-mask tests. **Realtime-path edits.** None. This slice is header text, one doc and one C test. + +## Decision record: #1317 (batch 15c) + +**Shipped.** D1-D4 and B15-R3, comment text and one C test; no declaration, symbol or Rust change. + +- Header (`crates/capi/include/miso_engine_v1.h`): "Starting an added stem in time" and the two-line + seek-report note are replaced by ten paragraphs in D1's order: the source-read clock, seek, + anchored seek, choosing the anchor, held, replacement, one seek in flight, seeks across a + structural transaction, far-past anchors, rate, and where a seek landed. A new comment block + before `MISO_ENGINE_V1_ABI_VERSION` states the growth rule (D2) with B15-R3's two sentences. The + cross reference in the live-edit paragraph now points at the new paragraphs, and the #1310 + paragraph is rewrapped to 100 columns (one line was 135 wide); "placeholder #1699" became "issue + 1699 is the successor issue for one". +- `docs/C_ABI_V1_QUALIFICATION.md` (D3): the `seek_at` paragraph no longer implies the bit protects + a directly linked host and names the source-read clock; a paragraph records the seek contract and + one "ABI growth" paragraph states D2 once. +- `crates/capi/tests/c/abi_smoke.c` (D4): `reports_every_defined_feature` requires one `&` per + `MISO_ENGINE_V1_FEATURE_*` symbol (nine); the `!= MISO_ENGINE_V1_FEATURE_MASK` comparison is gone. + The compile-time pins are kept. + +**Statements checked against the code** (file, function): clock origin 0 and `render.time.discontinuity` +(`plan.rs` `render_contiguous`, `plan_exchange.rs` `adopt_absolute_sample`, `ffi.rs` render test); +quantum-multiple anchor and the check order (`source/src/lib.rs` `try_seek`); producer switches to +`g` at the call and refuses other generations (`validate_submission_metadata`); held, replacement, +late apply and region end (`observe_seek_at_block_boundary`, `begin_block_play`, `settle_block`, +tests `a_late_anchored_seek_past_the_region_end_is_the_end_of_region`); one command slot +(`command_capacity = 1`, `source.seek.backpressure` in `host-core/src/source.rs`); consumer carry +including `held_seek` (`adopt_sources`); report words and 64-attempt bound (`publish_seek_report`, +`seqlock.rs`); source rate equals the session rate (`prepare.rs` pushes `compiled.sample_rate()`). + +**Departures.** + +- The text says a seek takes effect "no later than the first block that begins after the call + returns", not "at the next block render begins": a producer that pushes a seek and its PCM + between a block's command pop and its data pop is observed by the acquire's re-observe in that + same block (#1274 MINOR-3). +- Far-past text says the seek applies in the block that pops it (not "the next block after it is + popped"), as `observe_seek_at_block_boundary` pops and applies in one block. +- The end-of-region sentence is conditional on a region the host closed with `end_of_region`; with + no end known the block is an underrun. +- Feature mask is 511 (bit 8 landed with #1316), not the amendment's 255. +- #1320's prime has no host-visible API, so the header says nothing about it. #1396 has not landed, + so "equal to the render clock in this version" stands and #1396 is not named in the header. + +**Gates.** + +1. `bash scripts/check-capi-abi.sh`: ok (C11 and C++17 at `-Wall -Wextra -Werror -pedantic`, symbol + set, smoke run, shared and static); `--self-test`: ok. +2. Smoke is a bit test, by a local patch of `FEATURE_MASK` in `crates/capi/src/abi.rs`, reverted + after each run: clearing `FEATURE_SERVICE` makes `check-capi-abi.sh` exit 1; OR-ing in unknown bit + `1 << 9` passes. +3. `cargo test --locked -p capi`: 158 lib tests and every integration test pass (the Rust exact-mask + tests are unchanged). +4. Review gate: every D1 statement is checked above against its code anchor. +5. `bash scripts/check-workspace-policy.sh`, `cargo fmt --all -- --check`, + `RUSTDOCFLAGS='-D warnings' cargo doc --locked --workspace --no-deps`: pass. + +**Successor work.** #1355 must amend the "equal to the render clock in this version" sentence (the +header's first seek paragraph, the qualification record and `SourceControlSet::seek_at`'s doc) when +a warm successor makes the source-read offset nonzero; the amendment asks for this to be recorded in +#1355, whose spec does not yet carry it. #1699 owns the counter the header says does not exist yet. diff --git a/.github/ISSUE_SPECS/1355-adopt-a-warm-successor-with-a-raw-frame-prime-at-the-first-ready-block.md b/.github/ISSUE_SPECS/1355-adopt-a-warm-successor-with-a-raw-frame-prime-at-the-first-ready-block.md index 53ca67ba3..5f9b87dd7 100644 --- a/.github/ISSUE_SPECS/1355-adopt-a-warm-successor-with-a-raw-frame-prime-at-the-first-ready-block.md +++ b/.github/ISSUE_SPECS/1355-adopt-a-warm-successor-with-a-raw-frame-prime-at-the-first-ready-block.md @@ -233,3 +233,10 @@ widths. - *Carry meter and effect observation state across a plan swap* (#1327). - *Carry spectrum capture state across a plan swap* (#1395). - *Reset latency floors at a host-declared discontinuity* (#1323), D2: the rule D6 cites. + +**Note (controls coordinator, batch 15c, 2026-10-16).** #1317 documents the seek contract in the +C header with "equal to the render clock in this version" for the source-read clock. When a warm +successor makes the source-read offset nonzero, this slice amends that sentence in the header's +first seek paragraph, in `docs/C_ABI_V1_QUALIFICATION.md` and in the `SourceControlSet::seek_at` +doc. It also calls #1320's `prime_ready` with the source-read sample (S + O), not the render +sample, and wires the prime on both hosts. diff --git a/crates/capi/include/miso_engine_v1.h b/crates/capi/include/miso_engine_v1.h index 70073b696..760bcde63 100644 --- a/crates/capi/include/miso_engine_v1.h +++ b/crates/capi/include/miso_engine_v1.h @@ -78,10 +78,10 @@ extern "C" { * and a supersession are not in the response; the watermark's outcome flags report them. After * a live edit or a replacement the host goes on submitting every source the transaction left * unchanged, with no seek; after a replacement it acts only on a source the transaction added, - * changed or removed (see "Sources across a structural transaction" and "Starting an added stem - * in time" below). (miso_engine_v1_plan_resources describes whichever plan is active, so a - * replacement may change it, but an unchanged report does not mean that no replacement - * happened; the edit path is the signal.) + * changed or removed (see "Sources across a structural transaction" and the seek paragraphs from + * "The source-read clock" to "Where a seek landed" below). (miso_engine_v1_plan_resources + * describes whichever plan is active, so a replacement may change it, but an unchanged report + * does not mean that no replacement happened; the edit path is the signal.) * * Latency across a replacement (issue 1285). No node's latency, the output's included, drops * across a structural transaction: the successor plan is compiled with every surviving node's @@ -226,20 +226,21 @@ extern "C" { * changed against the newest committed session, so the host restarted its feed, and a source the * earlier transaction added, redeclared or removed and the new one does not keep as the waiting * replacement declares it starts fresh in the new plan (generation 1, frame 0, as an added one), - * also when the new one puts it back in its form in the plan that renders. No accepted chunk on a ring the waiting replacement created - * is dropped across a supersession: when the waiting replacement created a source's ring (the - * earlier transaction added the source, or redeclared it) and the session has accepted PCM, a seek - * or an end of region for that source since, a new transaction that does not keep the source as - * the waiting replacement declares it (it declares it as the plan that renders does, redeclares it - * another way, or removes it) is refused with MISO_ENGINE_V1_BACKPRESSURE before it commits: + * also when the new one puts it back in its form in the plan that renders. No accepted chunk on a + * ring the waiting replacement created is dropped across a supersession: when the waiting + * replacement created a source's ring (the earlier transaction added the source, or redeclared + * it) and the session has accepted PCM, a seek or an end of region for that source since, a new + * transaction that does not keep the source as the waiting replacement declares it (it declares + * it as the plan that renders does, redeclares it another way, or removes it) is refused with + * MISO_ENGINE_V1_BACKPRESSURE before it commits: * nothing changes, the waiting replacement stays published, no event is queued and the revision * does not move. Once render adopts the waiting replacement, a retry commits. A host that does not * render keeps getting MISO_ENGINE_V1_BACKPRESSURE for such a transaction until it renders. A * source the waiting replacement carries unchanged from the plan that renders is not refused this * way. A source the caller's own transaction removes discards its queued audio, whether the plan * that renders or the waiting replacement (carrying it unchanged) holds it: the caller asked for - * the removal, so this is not an unintended drop. No counter reports the discarded frames yet; a - * counter for the discarded frames is a successor issue (placeholder #1699). A resource cap counts + * the removal, so this is not an unintended drop. No counter reports the discarded frames yet; + * issue 1699 is the successor issue for one. A resource cap counts * every plan the session still holds and the new one: the running plan, the waiting replacement, * and a plan render has displaced until a later call reclaims it. A refusal is * MISO_ENGINE_V1_BACKPRESSURE when the new plan fit against the newest held plan alone, so a retry @@ -250,23 +251,125 @@ extern "C" { * watermark advances to its revision with MISO_ENGINE_V1_OUTCOME_SUPERSEDED set, and * superseded_count grows by the revisions the earlier replacements carried. * - * Starting an added stem in time (issue 1275). miso_engine_v1_source_seek_at is - * miso_engine_v1_source_seek with an anchor: source_frame enters the graph in the block that starts - * at absolute render sample anchor_sample, the clock miso_engine_v1_render_f32_planar takes and - * every plan swap continues; the output hears it the plan's latency later. The anchor must be a - * multiple of the plan's quantum, or the call returns MISO_ENGINE_V1_INVALID_ARGUMENT with - * "source.seek.anchor_unaligned". A host checks MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_AT before - * calling it. To start a stem a transaction added, after the command returns OK: pick A, a - * quantum multiple a few quanta past the last rendered block; call + * The source-read clock (issues 1275, 1316). A seek that names a block names it on the plan's + * source-read clock: the absolute sample of source time the block reads. In this version it is + * equal to the render clock. A plan's clock is 0 at compile. Each miso_engine_v1_render_f32_planar + * call must pass the clock's value as absolute_sample, renders one quantum and advances the clock + * by one quantum; a call at another sample renders nothing, moves nothing and returns + * MISO_ENGINE_V1_RENDER_REJECTED with "render.time.discontinuity". A plan that a structural + * transaction swaps in continues the clock of the plan it replaces, so an anchor stays valid + * across a swap. + * + * Seek. miso_engine_v1_source_seek(id, g, F) makes generation g play from source frame F. It + * takes effect no later than the first block that begins after the call returns; the PCM of the + * earlier generations that is still queued is discarded, not played. The call switches the source + * to g at once: from then on the host submits generation g in order from F, and a chunk of any + * other generation is refused as "source.generation.stale". Submit it before the block renders; + * a block of g with no PCM queued renders silence and counts as underrun frames. The refusals, + * all MISO_ENGINE_V1_INVALID_ARGUMENT: g not above the source's generation + * ("source.generation.stale"), g = 0 ("source.generation.zero"), F outside the source's region + * ("source.region.outside"), an unknown source ("source.id.unknown"). Where the seek landed is + * the seek report (below). Like every session call, a seek runs the service step first (see + * "Control service"). + * + * Anchored seek. miso_engine_v1_source_seek_at(id, g, F, A) is the seek with an anchor: F enters + * the graph in the block that starts at sample A of the source-read clock, and the output hears + * it the plan's latency later. A must be a multiple of the plan's quantum, or the call returns + * MISO_ENGINE_V1_INVALID_ARGUMENT with "source.seek.anchor_unaligned". Its other checks and + * results are the seek's, and it too switches the source to g at once, so the host submits g from + * F before the block at A renders. A host checks MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_AT before + * calling it. + * + * Choosing the anchor. To start a stem that a transaction added, after the command returns OK: + * pick A, a quantum multiple past the last rendered block by enough quanta to cover the time the + * host needs to decode generation g from F and submit it (a few quanta); call * miso_engine_v1_source_seek_at(id, g, F, A), where g is above the source's generation (2 for a - * just-added source) and F is the source frame the playing stems read at A; then submit generation - * g from F before the block at A renders. Until A the stem renders silence; from A it plays F on. - * Should the render reach the seek only after A, the stem starts at F plus the lateness, so it - * stays in time. + * just-added source) and F is the source frame the playing stems read at A; then submit g from F. + * Until A the stem renders silence; from A it plays F on. If g's PCM is not queued when the block + * at A renders, the seek applies there anyway: the blocks without PCM count as underrun frames, + * and PCM that falls behind the read position is discarded. A playing stem's seek report gives + * that frame: a stem whose report reads (generation, s, f) reads source frame f + (A - s) at A, + * until its next seek and inside its region. + * + * Held. From the block that pops an anchored seek until the block that applies it, the seek is + * held. Queued PCM of the playing generation plays meanwhile (the host cannot add to it, because + * the call switched the producer to g); when that runs out the source renders silence, and the + * frames count in the report's cumulative_held_frames, not in cumulative_underrun_frames. A + * planned wait is not starvation. A block without PCM while no seek is held is an underrun. + * + * Replacement. A newer seek, plain or anchored, replaces a held one: the held seek never applies, + * and the PCM already submitted for its generation is discarded. A plain seek applies as above; + * an anchored one holds in its place with its own anchor. + * + * One seek in flight. A source's command queue holds one seek. A seek call that arrives before + * render has popped the previous one, which happens when render begins a block, returns + * MISO_ENGINE_V1_BACKPRESSURE with "source.seek.backpressure" and changes nothing; the host + * retries after a block has begun, and a host that renders nothing keeps getting it. A held seek + * has been popped, so it does not block the next one. + * + * Seeks across a structural transaction (issues 1273, 1310). A source that a replacement plan + * keeps unchanged keeps its seek state with its ring: a seek queued and not yet popped, and a + * held seek, move to the new plan, and a held anchor stays on the continuing clock, so it applies + * in the anchor block of the new plan or, if that block has passed, as a late seek (below). The + * host does not repeat the seek. The same holds when a transaction supersedes an unadopted + * replacement. A source that the transaction added or changed restarts in a new ring at + * generation 1, frame 0, with its seek report at generation 0: a seek held or queued on the old + * ring is discarded with it, and the host seeks the new one after the command returns OK. A seek + * on a source that the transaction removed is refused as "source.id.unknown". + * + * Far past anchors. An anchor that render has already passed is not refused. The seek applies no + * later than the first block that begins after the call returns, at source frame F plus the + * lateness, the number of samples from A to that block's first sample, so the stem stays in time; + * a replacement plan adopted only after A does the same in its first block. If F plus the + * lateness lies past the end of a region that the host closed with end_of_region, the block + * renders silence as end of region: neither an underrun nor held. + * + * Rate. The lateness is added to F in source frames, one for one with samples of the source-read + * clock. That is exact because every chunk must be at the session's sample rate (refused as + * "source.rate.mismatch" otherwise); the engine converts no rates, so a host that decodes at + * another rate resamples before it submits. + * + * Where a seek landed (issue 1316). miso_engine_v1_source_seek_report(session, id, out) copies + * what render last published for the source of the newest committed session into out, a + * miso_engine_v1_seek_report (MISO_ENGINE_V1_SOURCE_SEEK_REPORT_SIZE bytes; the caller sets + * struct_size and zeroes reserved0 and reserved[]). generation is the generation in effect, 0 + * until the source's ring has rendered its first block; first_sample is the source-read sample of + * the first block in which that generation was in effect, and source_frame the source frame that + * block started at, for an anchored seek that render reached late F plus the lateness; so the + * source plays frame source_frame + (t - first_sample) in the block at t, until its next seek and + * inside its region. + * cumulative_underrun_frames and cumulative_held_frames count the source frames that blocks + * without PCM covered since the ring was made, split as under "Held"; a seek does not reset them. + * A plain seek lands in the block that applies it, and the report is how a host learns which: it + * keeps the previous generation until that block has rendered. The report is a level that render + * overwrites, not a queue. The ring owns it, so a swap that carries the source keeps it, and a + * source that a transaction added or changed starts at generation 0. The call returns + * MISO_ENGINE_V1_OK; MISO_ENGINE_V1_INVALID_ARGUMENT for a wrong struct_size, a nonzero reserved + * word, a malformed source ID, or an undeclared source ("source.id.unknown"); and + * MISO_ENGINE_V1_BACKPRESSURE with "source.report.busy", out untouched, when every bounded read + * (64) landed inside render's publication, so the host reads again. A host checks + * MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT before calling it. + */ + +/* + * ABI growth. MISO_ENGINE_V1_ABI_VERSION changes only on a break. Everything else grows in place: + * + * - Each added exported symbol, and each former reserved struct word that gets a name, gets its + * own MISO_ENGINE_V1_FEATURE_* bit in miso_engine_v1_capabilities.feature_mask: the next free + * one when it merges, never a reused one. + * - A new value of an existing enum or result code, and a field added in place to a V1 protocol + * message, takes no bit. So a host tolerates an enum value it does not know, and treats a + * result code it does not know as a failure. + * - A feature added as a protocol message is announced by its protocol capability flag in the + * CAPABILITIES reply, not by a feature bit. The control protocol registry, not this rule, + * governs protocol flags. * - * Where a seek landed (issue 1316). miso_engine_v1_source_seek_report copies the source's - * (generation, first_sample, source_frame) and its cumulative underrun and held frames; a host - * checks MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT before calling it. + * A host tests each bit it needs with &, tolerates bits it does not know (a newer library reports + * more bits than the header the host was built with), and never compares feature_mask with ==. + * MISO_ENGINE_V1_FEATURE_MASK is every bit this header defines; it is not a value to compare the + * report with. A feature bit protects only a host that resolves the symbol at run time: a host + * that links a symbol directly fails to load against a library that lacks it, before it can test + * any bit. */ #define MISO_ENGINE_V1_ABI_VERSION UINT32_C(0x00010000) diff --git a/crates/capi/tests/c/abi_smoke.c b/crates/capi/tests/c/abi_smoke.c index 92530678d..4bf745516 100644 --- a/crates/capi/tests/c/abi_smoke.c +++ b/crates/capi/tests/c/abi_smoke.c @@ -245,6 +245,23 @@ static int check_plan_watermark(miso_engine_v1_engine *engine) { return failed; } +/* A host tests each feature bit it needs with `&` and tolerates bits it does not know; it never + * compares the mask with `==` (header, "ABI growth"). This requires every bit the header defines, + * one test per symbol and never a bit number, so a library that stops reporting one fails and a + * library that reports an extra unknown bit passes. The exact mask is asserted by the Rust tests + * (`version_and_capabilities_are_exact`). */ +static int reports_every_defined_feature(uint64_t mask) { + return (mask & MISO_ENGINE_V1_FEATURE_IMMUTABLE_SESSION) != 0 && + (mask & MISO_ENGINE_V1_FEATURE_HOST_PLANAR_SOURCE) != 0 && + (mask & MISO_ENGINE_V1_FEATURE_SOURCE_SEEK) != 0 && + (mask & MISO_ENGINE_V1_FEATURE_PLANAR_STEREO_RENDER) != 0 && + (mask & MISO_ENGINE_V1_FEATURE_CAPABILITY_COMMAND) != 0 && + (mask & MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_AT) != 0 && + (mask & MISO_ENGINE_V1_FEATURE_PLAN_WATERMARK) != 0 && + (mask & MISO_ENGINE_V1_FEATURE_SERVICE) != 0 && + (mask & MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_REPORT) != 0; +} + int main(void) { miso_engine_v1_capabilities capabilities = {0}; miso_engine_v1_engine_config config = {0}; @@ -255,7 +272,7 @@ int main(void) { query_capabilities_signature(&capabilities) != MISO_ENGINE_V1_OK || capabilities.abi_version != MISO_ENGINE_V1_ABI_VERSION || capabilities.exact_launch_rate_mask != MISO_ENGINE_V1_EXACT_LAUNCH_RATE_MASK || - capabilities.feature_mask != MISO_ENGINE_V1_FEATURE_MASK) { + !reports_every_defined_feature(capabilities.feature_mask)) { return 1; } diff --git a/docs/C_ABI_V1_QUALIFICATION.md b/docs/C_ABI_V1_QUALIFICATION.md index ef78532e1..58325e596 100644 --- a/docs/C_ABI_V1_QUALIFICATION.md +++ b/docs/C_ABI_V1_QUALIFICATION.md @@ -107,10 +107,13 @@ A host starts a stem a structural transaction added in exact time through exported symbol, announced by `MISO_ENGINE_V1_FEATURE_SOURCE_SEEK_AT` (32) in the capability report, whose feature mask becomes 63; no struct, size or `ABI_VERSION` changes. It is `miso_engine_v1_source_seek` with an anchor: the seek's source frame enters the graph in the -block that starts at absolute render sample `anchor_sample`, which must be a multiple of the -quantum or the call returns `RESULT_INVALID_ARGUMENT` with `source.seek.anchor_unaligned`; its -thread rule, argument checks and every other result are the plain seek's. The host checks the bit -before calling the symbol, since a library older than #1275 does not export it. After the +block that starts at sample `anchor_sample` of the plan's source-read clock (equal to the render +clock in this version, #1316 D1), which must be a multiple of the quantum or the call returns +`RESULT_INVALID_ARGUMENT` with `source.seek.anchor_unaligned`; its +thread rule, argument checks and every other result are the plain seek's. A host that resolves +the symbol at run time checks the bit before calling it; a host that links the symbol directly +fails to load against a library older than #1275 before it can check anything (see "ABI growth" +below). After the transaction returns, the host picks `A` a few quanta past the last rendered block, calls `seek_at(id, g, F, A)` with `F` the frame the playing stems read at `A`, then submits generation `g` from `F`; the stem renders silence until `A` and plays `F` from `A` on, bit-identical to the @@ -216,6 +219,32 @@ and a wrong size, `header_smoke.cpp` pins the size, every field offset and the s capi runtime tests cover a plain seek, a late anchored seek, a scrub-then-align across a transaction, a carried and an added ring, and the two counters. +The seek contract in the header (#1317, decision 15 D15-12) states, from the code, the source-read +clock and its origin (0 at compile, one quantum per render call, continued by a swapped-in plan), +the plain and the anchored seek, how to choose the anchor, held behaviour (queued PCM of the +playing generation plays, then silence counted as held frames, never as underrun frames), +replacement of a held seek, the one-slot refusal (`BACKPRESSURE`, `source.seek.backpressure`), +displacement by a structural transaction, far-past anchors and the region end, the source-rate +assumption (`source.rate.mismatch`), and how `miso_engine_v1_source_seek_report` reports a seek's +landing. It promises nothing the code does not do; a source-read clock that leads the render clock +is a later change and amends the "equal to the render clock in this version" sentence. + +ABI growth (#1317, decision 15 D15-3). `MISO_ENGINE_V1_ABI_VERSION` changes only on a break. Each +added exported symbol, and each former reserved struct word given a name, gets its own +`MISO_ENGINE_V1_FEATURE_*` bit, the next free one when it merges. A new value of an existing enum +or result code, and a field added in place to a V1 protocol message, takes no bit, so a host +tolerates an enum value it does not know and treats a result code it does not know as a failure. A +feature added as a protocol message is announced by its protocol capability flag in the +`CAPABILITIES` reply, not by a feature bit; the control protocol registry, not this rule, governs +protocol flags. A host tests each bit it needs with `&`, tolerates bits it does not know and never +compares `feature_mask` with `==`. A feature bit protects only a host that resolves the symbol at +run time: a host that links a symbol directly fails to load against a library that lacks it, before +it can test anything. The exact mask is asserted by the library's own Rust tests +(`version_and_capabilities_are_exact` and the constant pins in `abi.rs`), not by the C smoke test: +`abi_smoke.c` tests one `&` per `MISO_ENGINE_V1_FEATURE_*` symbol (it fails when the library stops +reporting a defined bit and passes when it reports an extra unknown one) and keeps its compile-time +pins of the header's constants. + The first C11-static launch found one qualification-fixture error: it attempted generation-1 seek before the initial generation-1 submission and exited 13. No product byte or staged library was changed or rebuilt. The new consumer was corrected to submit generation 1 first, then seek and From d2ad596271e3549c633986545543ad682cd44130 Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 05:34:47 +0000 Subject: [PATCH 05/10] #1396: give a plan a source-read clock that leads its render clock A plan carries a source-read offset: the source section reads at render sample + offset (checked; overflow is TimeOverflow). The inventory records it; an ordinary successor inherits it and a discontinuity successor resets it to 0 (runs_behind_natural_timing gains the offset term). Render publishes the active plan's render and source-read clocks after each block (two Release stores into a preallocated record, 32 B native / 24 B wasm32); PlanPublisher reads them. In production the offset stays 0 until #1355; the header's "equal to the render clock in this version" stays true. realtime policy 151/33; audit capi unchanged. Co-Authored-By: Claude Opus 5.5 --- ...-read-clock-that-leads-its-render-clock.md | 56 ++++ crates/control-plane/src/compile.rs | 9 + crates/control-plane/src/control.rs | 221 ++++++++++++- crates/engine/src/realtime/mod.rs | 101 ++++++ crates/engine/src/realtime/plan.rs | 31 ++ crates/engine/src/realtime/plan_exchange.rs | 64 +++- crates/graph/src/lib.rs | 20 +- crates/host-core/src/lib.rs | 2 +- crates/host-core/src/prepare.rs | 42 ++- crates/host-core/tests/source_read_clock.rs | 295 ++++++++++++++++++ crates/source/src/lib.rs | 10 +- 11 files changed, 836 insertions(+), 15 deletions(-) create mode 100644 crates/host-core/tests/source_read_clock.rs diff --git a/.github/ISSUE_SPECS/1396-give-a-plan-a-source-read-clock-that-leads-its-render-clock.md b/.github/ISSUE_SPECS/1396-give-a-plan-a-source-read-clock-that-leads-its-render-clock.md index bee72f876..162855cea 100644 --- a/.github/ISSUE_SPECS/1396-give-a-plan-a-source-read-clock-that-leads-its-render-clock.md +++ b/.github/ISSUE_SPECS/1396-give-a-plan-a-source-read-clock-that-leads-its-render-clock.md @@ -239,3 +239,59 @@ one attempt. - **Batch placement.** "Neither is in the batch-15/16 list" (Dependencies on GitHub, about #1310 and #1344) is void. #1310 and #1344 are batch 15a; #1323 lands after #1310 (B15-R5). + +## Decision record: #1396 (batch 15c) + +**Shipped.** + +- D1: `PreparedRenderPlan::source_read_offset()` and `set_source_read_offset()` (control thread, + unpublished plan), forwarding to the new defaulted `PreparedPlanExecutor::set_source_read_offset`. + `GraphExecutor` (and its layout mirror `GraphExecutorWithoutSplitPairTable`) stores it, and the + source section's block entry reads from `time.absolute_sample.checked_add(offset)`; overflow + silences the block and returns `RenderError::TimeOverflow`, as the section's other refusals do. + `RenderTime` is unchanged. +- D2: `PlanStateInventory` records the offset (`source_read_offset()`); preparation sets every + plan's offset from `base.inventory` (0 for a first plan; `at_discontinuity` gives 0) and writes + the same value to the plan. `host_core::test_only_set_source_read_offset` (test-support) sets + plan and row together. +- D2's declaration term: `PlanStateInventory::runs_behind_natural_timing()` is now "a raised floor + or an offset above 0", the one test `SessionState::declare_discontinuity` already calls (#1323's + note and the method's own doc named this place). The control-plane hook is + `SessionState::test_set_next_source_read_offset` (a thread-local taken once by + `prepare_runtime`), with `test_render_clock` / `test_source_read_clock` readers, all test-support. +- D4: `plan_exchange` allocates one `Arc` clock record (two `AtomicU64`), a new row of + `plan_exchange_resource_report`. After each rendered block render stores the active plan's + source-read clock, then its render clock (`Release`); `PlanPublisher::render_clock()` and + `source_read_clock()` load one word each (`Acquire`). Before the first block both hold the + initial plan's values. No host surface (non-goal; #1360, #1361). + +**Departures.** + +- Gate 2's "fader-only prepared change" is a fader-gain change on a muted second track, so the + rebuilt run is bit-comparable with the run without the rebuild. +- The declaration term lives in host-core's `runs_behind_natural_timing` (called from the control + plane), not in a second test inside `crates/control-plane/src/`; gate 5 is in the control plane. +- Mechanical edits outside the paths: the re-export in `crates/host-core/src/lib.rs`, and two + comments in `crates/source/src/lib.rs` that said "#1396 passes the source-read sample here". +- The offset is nonzero only through the test hook until #1355, so the header's "equal to the + render clock in this version", `docs/C_ABI_V1_QUALIFICATION.md` and the `seek_at` doc stay true + and are unchanged. + +**Gates.** + +- Gate 1: `crates/host-core/tests/source_read_clock.rs::a_seek_applies_on_the_source_read_clock`. +- Gate 2: `an_ordinary_successor_inherits_the_offset` (same file). +- Gate 3: `a_declaration_resets_the_offset_and_loses_no_frame` (same file; ramp PCM). +- Gate 4: engine `realtime::tests:: + the_publisher_reads_both_clocks_of_the_active_plan_after_each_block`. +- Gate 5: control-plane + `control::tests::source_read_offset::a_declaration_resets_an_offset_with_no_raised_floor`. +- Red evidence: the graph passing the render sample turns gate 1 red; an ordinary successor at + offset 0 turns gate 2 red (also by its output alone); a discontinuity successor that keeps the + offset turns gate 3 red; render storing the outgoing plan's lead turns gate 4 red (checked in a + scratch copy); dropping the offset term turns gate 5 red. + +**Measured.** `realtime-policy` ok with no floor change (151 regions in 33 files). Audit capi +`pcm_digest` 4f52089e4a8cf83c, unchanged, zero violations. The capi `resource_lifecycle` ceilings +and the browser `expected.json` rows hold without a re-pin (`check-browser-expected-resources.py +--artifacts` green; the clock record is 32 bytes native, 24 on wasm32). diff --git a/crates/control-plane/src/compile.rs b/crates/control-plane/src/compile.rs index cbecca672..d316a4481 100644 --- a/crates/control-plane/src/compile.rs +++ b/crates/control-plane/src/compile.rs @@ -1162,6 +1162,15 @@ pub(crate) fn prepare_runtime( let (prepared, handles) = preparer .prepare(compiled, &caps, lanes, successor, superseded) .map_err(prepare_failure)?; + // #1396 deliverable 3: a test gives the next plan a source-read offset before publication. + #[cfg(feature = "test-support")] + let prepared = { + let mut prepared = prepared; + if let Some(offset) = crate::control::take_test_next_source_read_offset() { + host_core::test_only_set_source_read_offset(&mut prepared, offset); + } + prepared + }; // #1256 D2: keep only the producers. The strip ID list and the vectors this selection leaves // empty drop here, on the control thread; each producer keeps its own `track_id`. The // producer vector is built at exactly one entry per strip, so the boxed slice does not diff --git a/crates/control-plane/src/control.rs b/crates/control-plane/src/control.rs index 0ae32299f..8078b631a 100644 --- a/crates/control-plane/src/control.rs +++ b/crates/control-plane/src/control.rs @@ -408,6 +408,9 @@ thread_local! { const { core::cell::Cell::new(([None; 2], 0)) }; static TEST_NEXT_ADOPTION: core::cell::Cell> = const { core::cell::Cell::new(None) }; + /// The source-read offset the next plan this thread prepares is given (#1396 deliverable 3). + static TEST_NEXT_SOURCE_READ_OFFSET: core::cell::Cell> = + const { core::cell::Cell::new(None) }; static FORCE_FULL_PREPARATION: core::cell::Cell = const { core::cell::Cell::new(false) }; /// Whether the service step leaves displaced plans in the retirement queue (batch 15a review /// fold M1): a render preempted between claiming a candidate and retiring the plan it displaced. @@ -470,9 +473,17 @@ pub fn test_reset_lifecycle_observer() { TEST_FAULT_STATE.with(|state| state.set(([None; 2], 0))); TEST_OWNER_STATE.with(|state| state.set(TestOwnerCounters::default())); TEST_NEXT_ADOPTION.with(|state| state.set(None)); + TEST_NEXT_SOURCE_READ_OFFSET.with(|state| state.set(None)); TEST_HOLD_RECLAMATION.with(|state| state.set(false)); } +/// Takes the source-read offset [`SessionState::test_set_next_source_read_offset`] scheduled for +/// the next plan this thread prepares (#1396 deliverable 3), once. +#[cfg(feature = "test-support")] +pub(crate) fn take_test_next_source_read_offset() -> Option { + TEST_NEXT_SOURCE_READ_OFFSET.with(core::cell::Cell::take) +} + /// Makes this thread's control commands prepare their transactions by the full path (#1613): a /// clone of the model with the edits applied, compiled, as before the in-place preparation. A test /// runs one script both ways and compares the two sessions. @@ -1646,6 +1657,29 @@ where .find(|provider| provider.epoch == epoch) } + /// Gives the next plan this thread prepares the source-read offset `offset` (#1396 + /// deliverable 3): [`host_core::test_only_set_source_read_offset`] on that plan and its + /// inventory row, before it is published. Taken once, by the next preparation on this thread. + #[cfg(feature = "test-support")] + pub fn test_set_next_source_read_offset(&mut self, offset: u64) { + let _ = self; + TEST_NEXT_SOURCE_READ_OFFSET.with(|state| state.set(Some(offset))); + } + + /// The running plan's render clock, as render stored it after its last block + /// ([`PlanPublisher::render_clock`], #1396 D4). + #[cfg(feature = "test-support")] + pub fn test_render_clock(&self) -> u64 { + self.publisher.render_clock() + } + + /// The running plan's source-read clock, as render stored it after its last block + /// ([`PlanPublisher::source_read_clock`], #1396 D4). + #[cfg(feature = "test-support")] + pub fn test_source_read_clock(&self) -> u64 { + self.publisher.source_read_clock() + } + /// Schedules the next structural publication's adoption (#1348 gate 4): `command` reserves /// that one candidate with `adoption` instead of [`PlanAdoption::Next`]. Taken once, by the /// next structural publication on this thread. @@ -2090,7 +2124,9 @@ where /// /// After the service step, it rebuilds only if the newest plan (the pending candidate, else /// the running plan) runs behind its natural timing - /// ([`PlanStateInventory::runs_behind_natural_timing`]). It then prepares the committed model + /// ([`PlanStateInventory::runs_behind_natural_timing`]): a floor raised above a natural + /// arrival, or a source-read offset above 0 (#1396 D2), which a plan can hold with no raised + /// floor. The successor's offset is 0. It then prepares the committed model /// as a successor at a discontinuity (`SuccessorBase::discontinuity`): no floors, only the /// source consumers carried, every other owner at rest. It is published /// [`PlanAdoption::Next`] with the committed revision and its retirement credit reserved, as a @@ -4091,4 +4127,187 @@ mod tests { ); assert_eq!(EventError::Internal.diagnostic(), b"control.event.internal"); } + + /// #1396 gate 5: a declaration resets a source-read offset that no raised floor goes with. + #[cfg(feature = "test-support")] + mod source_read_offset { + use crate::{ + CompiledChildren, ControlAdapter, ControlLimits, PlanResources, RuntimePreparer, + compile_children, + }; + use engine::realtime::PlanarBufferMut; + use host_core::{ + CompiledSession, HostLiveControlHandles, HostLiveControlRequest, HostLiveLanes, + HostPrepareCaps, LiveValuesSuperseded, PrepareDiagnostics, PreparedHost, SuccessorBase, + prepare_host_runtime_with_live_lanes, prepare_host_runtime_with_live_lanes_successor, + }; + use protocol::{ + CommandPayload, ExpectedRevision, ProtocolCodec, RequestId, SessionEdit, + TypedCommandFrame, + }; + use session::{RouteSource, StableId, parse_session_json}; + + /// 48 kHz, quantum 128. + const SESSION: &str = + include_str!("../../../fixtures/session/v1/parametric-eq-nine-track.json"); + const QUANTUM: u64 = 128; + const OFFSET: u64 = 3 * QUANTUM; + + struct Adapter; + + impl ControlAdapter for Adapter { + type Row = PlanResources; + const ADAPTER_ALLOCATIONS: &'static [u64] = &[]; + } + + /// The C ABI's own preparation: host-core's live-lanes entries. + struct Preparer; + + impl RuntimePreparer for Preparer { + fn prepare( + &self, + compiled: &CompiledSession, + caps: &HostPrepareCaps, + lanes: HostLiveLanes, + successor: Option>, + superseded: &LiveValuesSuperseded, + ) -> Result<(PreparedHost, HostLiveControlHandles), PrepareDiagnostics> { + let live = HostLiveControlRequest::default(); + match successor { + Some(base) => prepare_host_runtime_with_live_lanes_successor( + compiled, caps, &live, lanes, base, superseded, + ), + None => prepare_host_runtime_with_live_lanes( + compiled, caps, &live, lanes, superseded, + ), + } + } + } + + fn limits() -> ControlLimits { + ControlLimits { + source_ring_frames: 1_024, + maximum_automation_spans_per_block: 128, + maximum_document_bytes: 1_000_000, + maximum_diagnostic_bytes: 4_096, + maximum_tracks: 100, + maximum_sources: 100, + maximum_routes: 100, + maximum_effects: 100, + maximum_graph_session_plus_plan_bytes: 100_000_000, + maximum_source_total_bytes: 10_000_000, + maximum_source_overhead_bytes: 10_000_000, + maximum_effect_state_bytes: 100_000_000, + maximum_effect_scratch_bytes: 100_000_000, + maximum_builtin_retained_bytes: 100_000_000, + maximum_control_retained_bytes: 10_000_000, + maximum_named_allocation_bytes: 100_000_000, + maximum_meter_streams: 1, + maximum_meter_items: 1, + maximum_meter_bytes: 1, + maximum_control_frame_bytes: 4_096, + maximum_replay_bytes: 8_192, + maximum_replay_entries: 16, + maximum_submixes: 0, + maximum_vcas: 0, + maximum_vca_reach_pairs: u64::MAX, + maximum_replacement_peak_bytes: u64::MAX, + } + } + + /// A structural transaction that raises no floor: a muted copy of the first track and + /// its route, so every natural arrival stays where it was. + fn add_muted_track(children: &mut CompiledChildren) { + let model = parse_session_json(SESSION).expect("fixture"); + let mut track = model.tracks[0].clone(); + track.id = StableId::parse("a-added").expect("track ID"); + track.fader.left_mute = true; + track.fader.right_mute = true; + let mut route = model.routes[0].clone(); + route.id = StableId::parse("a-added-main").expect("route ID"); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("track route") + }; + *track_id = track.id.clone(); + let edits = [ + SessionEdit::UpsertTrack { track }, + SessionEdit::UpsertRoute { route }, + ]; + let codec = ProtocolCodec::default(); + let frame = TypedCommandFrame { + request_id: RequestId::new(1).expect("nonzero request"), + expected_revision: ExpectedRevision::Exact( + children.session.test_controller().session().revision(), + ), + payload: CommandPayload::SessionTransactionApply(&edits), + }; + let mut bytes = vec![0_u8; codec.limits().max_frame_bytes]; + let length = codec + .encode_command_frame_into(&frame, &mut bytes) + .expect("typed command"); + children + .session + .command(&bytes[..length], 4_096) + .expect("structural transaction"); + } + + /// Render the next block, then run the service step. + fn render(children: &mut CompiledChildren, start: u64) { + let mut samples = [0.0_f32; 2 * QUANTUM as usize]; + let output = + PlanarBufferMut::try_new(&mut samples, 2, QUANTUM as usize, QUANTUM as usize) + .expect("output"); + children.plan.render(start, output).expect("render"); + children.session.service().expect("service"); + } + + /// One run: the running plan has no raised floor and offset `offset`; then declare. + /// Returns the plan epochs the declaration published. + fn declare_after(offset: u64) -> u64 { + super::super::test_reset_lifecycle_observer(); + let mut children = + compile_children::(SESSION, limits(), Box::new(Preparer)).expect("boot"); + if offset > 0 { + children.session.test_set_next_source_read_offset(offset); + } + add_muted_track(&mut children); + render(&mut children, 0); + let inventory = children.session.test_providers().test_inventory(); + assert!(inventory.raised_arrivals().is_empty(), "no floor is raised"); + assert_eq!(inventory.source_read_offset(), offset); + assert_eq!(children.session.test_render_clock(), QUANTUM); + assert_eq!(children.session.test_source_read_clock(), QUANTUM + offset); + + let epoch = children.session.newest_plan_epoch(); + children + .session + .declare_discontinuity() + .expect("declaration"); + let published = children.session.newest_plan_epoch() - epoch; + render(&mut children, QUANTUM); + assert_eq!( + children + .session + .test_providers() + .test_inventory() + .source_read_offset(), + 0 + ); + assert_eq!(children.session.test_render_clock(), 2 * QUANTUM); + assert_eq!( + children.session.test_source_read_clock(), + children.session.test_render_clock() + ); + published + } + + /// Test value: red if the declaration tests floors only (a plan that reads ahead with no + /// raised floor publishes nothing, so `ΣP` never resets and the source-read clock stays + /// ahead), or if a discontinuity successor inherits its predecessor's offset. + #[test] + fn a_declaration_resets_an_offset_with_no_raised_floor() { + assert_eq!(declare_after(OFFSET), 1, "the offset alone rebuilds"); + assert_eq!(declare_after(0), 0, "nothing to reset (#1323 gate 3)"); + } + } } diff --git a/crates/engine/src/realtime/mod.rs b/crates/engine/src/realtime/mod.rs index 4e09a061e..51fac52f7 100644 --- a/crates/engine/src/realtime/mod.rs +++ b/crates/engine/src/realtime/mod.rs @@ -223,6 +223,107 @@ mod tests { assert_eq!(plan.next_absolute_sample(), 8); } + /// A silent executor that records the source-read offset its plan hands it. + struct OffsetExecutor { + offset: std::sync::Arc, + } + + impl PreparedPlanExecutor for OffsetExecutor { + fn set_source_read_offset(&mut self, offset: u64) { + self.offset + .store(offset, std::sync::atomic::Ordering::Relaxed); + } + + fn render( + &mut self, + _arena: &mut BufferArena, + mut output: PlanarBufferMut<'_>, + _time: RenderTime, + ) -> Result<(), RenderError> { + output.plane_mut(0)?.fill(0.0); + Ok(()) + } + } + + /// #1396 gate 4: through 8 blocks and two swaps (offset 384, then 0), after each block the + /// publisher's render clock is the active plan's next sample and its source-read clock is that + /// plus the active plan's offset; before the first block both read the initial plan's. + /// + /// Test value: red if render stores the outgoing plan's offset after a swap, stores the clocks + /// before the block renders, or never stores them; or if a plan does not forward its offset + /// to its executor. + #[test] + fn the_publisher_reads_both_clocks_of_the_active_plan_after_each_block() { + const QUANTUM: u64 = 128; + let offset_plan = |id: u64, offset: u64| { + let seen = std::sync::Arc::new(std::sync::atomic::AtomicU64::new(u64::MAX)); + let mut plan = PreparedRenderPlan::prepare_with_executor( + PrepareRenderPlan { + plan_id: id, + envelope: RenderEnvelope { + sample_rate: SampleRateHz(48_000), + quantum: QuantumFrames(QUANTUM as u32), + output_channels: NonZeroUsize::new(1).expect("one"), + }, + scratch: &[], + }, + Box::new(OffsetExecutor { + offset: std::sync::Arc::clone(&seen), + }), + ) + .expect("plan"); + plan.set_source_read_offset(offset); + assert_eq!(plan.source_read_offset(), offset); + assert_eq!(seen.load(std::sync::atomic::Ordering::Relaxed), offset); + plan + }; + let mut initial = offset_plan(1, 0); + initial.set_source_read_offset(QUANTUM); + let (mut publisher, mut realtime, mut retirer) = plan_exchange( + initial, + PlanExchangeConfig { + retirement_capacity: NonZeroUsize::new(2).expect("two"), + }, + ) + .expect("exchange"); + assert_eq!(publisher.render_clock(), 0); + assert_eq!(publisher.source_read_clock(), QUANTUM); + + let mut active_offset = QUANTUM; + let mut samples = [0.0_f32; QUANTUM as usize]; + for block in 0..8_u64 { + let swap = match block { + 2 => Some(3 * QUANTUM), + 5 => Some(0), + _ => None, + }; + if let Some(offset) = swap { + let _ = publisher + .publish(offset_plan(block + 2, offset)) + .map_err(|_| "publish") + .expect("publish"); + } + let output = + PlanarBufferMut::try_new(&mut samples, 1, QUANTUM as usize, QUANTUM as usize) + .expect("output"); + let report = realtime + .render_contiguous(RenderIo { output }, block * QUANTUM) + .expect("block"); + if let Some(offset) = swap { + assert_eq!(report.swap, SwapOutcome::Applied); + active_offset = offset; + } + let next = (block + 1) * QUANTUM; + assert_eq!(publisher.render_clock(), next, "block {block}"); + assert_eq!( + publisher.source_read_clock(), + next + active_offset, + "block {block}" + ); + while retirer.try_reclaim().is_ok() {} + } + } + #[test] fn arena_is_fixed_and_disjoint() { let spec = PlanarBufferSpec { diff --git a/crates/engine/src/realtime/plan.rs b/crates/engine/src/realtime/plan.rs index 6ce2f9471..486c1958e 100644 --- a/crates/engine/src/realtime/plan.rs +++ b/crates/engine/src/realtime/plan.rs @@ -356,6 +356,15 @@ pub trait PreparedPlanExecutor: Send { fn prime_ready(&self, _block_start: u64, _lead_blocks: u32) -> bool { false } + /// Read this plan's sources `offset` samples ahead of its render clock (#1396 D1): a block + /// that renders from absolute sample `r` reads source time from `r + offset`, its + /// source-read sample, on which an anchored seek is compared. + /// + /// **Where it runs.** On the control thread, on an unpublished plan, through + /// [`PreparedRenderPlan::set_source_read_offset`]; never on the render thread. + /// + /// The default ignores the offset, which is right for an executor that reads no source. + fn set_source_read_offset(&mut self, _offset: u64) {} /// Invalidate owner-side observers after a render refusal that happened before dispatch. /// /// This is an internal hook so a prepared executor can discard a partial capture without @@ -760,6 +769,9 @@ pub struct PreparedRenderPlan { arena: BufferArena, rendered_blocks: u64, next_absolute_sample: u64, + /// `ΣP`, how far this plan reads its sources ahead of its render clock (#1396 D1). Set on the + /// control thread before publication; render reads it and never changes it. + source_read_offset: u64, response_snapshot_valid: bool, executor: Option>, /// This plan's revision gate (#1502 D1), built at revision 0 on the control thread. Render @@ -839,6 +851,7 @@ impl PreparedRenderPlan { arena: BufferArena::try_new(request.scratch)?, rendered_blocks: 0, next_absolute_sample: 0, + source_read_offset: 0, response_snapshot_valid: true, executor: None, gate: Arc::new(gate), @@ -925,6 +938,24 @@ impl PreparedRenderPlan { pub(crate) const fn adopt_absolute_sample(&mut self, absolute_sample: u64) { self.next_absolute_sample = absolute_sample; } + /// How far this plan reads its sources ahead of its render clock, `ΣP` (#1396 D1): a block + /// that renders from absolute sample `r` reads source time from `r + ΣP`, the plan's + /// source-read clock, on which every anchored seek is compared. 0 (the default) reads sources + /// on the render clock. + #[must_use] + pub const fn source_read_offset(&self) -> u64 { + self.source_read_offset + } + /// Set [`Self::source_read_offset`] and hand it to the executor + /// ([`PreparedPlanExecutor::set_source_read_offset`]). A control-thread call on an + /// unpublished plan: once published, a plan's offset never changes, and a swap does not carry + /// it (each plan is prepared with its own, #1396 D2). + pub fn set_source_read_offset(&mut self, offset: u64) { + self.source_read_offset = offset; + if let Some(executor) = self.executor.as_deref_mut() { + executor.set_source_read_offset(offset); + } + } /// Immutable structural program. #[must_use] pub const fn program(&self) -> &PreparedProgram { diff --git a/crates/engine/src/realtime/plan_exchange.rs b/crates/engine/src/realtime/plan_exchange.rs index c785ca33a..ca2767e82 100644 --- a/crates/engine/src/realtime/plan_exchange.rs +++ b/crates/engine/src/realtime/plan_exchange.rs @@ -58,7 +58,7 @@ use core::{ cell::Cell, fmt, num::NonZeroUsize, - sync::atomic::{AtomicUsize, Ordering}, + sync::atomic::{AtomicU64, AtomicUsize, Ordering}, }; use std::sync::Arc; @@ -75,8 +75,8 @@ pub struct PlanExchangeConfig { #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct PlanExchangeResourceReport { /// Sum of the mailbox (its state word and two cells with their adoption schedules), the - /// retirement SPSC header and backing, the shared credit-counter allocation and the - /// applied-revision watermark record. + /// retirement SPSC header and backing, the shared credit-counter allocation, the + /// applied-revision watermark record and the shared clock record (#1396 D4). pub retained_payload_bytes: u64, /// Largest single requested heap payload allocation among those rows. pub largest_allocation_bytes: u64, @@ -88,6 +88,29 @@ struct SharedCounterAllocation { weak: AtomicUsize, value: AtomicUsize, } + +/// The two clocks render publishes after each rendered block (#1396 D4): the active plan's next +/// render sample, and that sample plus the plan's source-read offset. Each is one word, stored +/// `Release` by render and loaded `Acquire` by the control thread, so neither read can tear. +struct PlanClocks { + render: AtomicU64, + source_read: AtomicU64, +} + +impl PlanClocks { + /// The clocks of `plan` as it stands: its next render sample and its source-read sample. + fn of(plan: &PreparedRenderPlan) -> (u64, u64) { + let render = plan.next_absolute_sample(); + (render, render.saturating_add(plan.source_read_offset())) + } +} + +#[repr(C)] +struct SharedClockAllocation { + strong: AtomicUsize, + weak: AtomicUsize, + clocks: PlanClocks, +} /// Result of one render-entry swap decision. #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub enum SwapOutcome { @@ -150,6 +173,7 @@ pub struct PlanPublisher { envelope: super::RenderEnvelope, retirement_credits: Arc, watermark: PlanWatermarkReader, + clocks: Arc, } /// Control-side retirement owner. Reclamation happens only by popping here. pub struct PlanRetirer { @@ -165,6 +189,8 @@ pub struct RealtimePlanOwner { /// Blocks the executor failed after the live snapshot (#1502 D7), saturating. live_block_errors: u64, watermark: WatermarkWriter, + /// The clock record render stores after each rendered block (#1396 D4). + clocks: Arc, _not_sync: Cell<()>, } /// Fixed report proving which complete plan owned a rendered block. @@ -344,6 +370,7 @@ pub fn plan_exchange_resource_report( retirement.slot_payload_bytes, Layout::new::().size(), watermark_retained_bytes(), + Layout::new::().size(), ]; let retained = rows.iter().try_fold(0_u64, |total, row| { total @@ -443,6 +470,12 @@ fn plan_exchange_inner( bounded_spsc_internal(config.retirement_capacity, QueueGeneration(2))?; let (mailbox_writer, mailbox_reader) = plan_mailbox(); let (watermark_writer, watermark_reader) = plan_watermark(initial_revision); + // #1396 D4: before the first block both clocks read the initial plan's values. + let (render, source_read) = PlanClocks::of(&initial); + let clocks = Arc::new(PlanClocks { + render: AtomicU64::new(render), + source_read: AtomicU64::new(source_read), + }); Ok(( PlanPublisher { mailbox: mailbox_writer, @@ -450,6 +483,7 @@ fn plan_exchange_inner( envelope, retirement_credits, watermark: watermark_reader, + clocks: Arc::clone(&clocks), }, RealtimePlanOwner { active: (PlanEpoch(0), initial), @@ -459,6 +493,7 @@ fn plan_exchange_inner( carry_mismatched: 0, live_block_errors: 0, watermark: watermark_writer, + clocks, _not_sync: Cell::new(()), }, PlanRetirer { @@ -545,6 +580,23 @@ impl PlanPublisher { self.watermark.clone() } + /// The running plan's render clock (#1396 D4): the absolute sample its next block renders + /// from, as render stored it after its last rendered block, or the initial plan's before the + /// first. One `Acquire` load of one word. The warm-successor deadline reads this clock. + #[must_use] + pub fn render_clock(&self) -> u64 { + self.clocks.render.load(Ordering::Acquire) + } + + /// The running plan's source-read clock (#1396 D4): the render clock plus the plan's + /// source-read offset ([`PreparedRenderPlan::source_read_offset`]), as render stored it after + /// its last rendered block, or the initial plan's before the first. One `Acquire` load of one + /// word. A host anchors a seek on this clock. + #[must_use] + pub fn source_read_clock(&self) -> u64 { + self.clocks.source_read.load(Ordering::Acquire) + } + /// Take back the last published candidate if render has not claimed it. /// /// One compare-and-swap marks its cell `Empty`. If it fails, render's claim won; render @@ -822,6 +874,12 @@ impl RealtimePlanOwner { let watermark = self .watermark .advance(render.live_snapshot.revision(), time.absolute_sample); + // #1396 D4: the plan that rendered the block, after it rendered: two stores. + let (render_clock, source_read_clock) = PlanClocks::of(&self.active.1); + self.clocks + .source_read + .store(source_read_clock, Ordering::Release); + self.clocks.render.store(render_clock, Ordering::Release); Ok((render, watermark)) } Err(failure) => { diff --git a/crates/graph/src/lib.rs b/crates/graph/src/lib.rs index f6b850fff..b3afe5d3f 100644 --- a/crates/graph/src/lib.rs +++ b/crates/graph/src/lib.rs @@ -3258,6 +3258,10 @@ struct GraphExecutor { /// would apply an `Observe` record still queued for a per-node effect, and so where a swap's /// carry applies it (#1281; a bank stage keeps its own, #1280 D3). Zero until a block renders. next_block_sample: u64, + /// How far this plan reads its sources ahead of its render clock (#1396 D1): the source + /// section's block entry reads from `time.absolute_sample + source_read_offset`. Set on the + /// control thread before publication. + source_read_offset: u64, /// Restores a swap's carry refused on a per-node successor instance, which no bank stage /// counts (#1281, #1279 D6). Saturating; read after render. node_carry_refused: u64, @@ -4555,6 +4559,7 @@ struct GraphExecutorWithoutSplitPairTable { carry_effects: Box<[GraphEffectMove]>, carry_scratch: Box<[u8]>, next_block_sample: u64, + source_read_offset: u64, node_carry_refused: u64, observer_carry: Option>, carry_delays: Box<[GraphDelayMove]>, @@ -4679,6 +4684,7 @@ impl GraphExecutor { carry_effects: Box::default(), carry_scratch: Box::default(), next_block_sample: 0, + source_read_offset: 0, node_carry_refused: 0, observer_carry: None, carry_delays: Box::default(), @@ -4701,6 +4707,10 @@ impl PreparedPlanExecutor for GraphExecutor { Some(self) } + fn set_source_read_offset(&mut self, offset: u64) { + self.source_read_offset = offset; + } + // REALTIME_POLICY_BEGIN /// Move the sources the installed [`GraphCarryProgram`] names out of the predecessor graph /// plan, and copy the strip input lanes [`install_builtin_input_carry`] names (issue #1276), @@ -4917,6 +4927,7 @@ impl PreparedPlanExecutor for GraphExecutor { carry_effects: _, carry_scratch: _, next_block_sample, + source_read_offset, node_carry_refused: _, observer_carry: _, carry_delays: _, @@ -4951,8 +4962,15 @@ impl PreparedPlanExecutor for GraphExecutor { probe.enter(test_only_phase_profile::SOURCE); } let source_validity = if let Some(source_set) = source_set.as_mut() { + // #1396 D1: the source-read sample, `ΣP` ahead of the render clock. + let Some(source_read_sample) = time.absolute_sample.checked_add(*source_read_offset) + else { + runtime.invalidate_observers_after_failure(time.absolute_sample); + host.silence(); + return Err(RenderError::TimeOverflow); + }; if let Err(error) = - source_set.begin_block(time.absolute_sample, source_set.envelope.quantum.0) + source_set.begin_block(source_read_sample, source_set.envelope.quantum.0) { runtime.invalidate_observers_after_failure(time.absolute_sample); host.silence(); diff --git a/crates/host-core/src/lib.rs b/crates/host-core/src/lib.rs index dc0bce084..c2036d47c 100644 --- a/crates/host-core/src/lib.rs +++ b/crates/host-core/src/lib.rs @@ -230,7 +230,7 @@ pub use prepare::{ test_only_prepare_host_runtime_on, test_only_prepare_host_runtime_successor_on, test_only_prepare_host_runtime_with_live_controls_on, test_only_prepare_host_runtime_with_live_controls_successor_on, - test_only_prepare_host_runtime_with_selected_meters_on, + test_only_prepare_host_runtime_with_selected_meters_on, test_only_set_source_read_offset, }; #[cfg(feature = "test-support")] pub use spectrum::{ diff --git a/crates/host-core/src/prepare.rs b/crates/host-core/src/prepare.rs index 314067a07..04aa6a71b 100644 --- a/crates/host-core/src/prepare.rs +++ b/crates/host-core/src/prepare.rs @@ -697,6 +697,16 @@ impl PreparedHost { } } +/// Set `host`'s source-read offset (#1396 deliverable 2): the plan's +/// ([`PreparedRenderPlan::set_source_read_offset`]) and its inventory row's together, so a +/// successor prepared from the inventory inherits it. Call it before the plan is published. The +/// one production writer of a nonzero offset is a warm successor's preparation (#1355). +#[cfg(feature = "test-support")] +pub fn test_only_set_source_read_offset(host: &mut PreparedHost, offset: u64) { + host.plan.set_source_read_offset(offset); + host.inventory.source_read_offset = offset; +} + /// One source of a prepared plan, as its successor's preparation sees it (issue #1272 D1). #[derive(Clone, Debug, Eq, PartialEq)] pub struct SourceInventoryRow { @@ -765,6 +775,10 @@ pub struct PlanStateInventory { /// ID: its latest incoming arrival, below its floored arrival in `input_floors`. Empty when the /// plan runs at its natural timing. raised_arrivals: Box<[(graph::GraphNodeId, u64)]>, + /// `ΣP`, how far the plan reads its sources ahead of its render clock (#1396 D2): its + /// predecessor's for an ordinary successor, 0 for a first plan and at a declared + /// discontinuity. + source_read_offset: u64, } /// One strip stage of a prepared plan, owner key `(strip ID, PostFader)` or `(strip ID, @@ -995,12 +1009,20 @@ impl PlanStateInventory { } /// Whether this plan runs behind its natural timing (#1323 D2): a floor raised some node's - /// input arrival above its natural one. A host-declared discontinuity rebuilds only such a - /// plan. (*Give a plan a source-read clock that leads its render clock*, #1396 D2, adds a - /// source-read offset above 0 to this test.) + /// input arrival above its natural one, or the plan reads its sources ahead of its render + /// clock (a [`Self::source_read_offset`] above 0, #1396 D2). A host-declared discontinuity + /// rebuilds only such a plan. The offset term matters because a plan can lead with no raised + /// floor: after a warm growth a later edit can make every natural arrival equal its floor. #[must_use] pub fn runs_behind_natural_timing(&self) -> bool { - !self.raised_arrivals.is_empty() + !self.raised_arrivals.is_empty() || self.source_read_offset > 0 + } + + /// `ΣP`, how far this plan reads its sources ahead of its render clock (#1396 D2): the + /// [`PreparedRenderPlan::source_read_offset`] its preparation set. + #[must_use] + pub const fn source_read_offset(&self) -> u64 { + self.source_read_offset } /// The base inventory of a successor at a host-declared discontinuity (#1323 D3): this @@ -1027,6 +1049,8 @@ impl PlanStateInventory { restarted: Box::default(), input_floors: Box::default(), raised_arrivals: Box::default(), + // #1396 D2: no read-ahead. + source_read_offset: 0, } } @@ -1055,6 +1079,7 @@ impl PlanStateInventory { restarted: Box::default(), input_floors: self.input_floors.clone(), raised_arrivals: self.raised_arrivals.clone(), + source_read_offset: 0, } } @@ -3407,6 +3432,12 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( restarted.extend(strip_delay_carry.restarted); restarted.sort_unstable(); restarted.dedup(); + // #1396 D2: an ordinary successor reads its sources as far ahead of its render clock as its + // predecessor does, so the source-read clock never moves backwards across a rebuild. A first + // plan reads on its render clock, and so does a discontinuity successor, whose base inventory + // is `at_discontinuity`'s. + let source_read_offset = successor.map_or(0, |base| base.inventory.source_read_offset); + plan.set_source_read_offset(source_read_offset); let inventory = PlanStateInventory { plan_identity: graph::plan_identity(&mut plan) .ok_or_else(|| graph_failure("host.inventory.identity"))?, @@ -3427,6 +3458,7 @@ fn prepare_host_runtime_with_live_controls_policy_and_spectrum( restarted: restarted.into_boxed_slice(), input_floors, raised_arrivals, + source_read_offset, }; let report = HostPrepareReport { sample_rate_hz: compiled.sample_rate().0, @@ -6081,6 +6113,8 @@ mod tests { restarted: Box::default(), input_floors: Box::default(), raised_arrivals: Box::default(), + // #1396 D2: no read-ahead. + source_read_offset: 0, } } diff --git a/crates/host-core/tests/source_read_clock.rs b/crates/host-core/tests/source_read_clock.rs new file mode 100644 index 000000000..4d48bff9d --- /dev/null +++ b/crates/host-core/tests/source_read_clock.rs @@ -0,0 +1,295 @@ +//! Issue #1396: a plan's source-read clock, which leads its render clock by the plan's +//! source-read offset `ΣP`. +//! +//! * **Gate 1, anchor on the source-read clock.** A plan prepared with offset `3 * 128` applies a +//! `seek_at` anchored at source-read sample `A` in the block that renders from `A - 384`: the +//! output equals a zero-offset reference anchored at `A - 384`. +//! * **Gate 2, an ordinary rebuild inherits.** An ordinary successor of that plan (a fader-only +//! prepared change) reports offset 384, and a seek held across the swap applies in the same +//! block as in a run without the rebuild. +//! * **Gate 3, a declaration resets, no frame lost.** A successor prepared at a declared +//! discontinuity reports offset 0, plays each source's next frame after the predecessor's last +//! played frame, and the publisher's two clocks are equal from its first block on. +//! +//! The session is the fixture reduced to one audible track `eq0` on [`SOURCE`] and a muted copy +//! of it, with no effect and no filter: its output is a fixed gain of the source PCM, so two runs +//! whose consumers read the same frames in the same blocks render the same bits. + +#![cfg(feature = "test-support")] + +#[path = "support/successor.rs"] +mod successor; + +use engine::realtime::{PlanAdoption, SwapOutcome}; +use graph_compiler::Backend; +use host_core::{ + PreparedHost, SourceControlSet, SourceSubmission, SuccessorBase, + test_only_prepare_host_runtime_successor_on, test_only_set_source_read_offset, +}; +use session::{RouteSource, SessionModel, StableId, parse_session_json}; +use successor::{QUANTUM, SUCCESSOR_REVISION, Session, caps, exchange, render}; + +const FIXTURE: &str = include_str!("../../../fixtures/session/v1/parametric-eq-nine-track.json"); +const SOURCE: &str = "fixture-source"; +const MUTED_TRACK: &str = "a-muted"; +/// `ΣP` of the plans under test: three quanta. +const OFFSET: u64 = 3 * QUANTUM as u64; +/// Blocks in every run. +const BLOCKS: usize = 12; +/// A seek's anchor on the source-read clock, as a block index: the offset plan renders it at +/// block `ANCHOR_BLOCK - 3`. +const ANCHOR_BLOCK: usize = 8; +/// The block before which the seek is queued. +const SEEK_BLOCK: usize = 2; +/// The block the successor renders first. +const SWAP_BLOCK: usize = 3; +/// The source frame the seek jumps to. +const TARGET_FRAME: usize = 40 * QUANTUM; + +fn id(value: &str) -> StableId { + StableId::parse(value).expect("stable ID") +} + +/// Track `eq0` on [`SOURCE`] and its muted copy [`MUTED_TRACK`], both console sections empty, no +/// inserts, input filters off. `muted_db` is the muted copy's fader gain. +fn session(muted_db: f32) -> Session { + let mut model: SessionModel = parse_session_json(FIXTURE).expect("fixture parses"); + model.console.pre_insert.clear(); + model.console.post_insert.clear(); + model.tracks.retain(|track| track.id.as_str() == "eq0"); + for track in &mut model.tracks { + track.console.clear(); + track.inserts.effects.clear(); + for channel in [&mut track.builtins.left, &mut track.builtins.right] { + channel.hpf_hz = 0.0; + channel.lpf_hz = 0.0; + } + } + model.routes.retain(|route| route.id.as_str() == "eq0-main"); + let mut muted = model.tracks[0].clone(); + muted.id = id(MUTED_TRACK); + muted.fader.left_mute = true; + muted.fader.right_mute = true; + muted.fader.left_db = muted_db; + muted.fader.right_db = muted_db; + model.tracks.push(muted); + let mut route = model.routes[0].clone(); + route.id = id(&format!("{MUTED_TRACK}-main")); + let RouteSource::Track { track_id, .. } = &mut route.source else { + panic!("a track route"); + }; + *track_id = id(MUTED_TRACK); + model.routes.push(route); + Session::compile(model) +} + +/// `session` prepared fresh with source-read offset `offset`. +fn prepared(session: &Session, offset: u64) -> PreparedHost { + let mut prepared = session.prepare(Backend::current()); + test_only_set_source_read_offset(&mut prepared, offset); + assert_eq!(prepared.plan.source_read_offset(), offset); + assert_eq!(prepared.inventory.source_read_offset(), offset); + prepared +} + +/// Ramp PCM: every sample of source frame `f` is `f`, so a repeated or skipped frame moves a bit. +fn ramp(frames: core::ops::Range) -> Vec { + frames.map(|frame| frame as f32).collect() +} + +/// Submit source frames `frames` of the ramp at `generation`, one quantum per chunk. +fn submit(sources: &mut SourceControlSet, generation: u64, frames: core::ops::Range) { + for start in frames.step_by(QUANTUM) { + let plane = ramp(start..start + QUANTUM); + sources + .submit( + SOURCE.as_bytes(), + SourceSubmission { + generation, + start_frame: start as u64, + sample_rate_hz: 48_000, + planes: &[&plane, &plane], + frames: QUANTUM as u32, + end_of_region: false, + }, + ) + .unwrap_or_else(|error| panic!("generation {generation} frame {start}: {error:?}")); + } +} + +/// How a run changes its plan at [`SWAP_BLOCK`]. +#[derive(Clone, Copy)] +enum Swap { + /// No rebuild. + None, + /// An ordinary successor of [`session`]`(-6.0)`. + Ordinary, + /// A successor at a declared discontinuity, of the same session. + Discontinuity, +} + +/// One run of `session(0.0)` prepared with `offset`, its blocks' bits, and per block the +/// publisher's `(render, source-read)` clocks after it. +/// +/// With `anchor` it queues, before [`SEEK_BLOCK`] renders, `seek_at(SOURCE, 2, TARGET_FRAME, +/// anchor)` and generation 2's PCM from [`TARGET_FRAME`]; generation 1 is queued, before anything +/// renders, for exactly the blocks before the anchor's render block, so a seek that applies late +/// renders held blocks. Without one it feeds the ramp one block ahead. +fn run(offset: u64, anchor: Option, swap: Swap) -> (Vec>, Vec<(u64, u64)>) { + let a = session(0.0); + let PreparedHost { + plan, + mut sources, + inventory, + .. + } = prepared(&a, offset); + let (mut publisher, mut owner, mut retirer) = exchange(plan); + let render_blocks = match anchor { + Some(anchor) => usize::try_from(anchor - offset).expect("anchor") / QUANTUM, + None => 1, + }; + submit(&mut sources, 1, 0..render_blocks * QUANTUM); + let mut successor_sources = None; + let mut bits = Vec::with_capacity(BLOCKS); + let mut clocks = Vec::with_capacity(BLOCKS); + for block in 0..BLOCKS { + if block == SEEK_BLOCK + && let Some(anchor) = anchor + { + sources + .seek_at(SOURCE.as_bytes(), 2, TARGET_FRAME as u64, anchor) + .expect("anchored seek"); + submit( + &mut sources, + 2, + TARGET_FRAME..TARGET_FRAME + (BLOCKS - render_blocks) * QUANTUM, + ); + } + if block == SWAP_BLOCK && !matches!(swap, Swap::None) { + let (b, discontinuity) = match swap { + Swap::Ordinary => (session(-6.0), false), + _ => (session(0.0), true), + }; + let mut next = test_only_prepare_host_runtime_successor_on( + &b.compiled, + &caps(), + SuccessorBase { + discontinuity, + ..SuccessorBase::new(&inventory, &a.model, SUCCESSOR_REVISION) + }, + Backend::current(), + ) + .unwrap_or_else(|failure| panic!("successor: {failure:?}")); + let expected = if discontinuity { 0 } else { OFFSET }; + assert_eq!( + next.plan.source_read_offset(), + expected, + "the plan's offset" + ); + assert_eq!( + next.inventory.source_read_offset(), + expected, + "the inventory row's offset" + ); + assert_eq!(next.sources.adopt_persisting(&mut sources), 1); + next.plan + .take_revision_gate() + .expect("the successor's handle") + .publish(SUCCESSOR_REVISION); + publisher + .reserve_replacement(next.plan, PlanAdoption::Next) + .unwrap_or_else(|_| panic!("reserve the successor")) + .commit(); + successor_sources = Some(next.sources); + } + if anchor.is_none() && block + 1 < BLOCKS { + let feeding = successor_sources.as_mut().unwrap_or(&mut sources); + submit(feeding, 1, (block + 1) * QUANTUM..(block + 2) * QUANTUM); + } + let rendered = render(&mut owner); + if block == SWAP_BLOCK && !matches!(swap, Swap::None) { + assert_eq!(rendered.swap, SwapOutcome::Applied); + } + bits.push(rendered.bits); + clocks.push((publisher.render_clock(), publisher.source_read_clock())); + while retirer.try_reclaim().is_ok() {} + } + (bits, clocks) +} + +/// The first block whose bits differ, if any. +fn first_difference(left: &[Vec], right: &[Vec]) -> Option { + left.iter() + .zip(right) + .position(|(left, right)| left != right) +} + +/// Gate 1 (the old #1287 gate 4, per D15-12). +/// +/// Test value: red if the graph compares a held seek with the render clock: the offset plan then +/// applies it three blocks late, and its blocks between hold. +#[test] +fn a_seek_applies_on_the_source_read_clock() { + let anchor = (ANCHOR_BLOCK * QUANTUM) as u64; + let (offset, clocks) = run(OFFSET, Some(anchor), Swap::None); + let (reference, _) = run(0, Some(anchor - OFFSET), Swap::None); + assert_eq!(first_difference(&offset, &reference), None); + // The jump is audible, and the run plays before and after it. + let jump = ANCHOR_BLOCK - 3; + assert_ne!(offset[jump - 1], offset[jump]); + assert!( + offset + .iter() + .all(|block| block.iter().any(|bits| *bits != 0)) + ); + for (block, (render, source_read)) in clocks.into_iter().enumerate() { + let next = ((block + 1) * QUANTUM) as u64; + assert_eq!( + (render, source_read), + (next, next + OFFSET), + "block {block}" + ); + } +} + +/// Gate 2. +/// +/// Test value: red if an ordinary successor defaults its offset to 0: the seek held across the +/// swap then applies three blocks late, and the blocks between hold. +#[test] +fn an_ordinary_successor_inherits_the_offset() { + let anchor = (ANCHOR_BLOCK * QUANTUM) as u64; + let (rebuilt, clocks) = run(OFFSET, Some(anchor), Swap::Ordinary); + let (unbroken, _) = run(OFFSET, Some(anchor), Swap::None); + assert_eq!(first_difference(&rebuilt, &unbroken), None); + for (block, (render, source_read)) in clocks.into_iter().enumerate() { + let next = ((block + 1) * QUANTUM) as u64; + assert_eq!( + (render, source_read), + (next, next + OFFSET), + "block {block}" + ); + } +} + +/// Gate 3. +/// +/// Test value: red if the reset rewinds or advances a consumer (a frame repeats or is skipped, +/// and the ramp moves a bit), or if the discontinuity successor keeps the predecessor's offset +/// (the source-read clock stays ahead). +#[test] +fn a_declaration_resets_the_offset_and_loses_no_frame() { + let (declared, clocks) = run(OFFSET, None, Swap::Discontinuity); + let (unbroken, _) = run(0, None, Swap::None); + assert_eq!(first_difference(&declared, &unbroken), None); + assert!(declared[SWAP_BLOCK].iter().any(|bits| *bits != 0)); + for (block, (render, source_read)) in clocks.into_iter().enumerate() { + let next = ((block + 1) * QUANTUM) as u64; + let expected = if block < SWAP_BLOCK { + next + OFFSET + } else { + next + }; + assert_eq!((render, source_read), (next, expected), "block {block}"); + } +} diff --git a/crates/source/src/lib.rs b/crates/source/src/lib.rs index 6f7e8c661..994eb130f 100644 --- a/crates/source/src/lib.rs +++ b/crates/source/src/lib.rs @@ -1297,9 +1297,9 @@ impl PcmSourceConsumer { } /// [`Self::begin_block`] for the block that reads source time from `source_read_sample`: the - /// plan's source-read clock, the absolute sample of source time this block reads; equal to - /// the render clock in this version (#1316 D1). *Give a plan a source-read clock that leads - /// its render clock* (#1396) is the issue that makes the two differ. + /// plan's source-read clock, the absolute sample of source time this block reads (#1316 D1): + /// the render clock plus the plan's source-read offset (#1396 D1). The offset is 0 unless a + /// plan is prepared to read ahead, so in this version the two clocks are equal. /// /// An anchored seek ([`SourceCommand::SeekAt`]) is held while `source_read_sample` is before /// its anchor; the block that starts at or past it applies the seek at `frame + @@ -2154,8 +2154,8 @@ impl GraphPreparedSourceSetDriver for SourceGraphSourceSetDriver { self.block_validity.source_underrun = true; continue; }; - // The plan's source-read clock, which an anchored seek waits on, equals the render - // clock in this version (#1316 D1); #1396 passes the source-read sample here. + // The graph passes the plan's source-read sample, which an anchored seek waits on: the + // render clock plus the plan's source-read offset (#1316 D1, #1396 D1). let report = consumer.begin_block_at(first_sample); self.block_validity.source_underrun |= report.underrun_event; self.block_validity.source_generation_changed |= report.generation_changed; From 544da3b1871a8f6e22b39f2df7ad061fef387e54 Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 05:46:11 +0000 Subject: [PATCH 06/10] Batch 15c review fold: policy comment attribution, literal-mask doc and redundant ffi pin, clock-pair snapshot note, rewraps Co-Authored-By: Claude Opus 5.5 --- crates/capi/include/miso_engine_v1.h | 84 ++++++++++----------- crates/capi/src/ffi.rs | 1 - crates/engine/src/realtime/plan_exchange.rs | 2 + crates/host-core/src/source.rs | 3 +- crates/source/src/lib.rs | 24 +++--- docs/C_ABI_V1_QUALIFICATION.md | 12 +-- tools/realtime-policy/src/policy.rs | 2 +- 7 files changed, 64 insertions(+), 64 deletions(-) diff --git a/crates/capi/include/miso_engine_v1.h b/crates/capi/include/miso_engine_v1.h index fa35c00ce..0b762ba57 100644 --- a/crates/capi/include/miso_engine_v1.h +++ b/crates/capi/include/miso_engine_v1.h @@ -180,20 +180,19 @@ extern "C" { * render has retired, brings the session's view of the running plan up to render, refreshes the * session counters, and stages render telemetry. Every other session call (source submit, seek, * seek_at, seek_report, submit_command, dequeue_event) that passes its argument checks runs the - * same step - * first; a call refused for its arguments returns before the step. So a host that makes those - * calls anyway need not also call service. The step never waits for render, never commits or + * same step first; a call refused for its arguments returns before the step. So a host that makes + * those calls anyway need not also call service. The step never waits for render, never commits or * acknowledges anything, and never takes or drops a pending edit; its cost does not grow with the * time since the last call. It returns MISO_ENGINE_V1_OK; a null session as * MISO_ENGINE_V1_INVALID_ARGUMENT and another handle as MISO_ENGINE_V1_WRONG_HANDLE; and, should * the session's plan epochs fail to synchronize, MISO_ENGINE_V1_INTERNAL with the session * diagnostic "capi.source.epoch". The host's duty: a pending edit progresses only while the host - * makes session calls, the same duty as draining events. A host that renders but never calls - * keeps retired plans allocated, and its next structural edit waits for that reclaim; a warm - * successor that misses its deadline falls back only at the host's next session call. The plan - * watermark never covers an edit before it is in effect. While edits are pending, call - * miso_engine_v1_service (or another session call) at least once per render-buffer period. A host - * checks MISO_ENGINE_V1_FEATURE_SERVICE before calling the symbol. + * makes session calls, the same duty as draining events. A host that renders but never calls keeps + * retired plans allocated, and its next structural edit waits for that reclaim; a warm successor + * that misses its deadline falls back only at the host's next session call. The plan watermark + * never covers an edit before it is in effect. While edits are pending, call miso_engine_v1_service + * (or another session call) at least once per render-buffer period. A host checks + * MISO_ENGINE_V1_FEATURE_SERVICE before calling the symbol. * * Engine memory (issue 1602). The engine owns no control-thread memory outside the compile * limits beyond its own handle; miso_engine_v1_engine_destroy frees that handle. Everything a @@ -235,40 +234,39 @@ extern "C" { * A transaction behind an unadopted replacement (issue 1310). A SESSION_TRANSACTION_APPLY that * needs a new plan while an earlier transaction's replacement plan has not been adopted by render * (a paused host never adopts one) commits as any other: it is not refused as backpressure because - * a replacement waits, with the one exception below. The engine takes the waiting replacement - * back, prepares one plan that carries both transactions against the plan that renders, and - * publishes it; nothing of the earlier transaction is lost. A source the new plan keeps from the - * waiting replacement or from the plan that renders keeps playing: the PCM accepted for it and a - * seek or seek_at queued on it move into the new plan, so the host neither refeeds nor reseeks it. - * An owner (a strip, an effect, a meter) that the earlier transaction restarted or removed and - * the new one puts back in its form in the plan that renders carries from the plan that renders, - * as one transaction of both would carry it (issue 1698). A source does not: its declaration - * changed against the newest committed session, so the host restarted its feed, and a source the - * earlier transaction added, redeclared or removed and the new one does not keep as the waiting - * replacement declares it starts fresh in the new plan (generation 1, frame 0, as an added one), - * also when the new one puts it back in its form in the plan that renders. No accepted chunk on a - * ring the waiting replacement created is dropped across a supersession: when the waiting - * replacement created a source's ring (the earlier transaction added the source, or redeclared - * it) and the session has accepted PCM, a seek or an end of region for that source since, a new - * transaction that does not keep the source as the waiting replacement declares it (it declares - * it as the plan that renders does, redeclares it another way, or removes it) is refused with - * MISO_ENGINE_V1_BACKPRESSURE before it commits: - * nothing changes, the waiting replacement stays published, no event is queued and the revision - * does not move. Once render adopts the waiting replacement, a retry commits. A host that does not - * render keeps getting MISO_ENGINE_V1_BACKPRESSURE for such a transaction until it renders. A - * source the waiting replacement carries unchanged from the plan that renders is not refused this - * way. A source the caller's own transaction removes discards its queued audio, whether the plan - * that renders or the waiting replacement (carrying it unchanged) holds it: the caller asked for - * the removal, so this is not an unintended drop. No counter reports the discarded frames yet; - * issue 1699 is the successor issue for one. A resource cap counts - * every plan the session still holds and the new one: the running plan, the waiting replacement, - * and a plan render has displaced until a later call reclaims it. A refusal is - * MISO_ENGINE_V1_BACKPRESSURE when the new plan fit against the newest held plan alone, so a retry - * after a render normally commits; the split is approximate, and such a retry may still be refused - * within a small band (a plan prepared on the adopted replacement differs by a few hundred bytes - * from one prepared across it). A refusal that does not fit even then is - * MISO_ENGINE_V1_COMPILE_REJECTED with the cap's diagnostic. When render adopts the new plan, the - * watermark advances to its revision with MISO_ENGINE_V1_OUTCOME_SUPERSEDED set, and + * a replacement waits, with the one exception below. The engine takes the waiting replacement back, + * prepares one plan that carries both transactions against the plan that renders, and publishes it; + * nothing of the earlier transaction is lost. A source the new plan keeps from the waiting + * replacement or from the plan that renders keeps playing: the PCM accepted for it and a seek or + * seek_at queued on it move into the new plan, so the host neither refeeds nor reseeks it. An owner + * (a strip, an effect, a meter) that the earlier transaction restarted or removed and the new one + * puts back in its form in the plan that renders carries from the plan that renders, as one + * transaction of both would carry it (issue 1698). A source does not: its declaration changed + * against the newest committed session, so the host restarted its feed, and a source the earlier + * transaction added, redeclared or removed and the new one does not keep as the waiting replacement + * declares it starts fresh in the new plan (generation 1, frame 0, as an added one), also when the + * new one puts it back in its form in the plan that renders. No accepted chunk on a ring the + * waiting replacement created is dropped across a supersession: when the waiting replacement + * created a source's ring (the earlier transaction added the source, or redeclared it) and the + * session has accepted PCM, a seek or an end of region for that source since, a new transaction + * that does not keep the source as the waiting replacement declares it (it declares it as the plan + * that renders does, redeclares it another way, or removes it) is refused with + * MISO_ENGINE_V1_BACKPRESSURE before it commits: nothing changes, the waiting replacement stays + * published, no event is queued and the revision does not move. Once render adopts the waiting + * replacement, a retry commits. A host that does not render keeps getting + * MISO_ENGINE_V1_BACKPRESSURE for such a transaction until it renders. A source the waiting + * replacement carries unchanged from the plan that renders is not refused this way. A source the + * caller's own transaction removes discards its queued audio, whether the plan that renders or the + * waiting replacement (carrying it unchanged) holds it: the caller asked for the removal, so this + * is not an unintended drop. No counter reports the discarded frames yet; issue 1699 is the + * successor issue for one. A resource cap counts every plan the session still holds and the new + * one: the running plan, the waiting replacement, and a plan render has displaced until a later + * call reclaims it. A refusal is MISO_ENGINE_V1_BACKPRESSURE when the new plan fit against the + * newest held plan alone, so a retry after a render normally commits; the split is approximate, and + * such a retry may still be refused within a small band (a plan prepared on the adopted replacement + * differs by a few hundred bytes from one prepared across it). A refusal that does not fit even + * then is MISO_ENGINE_V1_COMPILE_REJECTED with the cap's diagnostic. When render adopts the new + * plan, the watermark advances to its revision with MISO_ENGINE_V1_OUTCOME_SUPERSEDED set, and * superseded_count grows by the revisions the earlier replacements carried. * * The source-read clock (issues 1275, 1316). A seek that names a block names it on the plan's diff --git a/crates/capi/src/ffi.rs b/crates/capi/src/ffi.rs index 631f69e6b..d29a7dee0 100644 --- a/crates/capi/src/ffi.rs +++ b/crates/capi/src/ffi.rs @@ -1742,7 +1742,6 @@ mod tests { assert_eq!(query(&mut capabilities), RESULT_OK); assert_eq!(capabilities.abi_version, ABI_VERSION); assert_eq!(capabilities.exact_launch_rate_mask, 0x0f); - assert_eq!(capabilities.feature_mask, 0x3ff); assert_eq!(capabilities.feature_mask, crate::FEATURE_MASK); assert_ne!(capabilities.feature_mask & crate::FEATURE_SOURCE_SEEK_AT, 0); assert_ne!(capabilities.feature_mask & crate::FEATURE_PLAN_WATERMARK, 0); diff --git a/crates/engine/src/realtime/plan_exchange.rs b/crates/engine/src/realtime/plan_exchange.rs index ca2767e82..5f0b463d9 100644 --- a/crates/engine/src/realtime/plan_exchange.rs +++ b/crates/engine/src/realtime/plan_exchange.rs @@ -92,6 +92,8 @@ struct SharedCounterAllocation { /// The two clocks render publishes after each rendered block (#1396 D4): the active plan's next /// render sample, and that sample plus the plan's source-read offset. Each is one word, stored /// `Release` by render and loaded `Acquire` by the control thread, so neither read can tear. +/// The two loads are not a snapshot: render stores the source-read clock and then the render +/// clock, so a reader that needs both together must take them through `seqlock.rs`. struct PlanClocks { render: AtomicU64, source_read: AtomicU64, diff --git a/crates/host-core/src/source.rs b/crates/host-core/src/source.rs index db7c2362b..a93666584 100644 --- a/crates/host-core/src/source.rs +++ b/crates/host-core/src/source.rs @@ -221,7 +221,8 @@ impl SourceControlSet { /// /// `anchor_sample` is on the plan's source-read clock, which every plan swap continues and /// which equals the absolute render clock a host renders with in this version (#1316 D1); it - /// names when `frame` enters the graph, and the output hears it the plan's latency later. It must be a multiple of the quantum, or the seek is refused with + /// names when `frame` enters the graph, and the output hears it the plan's latency later. It + /// must be a multiple of the quantum, or the seek is refused with /// `source.seek.anchor_unaligned`. The region and generation rules are [`Self::seek`]'s. /// /// On success the source accepts the new generation's PCM from `frame` at once; submit it diff --git a/crates/source/src/lib.rs b/crates/source/src/lib.rs index 994eb130f..b8b82fa9d 100644 --- a/crates/source/src/lib.rs +++ b/crates/source/src/lib.rs @@ -67,14 +67,14 @@ pub enum SourceCommand { /// reads source time from `anchor_sample` (issue #1274). /// /// `anchor_sample` is on the plan's source-read clock -- the absolute sample of source time a - /// block reads, which the graph hands its source set each block and every plan swap - /// continues; equal to the render clock in this version (#1316 D1) -- and must be a multiple - /// of the quantum. The output hears `frame` the plan's latency later. The render consumer holds the + /// block reads, which the graph hands its source set each block and every plan swap continues; + /// equal to the render clock in this version (#1316 D1) -- and must be a multiple of the + /// quantum. The output hears `frame` the plan's latency later. The render consumer holds the /// seek until the first block starting at or after the anchor; observed late, it starts at - /// `frame + (block start - anchor_sample)`, so the source is in time either way. While the - /// seek is held, already queued PCM of the playing generation keeps playing and the new - /// generation's first submitted block waits as the pending block, so a host may prime the - /// ring from `frame` as soon as the seek is accepted. + /// `frame + (block start - anchor_sample)`, so the source is in time either way. While the seek + /// is held, already queued PCM of the playing generation keeps playing and the new generation's + /// first submitted block waits as the pending block, so a host may prime the ring from `frame` + /// as soon as the seek is accepted. SeekAt { /// The nonzero generation to make audible. generation: SourceGeneration, @@ -1303,11 +1303,11 @@ impl PcmSourceConsumer { /// /// An anchored seek ([`SourceCommand::SeekAt`]) is held while `source_read_sample` is before /// its anchor; the block that starts at or past it applies the seek at `frame + - /// (source_read_sample - anchor_sample)` (saturating). While it is held, queued PCM of the playing generation still - /// plays, the new generation's first block is kept as the pending block, and with nothing - /// playable the block is **held**, not an underrun (the host asked the source to wait; underrun - /// means the host owed PCM and did not deliver it). A newer command replaces a held one. This render-path - /// operation allocates nothing and never blocks. + /// (source_read_sample - anchor_sample)` (saturating). While it is held, queued PCM of the + /// playing generation still plays, the new generation's first block is kept as the pending + /// block, and with nothing playable the block is **held**, not an underrun (the host asked the + /// source to wait; underrun means the host owed PCM and did not deliver it). A newer command + /// replaces a held one. This render-path operation allocates nothing and never blocks. /// /// The block publishes the ring's seek report when a seek applies at it, at the ring's first /// block and when a cumulative counter moves ([`SourceSeekReport`], #1316 D3). diff --git a/docs/C_ABI_V1_QUALIFICATION.md b/docs/C_ABI_V1_QUALIFICATION.md index cfab42666..1839371fb 100644 --- a/docs/C_ABI_V1_QUALIFICATION.md +++ b/docs/C_ABI_V1_QUALIFICATION.md @@ -238,11 +238,11 @@ feature added as a protocol message is announced by its protocol capability flag protocol flags. A host tests each bit it needs with `&`, tolerates bits it does not know and never compares `feature_mask` with `==`. A feature bit protects only a host that resolves the symbol at run time: a host that links a symbol directly fails to load against a library that lacks it, before -it can test anything. The exact mask is asserted by the library's own Rust tests -(`version_and_capabilities_are_exact` and the constant pins in `abi.rs`), not by the C smoke test: -`abi_smoke.c` tests one `&` per `MISO_ENGINE_V1_FEATURE_*` symbol (it fails when the library stops -reporting a defined bit and passes when it reports an extra unknown one) and keeps its compile-time -pins of the header's constants. +it can test anything. The library's own Rust tests compare the reported mask with the library's `FEATURE_MASK` +(`version_and_capabilities_are_exact` and the constant pins in `abi.rs`); the C smoke test does not +compare the reported mask. `abi_smoke.c` tests one `&` per `MISO_ENGINE_V1_FEATURE_*` symbol (it +fails when the library stops reporting a defined bit and passes when it reports an extra unknown +one) and keeps its compile-time pins of the header's constants. The first C11-static launch found one qualification-fixture error: it attempted generation-1 seek before the initial generation-1 submission and exited 13. No product byte or staged library was @@ -986,7 +986,7 @@ that window is admitted against the plan that renders. exported symbol, announced by `MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY` (the next free bit, 256) in the capability report, whose feature mask grows to include it; no struct, size or `ABI_VERSION` changes. The mask is the OR of every feature symbol and is never compared with a - literal (`capi` `abi::tests::masks_and_result_codes_are_frozen` folds the symbols; `abi_smoke.c` + literal, except the C smoke test's header-constant pins (`capi` `abi::tests::masks_and_result_codes_are_frozen` folds the symbols; `abi_smoke.c` and `header_smoke.cpp` pin the bit inside the mask and the signature, and `abi_smoke.c` calls the symbol with a null session for `INVALID_ARGUMENT`). The frozen exported set is now 18 `miso_engine_v1_*` definitions. It is a session call, serialized with the others; it runs the diff --git a/tools/realtime-policy/src/policy.rs b/tools/realtime-policy/src/policy.rs index 0469f6ec9..1854d21cc 100644 --- a/tools/realtime-policy/src/policy.rs +++ b/tools/realtime-policy/src/policy.rs @@ -223,7 +223,7 @@ impl Policy { // `process_with`) and #1639's two (`crates/graph`: the jump-length cell's word decode // and its block-entry read) and #1630's one (`crates/builtins-compiler`: the cell // readers' carry marks, D15-2) and #1316's (`seqlock.rs`, the engine's one seqlock - // writer, B15-R1, and `crates/source`'s seek-report publication). The floors had fallen + // writer, B15-R1). The floors had fallen // behind the tree before #1637; they are set to its counts again. A change that adds a // marker raises them in the same change. marked_files_floor: 33, From 6fc1e26c39d16a2990adeec7373ec411bbdf7a24 Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 06:09:57 +0000 Subject: [PATCH 07/10] TBD44: model the engine seqlock under loom seqlock.rs takes its atomics and fences from the realtime sync shim (spsc.rs's mod sync, now visible inside crate::realtime), so loom models the protocol; a non-loom build resolves to the same core items. Loom model seqlock_loom_a_read_is_one_whole_record_or_busy (one writer, two records, one reader): a read is one whole record or Busy. Under cfg(loom) only, a read takes 2 attempts instead of 64 so loom can finish (4.3 s). Red with the reader's Acquire fence, the second sequence compare or the writer's Release fence removed (torn records). CI: a seqlock loom step next to the SPSC leg. No ordering changed; realtime policy 151/33. Co-Authored-By: Claude Opus 5.5 --- ...D44-model-the-engine-seqlock-under-loom.md | 112 ++++++++++++++++++ .github/workflows/qualification.yml | 4 + crates/engine/src/realtime/seqlock.rs | 47 +++++++- crates/engine/src/realtime/spsc.rs | 20 ++-- 4 files changed, 172 insertions(+), 11 deletions(-) create mode 100644 .github/ISSUE_SPECS/TBD44-model-the-engine-seqlock-under-loom.md diff --git a/.github/ISSUE_SPECS/TBD44-model-the-engine-seqlock-under-loom.md b/.github/ISSUE_SPECS/TBD44-model-the-engine-seqlock-under-loom.md new file mode 100644 index 000000000..8e0fdf6c3 --- /dev/null +++ b/.github/ISSUE_SPECS/TBD44-model-the-engine-seqlock-under-loom.md @@ -0,0 +1,112 @@ +# Model the engine seqlock under loom + +Root ruling of 2026-10-16, inside batch 15c. Follows #1316 (B15-R1). Feature sprint mode (owner, +2026-10-13): one implementer, normal tests, a batch review. Issue number: assigned at filing +(written as TBD44). + +## Product outcome + +The engine's one single-writer seqlock is proved under a weak memory model, not only stressed on +x86. Its two contracts, the applied-revision watermark and the per-ring seek report, are read by +hosts on iOS and Android (AArch64), where a missing fence or a missing sequence re-check tears a +record that x86's strong ordering hides. A loom model of the real code runs in CI on every +qualifying change. + +## Context + +- `crates/engine/src/realtime/seqlock.rs` (new in #1316, B15-R1): `PublishedWords`, its + writer's `publish` (the one `REALTIME_POLICY_BEGIN`/`END` region) and its reader's bounded + `read`. Before this issue it took `AtomicU64`, `Ordering`, `fence` and `Arc` from `core`/`std` + directly, so loom could not model it; its only concurrency test was the x86 stress test + `a_read_is_one_whole_publication_or_busy`. +- `crates/engine/src/realtime/watermark.rs` (applied-revision watermark, #1314) and + `crates/source/src/lib.rs` (seek report, #1316) are its two instances. +- `crates/engine/src/realtime/spsc.rs` owns the engine's `sync` shim: `core`/`std` in every + shipped build, `loom` under `--cfg loom`. The `spsc_loom_*` models run on it. +- CI: `.github/workflows/qualification.yml`, step "Realtime race model (engine SPSC loom leg)", + `CARGO_TARGET_DIR=target/ci/loom RUSTFLAGS='--cfg loom --check-cfg=cfg(loom)'`. + +## Decisions frozen + +- **D1. One shim.** The seqlock takes `Arc`, `AtomicU64`, `Ordering` and `fence` from the existing + `spsc.rs` `sync` shim; no second shim. A non-loom build resolves to the same `core`/`std` items + as before, so shipped code is unchanged. +- **D2. Orderings unchanged.** The protocol's orderings stay as #1316 shipped them unless loom + finds a defect; a defect is reported before it is fixed. +- **D3. The real code is the model.** The loom test calls `published_words`, `publish` and `read`; + no hand-written copy of the protocol. +- **D4. A loom-only read bound.** Under `--cfg loom` one read takes 2 attempts instead of + `MAXIMUM_READ_ATTEMPTS` (64): the no-tear argument holds per attempt, and 64 spinning attempts + are beyond loom's reach. Shipped builds keep 64. +- **D5. Own CI step.** The model runs in its own step next to the SPSC leg, same target dir and + flags, filtered to `seqlock_loom`, so neither filter can drop the other's models. + +## Authorized paths + +- `crates/engine/src/realtime/seqlock.rs` +- `crates/engine/src/realtime/spsc.rs` (the `sync` shim's visibility and its `fence` item only) +- `.github/workflows/qualification.yml` (the loom step only) +- this spec + +## Objective gates + +1. `seqlock_loom_a_read_is_one_whole_record_or_busy`: one writer publishes two records (`[1, 1]`, + `[2, 2]` over the initial `[0, 0]`) while one reader reads; every `Ok` is one whole record, and + a read after the writer is joined is `[2, 2]`. `Busy` is allowed. Passes under loom's default + bounds. +2. CI runs it: `CARGO_TARGET_DIR=target/ci/loom RUSTFLAGS='--cfg loom --check-cfg=cfg(loom)' + cargo test --locked --release -p engine --lib seqlock_loom`, and the `spsc_loom` leg still + passes. +3. Commands: `cargo fmt --all`; `cargo clippy --locked -q -p engine --all-targets --all-features + -- -D warnings`; `cargo test --locked -q -p engine --features engine/realtime-audit --lib`; + `cargo run --locked --release -q -p realtime-policy` (region count unchanged); + `RUSTDOCFLAGS='-D warnings' cargo doc --locked --workspace --no-deps`; + `python3 -B scripts/check-ci-path-routing.py`, `check-test-support-ci.py`, + `check-script-reachability.py`. + +## Test value + +Gate 1 turns red if the reader's `Acquire` fence, the reader's second sequence compare, or the +writer's `Release` fence is removed or weakened: each lets a reader on a weakly ordered target +(AArch64) return words of two publications as one record, which the x86 stress test cannot see. + +## Dependencies + +- #1316 (the seqlock module, B15-R1). + +## Decision record: TBD44 (batch 15c) + +**Shipped.** + +- `crates/engine/src/realtime/spsc.rs`: the `sync` module is `pub(super)`; `Arc`, `AtomicU64`, + `Ordering` and the new `fence` item are `pub(in crate::realtime)`, in both the `core`/`std` and + the `loom` variants. Everything else stays `pub(super)`. +- `crates/engine/src/realtime/seqlock.rs`: imports `Arc`, `AtomicU64`, `Ordering`, `fence` from the + shim (`core::sync::atomic::AtomicUsize` stays direct: it only sizes the retained-bytes mirror). + New private `READ_ATTEMPTS` (2 under loom, `MAXIMUM_READ_ATTEMPTS` otherwise, D4) bounds `read`. + New `loom_tests::seqlock_loom_a_read_is_one_whole_record_or_busy`. No ordering changed (D2): loom + found no defect. The publish region is still one marked region; `realtime-policy` reports 151 + regions in 33 files, as before. +- `.github/workflows/qualification.yml`: new step "Realtime race model (engine seqlock loom leg)" + after the SPSC leg, same `CARGO_TARGET_DIR` and `RUSTFLAGS`, filter `seqlock_loom` (D5). + +**Model size.** A first model with two reads per run and the full 64-attempt bound did not finish +in 9 minutes; two reads with the 2-attempt bound took 302 s. The shipped model (one read, 2 +attempts) explores in 4.3 s (about 28 s wall with the incremental build on the shared loom target +dir). The SPSC leg is unchanged: 15 models, 55 s. + +**Red evidence** (scratch copy of the workspace outside the tracked tree, same loom command): + +| Mutation | Result | +| --- | --- | +| Reader's `fence(Ordering::Acquire)` removed | FAILED: `a read returned a torn record [1, 2]` | +| Reader's second sequence compare removed (return after the word loads) | FAILED: `a read returned a torn record [0, 1]` | +| Writer's `fence(Ordering::Release)` removed | FAILED: `a read returned a torn record [1, 2]` | + +**Checks.** fmt, clippy (`-D warnings`), the engine lib tests with `realtime-audit` (91 passed), +both loom legs, `realtime-policy` (ok, 151 regions in 33 files), `cargo doc` with `-D warnings`, +and the three CI scripts all pass. + +**Known limit.** `cargo test` with a filter that matches no test passes; a rename of the model +that drops the `seqlock_loom` prefix would leave the CI step green and empty. The name carries the +prefix for that reason. diff --git a/.github/workflows/qualification.yml b/.github/workflows/qualification.yml index 3377b0cd9..c1547e6df 100644 --- a/.github/workflows/qualification.yml +++ b/.github/workflows/qualification.yml @@ -738,6 +738,10 @@ jobs: # invalidate the shared release target dir's fingerprints for every step after it. - name: Realtime race model (engine SPSC loom leg) run: CARGO_TARGET_DIR=target/ci/loom RUSTFLAGS='--cfg loom --check-cfg=cfg(loom)' cargo test --locked --release -p engine --lib spsc_loom + # TBD44: the seqlock behind the applied-revision watermark and the seek report, on the same + # target dir and flags as the SPSC leg, so it reuses that leg's build. + - name: Realtime race model (engine seqlock loom leg) + run: CARGO_TARGET_DIR=target/ci/loom RUSTFLAGS='--cfg loom --check-cfg=cfg(loom)' cargo test --locked --release -p engine --lib seqlock_loom # Issue #1609. host-core depends on `lane`, which refuses to compile without the pinned # x86-64-v3 features, and a RUSTFLAGS value replaces `.cargo/config.toml`'s rustflags, so this # leg names them. The `-C target-feature=+avx2,+fma` below is a copy of `.cargo/config.toml`'s diff --git a/crates/engine/src/realtime/seqlock.rs b/crates/engine/src/realtime/seqlock.rs index 3e01e151a..d31f7ec90 100644 --- a/crates/engine/src/realtime/seqlock.rs +++ b/crates/engine/src/realtime/seqlock.rs @@ -19,14 +19,23 @@ //! loads, retries, at most [`MAXIMUM_READ_ATTEMPTS`] times. There is no `unsafe` here: this module //! is inside the realtime root that `tools/realtime-policy` holds to its approved-unsafe list. -use core::sync::atomic::{AtomicU64, AtomicUsize, Ordering, fence}; -use std::sync::Arc; +use core::sync::atomic::AtomicUsize; + +// The ring's shim: core's atomics and `std`'s `Arc` in every shipped build, loom's under +// `--cfg loom`, so `seqlock_loom` explores this protocol itself (TBD44). +use super::spsc::sync::{Arc, AtomicU64, Ordering, fence}; /// The bound on one read's attempts. It keeps a read provably finite while a writer publishing in /// a tight loop still lets the reader through. A production writer publishes at most once per /// render block, so a read retries only when it lands inside one record store. pub const MAXIMUM_READ_ATTEMPTS: usize = 64; +/// The attempts one read takes. Under `--cfg loom` it is two, so `seqlock_loom` can explore every +/// interleaving of a read that retries once: each attempt's no-tear argument is independent of the +/// bound, and 64 spinning attempts are beyond loom's reach (TBD44). Every shipped build takes +/// [`MAXIMUM_READ_ATTEMPTS`]. +const READ_ATTEMPTS: usize = if cfg!(loom) { 2 } else { MAXIMUM_READ_ATTEMPTS }; + /// A read gave up after [`MAXIMUM_READ_ATTEMPTS`] torn attempts; the caller retries later. #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct PublishedWordsBusy; @@ -106,7 +115,7 @@ impl PublishedWordsReader { /// even counter reads, so what comes back is exactly one publication. pub fn read(&self) -> Result<[u64; N], PublishedWordsBusy> { let record = &*self.record; - for _ in 0..MAXIMUM_READ_ATTEMPTS { + for _ in 0..READ_ATTEMPTS { let before = record.sequence_lock.load(Ordering::Acquire); if !before.is_multiple_of(2) { continue; @@ -200,3 +209,35 @@ mod tests { assert!(newest <= PUBLICATIONS); } } + +#[cfg(all(test, loom))] +mod loom_tests { + use super::published_words; + + /// Words that are all equal within one publication and distinct across publications, so any + /// mixture of two records is detectable. + const RECORDS: [[u64; 2]; 3] = [[0, 0], [1, 1], [2, 2]]; + + /// TBD44: the real seqlock on loom's atomics and fences. One writer publishes two records + /// while one reader reads; under every interleaving loom explores (including the stale loads + /// its weak memory model allows), an `Ok` is one whole record -- the initial one or a + /// published one. `Busy` is allowed. After the writer is joined, a read is the last record. + #[test] + fn seqlock_loom_a_read_is_one_whole_record_or_busy() { + loom::model(|| { + let (mut writer, reader) = published_words(RECORDS[0]); + let publisher = loom::thread::spawn(move || { + writer.publish(&RECORDS[1]); + writer.publish(&RECORDS[2]); + }); + if let Ok(observed) = reader.read() { + assert!( + RECORDS.contains(&observed), + "a read returned a torn record {observed:?}" + ); + } + publisher.join().expect("writer"); + assert_eq!(reader.read(), Ok(RECORDS[2])); + }); + } +} diff --git a/crates/engine/src/realtime/spsc.rs b/crates/engine/src/realtime/spsc.rs index 61fa9720a..9783414cc 100644 --- a/crates/engine/src/realtime/spsc.rs +++ b/crates/engine/src/realtime/spsc.rs @@ -14,13 +14,16 @@ use sync::{Arc, AtomicU8, AtomicU32, AtomicU64, AtomicUsize, Ordering, UnsafeCel /// /// Under `--cfg loom` the real ring and the real plan mailbox are instantiated on loom's `Arc`, /// `AtomicUsize`, `AtomicU64` and `UnsafeCell`, so `spsc_loom` explores *this* code rather than a hand-written model of it -/// (#84 phase B). `cfg(loom)` is only a supported configuration for `cargo test`; a non-test -/// loom build is not a shape this crate promises to link. +/// (#84 phase B). The engine's seqlock (`seqlock.rs`) takes its `Arc`, `AtomicU64`, `Ordering` +/// and `fence` from here too, so `seqlock_loom` explores the real seqlock (TBD44). +/// `cfg(loom)` is only a supported configuration for `cargo test`; a non-test loom build is not a +/// shape this crate promises to link. #[cfg(not(loom))] -mod sync { +pub(super) mod sync { pub(super) use core::cell::UnsafeCell; - pub(super) use core::sync::atomic::{AtomicU8, AtomicU32, AtomicU64, AtomicUsize, Ordering}; - pub(super) use std::sync::Arc; + pub(super) use core::sync::atomic::{AtomicU8, AtomicU32, AtomicUsize}; + pub(in crate::realtime) use core::sync::atomic::{AtomicU64, Ordering, fence}; + pub(in crate::realtime) use std::sync::Arc; /// Exclusive access to a slot's storage, spelled the way loom's `UnsafeCell` spells it. #[inline(always)] @@ -41,10 +44,11 @@ mod sync { } } #[cfg(loom)] -mod sync { +pub(super) mod sync { pub(super) use loom::cell::UnsafeCell; - pub(super) use loom::sync::Arc; - pub(super) use loom::sync::atomic::{AtomicU8, AtomicU32, AtomicU64, AtomicUsize, Ordering}; + pub(in crate::realtime) use loom::sync::Arc; + pub(super) use loom::sync::atomic::{AtomicU8, AtomicU32, AtomicUsize}; + pub(in crate::realtime) use loom::sync::atomic::{AtomicU64, Ordering, fence}; pub(super) fn with_mut(cell: &UnsafeCell, body: impl FnOnce(*mut T) -> R) -> R { cell.with_mut(body) From f0f10f8f114bbde29acee92f0225dddc06d6986e Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 06:57:53 +0000 Subject: [PATCH 08/10] Batch 15c TBD44 verification minors: rewrap two qualification-doc paragraphs and the policy comment; the clock-pair doc says a reader needing both would need them published as one seqlock record Co-Authored-By: Claude Opus 5.5 --- crates/engine/src/realtime/plan_exchange.rs | 3 ++- docs/C_ABI_V1_QUALIFICATION.md | 29 +++++++++++---------- tools/realtime-policy/src/policy.rs | 5 ++-- 3 files changed, 19 insertions(+), 18 deletions(-) diff --git a/crates/engine/src/realtime/plan_exchange.rs b/crates/engine/src/realtime/plan_exchange.rs index 5f0b463d9..3c7b9cb33 100644 --- a/crates/engine/src/realtime/plan_exchange.rs +++ b/crates/engine/src/realtime/plan_exchange.rs @@ -93,7 +93,8 @@ struct SharedCounterAllocation { /// render sample, and that sample plus the plan's source-read offset. Each is one word, stored /// `Release` by render and loaded `Acquire` by the control thread, so neither read can tear. /// The two loads are not a snapshot: render stores the source-read clock and then the render -/// clock, so a reader that needs both together must take them through `seqlock.rs`. +/// clock, so a reader that needs both together would need them published as one record through +/// `seqlock.rs`; no reader needs that today. struct PlanClocks { render: AtomicU64, source_read: AtomicU64, diff --git a/docs/C_ABI_V1_QUALIFICATION.md b/docs/C_ABI_V1_QUALIFICATION.md index 754006672..d62541d1f 100644 --- a/docs/C_ABI_V1_QUALIFICATION.md +++ b/docs/C_ABI_V1_QUALIFICATION.md @@ -230,15 +230,15 @@ is a later change and amends the "equal to the render clock in this version" sen ABI growth (#1317, decision 15 D15-3). `MISO_ENGINE_V1_ABI_VERSION` changes only on a break. Each added exported symbol, and each former reserved struct word given a name, gets its own -`MISO_ENGINE_V1_FEATURE_*` bit, the next free one when it merges. A new value of an existing enum -or result code, and a field added in place to a V1 protocol message, takes no bit, so a host -tolerates an enum value it does not know and treats a result code it does not know as a failure. A -feature added as a protocol message is announced by its protocol capability flag in the -`CAPABILITIES` reply, not by a feature bit; the control protocol registry, not this rule, governs -protocol flags. A host tests each bit it needs with `&`, tolerates bits it does not know and never -compares `feature_mask` with `==`. A feature bit protects only a host that resolves the symbol at -run time: a host that links a symbol directly fails to load against a library that lacks it, before -it can test anything. The library's own Rust tests compare the reported mask with the library's `FEATURE_MASK` +`MISO_ENGINE_V1_FEATURE_*` bit, the next free one when it merges. A new value of an existing enum or +result code, and a field added in place to a V1 protocol message, takes no bit, so a host tolerates +an enum value it does not know and treats a result code it does not know as a failure. A feature +added as a protocol message is announced by its protocol capability flag in the `CAPABILITIES` +reply, not by a feature bit; the control protocol registry, not this rule, governs protocol flags. A +host tests each bit it needs with `&`, tolerates bits it does not know and never compares +`feature_mask` with `==`. A feature bit protects only a host that resolves the symbol at run time: a +host that links a symbol directly fails to load against a library that lacks it, before it can test +anything. The library's own Rust tests compare the reported mask with the library's `FEATURE_MASK` (`version_and_capabilities_are_exact` and the constant pins in `abi.rs`); the C smoke test does not compare the reported mask. `abi_smoke.c` tests one `&` per `MISO_ENGINE_V1_FEATURE_*` symbol (it fails when the library stops reporting a defined bit and passes when it reports an extra unknown @@ -988,13 +988,14 @@ that window is admitted against the plan that renders. exported symbol, announced by `MISO_ENGINE_V1_FEATURE_DECLARE_DISCONTINUITY` (the next free bit, 256) in the capability report, whose feature mask grows to include it; no struct, size or `ABI_VERSION` changes. The mask is the OR of every feature symbol and is never compared with a - literal, except the C smoke test's header-constant pins (`capi` `abi::tests::masks_and_result_codes_are_frozen` folds the symbols; `abi_smoke.c` - and `header_smoke.cpp` pin the bit inside the mask and the signature, and `abi_smoke.c` calls - the symbol with a null session for `INVALID_ARGUMENT`). The frozen exported set is now 18 + literal, except the C smoke test's header-constant pins (`capi` + `abi::tests::masks_and_result_codes_are_frozen` folds the symbols; `abi_smoke.c` and + `header_smoke.cpp` pin the bit inside the mask and the signature, and `abi_smoke.c` calls the + symbol with a null session for `INVALID_ARGUMENT`). The frozen exported set is now 18 `miso_engine_v1_*` definitions. It is a session call, serialized with the others; it runs the service step first. The browser's `miso_engine_web_v1_declare_discontinuity(handle)` forwards to - the same control-plane function, `control_plane::SessionState::declare_discontinuity` (root - ruling B15-R7); the browser SDK surface is #1296's. + the same control-plane function, `control_plane::SessionState::declare_discontinuity` (root ruling + B15-R7); the browser SDK surface is #1296's. - **Duty.** A host calls it at a stop, and before a seek of every source: the next block it renders need not continue the previous output. - **Behavior.** If the newest plan (the waiting replacement, else the running plan) runs behind its diff --git a/tools/realtime-policy/src/policy.rs b/tools/realtime-policy/src/policy.rs index 1854d21cc..4054afcdb 100644 --- a/tools/realtime-policy/src/policy.rs +++ b/tools/realtime-policy/src/policy.rs @@ -223,9 +223,8 @@ impl Policy { // `process_with`) and #1639's two (`crates/graph`: the jump-length cell's word decode // and its block-entry read) and #1630's one (`crates/builtins-compiler`: the cell // readers' carry marks, D15-2) and #1316's (`seqlock.rs`, the engine's one seqlock - // writer, B15-R1). The floors had fallen - // behind the tree before #1637; they are set to its counts again. A change that adds a - // marker raises them in the same change. + // writer, B15-R1). The floors had fallen behind the tree before #1637; they are set to + // its counts again. A change that adds a marker raises them in the same change. marked_files_floor: 33, marked_regions_floor: 151, render_thread_exports: table::parse(RENDER_THREAD_EXPORTS) From c48cc38755a8a32b57f6ba56fb87cdc1e64e0626 Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 11:01:33 +0000 Subject: [PATCH 09/10] Repin the browser render closure's enter_block site under its new impl index (Ms6 -> Ms7, #1396 added impl PlanClocks above it; count unchanged; owner-approved) Co-Authored-By: Claude Opus 5.5 --- scripts/check-web-audioworklet-callgraph.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/check-web-audioworklet-callgraph.py b/scripts/check-web-audioworklet-callgraph.py index 40c272804..9d31e929e 100644 --- a/scripts/check-web-audioworklet-callgraph.py +++ b/scripts/check-web-audioworklet-callgraph.py @@ -188,8 +188,10 @@ ), # Issue #1381a D4: the browser renders through the plan owner, so the block boundary's # swap decision is in the closure. The plan's own `render` entry is not: the owner calls - # `render_inner` and counts a failed block itself. - "_RNvMs6_NtNtCs_6engine8realtime13plan_exchangeNtB5_17RealtimePlanOwner11enter_block": ( + # `render_inner` and counts a failed block itself. The `Ms7` impl index moved from `Ms6` + # when #1396 added `impl PlanClocks` above `impl RealtimePlanOwner`; the sites and their + # count are unchanged (owner-approved pin edit, 2026-10-16). + "_RNvMs7_NtNtCs_6engine8realtime13plan_exchangeNtB5_17RealtimePlanOwner11enter_block": ( 4, "`running.prime_ready(...)`, the running plan's readiness hook, asked for a `Primed` " "candidate only (#1311), and `carry_from`'s executor hand-over at an adopted swap " From 35d016580186a315ada0bbe88f2fb5083c78503c Mon Sep 17 00:00:00 2001 From: BL Date: Fri, 16 Oct 2026 11:40:30 +0000 Subject: [PATCH 10/10] Number the seqlock loom model: #1702 Co-Authored-By: Claude Opus 5.5 --- ...-loom.md => 1702-model-the-engine-seqlock-under-loom.md} | 4 ++-- .github/workflows/qualification.yml | 2 +- crates/engine/src/realtime/seqlock.rs | 6 +++--- crates/engine/src/realtime/spsc.rs | 2 +- 4 files changed, 7 insertions(+), 7 deletions(-) rename .github/ISSUE_SPECS/{TBD44-model-the-engine-seqlock-under-loom.md => 1702-model-the-engine-seqlock-under-loom.md} (99%) diff --git a/.github/ISSUE_SPECS/TBD44-model-the-engine-seqlock-under-loom.md b/.github/ISSUE_SPECS/1702-model-the-engine-seqlock-under-loom.md similarity index 99% rename from .github/ISSUE_SPECS/TBD44-model-the-engine-seqlock-under-loom.md rename to .github/ISSUE_SPECS/1702-model-the-engine-seqlock-under-loom.md index 8e0fdf6c3..5a82645dd 100644 --- a/.github/ISSUE_SPECS/TBD44-model-the-engine-seqlock-under-loom.md +++ b/.github/ISSUE_SPECS/1702-model-the-engine-seqlock-under-loom.md @@ -2,7 +2,7 @@ Root ruling of 2026-10-16, inside batch 15c. Follows #1316 (B15-R1). Feature sprint mode (owner, 2026-10-13): one implementer, normal tests, a batch review. Issue number: assigned at filing -(written as TBD44). +(written as #1702). ## Product outcome @@ -74,7 +74,7 @@ writer's `Release` fence is removed or weakened: each lets a reader on a weakly - #1316 (the seqlock module, B15-R1). -## Decision record: TBD44 (batch 15c) +## Decision record: #1702 (batch 15c) **Shipped.** diff --git a/.github/workflows/qualification.yml b/.github/workflows/qualification.yml index c1547e6df..2c0dfe3a9 100644 --- a/.github/workflows/qualification.yml +++ b/.github/workflows/qualification.yml @@ -738,7 +738,7 @@ jobs: # invalidate the shared release target dir's fingerprints for every step after it. - name: Realtime race model (engine SPSC loom leg) run: CARGO_TARGET_DIR=target/ci/loom RUSTFLAGS='--cfg loom --check-cfg=cfg(loom)' cargo test --locked --release -p engine --lib spsc_loom - # TBD44: the seqlock behind the applied-revision watermark and the seek report, on the same + # #1702: the seqlock behind the applied-revision watermark and the seek report, on the same # target dir and flags as the SPSC leg, so it reuses that leg's build. - name: Realtime race model (engine seqlock loom leg) run: CARGO_TARGET_DIR=target/ci/loom RUSTFLAGS='--cfg loom --check-cfg=cfg(loom)' cargo test --locked --release -p engine --lib seqlock_loom diff --git a/crates/engine/src/realtime/seqlock.rs b/crates/engine/src/realtime/seqlock.rs index d31f7ec90..ab3b884fe 100644 --- a/crates/engine/src/realtime/seqlock.rs +++ b/crates/engine/src/realtime/seqlock.rs @@ -22,7 +22,7 @@ use core::sync::atomic::AtomicUsize; // The ring's shim: core's atomics and `std`'s `Arc` in every shipped build, loom's under -// `--cfg loom`, so `seqlock_loom` explores this protocol itself (TBD44). +// `--cfg loom`, so `seqlock_loom` explores this protocol itself (#1702). use super::spsc::sync::{Arc, AtomicU64, Ordering, fence}; /// The bound on one read's attempts. It keeps a read provably finite while a writer publishing in @@ -32,7 +32,7 @@ pub const MAXIMUM_READ_ATTEMPTS: usize = 64; /// The attempts one read takes. Under `--cfg loom` it is two, so `seqlock_loom` can explore every /// interleaving of a read that retries once: each attempt's no-tear argument is independent of the -/// bound, and 64 spinning attempts are beyond loom's reach (TBD44). Every shipped build takes +/// bound, and 64 spinning attempts are beyond loom's reach (#1702). Every shipped build takes /// [`MAXIMUM_READ_ATTEMPTS`]. const READ_ATTEMPTS: usize = if cfg!(loom) { 2 } else { MAXIMUM_READ_ATTEMPTS }; @@ -218,7 +218,7 @@ mod loom_tests { /// mixture of two records is detectable. const RECORDS: [[u64; 2]; 3] = [[0, 0], [1, 1], [2, 2]]; - /// TBD44: the real seqlock on loom's atomics and fences. One writer publishes two records + /// #1702: the real seqlock on loom's atomics and fences. One writer publishes two records /// while one reader reads; under every interleaving loom explores (including the stale loads /// its weak memory model allows), an `Ok` is one whole record -- the initial one or a /// published one. `Busy` is allowed. After the writer is joined, a read is the last record. diff --git a/crates/engine/src/realtime/spsc.rs b/crates/engine/src/realtime/spsc.rs index 9783414cc..0ff69130f 100644 --- a/crates/engine/src/realtime/spsc.rs +++ b/crates/engine/src/realtime/spsc.rs @@ -15,7 +15,7 @@ use sync::{Arc, AtomicU8, AtomicU32, AtomicU64, AtomicUsize, Ordering, UnsafeCel /// Under `--cfg loom` the real ring and the real plan mailbox are instantiated on loom's `Arc`, /// `AtomicUsize`, `AtomicU64` and `UnsafeCell`, so `spsc_loom` explores *this* code rather than a hand-written model of it /// (#84 phase B). The engine's seqlock (`seqlock.rs`) takes its `Arc`, `AtomicU64`, `Ordering` -/// and `fence` from here too, so `seqlock_loom` explores the real seqlock (TBD44). +/// and `fence` from here too, so `seqlock_loom` explores the real seqlock (#1702). /// `cfg(loom)` is only a supported configuration for `cargo test`; a non-test loom build is not a /// shape this crate promises to link. #[cfg(not(loom))]