diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 9a79326c6..774082206 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -710,3 +710,4 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1881](https://github.com/mudler/vllm.cpp/issues/1881) | `LTX25-DIT-ATTN-FLASH` | **The LTX-2.5 pixel A/B recorded a 75 KB LAUNCHER as its binary identity, and two builds a whole release window apart printed the SAME value while the 92 MB library that holds every measured kernel differed by 6,372,624 bytes.** Observed live on `dgx:gpu0` 2026-08-24 while the FA-2 ladder of [#1855](https://github.com/mudler/vllm.cpp/issues/1855) was building. `BINSHA=$(sha256sum "$BIN/ltx2-gen")` hashes 75,344 bytes of `main()`; `vt::Attention`, `vt::AttentionDenseFlash`, `vt::AttentionDenseFa2`, both VAEs and the loader are all in `libvllm.so.0.0.3`, which was hashed NOWHERE. Run `1612-r3` (source `3e2961ef0`, `libvllm` 85,703,328 B) and run `1853-fa2-r1` (source `62cbae10d`, `libvllm` `f046e75dcede2586...`, 92,075,952 B) both recorded `binary_sha256=834cec557c16cf77...`, each `binary_built=in-lease` with `BUILD_RC=0`. The launcher's own translation unit did not change, so its output is reproducible BY CONSTRUCTION - the one artefact whose hash was stable was the one containing none of the code under measurement. Consequence: §10.7's "the binary `834cec55...`", which #1743 and #1855 rest on, does NOT pin the code that produced them, and a later reader reads the same string and concludes the same code ran. **NOT an invalidation of 1612-r3**: its four arms ran from one build in one lease and each proved its own op from its own log; what its RECORD cannot do is tell its build from a later one. **FIXED IN FLOW for the pixel harness** - `LIBSHA` is computed, printed, written to `PROVENANCE` as `library_sha256` and added to every arm's `render.log` header, with three tripwires in `test_ltx25_pixel_ab_harness.py` that each red when their site is deleted. **STILL OWED**: `ltx25-dit-attn-flash-ab.sh` and `ltx25-dit-attn-fa2-hd128-ab.sh` carry the identical idiom, so every speed number they have recorded has the same hole. Owned by row `LTX25-DIT-ATTN-FLASH` and listed under `## Owed` | bug | | [#1886](https://github.com/mudler/vllm.cpp/issues/1886) | `LTX25-DIT-ATTN-FLASH` | **The FA-2 arm that every production render resolves reads `DIRECTIONAL` against `naive` on the audio, and the checked statistic ranks it BACKWARDS against `flash`.** Measured on `dgx:gpu0`, `rc` job `4dcdd916...`, `RUN_ID=1853-fa2-r1`, source `62cbae10d`, `libvllm.so.0.0.3` `f046e75dcede2586...` (92,075,952 B), `768x448/49f`, seed `20260820`, FOUR arms from ONE build in ONE lease. This is the render [#1855](https://github.com/mudler/vllm.cpp/issues/1855) said had never been taken: `ltx2_device.cpp:536` is the only reader of `VLLM_LTX2_DIT_FLASH_ATTN`, unset selects `vt::AttentionDenseFa2`, and nothing in `include/`, `examples/` or `tools/` sets it. **Every arm proved its own op and neither other, and the counts were re-derived from each arm's raw `render.log` rather than read off its `ARM` summary**: `fa2` op18=0 op21=0 op22=1 (2.223 s/forward), `naive` op18=1 op21=0 op22=0 (45.512 s), `fa2-ctl` op18=0 op21=0 op22=1 (2.256 s), `flash` op18=0 op21=1 op22=0 (6.360 s). **The control is a zero, exactly**: `fa2-ctl` is BIT-IDENTICAL to `fa2`, 49/49 frames and `audio.wav` byte-equal, `control/treatment = 0.000000` on luma against a treatment effect of 8.952578, so nothing here is run-to-run variation. **Correspondence passes in full** (frame margin **1.1928 > 1** at frame 29, 0 of 49 frames off `(0,0)`, audio lag **0**) and **the three video statistics are incoherent** (sharpness **0.020738**, blockiness **0.040532**, motion **0.061146**). **The audio fires**: `coherence.audio_rms` `K` **0.511574** over 376 windows, means 865.774 vs 892.84, the shipped arm **3.03% quieter**, `top10% = +0.952`. `READING DIRECTIONAL`, `VERDICT FAIL (exit 1)`. **The direction is ONE CHANNEL**: ch0 `K` **0.705886** (-6.40%), ch1 `K` **0.060690** (-0.29%) at **1.18x** its own 0.0516 floor, i.e. FA-2 does not touch channel 1. **This CORRECTS #1855's attribution rather than confirming it**: `flash` loses the SAME channel (ch0 0.756589), so the direction belongs to the reassociated f32 online-softmax order both kernels share -- by TWO DIFFERENT reassociations, see `vt::AttentionDenseFast`, `vt::AttentionDenseFlash` and `vt::AttentionDenseFa2` in `include/vt/ops.h` and not to the #1549 swap. §11.9 pre-registered exactly this test before the render existed. **THE DILUTION IS ARM-DEPENDENT AND FALLS HARDEST ON THE SHIPPED ARM, which needed all three pairs to see**: the checked mono term retains 89.1% of `flash`'s channel-0 direction (0.674002 of 0.756589) and only 72.5% of `fa2`'s (0.511574 of 0.705886), because `flash` moves both channels the same way while `fa2` leaves ch1 at -0.29% and the average dilutes its channel-0 loss against an effectively unmoved channel. That is what pushed `fa2` to within 2.3% of the constant while `flash` sits 35% above it. **A first draft of this row claimed an INVERSION -- that the checked statistic ranks the two arms the wrong way round -- and a fresh review falsified it before it was published**: on `K` against `K` there is no inversion, because ch0 `K` ranks `fa2` below `flash` (0.705886 vs 0.756589) and the mono term ranks them the same way. The apparent reversal only appeared when an AMPLITUDE measure (`fa2` loses 6.40% of ch0 against `flash`'s 5.40%) was set against a COHERENCE measure, which is the category error §11.3 exists to prevent. The claim is withdrawn. **`fa2` vs `flash` shows no direction anywhere** (audio `K` 0.177718, sharpness 0.001954, `SEPARATED, NOT DEGRADED`, exit 0), which locates the direction between both fast kernels and `naive`. **The `flash` vs `naive` row is NOT an independent second observation**: it reads every §11.8 figure to six decimals because its renders are BYTE-IDENTICAL to run `1612-r3` across `3e2961ef0` -> `62cbae10d` and a `libvllm` differing by 6,372,624 bytes, so #1855 is reproduced rather than replicated and the count of independent observations is still ONE. **TWO LIMITS**: the checked constant carries the mono verdict by only 2.3% (`0.511574` vs `0.5`), a PARTIAL direction that §11.3's two-population argument does not cover, so there `0.5` is a chosen bound and NOT repaired by moving it (§9); and 376 windows are not 376 independent observations because the 2.010 s track has one loud passage. **NOT ATTRIBUTED**: why a reassociated attention sum costs 6.4% of one audio channel while leaving the other at its floor and the picture directionless is unexplained. NOT FIXED IN FLOW: a finding about a change already on `main` whose mechanism needs its own investigation. Owned by row `LTX25-DIT-ATTN-FLASH` and listed under `## Owed`. Recorded in [`ltx25-dit-attn-flash.md`](specs/ltx25-dit-attn-flash.md) §12.6 | bug | | [#1887](https://github.com/mudler/vllm.cpp/issues/1887) | `LTX25-DIT-ATTN-FLASH` | **Three citations of `include/vt/ops.h:3304-3306` in [`ltx25-dit-attn-flash.md`](specs/ltx25-dit-attn-flash.md) were stale and asserted the OPPOSITE of the sentence they supported.** Found by the fresh review of [#1871](https://github.com/mudler/vllm.cpp/pull/1871) while that change was adding a FOURTH citation of the same anchor. At `def85d285` those lines sit inside the `AttentionRelPos` doc comment and read "Reductions are strictly sequential per output element => thread-count independent and byte-reproducible", while the three sites (§4 numerics, §10.2, and §11.3 -- the ratified criterion section the whole pixel lane rests on) cite them for the claim that `vt::Attention` and `vt::AttentionDenseFlash` "differ only in association". Correct at `90e8c3c85`/`ff8f72807`, stale by `c4ba829a3`. **Correct anchors, each verified unique by phrase**: `ops.h:3315-3316` (`Fast` NOT bit-identical to `Attention`, different head_dim partial-sum grouping), `ops.h:3328-3329` (`Flash` order UNCHANGED from `Fast`, bit-identical), `ops.h:3381-3382` (`Fa2` NOT bit-identical to `Fast`/`Flash`, `mma.sync` reassociates QK^T and PV). **A second defect the same reading exposed**: the anchor was being used to claim `flash` and `fa2` share ONE reassociated order, and they do not -- they are TWO DIFFERENT reassociations, so the supportable claim is a shared CLASS, which is what §12.6 measures. FIXED IN FLOW in #1871: all three pre-existing citations repaired and the fourth corrected before publication. Owned by row `LTX25-DIT-ATTN-FLASH` | bug | +| [#1884](https://github.com/mudler/vllm.cpp/issues/1884) | `LTX25-PHASE-RESIDUE` | **`CheckSeamAnchor` proves a seam anchor's POSITION and not its MAGNITUDE, so a zero-width anchor passes it green.** Filed by the change that introduced the gate ([#1668](https://github.com/mudler/vllm.cpp/issues/1668)), against its own work, and MEASURED: mutation M7 -- `artifacts.mux` closed immediately after it opens, dropping the late `Close()`, so the render's whole tail is un-named again -- passes **796 of 796 assertions, Status: SUCCESS**. Every clause survives a zero-width window: the count is 1, it is neither `span` nor `nested`, it still opens after `artifacts.audio` ends, nothing follows it so it is still the last leaf, and the containment clause holds VACUOUSLY because a zero-width window has no interior, so every other leaf ends at or before it or starts at or after it. The same argument applies to `load.dit_config`. **Third appearance of one shape**, after the `decode.audio.mel` partial transfer the file already records ("an anchor covering half its call passes 0.50 and is not detected here. Closing that needs a scope INSIDE the callee") and [#1568](https://github.com/mudler/vllm.cpp/issues/1568) one level down. **MUST NOT be closed by a share floor whose denominator is the INSTRUMENT or the wall**: a seam's honest share is a property of the box, which makes that floor `residue <= 2 * instrument` with a different name, and `## Design` 3 of the spec is the measured record of why that does not work -- 4 red in 45 at the table bound, 28 in 160 on a standalone probe, and a 20-run sample that saw none of its tail. **The one shape that prohibition does NOT cover, and where a next attempt should start**: the anchor's own extent against the SEAM its two neighbours define, both measured in the same run -- near 1 honest, near 0 collapsed, with the region being named as the denominator instead of the instrument, so it never asks how many seconds a seam SHOULD hold. It still needs a constant and a cross-box distribution, so it is a candidate and not a conclusion. Otherwise: an anchor INSIDE the callee, or a bound on a quantity the scheduler cannot move ([#1570](https://github.com/mudler/vllm.cpp/issues/1570), [#1439](https://github.com/mudler/vllm.cpp/issues/1439)). NOT FIXED IN FLOW: each closure is its own row with its own red-first tests and fresh review. Listed under `## Owed` in [`ltx25-phase-residue.md`](specs/ltx25-phase-residue.md) | bug | diff --git a/.agents/specs/ltx25-phase-residue.md b/.agents/specs/ltx25-phase-residue.md index bbda08854..8b2621dd4 100644 --- a/.agents/specs/ltx25-phase-residue.md +++ b/.agents/specs/ltx25-phase-residue.md @@ -175,8 +175,10 @@ and close point rather than by name: `src/vllm/multimodal/ltx2_video.cpp:794` an `6b48edb2c` separately repaired the `denoise` red on `main` by moving the share floor to 0.75 and adding assertion (1c). Its own comment records that as a holding action -- in substance rather than in those words, which the tree does not -use -- and the comment is still in the tree at -`tests/vllm/multimodal/test_ltx2_video.cpp:4324-4331`: +use -- and the comment is still in the tree, above the `denoise` entry of the `carrying` +table in `CheckRenderPhases` (`tests/vllm/multimodal/test_ltx2_video.cpp`; it +sat at :4324-4331 when this was written and moves with the file, which is why +the range is no longer quoted): > NAMING THE UN-NAMED TIME WOULD SETTLE IT PROPERLY, which is what #1439 asks for > first. A `denoise.update` scope over the sampler's per-step update would put the @@ -219,23 +221,205 @@ replacement does not work. What would close it is a bound on a quantity the scheduler cannot move, which is [#1570](https://github.com/mudler/vllm.cpp/issues/1570). -## Owed +## What landed, and what it is gated by + +**The three anchors and the counter land here.** #1668, on base `1724be38e`. +Written in the present tense of the merge that carries this file: while the pull +request is open they are on `row/LTX25-PHASE-RESIDUE-1668` and not on `main`. +The four items this row measured are now in two states rather than one, so the +earlier "nothing in this table is implemented" line above `## Owed` is retired +rather than edited: it was true when it was written and it is not +true now. + +| item | state | +|---|---| +| the load prologue | landed as `load.open` by `519303d15` (#1622), under `LTX25-DEVICE-RESIDENCY` | +| `Record::instrument_seconds` | landed by `be432e8e3` (#1711), under `LTX25-PHASE-INSTRUMENT`, with its docs | +| `load.dit_config` | **landed here** | +| `artifacts.mux` | **landed here** | +| `denoise.update` + `Ltx2ConditioningTrace::sampler_updates` | **landed here**, first-order arm only | + +### The gate is STRUCTURAL, and that is the whole point + +`## Design` 3 withdrew `residue <= 2 * instrument` because a seam's honest share +is a property of the box, and three fresh reviews measured its tail. Nothing +here re-proposes it, and **neither existing wall-clock ratio moved**. The two +seam anchors are held by `CheckSeamAnchor`, which carries **no constant at all**. +Its clauses are positions: + +- the anchor is emitted exactly once, is not a span, and is not `nested`; +- it opens at or after its predecessor's end, and closes at or before its + successor's start; +- no other leaf lies inside the window it claims. + +`artifacts.mux` has no successor — it closes one statement before +`WritePhaseLog` reads the clock — so its far-side clause is replaced by the +stronger one that nothing follows it: **it is the last leaf of its render**. + +Under a correct placement every clause holds by construction. Under a MISPLACED +one at least one fails outright, whatever the clock did — an anchor left open +across the next leaf marks that leaf `nested` and removes it from +`sum_leaf_seconds` altogether, and an anchor moved onto a neighbour is found +inside that neighbour's window. + +**This does NOT extend to an anchor that names NOTHING**, and an earlier draft of +this section said it did. A zero-width window satisfies every clause above, +because a window with no interior contains no leaf and a collapsed anchor still +opens after its predecessor and still precedes its successor. That is +[#1884](https://github.com/mudler/vllm.cpp/issues/1884), measured twice and +below. + +`denoise.update` is held by assertion (0), the record count, against +`Ltx2ConditioningTrace::sampler_updates`. That counter is maintained by the +RENDER, inside the scope it counts, for the reason `video_decode_chunks` is: a +denominator derived from the phase table cannot falsify a phase table. + +### `sampler_updates` is not a second name for `dit_evaluations` + +The two agree on the first-order arm — one update per evaluation — so that arm +cannot tell a real counter from an alias. The **res_2s arm can**, and it is in +the gate for that: `res2s_two_stage` runs its post-process and step inside +`Ltx2Res2sDenoisingLoop` behind `Ltx2Res2sHooks`, which the first-order +statement never reaches, so it counts 7 and 11 evaluations against **zero** +updates. Mutation M4 — the counter incremented beside `dit_evaluations` in the +shared `Evaluate` lambda, which is what an alias actually looks like — is green +on the first-order arm and red on that one. + +### Mutations, all six detected + +Run on the immutable head, each with its anchor asserted UNIQUE before it was +applied, its application verified by hash, its build's exit code checked, and +the tree restored and re-hashed byte-for-byte afterwards. The harness reported +its own defects first, which is why it exists: an early `M2` was rejected as +`ANCHOR-NOT-UNIQUE` (2 occurrences), an early `M4` as a semantic no-op that read +green, and an early `M5` as `BUILD-FAILED`, and a build failure would have left +the previous binary printing the previous green. + +| id | guarantee attacked | verdict | +|---|---|---| +| M1 | `load.dit_config` closes before `load.video_vae` opens | red, 4 assertions | +| M2 | `artifacts.mux` is the render's LAST leaf, not a window around the audio write | red, 4 assertions | +| M3 | `denoise.update` is emitted once per sampler update | red, 2 assertions | +| M4 | `sampler_updates` counts the sampler, not evaluations | red, 3 assertions | +| M5 | reachability: the `load.dit_config` production call site | red | +| M6 | reachability: the `artifacts.mux` production call site | red | +| **M7** | **the anchor still WRAPS the work: collapsed to zero-width. Filed on `artifacts.mux`; a fresh review then reproduced it on BOTH anchors at once** | **GREEN -- NOT detected, #1884** | + +### It is not load-flaky, which is the property it was built for + +The reason `## Design` 3 withdrew the instrument-relative bound is that its +value moves with the box, and this row's whole history is gates that red under +load. A structural gate should not have that failure mode, and this is the +measurement rather than the argument: **8 runs of 8 green at loadavg 25.98 to +37.92**, on one binary, while a second build was saturating the machine. Small +counts differ between runs (794 to 798 assertions) because the "seam holds +nothing else" clause iterates over the leaves the render actually emitted. + +That is a lower bound on stability and not a distribution. `## Design` 3 also +records why it cannot be more than that: a 20-run sample of a scheduler- +dependent quantity on this table did not see the tail that decided the gate. The +claim here is narrower and does not need one — these assertions read ORDER and +COUNT, and neither has a tail. + +M5 and M6 delete the production call site and leave a tree that COMPILES, which +is what `.agents/reachability.md` asks for: the gate has to be what notices. +Both anchors sit inside `Ltx2VideoEngine::Load` and `::Generate`, rather than in +a type a test constructs. Stated precisely, because an earlier draft was loose +about it: the case that reds on M5 and M6 enters through +`vllm::multimodal::LoadVideoEngine` and `VideoEngine::Generate` — the loader and +the production `Generate` it hands back. `.agents/reachability.md` names the +loader in those words; the `Generate` is the call the loader exists to reach. It is +NOT a C-ABI render, and a reader should not go looking for one. The C ABI +entry point `vllm_video_generate` is `src/capi/vllm_c.cpp:1623`, and it reaches +the same `Ltx2VideoEngine::Generate` at `:1668`; the server reaches it at +`src/vllm/entrypoints/openai/server_main.cpp:1616`. So the anchors are on the +shipped path by construction, and the separate `SUMS to wall` case is the one +that renders through `vllm_video_generate`. Both anchors were read out of the +file rather than recalled: an earlier draft of THIS sentence cited +`vllm_c.cpp:1667`, which is a blank line, and nothing in this tree gates a +citation. + +### M7 is GREEN, and it is the honest limit of a constant-free gate + +**A position is not a magnitude.** Close `artifacts.mux` immediately after +opening it and drop the late `Close()`, and all 796 assertions pass: the count +is 1, it is neither span nor nested, it still opens after `artifacts.audio`, it +is still the last leaf, and a zero-width window contains no other leaf +VACUOUSLY. The anchor reports ~0 s and the tail is back in +`unaccounted_seconds`, with the table green and a name on nothing. The same +argument applies to `load.dit_config`. + +That is [#1884](https://github.com/mudler/vllm.cpp/issues/1884), and it is +disclosed rather than closed because the obvious closure is the thing this row +already measured shut twice. + +**A fresh review found it independently and measured what it costs**, which is +the number this row should carry rather than the argument. Collapsing BOTH +anchors onto their own `Close()` — the exact shape of a refactor that hoists the +work out from under an anchor — is green on the focused case AND on the full +suite, 102 of 102 cases (4668 assertions on that run; the total moves a few +either way between runs, for the reason the counts above do), on a +line-count-preserving variant +so the reader-anchor ledger is undisturbed: + +| quantity, `attribution_multichunk/phase-log.json` | head | collapsed | +|---|---:|---:| +| `load.dit_config` duration | 0.000109239 s | **0.000016543 s** | +| gap `load.dit` -> `load.dit_config` | 0.000003296 s | **0.000141781 s** | +| gap `load.dit_config` -> `load.video_vae` | 0.000001042 s | **0.000087407 s** | +| gap `artifacts.audio` -> `artifacts.mux`, r1 / r2 | 0.000000983 / 0.000003287 s | **0.000036901 / 0.000066776 s** | + +So the collapse hands about **229 us back to the residue on a fixture whose +whole residue is about 1.2 ms** — roughly a fifth of the thing this row exists +to name — while the anchor that was supposed to name it measures 16.5 us, which +is instrument cost. The `(4b)` gap gate cannot see it either: `gap_count == +leaf_records + 1` and `gap_total == unaccounted` both still hold, because the +collapse only SPLITS one gap into two. + +**The next traceable hypothesis, named rather than left open.** The same review +proposes a discriminator whose denominator is NOT the instrument, which is what +separates it from the bound `## Design` 3 withdrew: the anchor's own extent +against the seam its two neighbours define, both measured in the same run — +`(end - start)` against `(next_start - previous_end)`. Honest, that ratio is +about 1 because a correctly placed anchor covers the seam except for two +boundaries; collapsed, it is about 0. That is a wide separation of the kind the +`decode.video.vae` floor already exploits, and it does not ask how many seconds +a seam SHOULD hold, which is the question that made the instrument-relative +bound a property of the box. It still needs a constant, so it is a candidate and +not a conclusion, and it belongs to #1884 with its own red-first measurement and +its own fresh review. Recorded here so the next attempt starts from it instead +of re-deriving it. + +What stays banned here is the shape `## Design` 3 withdrew: a floor whose +denominator is the INSTRUMENT, the wall, or a written-down number of seconds. +Each of those asks how many seconds a seam SHOULD hold, and the honest answer is +a property of the box. The seam-extent ratio above asks a different question -- +what fraction of THIS seam, in THIS run, did the anchor cover -- so the +prohibition does not reach it. An earlier draft of this paragraph banned "a +share floor here" without that qualification, which forbade the candidate the +paragraph had just proposed, and `## Stop conditions` points a reader at this +section as its authority. + +The hole itself is the same shape as the `decode.audio.mel` note in `test_ltx2_video` +("a partial transfer ... passes 0.50 and is not detected here. Closing that +needs a scope INSIDE the callee") and as #1568 one level down. Three +appearances, one repair, and the repair is an anchor inside the callee or a +bound on a quantity the scheduler cannot move -- which is #1570's open question +and, above it, #1439's. -Nothing in this table is implemented on `main`. Verified on `db648fb88` by a -tree-wide grep over `src/`, `include/`, `tests/` and `docs/`: `sampler_updates`, -`artifacts.mux` and `load.dit_config` return nothing, and `denoise.update` occurs -once, at `tests/vllm/multimodal/test_ltx2_video.cpp:4325`, asking for it. +## Owed | Issue | Owed | |---|---| -| [#1668](https://github.com/mudler/vllm.cpp/issues/1668) | **the three anchors and the instrument self-cost, as one implementable unit.** `load.dit_config`, `artifacts.mux`, `denoise.update` plus `Ltx2ConditioningTrace::sampler_updates`, and `Record::instrument_seconds` with its conservation invariant. The reference implementation, the gate report and the mutation table stay readable at `refs/pull/1556/head` = `b45ea3bbb` | -| [#1567](https://github.com/mudler/vllm.cpp/issues/1567) | the res_2s arm's `denoise.update` anchor. `Ltx2Res2sDenoisingLoop` runs its own post-process and step behind `Ltx2Res2sHooks`, so the anchor needs a hook rather than a statement. It lives in `ltx2_samplers.cpp`, is declared in `ltx2_samplers.h` beside the hooks struct, and is called from `ltx2_video.cpp`. **NOT `ltx2_res2s.cpp`**: #1556's spec named that file and it has never existed here, which `git log --all --diff-filter=A` confirms; #1567's forge text names no file at all, so the wrong anchor came from the spec rather than from the issue. No gate in this tree renders on that arm, so landing it beside the first-order arm would land dead code | -| [#1568](https://github.com/mudler/vllm.cpp/issues/1568) | the `denoise.step` / `denoise.update` seconds transfer. (1b') compares `start_seconds` only, so leaving `denoise.step` open across the post-process and emitting `denoise.update` empty after it preserves the alternation, both counters, containment, non-overlap, exclusivity, (1c) and (2), while moving 100% of the decomposed seconds onto one name. No (2b) floor separates it: the honest share of `denoise.update` runs 0.45% to 11.15% across four boxes and a transfer puts it at ~0%. Closing it needs an anchor INSIDE the callee | +| [#1668](https://github.com/mudler/vllm.cpp/issues/1668) | **LANDED.** `load.dit_config`, `artifacts.mux`, `denoise.update` and `Ltx2ConditioningTrace::sampler_updates` land with this file; `Record::instrument_seconds` landed separately as `be432e8e3` (#1711) under `LTX25-PHASE-INSTRUMENT`. See `## What landed, and what it is gated by`. What this issue does NOT close, and did not claim to: the res_2s arm (#1567), the seconds-transfer gate (#1568), and the instrument-share bound (#1570). The earlier reference implementation stays readable at `refs/pull/1556/head` = `b45ea3bbb`; it was not reused, and the anchors here were written and gated fresh | +| [#1567](https://github.com/mudler/vllm.cpp/issues/1567) | **UNBLOCKED by #1668 and still open.** The name `denoise.update` now exists on the first-order arm, so this issue is no longer waiting on a name that nothing defines — it is the SECOND arm of an anchor that ships. `Ltx2ConditioningTrace::sampler_updates` reads ZERO on res_2s, and `test_ltx2_video` now asserts that zero, so the arm's absence is measured rather than assumed and any hook that lands has a counter to be checked against. What is still owed: the res_2s arm's `denoise.update` anchor. `Ltx2Res2sDenoisingLoop` runs its own post-process and step behind `Ltx2Res2sHooks`, so the anchor needs a hook rather than a statement. It lives in `ltx2_samplers.cpp`, is declared in `ltx2_samplers.h` beside the hooks struct, and is called from `ltx2_video.cpp`. **NOT `ltx2_res2s.cpp`**: #1556's spec named that file and it has never existed here, which `git log --all --diff-filter=A` confirms; #1567's forge text names no file at all, so the wrong anchor came from the spec rather than from the issue. No gate in this tree renders on that arm, so landing it beside the first-order arm would land dead code | +| [#1568](https://github.com/mudler/vllm.cpp/issues/1568) | **UNBLOCKED by #1668 and still open, and its measured reason is now IN THE TREE.** The transfer this issue describes was hypothetical while `denoise.update` did not exist; the name ships now and `denoise` is a multi-part leaf, so (1b) is no longer vacuous on it. **(2b) still is**, and an earlier draft of this row said otherwise: both of this leaf's floors are 0.0 and the (2b) loop skips a 0.0 floor, so it executes zero CHECKs here. `decode.audio` remains the only leaf (2b) asserts anything about. `part_min_coverage` for `denoise.update` is **0.0**, and the note beside it in `test_ltx2_video` carries the reason rather than a constant: the honest share runs 0.45% to 11.15% across four boxes and the transfer puts it at ~0%, so the two distributions overlap and any floor that reds the transfer also reds an honest render this row has produced. Not closed, and deliberately not closed by a number. What is still owed: the `denoise.step` / `denoise.update` seconds transfer. (1b') compares `start_seconds` only, so leaving `denoise.step` open across the post-process and emitting `denoise.update` empty after it preserves the alternation, both counters, containment, non-overlap, exclusivity, (1c) and (2), while moving 100% of the decomposed seconds onto one name. No (2b) floor separates it: the honest share of `denoise.update` runs 0.45% to 11.15% across four boxes and a transfer puts it at ~0%. Closing it needs an anchor INSIDE the callee | | [#1569](https://github.com/mudler/vllm.cpp/issues/1569) | **CLOSED by `LTX25-PHASE-INSTRUMENT`** ([`ltx25-phase-instrument.md`](ltx25-phase-instrument.md)), which gates it over an 8000-record table where the copy and the sort are a measurable event, against a discriminator measured in the same run rather than written down as a constant. On the tree that lands, restoring `main`'s clock order (`M1`) reds it 10 runs of 10 at ratios of 2.080 to 3.418, and the partial regression (`N11`) reds it 10 of 10 at 0.892 to 1.145, against a bound of 0.5 and an honest 45-run maximum of 0.027616 at loadavg 19-26 — 0.018123 in a higher 56-113 regime. The earlier "1.004 against 0.0042" belonged to the WITHDRAWN one-number `copy + sort` budget, which a fresh review broke and `### 6` replaced with `min(copy, sort)`. What it originally owed: a gate on `WriteJson`'s clock ORDERING, **measured green under its own mutation**. Restoring the old order left the conservation case GREEN 10 of 10, at `wall 0.0608987s, unaccounted 0.000534223s, table charge 0.000301655s`, because the copy and sort of a three-record table are nanoseconds. Gating it needs a table with enough records for the sort to be measurable | | [#1570](https://github.com/mudler/vllm.cpp/issues/1570) | an upper bound on the instrument's own share of a leaf. `uncovered <= 2 * leaf_instrument` is stricter than the floor it replaces only while `leaf_instrument` stays small, and nothing bounds it. Moving the DiT `Tick` out of `Evaluate` would charge ~110 flushed writes to `denoise` and widen the gate while printing a small number | | [#1571](https://github.com/mudler/vllm.cpp/issues/1571) | **CLOSED by `LTX25-PHASE-INSTRUMENT`** ([`ltx25-phase-instrument.md`](ltx25-phase-instrument.md)). `phase-log.json` carries `gaps`, and the gate over it is an accounting identity rather than a tolerance: the gaps add to `unaccounted_seconds` by construction. On the fixture render it immediately named the NEXT region, `load.dit` -> `load.video_vae` at 0.627 ms, which is the `load.dit_config` anchor #1668 owes. What it originally owed: a per-gap decomposition IN the emitted table. The 92% region above was found with a scratch script; a reader of `phase-log.json` still cannot see it without one, and the same investigation will be re-derived the next time the residue moves | | [#1572](https://github.com/mudler/vllm.cpp/issues/1572) | assertion (1c)'s span slack reds intermittently on `main` — `decode.video` at `0.00256913` against a `0.00075` bound, 3.4x. Pre-existing from `6b48edb2c` and not this row's. **STILL OPEN, and two further shapes of that bound are now MEASURED SHUT.** `LTX25-DEVICE-RESIDENCY` built a fifth shape (`4 x` the worst boundary a 1 kHz sampler saw across the whole case) and a sixth (`4 x` the worst inside the record's OWN window) and withdrew both: the fifth lets one 200 ms descheduling of the sampler thread turn a real 20 ms un-named phase from red 9 of 9 into a GREEN case, and the sixth reds an unmutated tree 10 times in 45 consecutive runs at loadavg 21.8-61.5 -- of which **5 in 45 is the defensible figure**, because a second mechanism was identified and repaired while that population was still running, so its reds were measured on a binary that predates its own repair. See `.agents/specs/ltx25-device-residency.md` `### The span-slack bound, FIFTH and SIXTH shapes` for both distributions and for the one hypothesis that has not been tried | | [#1619](https://github.com/mudler/vllm.cpp/issues/1619) | **the `merge=union` driver duplicates a row, MEASURED on this row's own merges.** Both sides appended before the same trailing anchor rather than at the true end, so the driver concatenated two regions that each carried `#1546` and the resolved index held it TWICE, byte-identical, at 538 lines where the correct union is 537. `git merge-tree` called that merge clean and `check-issue-index-append-only.py` passed it, because a duplicate is an ADDITION and that checker only collects removals. `check-agent-record.py` did NOT pass it -- a claim #1556's spec made and this row REFUTED by reproduction: regenerating the raw driver output and running that same tree's checker returns rc=1 with `issue #1546 listed twice`, and the refusal has existed since `8dd6508da` (2026-08-09), before the merge. So the blind gate is exactly one checker, not two, and the gap is narrower than #1556 recorded. The de-duplication half is CONDITIONAL, and the condition is what #1556's spec omitted: the checker reds a repair only when the DUPLICATE IS ALREADY IN THE BASE. Measured at three pairings -- `--base e2a9e035d` against the real canonical 537-line file rc=0, against a synthetic 537 rc=0, and `--base --head <537 de-dup>` rc=1. It diffs `merge-base..HEAD`, so when the base predates the duplicate the addition and the removal CANCEL and it passes. Since `origin/main` is preflight's base, and is the shape this branch used, the gate does NOT red someone who repairs driver output before committing it -- only someone repairing a corruption that already landed. The same range property is why relocating a base-reachable row DOES red it: moving row `#168` to the end gives rc=1 and a `removed:` line naming it. So "de-duplicating in place FAILS the checker", as #1556's spec put it, is false unqualified and true once the duplicate is base-reachable. #1556's spec added that the same driver dropped `#838` on a later re-merge, making this a recurring class; that is WITHDRAWN as unreproducible. Re-running `git merge-file --union` at every later merge where `#838` was on a side leaves it present in all of them, and `git log -S` finds it absent from no committed state -- mechanically a union driver cannot drop a line that is an addition on one side. If it ever went missing, that points at a wholesale take-ours resolution rather than at the driver | +| [#1884](https://github.com/mudler/vllm.cpp/issues/1884) | **`CheckSeamAnchor` proves POSITION, not MAGNITUDE.** Filed by the change that wrote the gate, against its own work, and measured: mutation M7 -- `artifacts.mux` closed immediately after it opens, the whole tail un-named again -- passes 796 of 796 assertions. Every clause survives a zero-width window, the containment clause vacuously. Third appearance of one shape, after the `decode.audio.mel` partial transfer and #1568. **Must NOT be closed by a share floor against the INSTRUMENT or the wall**: that is `residue <= 2 * instrument` with a different name, and `## Design` 3 is the measured record of why it does not work. **The one shape that prohibition does NOT cover, and where to start**: the anchor's own extent against the SEAM its two neighbours define, both measured in the same run -- near 1 honest, near 0 collapsed, with the region being named as the denominator instead of the instrument. It still needs a constant and a cross-box distribution. Otherwise: an anchor INSIDE the callee, or a bound on a quantity the scheduler cannot move (#1570, #1439) | | [#1439](https://github.com/mudler/vllm.cpp/issues/1439) | **NOT closed by this row, and it must not be.** See `## Risks and decisions` D4 | | [#1470](https://github.com/mudler/vllm.cpp/issues/1470) | `test_ltx2_video` false-redded once on `main` under load and the failing case's identity was never captured. Untouched by THIS row, and **the identity and the rate are now measured** by `LTX25-DEVICE-RESIDENCY`: 1 red in 120 runs of the containment case at loadavg 40-155, on `artifacts.frames` render 2, where a 67.55 ms descheduling between `ppm_phase.Close()` and the leaf's destructor left 67.55 ms of a 74.87 ms leaf uncovered. It reds `covered >= 0.50 * leaf_seconds`. The same row also measured, and WITHDREW, the obvious repair: an instrument-relative second arm on that floor makes mutation `B-empty-ppm` — the writer's anchor opened after the write loop instead of around it, coverage 98% to 1.1% — pass. `artifacts.frames` is 0.2-7 ms on this fixture and one boundary on that host is 0.3-1.2 ms pinned to two idle cores, so no allowance built from the boundary is smaller than the leaf. The repair is an anchor, not a threshold | @@ -246,11 +430,42 @@ once, at `tests/vllm/multimodal/test_ltx2_video.cpp:4325`, asking for it. - Do not re-propose `residue <= 2 * instrument` without reading `## Design` 3 first, and never accept a 20-run distribution as evidence about it. - Do not close #1439 from this row. D4. +- Do not close #1884 with a share floor whose denominator is the INSTRUMENT, + the wall, or a written-down number of seconds. That is the withdrawn bound + wearing a different name. `## Design` 3, and D1. **One shape is NOT covered by + this prohibition, and `### M7` names it**: the anchor's own extent against the + SEAM its two neighbours define, both measured in the same run. Its denominator + is the region being named rather than the instrument, so it never asks how + many seconds a seam SHOULD hold -- the question that made the withdrawn bound + a property of the box. It still needs a constant and a cross-box distribution, + so it is where to START and not a licence to skip the measurement. ## Now -**This row's implementation is NOT on `main`, and #1556 is closed rather than -merged.** The pull request was measured, gate-run and through three fresh +**The three anchors #1668 owed land with this file.** `load.dit_config`, +`artifacts.mux`, `denoise.update` and `Ltx2ConditioningTrace::sampler_updates`, +from base `1724be38e`, held by a structural gate that carries no +constant, with six mutations detected and both wall-clock ratios untouched. +`Record::instrument_seconds`, the fourth item, had already landed as `be432e8e3` +(#1711). #1567 and #1568 are UNBLOCKED by that and stay open; neither is closed +here. The history below is kept because it is why the anchors have the shape +they have. + +**THIS ROW STILL HAS NO MATRIX ROW, and the implementation landing does not +create one.** The paragraph at the end of this section said that whoever picked +up #1668 would create the row with the implementation. That is declined, with a +reason: a matrix row carries a lifecycle state, and the state this work would +give it is not `DONE` — #1567, #1568, #1570, #1572 and #1884 are open under this spec +and three of them are gaps this row measured and could not close. A row created +now would enter the runnable population announcing a completion that four of its +own entries contradict. The spec is the record, `## Owed` names every owner, and +`git log --grep LTX25-PHASE-RESIDUE` is the history. Creating the row is owed to +whichever change closes the last of those five. + +### How it got here + +**This row's original implementation was NOT on `main`, and #1556 is closed +rather than merged.** The pull request was measured, gate-run and through three fresh reviews, and while it was in flight `519303d15` (#1622) landed the same load prologue repair under a different name. The pull request body — which `squash_merge_commit_message = PR_BODY` makes the permanent commit message — @@ -278,8 +493,11 @@ two floors this row was filed against, and it is NOT a claim that redding intermittently on `main`, which this row neither causes nor repairs. **LTX25-PHASE-RESIDUE has no matrix row, so it has no lifecycle state**, and -`scripts/now.py` and `audit-live-rows` will never surface it. That is deliberate -rather than an omission: with the implementation unlanded there is nothing to -give a state to, and creating a row would put an empty one in the runnable -population. The forward owner is [#1668](https://github.com/mudler/vllm.cpp/issues/1668), -and whoever picks it up creates the row with the implementation. +`scripts/now.py` and `audit-live-rows` will never surface it. That was deliberate +while the implementation was unlanded: there was nothing to give a state to, and +creating a row would have put an empty one in the runnable population. It said +here that whoever picked up #1668 would create the row with the implementation. +**#1668 lands and the row is still not created**, for the reason the top of +this section gives: the state it would carry is not `DONE`, and a row that +announces a completion five of its own `## Owed` entries contradict is worse +than no row. diff --git a/include/vllm/multimodal/ltx2_video.h b/include/vllm/multimodal/ltx2_video.h index 2838ff367..0c2d8f04d 100644 --- a/include/vllm/multimodal/ltx2_video.h +++ b/include/vllm/multimodal/ltx2_video.h @@ -897,6 +897,27 @@ struct Ltx2ConditioningTrace { // increment beside the res_2s loop's own returned `evaluations` would let the // two drift; the engine asserts they agree instead. int64_t dit_evaluations = 0; + // `sampler_updates` is every SAMPLER STEP this render took on the first-order + // arm — the post-process plus the Euler or ancestral update that turns a + // denoiser prediction into the next latent. Row LTX25-PHASE-RESIDUE, #1668. + // + // IT EXISTS AS A DENOMINATOR, exactly like `dit_evaluations` above and + // `video_decode_chunks` below. `denoise.update` anchors that work in the phase + // table, and the containment gate asserts one record per unit of work the + // RENDER counted, so the count cannot come from the table it is checking. + // Before this anchor the update ran inside the `denoise` leaf and inside no + // sub-scope, which is the whole of that gate's coverage miss: 49 us per step + // at nine frames against 343 us per step at 81, in one run of one binary. + // Instrument cost does not move 7x with the latent, so it is work. + // + // IT IS ZERO ON THE res_2s ARM, and that is recorded rather than hidden. That + // loop runs its own post-process and step inside `Ltx2Res2sDenoisingLoop` + // through `Ltx2Res2sHooks`, so anchoring it needs a hook rather than a + // statement and no gate in this tree renders on that arm. The zero is what + // separates this counter from a second name for `dit_evaluations`, and + // `test_ltx2_video` reads both arms to say so. See #1567 and `## Owed` in + // `.agents/specs/ltx25-phase-residue.md`. + int64_t sampler_updates = 0; // `dit_forwards` is every ACTUAL `Ltx2DitForward` this render ran, counted // inside the `Ltx2X0Model` lambda the guided denoiser drives. One evaluation // is one to four forwards — `cond`, `uncond`, `ptb`, `mod` diff --git a/src/vllm/multimodal/ltx2_video.cpp b/src/vllm/multimodal/ltx2_video.cpp index 8ba0b09f8..37b1a75f9 100644 --- a/src/vllm/multimodal/ltx2_video.cpp +++ b/src/vllm/multimodal/ltx2_video.cpp @@ -414,7 +414,7 @@ constexpr char kLtx2DurationHeadPathExtra[] = "duration_head_path"; // they are no longer trusted: the list below is derived from this file on every // run and compared, and the failure prints the replacement to paste in. // READER ANCHORS (derived and gated by test_ltx2_video): -// 930 940 941 1012 1108 1124 1213 1217 1310 1372 1480 1522 1564 1566 +// 930 940 941 1027 1123 1139 1228 1232 1335 1397 1505 1547 1589 1591 const char* const kKnownLoadExtras[] = { kLtx2AudioPromptEmbedsExtra, kLtx2PipelineKindExtra, kLtx2ModelVersionExtra, @@ -962,6 +962,21 @@ std::unique_ptr Ltx2VideoEngine::Load(const VideoModelParams& p im.dit = im.on_device ? Ltx2StreamDitToDevice(*im.queue, dit_file, dit_options) : Ltx2LoadDitFromSafetensors(dit_file, dit_options); } + // ── THE SECOND HOLE IN THE LOAD (row LTX25-PHASE-RESIDUE, #1668) ────────── + // + // `519303d15` named the load's PROLOGUE `load.open`, which was 92% of this + // render's un-named time. The gap decomposition `phase-log.json` now carries + // (#1571) named what was left, and the largest remaining one is this: from + // here to `load.video_vae`. It measured 0.822 ms of the residue on the + // 64x64x9 fixture on the box that filed #1571 and 0.109-0.127 ms on the box + // that landed this anchor — BOTH are quoted because a single number here + // reads as a property of the code and it is a property of the box, and + // NEITHER is asserted anywhere: no gate in this tree reads this magnitude + // (#1884). + // It is the DiT config resolution below, plus the recipe validation that reads + // it, and it sat inside the `load` span and inside no leaf — and `Sum` skips + // spans, so it was outside every number the table adds up. + phase::Scope dit_config_phase("load.dit_config"); // ── the config the SHAPES cannot see ────────────────────────────────────── // @@ -1237,6 +1252,16 @@ std::unique_ptr Ltx2VideoEngine::Load(const VideoModelParams& p // nothing else, and refusing it would break a caller who reuses one params // object across pipelines. if (params.video_vae_path.empty() && !im.recipe.audio_only) Fail("video_vae_path is required"); + // CLOSED BEFORE THE NEXT LEAF OPENS, for the reason `load.open` is closed + // before `load.dit`: a leaf opened while another leaf is live is marked + // `nested` and dropped from `sum_leaf_seconds` (`PhaseLog::Open`, + // `render_phase_log.cpp`). Left open across the VAE load, this anchor would + // take `load.video_vae` OUT of the table's sum instead of adding its own seam + // to it — naming 0.8 ms at the cost of un-naming a whole phase. + // + // Closed here rather than inside the `if` below so the audio-only recipe, + // which loads no video VAE, closes it at the same statement. + dit_config_phase.Close(); if (!params.video_vae_path.empty()) { const phase::Scope video_vae_phase("load.video_vae"); const SafetensorsFile f = SafetensorsFile::Open(params.video_vae_path); @@ -4520,6 +4545,36 @@ VideoResult Ltx2VideoEngine::Generate(const VideoGenParams& gen) { // first-order loop passes its own loop counter straight through // (samplers.py:45, :503) — which is what `should_skip_step` reads. Evaluate(video.latent, audio.latent, static_cast(sigma), step, v_raw, a_raw); + // ── THE SAMPLER'S OWN WORK (row LTX25-PHASE-RESIDUE, #1668) ───────── + // + // `denoise.step` above wraps the denoiser EVALUATION and closes when + // `Evaluate` returns. Everything below this line — the post-process and + // the Euler or ancestral update that turns the prediction into the next + // latent — is the sampler's own work. It sits BETWEEN two `denoise.step` + // records, inside the `denoise` leaf and inside no sub-scope, and until + // this anchor nothing named it. + // + // That is the whole of the containment gate's coverage miss, and the + // SHAPE of the miss is what proves it is work rather than instrument + // overhead: 49 us per step at nine frames against 343 us per step at 81 + // frames, in one run of one binary. Instrument cost does not move 7x + // with the latent. `test_ltx2_video`'s own comment asked for this scope + // by name while it tuned a threshold around the residue instead. + // + // NESTED, SO THE SUM DOES NOT MOVE. It opens while `denoise` is live and + // `PhaseLog::Open` marks it automatically, exactly as `denoise.step` + // beside it is marked; `Sum` skips both, so `sum_leaf_seconds` is + // unchanged and only the DECOMPOSITION of `denoise` gains a name. + // + // COUNTED BY THE RENDER, not by the instrument, for the reason + // `video_decode_chunks` is: the containment gate's record-count + // assertion is the only one there that is not a ratio, and a denominator + // derived from the phase table could not falsify a phase table. The + // increment sits INSIDE the scope so that the two cannot drift — a step + // that emits the record without counting it, or the reverse, is not + // reachable from one statement. + const ::vllm::multimodal::phase::Scope update_phase("denoise.update"); + im.trace.sampler_updates += 1; // `_step_state` (samplers.py:35) blends before it steps. const std::vector v_denoised = PostProcessLatent(v_raw, video); const std::vector a_denoised = PostProcessLatent(a_raw, audio); @@ -5367,6 +5422,22 @@ VideoResult Ltx2VideoEngine::Generate(const VideoGenParams& gen) { WriteFileBytes(result.audio_path, MiniMaxH3WriteWav(waveform, audio_channels, audio_samples, audio_rate)); } + // ── THE RENDER'S TAIL (row LTX25-PHASE-RESIDUE, #1668) ──────────────────── + // + // The result assembly and the mux argv build below ran after the last named + // leaf closed and before `WritePhaseLog` read the clock, so they were time + // nobody named in every table this instrument has ever written, and the last + // gap the decomposition reports. 0.210 ms of the fixture's residue on the + // box that filed #1571; 0.009-0.018 ms on the box that landed this anchor, + // which is the SAME ORDER as this instrument's own per-record charge, so on + // that box this anchor is close to naming its own cost. Quoted from two + // boxes for the reason above, and asserted on neither (#1884). + // + // NAMED `mux` RATHER THAN `finish`, and the distance between the two names is + // the point: `phase.finish` already names something else — what a RECIPE PHASE + // does after its sampler — and two names one letter apart for two different + // things is how a reader mis-ranks a lever. + phase::Scope mux_phase("artifacts.mux"); result.frame_count = rendered_frames; result.width = rendered_w; result.height = rendered_h; @@ -5392,6 +5463,9 @@ VideoResult Ltx2VideoEngine::Generate(const VideoGenParams& gen) { im.trace.completed = true; // W0 (#1010): the table, beside the frames it explains. The enclosing span is // closed BEFORE the write so this render's own span appears in its own file. + // The tail anchor closes first, so it names the assembly above and none of the + // writer's own cost — which the table charges to `instrument_seconds` instead. + mux_phase.Close(); generate_span.Close(); WritePhaseLog(gen.output_dir, kLtx2VideoFamily, phase_device, &result.phase_log_path); return result; diff --git a/tests/vllm/multimodal/test_ltx2_video.cpp b/tests/vllm/multimodal/test_ltx2_video.cpp index a1851da3c..7ab6b8676 100644 --- a/tests/vllm/multimodal/test_ltx2_video.cpp +++ b/tests/vllm/multimodal/test_ltx2_video.cpp @@ -4384,6 +4384,14 @@ void CheckRenderPhases(const nlohmann::json& table, REQUIRE(trace.completed); REQUIRE(trace.dit_evaluations > 0); REQUIRE(trace.video_decode_chunks > 0); + // AND THE THIRD ONE, which `denoise.update` is checked against below. Asserted + // rather than assumed: a counter that stayed at zero would make assertion (0) + // over that name read "emitted 0 times, and the render did the work 0 times", + // which is a vacuous pass wearing a green tick. This fixture renders on the + // first-order arm, where every step takes exactly one sampler update. + REQUIRE_MESSAGE(trace.sampler_updates > 0, + "the render counted no sampler updates at all, so the `denoise.update` count " + "below has nothing to falsify it"); // AND THE CHUNK COUNT IS RE-DERIVED rather than taken on faith, in the shape // the multi-chunk case above already uses: the tiling algebra the decoder runs @@ -4518,15 +4526,35 @@ void CheckRenderPhases(const nlohmann::json& table, // The audio decode is exactly one mel pass and one vocoder pass. const std::vector carrying = { {"denoise", - {"denoise.step"}, + {"denoise.step", "denoise.update"}, denoise_min_coverage, {}, - {trace.dit_evaluations}, - "Ltx2ConditioningTrace::dit_evaluations", - // ONE part, so (2) already IS the per-part assertion and a second copy of - // it would only be noise. `denoise.step`'s own placement debt is the - // third row of the anchor table in `### Owed out of W0`. - {0.0}, + {trace.dit_evaluations, trace.sampler_updates}, + "Ltx2ConditioningTrace::dit_evaluations and ::sampler_updates", + // TWO parts now, so (1b) stops being vacuous on this leaf — which is what + // the holding-action note above asked for. **(2b) does NOT**, and saying + // it did was wrong: both floors below are 0.0, and the loop that checks + // (2b) does `if (c.part_min_coverage[i] <= 0.0) continue;`, so it + // executes zero CHECKs here. `decode.audio` remains the only leaf where + // (2b) asserts anything. What this leaf gains is (1b), the count, and a + // reported share in the MESSAGE — not a second gated floor. + // `denoise.step`'s own placement debt is the third row of the anchor + // table in `### Owed out of W0`. + // + // NEITHER CARRIES A FLOOR, and for `denoise.update` that 0.0 is MEASURED + // debt rather than an oversight, in exactly the shape `decode.audio.mel` + // records beside it. The row's spec measured this name's honest share at + // 0.45% to 11.15% across four boxes, and the transfer it would have to + // catch — `denoise.step` left open across the post-process, with + // `denoise.update` emitted empty after it — puts it at ~0%. The two + // distributions therefore OVERLAP, and any floor that reds the transfer + // also reds an honest render this row has already produced. That is + // #1568, and it stays open here rather than being closed by a constant + // that cannot be justified. What binds this name instead is (0): the + // count comes from `Ltx2ConditioningTrace::sampler_updates`, which the + // RENDER maintains beside the update it counts, so no placement of a + // phase scope can move it. + {0.0, 0.0}, render}, {"decode.video", {"decode.video.chunk"}, @@ -4704,6 +4732,106 @@ void CheckRenderPhases(const nlohmann::json& table, } } +// ── A SEAM ANCHOR: the name that fills the hole between two named leaves ──── +// +// Row LTX25-PHASE-RESIDUE (#1668). `phase-log.json` carries a `gaps` array +// (#1571, landed by `LTX25-PHASE-INSTRUMENT`) that decomposes the residue into +// the intervals between consecutive leaves, and on the fixture render it names +// the regions nobody anchored. Two of them are anchored here, and this is the +// assertion that says the anchor went where the gap was. +// +// IT IS STRUCTURAL AND CARRIES NO TOLERANCE, which is deliberate and is the +// shape the note on the withdrawn `residue <= 2 * instrument` bound argues for. +// A seam anchor's guarantee is not "it measures at least X seconds" — its +// honest share is a property of the box, which is exactly what made that bound +// unusable. Its guarantee is a POSITION: the anchor lies inside the seam its +// two neighbours leave, and the seam holds nothing else. Under a correct +// placement all of that holds by construction; under any wrong one at least one +// clause fails outright, whatever the clock did: +// +// * opened before the previous leaf closes -> the anchor overlaps it, and +// `PhaseLog::Open` marks the INNER record `nested` besides; +// * left open across the next leaf -> the NEXT leaf is marked `nested` and +// leaves `sum_leaf_seconds` entirely, which is the failure mode the +// `load.setup`/`load.open` note in the driver warns about; +// * moved onto some other region -> it is no longer between these two names, +// and the leaf it landed beside shows up inside the seam. +// +// So no constant separates honest from defective here; the ORDER does. +void CheckSeamAnchor(const nlohmann::json& table, const std::string& previous, + const std::string& anchor, const std::string& next, int64_t render) { + INFO("seam anchor = " << anchor << ", between '" << previous << "' and '" << next << "'"); + const std::vector found = RecordsNamed(table, anchor, render); + REQUIRE_MESSAGE(found.size() == 1u, + "'" << anchor << "' was emitted " << found.size() + << " time(s) and the driver opens it exactly once. A region named zero " + "times is the un-named time this row was filed to name"); + const nlohmann::json& r = found.front(); + // A SEAM ANCHOR IS A LEAF, not a span and not nested. A span is skipped by + // `Sum`, so naming the seam with one would leave `sum_leaf_seconds` exactly + // where it was and change only the reader's impression that it had moved. + CHECK_MESSAGE(!r.value("span", false), + "'" << anchor << "' is a SPAN, so `Sum` skips it and the seconds it names are " + "still outside every leaf in the table's own sum"); + CHECK_MESSAGE(!r.value("nested", false), + "'" << anchor << "' is marked `nested`, so a leaf was still live when it " + "opened. Its seconds are dropped from `sum_leaf_seconds` and the seam is " + "still un-named"); + const double start = r["start_seconds"].get(); + const double end = r["end_seconds"].get(); + CHECK(end >= start); + + // The two neighbours that DEFINE the seam. Absent means this fixture does not + // run that phase, and the clause it would carry is skipped rather than faked; + // `REQUIRE(bounded)` below makes a call in which BOTH are absent a red instead + // of a vacuous pass. + bool bounded = false; + const std::vector before = RecordsNamed(table, previous, render); + if (!before.empty()) { + double previous_end = 0.0; + for (const nlohmann::json& p : before) { + previous_end = std::max(previous_end, p["end_seconds"].get()); + } + CHECK_MESSAGE(start >= previous_end - 1e-9, + "'" << anchor << "' opens at " << start << " and '" << previous + << "' is still running until " << previous_end + << ". The anchor is not filling the seam after that leaf — it is inside " + "it, charging a phase that already has a name"); + bounded = true; + } + const std::vector after = RecordsNamed(table, next, render); + if (!after.empty()) { + double next_start = std::numeric_limits::max(); + for (const nlohmann::json& n : after) { + next_start = std::min(next_start, n["start_seconds"].get()); + } + CHECK_MESSAGE(end <= next_start + 1e-9, + "'" << anchor << "' is still open at " << end << " and '" << next + << "' starts at " << next_start + << ". A leaf opened while this one is live is marked `nested` and leaves " + "`sum_leaf_seconds` altogether, so this placement REMOVES a phase from " + "the sum instead of adding one to it"); + bounded = true; + } + REQUIRE_MESSAGE(bounded, "neither '" << previous << "' nor '" << next + << "' is in this table, so this seam has no boundary and the " + "assertions above are vacuous"); + + // ...AND THE SEAM HOLDS NOTHING ELSE. Without this clause the two bounds above + // are satisfied by an anchor that moved anywhere into the interval, including + // on top of a THIRD leaf that happens to sit there — which is the transfer the + // rest of this file spends its length catching one level down. + for (const NamedInterval& iv : LeafIntervals(table, render)) { + if (iv.name == anchor) continue; + const bool outside = iv.end <= start + 1e-9 || iv.start >= end - 1e-9; + CHECK_MESSAGE(outside, "the leaf '" + << iv.name << "' runs [" << iv.start << ", " << iv.end + << "], inside the window '" << anchor << "' claims [" << start + << ", " << end << "]. The seam this anchor names is not empty, so " + "one of the two names is on the other's work"); + } +} + } // namespace TEST_CASE("ltx2 video: the three carrying phases contain their work and the load keeps its order") { @@ -4775,9 +4903,9 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load // (3c) AND NOTHING BUT AN ANCHOR IS NESTED. The assertion that sees a leaf // swallow a NEIGHBOUR, which the four above cannot: see the note on // `CheckOnlyAnchorsAreNested`. - CheckOnlyAnchorsAreNested(table, {"denoise.step", "decode.video.chunk", "decode.video.vae", - "decode.audio.mel", "decode.audio.vocoder", - "artifacts.frames.ppm"}); + CheckOnlyAnchorsAreNested(table, {"denoise.step", "denoise.update", "decode.video.chunk", + "decode.video.vae", "decode.audio.mel", + "decode.audio.vocoder", "artifacts.frames.ppm"}); // (4) THE FLOOR, read as "this name is not detached" and nothing more. See the // note above this case for why it is not tightened toward the measured share. @@ -4824,10 +4952,9 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load // every leaf in that state, and W2 and W3 must read it before they act on // `load.dit`. { - const std::vector load_order = {"load.open", "load.dit", - "load.video_vae", "load.audio_vae", - "load.upsampler", "load.text_encoder", - "load.prompt_embeds"}; + const std::vector load_order = { + "load.open", "load.dit", "load.dit_config", "load.video_vae", + "load.audio_vae", "load.upsampler", "load.text_encoder", "load.prompt_embeds"}; double previous = -1.0; std::string previous_name; int64_t seen = 0; @@ -4853,6 +4980,50 @@ TEST_CASE("ltx2 video: the three carrying phases contain their work and the load REQUIRE_MESSAGE(seen >= 4, "this fixture load names only " << seen << " load phases; the order it checks is vacuous"); } + + // (7) AND THE TWO SEAMS THE `gaps` DECOMPOSITION NAMED ARE ANCHORED. + // + // Row LTX25-PHASE-RESIDUE (#1668). The order above is a WEAK assertion by its + // own account, and it is weak in a specific direction: it constrains the + // leaves that exist and says nothing about the intervals BETWEEN them. That is + // where this render's residue actually was. #1571's gap decomposition, landed + // by `LTX25-PHASE-INSTRUMENT`, measured it on this fixture: after the load + // prologue — which `519303d15` named `load.open` — the largest remaining hole + // is `load.dit` -> `load.video_vae`, and the tail after `artifacts.audio` is + // the last one before the writer reads the clock. Both are now names rather + // than residue, and `CheckSeamAnchor` holds each to its position. + // + // THE LOAD RUNS ONCE, so its seam is read at render 0 like the order above. + CheckSeamAnchor(table, "load.dit", "load.dit_config", "load.video_vae", 0); + + // ...and the TAIL, which is checked on both renders because it is per-render. + // Its `next` is empty for a reason no other anchor here has: nothing follows + // it. `artifacts.mux` closes one statement before `WritePhaseLog` reads the + // clock, so the region it names is the last thing in the render and the seam + // has no leaf on its far side. The clause that would have bounded it is + // replaced by the stronger one below. + for (const int64_t r : {render_one, render_two}) { + CheckSeamAnchor(table, "artifacts.audio", "artifacts.mux", /*next=*/"", r); + // AND IT IS THE LAST LEAF OF ITS RENDER. This is what `next` cannot say. An + // `artifacts.mux` that opened early — around the audio write, say — still + // satisfies "after `artifacts.audio` ends" if the write is short, and still + // finds an empty window if it is narrow. It cannot satisfy this: some other + // leaf would then start after it, and the tail it claims to name would still + // be outside every leaf in the table. + const std::vector mux = RecordsNamed(table, "artifacts.mux", r); + REQUIRE(mux.size() == 1u); + const double mux_start = mux.front()["start_seconds"].get(); + for (const NamedInterval& iv : LeafIntervals(table, r)) { + if (iv.name == "artifacts.mux") continue; + CHECK_MESSAGE(iv.start <= mux_start + 1e-9, + "the leaf '" << iv.name << "' starts at " << iv.start << ", after '" + << "artifacts.mux" << "' opened at " << mux_start + << ". This anchor exists to name the render's TAIL — the result " + "assembly and the mux argv build that ran after the last named leaf " + "and before the table read the clock — and a leaf that follows it " + "means the tail is somewhere else"); + } + } } // ─── the CONSOLE copy is not a passenger of the FILE write ────────────── @@ -8893,6 +9064,36 @@ TEST_CASE("ltx2 video: the HQ pipeline evaluates the DiT twice per step") { // build that never ran the loop reports. Ruled out explicitly. CHECK(euler3.dit_evaluations > 0); + // ── AND `sampler_updates` IS A DIFFERENT NUMBER (row LTX25-PHASE-RESIDUE, + // #1668) ────────────────────────────────────────────────────────────────── + // + // `denoise.update`'s count assertion is taken from this counter, so the + // counter has to be worth taking. On the FIRST-ORDER arm the two agree — one + // sampler update per denoiser evaluation — and a counter that were merely a + // second name for `dit_evaluations` would agree there too, so that arm cannot + // tell them apart. THIS is where they separate: `res2s_two_stage` runs its + // post-process and step inside `Ltx2Res2sDenoisingLoop` behind + // `Ltx2Res2sHooks`, which the first-order arm's statement never reaches, so it + // counts 7 and 11 evaluations against ZERO updates. + // + // The zero is recorded rather than hidden, and it is #1567: anchoring that arm + // needs a hook rather than a statement, and no gate in this tree renders on it, + // so landing the anchor beside the first-order one would land dead code. What + // this case makes non-vacuous is that the counter counts the SAMPLER'S OWN + // WORK and not evaluations under another name. + INFO("sampler updates: res2s 3 -> " << hq3.sampler_updates << ", 5 -> " << hq5.sampler_updates + << "; euler 3 -> " << euler3.sampler_updates << ", 5 -> " + << euler5.sampler_updates); + CHECK(euler3.sampler_updates == 3); + CHECK(euler5.sampler_updates == 5); + // The DISCRIMINATOR. Same request, same fixture, same binary; the counters + // agree on one arm and disagree on the other, which no copy of + // `dit_evaluations` can do. + CHECK(euler3.sampler_updates == euler3.dit_evaluations); + CHECK(hq3.sampler_updates != hq3.dit_evaluations); + CHECK(hq3.sampler_updates == 0); + CHECK(hq5.sampler_updates == 0); + // THE BONG REFINEMENT IS REACHED ON THE PRODUCTION SCHEDULE, not only on the // hand-built fixtures in test_ltx2_pipeline. It changes the latent without // changing how many forwards ran, so the counter above is blind to it and this