evidence: track H3 Ref2VA on four logical Ascend 910C devices

evidence/h3-ref2va-910c-4logical-20261101-297/README.md

@@ -0,0 +1,50 @@
+# MiniMax-H3 Ref2VA: 910C four-logical-device campaign
+
+Governing acceptance issue: [InferRSI #297](https://github.com/InferMatrixLab/InferRSI/issues/297).
+Accountable owner: the requester's Codex H3 session, operating under Dong1017.
+Claim and plan: [owner receipt](https://github.com/InferMatrixLab/InferRSI/issues/297#issuecomment-6555458151).
+Final campaign acceptance remains with the requester.
+
+**Status: Draft tracking snapshot; campaign verdict INCONCLUSIVE.**
+No accepted 50% complete-HTTP latency reduction exists. All four final tracks
+remain incomplete. C1 passed 439 strict checks on each fixed 5 s/15 s request;
+one clean 5 s screening pair measured 291.248→286.667 s, a 1.5729% reduction,
+below the unchanged 50% bar. This is not final confirmation.
+
+C0–C23 cover 24 charged direction families. The tested fast lossy profiles have
+observed frozen quality failures. Most recently, C21/C22/C23 completed 5s-00
+but failed seven/nine/eight quality checks. C23 V2 passed 44 native tests and
+reduced a static 50-call attention cycle by 66.46%; its full 5 s diagnostic HTTP
+was 199.065 s. These observations do not qualify model quality or HTTP gains.
+No 15 s or clean timing extension follows for those rejected profiles.
+
+Start with the [experiment report and 24-direction ledger](results/report.md),
+[four-track execution snapshot](plan/execution-checkpoint.json), and
+[replay/availability notes](plan/reproduction.md). The inventory's publication
+hash map verifies available files; it does not qualify device or model behavior.
+
+Downstream code is tracked in [vLLM-Omni Draft PR #9745](https://github.com/vllm-project/vllm-omni/pull/9745).
+It ports only C1 to main, preserving newer piecewise/query-range dispatch, and
+passed [12 source-bound native tests](results/c1-upstream-main-native.json).
+The historical full-model C1 results remain tied to `be319692`; full-model
+quality and clean HTTP have not been rerun on the new upstream base.
+
+The scope is 5s/15s × lossless/lossy, 1344×768 at 24 FPS, 124/362 frames,
+original BF16 dense Ref2VA, 50 steps, four logical 910C devices. This is separate
+from the eight-device 910B1 campaign in #294 and the original eight-device requirement.
+
+- `plan/`: execution identity, original budget, protocol and frozen contract bytes.
+- `results/`: scoped numerical results, rejected operator variants and automatic quality comparisons.
+- `patch/`: tested private C1 and C10–C23 identities and their exact engine patches.
+- `validator/`: byte-preserved validator/scorer snapshots and checksums.
+
+The original B0 is retained. Migration and candidate sources are separate.
+Diagnostic HTTP values, operator improvements, unit tests and numerical equality
+are distinct conclusions. Missing quality samples remain incomplete; failures are
+not removed because other samples pass. The heldout clips have not been generated.
+See [execution checkpoint](plan/execution-checkpoint.json) and [results](results/README.md).
+
+Private connection details and original raw captures stay outside this repository.
+This Draft tracking snapshot contains sanitized projections and engine patch
+artifacts. Independent review, four-track acceptance and production qualification
+remain incomplete. The inventory registration checks layout/bytes only.

evidence/h3-ref2va-910c-4logical-20261101-297/patch/README.md

@@ -0,0 +1,70 @@
+# Patch status
+
+[C1 patch](c1-h3-bf16-prefix.patch) changes exactly three H3 source/test files.
+The [identity record](c1-source-identity.json) binds the tested private commit,
+parent, Git tree and file hashes. The engine commit remains private/unmerged;
+its diff is published here as an experiment artifact. Native focused
+tests passed (10 tests). Two complete fixed requests per duration passed strict
+comparison for both 5 s and 15 s, 439 checks each. The 15 s audit also recorded
+2,500 actual prefix-attention calls per rank per request. Clean performance
+screening measured a 1.5729% reduction at5s and failed the50% performance bar.
+Final qualification remains pending; no50% HTTP gain is accepted.
+
+The H3-owned override handles the explicit long-query BF16 fused selection with
+independent BNSD head batches and valid K/V-prefix slicing. Default and unsupported
+paths retain their parent behavior. It adds no generic backend API, MindIE binary
+or common environment change. Other failed candidates are retained in results;
+their existence does not establish upstream support. Any eventual engine PR
+needs current author checks and the upstream human-review/AI-disclosure gates.
+
+[C10 patch](c10-h3-audio-cache-guard.patch) changes four H3 source/test files.
+Its [identity](c10-source-identity.json) binds private commit `0af99184` to the
+native-tested bytes. Fifteen focused tests passed, including native NPU execution.
+The first full5s-01 request had no failures among39 observed gates. Independent
+5s-00 failed CLAP and audio-caption gates; the original C10 configuration is
+rejected for5s. Diagnostic HTTP was236.9/243.0s, with no clean performance claim.
+The second audit directly recorded50 actual wrapper forwards per rank and
+42checks/27vetoes agreeing across all4ranks. The first audit's missing runtime
+counters remain a historical evidence gap. No public API, dependency or frozen
+quality-gate change is included.
+
+## C12–C23 source artifacts
+
+Each C12/C13/C15–C23 `*-source-identity.json` records its tested engine head,
+parent, tree, raw file hashes and `patch_base`. The matching `*-candidate.patch`
+is cumulative over C1 `be319692b96413918973b5bd399cdda3f320c574`; apply C1
+first, then one selected cumulative patch. These branches are alternatives.
+`published: false` in an archived identity describes the engine commit's status
+at capture, not this evidence PR's publication state.
+
+C13 adapts [vLLM-Omni PR #9351](https://github.com/vllm-project/vllm-omni/pull/9351)
+ordering to the pinned chunked DLO implementation, retaining current manifests
+and double staging slots. C15–C23 are local experiments; their native tests and
+primitive results do not establish upstream support or complete model quality.
+C14 is a prerequisite probe and has no adopted engine patch.
+
+[The experiment report](../results/report.md) records dispositions and limits.
+In particular, C21/C22/C23 failed frozen quality and are not promoted for use.
+The separate [vLLM-Omni Draft PR #9745](https://github.com/vllm-project/vllm-omni/pull/9745)
+ports C1 to main; [its native checks](../results/c1-upstream-main-native.json)
+passed 12 tests. New-base full-model/HTTP qualification remains pending. These
+historical patch files are unchanged and retain their original tested revisions.
+No MindIE-SD code contribution is included.
+MindIE-SD source, installed provider binaries and shared environments were not
+modified. Future upstream engine contributions require their own focused checks
+and review. [Source reconstruction notes](../plan/reproduction.md) state the
+external artifacts needed for replay.
+
+
+[C11 dual-sigma patch](c11-modality-sigma-residual.patch) changes five H3 paths.
+Its [identity](c11-source-identity.json) binds61b8efa and20 native-focused tests.
+Fn1/Bn0 failed5s-00 audio adherence; Fn1/Bn1 failed its frozen audio-level gate.
+These are rejected configurations, not production acceleration claims.
+
+[FP32 residual revision](c11-sigma-fp32-residual.patch) preserves the same
+five-path overall boundary and adds a two-file delta over61b8efa. Its
+[identity](c11-fp32-source-identity.json) bindsd4d25b4 and23 passed tests,
+including three executed NPU tests. The native attempt projection retains two
+test-reference failures and the later correction. The production bytes stayed
+unchanged across these native attempts. Full5s-00 and15s-00 still failed frozen
+quality guards; the patch is not promoted. No frozen scorer/threshold changed.

evidence/h3-ref2va-910c-4logical-20261101-297/patch/c1-h3-bf16-prefix.patch

@@ -0,0 +1,200 @@
+diff --git a/tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py b/tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py
+new file mode 100644
+index 0000000..b805402
+--- /dev/null
++++ b/tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py
+@@ -0,0 +1,91 @@
++# SPDX-License-Identifier: Apache-2.0
++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
++
++import pytest
++import torch
++
++from tests.helpers.mark import hardware_marks
++from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata
++from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl
++from vllm_omni.diffusion.models.minimax_h3.attention import flash_h3
++from vllm_omni.platforms import current_omni_platform
++
++pytestmark = [pytest.mark.core_model, pytest.mark.diffusion, *hardware_marks(res={"npu": "A3"}, num_cards=1)]
++
++
++@pytest.mark.parametrize(
++    "mode", ["fused", "default", "laser", "causal", "masked", "compact", "gqa", "unresolved", "short"]
++)
++def test_h3_prefix_dispatch_preserves_unselected_paths(monkeypatch, mode):
++    query = torch.arange(16384, dtype=torch.float32).to(torch.bfloat16).reshape(1, 4096, 2, 2)
++    key, value = query + 10, query + 20
++    calls = []
++
++    def parent(self, q, k, v, attn_metadata=None):
++        calls.append("parent")
++        return torch.zeros_like(q)
++
++    def prefix(q, k, v, scale):
++        calls.append("prefix")
++        assert torch.equal(q, query)
++        assert torch.equal(k, key[:, :4093]) and torch.equal(v, value[:, :4093])
++        assert scale == 0.17
++        return torch.ones_like(q)
++
++    monkeypatch.setattr(FlashAttentionImpl, "forward_fa_npu", parent)
++    monkeypatch.setattr(flash_h3, "_prefix_bnsd", prefix)
++    impl = object.__new__(flash_h3.MiniMaxH3FlashAttentionImpl)
++    impl.causal, impl.softmax_scale = mode == "causal", 0.17
++    result = None if mode == "unresolved" else ([4093, 4096], [4093, 4096])
++    monkeypatch.setattr(impl, "_resolve_packed_seq_npu", lambda q, k, extra: result)
++    extra = {"npu_attn_varlen": True}
++    if mode == "compact":
++        extra["minimax_h3_compact_reference_rows"] = 2
++    mask = torch.ones(1, 4096, dtype=torch.bool) if mode == "masked" else None
++    metadata = AttentionMetadata(attn_mask=mask, extra=extra)
++    if mode == "default":
++        monkeypatch.delenv("MINDIE_SD_FA_TYPE", raising=False)
++    else:
++        monkeypatch.setenv("MINDIE_SD_FA_TYPE", "ascend_laser_attention" if mode == "laser" else "fused_attn_score")
++    actual_query, actual_key, actual_value = query, key[:, :, :1] if mode == "gqa" else key, value
++    if mode == "short":
++        actual_query, actual_key, actual_value = (tensor[:, :128] for tensor in (query, key, value))
++    output = impl.forward_fa_npu(actual_query, actual_key, actual_value, metadata)
++    assert calls == ["prefix" if mode == "fused" else "parent"]
++    assert torch.equal(output, torch.ones_like(actual_query) if mode == "fused" else torch.zeros_like(actual_query))
++
++
++@pytest.mark.skipif(not current_omni_platform.is_npu(), reason="Requires native Ascend fused attention")
++def test_native_prefix_bnsd_excludes_padding_and_preserves_inputs(monkeypatch):
++    monkeypatch.setenv("MINDIE_SD_FA_TYPE", "fused_attn_score")
++    torch.manual_seed(19)
++    device = torch.device(current_omni_platform.device_type)
++    query, key, value = [torch.randn(1, 4096, 2, 64, dtype=torch.bfloat16).to(device) for _ in range(3)]
++    original = [tensor.clone() for tensor in (query, key, value)]
++    used, scale = 4093, 0.17
++    cu = torch.tensor([0, used, 4096], dtype=torch.int32, device=device)
++    metadata = AttentionMetadata(
++        extra={
++            "npu_attn_varlen": True,
++            "cu_seqlens_q": cu,
++            "cu_seqlens_k": cu,
++            "max_seqlen_q": used,
++            "max_seqlen_k": used,
++            "laser_input_scale": 256.0,
++        }
++    )
++    impl = flash_h3.MiniMaxH3FlashAttentionImpl(num_heads=2, head_size=64, softmax_scale=scale, qkv_layout="BSND")
++    with torch.inference_mode():
++        output = impl.forward_fa_npu(query, key, value, metadata)
++        dirty_key, dirty_value = key.clone(), value.clone()
++        dirty_key[:, used:] = 1000
++        dirty_value[:, used:] = -1000
++        repeated = impl.forward_fa_npu(query, dirty_key, dirty_value, metadata)
++    q, k, v = query[:, :used].float().cpu(), key[:, :used].float().cpu(), value[:, :used].float().cpu()
++    scores = torch.einsum("bqhd,bkhd->bhqk", q, k) * scale
++    expected = torch.einsum("bhqk,bkhd->bqhd", scores.softmax(dim=-1), v)
++    assert output.is_contiguous() and output.dtype == torch.bfloat16
++    assert output.shape == query.shape and torch.isfinite(output).all()
++    torch.testing.assert_close(output[:, :used].float().cpu(), expected, rtol=1e-2, atol=2e-3)
++    assert torch.equal(output[:, :used], repeated[:, :used])
++    assert all(torch.equal(before, after) for before, after in zip(original, (query, key, value)))
+diff --git a/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py
+new file mode 100644
+index 0000000..6ebfb10
+--- /dev/null
++++ b/vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py
+@@ -0,0 +1,76 @@
++# SPDX-License-Identifier: Apache-2.0
++# SPDX-FileCopyrightText: Copyright contributors to the vLLM-Omni project
++"""Model-owned BF16 prefix attention for H3's explicit NPU fused selection."""
++
++from __future__ import annotations
++
++import os
++
++import torch
++
++from vllm_omni.diffusion.attention.backends.abstract import AttentionMetadata
++from vllm_omni.diffusion.attention.backends.flash_attn import FlashAttentionImpl
++
++
++def _prefix_bnsd(query: torch.Tensor, key: torch.Tensor, value: torch.Tensor, scale: float) -> torch.Tensor:
++    from mindiesd import attention_forward
++
++    batch, rows, heads, dim = query.shape
++
++    def fold(tensor: torch.Tensor) -> torch.Tensor:
++        return tensor.transpose(1, 2).contiguous().reshape(batch * heads, 1, tensor.shape[1], dim)
++
++    output = attention_forward(
++        fold(query),
++        fold(key),
++        fold(value),
++        attn_mask=None,
++        scale=scale,
++        head_first=True,
++        opt_mode="manual",
++        op_type="fused_attn_score",
++        layout="BNSD",
++    )
++    # Match the original packed varlen output layout before shared SP restores
++    # local sequence shards. Padding query rows are not consumed by H3.
++    return output.reshape(batch, heads, rows, dim).transpose(1, 2).contiguous()
++
++
++class MiniMaxH3FlashAttentionImpl(FlashAttentionImpl):
++    """Use independent head batches for explicit long BF16 prefix attention.
++
++    Default and other operator selections preserve their parent behavior.
++    Mathematical equivalence alone does not promise bit-exact model outputs;
++    each numerical qualification must cover the actual workload.
++    """
++
++    def forward_fa_npu(
++        self,
++        query: torch.Tensor,
++        key: torch.Tensor,
++        value: torch.Tensor,
++        attn_metadata: AttentionMetadata | None = None,
++    ) -> torch.Tensor:
++        if os.environ.get("MINDIE_SD_FA_TYPE") != "fused_attn_score" or attn_metadata is None:
++            return super().forward_fa_npu(query, key, value, attn_metadata)
++        extra = attn_metadata.extra
++        if (
++            self.causal
++            # Keep the short text refiners on their original varlen kernels.
++            or query.shape[1] < 4096
++            or attn_metadata.attn_mask is not None
++            or not extra.get("npu_attn_varlen", False)
++            or extra.get("minimax_h3_compact_reference_rows", 0)
++            or query.dtype != torch.bfloat16
++            or query.dtype != key.dtype
++            or query.dtype != value.dtype
++            or query.shape[2:] != key.shape[2:]
++            or query.shape[2:] != value.shape[2:]
++            or key.shape[1] != value.shape[1]
++        ):
++            return super().forward_fa_npu(query, key, value, attn_metadata)
++        resolved = self._resolve_packed_seq_npu(query, key, extra)
++        if resolved is None:
++            return super().forward_fa_npu(query, key, value, attn_metadata)
++        used_k = resolved[1][0]
++        return _prefix_bnsd(query, key[:, :used_k], value[:, :used_k], self.softmax_scale)
+diff --git a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py
+index 28931c9..09f2e7d 100644
+--- a/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py
++++ b/vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py
+@@ -208,6 +208,7 @@ class MiniMaxH3Attention(nn.Module):
+         self.vdn: VDNH3HybridAttention | None = None
+         self._prefix = prefix
+         from .attention.fastvideo_h3 import MiniMaxH3VSAImpl
++        from .attention.flash_h3 import MiniMaxH3FlashAttentionImpl
+
+         self.attention = Attention(
+             num_heads=self.num_heads,
+@@ -221,7 +222,7 @@ class MiniMaxH3Attention(nn.Module):
+             role_category=role_category,
+             skip_sequence_parallel=skip_sequence_parallel,
+             prefix=prefix,
+-            impl_overrides={"FASTVIDEO_VSA": MiniMaxH3VSAImpl},
++            impl_overrides={"FASTVIDEO_VSA": MiniMaxH3VSAImpl, "FLASH_ATTN": MiniMaxH3FlashAttentionImpl},
+         )
+
+     def enable_vsa_gate(self) -> None:

evidence/h3-ref2va-910c-4logical-20261101-297/patch/c1-source-identity.json

@@ -0,0 +1,12 @@
+{
+  "head": "be319692b96413918973b5bd399cdda3f320c574",
+  "tree": "8dfb3c6edbac7ca3a92edfa9659cc7a7eb58bbc3",
+  "base": "922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb",
+  "branch": "rsi/h3-297-bf16-prefix",
+  "files": {
+    "vllm_omni/diffusion/models/minimax_h3/attention/flash_h3.py": "6e175317c8db76b1e4e1b094a4c43923c8fbe3a5f039405f1cba070772497fe6",
+    "vllm_omni/diffusion/models/minimax_h3/minimax_h3_blocks.py": "e7e26ad0b2a9120b99281bf59a67eced7f7207f5679a8f02f2ede5859695be1a",
+    "tests/diffusion/models/minimax_h3/test_minimax_h3_flash_attention.py": "34fb26fb848556d8a67af91f7d3ee6443e77018e149672aed0450fd970be1968"
+  },
+  "published": false
+}