Repository navigation
[Campaign] MiniMax-H3 Ref2VA Ascend NPU latency optimization with video/audio quality gates #293
Description
Current state: PR #294 merged on 2026-11-02 (919ed42c) with historical five-second evidence: 407.27 → 108.12 s HTTP end-to-end latency (73.45% lower) on eight 910B1 NPUs, all 14 parity comparisons passing. Amy now reports a requester-narrowed lossless 5 s / 15 s follow-up (scope record); Draft #318 (e59b39a3) publishes validator repairs and author-reported initial screening: one 5 s baseline completed, but the candidate failed before inference due to conflicting offload settings. Corrected matched screening is queued; no paired lossless speedup or campaign acceptance is established.
Decision needed: none
Next action: Amy's existing follow-up tracks expected-duration audio validation, LPIPS weight identity, baseline repeatability and independent final-combination evidence at both durations. Review and qualification remain pending; the historical five-second result does not establish the new lossless outcome. Preserve the frozen scope/evidence below; the reported scope change applies to this follow-up and does not alter the separate 910C campaign. Final acceptance stays with the requester.
Goal and authority
User-requested campaign: minimize MiniMax-H3 Ref2VA delivered-video latency on Ascend NPU, retaining quality acceptance. The requester explicitly allows quantization and caches with quality gates. Codex acts as the implementation agent for the authenticated contributor. This authorizes this campaign only, not closure of other research issues or paid model evaluation.
Deliver accepted vLLM-Omni improvements in one PR, with each optimization's source and individually measured effect. Deliver reusable RSI experience and rejected experiments to InferRSI (the canonical successor of OmniRSI). No global optimality claim: report the best validated configuration within the declared search space.
Frozen initial scope
- Baseline source: vllm-project/vllm-omni
c7f56f0155953c7d4a77276a2f4fcbd844ab0da2. - Hardware: one node, 8 Ascend 910B1, 64 GiB/device, fixed physical allocation inside
npu run; no unreserved workload. - Environment image:
quay.io/ascend/vllm-omni:nightly-20261022-6b35bca, local image SHA2567fc6c73bd19a6ef4ee8d6048009972234796a1fc1bd3067168a425343cc476f7. Runtime compatibility is currently being checked; a changed environment requires a new frozen RunSpec before comparisons. - Checkpoint: original MiniMaxAI/MiniMax-H3 Ref2VA BF16; local content identity to be recorded before formal measurements.
- Inputs: official Ref2VA test prompt plus video SHA256
d6edc20ac82aa980c9cbd0b05b371f60d12dc8d018492e571b42de0086819c41and audio SHA2562a48d447b343b38a2c504d5281a0c97c688db0ba87931ed9282c598bd6752454. Downloaded content matches the pinned upstream accuracy test. - Workload: 1344x768, 24 FPS, output durations 5 s and 15 s, seed 0, 50 configured inference steps, flow shifts 12/3, concurrency 1. Seeds/reference variations are held-out checks before broader claims. Short-step bringup is diagnostic only.
- Base serving: USP8/ring1, text TP8, VAE tile8, DLO, dense LaserAttention, original BF16. Startup, warmup, request upload-to-full-MP4 delivery, and stage timings remain separate.
- Quality gates: decoded-video SSIM >=0.97 and PSNR >=34 dB (from upstream H3 reference test), LPIPS <=0.20, audio multi-resolution spectral cosine >=0.80 and RMS ratio in [0.50,2.00] (from upstream H3 quantization test). Apply to matched baseline/candidate outputs; these are campaign adoption of existing thresholds, not evidence that the original CUDA tests qualify NPU. Require matching frame count/dimensions/FPS, audio rate/channels/sample count, finite values and successful decoding. Record baseline rerun variance. Temporal/perceptual generalization remains a separate limitation.
- Formal comparisons: alternating AB/BA, independent server processes, a warmup per workload/arm, at least three measured requests per workload/arm; fixed inputs/device/config except the named candidate. No estimated p95 from means; retain raw samples and report small sample limits.
Search and attribution
Profiling first, then one hypothesis per step: residency/transfer, fused operators/packing, reference-KV reuse, TeaCache/Cache-DiT, INT8 and RainFusion. Existing main features are configuration baselines, not newly authored code. Do not use T2VA-only FastH3 distillation for Ref2VA. Test individual effects and the final combination directly; do not add percentages from separate cohorts.
Sources: task benchmark selection #60, engineering benchmark scope #252, pinned vLLM-Omni NPU recipe, H3 accuracy/quantization tests, and InferRSI source-attributed H3 records. Imported CUDA/older NPU results remain unverified on this campaign.
Acceptance and delivery
- Runnable baseline, source/model/environment manifests and lifecycle evidence.
- Profiling-backed candidates with exact source attribution and rejected outcomes retained.
- Independent video/audio quality checks and matched repeated E2E for every accepted candidate.
- Direct original-baseline versus final-combination test at both durations.
- One vLLM-Omni PR with individual effects and exact evidence scope.
- InferRSI contribution using existing evidence/knowledge contracts; source and optimizer improvement claims separated.
Current status: INCONCLUSIVE, bringup underway, no optimization result claimed. Stop candidate promotion on failed quality, request failures, resource interference or incomplete process cleanup. Implementation readiness and campaign acceptance are separate; final acceptance remains with the requester. Any unavailable measurement is reported explicitly.
Activity
Dong1017 commentedon Oct 31, 2026
同步一下：我们这边也会做 4×910C 的 H3 Ref2VA campaign。
当前这个 Issue 和 amy 的关联 PR #294 针对 单机 8×910B1（64 GiB/卡）。4×910C 的 baseline 目前还不清楚，我们会先做 preflight，确认环境、模型/代码版本、四卡运行配置与 baseline 的可运行条件，再讨论性能与质量对照。现阶段是计划，尚无 910C 实测结果。
两边的环境、性能和质量证据会分开记录，不直接沿用 910B1 的结论；amy 现有 campaign 的归属和范围保持不变。4×910C 拟按完整 campaign 跟踪：baseline 建立、瓶颈分析、优化筛选、性能与质量对照、最终组合验证及交付沉淀；preflight/baseline 只是第一阶段，不是整个任务。独立子 Issue、具体工作负载矩阵与验收指标待进一步讨论。
amy-why-3459 commentedon Nov 2, 2026
The requester has explicitly narrowed the next delivery to lossless Ref2VA acceleration with no audio-quality degradation, and requires both 5-second and 15-second outputs. This supersedes the earlier permission to select approximate caches/quantization for this follow-up; the historical five-second lossy evidence remains unchanged.
The existing engine delivery remains vLLM-Omni #9698. Reusable validation and new evidence will be a follow-up InferRSI contribution, because #294 is already merged. InferRSI is the OmniRSI successor.
Initial control: engine
922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb, native BF16, dense attention, 50 steps, no Cache-DiT, no reference-KV reuse, no quantization or sparse attention. Hardware remains eight 910B1 devices reserved together. Preserve the original source baselinec7f56f0155953c7d4a77276a2f4fcbd844ab0da2for the final direct comparison; matched shutdown budgets must be disclosed.First search slice: exact weight placement/transfer, using the existing DLO
rank-localandresident_layersimplementation, with unchanged arithmetic and weights. Screen the current AllGather control against rank-local transfer and a bounded resident prefix before selecting a candidate. Configuration reuse is attributed to the existing implementation, not claimed as a new kernel. Any final candidate needs independent AB/BA runs at both durations, warmups, raw HTTP E2E samples, separate stage profiling, every-output quality and actual worker exits. Do not add isolated percentage gains.Quality work precedes promotion: add a new validator version without changing the old frozen bundle. Check audio/video coverage and timestamps against expected output length; authenticate actual LPIPS weight bytes before loading; preserve upstream media metrics. The old 0.80 audio cosine gate alone cannot demonstrate no degradation. Retained dense baseline repetitions already have different decoded PCM hashes, so bitwise media equality cannot be presumed even for baseline-vs-baseline. Establish current control repeatability, retain exact-equality diagnostics, and compare candidate deviations against matched control variability. If evidence cannot establish preservation, retain INCONCLUSIVE instead of describing a perceptual threshold as losslessness.
Status: INCONCLUSIVE, preparation only. No new lossless speedup measured. This update does not request campaign closure or alter the independent 910C campaign.
Agent-assisted-by: Codex