Skip to content

evidence: track H3 Ref2VA on four logical Ascend 910C devices - #317

Draft
Dong1017 wants to merge 2 commits into
InferMatrixLab:mainfrom
Dong1017:codex/h3-297-campaign
Draft

Dong1017 wants to merge 2 commits into
InferMatrixLab:mainfrom
Dong1017:codex/h3-297-campaign

Conversation

@Dong1017

@Dong1017 Dong1017 commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Governing Issue and delivery slice

Refs #297

Downstream engine contribution

vLLM-Omni Draft PR #9745
publishes only the C1 BF16 prefix-attention implementation, tests and recipe
against main: base b84bade9b9be8fdad0a1a9b3e8dd3ab4273dc4d9, head
7a20cf9b987b22fe3ce00221cb30cc3c6fb90e63. Its new-base focused native suite
passed 12 tests with actual Ascend fused attention, source binding and cleanup.
Portable native record. New-base full-model 5 s/15 s quality and
clean HTTP A/B are not run. Historical C1 strict/latency results below retain
their original revision; the 50% campaign target remains unmet.

Publish the requested Draft tracking snapshot for MiniMax-H3 Ref2VA on
four logical Ascend 910C devices, covering 5 s/15 s × lossless/lossy.
Campaign verdict: INCONCLUSIVE. No accepted 50% complete-HTTP reduction.
This is the issue owner's evidence delivery slice; final goal acceptance remains
with the requester.

The structure follows PR #294:
frozen plans, source attribution, results, rejected attempts, validators,
integrity hashes and replay limits. Its eight-device 910B1 campaign and accepted
result remain separate from this four-logical-device 910C campaign.

  • Accepted scope / delegation: the requester authorized the isolated
    campaign and this tracking PR. The target remains at least 50% matched-B0
    complete-HTTP reduction for each track, strict bit-exact lossless checks and
    frozen automatic lossy gates without human media inspection.
  • What this PR delivers: 96 campaign files, a four-track status/report,
    C0–C23 direction ledger, 28 automatic quality comparisons, 40 C12–C23 run
    projections, experimental engine diffs and source identities, eight unchanged
    frozen snapshots, plus inventory registration and count assertions.
  • What it does not deliver: a selected fast quality-qualified configuration,
    final four-track acceptance, a production engine contribution or a
    self-contained device replay environment.

Current results

Evidence cutoff: 2026-11-02 15:52 +08.

Track Numerical / quality evidence Clean complete-HTTP evidence Target status
5 s lossless C1 direct-B0 strict 439 PASS; C13 fixed-request strict 439 PASS One C1 screening pair: 291.247763→286.666760 s, 1.572888689% lower Performance FAIL at 50%; final confirmation incomplete
15 s lossless C1 direct-B0 strict 439 PASS Not run INCONCLUSIVE
5 s lossy Tested fast profiles have observed frozen quality failures No qualified candidate's clean pair No accepted candidate
15 s lossy Tested profiles have independent visual/audio failures at this duration No qualified candidate's clean pair No accepted candidate

The clean 5 s pair used two warmups and three measured complete MP4 responses
per fresh server, with audit/profiling disabled. Source integrity, cleanup and
strict paired checks passed. It is one screening pair, not the final three-pair
AB/BA/AB protocol. The delta includes migration+C1; no isolated migration-only
clean timing arm exists. Statistical significance is not claimed.

Recent completed diagnostic 5s-00 requests:

Candidate Audited HTTP Frozen quality disposition
C21 pre-gate activation cache 163.806 s FAIL: 7 observed violations
C22 sigma-secant pre-gate cache 166.859 s FAIL: 9 observed violations
C23 fresh conditioning / cached video context 199.065 s FAIL: 8 observed violations

Each passed 134 input checks and cleanup; none proceeds to 15 s or clean timing.
C23 V2 also passed 44 native tests, including three executed four-rank suites,
and reduced its static 50-call attention cycle by 66.46%. That primitive
measurement excludes FFN, changing-input quality, complete-model timing and HTTP;
the complete model's quality failure prevents promotion.

Changed artifacts and evidence

Requirement Artifact Evidence and remaining gap
Workload, revisions and budget Execution checkpoint, plans Original model/50 steps, isolated stack, current requester ceiling24 directions/32 h/128 logical-device-hours; original start/deadline retained
Independent quality contracts Lossless, lossy, frozen hashes Exact validator/scorer bytes preserved; quality corpus and heldout execution incomplete
Candidate results and origins Report and direction ledger, run projections, quality comparisons Includes negative/inconclusive outcomes, actual dispatch counters, C13 baseline-reproduced regression failure and retained C21/C22 controller-alias repair
Engine sources Patch notes and identities Experimental patches, including the chunked-DLO adaptation of upstream PR9351; private engine commits remain unmerged and are not adopted production support
Availability and integrity Replay notes, publication digests Available bytes are verifiable; original weights/image/media/tensors/logs and private bootstrap/runtime remain external

Baseline engine: df1c1b05936381dbaa3891c1ae4c4b21ab375a5d.
Migration source: 922f1e9f1cb85b9f2b66d86e1eb500e6872b2dcb.
Executed InferRSI: 50c19da43e0b358e6f990c00719891ad308a8ac1.
Latest rejected engine candidate C23: 6eb7bb094effb40853431208d1631effaf452a3f.
The publication commit is not a new experiment runtime revision.

Checks and dependencies

  • python tools/validate_evidence.py: PASS, 21 bundles / 528 hash checks,
    zero errors. Its 16 external/unavailable records across the inventory remain
    explicit; successful byte checks do not replay devices or change verdicts.
  • ruff check src tests tools: PASS.
  • pytest tests/test_evidence_inventory.py tests/test_smoke.py tests/test_claim.py:
    24 passed, 1 skipped. The symlink-escape fixture skipped because symlink
    creation is unavailable in this Windows environment.
  • All 96 staged evidence files matched their working bytes; eight original frozen
    snapshots remained exact. Relative Markdown links and the pending artifact
    scan passed. Private host identifiers, paths, credentials, raw media and weights
    are excluded. These checks do not substitute for independent review.
  • Device results are carried from their pinned executions; no model/device run
    was started for this publication. Original MindIE-SD source/binaries, shared
    checkouts and global SDK/driver environments were not changed.
  • Hosted checks apply to this PR head. Full repository/hardware qualification
    and independent review are not claimed by the local focused tests.

Knowledge authorship declaration

Not applicable under the current protected-path policy: this changes campaign
evidence, its inventory entry and inventory count assertions only.

Agent-assisted-by: OpenAI Codex (author)

Remaining work and Issue closure

All 24 direction families are charged. The original active deadline is
2026-11-03 01:19 +08, with no extra final-confirmation budget. Further new
mechanism search requires a requester extension of the direction ceiling;
this tracking PR grants no execution budget or new device run.

Four-track final confirmation, full lossy quality coverage and independent review
remain incomplete. The observed failed configurations stay FAIL; missing samples
are not PASS. Goal acceptance belongs to the requester and is not claimed.
Keep this PR Draft for tracking and keep the acceptance Issue open.

Publish frozen contracts, source-bound results and rejected candidates
with four-track status, reproduction limits and inventory hashes.
Keep numerical, model-quality and complete-HTTP conclusions separate.

Agent-assisted-by: OpenAI Codex (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>
Add the main-based native qualification and distinguish it from
historical full-model/HTTP campaign evidence.

Agent-assisted-by: OpenAI Codex (author)
Signed-off-by: GUOGUO <xwdong1998@163.com>

@Dong1017 Dong1017 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Static evidence review at a0677a03: one historical-validator limitation should be documented before reuse. The reported single-pair 1.57% result, rejected candidates and incomplete qualification remain appropriately bounded; this is not campaign acceptance or an unconditional LGTM.

  • Please preserve frozen validator bytes/receipts and document the mixed FAIL/missing-evidence precedence limitation below. A future qualified version should preserve observed FAIL while also reporting missing evidence.
  • Reuse the existing cross-PR inventory finding when integrating both bundles; it remains unresolved.
  • Software CI passed, but no device/model execution or independent raw-artifact replay was performed in this review. New-base full-model and HTTP qualification remain NOT_RUN.
Inspected evidence

Head: a0677a0; base: 2baebf6. Static inspection covered archived validators/scorers, frozen contracts, principal reports/projections, inventory and upstream receipt. The 28 quality projections retain 24 FAIL / 4 INCONCLUSIVE; 40 run projections agree with the report. Current-head software run: https://github.com/InferMatrixLab/InferRSI/actions/runs/41418369337 . No hardware efficacy inferred.

same({k: reference['media'][k] for k in MEDIA_KEYS},
{k: value['media'][k] for k in MEDIA_KEYS},
f'{label}:sample-{index}:decoded_pixels_and_pcm')
verdict = 'INCONCLUSIVE' if missing else 'FAIL' if failures else 'PASS'

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 — historical verdict precedence differs from the documented contract. An observed cleanup failure (or tensor mismatch) plus a missing rank configuration yields INCONCLUSIVE here, even though failures contains the violation; validator/README.md says observed gate violations remain FAIL. Please annotate this limitation while retaining frozen bytes and receipts, and use a separately versioned FAIL-first validator/wrapper with a mixed failure+missing regression before future qualification. This does not invalidate the reported C1/C13 PASS receipts, whose failure and missing lists are empty; no affected mixed-outcome receipt is established by this static review.

Copy link
Copy Markdown
Collaborator

Review at head a0677a03 (draft): not approving yet.

  1. Merge conflict: the PR is currently not mergeable with main (dirty). Please rebase.
  2. Open validator finding (evidence/h3-ref2va-910c-4logical-20261101-297/validator/strict_compare.py vs validator/README.md:19): an observed cleanup failure or tensor mismatch combined with a missing rank configuration yields INCONCLUSIVE, while the README says observed gate violations remain FAIL. No commit has addressed or documented this since the review on 2026-11-02. Expected: document this as a known historical-validator limitation in the README (or fix precedence if the validator is not frozen).
  3. Size: +27,509 lines / 98 files of evidence; please confirm this fits the evidence byte-budget discussion in [Investigation] Agent-led workflow gaps: closure, decision visibility, review provenance and focus #273 or note which artifacts should move to external storage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:workbench CLI workbench: profile, A/B loop, quality gates, evidence reports

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants