Skip to content

gate_nvtier: load the all-RAM arm before the paged arm, and declare its real host need - #633

Merged
juliankang4 merged 2 commits into
midagedev:mainfrom
juliankang4:load-host-need-once
Oct 21, 2026
Merged

juliankang4 merged 2 commits into
midagedev:mainfrom
juliankang4:load-host-need-once

Conversation

@juliankang4

Copy link
Copy Markdown
Collaborator

gate_nvtier가 all-RAM 팔을 paged 팔보다 먼저 load하도록 순서를 바꾸고, 이 gate의 자원 선언을 실제 필요량으로 고칩니다.

Fixes the gate_nvtier item in #524.

What and why

gate_nvtier refused to load its all-RAM arm on a 128 GB class host, although that arm fits there when it loads alone. The gate loaded the paged arm first. That arm's NVMe tier keeps a RAM arena of 23,494,107,136 B alive until the stepunion arms, because the clauses after the two arms read the tier. The host check at the all-RAM arm's load reads MemAvailable, and that arena was already gone from it. On an RTX 5090 machine the refusal read: the host has 102,243,987,456 B available, under the 105,818,045,440 B this load needs.

This pull request makes two changes:

  • crates/gpu-gates/src/bin/gate_nvtier.rs loads the all-RAM arm first, so its host check sees the whole host. The clauses that compare the two arms read only values that do not depend on the order (tokens, digests, split walk, stats). shared/nvtier_family.rs already loads its all-RAM arm first.
  • tools/gate-batch-resources.tsv declares gate-gpu-nvtier at 105,818,045,440 B, the need the all-RAM arm's load check reports, instead of 92,054,154,240 B. The paged arm loads after it with a smaller need of 5,495,181,312 B plus its 23,494,107,136 B arena, so the all-RAM arm sets the maximum. A comment in the file shows the sum.

The model load path is unchanged. placed_pre in crates/gpu/src/model.rs reads host_available() and checks HostNeed::of(plan, extra).grant once, before the card upload and before any host set is read. The only earlier allocation is the CUDA context, so the load does not count its own allocations twice. The refusal came from the gate holding an earlier arm's arena, not from the load.

Release notes: internal: the nvtier gate loads its all-RAM arm first so it runs wherever that arm fits alone, and its resource row matches that arm's host need

Code shape

  • No second copy. No new logic. The arm order now matches shared/nvtier_family.rs.
  • Model code holds only architecture facts. No model code changes.
  • Dead code is gone. Nothing was made unused.
  • No silent failure. No error path changes.
  • Comments state what is true now.
  • No relaxed gate. No band, tolerance or lint moved. The declared host need went up to the measured value.
  • cargo fmt and cargo clippy are clean on what I touched; the lint count did not go up.

Duplication found or removed (path:line): none.

What I ran

The pull request is two commits on upstream main 58356c00. Both runs below used the arm-order change; the head differs from the measured build only in comment text and the resource table row.

On a Mac:

cargo fmt --all -- --check
rc 0
bash tools/check-comments.sh
rc 0

The gate on an RTX 5090 machine, once with the paged arm first (main's order) and once with the all-RAM arm first (this pull request):

BLOOMERY_BOX_LOCAL=1 just gate-gpu-nvtier
--- main's order
gate_nvtier: GpuModel::load_placed: the host has 102243987456 B available (MemAvailable, or the smaller room under a cgroup v2 limit
gpu-gate.sh: gate_nvtier ran 30 s (exit 1)
GPU GATE RED: gate_nvtier (exit 1)
--- this pull request
gate_nvtier: every clause passed
gpu-gate.sh: gate_nvtier ran 98 s (exit 0)

With this pull request every clause passed, including the clauses that compare the two arms (bits, split, floor-serve, counters). At the all-RAM arm's load the gate process held about 144 MiB of anonymous memory and MemAvailable was 120,034 MiB. Under main's order MemAvailable read 97,045 MiB while the paged arm's arena was held.

What I ran into

This does not block the pull request. shared/gate_card.rs opens the real tier by name only on an A6000 or a 3090, which are the cards this gate names, and an RTX 5090 is not among them. To run the gate on the 5090 I added one line during the run, .or(device.name.contains("5090").then_some("5090")), and reverted it afterwards. The plan used the 3090's bytes, and the run printed move proof card bytes ... PASS. That line is not in this pull request. I did not run the gate on an A6000 or a 3090.

Hardware (if the change is for a machine)

nvidia-smi prints this GPU name, driver and CUDA version:

NVIDIA GeForce RTX 5090, 595.99.02
| NVIDIA-SMI 595.99.02              Driver Version: 595.99.02      CUDA Version: 13.2     |

The gate ran against Qwen3.8-Flash-Next-UD-Q4_K_XL. The host has 128 GB class memory (MemAvailable 120,328 MiB at the gate's start). The arena of 23,494,107,136 B comes from the gate's plan, which uses the 3090's card bytes.

AI help

AI 코딩 에이전트가 코드를 작성한 후 설명을 썼고, 테스트를 마쳤습니다. diff 리뷰와 PR 초안 리뷰는 제가 직접 했습니다.

The paged arm's tier, which the room, advice, footprint and counters
clauses read after both arms, kept its RAM arena (about 23.5 GB on the
gate plan) mapped while the all-RAM arm loaded. That arm's host need
(105.8 GB on the gate plan) does not count the arena, but the host's
reading had already lost it, so a 128 GB host refused the load
although the arm alone fits. The all-RAM arm now loads first, as the
nvtier family gate already orders its arms.
The row declared 92,054,154,240 B, the qwen4exp plan's need, but the
gate's all-RAM arm also holds the 14.6 GB residency churn pool and needs
105,818,045,440 B.
@github-actions github-actions Bot added gpu The card engine: crates/gpu*, crates/app needs-gpu Changes code a card must check, and carries no gate run on a card tools Tools, recipes, CI and the toolchain: tools/, justfile, .github/, Cargo labels Oct 21, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CLEAN: The load order removes the retained-arena conflict, and the resource declaration matches the all-RAM load's stated need.

Non-blocking

⚠ Complete gate evidence: crates/gpu-gates/src/bin/gate_nvtier.rs:2002, the PR identifies the upstream base but omits the exact revisions of the two measured builds. It also reports a temporary 5090 card-selection edit. Record the measured revisions alongside that edit and the run output to make the comparison reproducible, as CONTRIBUTING.md requires.

Checked

Duplicates: 0 new functions, types or constants; no dryer pairs supplied. The sibling family gate already loads all-RAM first.
Complexity: crapper flagged no functions; the reorder adds no branches.
Silent failure: 0 added sites; both arm loads still propagate errors.
Tests: assertions unchanged. Restoring paged-first is the reported mutant; it fails at the host-load check on the reported 128 GB host. GPU execution was not repeated here.
Gates: no assertions, bands or pins moved; the host-resource declaration increased. Its sum is 105,818,045,440 B [derived from the listed terms].
Adversarial checks: the all-RAM session drops before the paged load; footprint preparation drops warmed pages; grant children still precede both arms.
Performance: gate load order only; no engine hot path changed.
Escape catalog: no patterns matched.

Notes

dryer did not run. Diagnostics says no changed Rust file is under crates/, although gate_nvtier.rs is changed.

Card check: needs-gpu. This pull request changes crates/gpu-gates/src/bin/gate_nvtier.rs and carries no gate run on a card (CONTRIBUTING's evidence list); a maintainer adds gpu-verified after one.

🔨 Shingler, bloomery's review bot: approves a clean pull request and requests changes on a must-fix (how it works)

@juliankang4

Copy link
Copy Markdown
Collaborator Author

The review is done and the contribution check passed, so I'm merging this.

@juliankang4
juliankang4 merged commit e943e1a into midagedev:main Oct 21, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gpu The card engine: crates/gpu*, crates/app needs-gpu Changes code a card must check, and carries no gate run on a card tools Tools, recipes, CI and the toolchain: tools/, justfile, .github/, Cargo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant