Repository navigation
Sync upstream/main (2026-10-14) / 同步上游 main（2026-10-14） - #37
Merged
Merged
Conversation
Update six Claude Code workflow invocations to claude-opus-5-5.
…SemiAnalysisAI#2542) * refactor: standardize srt-slurm cluster setup hooks Per-allocation host checks live in runners/srt-slurm/hooks/<cluster>/setup.sh and run through the cluster profile's default_host_setup. Shared host-check functions move to hooks/common.sh. MI355X gets its RDMA/QoS preflight, GPU drain gate, and hugepage reclaim. * feat(amd): add disaggregated SGLang and vLLM to synthetic acceptance * feat(amd): run MI355X multi-node recipes through srt-slurm Multi-node jobs with a CONFIG_FILE use the shared native path: srt-slurm owns allocation, serving, and post-eval; results and eval artifacts are collected with the shared helpers. The MI355X profile adds the multi-node fabric, model, and cache settings. * feat(amd): run MI355X multi-node fixed-sequence configs on srt-slurm Move the Qwen3.5 FP8/MXFP4 and DeepSeek-R1 FP8/MXFP4 disaggregated configs to srt-slurm recipes. Recipes write results in the shared sa-bench layout, and eval runs drop fake expert dispatch. The legacy multi-node path now only serves AgentX. * refactor(amd): remove legacy multi-node fixed-sequence orchestration Delete the retired MI355X disaggregated launch scripts, the amd_utils vLLM and ATOM engine paths, the fixed-sequence bench client, and the per-model fixed-sequence server settings. AgentX keeps its SGLang and TileRT paths. * fix(amd): serve Qwen3.5 FP8 disagg from the full MI355X checkpoint The cached Hub snapshot's config is text-only (qwen3_5_moe_text, no vision_config), so SGLang's Qwen-VL processor crashed at startup. The legacy path loaded /it-share/data/Qwen3.5-397B-A17B-FP8; register that as a model alias and use it in the recipe.
…pes (SemiAnalysisAI#3413) * refactor(amd): share one fixed-sequence client across multi-node recipes The four MI355X multi-node recipes each inlined the same benchmark loop. Move it to benchmarks/multi_node/srt_fixed_sequence.sh, like the single-node client; recipes set only the model, tokenizer, client backend and chat template in benchmark.env. * feat(srt): standard fixed-sequence client defaults and power windows Default the multi-node client to the repo standard: range ratio 0.8, chat template on, --trust-remote-code, 10x prompts and 2x warmup at request rate inf. Keep the client backend configurable. When srt-slurm exports SRT_MEASUREMENT_WINDOW_DIR, write each concurrency's measured boundary as a custom-benchmark window so power lanes validate without sa-bench. * refactor(power): move the fixed-sequence window writer into infx Replace the inline Python in srt_fixed_sequence.sh with infx.results.power.window, which writes srt-slurm's custom-benchmark measurement window from the client's result. * refactor(power): model the measurement window as a dataclass * test: drop the tautological power window test * fix(srt): request the model name the frontend serves Recipes hardcoded MODEL, but the served name depends on the cluster's model path and can differ from both the recipe and the workflow's HF id (Dynamo TRT-LLM served DeepSeek-R1-0528 while the recipe said dsr1, so every request got 404). Read it from /v1/models instead. * style: wrap window writer call for ruff
…nceX client (SemiAnalysisAI#3415) * refactor(nvidia): run multi-node fixed-sequence recipes on the InferenceX client Move the 228 recipe files behind the 27 active NVIDIA multi-node fixed-sequence configs from srt-slurm's sa-bench to benchmarks/multi_node/srt_fixed_sequence.sh. Each recipe keeps its ISL/OSL and sends the same served model name and /model tokenizer sa-bench used; concurrencies come from the matrix. Telemetry recipes keep benchmark.concurrencies, which srt-slurm needs for their expected power windows. Deprecated recipes are unchanged. * fix(power): expect windows for the matrix concurrencies on fixed-sequence power lanes The client runs CONC_LIST, but GB200, GB300 and B200 injected it into benchmark.concurrencies only for AgentX, so a narrowed fixed-sequence power run expected windows it never measured. Inject on every DCGM power lane, as H200 already does. * fix(nvidia): omit --publish-metrics on Dynamo TRT-LLM recipes The pinned srt-slurm passes --publish-metrics to Dynamo TRT-LLM workers by default, which Dynamo builds before ai-dynamo/dynamo#12162 reject, so workers exited at startup. Set engine.publish_metrics: false on the 156 active TRT-LLM recipes. * fix(nvidia): give H100 etcd and NATS time to start The first container step on an H100 node takes about eight minutes to set up the job image, and etcd is that step, so it missed srt-slurm's 300s readiness default. Allow 1200s in the 12 active H100 recipes. * refactor(nvidia): let the client read the served model name Drop the hardcoded MODEL from the converted recipes; the client now asks the frontend, which serves the name the workers registered. * fix(nvidia): give H100 workers an hour to become healthy Node container setup plus loading DeepSeek-R1 FP8 weights over NFS took longer than srt-slurm's 30-minute health default. Use 360 attempts, as the GB200 recipes do. * fix(nvidia): run H100 etcd and NATS in the nginx image Each service ran in its own ~40 GB instance of the job image on the head node, beside prefill rank 0, which was then killed while loading weights. etcd and NATS are static binaries from /configs, so use the small nginx image the profile already provides. * fix(nvidia): let TRT-LLM pick the UCX interface on B300 TRTLLM_UCX_INTERFACE named RDMA devices (mlx5_*), not a network interface, so TRT-LLM could not find a local IP ("Can't get local ip from NIC Interface") and KV cache never reached decode. No other cluster sets it; drop it from the 26 active B300 TRT-LLM recipes. * fix(nvidia): prewarm H100 weights so SGLang TP ranks load in step SGLang aborts when a TP rank finishes loading 480 s before the slowest one. On H100 the weights come over NFS, and a node reused from an earlier run loads from page cache, so the head node's ranks finished first and the run failed on every job that reused it. Read the weights into each worker node's page cache in host_setup before workers start. * Revert "fix(nvidia): let TRT-LLM pick the UCX interface on B300" B300 TRT-LLM disagg is broken on the DSXE cluster regardless; leave these recipes unchanged here.
…isAI#3370) * CollectiveX: NCCL EP - enable zero-copy in HT mode Align with other backends that take advantage of zero-copy. Signed-off-by: Ludwig Schneider <lschneider@nvidia.com> * CollectiveX: NCCL EP - switch to rank-major layout for LL mode Take advantage of NCCL EP features for low-latency mode: * use rank-major layout * leverage zero-copy when supported Signed-off-by: Ludwig Schneider <lschneider@nvidia.com> * CollectiveX: introduce CUDA graph support for relevant backends Many inference deployments rely on CUDA graphs * Enable this path for all backends that support it. * Set it by default. Signed-off-by: Ludwig Schneider <lschneider@nvidia.com> * CollectiveX: remove redundant NCCL EP completion Signed-off-by: Ludwig Schneider <lschneider@nvidia.com> --------- Signed-off-by: Ludwig Schneider <lschneider@nvidia.com> Co-authored-by: Ludwig Schneider <lschneider@nvidia.com>
Restore the previous model selections in the five Claude Code workflows to diagnose the sign-off failure.
…TP on MI355X / 在 MI355X 上添加 DeepSeek-V4-Pro FP4 ATOM PD 分离式 AgentX hicache MTP recipe (SemiAnalysisAI#3158) * feat: add DeepSeek-V4-Pro FP4 ATOM PD-disagg AgentX hicache MTP on MI355X / 在 MI355X 上添加 DeepSeek-V4-Pro FP4 ATOM PD 分离式 AgentX hicache MTP recipe Add the ATOM PD-disaggregated agentic arm for DeepSeek-V4-Pro FP4 on MI355X (dsv4-fp4-mi355x-atom-disagg-agentic-hicache-mtp), the ATOM counterpart to the existing sglang-disagg agentic arm, modeled on ATOM's DeepSeek-V4-Agentic-PD-Max recipe (TP / DP-attention / DP+CPU-offload tiers). - configs/amd-master.yaml: new config key (1P1D TP8, atomesh router, mooncake KV transfer); TP EP1 conc 1/2/8/16, DP EP8 conc 64/128 (no offload), DP EP8 conc 256 with lmcache_offload DRAM tier. - benchmarks/multi_node/agentic/dsv4_fp4_mi355x_atom-disagg.sh: agentic launcher. - amd_utils/models_atom.yaml: DeepSeek-V4-Pro-AgentX entry (prefix-cache env, prefill-only TBO, DP session-affinity env, block-256, agentic server knobs). - amd_utils/server_atom.sh: IS_AGENTIC branch (prefix caching, max-num-seqs=2x conc, lmcache_offload multi-connector, dp-sticky/idx2idx router, synthetic AL, trace_replay.sh client). - perf-changelog.yaml: changelog entry. 在 MI355X 上新增 DeepSeek-V4-Pro FP4 的 ATOM PD 分离式 agentic 分支 （dsv4-fp4-mi355x-atom-disagg-agentic-hicache-mtp），作为现有 sglang-disagg agentic 分支的 ATOM 对照，参考 ATOM 的 DeepSeek-V4-Agentic-PD-Max recipe （TP / DP-attention / DP+CPU 卸载三档）。搜索空间：TP EP1 并发 1/2/8/16， DP EP8 并发 64/128（无卸载），DP EP8 并发 256（lmcache_offload DRAM 卸载档）。 server_atom.sh 新增 IS_AGENTIC 分支：开启前缀缓存、max-num-seqs=2x 并发、 lmcache_offload multi 连接器、dp-sticky/idx2idx 路由、合成接受长度、trace_replay 客户端。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: set perf-changelog pr-link to SemiAnalysisAI#3158 / 将 perf-changelog 的 pr-link 设为 SemiAnalysisAI#3158 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: resolve DeepSeek-V4-Pro-AgentX recipe in server_atom.sh on agentic runs / 修复 server_atom.sh 在 agentic 运行时解析 DeepSeek-V4-Pro-AgentX recipe server_atom.sh looked up models_atom.yaml by the bare MODEL_NAME, so agentic runs silently loaded the throughput DeepSeek-V4-Pro entry instead of the new DeepSeek-V4-Pro-AgentX entry (block-256, mem_frac 0.9, batched-tokens, the attn/state/level/AL knobs, DP env, prefill-only TBO were all ignored). Mirror server_sglang.sh: derive the '<model>-AgentX' key when IS_AGENTIC, falling back to the base entry so the non-agentic atom-disagg arm is unchanged. server_atom.sh 之前按裸 MODEL_NAME 查 models_atom.yaml，导致 agentic 运行 静默加载吞吐版 DeepSeek-V4-Pro 条目，而非新的 DeepSeek-V4-Pro-AgentX （block-256、mem_frac 0.9、batched-tokens、attn/state/level/AL 等旋钮、DP 环境变量、仅 prefill 的 TBO 全部被忽略）。改为与 server_sglang.sh 一致： IS_AGENTIC 时解析 '<model>-AgentX'，缺失时回退到基础条目，保证非 agentic 的 atom-disagg 分支不变。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: remove dsv4-fp8-mi325x-vllm and -mtp configs / 移除 dsv4-fp8-mi325x-vllm 及其 mtp 配置 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: align atom PD agentic recipe with DeepSeek-V4-Agentic-PD-Max reference / 使 atom PD agentic recipe 与 DeepSeek-V4-Agentic-PD-Max 参考对齐 Three deviations from the ATOM reference recipe: - DP tiers (conc 64/128/256) used ep=8, adding --enable-expert-parallel that the recipe omits. Switch to ep=1 so the TP+DPA path emits exactly --enable-dp-attention (--enable-tbo on prefill only), no expert-parallel. - OFFLOAD_COPY_WORKERS=1 / OFFLOAD_MIN_LOAD_TOKENS=8192 were exported by the launcher outside the SLURM/Docker boundary and never reached the server. Export them in server_atom.sh's in-container offload branch instead. - ATOM_HOST_IP was never set; the recipe exports it per node. Default it to the node's resolved host_ip (matches the mooncake proxy_ip). 与 ATOM 参考 recipe 的三处偏差： - DP 档（并发 64/128/256）使用 ep=8，引入了 recipe 未使用的 --enable-expert-parallel。改为 ep=1，使 TP+DPA 路径仅输出 --enable-dp-attention（--enable-tbo 仅 prefill），不含 expert-parallel。 - OFFLOAD_COPY_WORKERS=1 / OFFLOAD_MIN_LOAD_TOKENS=8192 之前由启动脚本在 SLURM/Docker 边界外导出，未传入服务器进程。改在 server_atom.sh 容器内的 卸载分支导出。 - ATOM_HOST_IP 从未设置；recipe 每个节点都会导出它。默认设为本节点解析出的 host_ip（与 mooncake proxy_ip 一致）。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: reduce atom PD agentic sweep to one concurrency per tier / 将 atom PD agentic 扫描精简为每档一个并发点 Trim dsv4-fp4-mi355x-atom-disagg-agentic-hicache-mtp to a single representative concurrency per tier (1 TP, 64 DP, 256 DP+offload) for bring-up; the full conc lists are retained as comments. 将 dsv4-fp4-mi355x-atom-disagg-agentic-hicache-mtp 精简为每档一个代表性并发点 （1 TP、64 DP、256 DP+卸载）用于 bring-up；完整并发列表以注释保留。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: set INFMAX_CONTAINER_WORKSPACE for in-container ATOM agentic replay / 为容器内 ATOM agentic 回放设置 INFMAX_CONTAINER_WORKSPACE The in-container trace_replay.sh path aborted immediately because INFMAX_CONTAINER_WORKSPACE was unset (benchmark_lib.sh uses it to locate utils/aiperf + utils/agentic-benchmark). The SGLang client-image path sets it in its env-file; the ATOM path never did. Derive it from ATOM_WS_PATH (-> the container repo root /workspace) in server_atom.sh's agentic branch. Also narrow the sweep to the conc-1 TP tier (c64/c256 commented out) to validate this fix cheaply before committing GPU time to the DP/offload tiers. 容器内 trace_replay.sh 因 INFMAX_CONTAINER_WORKSPACE 未设置而立即中止 （benchmark_lib.sh 用它定位 utils/aiperf 与 utils/agentic-benchmark）。SGLang 客户端镜像路径在其 env-file 中设置了它；ATOM 路径没有。改为在 server_atom.sh 的 agentic 分支中由 ATOM_WS_PATH 推导（即容器仓库根 /workspace）。 同时将扫描收敛到 conc-1 TP 档（注释掉 c64/c256），以低成本验证此修复， 再对 DP/卸载档投入 GPU 时间。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: forward AIPERF_EXPERIMENTAL_FAST into ATOM agentic container / 将 AIPERF_EXPERIMENTAL_FAST 传入 ATOM agentic 容器 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: forward REQUIRE_POWER into ATOM agentic container / 将 REQUIRE_POWER 传入 ATOM agentic 容器 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore: expand atom PD agentic sweep to full concurrency grid / 将 atom PD agentic 扫描扩展为完整并发网格 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(atom): set MC_ENABLE_DEST_DEVICE_AFFINITY=1 for dsv4 agentic PD across all concurrencies Add MC_ENABLE_DEST_DEVICE_AFFINITY=1 to the DeepSeek-V4-Pro-AgentX base env so it applies to every concurrency tier on both prefill and decode nodes, matching ATOM recipes/DeepSeek-V4-Agentic-PD-Max.md. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update runtime_settings.sh * test(atom): align dsv4 agentic PD with DeepSeek-V4-Agentic-PD-Max, c1/c16 Apply the updated ATOM recipe to the dsv4 agentic PD-disagg arm and test the TP tier on rocm/atom-dev:nightly_202609210314: - drop MC_ENABLE_DEST_DEVICE_AFFINITY, add ATOM_MOONCAKE_MATCHED_RAILS=auto - add --index-cache-dtype fp4 - atomesh: cache_aware policies + balance thresholds on the DP tiers, round_robin on the TP tier, --atom-pd-rank-mapping-policy none on both - dense cudagraph capture ladder 1..min(64, 2*conc) and --cudagraph-mode FULL - ATOM_ENABLE_PREFILL_DELAYER=0 on the DP-attention tiers - spec decode --method dspark with synthetic acceptance 3.01 Throwaway test branch: the sweep is trimmed to conc 1 and 16 for an A/B against run 35566949648, which ran the pre-recipe config on the same image. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(atom): DSpark needs the -0813 checkpoint draft head Run 35576547691 died in drafter.arm_aux_capture on both cells: ValueError: DSpark requires dspark_target_layer_ids on the draft config. --method dspark reads dspark_block_size / dspark_markov_rank / dspark_target_layer_ids out of the served checkpoint's config.json. Only DeepSeek-V4-Pro-0813 bundles that draft head; the base Pro checkpoint does not, so the drafter had nothing to arm and every ModelRunner aborted after weight load. models.yaml already records this on the SGLang side. Point the arm at DeepSeek-V4-Pro-0813 -- the recipe's stated model, and the same checkpoint the sglang-disagg DSpark arm serves -- and alias DeepSeek-V4-Pro-0813-AgentX to the existing entry so server_atom.sh resolves the recipe under the new name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(atom): align dsv4 agentic PD with all recipe concurrency tiers Bring the ATOM PD-disagg agentic arm into full agreement with recipes/DeepSeek-V4-Agentic-PD-Max.md beyond the c1/c16 TP tier: - amd-master.yaml: restore the DP-attention tier (conc 64,128, dp-attn true, prefill TBO) and the DP+CPU-offload tier (conc 256, lmcache). - server_atom.sh: make decode dense CUDA-graph capture tier-aware -- TP decode keeps 1..min(64,2*conc); DP-attention decode uses the recipe's per-rank 1..(conc/4). - models_atom.yaml: drop ATOM_DP_SESSION_AFFINITY / ATOM_DP_LB_REQ_EQUIV from the DP env; the cache-aware --dp-aware router sends rank hints that override them, so they are not needed on this path (recipe). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(atom): scope GPU_MAX_HW_QUEUES to the DP-attention prefill node The recipe sets GPU_MAX_HW_QUEUES=5 only on the DP-attention prefill node; decode leaves it unset. The shared DP env was exported on every node, so it leaked onto decode. Move it into a prefill-only prefill_dp_env field and apply it in server_atom.sh only on prefill nodes when DP is on. ATOM_ENABLE_PREFILL_DELAYER=0 stays shared (recipe sets it on both roles). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore(config): rename DSV4-Pro MI355X ATOM disagg agentic key to lmcache-dspark Rename the config key dsv4-fp4-mi355x-atom-disagg-agentic-hicache-mtp to dsv4-fp4-mi355x-atom-disagg-agentic-lmcache-dspark to reflect the LMCache KV offload backend and DSpark speculative decoding used by this recipe. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * chore(config): bump DSV4-Pro atom-disagg image and lower dram-utilization to 0.50 Update dsv4-fp4-mi355x-atom-disagg-agentic-lmcache-dspark: bump the ATOM image from nightly_202609210314 to nightly_202609221542 and reduce the agentic-coding dram-utilization from 0.80 to 0.50 (smaller LMCache host-offload budget). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(config): isolate DSV4-Pro atom-disagg lmcache tier to conc-256 only Comment out the conc [1,16] and [64,128] tiers of dsv4-fp4-mi355x-atom-disagg-agentic-lmcache-dspark so the sweep exercises only the conc-256 DP-attention CPU-offload (LMCache) tier. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(config): run DSV4-Pro atom-disagg conc-256 with kv-offloading none Switch the active conc-256 tier of dsv4-fp4-mi355x-atom-disagg-agentic-lmcache-dspark from dram/LMCache offload to kv-offloading none (the dram/LMCache variant is kept commented out) to A/B the conc-256 point without the CPU-offload tier. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(atom-disagg): align DSV4-Pro PD launcher env with recipe Enable the conc-256 CPU-offload tier (dram + lmcache, dspark) in amd-master.yaml, and make server_atom.sh/models_atom.yaml reproduce the DeepSeek-V4-Agentic-PD-Max recipe exactly: NUMA binding and the ATOM_DP_* ports are now unconditional (the recipe sets them in the TP tier too, where DP attention is off), and PYTHONHASHSEED=0 is exported on the prefill offload node for consistent LMCache prefix hashes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(atom-disagg): use dp_sticky router policy for conc-256 At concurrency 256 the DP-attention router now pins each session to a fixed DP rank (dp_sticky prefill/decode policy) and exports AIPERF_HTTP_X_SESSION_ID_FROM_CORRELATION_ID=1 so aiperf derives the session id from the request correlation id. Conc 64/128 keep cache_aware and the TP tier keeps round_robin. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test(atom-disagg): restore conc 1/16/64/128/256 on nightly_202609221542 Restore the full concurrency sweep for the DSV4-Pro ATOM PD-disagg AgentX arm. conc 1/16/64 keep the run-35643358891 shape with only the image bumped; conc 128 matches run 35810485934 (MTP, DP attention, GPU-resident KV) and conc 256 matches run 35825955863 (DSpark draft model, lmcache DRAM tier, dram-utilization 0.50), so the two measured cells reproduce exactly. The MTP tiers keep dram-utilization 0.80 in their own scenario block. Carry the server_atom.sh PORT export so the lm-eval cells can run: without it run_lm_eval's check_env_vars guard aborts the eval and hangs the allocation (run 35643358891). Record the measured conc-128 / conc-256 throughput and the conc-128 gsm8k accuracy in perf-changelog.yaml, and fix the entry's config key, which still named the pre-rename hicache-mtp key. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(perf-changelog): note online_quant unused for DSV4-Pro MI355X ATOM PD-disagg AgentX Add the ATOM PD-disaggregated agentic arm for DeepSeek-V4-Pro-0813 MXFP4 on MI355X -- dsv4-fp4-mi355x-atom-disagg-agentic-lmcache-dspark -- the ATOM counterpart to dsv4-fp4-mi355x-sglang-disagg-agentic-hicache-mtp, so the two engines are directly comparable on the AgentX trace workload. 1P1D TP8, mooncake RDMA KV transfer, atomesh PD router, cluster:mi355x-amds. Modeled on ATOM recipes/DeepSeek-V4-Agentic-PD-Max.md. Docker image: rocm/atom-dev:nightly_202609221542 Recipe / search space: - TP8 (EP1): conc 1, 16 -- MTP, GPU-resident KV, dram-utilization 0.80. - DP-attention: conc 64, 128 -- MTP, GPU-resident KV, dram-utilization 0.80. - DP-attention + CPU offload: conc 256 -- DSpark draft model + lmcache host KV tier (multi connector, max_pending_saves=8, slot_sidecar_staging_slots=4, LRU, 256-token chunks), dram-utilization 0.50, dp-sticky idx2idx routing. - conc 1/16/64 keep the shape run 35643358891 used and change only the image. Per ATOM DeepSeek-V4-Agentic-PD-Max recipe: prefix caching on, FP8 KV and index cache, block-size 256, TBO on prefill only, max-num-seqs = 2x concurrency. online_quant: this config does not use online_quant. server_atom.sh only emits --online_quant_config (ONLINE_QUANT_ARG) when the model entry sets online_quant_config / online_quant_dpa_config in models_atom.yaml (read at server_atom.sh:65-66, assembled at :156-172, injected into the prefill/decode server commands at :317/561/652). The DeepSeek-V4-Pro-AgentX entry and its -0813-AgentX alias define neither field, so ONLINE_QUANT_ARG stays empty and --online_quant_config is never passed. Online (runtime) quantization is unnecessary here because the served DeepSeek-V4-Pro-0813 checkpoint is already offline MXFP4-quantized; online_quant (e.g. the ptpc_fp8 configs in deprecated/models_atom.yaml) is for BF16 / unquantized checkpoints. DSpark accuracy is not touched by online_quant. Because online_quant is off, no target layer is re-quantized at runtime, so it cannot perturb draft/target agreement (acceptance length) or gsm8k. The DSpark draft head is read from the -0813 config.json and verifies against the unchanged MXFP4 target; the gsm8k numbers reflect the offline MXFP4 checkpoint + DSpark only. Measured results: - conc-128 (run 35810485934, 3627 s, 13917/15339 profiled, 0 errors): 464105 total tok/s (29007 tok/s/GPU over 16 GPUs), 3934 output tok/s, TTFT mean 9.75 s / p90 15.89, TPOT mean 12.98 ms / p90 14.43, interactivity mean 77.0 / p90 69.3, e2e mean 23.01 s, 3.84 QPS, 96.6% theoretical prefix-cache hit. - conc-256 (run 35825955863, 3630 s, 21317/24162 profiled, 0 errors): 776348 total tok/s (48522 tok/s/GPU), 5794 output tok/s, TTFT mean 18.10 s / p90 32.78, TPOT mean 15.95 ms / p90 17.57, interactivity mean 62.7 / p90 56.9, e2e mean 33.89 s, 5.88 QPS, 96.8% theoretical prefix-cache hit. Not a clean scaling comparison vs conc-128 -- this tier also swaps MTP for the DSpark draft model and adds the lmcache host KV tier. Accuracy: conc-128 DP-attention cell (run 35851118405, lm-eval gsm8k 5-shot, eval concurrency 128, n=1319): em_strict 0.9719 +/- 0.0045, em_flexible 0.9712 +/- 0.0046. conc 1, 16, 64 are unmeasured on this image. Note: server_atom.sh exports PORT before sourcing benchmark_lib.sh on the eval path: run_lm_eval's check_env_vars guard runs before it parses --port, and job.slurm's docker -e allowlist forwards ROUTER_PORT but never PORT, so every multi-node lm-eval cell aborted ~8 s in and hung the decode node in "Waiting until router closes..." until cancelled (run 35643358891). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(perf-changelog): note online_quant unused for DSV4-Pro MI355X ATOM PD-disagg AgentX Add the ATOM PD-disaggregated agentic arm for DeepSeek-V4-Pro-0813 MXFP4 on MI355X -- dsv4-fp4-mi355x-atom-disagg-agentic-lmcache-dspark -- the ATOM counterpart to dsv4-fp4-mi355x-sglang-disagg-agentic-hicache-mtp, so the two engines are directly comparable on the AgentX trace workload. 1P1D TP8, mooncake RDMA KV transfer, atomesh PD router, cluster:mi355x-amds. Modeled on ATOM recipes/DeepSeek-V4-Agentic-PD-Max.md. Docker image: rocm/atom-dev:nightly_202609221542 Recipe / search space: - TP8 (EP1): conc 1, 16 -- MTP, GPU-resident KV, dram-utilization 0.80. - DP-attention: conc 64, 128 -- MTP, GPU-resident KV, dram-utilization 0.80. - DP-attention + CPU offload: conc 256 -- DSpark draft model + lmcache host KV tier (multi connector, max_pending_saves=8, slot_sidecar_staging_slots=4, LRU, 256-token chunks), dram-utilization 0.50, dp-sticky idx2idx routing. - conc 1/16/64 keep the shape run 35643358891 used and change only the image. Per ATOM DeepSeek-V4-Agentic-PD-Max recipe: prefix caching on, FP8 KV and index cache, block-size 256, TBO on prefill only, max-num-seqs = 2x concurrency. online_quant: this config does not use online_quant. server_atom.sh only emits --online_quant_config (ONLINE_QUANT_ARG) when the model entry sets online_quant_config / online_quant_dpa_config in models_atom.yaml (read at server_atom.sh:65-66, assembled at :156-172, injected into the prefill/decode server commands at :317/561/652). The DeepSeek-V4-Pro-AgentX entry and its -0813-AgentX alias define neither field, so ONLINE_QUANT_ARG stays empty and --online_quant_config is never passed. Online (runtime) quantization is unnecessary here because the served DeepSeek-V4-Pro-0813 checkpoint is already offline MXFP4-quantized; online_quant (e.g. the ptpc_fp8 configs in deprecated/models_atom.yaml) is for BF16 / unquantized checkpoints. DSpark accuracy is not touched by online_quant. Because online_quant is off, no target layer is re-quantized at runtime, so it cannot perturb draft/target agreement (acceptance length) or gsm8k. The DSpark draft head is read from the -0813 config.json and verifies against the unchanged MXFP4 target; the gsm8k numbers reflect the offline MXFP4 checkpoint + DSpark only. Measured results: - conc-128 (run 35810485934, 3627 s, 13917/15339 profiled, 0 errors): 464105 total tok/s (29007 tok/s/GPU over 16 GPUs), 3934 output tok/s, TTFT mean 9.75 s / p90 15.89, TPOT mean 12.98 ms / p90 14.43, interactivity mean 77.0 / p90 69.3, e2e mean 23.01 s, 3.84 QPS, 96.6% theoretical prefix-cache hit. - conc-256 (run 35825955863, 3630 s, 21317/24162 profiled, 0 errors): 776348 total tok/s (48522 tok/s/GPU), 5794 output tok/s, TTFT mean 18.10 s / p90 32.78, TPOT mean 15.95 ms / p90 17.57, interactivity mean 62.7 / p90 56.9, e2e mean 33.89 s, 5.88 QPS, 96.8% theoretical prefix-cache hit. Not a clean scaling comparison vs conc-128 -- this tier also swaps MTP for the DSpark draft model and adds the lmcache host KV tier. Accuracy: conc-128 DP-attention cell (run 35851118405, lm-eval gsm8k 5-shot, eval concurrency 128, n=1319): em_strict 0.9719 +/- 0.0045, em_flexible 0.9712 +/- 0.0046. conc 1, 16, 64 are unmeasured on this image. Note: server_atom.sh exports PORT before sourcing benchmark_lib.sh on the eval path: run_lm_eval's check_env_vars guard runs before it parses --port, and job.slurm's docker -e allowlist forwards ROUTER_PORT but never PORT, so every multi-node lm-eval cell aborted ~8 s in and hung the decode node in "Waiting until router closes..." until cancelled (run 35643358891). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: hyukjlee <hyukjlee@amd.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com>
* ci: run Claude Opus 5.5 with a supported CLI Pin Claude Code 2.1.282 and update the Claude action in each workflow before selecting Opus 5.5. * ci: document pinned Claude CLI installation The Claude CLI package has no regular dependencies, and all native optional packages are version pinned to 2.1.282. Record the narrow Zizmor exception for this explicit compatibility pin.
…e to v0.5.20-rocm720-mi35x-20260924 (SemiAnalysisAI#3423) Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rrying vllm#58510 / 在首个包含 vllm#58510 的 ROCm 10.0 nightly 上重新扫描 MI355X (SemiAnalysisAI#3420)
…-Flash ATOM TP2/TP4 (SemiAnalysisAI#3387) * feat(atom): add DeepSeek-V4.1-Flash MI355X AgentX recipe 新增 DeepSeek-V4.1-Flash 在 MI355X 上的 ATOM TP2/TP4 AgentX 配方，使用指定 nightly 镜像、5-token DSpark 和 golden AL 3.51；同步中英文文档及性能变更记录。 * docs: link ATOM Flash recipe changelog to PR 3386 回填 DeepSeek-V4.1-Flash ATOM 配方的实际 PR 链接。 * docs: point ATOM Flash changelog to upstream PR 3387 将性能变更记录指向 upstream 分支提交的 PR 3387。 * chore(atom): bump DeepSeek-V4.1-Flash image to nightly_202609241513 将 DeepSeek-V4.1-Flash ATOM 配方镜像更新为 rocm/atom-dev:nightly_202609241513（digest sha256:d04ec105），同步中英文文档及性能变更记录。 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * chore(atom): bump DeepSeek-V4.1-Flash image to nightly_202609250902 将 DeepSeek-V4.1-Flash ATOM 配方镜像更新为 rocm/atom-dev:nightly_202609250902（digest sha256:9aaeed59），同步中英文文档及性能变更记录。 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(atom): document DSpark layer-37-39 dtype preservation for DSV4.1-Flash PR SemiAnalysisAI#3387 adds the DeepSeek-V4.1-Flash FP4 ATOM AgentX recipe on MI355X (TP2/TP4, five-token DSpark, BF16 KV, FP8 index cache, no EP/KV offload). This commit adds a changelog note recording the data-type-preserved audit: DSpark draft layers 37-39 keep their checkpoint precision. The recipe passes no --online_quant_config, so make_v4_quant_config builds the same per-layer quant spec for the drafter as the target; the mtp.0/1/2 weights load inline from the same checkpoint (spec_decode=True) and config.get_layer_quant_config has no mtp.* special case, so ATOM neither re-quantizes nor up/down-casts them. --kv_cache_dtype/--index-cache-dtype affect cache storage only, not draft/MTP weights. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * feat(atom): add DeepSeek-V4.1-Flash FP4 ATOM AgentX on MI355X (TP2/TP4 DSpark) Add the DeepSeek-V4.1-Flash FP4 ATOM AgentX recipe on MI355X following the upstream TP2/TP4 recipe: TP2 conc [1,2,8,16,32,64], TP4 conc [2,8,16,32,64], no expert parallelism or KV offload. Uses rocm/atom-dev:nightly_202609250902, BF16 KV, FP8 index cache, max-num-seqs 128, 16K batching/prefill chunks, prefix caching with block size 16, 8K state checkpoints, level 3 + FULL graphs. Five-token DSpark with thinking_on golden AL 3.51 for throughput, real acceptance for eval, dsml_v41 tool parsing. Data-type preserved: DSpark draft layers 37-39 keep their checkpoint precision. The recipe passes no --online_quant_config, so make_v4_quant_config builds the same per-layer quant spec for the drafter as the target; the mtp.0/1/2 weights load inline from the same checkpoint (spec_decode=True) and config.get_layer_quant_config has no mtp.* special case, so ATOM neither re-quantizes nor up/down-casts them. --kv_cache_dtype/--index-cache-dtype affect cache storage only, not draft/MTP weights. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: honglie <275632494+yhl-amd@users.noreply.github.com> Co-authored-by: Honglie Yi <hyi@crsuse2-m2m-v2-020.us-east2-a.compute.internal> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: seungrokj <seungrok.jung@amd.com>
…image to v0.5.20-rocm720-mi30x / 将 glm5.2-fp8-mi325x-sglang-agentic-mtp 的 SGLang ROCm 镜像更新至 v0.5.20-rocm720-mi30x (SemiAnalysisAI#3362) * feat(glm5.2-mi325x): update SGLang ROCm image to v0.5.20-rocm720-mi30x Update glm5.2-fp8-mi325x-sglang-agentic-mtp from lmsysorg/sglang:v0.5.19-rocm720-mi30x to lmsysorg/sglang:v0.5.20-rocm720-mi30x and rename the retired --cuda-graph-max-bs alias to --cuda-graph-max-bs-decode in the recipe (sgl-project/sglang#38375). 将 glm5.2-fp8-mi325x-sglang-agentic-mtp 的 SGLang ROCm 镜像从 v0.5.19-rocm720-mi30x 更新至 v0.5.20-rocm720-mi30x，并将配方中已被上游移除的 --cuda-graph-max-bs 别名 改为 --cuda-graph-max-bs-decode（sgl-project/sglang#38375）。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(changelog): record glm5.2-fp8-mi325x-sglang-agentic-mtp v0.5.20 image refresh Append the perf-changelog entry for the SGLang ROCm image update to v0.5.20-rocm720-mi30x and the --cuda-graph-max-bs-decode rename. 为 SGLang ROCm 镜像更新至 v0.5.20-rocm720-mi30x 及 --cuda-graph-max-bs-decode 重命名追加 perf-changelog 条目。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com>
…mage to v0.30.0 / 将 minimaxm3-fp8-mi300x-vllm-agentic-mtp 的 vLLM ROCm 镜像更新至 v0.30.0 (SemiAnalysisAI#3361) * chore(amd): update minimaxm3-fp8-mi300x-vllm-agentic-mtp vLLM ROCm image to v0.30.0 Move the master image for minimaxm3-fp8-mi300x-vllm-agentic-mtp from vllm/vllm-openai-rocm:v0.29.0 to the vllm/vllm-openai-rocm:v0.30.0 release image (digest sha256:2e7da1ad1c66836802072588adea75f9f4991da5f9545b4318e91d422c22ce6a). The recipe script, search space and evals are unchanged. 将 minimaxm3-fp8-mi300x-vllm-agentic-mtp 的主镜像从 vllm/vllm-openai-rocm:v0.29.0 更新为 vllm/vllm-openai-rocm:v0.30.0 发布镜像（digest sha256:2e7da1ad1c66836802072588adea75f9f4991da5f9545b4318e91d422c22ce6a）。 配方脚本、搜索空间与评测均未改动。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(changelog): record the minimaxm3-fp8-mi300x-vllm-agentic-mtp v0.30.0 image update Append the perf-changelog entry for the vLLM ROCm image update from v0.29.0 to v0.30.0 on minimaxm3-fp8-mi300x-vllm-agentic-mtp after the smoke run (35729587789) passed its c2 and c16 LMCache throughput points and both minimax_m3_full evals. 为 minimaxm3-fp8-mi300x-vllm-agentic-mtp 的 vLLM ROCm 镜像从 v0.29.0 更新至 v0.30.0 追加 perf-changelog 条目；smoke 运行（35729587789）的 c2 与 c16 LMCache 吞吐点及两项 minimax_m3_full 评测均已通过。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Klaud Cold <klaud-cold@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com>
…mage to v0.5.20-cu130 / 将 dsv4-fp4-b200-sglang-agentic-hicache-mtp 的 SGLang 镜像更新至 v0.5.20-cu130 (SemiAnalysisAI#3334) * feat(dsv4): update B200 SGLang AgentX HiCache MTP image to v0.5.20-cu130 Update dsv4-fp4-b200-sglang-agentic-hicache-mtp from lmsysorg/sglang:v0.5.19-cu130 to lmsysorg/sglang:v0.5.20-cu130 (sglang v0.5.20, build commit 94602c9c). SGLang v0.5.20 removed the deprecated --cuda-graph-max-bs alias (sgl-project/sglang#38375), so the recipe passes the same value through --cuda-graph-max-bs-decode. Model, topology, DSpark settings, HiCache settings and all points are unchanged. 将 dsv4-fp4-b200-sglang-agentic-hicache-mtp 的镜像从 lmsysorg/sglang:v0.5.19-cu130 更新为 lmsysorg/sglang:v0.5.20-cu130（sglang v0.5.20，构建提交 94602c9c）。SGLang v0.5.20 移除了已弃用的 --cuda-graph-max-bs 别名（sgl-project/sglang#38375），因此配方改用 --cuda-graph-max-bs-decode 传入同一数值。模型、拓扑、DSpark 设置、HiCache 设置及所有测试点保持不变。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore(changelog): record the B200 DeepSeek-V4-Pro SGLang AgentX image update Append the perf-changelog entry for dsv4-fp4-b200-sglang-agentic-hicache-mtp moving to lmsysorg/sglang:v0.5.20-cu130 (PR #3334). 为 dsv4-fp4-b200-sglang-agentic-hicache-mtp 更新至 lmsysorg/sglang:v0.5.20-cu130 追加 perf-changelog 条目（PR #3334）。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(ci): verify only newly submitted signoffs (#3340) 仅对新提交的签核自动运行验证，编辑清单后需手动重试，并修正文档及验证器提示。 * perf(amd): enable ATOM DSpark K6 with RCCL DEP / ATOM 全并发启用 DSpark K6 与 RCCL DEP (#2912) * perf(amd): switch DSV4 AgentX to native RCCL DEP 将 DeepSeek-V4-Pro MI355X ATOM AgentX 的 c48 及以上测试切换到本地验证过的原生 RCCL DEP 配置，并保持低并发 TP 测试不变。 同步固定的 post-merge ATOM 镜像、EP8 元数据、关闭 TBO/EPLB、真实 MTP 接受率以及本地验证过的路由和 AIPerf 参数。 * docs(perf): link DSV4 RCCL DEP pull request 将 DeepSeek-V4-Pro RCCL DEP 性能变更记录中的占位链接替换为实际的 InferenceX PR 链接。 * fix(amd): trim DSV4 AgentX DEP overrides 精简 DeepSeek-V4-Pro AgentX RCCL DEP 配置，移除与 CLI 或公共默认值重复的环境变量，并补齐本地验证使用的 3600 秒 warmup grace。 * fix(amd): restore DSV4 AgentX scheduling controls Restore the request-equivalent weight, prefill delayer, and decode interval requested for the AgentX run. Remove the newly introduced terminal MTP overrides while keeping the rest of the cleanup unchanged. * fix(amd): remove redundant AgentX timeout overrides Keep the 3600-second agentic warmup allowance, but rely on the server keep-alive setting and AIPerf default benchmark grace period. * fix(amd): keep DSV4 state checkpoints at 8K Remove the DEP-only 32K override so both TP and DEP retain the original 8192-token state checkpoint interval. * perf(amd): capture dense DSV4 DEP decode graphs 为 DSV4-Pro 的 DPA/DEP 路径补齐小 batch CUDA graph，避免非二次幂 decode batch 向上填充造成的 attention、MoE 和 RCCL 无效计算。保留 TP 路径默认行为，并为高并发配置保留大 batch graph。 * perf(amd): include upstream DSV4 DEP fixes in AgentX Pin the September 12 ATOM nightly containing the RCCL EP sentinel fix and AITER stage2 tuning fix. Record the server's installed sources and bundled tuning rows in the AgentX artifacts while retaining dense decode graphs, FP8 KV, and the current scheduler settings. 固定到包含 RCCL EP sentinel 与 AITER stage2 调优修复的 9 月 12 日 ATOM nightly；保存服务端实际源码版本及镜像内 tuning 行，保留 dense decode graph、FP8 KV 和现行调度设置。镜像包含 AITER#4159，但本配方不启用其 BF16 FlyDSL paged SWA 分支。 * fix(ci): provide exact PR context to priority classifier Supply immutable base/head revisions and the permitted diff command, document read-only tool use, and allow 16 turns for structured classification. 为优先级分类器提供确切的 Base/Head 版本及允许执行的 Diff 命令，明确只读工具用法，并将结构化分类的轮数上限调整为 16。 * perf(amd): enable DSpark K6 across the ATOM AgentX sweep 将 PR #2912 的十个 ATOM AgentX 性能点切换为固定版本 0813 checkpoint、 DSpark K6/q7 和 golden AL 3.77；C256 全量 GSM8K 保留真实验证。 修复 draft_model 路由与共享缓存挂载，统一服务与客户端 tokenizer， 增加 DSpark head、66 个分片及 tokenizer 检查并记录运行身份。 保留既有镜像优化、原生 RCCL DEP 和 dense FULL graph 配置。 * fix(amd): preserve inference mode in DSpark TP MoE callbacks 修复 DSpark q7 触发 AITER TP 通信融合 MoE 补齐路径时的 inference tensor 原地写入错误。仅对固定源码应用 InferenceMode callback 补丁，保留通信融合与 graph，保存补丁哈希，并覆盖原始失败、补齐、对齐和重复应用场景。 * fix(amd): use upstream ATOM inference-mode fix 将 MI355X ATOM AgentX 配方切换到包含 ROCm/ATOM#2233 的 pr2233-4f3a808 镜像，并移除运行时 AITER 源码补丁、对应测试和补丁清单记录。 * perf(amd): use BF16 KV through concurrency 16 将 MI355X ATOM AgentX 的 C1、C2、C4、C8 和 C16 任务切换为 BF16 KV cache；C48 及以上 DEP 任务继续使用 FP8 KV cache。 * perf(amd): pin DSv4 ATOM AgentX to official nightly_202609161445 Restore run-sweep.yml to the PR base and replace the temporary rocm/atom-dev:pr2233-4f3a808 image with official nightly_202609161445, which includes the merged ROCm/ATOM#2233 inference-mode fix. 将 run-sweep.yml 还原到 PR base，并把临时镜像 pr2233-4f3a808 换成 包含已合入 ATOM#2233 修复的官方 nightly_202609161445。 * fix: restore missing config-keys in perf-changelog merge The main merge left the first PR #2912 changelog entry without its - config-keys: header, so process_changelog could not parse the added YAML. 修复合并后第一条 2912 changelog 条目缺少 - config-keys: 导致的 YAML 解析失败。 * docs: drop unrelated CI procedure changes 将两份 CI 流程文档恢复为 main 版本，使其不再出现在本 PR 的变更中。 * docs: align CI procedures with PR base 将中英文 CI 流程文档恢复为 PR 当前基础版本，确保它们不再出现在本 PR 的变更中。 * docs(perf): correct DSv4-Pro ATOM AgentX changelog image to nightly_202609161445 Align the perf-changelog entry with the recipe's actual image (rocm/atom-dev:nightly_202609161445 in configs/amd-master.yaml), replacing the stale nightly_202609071454 reference and its now-mismatched digest and build-commit details. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(perf): align dsv4-fp4-mi355x-atom-agentic-mtp changelog with PR 2912 Rewrite the dsv4-fp4-mi355x-atom-agentic-mtp entry to match the final recipe: EAGLE MTP -> DSpark (draft_model) on the DeepSeek-V4-Pro-0813 checkpoint, NUM_SPEC_TOKENS 3 -> 6 (K6), SPEC_DECODE_AL 2.49 -> 3.77, and the RCCL DEP band on TP8/DPA8/EP8. Correct the state checkpoint interval to STATE_CHECKPOINT_INTERVAL_TOKENS=8192 (was mis-stated as 32768) and note eval verifies real DSpark drafts, not MTP. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(perf): align dsv4-fp4-mi355x-atom-agentic-mtp changelog with PR 2912 Rewrite the dsv4-fp4-mi355x-atom-agentic-mtp entry to match the final recipe: EAGLE MTP -> DSpark (draft_model) on the DeepSeek-V4-Pro-0813 checkpoint, NUM_SPEC_TOKENS 3 -> 6 (K6), SPEC_DECODE_AL 2.49 -> 3.77, and the RCCL DEP band on TP8/DPA8/EP8. Correct the state checkpoint interval to STATE_CHECKPOINT_INTERVAL_TOKENS=8192 (was mis-stated as 32768) and note eval verifies real DSpark drafts, not MTP. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: Chun Fang <chun.fang@amd.com> Co-authored-by: seungrokj <seungrok.jung@amd.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * docs: define draft precision as serve-the-draft-as-it-ships (#3348) Rewrite the Draft-model precision rule in CONTRIBUTING.md, its Chinese mirror, the checklist item, and verifier Check 13 so the baseline is the draft that ships with the served checkpoint, at its stored precision, through the pinned upstream image's default handling. Forbid only submission-side changes that lower draft precision below that default. Add the Qwen FP8 embedded MTP head and the SGLang DSpark wo_a FP8-to-BF16 load conversion as worked examples of what is compliant. 将 CONTRIBUTING.md 中的 Draft 模型精度规则、其中文镜像、审阅清单条目以及 验证器检查 13 重写为：基线是随所服务 checkpoint 一同发布的 draft，保持其 存储精度，并采用锁定上游镜像的默认加载处理。仅禁止提交方将 draft 精度降到 该默认值以下的改动。新增 Qwen FP8 内嵌 MTP head 与 SGLang DSpark wo_a FP8 转 BF16 加载转换两个合规示例。 * Fix MI300X AMD partition and match its eight-node runner pool (#3350) Submit to compute-0 and retire mi300x-amd_08 from both inventory groups. Benchmark commands and resource settings are unchanged. * Consolidate deprecated configs and remove retired runtime support (#3349) * chore: consolidate deprecated configs by vendor * style: separate deprecated configs with blank lines * chore: remove retired model routing and stale agent guidance * chore: archive retired server registry entries and fix deprecation guidance * chore: archive remaining retired disaggregated registry variants * chore: remove retired DSV4 fixed-sequence helper branches * docs: correct remaining GLM AgentX policy comment * Add B300 Qwen3.5 397B FP8 disaggregated AgentX recipes (#3268) * feat: add B300 Qwen3.5 FP8 disaggregated AgentX starter * chore: link B300 starter performance changelog * perf: colocate B300 Qwen3.5 prefill and decode workers * fix: match B300 Mamba cache flag to pinned SGLang * perf: use latest SGLang nightly for B300 Qwen3.5 FP8 * fix: use Dynamo compatibility with latest SGLang * fix: use intra-node NVLink for colocated B300 PD * perf: select measured B300 FP8 HiCache PD candidate * fix: propagate failed B300 Slurm runs after collecting artifacts * Prepare expanded measured B300 FP8 qualification curve * perf(b300): qualify measured concurrency 48 PD point * perf(b300): preserve existing aggregate publication scopes * style(b300): normalize final config newline * fix: require validated B300 Qwen3.5 AgentX power telemetry * fix: bind B300 power windows to matrix concurrency * perf: qualify B300 FP8 write-through PD at concurrency 96 * style: format B300 power contract fixture * fix: declare the B300 AgentX formal power window contract * docs: consolidate the B300 FP8 performance changelog * Avoid unverified host-memory pressure for B300 H298 recipes * fix: allow complete B300 AgentX runs on eligible nodes * perf: retain B300 FP8 candidates supported by canonical qualification * feat(models): acquire verified original BF16 MTP shards atomically * test(models): reject corrupted provenance and unexpected asset files * feat(b300): stage original BF16 MTP assets for Qwen FP8 recipes * feat(b300): prepare C48 disaggregation with 64 decode slots * perf(b300): add screened TP2 EP2 write-through candidate * perf(b300): select measured C24 decode capacity * perf: expand B300 FP8 disaggregated AgentX frontier * fix: allow full B300 TP2 sweep startup and drain time * perf: select B300 1P2D throughput frontier point * perf: select B300 1P2D C32 frontier point * perf: add B300 1P2D C64 frontier point * perf: refine B300 FP8 disaggregation frontier candidates * Refine B300 FP8 frontier with decode cache and DP attention * perf: refine B300 FP8 disaggregated frontier * perf(b300): qualify six FP8 disaggregated frontier candidates * perf(b300): qualify C4 latency endpoint and drop regressed C40 * perf(b300): select basic DCGM counters for Qwen AgentX * perf(b300): add higher-capacity C48 FP8 disaggregation candidate * perf: expand B300 FP8 sweep with screened C12 and C44 points * perf: add screened B300 FP8 C56 ReplaySSM capacity point * test: remove B300 SRT status test * test: remove MTP acquisition test * fix(b300): use the official FP8 embedded MTP head * fix(b300): disable inherited AgentX power collection * fix(b300): narrow node exclusion and simplify recipe metadata * [Klaud Cold] Update qwen3.5-fp8-h200-sglang-agentic-hicache-mtp SGLang image to nightly-dev-cu13-20260916-c9a8fba9 / 将 qwen3.5-fp8-h200-sglang-agentic-hicache-mtp 的 SGLang 镜像更新至 nightly-dev-cu13-20260916-c9a8fba9 (#3200) * perf: update qwen3.5-fp8-h200-sglang-agentic-hicache-mtp SGLang image to nightly-dev-cu13-20260916-c9a8fba9 Move the H200 Qwen3.5 FP8 AgentX HiCache MTP recipe from lmsysorg/sglang:nightly-dev-cu13-20260914-4358a161 to lmsysorg/sglang:nightly-dev-cu13-20260916-c9a8fba9 (sgl-project/sglang@c9a8fba9, index digest sha256:509c2742b9441fe9182ae3dd7dac8840ba48e03d2d44b7b24e80d428f117fd85). The 09-16 nightly is the first cu13 build carrying sgl-project/sglang#39516, the HiCache startup fallback for the pinned sglang-kernel 0.4.7 wheel that broke every registered host pool on the 09-15 build. Recipe script, MTP settings, HiCache arms and the concurrency grid are unchanged. 将 H200 Qwen3.5 FP8 AgentX HiCache MTP 配方的 SGLang 镜像从 lmsysorg/sglang:nightly-dev-cu13-20260914-4358a161 更新为 lmsysorg/sglang:nightly-dev-cu13-20260916-c9a8fba9（sgl-project/sglang@c9a8fba9）。 09-16 nightly 是首个包含 sgl-project/sglang#39516 的 cu13 构建，该修复为固定的 sglang-kernel 0.4.7 wheel 提供 HiCache 启动回退，09-15 构建中所有已注册主机池因此无法启动。 配方脚本、MTP 设置、HiCache 分支及并发网格保持不变。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * perf: add changelog entry for the qwen3.5-fp8-h200-sglang-agentic-hicache-mtp image bump Append the perf-changelog entry for moving qwen3.5-fp8-h200-sglang-agentic-hicache-mtp to lmsysorg/sglang:nightly-dev-cu13-20260916-c9a8fba9 after the targeted smoke run (35135979333) passed the c2 throughput point and the c24 gsm8k eval. 为 qwen3.5-fp8-h200-sglang-agentic-hicache-mtp 更新至 lmsysorg/sglang:nightly-dev-cu13-20260916-c9a8fba9 追加 perf-changelog 条目； 定向冒烟运行（35135979333）已通过 c2 吞吐点与 c24 gsm8k 评测。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com> * [Klaud Cold] Update minimaxm3-fp8-mi300x-vllm-agentic-mtp vLLM ROCm image to v0.29.0 / 将 minimaxm3-fp8-mi300x-vllm-agentic-mtp 的 vLLM ROCm 镜像更新至 v0.29.0 (#3063) * feat: update minimaxm3-fp8-mi300x-vllm-agentic-mtp vLLM ROCm image to v0.29.0 Move the MI300X MiniMax-M3 MXFP8 AgentX EAGLE3 recipe from the unstable commit nightly vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 to the v0.29.0 release image vllm/vllm-openai-rocm:v0.29.0 (digest sha256:e5e47f6aaab675c252c381f0dac237b31b10d87bb74d092b07fb4065efd7f5a1). Recipe, search space and evals are unchanged. 将 MI300X MiniMax-M3 MXFP8 AgentX EAGLE3 配方的镜像从不稳定的提交 nightly vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 更新为 v0.29.0 发布镜像 vllm/vllm-openai-rocm:v0.29.0 （digest sha256:e5e47f6aaab675c252c381f0dac237b31b10d87bb74d092b07fb4065efd7f5a1）。 配方、搜索空间与评测保持不变。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: add perf-changelog entry for the minimaxm3-fp8-mi300x-vllm-agentic-mtp v0.29.0 image bump Record the vLLM ROCm image move from nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 to the v0.29.0 release image for PR #3063 after the smoke run passed. 为 PR #3063 记录 vLLM ROCm 镜像从 nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 更新至 v0.29.0 发布镜像的 perf-changelog 条目（冒烟运行已通过）。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: shorten PR 3063 performance changelog 缩短 PR 3063 的性能变更日志条目。 * docs: keep PR 3063 changelog concise 保持 PR 3063 性能变更日志简洁。 --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com> * [Klaud Cold] Update minimaxm3-fp8-mi300x-vllm-agentic-mtp vLLM ROCm image to v0.29.0 / 将 minimaxm3-fp8-mi300x-vllm-agentic-mtp 的 vLLM ROCm 镜像更新至 v0.29.0 (#3063) * feat: update minimaxm3-fp8-mi300x-vllm-agentic-mtp vLLM ROCm image to v0.29.0 Move the MI300X MiniMax-M3 MXFP8 AgentX EAGLE3 recipe from the unstable commit nightly vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 to the v0.29.0 release image vllm/vllm-openai-rocm:v0.29.0 (digest sha256:e5e47f6aaab675c252c381f0dac237b31b10d87bb74d092b07fb4065efd7f5a1). Recipe, search space and evals are unchanged. 将 MI300X MiniMax-M3 MXFP8 AgentX EAGLE3 配方的镜像从不稳定的提交 nightly vllm/vllm-openai-rocm:nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 更新为 v0.29.0 发布镜像 vllm/vllm-openai-rocm:v0.29.0 （digest sha256:e5e47f6aaab675c252c381f0dac237b31b10d87bb74d092b07fb4065efd7f5a1）。 配方、搜索空间与评测保持不变。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: add perf-changelog entry for the minimaxm3-fp8-mi300x-vllm-agentic-mtp v0.29.0 image bump Record the vLLM ROCm image move from nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 to the v0.29.0 release image for PR #3063 after the smoke run passed. 为 PR #3063 记录 vLLM ROCm 镜像从 nightly-d9105ea8001e0a6d77a96327d17515bb5791fb36 更新至 v0.29.0 发布镜像的 perf-changelog 条目（冒烟运行已通过）。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: shorten PR 3063 performance changelog 缩短 PR 3063 的性能变更日志条目。 * docs: keep PR 3063 changelog concise 保持 PR 3063 性能变更日志简洁。 --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com> * [Klaud Cold] Update glm5.2-fp8-mi325x-sglang-agentic-mtp SGLang ROCm image to v0.5.19-rocm720-mi30x / 将 glm5.2-fp8-mi325x-sglang-agentic-mtp 的 SGLang ROCm 镜像更新至 v0.5.19-rocm720-mi30x (#2980) * chore: bump glm5.2-fp8-mi325x-sglang-agentic-mtp SGLang image to v0.5.19-rocm720-mi30x Update the GLM-5.2 FP8 MI325X SGLang AgentX MTP family from lmsysorg/sglang:v0.5.16-rocm720-mi30x to the v0.5.19 release image lmsysorg/sglang:v0.5.19-rocm720-mi30x (Docker Hub digest sha256:a0ffcdd013af6d79c9b7c4348c42b14a79a48f6c43dd3ca0235ec3802da45f75). Model, precision, topology, speculative settings, workloads and recipe script are unchanged. 将 GLM-5.2 FP8 MI325X SGLang AgentX MTP 配方的镜像从 lmsysorg/sglang:v0.5.16-rocm720-mi30x 更新为 v0.5.19 发布镜像 lmsysorg/sglang:v0.5.19-rocm720-mi30x。模型、精度、拓扑、投机解码参数、 工作负载与配方脚本保持不变。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * chore: add perf-changelog entry for glm5.2-fp8-mi325x-sglang-agentic-mtp image bump Append the changelog entry for updating the GLM-5.2 FP8 MI325X SGLang AgentX MTP image to lmsysorg/sglang:v0.5.19-rocm720-mi30x (PR #2980). 为 glm5.2-fp8-mi325x-sglang-agentic-mtp 镜像更新至 lmsysorg/sglang:v0.5.19-rocm720-mi30x 追加性能变更日志条目（PR #2980）。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs: shorten PR 2980 performance changelog 缩短 PR 2980 的性能变更日志条目。 --------- Co-authored-by: Klaud-Cold <klaud-cold@users.noreply.github.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com> * [Klaud Cold] Add MI325X TP4 and TP2 DeepSeek-V4.1-Flash vLLM AgentX arms with Engram host offload / 新增 Engram 主机卸载的 MI325X TP4 与 TP2 DeepSeek-V4.1-Flash vLLM AgentX 臂 (#3337) * [Klaud Cold] Add MI325X TP4 and TP2 DeepSeek-V4.1-Flash vLLM AgentX arms with Engram host offload / 新增 Engram 主机卸载的 MI325X TP4 与 TP2 DeepSeek-V4.1-Flash vLLM AgentX 臂 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Repin dsv41flash-mi325x-tp2-tp4-engram-offload to the ROCm 10.0 nightly channel / 将镜像重新固定到 ROCm 10.0 nightly 渠道 Match SemiAnalysisAI/InferenceX#3326, which moved MI355X to nightly-rocm100-3df4ae15 on the ROCm 10.0 channel. Same vLLM commit, different ROCm runtime. 与 SemiAnalysisAI/InferenceX#3326 保持一致：相同的 vLLM commit，ROCm 10.0 运行时。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: refresh PR #3337 for sweep reuse [skip-sweep] Sync with origin/main after the green sweep run 35637516947; the reuse gate authorizes that run on this head. 在绿色 sweep 运行 35637516947 之后与 origin/main 同步；reuse gate 在此 head 上授权该运行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore(ci): run the priority classifier on Opus 5 at low effort [skip-sweep] (#3360) * chore: bump srt-slurm to v2.18.0 and stream benchmark output (#3367) Move the utils/srt-slurm submodule from 984180e (#448) to v2.18.0 (2ac4eb1), 28 commits ahead, which adds benchmark.stream_output (NVIDIA/srt-slurm#483). Enable it for every srt-slurm recipe through setup_srt_slurm so benchmark.out is mirrored into the Actions job log while the client runs. The TileRT fork pin predates the option and is left unchanged. * Add DSV4 B200 Dynamo+SGLang AgentX configs / 添加 DSV4 B200 Dynamo+SGLang AgentX 配置 (#3257) * feat(dsv4): add B200 Dynamo SGLang AgentX configs * docs(perf): link DSV4 B200 AgentX PR * fix(dsv4): expand c256 decode KV headroom --------- Co-authored-by: hshrivastava-droid <hshrivastava@nvidia.com> Co-authored-by: Rohit Nagraj <rohitnagraj.99@gmail.com> * [Klaud Cold] Add MI300X TP4 DeepSeek-V4.1-Flash vLLM AgentX arm with Engram host offload / 新增 Engram 主机卸载的 MI300X TP4 DeepSeek-V4.1-Flash vLLM AgentX 臂 (#3336) * [Klaud Cold] Add MI300X TP4 and TP2 DeepSeek-V4.1-Flash vLLM AgentX arms with Engram host offload / 新增 Engram 主机卸载的 MI300X TP4 与 TP2 DeepSeek-V4.1-Flash vLLM AgentX 臂 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Repin dsv41flash-mi300x-tp2-tp4-engram-offload to the ROCm 10.0 nightly channel / 将镜像重新固定到 ROCm 10.0 nightly 渠道 Match SemiAnalysisAI/InferenceX#3326, which moved MI355X to nightly-rocm100-3df4ae15 on the ROCm 10.0 channel. Same vLLM commit, different ROCm runtime. 与 SemiAnalysisAI/InferenceX#3326 保持一致：相同的 vLLM commit，ROCm 10.0 运行时。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Drop the MI300X TP2 arm: measured infeasible on a 192 GB card / 移除 MI300X TP2 臂：在 192 GB 卡上实测不可行 Run 35671005506 reported "Available KV cache memory: -13.51 GiB" at TP2 concurrency 1 and the engine refused to start, even with the indexer buffer already halved to 4096 batched tokens. TP8 and TP4 are unaffected; TP4 passed at concurrency 1 and 32 in the same run. 运行 35671005506 在 TP2 并发 1 下报告 "Available KV cache memory: -13.51 GiB" 并拒绝启动，此时 indexer 缓冲区已减半至 4096。TP8 与 TP4 不受影响。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: refresh PR #3336 for sweep reuse [skip-sweep] Sync with origin/main after the green sweep run 35678619120; the reuse gate authorizes that run on this head. 在绿色 sweep 运行 35678619120 之后与 origin/main 同步；reuse gate 在此 head 上授权该运行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: warn when sign-off lacks a reuse command (#3368) 中文：签署审核缺少授权复用命令时改为警告，同步裁定发布逻辑、双语文档和回归测试。 * [PowerX] retain healthy window measurements in audit sidecars / 在审计文件中保留健康窗口测量 (#3357) * fix: retain healthy power windows in audit sidecars 保留健康窗口的独立测量和逐点失败证据，保持包完整性检查、required-power 及发布判定不变。 * docs: clarify retained power diagnostics 精简中英文窗口保留说明，明确审计顶层逐 GPU 诊断字段不构成发布许可。 * [B200][SGLang][AgentX] Improve DeepSeek V4.1 Flash performance (#3346) * perf(b200): tune DeepSeek V4.1 host Engram layout on latest SGLang * chore(b200): link performance candidate to PR 3346 * perf(b200): add STP and retain native DSpark FP8 projections * fix(b200): import pinned images and handle strided draft inputs * perf(b200): compare GPU-resident Engram tables for STP * fix(b200): use shipped default DSpark implementation * perf(b200): service STP decodes between long prefills * perf(b200): probe TP2 DSpark with bounded prefill memory * perf(b200): schedule DSpark decode between long prefills * perf(b200): compare GPU-resident Engram for TP4 DSpark * fix(b200): release completed MXFP4 loader temporaries * fix(b200): mark loader patch utility executable * perf(b200): test more DSpark SWA prefix tails * fix(b200): avoid TP2 loader allocation fragmentation * perf(b200): compare DSpark prefill service at interval four * perf(b200): compare host DSpark with larger SWA retention * perf(b200): qualify TP2 DSpark across concurrency * perf(b200): test more TP2 SWA prefix retention * docs(b200): document TP2 loader memory patch waiver * perf(b200): select DSpark curve with concurrency-scaled SWA capacity * perf(b200): qualify stock TP2 loader with expandable allocator * Test stock B200 loader on latest SGLang nightly AI-assisted by gpt-6-astra with high reasoning. * Document latest-nightly stock B200 loader qualification * refactor: remove redundant B200 recipe startup checks * [B300][SGLang][AgentX] Improve DeepSeek V4.1 Flash performance (#3342) * feat(b300): add DeepSeek V4.1 SGLang nightly AgentX recipe * fix(b300): record recipe pull request provenance * feat(b300): add non-speculative SGLang AgentX arm * docs: consolidate performance changelog into one PR entry * fix(b300): preserve native DSpark FP8 draft projections * fix(b300): normalize strided native FP8 projection inputs * fix: restore default SGLang DSpark precision * perf(b300): expand DSpark parallelism and interleave decode * Launch B300 SGLang AgentX through an owned Slurm batch job * Match B300 batch PMI environment to non-MPI container step * Use supported PMIx container steps for B300 AgentX batch jobs * perf(b300): scope final AgentX sweep to DSpark candidates * Test B300 low-concurrency prefix retention and checkpoint prefetch * Scale B300 TP4 prefix retention within measured KV budget * Pin B300 to the September 22 SGLang nightly * Test larger B300 TP2 KV budget for high concurrency * Increase high-concurrency TP2 cache memory on B300 * Screen higher prefill duty for B300 TP2 DSpark * perf(amd): tune MiniMax-M3 ATOM AgentX memory, graphs, indexer CP, and offload / 调优 MiniMax-M3 ATOM AgentX 显存、图捕获、Indexer CP 与卸载 (#3189) * perf(minimaxm3-atom): raise util, capture the real batch sizes, add indexer CP and a TP2 offload curve Six changes to the MiniMax-M3 MXFP4 ATOM agentic recipe, all measured on 8xMI355X against the same cc-traces replay: - gpu-memory-utilization 0.90 -> 0.95. - Declare cudagraph capture sizes. The stock list pads a running batch of 9 to 16 and 17 to 32; this workload keeps only 22-41% of the offered concurrency in decode at any instant, so those are the batches that actually replay. - Enable ATOM_M3_INDEXER_CP on TP4 past CONC=24. Each rank scores all four index heads over 1/TP of the blocks instead of one head over all of them. - Restore EAGLE3 on the CONC=40/48 bands. The forward step is 37-41 ms with or without it, so dropping it handed back the whole acceptance-length multiplier. - Route CONC=20/25/30 to the chunk-256 + ATOM_SLRU offload tier and derive LMCACHE_LOOKUP_SERVER_WORKER_IDS from TP instead of hardcoding four ranks. - Spread TP2 offload ranks across NUMA nodes; pinning 256 GB/rank takes 45 min with both ranks on node 0 and 21 s with one per node. Matrix: drop TP2 c5, add the TP2 c20/25/30 offload curve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(minimaxm3-atom): drop the dead per-band knobs The per-CONC case set MAX_NUM_SEQS, MAX_NUM_BATCHED_TOKENS and GPU_MEM_UTIL in all seven bands and two lines later overwrote all three unconditionally: 21 of its 49 assignments could never be read. What actually survived it was NUM_SPEC_TOKENS, SPEC_DECODE_AL and STATE_OFFLOAD_CPU_GIB, and after restoring EAGLE3 on 40/48 only CONC=56 differs on the first two. Collapsed to defaults plus three overrides; STATE_OFFLOAD_CPU_GIB keeps its exact per-band values. ATOM_ENABLE_REPLAYSSM is removed rather than collapsed: it is only read by gdn_attn, which gates the gated-delta-net path, and MiniMax-M3 is pure attention with no SSM layers, so the value never reached anything. Verified by replaying every point in the amd-master matrix (19 combinations of TP, CONC and offload) through a stubbed benchmark_lib and diffing the resulting server argv plus every exported env var: byte-identical apart from the removed variable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor(minimaxm3-atom): one table per concurrency, and drop the tier M3 cannot use The hybrid offload tier is unreachable for this model. ATOM's M3 connector is PAGE-only -- the docstring says MiniMax-M3 has no recurrent state and no SLOT sidecar, and it raises if slot_regions is non-empty -- so OFFLOAD_STATE and the staging knobs behind STATE_OFFLOAD_CPU_GIB never applied. Its LMCACHE_CHUNK_SIZE is justified by "block-size(128) x dcp(8)", but this model's search space sets no dcp-size at all. Removed, along with the comment claiming M3 carries a per-request recurrent state. An unlisted concurrency now fails instead of silently taking a tier that does not fit. CONC=56 is removed for the same reason: nothing in the matrix runs it here (the 56 in this file's neighbourhood belongs to Kimi K3, which has its own script), and it was the only band that disabled the draft model. With it gone NUM_SPEC_TOKENS is constant, so the guard around SPEC_ARGS goes too. The remaining per-concurrency facts now live in one table. Previously the CP gate was a numeric comparison and the offload tier an enumeration, so adding a concurrency above 24 would silently enable indexer CP on a point nobody measured, while forgetting the tier failed loudly. Both are declared per band now: the loud failure is kept, the silent one becomes a missing speedup. Verified the same way as the previous cleanup: all 19 matrix points replayed through a stubbed benchmark_lib, server argv and every exported env var byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * perf(minimaxm3-atom): spread TP4 offload ranks across NUMA nodes, move to the rocm7 nightly Pinning 256 GB/rank with all four TP4 ranks on NUMA node 0 takes 27m08s (12:13:10 "NUMA mapping" -> 12:40:18 "Created backend: LocalCPUBackend"); the same pinning with TP2 split one rank per node takes 21 s. TP4 now selects 0,1,4,5 for the same reason TP2 selects 0,4. Only applied when the caller has not already pinned a device set, and only on the offload path -- a run without the CPU tier pins no host memory. The image moves to the rocm7 line of ATOM's nightly release (232f62e3), which carries the M3 dense-split work. Checked against the measured baseline rather than assumed: triton 3.7.0 and torch 2.10.0+rocm7.2.4.git3d3aa833 are identical to the image every number in this PR was measured on, which matters because the rocm10.0 line of the same build carries triton 3.8.0 and that version regresses this model's Gluon kernels. LMCache in the new image is 0.5.5rc3, not the 0.4.5 the matrix declared, so the two MiniMax-M3 offload rows are corrected; the other models keep their own versions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(perf): record MiniMax-M3 AgentX tuning sweep 记录 MiniMax-M3 ATOM AgentX 调优、矩阵变化、测试结果与已知 CPU DRAM 元数据限制，以便上游 PR 启动完整 AgentX sweep。 * docs(perf): link MiniMax-M3 tuning pull request 将 MiniMax-M3 ATOM AgentX 性能 changelog 的占位链接替换为上游 PR #3189。 * fix(ci): provide exact PR context to priority classifier Supply immutable base/head revisions and the permitted diff command, document read-only tool use, and allow 16 turns for structured classification. 为优先级分类器提供确切的 Base/Head 版本及允许执行的 Diff 命令，明确只读工具用法，并将结构化分类的轮数上限调整为 16。 * ci(agentx): run MiniMax-M3 throughput only 禁用本次 changelog 的自动 vendor eval，只保留 19 个 AgentX 吞吐任务，避免为每个并发额外展开评测。 * perf(minimaxm3-atom): extend indexer CP down to CONC=15 on TP4 The gate was CONC > 24, chosen from a kernel microbenchmark over uniform contexts. Three same-commit A/B pairs (1800 s per arm, TP4 + EAGLE3) put the real crossover below 15: CONC 15 interactivity p90 +3.7% ISL-normalised throughput +0.67% CONC 24 +12.3% +0.24% CONC 28 +16.7% +2.7% Monotone and never negative. Throughput is neutral everywhere (only CONC=28 clears the 0.6% noise floor) while interactivity rises with the running batch, which is what CP actually shortens. Prefix hit rates are identical within each pair (96.7/96.7, 97.5/97.5, 97.2/97.3), so the gain is not a caching artifact. CONC=20 is interpolated between the two measured points, not run on its own. Nothing below 15 is measured and the microbenchmark is negative at batch ~2, so the band stops there. CONC=20 appears in both search-space rows, so it is split out of the offload enumeration: TP4 gets CP without a tier, and the tp_size == 4 gate keeps CP off the TP2 offload curve. 25 and 30 are TP2-only and stay without a CP declaration, which would never take effect but would misreport the point. Verified with the 19-point argv+env replay: the only difference against the previous revision is ATOM_M3_INDEXER_CP="1" on tp4_c15, tp4_c20 and tp4_c24. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(perf): record expanded MiniMax-M3 indexer CP band 记录 TP4 C15/C20/C24 新增 indexer-only CP 的 A/B 证据，并说明 TP2 C20 仍由 TP-size gate 保持关闭。 * ci: restore default PR priority classifier Revert the 16-turn exact-diff prompt so this recipe PR does not change shared sweep CI. 还原默认的优先级分类器配置，避免本配方 PR 修改共享 sweep workflow。 Co-authored-by: Cursor <cursoragent@cursor.com> * perf(minimaxm3-atom): switch to official nightly_202609170640_minimax_m3 image Replace the developer-named ATOM nightly so this submission uses an official tag. 将 ATOM 镜像替换为不含开发者用户名的官方 tag nightly_202609170640_minimax_m3。 Co-authored-by: Cursor <cursoragent@cursor.com> * switch to 0917 nightly atom image * perf(minimaxm3): enable agentic eval by removing no-evals opt-out Drop `no-evals: true` from the minimaxm3-fp4-mi355x-atom-agentic-mtp changelog entry so plan generation no longer skips eval rows; the model's automatic minimax-vendor (minimax_m3_full) agentic eval then runs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs(perf): correct MiniMax-M3 ATOM AgentX changelog image to nightly_202609171455 Align the perf-changelog entry with the recipe's actual image (rocm/atom-dev:nightly_202609171455 in configs/amd-master.yaml), replacing the stale nightly_202609170640_minimax_m3 reference in both the English and Chinese descriptions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(minimaxm3): size LMCache CPU pool per-rank from TOTAL_CPU_DRAM_GB/TP The cpu256 offload tier hardcoded LMCACHE_MAX_LOCAL_CPU_SIZE=256 per rank, independent of the matrix generator's TOTAL_CPU_DRAM_GB budget. Derive it as TOTAL_CPU_DRAM_GB / TP so the per-rank pool respects the declared budget and is TP-independent (node_DRAM * dram-utilization / 8). Bump the atom arm's dram-utilization 0.20 -> 0.687 so the offload arms land ~256 GB/rank. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update perf-changelog.yaml * docs(minimaxm3): expand ATOM AgentX changelog and record EAGLE3-GQA draft online-quant finding Update the minimaxm3-fp4-mi355x-atom-agentic-mtp changelog entry to reflect every change in SemiAnalysisAI/InferenceX#3189 and to record the EAGLE3-GQA draft online-quant verification: - Raise ATOM GPU memory utilization 0.90 -> 0.95 and declare a fine-grained CUDA graph ladder (capture sizes 1-64) sized for the 22-41%-of-CONC running batch, trimmed by ModelRunner to min(2*CONC, 8192). TP4 C24 A/B: throughput +9.2%, interactivity +50.7%, unchanged estimated graph memory. - Enable ATOM_M3_INDEXER_CP for TP4 C15/C20/C24/C28/C32/C40/C48; the TP-size gate (tp_size == sparse_num_index_heads == 4) keeps it off the TP2 C20 offload point. Restore EAGLE3 K3 (golden AL 2.78) at C40/C48. - Collapse per-band knobs into one case; add the TP2 LMCache DRAM-offload curve at C20/C25/C30 on the chunk-256 ATOM_SLRU tier; size LMCACHE_MAX_LOCAL_CPU_SIZE as TOTAL_CPU_DRAM_GB/TP; cover every rank in LMCACHE_LOOKUP_SERVER_WORKER_IDS. Remove TP2 C5 and the unreachable hybrid/state-offload and C56 branches. - NUMA-spread offload GPUs (0,4 for TP2; 0,1,4,5 for TP4), cutting TP2 host-memory pinning from 45 min to 21 s. Move to rocm/atom-dev:nightly_202609171455 and LMCache 0.5.5rc3+rocm7.2.4 on both offload arms. - Set dram-utilization to 0.687 so the recorded budget matches the real ~256 GB per-rank pinning (512 GB total TP2, 1,024 GB TP4), independent of TP, replacing the earlier 0.20 metadata that under-described the allocation. - Verified against ATOM commit b6e2a39433d73b586c366ca90e661f098eb463e3 that --online_quant_config ptpc_fp8 does not change the Inferact/MiniMax-M3-EAGLE3-GQA draft precision: eagle3_llama.py builds every draft linear (qkv_proj, o_proj, gate_up_proj, down_proj, fc) with no quant_config, so should_stream_online_quant short-circuits to False and OnlineQuantStreamer.maybe_create finds no candidates. The draft runs at native bf16; only the target MiniMax-M3 weights are online- quantized to ptpc_fp8, and exclude_layer applies to the target alone. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Wang Yiting <yitiwang@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: billishyahao <bill.he@amd.com> Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: seungrokj <seungrok.jung@amd.com> * feat(agentx): retune GLM5.2 FP4 MI355X ATOM recipe on _0921 docker (#3359) * feat(agentx): retune GLM5.2 FP4 MI355X ATOM recipe on _0921 docker Signed-off-by: zhuyuhua-v <yuhzhu@amd.com> * docs(glm5.2): note MTP block stays BF16 under --online_quant_config (PR #3359) Changelog for the GLM-5.2 FP4 MI355X ATOM AgentX recipe retune: - Track the updated GLM-5.2 standalone recipe (ROCm/ATOM PR 2345) on the MI355X ATOM AgentX config. - Bump the image from rocm/atom-dev:nightly_202609151450 to rocm/atom-dev:nightly_202609211553, which carries ROCm/ATOM PR 2345. - Drop the LMCache DRAM offload from the small-concurrency TP4 arm: conc [2, 4, 8, 10] moves from kv-offloading dram with the lmcache 0.4.5 backend to kv-offloading none, so those points serve off the native GPU prefix cache alone and no longer start an LMCache CPU tier. The TP4+DCP4 arm at conc [16, 24, 32, 40, 48] keeps LMCache at 256 GiB per rank (dram-utilization 0.171 unchanged), and the TP8 arm stays GPU-resident as before. - Add --index_cache_dtype fp4 and --block-size 64 to the ATOM server launch on every point. Neither flag was set before, so both took the ATOM default. - Drop ATOM_USE_FLYDSL_GATHER_KV_B_PROJ=0, which pinned gather_kv_b_proj to the Triton path on the DCP prefill-context and MTP verify paths; the updated recipe no longer sets it. - The MTP block data precision is not touched by --online_quant_config: the exclude_layer list adds "model.layers.78.*" so layer 78 (the GLM-5.2 MTP head) is skipped by online quantization and stays in native BF16, as it ships unquantized in amd/GLM-5.2-MXFP4. The expert excludes only reach layers 0-77, so without this exclude the whole MTP block would be online-quantized to ptpc_fp8 while the target's experts stay MXFP4. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Signed-off-by: zhuyuhua-v <yuhzhu@amd.com> Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: seungrokj <seungrok.jung@amd.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * config(dsv41flash): sweep MI355X at TP=2 and TP=4 through concurrency 128 / MI355X DSv4.1-Flash 在 TP=2 与 TP=4 下扫描至并发 128 (#3326) * config(dsv41flash): sweep MI355X at TP=2 and TP=4 through concurrency 128 Add a TP=2 search space next to TP=4 for the MI355X DSv4.1-Flash AgentX entry, and extend both rows to concurrency 128. TP=2 became feasible once the Engram tables moved to host memory: at 47.2 GiB of device memory per rank they previously forced four GPUs per server, and two GPUs per server doubles the servers per node. The image moves to the ROCm 10 nightly and carries a placeholder tag until that build is published. The digest is pinned before the sweep runs, which is why the PR opens as a draft. 中文：为 MI355X DSv4.1-Flash AgentX 配置在 TP=4 之外新增 TP=2 搜索空间，并将两者 的并发扩展到 128。Engram 表已驻留主机内存，不再占用每 rank 47.2 GiB 设备内存 （此前这迫使每个服务器使用四张 GPU），因此 TP=2 可行，且每节点可容纳的服务器数 量翻倍。镜像切换到 ROCm 10 nightly，在该构建发布前暂用占位 tag；digest 固定后再 运行 sweep，因此本 PR 以 draft 形式提交。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(dsv41flash): record the sweep PR link in the changelog 中文：在 changelog 中记录本次 sweep 的 PR 链接。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(dsv41flash): let vLLM pick max_num_seqs for the MI355X AgentX recipe A fixed max_num_seqs of 128 caps in-flight sequences at the outer concurrency once the sweep reaches c128, leaving no headroom for AgentX subagent fan-out. Use the MI355X API-server default of 1024 instead, and derive the CUDA graph ceiling from max(2 * CONC, 128): 1024 through c64, 2048 at c128. Supersedes PR #3111, which carried this recipe change and was closed in favour of this sweep. 中文：扫描到 c128 时，固定的 max_num_seqs 128 会把在途序列数限制在外层并发上， 使 AgentX 子代理扇出没有余量。改用 MI355X API server 默认值 1024，并由 max(2 * CONC, 128) 推导 CUDA graph 上限：c64 及以下为 1024，c128 为 2048。 本次提交取代已关闭的 PR #3111（该 PR 原本包含此 recipe 改动）。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(dsv41flash): keep the placeholder on the existing ROCm nightly channel Per #3215, the MI355X DSv4.1-Flash entry stays on the existing upstream ROCm nightly channel rather than moving to nightly-rocm100. Drop the rocm10 marker from the placeholder tag; the pin is filled in with the nightly this sweep actually runs on before dispatch. 中文：依据 #3215，MI355X DSv4.1-Flash 条目继续使用现有的上游 ROCm nightly 渠道， 不切换到 nightly-rocm100。占位 tag 去掉 rocm10 标记；派发前再填入本次 sweep 实际运行的 nightly。 Co-authored-by: Cursor <cursoragent@cursor.com> * Revert "config(dsv41flash): keep the placeholder on the existing ROCm nightly channel" This reverts commit 039194e50d92a80c847d770729e106237e2b36c4. The MI355X DSv4.1-Flash AgentX sweep runs on the ROCm 10 nightly after all, so restore the nightly-rocm100 placeholder. The pin is filled in with the published nightly before dispatch. 中文：还原提交 039194e50d92a80c847d770729e106237e2b36c4。 MI355X DSv4.1-Flash AgentX sweep 最终仍在 ROCm 10 nightly 上运行，因此恢复 nightly-rocm100 占位 tag。派发前再填入已发布的 nightly。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(dsv41flash): pin the MI355X AgentX sweep to the ROCm 10 nightly Replace the placeholder with vllm/vllm-openai-rocm:nightly-rocm100-3df4ae153eb385e27b52f26c81f8edb9e20b9984, published 2026-09-21T05:50:03Z with digest sha256:eccb72b74b8c04ce7406d9212e200b129f58e24be642795a857f755a94fdb3a1. The tag is active and linux/amd64, and its manifest resolves from registry-1.docker.io at that digest. This sweep qualifies the new image; the previous green run on nightly-eed1f3d0 is not evidence for it. 将占位 tag 替换为 vllm/vllm-openai-rocm:nightly-rocm100-3df4ae153eb385e27b52f26c81f8edb9e20b9984， 该镜像于 2026-09-21T05:50:03Z 发布，digest 为 sha256:eccb72b74b8c04ce7406d9212e200b129f58e24be642795a857f755a94fdb3a1。 该 tag 状态为 active、架构为 linux/amd64，其 manifest 可从 registry-1.docker.io 按该 digest 解析。 本次 sweep 用于验证新镜像；此前在 nightly-eed1f3d0 上的绿色运行不能作为其证据。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(dsv41flash): offload Engram to host memory at TP=2 on MI355X vllm-project/vllm#57491 widened the two ROCm is_cuda() gates to is_cuda_alike(), so the pinned nightly resolves an Engram config on gfx950 and offloads the tables to pinned host memory by default. Set cpu_offload explicitly per TP instead of taking that default. TP=2 needs the offload: the tables cost 47.2 GiB per rank at TP=4, so 94.4 GiB at TP=2, which does not fit beside half of the 511 GB checkpoint on a 288 GiB card. TP=4 keeps them resident so it stays comparable with the validated concurrency 1-32 run. vllm-project/vllm#57491 将 ROCm 上的两处 is_cuda() 判断放宽为 is_cuda_alike()，因此所固定的 nightly 在 gfx950 上会解析 Engram 配置， 并默认将表下放到锁页主机内存。这里按 TP 显式设置 cpu_offload，而不是 沿用该默认值。 TP=2 需要该下放：表在 TP=4 时每 rank 占用 47.2 GiB，TP=2 时为 94.4 GiB， 无法与 511 GB 检查点的一半同时放入 288 GiB 的单卡。TP=4 保持表常驻 GPU， 以便与已验证的并发 1-32 运行保持可比。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(dsv41flash): disable SWA bounded replay on MI355X Every TP=2 and TP=4 point of run 35567570539 died with HSA_STATUS_ERROR_MEMORY_FAULT, taking the worker and then the engine core down and aborting AgentX warmup. The fault follows the first launch of _pad_replayed_slots_kernel in all 16 server logs, which is the first prefix hit carrying a replay start. vllm-project/vllm#56227 added SWA bounded replay, default on, after the eed1f3d0 pin and before nightly-rocm100-3df4ae153. It pads the replayed tokens' slots in the prefix-cacheable groups, but the window clamp it relies on landed in the FlashInfer and FlashMLA kernels; the ROCm sparse SWA path only gained the replay_start kwarg, and consumes it on the decode path while the fault is in prefill. Pass --no-swa-bounded-replay until ROCm clamps too. Prefix caching itself stays on. 运行 35567570539 的所有 TP=2 与 TP=4 数据点都以 HSA_STATUS_ERROR_MEMORY_FAULT 崩溃，先后带崩 worker 与 engine core， 并中止 AgentX warmup。16 份 server 日志中，该故障均紧随 _pad_replayed_slots_kernel 的首次启动，即首个带 replay start 的前缀命中。 vllm-project/vllm#56227 在 eed1f3d0 与 nightly-rocm100-3df4ae153 之间 引入了默认开启的 SWA bounded replay。它会填充被重放 token 在可前缀缓存 分组中的 slot，但其依赖的窗口钳制只落在 FlashInfer 与 FlashMLA 内核中； ROCm 稀疏 SWA 路径仅新增了 replay_start 参数，且只在 decode 路径消费它， 而故障发生在 prefill。在 ROCm 同样实现钳制之前传入 --no-swa-bounded-replay。前缀缓存本身保持开启。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(dsv41flash): fix the MI355X KV split at high concurrency Concurrency 64 collapsed on both arms of run 35574132719 because the KV pool was too small to hold the AgentX working set. TP=2 fell to a 17.6% prefix cache hit rate, 187 s TTFT and 150 tok/s, against 94.8%, 1.3 s and 957 tok/s at c32. Two settings were spending device memory that the KV pool needed. The Engram tables stayed resident at TP=4. vllm-project/vllm#57491 widened the two ROCm is_cuda() gates to is_cuda_alike(), so gfx950 can now offload them to pinned host memory as every NVIDIA arm has since #2963. Measured here at TP=4 with 16384 batched tokens, resident tables leave 37.96 GiB of KV cache and 14.15x maximum concurrency at 1M context, against 84.54 GiB and 31.52x offloaded. The sparse-attention indexer and its companion per-rank buffers scale at roughly 4.4 MiB per batched token, so the upstream 16384 was the larger cost. Size it per TP instead, 4096 at TP=2 and 8192 at TP=4, and cap max_num_seqs at the shape graph capture already covers rather than the MI355X API-server default of 1024. TP=2 then keeps 79.34 GiB (39.44x) and TP=4 keeps 121.03 GiB (54.15x), against the B300 arm's 132.48 GiB (60.17x). B300 runs 8192 at TP=4 and the Blackwell TP=2 arms run 4096 (#3320, #3321). Sweep concurrency 64 and 128 to confirm the high end first. Concurrency 1-32 is backfilled once those land, since every point now runs the new memory split. Co-authored-by: Cursor Agent <cursoragent@cursor.com> Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> * config(dsv41flash): change the MI355X memory split only where KV ran short The previous commit applied one memory split per TP across every concurrency. Concurrency 1-32 never ran short of KV, so it paid for changes it did not need: offloaded Engram lookups go to pinned host memory over UVA, and a smaller prefill chunk costs TTFT. Scope each change to the points that were actually starved, which also leaves concurrency 1-32 and TP=4 c64 byte-identical to the settings run 35574132719 already measured on this image and flag set, so those points are combined with this sweep rather than re-run. Dividing measured KV tokens by concurrency gives the per-request budget and separates the failures cleanly. Every point that held had 232K or more; TP=2 c64 collapsed at 122K. TP=4 c1-c64 resident, 16384 14.83M tokens 232K and up TP=4 c128 offload, 8192 49.63M 388K TP=2 c1-c32 offload, 16384 7.84M 245K and up TP=2 c64 offload, 8192 24.68M 386K TP=2 c128 offload, 4096 33.34M 260K Probing the configurations directly also turned up two things. TP=2 c128 at 4096 with the API-server default of 1024 sequences faults during profiling with HSA_STATUS_ERROR_EXCEPTION at M=1024, N=129280, K=256, reproduced on two GPU pairs: DSpark verifies 1+5 tokens per sequence, so a decode batch of max_num_seqs needs six times that many token slots and 4096 leaves four. Cap max_num_seqs at the graph-capture shape wherever the chunk falls below that bound, which is TP=2 c128 alone. And the reason c128 never completed before is that at 16384 with capture at 2048, TP=2 c128 held 7.72 GiB, 3.02M tokens, 23.6K per request. Co-authored-by: Cursor Agent <cursoragent@cursor.com> Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> * config(dsv41flash): measure the full MI355X concurrency range on the new pin * docs(dsv41flash): retire the MI355X draft status and repoint GPU validation The MI355X section still described a draft recipe at TP4 concurrency 1-32 and cited run 34710937012 on nightly-eed1f3d0 as the pinned image. Both predate this branch: the arm now sweeps TP4 and TP2 at concurrency 1-128 on nightly-rocm100-3df4ae15. Lead with the current pin, keep the earlier run only as the superseded-pin note explaining why its points do not carry onto this image, and add the merged upstream recipe #1006 beside #968. MODELS.md and MODELS_zh.md carried a second DeepSeek-V4.1-Flash row holding the MI355X arm as pending GPU validation. Drop it; the active row already covers the model. --------- Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com> Co-authored-by: Chun Fang <chun.fang@amd.com> * [AMD] Update GLM-5.2 MI355X image, HiCache capacity, and sweep / 更新 GLM-5.2 MI355X 镜像、HiCache 容量与 sweep (#3329) * perf: update GLM-5.2 ROCm image and HiCache path 将 GLM-5.2 ROCm 镜像更新到 20260920，切换 HiCache kernel/page_first，并调整并发矩阵。 * perf: split GLM-5.2 low and high concurrency points TP8/EP1 仅保留并发 1、2、4；TP4/EP4 HiCache 仅运行并发 12、14、16，并补充实际 PR 链接。 * Keep existing GLM-5.2 HiCache settings * Use default GLM-5.2 HiCache backend and layout * Add missing GLM-5.2 concurrency points * perf: increase GLM-5.2 HiCache capacity 将 GLM-5.2 TP4/EP4 HiCache 调整为每 rank 180 GB，并将 SA DRAM 利用率提高到 0.85。 * fix changelog --------- Co-authored-by: Chun Fang <chun.fang@amd.com> * fix(review): bind verifier verdicts to signoffs (#3371) 让清单编辑触发验证，并按签署资源隔离验证评论。 * [TileRT] GLM-5.3 MI355X AgentX parallel KV inject experiments, reverted pending tile-ai/TileRT#66 / GLM-5.3 MI355X AgentX 并行 KV 注入实验（已回退，待 tile-ai/TileRT#66） (#3376) * [GB300][SGLang][AgentX] Improve DeepSeek V4.1 Flash performance (#3344) * perf(gb300): add nightly SGLang V4.1 Flash DSpark candidates * chore(gb300): link performance changelog to PR 3344 * perf(gb300): add non-speculative SGLang AgentX candidates * chore(gb300): consolidate performance changelog entry * perf(gb300): resolve local SGLang checkpoint for Engram cache advice * perf(gb300): retain native FP8 DSpark draft projections * fix(gb300): make native draft FP8 inputs contiguous * perf(gb300): test GPU-resident Engram tables on TP4 * Use shipped nightly DSpark precision on GB300 * Interleave GB300 STP decode with long prefills * perf(gb300): prioritize DSpark across the AgentX curve * perf(gb300): test larger SWA prefix cache for DSpark * Test larger SWA cache on GB300 TP2 DSpark * Select DSpark and scale SWA retention for the GB300 sweep * perf(gb300): select qualified prefill capacity on latest nightly * chore(gb300): remove documentation changes from performance PR * [H200][SGLang][AgentX] Improve DeepSeek V4.1 Flash performance (#3341) * perf(h200): qualify DeepSeek V4.1 Flash on latest SGLang nightly * chore(h200): link performance qualification PR * perf(h200): enable huge pages for Engram host shards * perf(h200): add native non-speculative SGLang throughput arm * chore(h200): consolidate performance changelog entry * fix(sglang): retain native FP8 DSpark attention projections * style(sglang): sort native projection patch imports * chore(sglang): store context-free native projection patch * fix(h200): make native draft FP8 inputs contiguous * perf(h200): test GPU-resident Engram for non-speculative serving * perf(h200): interleave STP decode with long prefix prefill * fix: restore default SGLang DSpark precision * perf(h200): add TP4 host Engram throughput comparison * perf(h200): prioritize TP4 DSpark and prevent prefill starvation * perf(h200): tune block32 kernels and extend DSpark concurrency * perf(h200): compare native Marlin MoE with stock DSpark * perf(h200): retain DSpark after non-speculative comparisons * perf(h200): retain more SWA prefixes within the KV budget * perf(h200): select measured DSpark MoE backends by TP * perf: consolidate H200 DSpark topology and launch tuning * perf: qualify H200 DSpark with exact EP1 baseline topologies * perf: select complete H200 EP1 qualification grid * fix(h200): allow full high-concurrency AgentX warmup * [CI] select Python 3.12 for result processing / 固定结果处理 Python 3.12 (#3397) * fix: select Python 3.12 for result processing 中文：在分配 GPU 前准备独立的结果处理 Python，并在固定序列和 AgentX 路径中显式使用；配置缺失时仍保留失败产物。 * fix: use managed Python for H200 AgentX power results 在 H200 AgentX DCGM 结果处理路径中校验并使用受管 Python 解释器，保留失败产物；补充回归测试和中英文恢复说明。 * docs(review): explicitly ban FP8 NextN MoE flag (#3403) 明确禁止启用 SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE，并同步中英文贡献指南及审阅清单。 * [TileRT] Bump GLM-5.3 FP8 MI355X AgentX to tilert 0.1.6.post2 / GLM-5.3 FP8 MI355X AgentX 升级至 tilert 0.1.6.post2 (#3389) * [TileRT] Bump GLM-5.3 FP8 MI355X AgentX to tilert 0.1.6.post2 * Set changelog pr-link * [MI355X][SGLang][AgentX] Improve DeepSeek V4.1 Flash performance Add the native SGLang DeepSeek V4.1 Flash DSpark recipe for MI355X, with radix caching enabled and supported long-context memory limits. Validated by official sweep 35943469224: six canonical performance cells and six full GSM8K evaluations. Reuse this sweep for production publication. * [GB200][SGLang][AgentX] Improve DeepSeek V4.1 Flash performance Qualify the native SGLang DeepSeek V4.1 Flash DSpark recipe on GB200 with radix caching and full context. Official sweep 35876607563 passed sixteen canonical performance cells and sixteen full GSM8K evaluations across TP2/EP1 and TP4/EP1 at C1 through C128. Reuse the qualified sweep for production publication. * [H200][SGLang][AgentX] Qualify DSpark decoder SWA bounded replay (#3392) Enable decoder SWA bounded replay and pin the qualified SGLang nightly for H200 DSpark. Reuse full sweep 35893977246: 16 canonical performance cells and 16 complete GSM8K evaluations passed; final staging verified. * [H100][SGLang][AgentX] Improve DeepSeek V4.1 Flash performance (#3345) Restore the first fully qualified H100 SGLang recipe and pinned September 22 image, including TP8/EP8 and DP8 coverage. Reuse successful full sweep 35690159495. Preserve native context, prefix caching and shipped DSpark precision. * refactor: migrate fixed-sequence recipes to SRT-Slurm / 将定长配方迁移至 SRT-Slurm (#3352) * refactor: start native single-node SRT-Slurm migration 开始原生单节点 SRT-Slurm 迁移，添加 H200 SGLang 8k1k 并行候选配方并复用现有基准客户端；生产路由保持不变。 * docs: link single-node migration changelog to draft PR 为单节点迁移的 changelog 条目补充草稿 PR 链接。 * feat: wire native H200 SRT pilot into end-to-end workflow 将原生 H200 SRT 试点接入端到端工作流，显式校验配方、保留结果和功耗产物，并保持生产路由不变。 * test: cover native single-node job failures and artifacts 覆盖原生单节点作业的提交失败、Slurm 失败、产物保留及定向取消行为。 * fix: drop unused AIPerf inputs from SRT setup 删除 SRT 准备阶段未使用的 AIPerf 排空参数要求，使固定序列工作流能进入实际提交；新增缺省参数回归覆盖。 * fix: let native SRT stage the pilot container image 将配方指定镜像直接交给原生 SRT/Pyxis，移除试点对旧 squash 缓存就绪状态的依赖，保留模型预检查。 * fix: bootstrap native SRT binaries before pilot submission 在试点提交前执行原生 SRT 二进制准备，并覆盖准备失败时不提交作业的行为。 * feat: port H200 MTP and Qwen recipes to native SRT 将 H200 DeepSeek-R1 MTP 与 Qwen3.5 EP8 配方迁移到并行原生 SRT 试点，保留聊天模板及随并发变化的图捕获，并使用 4/16/64 逐点验证回归。 * docs: remove migration guide 移除独立迁移指南及入口链接，迁移范围与验证证据保留在 PR 中。 * feat: migrate fixed-sequence SGLang recipes to native SRT 将 H100、H200、B200 和 B300 的 21 个活跃 SGLang 定长配方及全部 181 个配置点切换到原生 SRT-Slurm。共享提交、eval 和产物处理，并保留逐点拓扑与推测解码设置。 * feat: migrate fixed-sequence TRT recipes to native SRT 将八个定长 TRT 配方迁移至原生 SRT，保留 63 个测试点的引擎参数、客户端和 eval token 预算。 * feat: convert remaining fixed-sequence recipes to native SRT 将剩余 AMD SGLang/ATOM 与 RTX 定长配方切换到原生 SRT 配置，保留测试点及服务参数；固定直接 ATOM 服务草稿依赖并扩展行为验证。 * fix: forward native Docker eval model identity 为 RTX 原生 Docker 路径显式传递 eval 模型名称，并验证实际启动器的命令、失败传播及容器清理。 * fix: apply AMD container options as native leaf overrides 按原生 SRT 语义逐项传递容器选项，增加映射行为回归测试并输出提交失败原因。 * refactor: reuse native job workspace for AMD scratch files 复用作业检出目录存放 AMD 原生运行时临时文件，由现有 runner 清理流程回收，移除额外 scratch-root 配置。 * fix: complete AMD native launch and terminal status handling 修复 AMD 原生启动命令的尾部换行，在 accounting 不可用时通过 Slurm 控制器验证终态，并等待启动失败的日志进程退出。 * refactor: require native recipes for the Slurm fixed-sequence cutover 移除 Docker 迁移支持和 50 个已被原生配方替代的定长 Bash 实现。Slurm 定长作业必须提供 SRT 配方，保留 AgentX、多节点和显式采集器的独立执行路径。 * refactor: make single-node fixed-sequence coverage SRT-only 归档仅支持 Docker 的 RTX 定长配置及脚本，移除无调用方的 runner 路由；保留历史 changelog 条目，但不为已归档的精确配置键生成作业。 * fix: allow client dependencies in container virtual environments 修复容器虚拟环境中的客户端依赖安装，移除不兼容的用户目录安装参数。 * fix: limit native single-node steps to serving GPUs 保留节点独占预留，同时将原生单节点服务和客户端步骤限制为配方指定的 GPU 数量，避免采集空闲设备功耗。 * fix: restore repository workdir for native container steps 原生容器步骤从已有仓库挂载目录启动，避免 Python 动态模块在根目录下导入失败，并保留调用方运行时覆盖。 * fix: stream AMD power samples without input buffering 逐行写出 AMD 功耗 CSV，避免 awk 输入缓冲在采样器关闭时丢失末尾数据；新增真实流式读取回归测试。 * fix: match registered H200 runner labels 将 H200 DGXC runner 清单对齐到实际注册的两位数字标签，恢复精确 smoke 选择和 sweep 调度。 * chore: pin srt-slurm submodule to main [skip ci] * refactor(srt): return single-node migration to NVIDIA upstream 将单节点迁移切回 NVIDIA 上游 srt-slurm。ATOM 使用原生 AToMesh 及单独固定的官方路由器镜像；TRT-LLM 使用原生模型名称转发。保留 worker 镜像、服务参数、并发范围与拓扑，不增加 Bash 分支。 * chore: bump srt-slurm submodule to v2.23.2 Move the shared SRT submodule from v2.22.1 (3cbc5dd) to upstream release v2.23.2 (8dace5f). Upstream delta is three additive changes: vllm-router multi-node hybrid-DP rank offsets, SGLang leader IP honoring the cluster network interface, and nsys/DSight coverage for SGLang and Dynamo. No recipe or connector changes required; utils/ and runners/ tests pass. * refactor(srt): check MI300X firmware in a setup hook Replace the MI300X container preamble with a host setup hook that fails on MEC firmware older than 177 instead of exporting HSA_NO_SCRATCH_RECLAIM in every container. * fix(srt): apply pending upstream srt-slurm patches at setup Post-eval steps dropped the recipe's srun_options, so ATOM evals on MI355X ran in a read-on…
…ud 基线预检并报告失败原因 (SemiAnalysisAI#3443) * fix: preserve Klaud baseline preflight and classify failures 在候选任务中复用规划阶段的基线预检，并记录脱敏失败原因。 * test: recognize Git options in SRT clone fixture 让 SRT 测试夹具识别 git -c 参数后的 clone 子命令。
* feat(srt): run single-node AgentX on native srt-slurm Single-node AgentX points with an srt-recipe now take the native single-node path fixed-sequence points already use. The adapter accepts AgentX points and DSpark speculation. Single- and multi-node AgentX share one client, benchmarks/srt_agentic.sh (moved from benchmarks/multi_node/agentic_srt.sh), which replays one CONC point or a CONC_LIST batch against the srt-slurm frontend and reads engine metrics from the workers behind a router frontend. The H200 DeepSeek-V4.1-Flash SGLang config is the first one ported. * fix(amd): drop a duplicate DSV4 prefill key and restore the MI355X Qwen3.5 fixed-sequence recipe srt-slurm parses recipes strictly and rejected the repeated disable-cuda-graph key. Removing the unported MI355X Qwen3.5 AgentX recipe had also deleted the fixed-sequence recipe beside it. * feat(srt): accept vLLM points in the single-node adapter vLLM points validate their topology as tensor x data parallel GPUs, with DP attention as data-parallel ranks and expert parallelism as enable-expert-parallel, and eval-only runs set max-model-len. * feat(agentx): port the DSV4.1 Flash B300 SGLang AgentX config to srt-slurm * feat(agentx): let srt-slurm AgentX recipes apply the chat template client-side Several legacy AMD AgentX scripts appended --apply-chat-template to the replay command. AIPERF_APPLY_CHAT_TEMPLATE=true in a recipe's benchmark env keeps that behavior on the shared srt-slurm client. * feat(agentx): run GB200/GB300 single-node AgentX natively and port DSV4.1 Flash SGLang there GB launchers submit recipe points through launch_srt_single_node with an aarch64 srt-slurm setup; the squash is imported on a compute tray first. * feat(agentx): let srt-slurm AgentX recipes apply the chat template client-side Several legacy AMD AgentX scripts appended --apply-chat-template to the replay command. AIPERF_APPLY_CHAT_TEMPLATE=true in a recipe's benchmark env keeps that behavior on the shared srt-slurm client. * feat(agentx): let srt-slurm AgentX recipes apply the chat template client-side Several legacy AMD AgentX scripts appended --apply-chat-template to the replay command. AIPERF_APPLY_CHAT_TEMPLATE=true in a recipe's benchmark env keeps that behavior on the shared srt-slurm client. * feat(agentx): let srt-slurm AgentX recipes set the AIPerf benchmark grace period The GLM-5.2 MI325X legacy script bounded the post-window drain with --benchmark-grace-period 1800. AIPERF_BENCHMARK_GRACE_PERIOD in a recipe's benchmark env passes it through the shared client. * feat(agentx): port the DSV4.1 Flash B200 vLLM AgentX config to srt-slurm * feat(srt): force ATOM AgentX golden acceptance with its server flag ATOM pins acceptance with --spec-decode-acceptance-length rather than an environment variable. The adapter now sets it from the golden curve for AgentX throughput and removes it for evals, reading the draft model for the MiniMax GQA curve and the probabilistic sampler for Kimi DSpark. * feat(srt): accept draft_model speculation in the single-node adapter DeepSeek-V4 configs mark their bundled DSpark draft as draft_model; the recipe still speculates natively, so the point binds as speculative. * fix(mi355x): mount the shared HF cache for native AgentX checkpoints The legacy MI355X AgentX path read DeepSeek-V4-Pro (vLLM/ATOM), DeepSeek-V4.1 Flash, MiniMax-M3 and GLM-5.2-FP8 from /it-share rather than node-local NVMe. The native srt-slurm path keeps that mount for those AgentX checkpoints. * feat(agentx): port the Qwen3.5 MI300X SGLang AgentX config to srt-slurm * feat(agentx): port the Qwen3.5 MI325X SGLang AgentX config to srt-slurm * feat(agentx): port the GLM-5.2 MI325X SGLang AgentX config to srt-slurm * feat(agentx): port the DSV4.1 Flash H100 SGLang AgentX configs to srt-slurm * feat(agentx): port the Qwen3.5 H100 SGLang AgentX configs to srt-slurm * feat(agentx): port the Qwen3.5 H200 SGLang HiCache EP1 AgentX config to srt-slurm * feat(srt): accept the draft_model label for single-node DSpark points * feat(agentx): port the DSV4.1 Flash MI300X vLLM AgentX config to srt-slurm * feat(agentx): port the DSV4.1 Flash MI325X vLLM AgentX config to srt-slurm * fix(srt): bind single-node draft_model and EAGLE3 AgentX points The DSV4 B300 DSpark config labels its points spec-decoding: draft_model and the MiniMax-M3 TRT-LLM config drafts with EAGLE3; accept both as speculative recipes. Golden acceptance already resolves both curves. * feat(srt): accept vLLM EAGLE3 speculation in the single-node adapter * feat(srt): accept EAGLE3 speculation in the single-node adapter MiniMax-M3 vLLM AgentX recipes speculate with the EAGLE3 GQA draft, which the golden acceptance lookup already maps to minimaxm3_eagle3_gqa. * feat(srt): accept EAGLE3 speculation in the single-node adapter MiniMax-M3 vLLM AgentX recipes speculate with the EAGLE3 GQA draft, which the golden acceptance lookup already maps to minimaxm3_eagle3_gqa. * feat(agentx): port the MiniMax-M3 MI325X vLLM AgentX config to srt-slurm * feat(agentx): port the Qwen3.5 FP4 B300 SGLang AgentX config to srt-slurm * feat(agentx): port the Qwen3.5 FP8 B300 SGLang AgentX config to srt-slurm * feat(agentx): port the GLM-5.2 FP4 B300 SGLang AgentX config to srt-slurm * feat(agentx): port the GLM-5.2 FP8 B300 SGLang AgentX config to srt-slurm * feat(agentx): port the DSV4 Pro B300 SGLang AgentX config to srt-slurm * feat(agentx): port the MiniMax-M3 B300 TRT-LLM AgentX config to srt-slurm * fix(b300): serve Qwen3.8-Flash-Next NVFP4 natively from the writable model root The checkpoint is not staged on node-local NVMe; the legacy AgentX path downloads it to the shared writable model directory. * feat(agentx): port the Qwen3.8-Flash-Next FP4 B300 SGLang AgentX config to srt-slurm * feat(srt): accept draft_model speculation in the single-node adapter DeepSeek-V4 configs mark their bundled DSpark draft as draft_model; the recipe still speculates natively, so the point binds as speculative. * feat(srt): force ATOM AgentX golden acceptance with its server flag ATOM pins acceptance with --spec-decode-acceptance-length rather than an environment variable. The adapter now sets it from the golden curve for AgentX throughput and removes it for evals, reading the draft model for the MiniMax GQA curve and the probabilistic sampler for Kimi DSpark. * fix(mi355x): mount the shared HF cache for native AgentX checkpoints The legacy MI355X AgentX path read DeepSeek-V4-Pro (vLLM/ATOM), DeepSeek-V4.1 Flash, MiniMax-M3 and GLM-5.2-FP8 from /it-share rather than node-local NVMe. The native srt-slurm path keeps that mount for those AgentX checkpoints. * feat(srt): accept draft-model labeled speculation in the single-node adapter The DSV4 B200 vLLM AgentX matrix labels its native DSpark drafter draft_model rather than mtp. * feat(agentx): port the DSV4.1 Flash H100 and H200 vLLM AgentX configs to srt-slurm * feat(agentx): port the MiniMax-M3 H100 and H200 vLLM AgentX configs to srt-slurm * feat(agentx): port the Qwen3.5 MI355X SGLang AgentX config to srt-slurm * feat(agentx): port the DSV4.1 Flash MI355X SGLang AgentX config to srt-slurm * feat(agentx): port the GLM-5.2 FP4 MI355X SGLang AgentX config to srt-slurm * feat(agentx): port the GLM-5.2 FP8 MI355X SGLang AgentX config to srt-slurm * feat(agentx): port the DSV4 MI355X SGLang AgentX config to srt-slurm * feat(agentx): port the DSV4 MI355X ATOM AgentX config to srt-slurm * feat(agentx): port the GLM-5.2 MI355X ATOM AgentX config to srt-slurm The DCP4 LMCache entry stays on the legacy script: srtctl reserves ATOM's kv-transfer-config for disaggregated workers. * feat(agentx): port the Kimi-K3 MI355X ATOM AgentX config to srt-slurm The DCP8 LMCache entries stay on the legacy script: srtctl reserves ATOM's kv-transfer-config for disaggregated workers. * feat(agentx): port the MiniMax-M3 MI355X ATOM AgentX config to srt-slurm The LMCache entries stay on the legacy script: srtctl reserves ATOM's kv-transfer-config for disaggregated workers. * feat(srt): accept EAGLE3 speculation in the single-node adapter * feat(agentx): port the DSV4.1 Flash B300 vLLM AgentX config to srt-slurm * feat(agentx): port the MiniMax-M3 B200 and B300 vLLM AgentX configs to srt-slurm * feat(agentx): port the DSV4 B200 and B300 vLLM AgentX configs to srt-slurm * feat(agentx): port the Qwen3.5 FP4 B200 SGLang AgentX config to srt-slurm * feat(agentx): port the Qwen3.5 FP8 B200 SGLang AgentX config to srt-slurm * feat(agentx): port the Qwen3.8 Next FP4 B200 SGLang AgentX config to srt-slurm * fix(h100): point native single-node uv caches at shared NFS * feat(agentx): port the GLM-5.2 FP4 B200 SGLang AgentX config to srt-slurm * feat(agentx): port the GLM-5.2 FP8 B200 SGLang AgentX config to srt-slurm * feat(agentx): port the DSV4 FP4 B200 SGLang AgentX config to srt-slurm * feat(agentx): port the MiniMax-M3 FP4 B200 TRT-LLM AgentX config to srt-slurm * feat(agentx): let srt-slurm AgentX recipes set the AIPerf benchmark grace period The GLM-5.2 MI325X legacy script bounded the post-window drain with --benchmark-grace-period 1800. AIPERF_BENCHMARK_GRACE_PERIOD in a recipe's benchmark env passes it through the shared client. * feat(agentx): port the DSV4.1 Flash GB200 and GB300 vLLM AgentX configs to srt-slurm * feat(agentx): port the MiniMax-M3 MI300X vLLM AgentX config to srt-slurm The LMCache DRAM point starts one MP server per TP rank from a setup script inside the worker container, as the legacy script did. * feat(agentx): port the DSV4.1 Flash MI355X vLLM AgentX config to srt-slurm * feat(agentx): port the MiniMax-M3 MI355X vLLM AgentX config to srt-slurm * feat(srt): accept vLLM decode context parallelism and non-drafting points in the single-node adapter Kimi-K3 B300 serves TP8 with decode-context-parallel-size 8, and stops drafting above conc 16 while its matrix labels every point mtp; such a variant declares SPEC_DECODING in its benchmark env. * feat(agentx): port the Kimi-K3 B300 vLLM AgentX config to srt-slurm * feat(agentx): port the Kimi-K3 MI355X vLLM AgentX DCP1 points to srt-slurm The DCP8 no-draft points keep the legacy script: the single-node adapter requires DCP_SIZE=1 and speculation matching the matrix label. * fix(matrix): read node counts from named srt-slurm override variants recipe_node_count returned None for any CONFIG_FILE carrying a selector, so rows pointing at `file.yaml:override_<name>` fell back to the master topology estimate. Resolve `base` and `override_<name>` the way srtctl does (deep merge, null deletes, top-level schema carried into the variant) so the recipe allocation stays authoritative. Zip groups and non-schema-2 variant files keep the estimate. * refactor(agentx): consolidate DSV4 multi-node AgentX recipes into override variants Each per-configuration recipe becomes an `override_<name>` block over a shared `base` in one `*-variants.yaml` per master-config entry, and the master entries select it with `CONFIG_FILE=...:override_<name>`. Every selected variant resolves, through the pinned srtctl, to exactly the recipe it replaces, including its original `name`. Power recipes with top-level telemetry stay standalone because launchers detect them as text. * refactor(agentx): consolidate GLM-5.2 multi-node AgentX recipes into override variants Each per-configuration recipe becomes an `override_<name>` block over a shared `base` in one `*-variants.yaml` per master-config entry, and the master entries select it with `CONFIG_FILE=...:override_<name>`. Every selected variant resolves, through the pinned srtctl, to exactly the recipe it replaces, including its original `name`. Power recipes with top-level telemetry stay standalone because launchers detect them as text. * refactor(agentx): consolidate Kimi-K3 GB200 AgentX recipes into override variants Each per-configuration recipe becomes an `override_<name>` block over a shared `base` in one `*-variants.yaml` per master-config entry, and the master entries select it with `CONFIG_FILE=...:override_<name>`. Every selected variant resolves, through the pinned srtctl, to exactly the recipe it replaces, including its original `name`. Power recipes with top-level telemetry stay standalone because launchers detect them as text. * refactor(agentx): consolidate MiniMax-M3 multi-node AgentX recipes into override variants Each per-configuration recipe becomes an `override_<name>` block over a shared `base` in one `*-variants.yaml` per master-config entry, and the master entries select it with `CONFIG_FILE=...:override_<name>`. Every selected variant resolves, through the pinned srtctl, to exactly the recipe it replaces, including its original `name`. Power recipes with top-level telemetry stay standalone because launchers detect them as text. * refactor(agentx): consolidate Qwen3.5 multi-node AgentX recipes into override variants Each per-configuration recipe becomes an `override_<name>` block over a shared `base` in one `*-variants.yaml` per master-config entry, and the master entries select it with `CONFIG_FILE=...:override_<name>`. Every selected variant resolves, through the pinned srtctl, to exactly the recipe it replaces, including its original `name`. Power recipes with top-level telemetry stay standalone because launchers detect them as text. * docs(recipes): describe AgentX override-variant bundles * feat(agentx): port the DSV4 MI355X vLLM AgentX config to srt-slurm The DEP8 points run behind srt-slurm's vLLM Router frontend with consistent-hash session routing, as the legacy script's router did. * fix(agentx): give the MiniMax-M3 Hopper Mooncake master time to install * fix(agentx): give the AMD SGLang AgentX recipes an hour to become healthy The MI300X Qwen3.5 server needed ~27 minutes to load and finish first-request kernel tuning, past srt-slurm's 1800 s default. * fix(agentx): extend the DSV4.1 Flash H100 SGLang health window for cold NFS loads * fix(agentx): extend the MiniMax-M3 H100 vLLM load window for cold NFS loads * fix(h100): keep one uv cache per runner for native single-node jobs * fix(b300): give srt-slurm jobs the workflow time limit instead of srtctl's one hour The B300 profile set no default_time_limit, so srtctl submitted native single-node AgentX jobs with --time=01:00:00 and a c32 point timed out. * fix(agentx): report RDMA port states when the Kimi-K3 B300 Mooncake rail probe fails * fix(mi355x): let native single-node points use a squash staged on /it-share Node-local /var/lib/squash is not visible from the runner host, so native points always pulled the image, which fails once a nightly tag is pruned. * fix(agentx): probe DSXE rdmap RDMA rails for the Kimi-K3 B300 Mooncake store * fix(srt): let single-node AgentX evals run without a fixed-sequence context * fix(srt): evaluate single-node AgentX points with the workflow's framework at native context The single-node post-eval required MAX_MODEL_LEN and forced lm-eval, but benchmark_lib clears MAX_MODEL_LEN for AgentX, so every AgentX eval-only point failed. AgentX now runs run_eval as the multi-node post-eval does. * fix(b300): download Qwen3.8-Flash-Next NVFP4 into the shared HF cache The checkpoint is staged neither on node-local NVMe nor in the writable model root, so native jobs resolve it by HF id like the other launchers. * fix(srt): stage single-node AgentX eval artifacts once * fix(srt): stage single-node AgentX eval artifacts once * fix(b300): read models without node-local staging from the shared model root DeepSeek-V4.1-Flash is not in STAGED_MODELS and exists on /scratch only on some nodes, so native points failed wherever it was missing. * fix(agentx): give GB200/GB300 DSV4.1 Flash servers a two-hour health budget Cold weight loading from the shared HF cache plus graph capture took just under 30 minutes on GB200 TP2, past srt-slurm's 1800 s default. * chore(agentx): keep the pruned-image MI355X vLLM configs on their legacy scripts Their nightly images are gone from Docker Hub; the legacy path still runs from node-local squashes. The recipes stay in place for when a squash is staged on /it-share. * fix(agentx): give the DSV4.1 Flash GB200 and GB300 vLLM recipes a two-hour readiness window * fix(agentx): let Mooncake pick the GID on DSXE InfiniBand rails for Kimi-K3 B300 * fix(agentx): give the MiniMax-M3 B300 TRT-LLM server a two-hour health budget * style(agentx): write srt-slurm recipe overrides as block YAML Content is unchanged; flow mappings in override variants become block mappings. vLLM compilation/kv-transfer/speculative configs stay quoted JSON strings, which srtctl passes to vLLM verbatim. * fix(agentx): give the Qwen3.8-Flash-Next B300 SGLang server a two-hour health budget * fix(srt): keep GLM-5.2's 150-step SWE-bench budget for single-node AgentX evals The legacy GLM-5.2 scripts raised SWEBENCH_AGENT_STEP_LIMIT to 150 for eval-only runs. srt-slurm's post-eval only sees the runner's environment, so the recipe cannot set it; srt_eval.sh does. * fix(agentx): give the MiniMax-M3 B200 TRT-LLM server a two-hour health budget * fix(b200): default the Qwen3.8-Flash-Next NVFP4 path when no pool setting names it * chore(minimaxm3): drop the vLLM SimpleCPUOffload patch The MiniMax-M3 B200/B300 DRAM points no longer patch vLLM's SimpleCPUOffload worker. The pinned nightly is pruned, so these points cannot run until the image moves to a build that no longer needs the patch (see SemiAnalysisAI#3131, SemiAnalysisAI#3132, SemiAnalysisAI#3435). * style(srt): format single-node adapter 按 Ruff 格式化单节点适配器。 --------- Co-authored-by: adibarra <93070681+adibarra@users.noreply.github.com>
… nightly / 将 Kimi-K3 MI355X vLLM 镜像刷新至 2026-09-24 的 ROCm 10.0 nightly (SemiAnalysisAI#3419) * config(kimik3): refresh MI355X vLLM image to the 2026-09-24 ROCm 10.0 nightly Repin kimik3-fp4-mi355x-vllm-agentic-mtp from nightly-rocm100-af1c0149 to nightly-rocm100-7f1a5398 and re-measure all 9 points. Only the image changes; the search space and kimik3_fp4_mi355x_mtp.sh are untouched. The old pin is from 2026-09-16 and the new one is 441 commits ahead, carrying Kimi-K3 fixes vllm-project/vllm#51483, #57098 and #57430 plus ROCm CPU KV-offload work this arm exercises above concurrency 4. 将 kimik3-fp4-mi355x-vllm-agentic-mtp 的镜像由 nightly-rocm100-af1c0149 更新为 nightly-rocm100-7f1a5398，并重新测量全部 9 个数据点。本次仅更换镜像， 搜索空间与 kimik3_fp4_mi355x_mtp.sh 均未改动。 旧镜像切出于 2026-09-16，新镜像领先 441 个提交，包含 Kimi-K3 修复 vllm-project/vllm#51483、#57098 与 #57430，以及本臂在并发大于 4 时会使用的 ROCm CPU KV 卸载改动。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(kimik3): point the changelog entry at PR 3419 Replace the XXX placeholder now that the PR exists. 在 PR 建立后，将 changelog 条目中的 XXX 占位符替换为实际链接。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(kimik3): refresh MI355X vLLM image to the 2026-09-24 ROCm 10.0 nightly Refresh the image on kimik3-fp4-mi355x-vllm-agentic-mtp and re-sweep it. The image: line is the only change; the search space and benchmarks/single_node/agentic/kimik3_fp4_mi355x_mtp.sh are untouched (nine points, same curve shape: TP8 conc 1/4 GPU-resident KV, conc 8/10/12/14 on vllm-simple DRAM offload, conc 44/48/70 on TP8/DCP8 vllm-simple). Old pin nightly-rocm100-af1c0149 (vLLM main 2026-09-16), new pin nightly-rocm100-7f1a5398 (vLLM main 2026-09-24), 441 commits apart. The jump carries three Kimi-K3 correctness fixes (vllm#51483, #57098, #57430) plus ROCm CPU KV-offload work (vllm#57160) exercised by the seven SimpleCPUOffloadConnector points above conc 4. This sweep is the qualification for the new image. Data type preserved: this image bump does not change the DSpark draft model data type. The script is unchanged; the draft loads unmodified from the published Inferact/Kimi-K3-DSpark checkpoint via --speculative-config (model=Inferact/Kimi-K3-DSpark, method=dspark). The only dtype in that speculative-config is kv_cache_dtype=fp8, which sets the draft KV-cache storage precision, not the draft weights. No flag overrides or re-quantizes the draft-model weights, so the draft dtype is preserved from its checkpoint across this re-sweep. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: seungrokj <seungrok.jung@amd.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…benchmarks/single_node/agentic (SemiAnalysisAI#3460) SemiAnalysisAI#3428 moved single-node AgentX onto native srt-slurm recipes. Delete every script in benchmarks/single_node/agentic that no master config reaches any more. Keep the nine scripts still reached by search-space entries without an srt-recipe, plus the H100/H200 block32 installers and kernel_configs that the multi-node DeepSeek-V4.1-Flash recipes use. The README now marks the folder as legacy. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Single-node AgentX runs as native srt-slurm recipes since SemiAnalysisAI#3428, and SemiAnalysisAI#3460 removed the unused scripts. Delete the rest of the legacy folder. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…sAI#3464) Every active single-node fixed-sequence config runs as an srt-slurm recipe since the SRT-only cutover (2026-09-22); only deprecated/ scripts remained, and nothing executes or sources them. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… delete configs/deprecated and amd_utils/deprecated (SemiAnalysisAI#3463) * [Klaud Cold] Remove everything that pointed at the deleted benchmarks/single_node/agentic reached it: - Drop the 17 search-space entries without srt-recipe: six configs that were entirely bash-based (both B300 Qwen3.5 power A/B keys, qwen3.8next H100, Kimi-K3 and MiniMax-M3 MI355X vLLM, DSv4.1 Flash MI355X ATOM) and the ATOM LMCache DRAM-offload arms of Kimi-K3, MiniMax-M3 and GLM-5.2 on MI355X. - Remove the dead dsv41flash bash branches in the GB200/GB300 launchers and the MiniMax-M3 ATOM script special case in the MI355X launcher. - Restore the DSv4.1 Flash H100/H200 block32 installers and kernel configs under benchmarks/multi_node/srt-slurm-recipes/configs/dsv41flash-block32/ and repoint the setup scripts, which the live single-node H100/H200 SGLang AgentX recipes run and SemiAnalysisAI#3461 broke. - Point docs and comments at the srt-slurm recipes and srt_agentic.sh. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Drop the DSv4.1 Flash H100/H200 block32 kernel tilings instead of restoring them Per review: do not re-add the block32 installers and kernel configs. Remove the restored dsv41flash-block32 assets, the two block32 setup scripts, the recipes' setup_script entries and the H200 DSV41_BLOCK32_TP env vars; the H100/H200 SGLang AgentX recipes now run with SGLang's default tilings. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Drop the KLAUD_DEBUG script pointer (fixed_seq_len is deleted in SemiAnalysisAI#3464) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Delete configs/deprecated and drop remaining fixed_seq_len pointers - Delete configs/deprecated/{amd,nvidia}-master.yaml; nothing reads them. AGENTS.md, docs/configuration-procedures and the Claude review prompt now say retired entries are deleted, with git history and perf-changelog.yaml as the record. - Rewrite the add-model-hardware playbook and configuration procedures for srt-slurm recipes instead of bash scripts under fixed_seq_len/. - Point MODELS links, the KLAUD_DEBUG example and a deprecated multi-node comment away from the deleted folders. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Delete benchmarks/multi_node/amd_utils/deprecated Nothing loads these archived AMD server-registry files or glm5_setup.sh; the live registries are the models*.yaml and node_excludes.yaml beside them. configuration-procedures now says retired AMD registry entries are deleted rather than moved into deprecated/. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Revert the docs/waiver/2508.md edit Leave the historical waiver record unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…3467) Nothing resolves or sources these 12 retired multi-node scripts. Drop the configuration-procedures link to the llmd-vllm wrapper (it pointed at a path that no longer existed) and reword the ATOM agentic comment that named its retired SGLang sibling. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…#3469) * [Klaud Cold] Remove the inferencemax-repos MCP server Delete the vLLM/SGLang source MCP server added in SemiAnalysisAI#494 (.claude/mcp, .mcp.json, .github/mcp-ci.json) and everything wired to it: the --mcp-config/allowedTools entries and INFERENCEMAX_ROOT env in the Claude and CODEOWNER sign-off workflows, the prompt sections that told reviewers to use it, the CI compatibility step and path triggers, the mcp dependency group (uv.lock regenerated with uv 0.12.16; deletions only), and the docs. The fetch MCP server used by those workflows is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Drop the empty env block left in the Claude review step INFERENCEMAX_ROOT was its only entry; an empty env: is null and fails the workflow schema (Zizmor: jobs.review.steps.3.env null). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…emiAnalysisAI#3471) * [Klaud Cold] Remove Kimi-K2.5, Kimi-K2.6 and Kimi-K2-Thinking Delete every Kimi K2-family asset and reference outside MODELS.md and the append-only perf-changelog: - multi-node srt-slurm recipes: kimik2.5/vllm (GB200/GB300) and kimik2.6/vllm (B200) - the retained Kimi-K2.5 SPEED-Bench collector, its EAGLE3 golden AL file and README rows, its CI priority score, the kimik2.5 eagle3 special case in synthetic_acceptance, the gsm8k threshold, the run-sweep criteria enum value, and the Kimi-K2.6-NVFP4 B300 staging entry and SRT model alias - the Kimi-K2-Thinking suite at the top of experimental/token_position_decode_slo - 516 kimik2.5 entries from the OperatorX testlists - stale comments and README/KLAUD_DEBUG mentions Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Keep Kimi K2.5/K2.6/K2-Thinking material under experimental/ Restore the Kimi-K2-Thinking token_position_decode_slo suite, the OperatorX kimik2.5 testlist entries and the testlist generator comments. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Keep the Kimi K2.5 news line in README Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…miAnalysisAI#3472) Remove the runner-registration screenshot and its embed in RUNNER_SETUP.md; the step's text and link to the registration page remain. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… GLM-5/GLM-5.1 recipes (SemiAnalysisAI#3473) Delete the multi-node srt-slurm 8k1k recipes for DSv4 (SGLang/TRT-LLM/vLLM) and MiniMax-M3 (vLLM), and all glm5/ and glm5.1/ recipes. DSR1 and Qwen3.5 8k1k recipes stay. Remove glm5.1-fp8-b200-tilert-agentic, whose only recipe was glm5.1/tilert/b200-fp8. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…anary 选择 (SemiAnalysisAI#3462) * fix(ci): run AgentX canary before sweep fan-out 将单节点 AgentX 吞吐量任务纳入 Canary 选择，沿用场景和运行时长，并从后续矩阵移除已选任务。 * fix(ci): gate multi-node AgentX sweeps with a canary 为多节点 AgentX 扫描添加 Canary 门禁。
…I#3952) 为 NVIDIA 模型与框架组合增加官方配方概览缺失警告，排除非 NVIDIA 硬件，并更新发布器与测试。
…e points / DSv4.1-Flash ATOM AgentX 更新至 1012 nightly 并去掉五个点 (SemiAnalysisAI#3928) * perf(amd): bump DSv4.1-Flash ATOM AgentX to 1012 nightly and drop four points Move dsv41flash-fp4-mi355x-atom-agentic-dspark to rocm/atom-dev:nightly_202610121516 (ATOM a0527c9db) and drop TP2 c5/c128 and TP4 c4/c6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: set perf-changelog pr-link to SemiAnalysisAI#3928 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(amd): drop DSv4.1-Flash ATOM TP4 concurrency 5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * perf(amd): align DSv4.1-Flash ATOM AgentX 1012 nightly changelog with the diff - Update the image from rocm/atom-dev:nightly_202610100929 to nightly_202610121516 (digest sha256:a60811a8467b43e3e50e9e9eb8cec34f68b7bca03f8c6dbd6a0ff813dde49d4c; ATOM 0.1.8.dev29+ga0527c9db, ROCm 7.2.4) and update model.container and the AToMesh frontend image to match. - Drop TP2 concurrency 5 and 128 and TP4 concurrency 4, 5 and 6. TP2 runs concurrency 1, 2, 4, 6, 8, 16, 32 and 64 and TP4 runs 1, 8 and 32: 11 points instead of 16. - Server settings are unchanged. Upstream recipes/DeepSeek-V4.1-Flash-Agentic.md is identical between 73defd9b8 and a0527c9db. - ROCm/ATOM#2545 reworks decoder SWA bounded replay to follow the V4.1 report: the forward that finishes a prompt runs the encoder (layers 0-19) and layer 20's compression on every row, layers 20-38 on the last 128 rows with their windows truncated there, and layer 39 on the last row only. - ROCm/ATOM#2545 also turns on encoder SWA bounded replay by default: a prefix hit rebuilds the encoder's sliding-window state by recomputing one KV page below the hit instead of reading a state checkpoint, and no state checkpoints are kept. The DeepSeek-V4.1 KV page, which is also the prefix-cache hash block, drops from 256 to 128 tokens. - ATOM's mono decode path (on by default) now takes any decode step of up to 48 rows, i.e. concurrency 1-8 with five DSpark drafts, not only DSpark verify steps. Steps of 12 or more rows that are a multiple of TP run its sublayers token-parallel, with reduce-scatters instead of all-reduces. The draft runner never runs token-parallel. - No data-type or precision change to the DSpark draft. The checkpoint's bundled mtp.0-2 stages are stored like the target's layers: FP8 E4M3 weights with 32x32 UE8M0 block scales for the attention projections, shared experts and main_proj; MXFP4 routed experts; BF16 norms, router gate, Markov head and confidence head; F32 attention sinks and hc mixers. ATOM builds the draft with the target's native_quant_config() and make_v4_quant_config(), with no online quantization in this recipe, so it runs as shipped: FP8 activations on the FP8 GEMMs and the target's BF16 KV cache. Between 73defd9b8 and a0527c9db only the rows the draft reads under bounded replay change. - 镜像从 rocm/atom-dev:nightly_202610100929 更新为 nightly_202610121516（digest sha256:a60811a8467b43e3e50e9e9eb8cec34f68b7bca03f8c6dbd6a0ff813dde49d4c；ATOM 0.1.8.dev29+ga0527c9db，ROCm 7.2.4），model.container 和 AToMesh frontend 镜像同步更新。 - 去掉 TP2 的并发 5 和 128，以及 TP4 的并发 4、5 和 6。TP2 跑并发 1、2、4、6、8、16、32 和 64，TP4 跑 1、8 和 32：共 11 个点，原来是 16 个。 - server 参数不变。上游 recipes/DeepSeek-V4.1-Flash-Agentic.md 在 73defd9b8 和 a0527c9db 之间没有改动。 - ROCm/ATOM#2545 按 V4.1 报告重写了 decoder SWA bounded replay：完成 prompt 的那次 forward 对所有行运行 encoder（第 0-19 层）和第 20 层的压缩，第 20-38 层只算最后 128 行并在此截断窗口，第 39 层只算最后一行。 - ROCm/ATOM#2545 还默认开启 encoder SWA bounded replay：前缀命中时，从命中点下方重算一个 KV page 来重建 encoder 的滑动窗口状态，不再读取 state checkpoint，也不再保留 state checkpoint。DeepSeek-V4.1 的 KV page（也是前缀缓存的 hash block）从 256 降为 128 个 token。 - ATOM 的 mono decode 路径（默认开启）现在接管任何不超过 48 行的 decode step（五个 DSpark draft 时对应并发 1-8），不再只限于 DSpark verify step。行数不少于 12 且为 TP 倍数的 step 以 token 并行运行子层，用 reduce-scatter 代替 all-reduce。draft runner 从不使用 token 并行。 - DSpark draft 的数据类型和精度不变。checkpoint 自带的 mtp.0-2 与 target 层的存储方式相同：attention 投影、shared experts 和 main_proj 为 FP8 E4M3 权重加 32x32 UE8M0 block scale；routed experts 为 MXFP4；norm、router gate、Markov head 和 confidence head 为 BF16；attention sink 和 hc mixer 为 F32。ATOM 用与 target 相同的 native_quant_config() 和 make_v4_quant_config() 构建 draft，本 recipe 没有 online quantization，因此 draft 按原样运行：FP8 GEMM 使用 FP8 激活，KV cache 与 target 相同为 BF16。在 73defd9b8 和 a0527c9db 之间，只有 bounded replay 下 draft 读取的行发生变化。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: seungrokj <seungrok.jung@amd.com>
… --monokernel on the latest ROCm 10.0 nightly / [AMD][MI355X] DSv4.1 Flash vLLM：在最新 ROCm 10.0 nightly 上启用 SWA bounded replay 与 --monokernel 并重测 (SemiAnalysisAI#3959) * perf(amd): re-sweep DSv4.1-Flash MI355X vLLM AgentX with SWA bounded replay and --monokernel / DSv4.1-Flash MI355X vLLM AgentX 启用 SWA bounded replay 与 --monokernel 并重测 Bump the image to nightly-rocm100-5555a2d8f7116b2b4877f66516f4128176cf722a, which contains vllm-project/vllm#59039 and #61082. Drop --no-swa-bounded-replay and replace VLLM_ROCM_MONO_DECODE=1 with --monokernel. 镜像更新至 nightly-rocm100-5555a2d8f7116b2b4877f66516f4128176cf722a，包含 vllm-project/vllm#59039 与 #61082。移除 --no-swa-bounded-replay，并用 --monokernel 取代 VLLM_ROCM_MONO_DECODE=1。 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(changelog): point the perf-changelog entry at PR 3959 / 将 perf-changelog 条目指向 PR 3959 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * docs(changelog): describe the DSv4.1-Flash MI355X vLLM re-sweep from the diff - Update the image from nightly-rocm100-7d0b4e57aac4c4323225b0eee9ae960ce75ffffe to nightly-rocm100-5555a2d8f7116b2b4877f66516f4128176cf722a, the 2026-10-13 ROCm 10.0 nightly, which contains vllm-project/vllm#59039 and #61082. Update model.container to match. - No point changes: concurrency 1, 2, 4, 8, 16, 32 and 64 on TP2 and TP4 (14 points), all re-measured on the new image. - Remove --no-swa-bounded-replay, which turns on decoder SWA bounded replay from vllm-project/vllm#59039: in eager prefill steps, layer 20 (the last KV source layer) writes its KV for every token, and the rest of layer 20 and layers 21-39 run on each request's last 128 tokens only. Prefix-cache SWA replay stays off, since vLLM still forces it off on ROCm (the window clamp it needs lives only in the FlashInfer and FlashMLA prefill kernels). - Replace VLLM_ROCM_MONO_DECODE=1 with --monokernel. vllm-project/vllm#61082 moves the switch to KernelConfig.monokernel and removes the environment variable, so the old setting would leave the mono decode kernels off. - vllm-project/vllm#61082 also runs all 40 backbone layers whole as two persistent launches. Before, 30 ran whole and the other 10 (layers 0 and 1 and the compressor / indexer layers 2, 8, 14, 20, 24, 28, 32 and 36) kept vLLM's attention plus the FFN launch. The step limit is unchanged at 48 rows, i.e. concurrency 1-8 with five DSpark drafts; larger steps take the regular path. - No data-type or precision change to the DSpark draft. The checkpoint's bundled mtp.0-2 stages are stored like the target's layers: FP8 E4M3 weights with 32x32 UE8M0 block scales for the attention projections, shared experts and main_proj; MXFP4 routed experts; BF16 norms, router gate, Markov head and confidence head; F32 attention sinks and hc mixers. vLLM builds the draft with the checkpoint's own quantization config (get_draft_quant_config) and, with no kv_cache_dtype in speculative-config, the target's fp8_ds_mla KV cache, so it runs as shipped. The mono decode kernels skip the draft layers, and the draft load path is the same in 7d0b4e57 and 5555a2d8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: seungrokj <seungrok.jung@amd.com>
…emiAnalysisAI#3973) 中文：power_model 的改动须附带 InferenceX-app 移植版的配套 PR，并互相链接。
…200 Dynamo-SGLang MTP 配置 (SemiAnalysisAI#2594) * feat(config): add Qwen3.5 FP4 GB200 Dynamo-SGLang MTP 新增 Qwen3.5 FP4 GB200 Dynamo-SGLang MTP 配置。 * chore(changelog): link PR SemiAnalysisAI#2594 在性能变更日志中补充 PR SemiAnalysisAI#2594 链接。 * fix(changelog): link PR SemiAnalysisAI#2594 to Qwen3.5 entry 将 PR SemiAnalysisAI#2594 链接补充到对应的 Qwen3.5 变更日志条目。 * Migrate Qwen3.5 GB200 recipes to srt-slurm schema 2 Applied srtctl migrate from the srt-slurm commit pinned by main; resolved prefill/decode nodes, workers, env, engine args, frontend, model and benchmark settings are unchanged. --------- Co-authored-by: hshrivastava-droid <hshrivastava@nvidia.com> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: Ankur-singh <ankusingh@nvidia.com>
…ith DEP2 and TP4 EP1 / [AgentX] GB300 DSV4.1 Flash SGLang：升级 20261012 nightly，精简为 7 个点（DEP2、TP4 EP1） (SemiAnalysisAI#3859) * perf(agentx): fold DSpark sampling and retune c64/c128 for GB300 DSV4.1 Flash SGLang Bump the image to nightly-dev-cu13-20260929-79cafec0 and set SGLANG_DSPARK_FOLDED_SAMPLING=2 on every point: the AUTO default keeps DSpark draft sampling eager for DeepSeek-V4.1 TP shards, which left low-concurrency decode launch-bound. Run TP4 c64 with 8K chunks and a 16-step prefill-decode interval, and let TP2 c128 admit all 128 sessions with a 4-step interval and mem-fraction-static 0.85. 中文：将镜像升级到 nightly-dev-cu13-20260929-79cafec0，并在所有点上设置 SGLANG_DSPARK_FOLDED_SAMPLING=2：AUTO 默认值对 DeepSeek-V4.1 的 TP 分片 保持 DSpark draft 采样为 eager 执行，导致低并发 decode 受 kernel 发射限制。 TP4 c64 改为 8K prefill chunk 与 16 步 prefill-decode interval；TP2 c128 接纳全部 128 个会话，使用 4 步 interval 与 mem-fraction-static 0.85。 * chore(changelog): link the GB300 DSV4.1 Flash SGLang entry to SemiAnalysisAI#3782 中文：将 GB300 DSV4.1 Flash SGLang 的 changelog 条目链接到 SemiAnalysisAI#3782。 * perf(agentx): bump GB300 DSV4.1 Flash SGLang to the 20261008 nightly, 16K chunks at TP4 c1 Root cause: on the 20260929 nightly the Python radix-tree insert walk slices the remaining key at every node, which delays the last prefill chunk on 200K-400K-token AgentX prompts. At TP4 c1, 4K chunks with a 16-step prefill-decode interval split long cache-miss prefills into many steps and made subagent requests wait behind them. Fix: pin lmsysorg/sglang:nightly-dev-cu13-20261008-7760d526 by digest in the recipe and the master config. The image makes the Rust radix-tree core the default. TP4 c1 now runs 16K chunks with no prefill-decode interval. Validation: local GB300 TP4 c1 30-minute AgentX runs, synthetic acceptance 3.51. Image bump: 5,570 -> 5,577 tok/s/GPU, P90 interactivity 521.5 -> 562.5. Chunking (new image, fastokens backend in both arms): 5,826 -> 5,957 tok/s/GPU, P90 TTFT 1.38 -> 0.46 s. Other points were not rerun on the new image. * chore(changelog): link the GB300 DSV4.1 Flash SGLang 20261008 entry to SemiAnalysisAI#3859 * perf(agentx): GB300 DSV4.1 Flash SGLang 20261012 nightly, tuned TP2 c8-c32, DEP2 c64/c128 Bump to nightly-dev-cu13-20261012-e6072935, which carries DP attention with MegaMoE. TP2 c8/c16/c32 run 16K chunks with decoder SWA bounded replay and a longer prefill-decode interval; c64 and c128 move to DEP2 behind sglang-router with shortest-prefill-first scheduling. * perf(agentx): GB300 DSV4.1 Flash SGLang: trim to 7 Pareto points, TP4 EP1 at c4/c16 Keep TP4 c1/c4/c16, TP2 c16/c32 and DEP2 c64/c128; the other points are dominated. TP4 c4 and c16 run EP1, and c16 adds a 192-step prefill-decode interval, 4 tokenizer workers and soft NUMA binding. * chore: credit the GB300 DSV4.1 Flash config-only tuning campaign Co-authored-by: nbommi19 <227267412+nbommi19@users.noreply.github.com> --------- Co-authored-by: weireweire <20922698+weireweire@users.noreply.github.com> Co-authored-by: nbommi19 <227267412+nbommi19@users.noreply.github.com> Co-authored-by: Ankur-singh <ankusingh@nvidia.com>
…d] 将 DSR1 FP8 MI355X ATOM 镜像更新至 v0.1.7 (SemiAnalysisAI#3911) * chore(amd): bump dsr1-fp8-mi355x-atom to ATOM v0.1.7 image Pin rocm/atom-dev:nightly_202610080328 by digest; it is built from the ATOM v0.1.7 release commit a526f0d. 将 dsr1-fp8-mi355x-atom 更新为 ATOM v0.1.7 镜像：按摘要固定 rocm/atom-dev:nightly_202610080328，该镜像构建自 ATOM v0.1.7 发布提交 a526f0d。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(changelog): add dsr1-fp8-mi355x-atom ATOM v0.1.7 entry 为 dsr1-fp8-mi355x-atom 的 ATOM v0.1.7 镜像更新追加 perf-changelog 条目。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore: refresh PR SemiAnalysisAI#3911 for sweep reuse [skip-sweep] --------- Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: functionstackx <47992694+functionstackx@users.noreply.github.com>
… / fix(review): 让签核验证器发现并修正结论格式问题 (SemiAnalysisAI#3958) * fix(review): stop discarding sign-off verdicts over row formatting slips The sign-off verifier sometimes marks an N/A row with the PASS emoji (✅ instead of ➖), most often Check 12 when it is the only N/A row. signoff_publish then discards the whole verdict and posts "did not produce a valid verdict". All 3 rejections among the 36 verdict artifacts from 2026-10-06 to 2026-10-13 had this cause. N/A already counts as a pass, so the publisher now rewrites ✅ to ➖ on N/A rows before validation. Every other emoji/status mismatch is still rejected. The verifier prompt now states that the status word alone selects the emoji, and that each row stays on one line with its reason after " — ". A later run ended the Check 13 row at "PASS" and moved the reason into sub-bullets, which the publisher also rejects. 签核验证器有时会把 N/A 行标成通过的表情符号（✅ 而不是 ➖），最常见于 Check 12 是唯一 N/A 行的情况。signoff_publish 随后会丢弃整个结论，并 发布 “did not produce a valid verdict”。2026-10-06 至 2026-10-13 的 36 个结论产物中，3 次拒绝全部由此引起。 N/A 本来就视为通过，因此发布器现在会在校验前把 N/A 行的 ✅ 改为 ➖； 其他表情符号与状态不一致的情况仍会被拒绝。 验证器提示词现在明确说明：表情符号只由状态词决定，且每一行都须写在同 一行内，原因放在 " — " 之后。后来的一次运行把 Check 13 行停在 "PASS"， 把原因移到了子列表中，发布器同样会拒绝这种行。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(review): let the sign-off verifier check its verdict before publishing Three different formatting slips discarded the same PR's verdicts: an N/A row marked ✅, a Check 13 row whose reason moved into sub-bullets, and a verdict that left out Check 4. Prompt wording alone cannot rule these out. signoff_publish now collects every reason a verdict is rejected (parse_verdict/verdict_errors) instead of stopping at the first one; publication outcomes are unchanged. The new read-only command `python -m infx.workflows.signoff_validate <verdict.md>` prints those reasons, or "Verdict format is valid.", without publishing. The verify job allows that one command, and the prompt requires the verifier to run it and fix any format problem before finishing, without changing a check's conclusion. 同一个 PR 的签核结论曾因三种不同的格式问题被丢弃：N/A 行标成 ✅、 Check 13 行的原因被移到子列表中，以及漏写 Check 4。仅靠提示词措辞无法 完全避免这些问题。 signoff_publish 现在会收集结论被拒绝的所有原因 （parse_verdict/verdict_errors），而不是在遇到第一个问题时就停止；发布 结果保持不变。新增的只读命令 `python -m infx.workflows.signoff_validate <verdict.md>` 会列出这些原因， 或输出 "Verdict format is valid."，但不会发布。验证任务只额外允许这一条 命令，提示词要求验证器在结束前运行它并修正格式问题，且不得因此改变任何 检查的结论。 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(review): drop added sign-off verdict tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Cameron Quilici <60715037+cquil11@users.noreply.github.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com> Co-authored-by: Cameron Quilici <60715037+cquil11@users.noreply.github.com>
…grid to c28 / [NVIDIA][AgentX] 调优 GLM-5.2 FP8 B300 SGLang MTP 配方并将并发扩展至 c28 (SemiAnalysisAI#3836) Tune MTP depth, admission caps and HiCache scheduling with one recipe per concurrency (c1 K6/cap2, c4 K4/cap3, c8 unchanged, c12 K5/cap5, c16 K4/cap7 with prefill-decode-interval 8 and write_through), pin the upstream SGLang image by digest, and add c20, c24 and c28 with uncapped admission (2x CONC). 按并发调优 MTP 深度、准入上限和 HiCache 调度，每个并发一个配方；按摘要固定 上游 SGLang 镜像；新增 c20、c24、c28（不限制准入，2x CONC）。 Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> Co-authored-by: Po-Han Huang <pohanh@nvidia.com>
…axm3 B300 FP4 vllm 升级每日构建 (SemiAnalysisAI#4000) * [AgentX nightly] minimaxm3 B300 FP4 vllm mtp [AgentX nightly] minimaxm3 B300 FP4 vllm 升级每日构建 * Record canonical PR link for SemiAnalysisAI#4000 记录 SemiAnalysisAI#4000 的正式 PR 链接。
…#3802) * chore(recipes): delete unreferenced srt-slurm recipes Remove recipe files and override blocks that no master config, launcher table, test, or doc references: - 35 multi-node recipes under seven retired 1k1k directories - 6 minimaxm3 gb200-fp4 agentx per-point recipes superseded by bundles - qwen3.5 gb200-fp4 agentx agg-tp4-mtp-hicache (superseded by agg-variants.yaml:override_tp4) - single-node minimaxm3 vllm mi355x-fp4-mtp agentic recipe - the four MTP overrides in dsr1 b200-fp4 8k1k disagg-stp-mtp-variants.yaml, whose master entries only select the STP variants The generated sweep matrix is byte-identical before and after. * chore(recipes): keep retired 1k1k recipes for reference Restore the 35 multi-node recipes under the retired 1k1k directories. They are not referenced by any master config but are kept as a historical record. * chore(recipes): drop the hard-coded variant count from a usage comment
…t-recipe (SemiAnalysisAI#3805) * refactor(matrix): select srt-slurm recipes with srt-recipe-dir and srt-recipe Master configs name each row's recipe with srt-recipe (and eval-srt-recipe) relative to a per-entry srt-recipe-dir instead of CONFIG_FILE and EVAL_CONFIG_FILE additional-settings. Validation requires them on every srt-slurm row, checks the file and selected variant exist at load time, and rejects the old settings; llmd-vllm entries keep CONFIG_FILE. The generator emits project-relative paths, counts nodes from the row's recipe, and the launcher reads SRT_RECIPE/EVAL_SRT_RECIPE, staging the same recipes/<path>:<selector> argument for srtctl. Also drops the unread SYNTHETIC_ACCEPTANCE* and SLURM_PARTITION settings. * chore(recipes): drop stale CONFIG_FILE header comments Master rows now select bundle variants with srt-recipe, so the recipe headers no longer describe how to pick one. * fix(matrix): resolve srt recipes under the loaded config's project Recipe existence, selector checks and recipe node counts read files from the project that holds the loaded configs/ (or the root a caller passes) instead of the tooling checkout's repository_root(). The generator CLI and planner pass config_root(config files), Klaud live families pass the checkout root, and Klaud's canonical matrix fetches the candidate's recipes from the candidate head. * test(operatorx): give the generator fixture a project-layout srt recipe Master configs now name an srt-recipe resolved under the project that holds configs/, so the fixture places its master in configs/ beside a recipe. * fix(configs): select the DSv4.1-Flash ATOM AgentX recipe with srt-recipe-dir srt-recipe paths, which validation rejects now that every srt-slurm entry needs srt-recipe-dir. The entry sets srt-recipe-dir and its rows select agentic.yaml; the generated rows are unchanged. * chore(recipes): drop stale CONFIG_FILE headers from the DSv4.1-Flash GB300 AgentX recipes SemiAnalysisAI#3598 added the GB300 Dynamo+SGLang recipes with headers that say to select a variant with CONFIG_FILE. Master rows select them with srt-recipe now, so the headers stop at the variant layout like the other AgentX bundles. * refactor(matrix): deslop srt-recipe selection Set an AgentX row's srt-recipe fields once after the topology branch, and fetch each recipe file a Klaud family references once rather than once per row. * test: deslop srt-recipe selection tests The installed-package test no longer checks recipe node counts, which take an explicit root since srt-recipe and so no longer exercise caller-repository resolution. Drop unused monkeypatch fixtures and a history comment. * docs: deslop srt-recipe selection guidance Show srt-recipe in the CONFIGS.md search-space example, shorten its selection note, name srt-slurm recipes in the revision snapshot description, and shorten the debug-runs recipe path. * fix(configs): select each GB200 DSR1 FP8 low-latency variant explicitly The low-latency row named disagg-tp8-stp-variants.yaml without a selector, so one job submitted all three zip variants under a 4-node, 1-decode-worker label. Each variant now runs only the concurrencies it is tuned for, with its selector and its 7 TP8 decode workers on 16 nodes: zip_override_lowlat[0] (c8) at 4 and 8, zip_override_lowlat[1] (c16-32) at 16. zip_override_lowlat[2] is tuned for concurrency 64, which the entry never ran, so it has no row. * fix(configs): select the Qwen3.8 and Qwen3.5 GB300 AgentX recipes with srt-recipe-dir SemiAnalysisAI#3816 and SemiAnalysisAI#3840 added qwen3.8next-fp4-gb300-sglang-agentic-mtp and the Qwen3.5 FP8 GB300 midcurve entries with a project-relative srt-recipe path and CONFIG_FILE settings, which validation rejects now that every srt-slurm entry needs srt-recipe-dir. The entries set srt-recipe-dir and their rows select the same recipes and DEP16 overrides; the generated rows are unchanged. * fix(matrix): report a malformed srt-recipe as a validation error A recipe that fails to parse raised a raw yaml.YAMLError out of validate_master_config, so Klaud's canonical_matrix crashed on a malformed candidate recipe instead of rejecting it like other bad srt-recipe references. It now raises the same ValueError, naming the recipe file. * fix(configs): select the Qwen3.5 GB200 FP4 V2 MTP recipes with srt-recipe-dir SemiAnalysisAI#2594 added qwen3.5-fp4-gb200-dynamo-sglang-mtp with CONFIG_FILE settings, which validation rejects now that every srt-slurm entry needs srt-recipe-dir. The entry sets srt-recipe-dir and each row selects the same recipe; the generated rows are unchanged.
… the matrix point (SemiAnalysisAI#3809) * refactor(srt): compose fixed-sequence recipes from fragments and bind the point Active fixed-sequence srt-slurm recipes keep only recipe-specific settings. At launch the lane's shared block (configs/srt-recipes/) is merged under the fragment, the row's variant is selected, and the binder writes the matrix point: model path, image, precision, ISL/OSL, single-node client values (MODEL, CONC, RANDOM_RANGE_RATIO, USE_CHAT_TEMPLATE) and telemetry concurrencies. A fragment that sets a bound key fails before submission. The hf:<MODEL> model path maps to the cluster's staged checkpoint and the master image to its staged container, so recipes stop naming aliases. The mi355x-amds DeepSeek-R1 MXFP4 checkpoint moves into models.entries. Six b300-fp8 trtllm recipes declared fp4 precision; it is now bound to fp8. * feat(srt): add infx generate for bound fixed-sequence recipes `infx generate --config-key <key|glob> --output-dir <dir>` writes the bound recipe of every fixed-sequence point and variant, validated by the pinned srtctl, with a manifest mapping files to matrix points. srtctl's import dependencies sit behind the new `recipes` extra. RECIPES.md documents fragments, composition and the command. * refactor(srt): set the multi-node fixed-sequence TOKENIZER in the shared block Every active multi-node fixed-sequence fragment repeated benchmark.env.TOKENIZER: /model. The shared block configs/srt-recipes/fixed-sequence-multi.yaml now sets it and the fragments drop it, along with the benchmark blocks it was the only entry of. The three mi355x fragments that lacked it (DeepSeek-R1 FP8 disagg variants, Qwen3.5 FP4 and FP8 1p1d) now tokenize with the mounted checkpoint too instead of the served model id. Every other bound recipe is unchanged. * fix(srt): bind exactly one multi-node variant in infx generate infx generate bound every variant a multi-node selector matched, while the launcher's binder refuses anything but one, so it wrote recipes for points that launch rejects, such as dsr1-fp8-gb200-dynamo-sglang. Both now share bind_multinode, and generate fails such a point as launch does. * refactor(srt): deslop the fixed-sequence binder The binder parses CONC_LIST with the launcher's parse_concurrencies, now in workload.py, so telemetry concurrencies get its canonical and unique checks. Define the bound recipe file name once, drop generate's MODEL_PREFIX and RUN_EVAL inputs that nothing on its path reads, name what _validated does, and drop a model guard the binder makes redundant. Fragment checks drop two cases that repeat a covered branch. * test(srt): drop assertions on where the bound recipe lives The content checks that follow already prove srtctl ran the bound variant. * docs: deslop fixed-sequence fragment guidance Drop restatements of the shared blocks and alias history from RECIPES.md, and correct which keys add-model-hardware says fragments omit. * build: drop test pins the recipes extra already declares CI and the testing guide install every extra with the test group. * fix(srt): fail fast when a fixed-sequence point's cluster stages no checkpoint Fragments no longer name model.path aliases, so the alias guard never fired for fixed-sequence recipes: an unstaged MODEL left model_paths empty and srtctl resolved the bound hf:<MODEL> from the Hub on the compute nodes. model_paths now raises before submission instead, unless an Override row sets hub to load the Hub snapshot. The mi355x-amds DeepSeek-R1 FP8 and Qwen3.5 FP4 SGLang disagg lanes, whose fragments named hf: paths before they were bound, set it and keep loading the Hub snapshot. * refactor(srt): compose the Qwen3.5 GB200 FP4 V2 MTP recipes from fragments SemiAnalysisAI#2594 added nine standalone recipes for qwen3.5-fp4-gb200-dynamo-sglang-mtp that name a model alias, the image and the precision, and run srt-slurm's sa-bench. They become fragments: the binder writes the master model, image and precision, and the shared fixed-sequence-multi.yaml block runs the InferenceX client, as every other multi-node fixed-sequence recipe has since SemiAnalysisAI#3415. The client keeps sa-bench's settings: the point's lengths, the workflow's 0.8 range ratio, ten prompts and two warmups per concurrency, an unthrottled request rate, the chat template, ignore-EOS and the /model tokenizer. The c12288 point's sa-bench reuse_http_connections has no counterpart; the client opens a connection per request, as it does for every other point.
…r row (SemiAnalysisAI#3818) * refactor(srt): compose AgentX recipes from fragments and set power on master rows AgentX recipes, single- and multi-node, become fragments like the fixed-sequence ones. configs/srt-recipes/agentic-{single,multi}.yaml supply the client command; the binder writes the master model, image, precision and identity, and the launcher hands multi-node clients RESULT_DIR, AIPERF_DATASET_MMAP_CACHE_DIR and HF_HUB_CACHE from the volumes it mounts (agentic-volume-mounts replaces lanes._AGENTIC_CACHES). Dead keys and copies of the workflow environment leave the fragments; a fragment that sets a bound key fails before submission. A multinode search-space row with power: true replaces recipe telemetry blocks: the shared telemetry-dcgm.yaml is merged with the cluster's power-exporter-port and benchmark.concurrencies is bound after variant selection, so bundles can measure power. POWER_LANES now judges the row's field instead of reading recipe text, and inject_concurrencies is gone. Selected zip variants drop null-valued keys as srtctl apply does, and h200-dgxc GLM-5.2 FP8 AgentX keeps serving the Hub snapshot. * feat(srt): generate the bound recipes of AgentX points too infx generate now binds every srt-slurm point of the selected keys, AgentX as well as fixed-sequence, single- and multi-node, through the binder the launcher runs. The workflow environment it reproduces gains the AgentX inputs (IS_AGENTIC, KV_OFFLOADING, TOTAL_CPU_DRAM_GB, THINKING_MODE) and the multi-node request fields. A multi-node point takes the binder inputs the launcher derives from the one cluster its runner label schedules on: launch_path, the lane's POWER_LANES decision and the cluster's power-exporter-port give the DCGM telemetry block and bound concurrencies, and config.agentic_client_env, which no longer needs an SrtRun, gives the AgentX client paths. Such a point fails when its label spans several clusters or the lane refuses its power. The manifest records the cluster. * refactor(srt): compose the DSv4.1-Flash ATOM AgentX recipe from a fragment so binding it fails on its model block and benchmark MODEL. Like the other AgentX fragments it drops them, the client command its shared block supplies, and WEKA_LOADER_OVERRIDE, which names the corpus dsv41flash replays by default. Every point binds to main's selected variant without that override. * refactor(srt): deslop AgentX composition The multi-node binder reads IS_AGENTIC instead of defaulting a missing value to fixed-sequence, model_paths drops a guard the binder makes redundant, and RECIPES.md points at the bound-key tables instead of copying them. * refactor(srt): drop checks and env placeholders the binder makes dead select_recipe binds each variant before validate_recipe sees it, and the binder writes model.path, model.container, model.precision and benchmark.env.MODEL from the same environment, so those four comparisons can never fail. infx generate set KV_OFFLOADING and TOTAL_CPU_DRAM_GB to empty or zero placeholders for fixed-sequence points that nothing reads; it now sets them only for AgentX points. * refactor(srt): share the multi-node binder inputs with infx generate The launcher and infx generate each derived the DCGM exporter port and the AgentX client paths from the cluster, lane, request and power decision, and infx generate must match the launcher. Both now call config.binder_inputs. infx generate imports the launcher modules at the top: they import nothing that imports it back. * test(srt): drop binder tests that echo their inputs or repeat launcher tests The bound AgentX client test asserted values it passed in, and the CLI test's power and client-env tail repeated the srt driver tests that run the same module with those flags. The fragment-rejection test no longer pins the explanatory suffix, and an unused constant goes. * docs(srt): condense infx generate guidance RECIPES.md states the infx generate binder inputs in fewer words and drops a sentence the power-row table already makes. CONFIGS.md puts mounts back beside volume-mounts, which it describes, and the GLM-5.2 GB200 power comment moves onto the power field. * docs: drop recipe image and telemetry rules the binder now owns Recipes may no longer set model.container, identity.container.image or telemetry.enabled, yet the Klaud prompt and docs still asked for matching images, nuke.md still ran a script that rewrites model.container (it now fails on every recipe), and the power docs still gated energy collection on recipe telemetry. They now say the binder writes the master image and a power: true master row enables DCGM telemetry. * refactor(srt): compose main's new GB300 and B300 AgentX recipes from fragments SemiAnalysisAI#3816, SemiAnalysisAI#3840 and SemiAnalysisAI#3744 added full recipes: the Qwen3.8 GB300 and Qwen3.5 GB300 DEP16 recipes carry model blocks, the client command, benchmark MODEL and the client paths the launcher hands multi-node points, and both new recipes and the DSv4.1-Flash B300 one set the identity repo and image the binder writes. Like the other AgentX fragments they drop them, and WEKA_LOADER_OVERRIDE, which names both models' default corpus. The Qwen3.8 recipe keeps the TOTAL_CPU_DRAM_GB and KV_OFFLOADING the launcher checks against the point. Each point binds to main's selected variant, with the Qwen3.5 model alias bound as its Hub id like the other GB300 Qwen3.5 AgentX points. * refactor(srt): share one agentic.yaml block between the AgentX lanes agentic-single.yaml and agentic-multi.yaml were byte-identical. Single- and multi-node AgentX points now compose with configs/srt-recipes/agentic.yaml; the bound recipes are unchanged. * test(srt): reject a single-node variant sized for another DRAM budget The launcher rejects a variant whose TOTAL_CPU_DRAM_GB differs from the point's, the check that ties a recipe's literal host pools to the row's dram-utilization. No test covered it. * fix(srt): bind the AgentX client's IS_MULTINODE and power switch Aggregated multi-node AgentX recipes pinned IS_MULTINODE=false so the client would not measure power, and so published a single-node result shape for jobs that span nodes (the DSv4 GB300 TP8 agg point runs on two). The client's conc_N naming had to accept the workflow's CONC_LIST to keep collecting them. The Qwen3.5 B300 disagg recipe pinned ENABLE_AGENTX_POWER=0 instead. The binder now writes both into every multi-node AgentX recipe from the launcher's own state: IS_MULTINODE from the request, and ENABLE_AGENTX_POWER=1 exactly when the power decision collects AgentX or adapter power. A fragment that sets either fails composition. The pins are deleted, and so are the TP, PP_SIZE, PCP_SIZE and EP_SIZE copies: only single-node points read them, and multi-node results take their topology from the workflow's PREFILL_* and DECODE_* env. The client names results per concurrency on IS_MULTINODE alone. The DSv4 GB300 vLLM 1P4D recipe also pinned RUNNER_TYPE=gb300 and PREFILL_TP=1. Its prefill worker is DEP4 (tensor-parallel-size 1, data-parallel-size 4, expert parallel) on four GPUs, which the master row records as tp 4, ep 4, dp-attn like its DEP4 siblings; the pin published prefill_tp 1 and counted one prefill GPU instead of four. Those pins and the unread OFFLOADING copy are deleted, so the row publishes its runner label and topology like its siblings. The client's power mode, result directory and file name are unchanged for every point, except that the 192 multi-node points without power switch from an invalid power verdict (multinode_power_contract_missing) to none. The 35 formerly pinned points publish is_multinode true with prefill and decode topology. * refactor(srt): stop pinning workflow env in recipe fragments Fragments still set values the workflow exports from the matrix point or from runtime_settings.sh. The Kimi-K3 GB200 and GB300 fragments pinned REQUIRE_POWER=1, two TensorRT-LLM fragments pinned MAX_MODEL_LEN, MiniMax-M3 GB200 TRT copied KV_OFFLOAD_BACKEND_METADATA, the DSv4 GB300 frontends copied MODEL, MODEL_PREFIX, FRAMEWORK, PRECISION, DURATION, KV_OFFLOADING and CONC, and GLM-5.3 TileRT set MODEL=/model, which it published as the model. The binder now writes REQUIRE_POWER into every multi-node AgentX recipe: 1 when the workflow requires power or the lane's power rule does. The gb200-nv and gb300-nv Kimi-K3 AgentX rules require it, which keeps the pinned points strict and no others. BOUND_ENV rejects every name the workflow exports from the matrix point. MAX_MODEL_LEN goes (the AgentX client never reads it; evals take the job's), and so do the frontend copies, which the Dynamo frontend never reads. GLM-5.3 names its local checkpoint as the client's TOKENIZER, which the replay reads before MODEL, so it publishes zai-org/GLM-5.3 and keeps its tokenizer. Copies that equal the workflow's value go too: VLLM_ENGINE_READY_TIMEOUT_S 3600, PYTHONDONTWRITEBYTECODE, AIPERF_DYNAMO_SESSION_TIMEOUT_SECONDS 3600 and the MORI, NCCL and SGLang settings runtime_settings.sh exports for sglang-disagg. Every process keeps its effective value through the job environment. Fragments keep values that override the workflow's on purpose: the Dynamo routing and session settings, longer engine-ready timeouts, MORI_IO_QP_MAX_SEND_WR 32767 and single-node variant labels.
…affected config keys (SemiAnalysisAI#3812) * feat(matrix): fingerprint the concrete recipe each benchmark row runs recipe-fingerprint hashed only the generated matrix row, so editing a fragment, a shared block or an AgentX variant left it unchanged. It now hashes the row's fields (all but conc, exp-name and the fingerprint) with the one variant the launcher submits for the row: the fragment composed with its shared block (and the telemetry block for a power row), the selected variant and the bound image, model, precision and, for fixed sequences, lengths. Concurrency values and the job name stay out, and the cluster facts the launcher adds (staged paths, mounts, client cache paths, exporter port, fabric) never enter it, so one recipe keeps one fingerprint across concurrencies and clusters. An eval-srt-recipe contributes only its path. Rows without an srt-slurm recipe keep the row hash. The planner runs without srtctl, so srt_slurm/variants.py reproduces srtctl's override-bundle expansion and the planner reuses the launcher's variant selection and binding. A row that no variant serves now fails planning instead of its launch. Klaud matches published fingerprints when it freezes a baseline. Producer revisions now recompute theirs with their own infx.matrix.fingerprint, or the row hash when they predate it, and the canonical family fetches the head's shared blocks with its recipes. * feat(matrix): list the config keys a change affects `python -m infx.matrix.changed --base <ref>` generates the base matrix with the base revision's own generator, fingerprints both sides under their own recipes and prints every config key whose points (recipe fingerprint and concurrency) were added or removed, with point counts. --json prints the same report for tools. Authors use it to decide which keys a perf-changelog entry should rerun. * test(matrix): drop assertions other tests already make test_expansion_matches_srtctl already compares the zip, override and unselected expansions in full against srtctl, and the change report's added and removed concurrencies already prove the edited fragment's fingerprints changed. * refactor(matrix): deslop recipe fingerprints variants.py owns without_nulls and the zip selector pattern that synthetic_acceptance duplicated. bound_variant no longer gates power_port on the row's power, which only the fingerprint needed; the fingerprint passes its stand-in port for power rows itself. Drop the CONC_LIST exclusion that no bound recipe can carry, the listed flag in producer regeneration, and defensive exp-name lookups, and trim narrating docstrings. * docs: condense recipe fingerprint guidance * test(fingerprint): edit the shared agentic.yaml block in the AgentX case
… refs (SemiAnalysisAI#3820) * refactor(srt): take cluster fabric facts from runners.yaml Recipes stop hardcoding the cluster's network hardware. Each srt-slurm cluster profile gains a typed fabric record: ucx-net-devices, nccl-ib-hca, ib-devices, socket-ifname and mooncake-devices are lists, and mooncake-gid-index, mori-rdma-tc and mori-io-tc are numbers. A recipe value '@fabric.<name>' takes the job cluster's field after binding, lists comma-joined and numbers as decimal strings. The launcher hands both binders the cluster's rendered fabric; an unknown name, a reference inside a longer string, or a field the cluster does not set fails before submission. Nothing resolves references at plan time. Literals equal to their cluster's fact become references in 72 active recipes on gb200-nv, gb300-nv, b200-nscale, b300-dsxe, h200-dgxc and mi355x-amds. Deliberate per-recipe values stay literal: the MI355X UMBP recipe's permuted rail order, B200 Mooncake's four of eight HCAs, and empty Mooncake device lists that leave discovery to Mooncake. infx generate resolves references for a point whose runner label names one cluster and records that cluster in the manifest; otherwise references stay and the manifest's cluster is null. * feat(srt): check recipe fabric references at matrix generation Matrix generation now fails a row whose selected recipe variant (srt-recipe or eval-srt-recipe) has an '@fabric.<name>' value naming no Fabric field, or a field that a cluster the row's runner label reaches does not set. The launcher still resolves references at bind time and fingerprints hash them as written. Cluster env and srt-slurm.host-setup.env values may reference the cluster's own fabric the same way and resolve when the record loads; env keeps rejecting commas after resolution. mi355x-amds takes MORI_RDMA_TC and IBDEVICES from its fabric instead of repeating them. * chore: reconcile srt-fabric-refs with lower branch after linearizing * fix(srt): keep the fabric reference constant importable on Python 3.10 The binder runs in srtctl's Python 3.10 venv, where importing infx.clusters fails on typing.Self. * refactor(srt): merge RDMA NIC fabric fields into rdma-devices nccl-ib-hca, mooncake-devices and ib-devices held the same RDMA HCA list on every cluster that set them. One Fabric field, rdma-devices, now carries that list for NCCL_IB_HCA, the Mooncake device_name, SGLang disaggregation-ib-device and IBDEVICES, and each cluster lists its HCAs once. Bound recipes are unchanged. Rows whose recipes reference the field get new fingerprints, since references are hashed as written. * refactor(srt): take the GB300 DSv4 Mooncake RDMA devices from the fabric prefill role. Like the other recipes' NIC lists, both read '@fabric.rdma-devices', which renders the same mlx5_0..mlx5_3 list on gb300-nv. * test(srt): drop fabric assertions other tests already make The workload tests already prove the multi-node binder resolves --fabric, the single-node driver case proves the launcher passes the job cluster's rendering, and the matrix tests already cover an unknown fabric name. * refactor(srt): fail fabric lookups for clusters without a record infx generate now raises the binder inputs' missing srt-slurm settings error instead of binding an empty fabric, and the matrix check reads the record a cluster label names instead of treating a missing one as an empty fabric. * docs: condense fabric reference guidance * refactor(srt): take the GB200 DSv4 Mooncake RDMA devices from the fabric the decode stores and the prefill role. Like the GB300 recipes, both read '@fabric.rdma-devices', which renders the same mlx5_0..mlx5_3 list on gb200-nv. * refactor(srt): resolve every fabric reference with resolve_fabric Cluster env and host-setup.env values and the matrix plan-time check now call the binder's resolve_fabric instead of Fabric.field_name, Fabric.resolve and a second string walk. An unset field reads "this cluster sets no srt-slurm.fabric.<name>" everywhere, and the matrix error names the recipe and the first cluster that fails. A runner that no cluster label lists, such as gb300-nv, still has its reference names checked. A cluster env: null is now rejected instead of read as an empty mapping. The embedded-reference generation case is dropped: it ran the same resolve_fabric branch as the binder's IBDEVICES case. * refactor(srt): name fabric fields after cluster RDMA facts Fabric fields now name what the cluster has rather than the software that reads it: ucx-net-devices becomes rdma-device-ports, mooncake-gid-index becomes rdma-gid-index, and mori-rdma-tc and mori-io-tc, both 104 on mi355x-amds, become one rdma-tc. Recipes and the mi355x-amds cluster env keep their engine variable names (UCX_NET_DEVICES, MC_GID_INDEX, MORI_RDMA_TC, MORI_IO_TC) and point them at the new references, so every bound recipe renders the same values.
…er config (SemiAnalysisAI#3821) * refactor(srt): bind script-installed component versions from the master config Repo setup scripts that install a component now read its version from the recipe instead of pinning it, and the binder writes the point's master version after variant selection, so the submitted recipe shows what gets installed: - vllm-router.sh (was vllm-router-0.1.14.sh) reads ROUTER_VERSION. - vllm-mooncake.sh (was vllm-mooncake-0.3.11.sh) reads KV_OFFLOAD_BACKEND_VERSION; kimik3-b300-mooncake.sh and the mooncake-master preambles of the Kimi-K3 B300 and MiniMax-M3 H100 recipes run it instead of pinning their own pip install. - lmcache-mp-rocm.sh reads KV_OFFLOAD_BACKEND_VERSION. - glm5.3-tilert-rocm.sh reads ROUTER_VERSION, the tilert-pd-router release; the GLM-5.3 recipe drops TILERT_VERSION. The binder writes the version into the top-level environment when setup_script is one of these installers (workers and static routers) and into the env of each service whose preamble runs one, since services do not inherit environment. A point that does not declare the version fails before submission, a vllm-router pin in SETUP_PIP_PACKAGES must match the master router, and a fragment may not set the version variables. The launcher and infx generate also fail a bound recipe whose setup_script is in neither configs/ nor the srt-slurm checkout's, where srtctl itself only warns. * test(srt): drop duplicate component version tests The router half of the binding test reruns the setup_script branch the Mooncake half covers, the pinned-router acceptance only echoed its input, and the setup-script check is called directly instead of through a copy of the multi-node CLI harness. * refactor(srt): deslop component version binding _fragment_versions yields its findings, the one-entry PIP_COMPONENTS registry becomes the vllm-router check it was, and lmcache-mp-rocm.sh reads KV_OFFLOAD_BACKEND_VERSION without an alias. * docs: condense component version guidance * refactor(srt): drop the binder's fragment path from component version errors bind_workload took a required source only to prefix two component version errors with the fragment path. No other binder error names the fragment, and the planner's recipe fingerprint already prefixes the srt-recipe, so those errors printed the path twice. check_setup_script keeps its source.
… config key (SemiAnalysisAI#3824) * fix(launch): apply lane job edits to the bound recipe prepare_recipe edited the staged fragment before the binder ran. Its text edits only find a flat recipe's top-level name and six-space watchdog-timeout, so bundle variants kept their recipe names as srtctl's run name and never got the cluster's dist-timeout. The edits now apply to the recipe the binder writes, which is flat for every variant. Slurm job names were unaffected: srtctl names the job from RUNNER_NAME. * refactor(recipes): fold per-point recipe families into one bundle per entry Master-config entries whose rows each named their own standalone recipe now select overrides of one <agg|disagg>[-axis]-variants.yaml: base holds what the points share and each override_<topology>_c<conc> holds only its point's deltas, including the job name. Ten AgentX entries (50 recipes) and seventeen fixed-sequence entries (184 recipes) move into 27 bundles. Every moved row's selected variant equals its old recipe after composition, with and without the power telemetry block, and the full-sweep matrices differ only in those rows' srt-recipe strings. Entries whose points mix recipe designs, such as a TP low-latency recipe beside wide-EP ones, keep their standalone files. Section-header comments, commented-out settings and per-point labels that restate the topology are not carried into the bundles. RECIPES.md describes the convention. * refactor(recipes): fold the GLM-5.2 GB300 SGLang AgentX point recipes into bundles the other AgentX families, the three aggregate points select overrides of agg-variants.yaml and the three low-latency 1PxD points overrides of disagg-variants.yaml; each override selects exactly its old recipe. The DEP8/DEP16 2P1D point is a different design and keeps its own file. * test(launch): drop recipe edit assertions other tests already make The health-check floor and model path of a bound bundle variant are already covered by test_srt_config and the multi-node lane test, and the bundle-shaped health budget test exercises inputs that only reach raise_health_attempts as the binder's flat output. * refactor(launch): inline recipe_relpath into recipe_mirror_path Its other caller went away when the lane edits moved to the bound recipe. * docs: drop the per-override name rule and tidy bundle comments Every launch renames the bound recipe and fingerprints exclude name, so bundles need not set one per override. Drop a misindented commented-out flag and move the NATS comment above request_plane. * refactor(recipes): select the Qwen3.5 GB300 FP8 AgentX midcurve points from the pareto bundles of the pareto entries by naming those points' standalone recipes, which the bundles replace. Their rows select the matching pareto overrides, so each still runs exactly the pareto point's recipe rather than a copy that could drift; the DEP16 points keep disagg-dep16-mtp6-variants.yaml. RECIPES.md notes that an entry rerunning another entry's points selects its overrides. * refactor(launch): edit the bound recipe as data prepare_recipe now edits the binder's safe_dump output through one safe_load/safe_dump round trip instead of regex and line edits that depended on PyYAML's indentation: it sets the top-level name, raises health_check.max_attempts to the floor, and adds dist-timeout to each role whose args set watchdog-timeout. For all 589 bound multi-node points of every master key and every multi-node recipe variant, with and without a dist timeout, the written recipe parses identically to the old edits. The helper tests fold into one prepare_recipe test. * refactor(recipes): drop the per-override names from the folded bundles The 29 bundles folded from per-point recipes carried each old file's top-level name into its override, and some no longer matched the topology. Nothing reads them: every launch renames the bound recipe to the runner's job name, srtctl names the Slurm job from RUNNER_NAME, and fingerprints exclude name. Variants now take srtctl's <base name>_<override key> fallback, which only changes infx generate output.
…idence (SemiAnalysisAI#4036) 明确参数化命令和适用官方文档可满足配方覆盖要求；将缺少草稿精度证据改为警告，保留已证实违规的失败判定，并同步清单说明和发布器回归测试。 Signed-off-by: Xin Li <xinli@nvidia.com>
…-K3 PD 配置 (SemiAnalysisAI#3582) * feat(launch): support read-only Kimi-K3 assets 为 Kimi-K3 PD 声明模型资源与只读 draft/provider 挂载；保留现有可写缓存目录的创建行为，不改共享镜像准备和任务生命周期。 * feat(config): add native Kimi-K3 PD recipes Add topology-named native variants with a digest-pinned official worker image, role-owned serving settings and default evaluation selection. Preserve upstream request-ID randomization and keep benchmark and cluster configuration separate. 新增按拓扑命名的原生配置变体，使用 digest 固定的官方 worker 镜像、按角色管理的 serving 参数及默认精度评测；遵循上游 request-ID 随机化，将 benchmark 和集群配置分层。 * fix(srt-slurm): backport discovery connector template binding Carry NVIDIA/srt-slurm SemiAnalysisAI#508 through the existing patches directory at the shared submodule API. Exercise real job-local patch application and conflict handling, then run the upstream P/D command-rendering regressions in an isolated checkout. Remove the patch once the shared dependency includes it. 沿用既有 patches 目录回补 NVIDIA/srt-slurm SemiAnalysisAI#508，与共享 submodule API 保持一致。验证任务私有 checkout 的补丁应用和冲突退出，再在隔离进程中执行上游 P/D 命令渲染回归；共享依赖包含该修复后删除补丁。 * refactor: use native asset mounts for Kimi-K3 recipes Declare draft checkpoint and Ionic provider mounts through existing recipe and cluster configuration. Keep shared launch code unchanged, carry only the SemiAnalysisAI#508 runtime backend fix, and record recipe details in YAML and the changelog. 中文：通过现有 recipe 和集群配置声明 draft checkpoint 与 Ionic provider 挂载。共享 launch 代码保持不变，仅携带 SemiAnalysisAI#508 的运行时 backend 修复，将 recipe 信息统一记录在 YAML 和 changelog。 * perf(config): tune and simplify Kimi-K3 PD recipe settings Enable INT4 QuickReduce with a 256 MiB limit and FlyDSL stage-2 FP8 on both roles. Use default nonblocking waits and breakable-graph settings. Scope draft and Ionic mount paths to the recipe. Configure 1P1D c48/c56, 1P2D c24/c48 and 1P3D c12/c24 with prefill-only CPU offload and point-specific decode graph coverage. Update the existing performance changelog entry. 中文：在 P/D 两侧启用 INT4 QuickReduce、256 MiB 大小上限和 FlyDSL stage-2 FP8，使用默认非阻塞等待与 breakable graph 设置。将 draft 和 Ionic 挂载路径收束到 recipe。配置 1P1D c48/c56、1P2D c24/c48 和 1P3D c12/c24，使用仅 prefill 侧 CPU offload 与各点对应的 decode graph 覆盖，并更新当前 PR 的性能变更记录。 * fix(config): use default AgentX failure threshold for Kimi-K3 PD Remove the recipe's live failure threshold override so all Kimi-K3 PD variants use the pinned AgentX scenario's 10% default. Keep the final result threshold, both roles' stream interval, and optional server-metrics validation unchanged. 移除 Kimi-K3 PD 配方的运行中错误率覆盖，所有变体沿用固定版本 AgentX 场景的默认 10% 门槛。最终结果校验门槛、P/D 流式输出间隔及可选 server-metrics 校验保持不变。 * fix(srt): clarify discovery patch scope 说明 SemiAnalysisAI#508 仅修复 srt-slurm 启动器的 discovery 配置渲染，不修改推理引擎、kernel、模型计算或镜像。补丁应用后的代码保持不变。 * fix(kimik3): migrate PD recipe to srt-recipe selection and AgentX fragments Main now selects srt-slurm variants with srt-recipe-dir/srt-recipe and binds model, image, precision, client env and concurrencies from the master row, so the recipe drops those keys and the shared agentic.yaml supplies the client command. --------- Co-authored-by: Cam Quilici <cjquilici@gmail.com> Co-authored-by: billishyahao <bill.he@amd.com>
* chore: remove comments from srt-slurm recipe YAML 删除单节点和多节点 srt-slurm 配方 YAML 中的注释，保持配置内容不变。 * chore: keep explanatory recipe comments Restore inline comments that explain non-obvious settings (workarounds, timeouts, memory sizing, bug references). Header blocks, section labels, topology/provenance descriptions, perf result notes and commented-out config stay removed. --------- Co-authored-by: cquil11 <60715037+cquil11@users.noreply.github.com>
…/ 清理 infx 注释并保留指定目录 (SemiAnalysisAI#4039) * chore: remove infx comments outside bench_serving 删除 infx 中 bench_serving 目录之外的说明性注释，保留功能性指令和代码行为。 * chore: preserve golden_al_distribution comments 按要求保留 golden_al_distribution 的全部注释和原始内容。
Ban header comments and historical comments; comments must give succinct technical explanation of a specific line or block. 禁止文件头注释和描述代码历史演变的注释；注释只能简洁地从技术角度解释具体代码或配置。
…Agent 指南 (SemiAnalysisAI#4042) * docs: remove redundant pipeline guides 删除三份重复说明文档并修复引用，保留 Claude 命令、技能、已知限制及其操作文档。 * docs: consolidate result and ingest recovery instructions 将结果检查和恢复流程合并到 Claude 恢复命令，删除重复指南并保留验证与权限边界。 * docs: remove requested guides and repair agent references 删除指定的说明文档，保留四项核心文档与全部 Claude 命令，将必要操作规则移入现有入口并修复引用。
从 AMD 主配置的 CODEOWNERS 条目中移除无效所有者 @yctseng0211，保留其余所有者。
…SemiAnalysisAI#3781) * feat(power): bind native GPU telemetry into composed recipes * fix(power): include native telemetry in revision recipe loading * chore: remove PR 3781 perf changelog entry * docs: trim AMD exporter notes in CONFIGS.md * test(power): remove producer replay test and dependency * refactor(power): bind single-node required power from REQUIRE_POWER The launcher no longer re-reads the bound recipe, patches its request, or writes $GITHUB_ENV. REQUIRE_POWER alone sets telemetry.required, so a single-node fragment may not set it, and result processing takes the producer SHA from the launcher stamp as the multi-node workflow does. * refactor(power): gate single-node telemetry only when composing select_recipe already omits the native telemetry block for eval-only points, so the binder no longer rewrites telemetry.enabled. * refactor(power): share the AgentX power adapter invocation Single-node and multi-node AgentX run the adapter through one helper, and the single-node adapter writes its sidecar into the workspace instead of the launcher copying it there. * refactor(power): derive single-node AgentX GPU count from the aggregate The aggregate already records num_gpus from the same environment as GPU_COUNT, so the adapter drops --expected-num-gpus. * chore(runners): share the AMD exporter settings through YAML anchors * ci(power): drop power-audit upload paths no job writes --------- Co-authored-by: Cam Quilici <cjquilici@gmail.com>
…wer_model] 支持跨系统的实测 CPU socket 功耗输入 (SemiAnalysisAI#4031) * feat: add measured x86 CPU socket input to power_model 中文：为 power_model 新增实测 x86 CPU socket 功耗输入，仅替代建模的 x86 CPU 功耗 * refactor: make x86 CPU socket validation explicit 中文：明确检查 x86 CPU socket，并删除重复的测试注释。 * feat: apply both measured socket inputs across system families 中文：让两种实测 socket 功耗输入适用于 HGX 和机架系统。
# Conflicts: # .github/workflows/benchmark-tmpl.yml # benchmarks/benchmark_lib.sh # configs/runners.yaml
|
Thanks for the contribution!
中文感谢你的贡献！
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
upstream/main(through69f12085e, 645 commits) into the fork'smain, preserving the fork's BYO-endpoint (cluster:remote-bench) feature..github/workflows/benchmark-tmpl.yml(keeps fork'sswebench-gen-mode+ remote-bench inputs/env alongside upstream's newrequire-powerdefault), and keeps fork-modifiedbenchmarks/benchmark_lib.sh+configs/runners.yamlat their legacy paths (upstream deleted them when it moved the e2e tree intoinferencex-e2e/).utils/aiperfsubmodule entry (pinfaf3f8b3) that upstream removed; upstream now installs the PyPIagentx-harness==1.0.7per-job venv instead.Known follow-up
The new upstream workflow runs from
inferencex-e2e/and readsinferencex-e2e/configs/runners.yaml, which has nocluster:remote-benchrunner. The fork's remote-bench recipes underbenchmarks/single_node/agentic/*-remote-bench.share kept but are not dispatched by the new launcher until ported to theinferencex-e2e/layout.中文
摘要
upstream/main（至69f12085e，共 645 个提交）合并到 fork 的main，保留 fork 的 BYO endpoint（cluster:remote-bench）功能。.github/workflows/benchmark-tmpl.yml的冲突（保留 fork 的swebench-gen-mode与 remote-bench 输入/环境变量，同时合入 upstream 新的require-power默认值）；上游把端到端目录迁入inferencex-e2e/时删除了benchmarks/benchmark_lib.sh和configs/runners.yaml，本 PR 在原路径保留 fork 修改后的版本。utils/aiperf子模块条目（pinfaf3f8b3）；上游现在改为按任务安装 PyPI 包agentx-harness==1.0.7。已知后续工作
上游新工作流在
inferencex-e2e/下运行并读取inferencex-e2e/configs/runners.yaml，其中没有cluster:remote-benchrunner。fork 位于benchmarks/single_node/agentic/*-remote-bench.sh的 remote-bench 脚本已保留，但在迁移到inferencex-e2e/布局之前，新 launcher 不会调度它们。