feat(serving): replicated dynamo.vllm behind the Dynamo frontend + LiveKit 2xTP4 arm / 多副本 dynamo.vllm 与 LiveKit 2xTP4 配置

.agents/skills/serving-bench/SKILL.md

@@ -16,6 +16,8 @@ image: vllm/vllm-openai:v0.29.0     # required
 model: Qwen/Qwen3.8-27B             # HF repo id, never a local path
 gpus: 1                             # GPUs the command needs
 replicas: 1                         # optional: N engine copies, gpus/N GPUs each; N > 1 needs router: with {{engine_urls}}
+host_cpus: 64                       # optional, hardware-hcm-8x_* only: job CPUs / RAM GB (default 8 CPU + 64 GB per GPU
+host_mem_gb: 192                    #   plus HiCache/Mooncake host pools); set only to ask for something else
 precision: bf16                     # optional (bf16 default), naming only
 env:                                # optional, the engine's env vars; quote booleans
   PYTORCH_CUDA_ALLOC_CONF: expandable_segments:True
@@ -76,11 +78,14 @@ It validates first. On `ERROR <field>: <fix>` lines, fix the YAML and rerun; not
 
 ## 4. Results
 
-- Status: `gh run view <run-id> --repo vngcloud/InferenceX`. One job per `conc`. Expect roughly 20 min of image pull, model load and warmup on top of the scenario duration (180 s smoke ≈ 25 min).
-- Artifacts (`gh run download <run-id> --repo vngcloud/InferenceX -n <artifact>`), names start with `<name>_c<conc>`:
-  - `agentic_<name>_c<conc>_…`: `results/` with `serving.yaml`, `router_command.txt`, `router.log`, `server_command.txt`, `server.log`, `benchmark_command.txt`, `benchmark.log`, `aiperf_artifacts/`
-  - `bmk_agentic_<name>_c<conc>_…`: aggregate result JSON
-  - `server_logs_<name>_c<conc>_…`: server logs
+- Status: `gh run view <run-id> --repo vngcloud/InferenceX`. One job per `conc`. Expect roughly 20 min of image pull, model load and warmup on top of the scenario duration (180 s smoke ≈ 25 min), plus the wait for the runner (hours when the box is busy). An old `gh` that prints no jobs: `gh api repos/vngcloud/InferenceX/actions/runs/<run-id>/jobs`.
+- Artifacts: `gh run download <run-id> --repo vngcloud/InferenceX -D <dir>` takes all of them (`-n <artifact>` one). Names start with `<name>_c<conc>`:
+  - `agentic_<name>_c<conc>_…`: `serving.yaml`, `dataset.txt`, `router_command.txt`, `router.log`, `server_command.txt`, `server.log`, `benchmark_command.txt`, `benchmark.log`, `gpu_metrics.csv`, `aiperf_artifacts/` (with `replicas: N`, replica `i` > 0 also has `server_command.<i>.txt` / `server.<i>.log`)
+  - `bmk_agentic_<name>_c<conc>_…` and `results_bmk/agg_bmk.json` (all concs): the aggregate result
+  - `server_logs_…`, `gpu_metrics_…`, `power_audit_…`, `run-stats`
+- Read the aggregate: `request_metrics.latency.ttft` (s; mean/p50/p90/p95), `request_accounting` (`records_profiled`, `records_error_dropped`, `error_categories`; warmup drops are normal), `server_metrics.cache.gpu_cache_hit_rate`, `request_metrics.throughput`, `avg_total_gpu_power_w`. `tp` and the `tp<N>` in names are the YAML's `gpus`, not the engine's TP: with `replicas` read the TP from `server_command.txt`. `scenario_type` says `agentic-coding` for every agentic scenario.
+- LiveKit scenarios (`custom-agentic-livekit111868-*`): after downloading the whole run, from the repo root run `uv run -q --no-project --python 3.12 --with pyyaml --with huggingface_hub python utils/serving/livekit_ttft.py <dir>`. It prints TTFT per conc split by a room's first turn vs later turns; report both, since first turns set the p95 tail. It fetches the plan CSV from the lock (`<dataset>-plan`, a public repo: no token needed); `--plan <csv>` uses a local copy. On `-v1-warm` (dial warm-up) runs, warm-up requests are excluded from the TTFT views and reported on their own last line; report their p95 too, since they must finish before the room's first turn.
+- Not failures: `503 Service Unavailable` lines in `server.log` / `router.log` before the warmup starts (readiness polling while the model loads), and errors in `router.log` at teardown after the profiling phase. Check the timestamps against `benchmark.log`.
 - Dataset errors appear as `ERROR dataset <name>: …` lines in the job log (`gh run view <id> --repo vngcloud/InferenceX --log-failed`); the first tau2 run downloads ~1.1 GB.
 - A failed job: read `server.log` first (engine flags, OOM, parser errors), then `benchmark.log` (aiperf). Router path: read `router.log` first when the job fails before aiperf starts. `gh run view <run-id> --repo vngcloud/InferenceX --log-failed | tail -100` shows the launcher side (slurm, enroot).
 - If the engine crashes or the job fails, report the error and the config as given. Do not change flags or env, or redispatch a modified config, on your own: the config is the user's, a change is their decision.

.github/workflows/benchmark-tmpl.yml

@@ -227,11 +227,11 @@
   # scenario in one run). scenario-map is keyed by the row's exp-name; the flat
   # serving.scenario is the single-scenario fallback, empty for scenario lists
   # so a missing map entry fails loudly in serve_generic.sh (check_env_vars).
   SERVING_SCENARIO: ${{ inputs.serving != '' && (fromJSON(inputs.serving).scenario-map[fromJSON(inputs.config).exp-name].scenario || fromJSON(inputs.serving).scenario) || '' }}
   SERVING_LOADER: ${{ inputs.serving != '' && (fromJSON(inputs.serving).scenario-map[fromJSON(inputs.config).exp-name].loader || fromJSON(inputs.serving).loader) || '' }}
   # '1' for sweep scenarios (adaptive_search), '0' otherwise — never '': check_env_vars
   # rejects empty values, so '' would abort every non-sweep serving run in serve_generic.sh.
   SERVING_SEARCH: ${{ inputs.serving != '' && ((fromJSON(inputs.serving).scenario-map[fromJSON(inputs.config).exp-name].search || fromJSON(inputs.serving).search) && '1' || '0') }}
   SERVING_YAML: ${{ inputs.serving != '' && fromJSON(inputs.serving).yaml || '' }}
   # Router sidecar (any router, spec 2026-10-14); only runners/launch_hardware-hcm.sh reads these.
   ROUTER_IMAGE: ${{ inputs.serving != '' && fromJSON(inputs.serving).router-image || '' }}
@@ -240,6 +240,9 @@
   ROUTER_PROFILE: ${{ inputs.serving != '' && fromJSON(inputs.serving).router-profile || '' }}
   ROUTER_METRICS_PORT: ${{ inputs.serving != '' && fromJSON(inputs.serving).router-metrics-port || '' }}
   REPLICAS: ${{ inputs.serving != '' && fromJSON(inputs.serving).replicas || '' }}
+  # Host CPUs / RAM GB of the allocation (empty: launcher default); only launch_hardware-hcm.sh reads these.
+  HOST_CPUS: ${{ inputs.serving != '' && fromJSON(inputs.serving).host-cpus || '' }}
+  HOST_MEM_GB: ${{ inputs.serving != '' && fromJSON(inputs.serving).host-mem-gb || '' }}
   # Storage sidecar (Mooncake master for HiCache L3 etc.); only launch_hardware-hcm.sh reads these.
   STORAGE_IMAGE: ${{ inputs.serving != '' && fromJSON(inputs.serving).storage-image || '' }}
   STORAGE_CMD: ${{ inputs.serving != '' && fromJSON(inputs.serving).storage-cmd || '' }}
@@ -261,7 +264,7 @@
 
 jobs:
   benchmark:
     runs-on: >-
       ${{ fromJSON(
         vars.PRIORITY_SCHEDULER_ENABLED == 'true' &&
         (

benchmarks/dynamo-vllm/Dockerfile

@@ -0,0 +1,17 @@
+# vLLM + Dynamo (aggregated dynamo.vllm workers behind python3 -m dynamo.frontend) for serving YAMLs.
+#
+# vLLM stays the base image's: ai-dynamo is installed without its vllm extra, and aggregated
+# serving needs no NIXL. 1.6.0.dev20261018 is the first ai-dynamo whose vLLM worker publishes
+# Gemma-4's hybrid KV groups to the KV router (vLLM >= 0.31's get_kv_cache_group_metadata).
+#
+#   docker build -t <registry>/dynamo-stack/vllm-runtime:0.31.0-dyn1.6.0.dev20261018 benchmarks/dynamo-vllm
+ARG VLLM_IMAGE=vllm/vllm-openai:v0.31.0
+FROM ${VLLM_IMAGE}
+ARG DYN_VERSION=1.6.0.dev20261018
+RUN pip install --no-cache-dir "ai-dynamo==${DYN_VERSION}" "ai-dynamo-runtime==${DYN_VERSION}" blake3 uvloop \
+ && python3 - <<'PY'
+import vllm, dynamo.vllm, dynamo.frontend, dynamo._core
+from vllm.v1.engine.core import EngineCore
+assert hasattr(EngineCore, "get_kv_cache_group_metadata"), "vLLM lacks get_kv_cache_group_metadata"
+print("vllm", vllm.__version__)
+PY

benchmarks/dynamo-vllm/Dockerfile.g4strip

@@ -0,0 +1,19 @@
+# The vllm-runtime image with a patched ai-dynamo-runtime: Gemma 4 keeps its reasoning parser on
+# with thinking off, so the empty `<|channel>thought\n<channel|>` the model opens after a tool
+# result leaves `content` (ai-dynamo/dynamo#16781; vLLM's gemma4 reasoning parser does the same).
+# Perf is unchanged; only the tool-result turn's content differs.
+#
+# The wheel is built from ai-dynamo main 3763cee608 plus the two-line gate change in
+# lib/llm/src/preprocessor.rs; it is not published, so it goes in the build context:
+#   cp ai_dynamo_runtime-1.6.0-cp310-abi3-linux_x86_64.whl <ctx>/
+#   docker build -f Dockerfile.g4strip -t <registry>/dynamo-stack/vllm-runtime:0.31.0-dyn1.6.0.dev20261018-g4strip <ctx>
+ARG BASE_IMAGE=vcr.vngcloud.vn/108942-public69/dynamo-stack/vllm-runtime:0.31.0-dyn1.6.0.dev20261018
+FROM ${BASE_IMAGE}
+COPY ai_dynamo_runtime-1.6.0-cp310-abi3-linux_x86_64.whl /tmp/
+RUN pip install --no-cache-dir --no-deps --force-reinstall /tmp/ai_dynamo_runtime-1.6.0-cp310-abi3-linux_x86_64.whl \
+ && rm /tmp/ai_dynamo_runtime-*.whl \
+ && python3 - <<'PY'
+import vllm, dynamo.vllm, dynamo.frontend, dynamo._core
+assert vllm.__version__.startswith("0.31.0"), vllm.__version__
+print("vllm", vllm.__version__)
+PY

benchmarks/single_node/agentic/serve_generic.sh

@@ -21,6 +21,7 @@ umask 0000
 #   ROUTER_METRICS_PORT generic router: its /metrics port, empty = the router port
 #   ENGINE_PORT        generic router: the engine's HTTP port (launcher)
 #   DYN_SYSTEM_PORT    Dynamo engine metrics/health port (launcher)
+#   DYN_SYSTEM_PORTS, DYN_KV_EVENT_PORTS  Dynamo replicas: one system / vLLM KV event port per replica (launcher)
 
 source "$(dirname "$0")/../../benchmark_lib.sh"
 
@@ -72,8 +73,12 @@ HF_HUB_OFFLINE= TRANSFORMERS_OFFLINE= HF_DATASETS_OFFLINE= "$AIPERF_HF_CLI" down
 if [[ -n "${ROUTER_CMD:-}" ]]; then
     if [[ "${ROUTER_PROFILE:-}" == dynamo ]]; then
         check_env_vars DYN_SYSTEM_PORT
-        # Frontend (dynamo_frontend_*) + engine (sglang:* via the Dynamo system port).
-        export AIPERF_SERVER_METRICS_URLS="http://localhost:${PORT}/metrics,http://localhost:${DYN_SYSTEM_PORT}/metrics"
+        # Frontend (dynamo_frontend_*) + every engine (via its Dynamo system port).
+        metrics_urls="http://localhost:${PORT}/metrics"
+        for system_port in ${DYN_SYSTEM_PORTS:-$DYN_SYSTEM_PORT}; do
+            metrics_urls+=",http://localhost:${system_port}/metrics"
+        done
+        export AIPERF_SERVER_METRICS_URLS="$metrics_urls"
     else
         check_env_vars ENGINE_PORT
         # Router (its own metrics port, else the router port) + every engine's own /metrics.
@@ -94,7 +99,41 @@ esac
 
 SERVER_LOG="$RESULT_DIR/server.log"
 eval "set -- $SERVE_CMD"
-if [[ "${REPLICAS:-1}" -gt 1 ]]; then
+if [[ "${REPLICAS:-1}" -gt 1 && "${ROUTER_PROFILE:-}" == dynamo ]]; then
+    # Dynamo replicas (schema: dynamo.vllm only): N engines, each on its own GPU slice, Dynamo
+    # system port (health + metrics, DYN_SYSTEM_PORTS) and vLLM KV event port (DYN_KV_EVENT_PORTS);
+    # the frontend finds them through file discovery. Replica 0 keeps server.log, as below.
+    read -r -a replica_ports <<< "${DYN_SYSTEM_PORTS:-}"
+    read -r -a kv_ports <<< "${DYN_KV_EVENT_PORTS:-}"
+    if [[ -n "${CUDA_VISIBLE_DEVICES:-}" ]]; then
+        IFS=, read -r -a replica_gpus <<< "$CUDA_VISIBLE_DEVICES"
+    else
+        mapfile -t replica_gpus < <(nvidia-smi --query-gpu=index --format=csv,noheader)
+    fi
+    if (( ${#replica_ports[@]} != REPLICAS || ${#kv_ports[@]} != REPLICAS || ${#replica_gpus[@]} == 0 || ${#replica_gpus[@]} % REPLICAS )); then
+        echo "ERROR: replicas $REPLICAS needs $REPLICAS ports in DYN_SYSTEM_PORTS/DYN_KV_EVENT_PORTS (got '${DYN_SYSTEM_PORTS:-}' / '${DYN_KV_EVENT_PORTS:-}') and a GPU count divisible by $REPLICAS" >&2
+        exit 1
+    fi
+    gpus_per_replica=$(( ${#replica_gpus[@]} / REPLICAS ))
+    replica_pids=()
+    for ((replica = 0; replica < REPLICAS; replica++)); do
+        suffix=""
+        (( replica == 0 )) || suffix=".$replica"
+        replica_cmd=("$@" --kv-events-config "{\"enable_kv_cache_events\":true,\"publisher\":\"zmq\",\"endpoint\":\"tcp://*:${kv_ports[replica]}\"}")
+        write_command "$RESULT_DIR/server_command${suffix}.txt" "${replica_cmd[@]}"
+        DYN_SYSTEM_PORT="${replica_ports[replica]}" \
+        CUDA_VISIBLE_DEVICES="$(IFS=,; echo "${replica_gpus[*]:replica*gpus_per_replica:gpus_per_replica}")" \
+            "${replica_cmd[@]}" > "$RESULT_DIR/server${suffix}.log" 2>&1 &
+        replica_pids+=("$!")
+    done
+    for ((replica = 0; replica < REPLICAS; replica++)); do
+        suffix=""
+        (( replica == 0 )) || suffix=".$replica"
+        wait_for_server_ready --port "${replica_ports[replica]}" --server-log "$RESULT_DIR/server${suffix}.log" \
+            --server-pid "${replica_pids[replica]}"
+    done
+    SERVER_PID="${replica_pids[0]}"
+elif [[ "${REPLICAS:-1}" -gt 1 ]]; then
     # Serving-YAML replicas (spec 2026-10-16): N copies of SERVE_CMD, each on its own GPU slice and
     # ENGINE_PORTS entry. Replica 0 keeps server.log / server_command.txt: the vLLM server-metrics
     # adapter reads "GPU KV cache size" from server.log (replicas are identical).

docs/serving-yaml-procedures.md

@@ -61,6 +61,10 @@ The first run of a large model spends extra time downloading it and uses node di
    The script runs the same validation CI runs. On `ERROR <field>: <fix>` lines nothing was dispatched: fix the YAML and rerun. On success it prints the run URL. Use `--ref <branch>` to dispatch a branch other than the current one.
 4. Do not redispatch if the script reports that the run did not show up; check `gh run list --repo vngcloud/InferenceX -w e2e-tests.yml -L 5` first.
 
+### Host CPU / RAM (hardware-hcm)
+
+The `hardware-hcm-8x_*` launcher takes 8 CPUs + 64 GB RAM per GPU plus the engines' host KV pools (`--hicache-size`, `MOONCAKE_GLOBAL_SEGMENT_SIZE`), capped at 1950 GB. A YAML can override both with `host_cpus` / `host_mem_gb` (whole job: engines, router, sidecars, aiperf), e.g. 2× TP4 vLLM Gemma-4-31B at c96 peaks at ~70 GB: `host_cpus: 64`, `host_mem_gb: 192`. Slurm does not enforce `--mem` here, so asking for less than the peak oversubscribes the node.
+
 ### Replicas
 
 Several copies of one engine (e.g. 2× TP1): set `replicas: N` and `gpus` = N × per-engine GPUs; the router command lists the engines as `{{engine_urls}}` (space-separated URLs). Not with the Dynamo frontend.
@@ -115,7 +119,8 @@ The `scenarios/sweep-*.yaml` family (generated by [`utils/serving/gen_sweep_scen
 ## Read results and failures
 
 - Status: `gh run view <run-id> --repo vngcloud/InferenceX`. One job per `conc` value. Expect about 20 minutes of image pull, model load and warmup on top of the scenario duration.
-- Artifacts (`gh run download <run-id> --repo vngcloud/InferenceX -n <artifact>`) start with `<name>_c<conc>`; `results/` holds `serving.yaml`, `server.log`, `benchmark.log`, `router.log` (router path) and `aiperf_artifacts/`.
+- Artifacts (`gh run download <run-id> --repo vngcloud/InferenceX -D <dir>` for all, `-n <artifact>` for one) start with `<name>_c<conc>`; `agentic_<name>_c<conc>_…` holds `serving.yaml`, `dataset.txt`, `server.log`, `benchmark.log`, `router.log` (router path) and `aiperf_artifacts/` (with `replicas: N`, replica `i` > 0 also logs to `server.<i>.log`). The aggregate result is `bmk_agentic_…` / `results_bmk/agg_bmk.json`; its `tp` is the YAML's `gpus`, not the engine TP. `503` lines before warmup are readiness polling, not failures.
+- LiveKit scenarios (`custom-agentic-livekit111868-*`): `gh run download <run-id> --repo vngcloud/InferenceX -D <dir>`, then `uv run -q --no-project --python 3.12 --with pyyaml --with huggingface_hub python utils/serving/livekit_ttft.py <dir>` from the repo root prints TTFT per conc split by a room's first turn vs later turns (first turns set the p95 tail). It fetches the trace's plan CSV from the lock (`<dataset>-plan`); `--plan <csv>` uses a local copy. On `-v1-warm` (dial warm-up) runs the warm-up requests are left out of every TTFT view and get their own last line (latency, share done within 0.5/1/1.5/2 s).
 - On failure, read `server.log` first, then `benchmark.log`. For the router path, read `router.log` first when the job fails before aiperf starts. `gh run view <run-id> --repo vngcloud/InferenceX --log-failed | tail -100` shows the launcher side (Slurm, enroot).
 - An engine crash describes the config as given. Changing flags is the config owner's decision, not the benchmark's.
 

docs/serving-yaml-procedures_zh.md

@@ -61,6 +61,10 @@
    脚本执行与 CI 相同的校验。出现 `ERROR <field>: <fix>` 时不会派发任何任务：修改 YAML 后重新运行。成功时会输出运行 URL。使用 `--ref <branch>` 可派发当前分支以外的分支。
 4. 如果脚本提示运行未出现，不要重复派发；先执行 `gh run list --repo vngcloud/InferenceX -w e2e-tests.yml -L 5` 检查。
 
+### 主机 CPU / 内存（hardware-hcm）
+
+`hardware-hcm-8x_*` 启动器默认每块 GPU 申请 8 个 CPU + 64 GB 内存，再加上引擎的主机 KV 池（`--hicache-size`、`MOONCAKE_GLOBAL_SEGMENT_SIZE`），上限 1950 GB。YAML 可用 `host_cpus` / `host_mem_gb` 覆盖两者（整个作业：引擎、路由、sidecar、aiperf），例如 2× TP4 vLLM Gemma-4-31B c96 峰值约 70 GB：`host_cpus: 64`、`host_mem_gb: 192`。这里 Slurm 不强制 `--mem`，申请低于峰值会导致节点超分。
+
 ### 副本（replicas）
 
 「**副本（replicas）**」：同一引擎的多个副本（如 2×TP1）。设置 `replicas: N`，`gpus` = N × 每个引擎的 GPU 数；路由命令用 `{{engine_urls}}`（空格分隔的 URL）列出全部引擎。不支持 Dynamo frontend。
@@ -115,7 +119,8 @@ router:
 ## 查看结果与失败
 
 - 状态：`gh run view <run-id> --repo vngcloud/InferenceX`。每个 `conc` 值对应一个任务。除场景时长外，镜像拉取、模型加载与预热约需 20 分钟。
-- 产物（`gh run download <run-id> --repo vngcloud/InferenceX -n <artifact>`）以 `<name>_c<conc>` 开头；`results/` 包含 `serving.yaml`、`server.log`、`benchmark.log`、`router.log`（Router 路径）与 `aiperf_artifacts/`。
+- 产物（全部：`gh run download <run-id> --repo vngcloud/InferenceX -D <dir>`；单个：`-n <artifact>`）以 `<name>_c<conc>` 开头；`agentic_<name>_c<conc>_…` 包含 `serving.yaml`、`dataset.txt`、`server.log`、`benchmark.log`、`router.log`（Router 路径）与 `aiperf_artifacts/`（`replicas: N` 时，副本 `i` > 0 另写 `server.<i>.log`）。聚合结果为 `bmk_agentic_…` / `results_bmk/agg_bmk.json`；其中 `tp` 是 YAML 的 `gpus`，不是引擎 TP。warmup 之前的 `503` 行是就绪轮询，不是失败。
+- LiveKit 场景（`custom-agentic-livekit111868-*`）：`gh run download <run-id> --repo vngcloud/InferenceX -D <dir>`，然后在仓库根目录运行 `uv run -q --no-project --python 3.12 --with pyyaml --with huggingface_hub python utils/serving/livekit_ttft.py <dir>` 按 conc 输出 TTFT，并区分房间首轮与后续轮次（p95 尾部由首轮决定）。脚本从锁文件（`<dataset>-plan`）获取 trace 的 plan CSV；`--plan <csv>` 使用本地副本。`-v1-warm`（拨号预热）运行中，预热请求不计入任何 TTFT 视图，单独在最后一行输出（延迟，以及 0.5/1/1.5/2 s 内完成的比例）。
 - 失败时先看 `server.log`，再看 `benchmark.log`。Router 路径下，若任务在 aiperf 启动前失败，先看 `router.log`。`gh run view <run-id> --repo vngcloud/InferenceX --log-failed | tail -100` 显示启动脚本侧（Slurm、enroot）的日志。
 - 引擎崩溃反映的是原样提交的配置。是否修改参数由配置所有者决定，而不是由基准测试决定。
 

infx/matrix/validation.py

@@ -327,9 +327,7 @@ class SingleNodeAgenticMatrixEntry(BaseModel):
     kv_offload_backend: KVOffloadBackendMetadata | None = Field(
         default=None, alias=Fields.KV_OFFLOAD_BACKEND.value
     )
-    hicache_ratio: float | None = Field(
-        default=None, alias=Fields.HICACHE_RATIO.value, gt=0
-    )
+    hicache_ratio: float | None = Field(default=None, alias=Fields.HICACHE_RATIO.value, gt=0)
     router: ComponentMetadata | None = None
     total_cpu_dram_gb: int = Field(alias=Fields.TOTAL_CPU_DRAM_GB.value, ge=0)
     duration: int = Field(alias=Fields.DURATION.value)
@@ -376,9 +374,7 @@ class MultiNodeAgenticMatrixEntry(BaseModel):
     kv_offload_backend: KVOffloadBackendMetadata | None = Field(
         default=None, alias=Fields.KV_OFFLOAD_BACKEND.value
     )
-    hicache_ratio: float | None = Field(
-        default=None, alias=Fields.HICACHE_RATIO.value, gt=0
-    )
+    hicache_ratio: float | None = Field(default=None, alias=Fields.HICACHE_RATIO.value, gt=0)
     router: ComponentMetadata | None = None
     kv_p2p_transfer: str | None = Field(
         default=None, alias=Fields.KV_P2P_TRANSFER.value, min_length=1
@@ -650,9 +646,7 @@ class AgenticCodingSearchSpaceEntry(BaseModel):
     kv_offload_backend: KVOffloadBackendMetadata | None = Field(
         default=None, alias=Fields.KV_OFFLOAD_BACKEND.value
     )
-    hicache_ratio: float | None = Field(
-        default=None, alias=Fields.HICACHE_RATIO.value, gt=0
-    )
+    hicache_ratio: float | None = Field(default=None, alias=Fields.HICACHE_RATIO.value, gt=0)
     router: ComponentMetadata | None = None
     kv_p2p_transfer: str | None = Field(
         default=None, alias=Fields.KV_P2P_TRANSFER.value, min_length=1