Support benchmarking an existing/remote inference endpoint (BYO endpoint) #26

Summary

Investigated whether InferenceX could benchmark an existing/remote inference endpoint instead of always launching the server on the GPU runner itself (context: analysis of run 29940925355, the GLM-5.2 W4AFP8 H200 DRAM-KV-offload/HiCache sweep). Today the whole pipeline assumes "launch server, then benchmark it, on the same box" — the client always talks to http://0.0.0.0:$PORT.

Current blocker

benchmarks/benchmark_lib.sh hardcodes http://0.0.0.0:$port in three places: run_benchmark_serving, run_lm_eval, and (the one that actually matters for the agentic path the analyzed run used) build_replay_cmd's --url.

Status: implemented on feat/remote-bench-endpoint, not yet merged

• 1. Shared library change — --base-url/REMOTE_BASE_URL override added to run_benchmark_serving, run_lm_eval, and build_replay_cmd in benchmarks/benchmark_lib.sh. Defaults to today's behavior when unset.
• 2. The -remote-bench recipe — benchmarks/single_node/agentic/glm5.2_fp4_sglang-remote-bench.sh. Required pre-flight check on REMOTE_BASE_URL/REMOTE_GPU_TELEMETRY_URL/REMOTE_ENGINE_METRICS_URL, optional REMOTE_RESET_URL cache-reset hook, self-reported RUNNER_TYPE override, same artifact schema as local recipes. One file per model+precision+framework (not per hardware, since remote-bench never launches a local server — hw-specific tuning doesn't apply).
• 3. Runner / config registration — cluster:remote-bench label registered on a real controller box (bench-client_01, no GPU), added to configs/runners.yaml (no hardware: entry needed), new runners/launch_bench-client.sh dispatcher (no docker/GPU mounts, unlike every other launcher — also has to point RESULT_DIR/INFMAX_CONTAINER_WORKSPACE at the real host paths since there's no bind mount). Wired up via new optional inputs on benchmark-tmpl.yml (remote-base-url etc., all empty-default/backward-compatible) plus a new dedicated workflow_dispatch entrypoint (remote-bench.yml) — production sweeps dispatch via perf-changelog.yaml/validation.py's closed schema, which we left untouched rather than extending it for per-run endpoint values.

Prior art

No existing issue or PR covers this. Closest adjacent asks upstream: #1045 (Kubernetes-native runner for distributed benchmarking, llm-d) and #1507 (engine startup time polluting GPU metric capture) — both still assume we control the server, just orchestrated differently.

Next steps

Smoke-test a live dispatch against a real remote endpoint, then open the PR.