From 1bef9a281d5225f5871e9e170a3f8346c38191bb Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Sat, 22 Aug 2026 21:52:52 +0000 Subject: [PATCH 1/3] record(SERVER-STALL-1737): the #931 hang shape reproduces off-GPU, and the advertised context is one block too large (#1737) The 2026-08-22 comment on #931 reported a single `/v1/completions` request that never returns on `dgx:gpu0` -- zero bytes, no status line, GPU idle -- and could not pursue it, because that box is bounded by #1647. #931 itself is closed and its cause (the SSE keepalive, #971) is on main, and the failing request is non-streaming, so it never reaches that frame. This is a second defect with the same client-visible signature, and it does not need the box. It reproduces on a CPU build of `vllm-server` against `facebook/opt-125m` in two flags. With `--num-blocks 4 --block-size 16`, one non-streaming request of 6 prompt plus 56 output tokens returns `curl: (28) ... with 0 bytes received` after 150 s while `/health` answers 200. Scraped during the stall, `/metrics` reads `num_requests_waiting 1`, `num_requests_running 0`, `kv_cache_usage_perc 0.0` and `num_preemptions_total 1`: the KV cache is empty and the request is still refused. `VT_SERVER_VERBOSE=1` emits 48,285 consecutive `core-step end model_executed=0` heartbeats, which is the CPU equivalent of the `0 %, 11.43 W` reading in that comment. The mechanism was measured with a scratch scheduler trace, since reverted byte-for-byte. `ResolveMaxModelLen` reconstructs `available` as `num_blocks * bytes_per_block` and advertises `max_model_len = num_blocks * block_size`, while only `(num_blocks - 1) * block_size` is allocatable, because block 0 is the reserved null block. A request inside the advertised context is therefore admitted, decodes until the pool is exhausted, preempts itself as the only request in flight, and is refused re-admission on every step thereafter. The boundary is exact: at `--num-blocks 2` a 17-token request returns 200 and an 18-token one hangs. Nothing is fixed here, and the reason is the in-flow rule's own criterion. The silent-wedge half is shared with vLLM, whose v1 scheduler breaks out of the same loop and leaves the request waiting, so removing it is a divergence that needs a decision rather than a patch. The off-by-one half is ours, but correcting it tightens an assertion `test_loaded_engine_dense.cpp` currently pins, so it needs its own row, spec and fresh review. It is not established that this arithmetic is what the #931 comment hit, and the records say so: that request was 22 tokens against a 27B checkpoint, nowhere near any advertised context. What transfers is the shape, and a discriminator that needs no new code -- read the `VT_SERVER_VERBOSE=1` heartbeat during the stall, because `model_executed=0` against a non-zero `unfinished` count puts the search on the KV pool rather than on `AsyncLLM`. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/issue-index.md | 1 + .agents/specs/server-concurrency-failures.md | 38 ++++++++++++++++++++ 2 files changed, 39 insertions(+) diff --git a/.agents/issue-index.md b/.agents/issue-index.md index 46dff4cd8..628a7593b 100644 --- a/.agents/issue-index.md +++ b/.agents/issue-index.md @@ -635,6 +635,7 @@ rather than merged. `scripts/check-agent-record.py` gates both. | [#1724](https://github.com/mudler/vllm.cpp/issues/1724) | `LTX25-DIT-ATTN-FA2-HD128` | **`LaunchDenseFA2Bf16`'s head-dim `else throw` is unreachable while the admissibility guard above it stands, so no test enters it and no mutation can turn it red.** `cuda_flash_attn_fa2.cu::LaunchDenseFA2Bf16` now ends its dispatch `if (d == 64) { ... } else if (d == 128) { ... } else { throw ... }`. The arm is RIGHT and the fresh review of [PR #1704](https://github.com/mudler/vllm.cpp/pull/1704) asked for it: the head dim is a TEMPLATE parameter, so the two compiled instantiations are two call sites and not one call with an argument, and the shape it replaced put the 128 call in a bare `else` on the reasoning that a head dim with no instantiation would be a LINK error. That reasoning is wrong -- the set the function serves is decided by the `d != 64 && d != 128` guard earlier in the same function, not by the linker -- so widening that guard alone, the exact first edit a head_dim-192 rung makes, links fine and sends 192 into the 128 kernel, which reads 128 of its 192 channels and returns a SILENTLY TRUNCATED answer. The two guards are ORDERED, which is what leaves the arm without coverage: no input reaches it, no test can enter it without first making the very edit it guards, and deleting the arm or inverting its condition leaves every case green. NOT a regression and NOT a capability that landed dead -- the same input previously produced a silently truncated answer with no diagnostic, so the arm strictly replaces silence with a named refusal. The owning suite `test_ops_attention_dense_fa2` is CUDA-gated in all 12 of its cases -- each returns early on `!HasCuda()` -- so doctest reports 12 cases and ZERO assertions on a CPU build, so it gives the arm no coverage on the authoring host either. OWES: the first commit that widens the `d != 64 && d != 128` guard -- for head_dim 192, for f32, or for any new rung -- owes a RED-FIRST case proving the throw fires for a head dim the widened guard admits and the launcher has no instantiation for, taken BEFORE the instantiation that makes the throw unreachable again is added; that widening is the only moment at which the guarantee is both reachable and provable. NOT fixed in flow, because the only change that makes it testable is the guard widening this issue conditions, which is a separate rung with its own row. Under `## Owed` in [`ltx25-dit-attn-fa2-hd128.md`](specs/ltx25-dit-attn-fa2-hd128.md) | bug | | [#1734](https://github.com/mudler/vllm.cpp/issues/1734) | `LTX25-DIT-ATTN-FA2-HD128` | **`scripts/ltx25-dit-attn-fa2-hd128-ab.sh` prints an EMPTY `memavail low-water:` for every arm, and the cause is the writer rather than the reducer that prints it.** Found by this row's own A/B on `dgx:gpu0` (rc job `91e0b5d9-b7f7-4b69-bf3f-d593aa25f871`, artifacts `/mnt/nas_share/rc/ltx25-fa2hd128/out/20260822T203535Z/`). At line 367, `n=$(grep -c 'last=' "$log" 2>/dev/null \|\| echo 0)` emits TWO lines when the count is zero, because `grep -c` prints `0` AND exits 1, so the `\|\| echo 0` arm fires as well. Line 369 then writes that value into a tab-separated record, which lands on disk split across two lines: `watch-flash.tsv` is 85 lines with `NF=3`, 85 with `NF=2` and 16 with `NF=4` (186 total), and `watch-fa2.tsv` reads 85 / 85 / 6. The 85 pairs are exactly the 5-second polls taken during the ~7 minutes of model load, before any `last=` line existed; once the log has a match `grep -c` exits 0 and the record is well formed, which is why a minority of lines are intact. TWO consequences. The reducer at line 390 reads `$4` over a file where `$4` is empty on 170 of 186 lines, and an empty string sorts first under `sort -n`, so `head -1` returns it -- that is the reported symptom, and the reducer is independently fragile because it strips the `memavail_gib=` prefix off `$4` alone and so depends on a field position it cannot rely on. The sample cap's own test `[ "${n:-0}" -ge "$WANT_SAMPLES" ]` also receives the two-line value for those 85 iterations; harmless here because the cap cannot fire before a sample exists, but a non-integer reaching an integer comparison inside the guard that stops a job on a shared box is not something to leave standing. TOUCHES NO NUMBER in the measurement: both arms report `stopped_by=sample-cap`, which is the direct evidence that neither was stopped by memory pressure, and re-derived with a prefix-stripping match instead of a positional one the low-water is **40.3 GiB on both arms** against `MEM_FLOOR_GIB=12.0`, so the run stayed 3.36x above its own floor. NOT fixed in the record commit that found it, because that commit is `.agents/`-only by scope and a `scripts/` edit owes a red-first case over a fixture `watch-*.tsv` asserting the record is one line and the low-water reduces to the expected value. Under `## Owed` in [`ltx25-dit-attn-fa2-hd128.md`](specs/ltx25-dit-attn-fa2-hd128.md) | bug | | [#1649](https://github.com/mudler/vllm.cpp/issues/1649) | `GATE-WINDOWS-PORTABILITY-TARGET-SCOPE` | **`check-windows-portability.py` read `/w` anywhere in `CMakeLists.txt` as a project-wide negation of `/W4 /WX`, so the vendored BoringSSL targets' PRIVATE `/w` red `windows-msvc-cpu` and `windows-msvc-vulkan` on main and on every PR.** `check()` set `warnings = cmake` -- the whole file -- concatenated `cmake/CompilerWarnings.cmake` and did a flat token search, which cannot tell a global `add_compile_options(/w)` from a `target_compile_options( PRIVATE /w)`; only the first negates the policy. The refusal lands before any translation unit is read, so the job carries no `error C####`. SECOND red, which #1649 did not record: the same defect fails this checker's own suite -- `test_real_tree_msvc_warning_policy_reaches_the_cxx_compile` with `negation='/w'` on main at `8540a2755` (78 tests, 1 failure) -- so the tree asserted the contradiction in two places at once and one fix clears both. FIXED IN FLOW: `without_foreign_target_compile_options` blanks `target_compile_options` spans whose target PROVABLY names only targets this project never declares (`add_library`/`add_executable` first arguments, with `foreach` bindings resolved); anything unresolved stays in scope, and `cmake/CompilerWarnings.cmake` is kept whole because it applies the policy through an unresolvable function parameter. The widening is bounded by three guard properties proved discriminating by MUTATION -- forcing `_target_is_foreign` to `return True` reds exactly those three and nothing else (82 tests, 3 failures), tree restored byte-for-byte. 82 tests OK; the real tree now prints `Windows portability contract OK`. RESIDUAL, stated not hidden: a negation reaching a project target through a `set()` binding rather than a `foreach()` one is still not caught; no such construct exists in the tree today. Recorded under [`gate-windows-portability-target-scope.md`](specs/gate-windows-portability-target-scope.md) `## 4. Risks / decisions` | bug | +| [#1737](https://github.com/mudler/vllm.cpp/issues/1737) | — | **A request the scheduler can never admit hangs forever with zero bytes, and our advertised `max_model_len` is one block larger than the servable KV.** Found chasing the 2026-08-22 comment on [#931](https://github.com/mudler/vllm.cpp/issues/931), whose own cause (the SSE keepalive) is fixed and closed — this is a SECOND defect with the same client-visible signature, and unlike that report it reproduces OFF-GPU. CPU build of `vllm-server`, `facebook/opt-125m`, `--num-blocks 4 --block-size 16`, one non-streaming `/v1/completions` of 6 prompt + 56 output tokens: `curl: (28) ... with 0 bytes received` after 150 s, no status line and no error body, while `/health` answers 200 and `/metrics` reads `num_requests_waiting 1`, `num_requests_running 0`, `kv_cache_usage_perc 0.0`, `num_preemptions_total 1`. `VT_SERVER_VERBOSE=1` shows 48,285 consecutive `core-step end model_executed=0` heartbeats, 48,284 of them carrying an admission refusal — the CPU equivalent of the `0 %, 11.43 W` GPU reading in the #931 comment. MECHANISM, measured with a scratch scheduler trace since reverted: `ResolveMaxModelLen` (`model_loader.cpp:1508`) reconstructs `available` as `num_blocks * bytes_per_block` and advertises `max_model_len = num_blocks * block_size`, while only `(num_blocks - 1) * block_size` is allocatable because block 0 is the reserved null block (`block_pool.cpp:53-57`, `:453`); a request inside the advertised context is admitted, decodes until the pool is exhausted, preempts ITSELF as the only request in flight, and is then refused re-admission on every subsequent step forever. Boundary is exact: at `--num-blocks 2` (advertised 32, servable 16) 17 total tokens return 200 and 18 hang. NOT FIXED HERE, and the reason is the in-flow rule's own criterion: the silent-wedge half is shared with vLLM (`sched/scheduler.py:934-938` at the pinned `555967922` breaks out of the same loop and leaves the request waiting), so changing it is a DIVERGENCE needing a decision; and the off-by-one half tightens an assertion `tests/vllm/entrypoints/test_loaded_engine_dense.cpp:635-660` currently pins, so it needs its own row, spec and fresh review. It is NOT established that this is what the #931 comment hit — that report used a 22-token request against a 27B checkpoint, nowhere near any advertised context — but the SHAPE transfers, and so does a one-command discriminator that needs no new code: `VT_SERVER_VERBOSE=1` and read the heartbeat during the stall. Owed by [`server-concurrency-failures.md`](specs/server-concurrency-failures.md) `## Owed` | bug | | [#961](https://github.com/mudler/vllm.cpp/issues/961) | `BACKEND-CUDA-SM110` | `tests/tools/test_online_gate_startup.py:259` guards `shellcheck` absence with a check that cannot fire, so an absent instrument reads as a code verdict: `test_serve_low_tools` raises `FileNotFoundError: 'shellcheck'` instead of skipping. Filed by the sm_110 baseline lane ([#955](https://github.com/mudler/vllm.cpp/issues/955)), where the leased `thor:gpu0` worker carries no `shellcheck`, so the baseline names the failure as a known entry rather than an sm_110 fact. Indexed late: the issue was opened 2026-08-15 and its index row was lost with the unmerged repair of PR [#956](https://github.com/mudler/vllm.cpp/pull/956) | bug | | [#962](https://github.com/mudler/vllm.cpp/issues/962) | `BACKEND-CUDA-SM110` | NVFP4 marlin disagrees with itself on sm_110: `test_ops_moe_grouped:1262` fails `CHECK(bitdiff == 0)` on `NVFP4 block8-vs-block16 M=8 K=4096 N=4096 bitdiff=15/32768`, against a kernel configure reports as `marlin-nvfp4: ENABLED for [110]`. A live kernel defect on an ENABLED feature, not an absent one, and the only substantive standing sm_110 finding in the [#955](https://github.com/mudler/vllm.cpp/issues/955) baseline. Indexed late, same cause as the row above | bug | | [#1363](https://github.com/mudler/vllm.cpp/issues/1363) | `BACKEND-CUDA-SM110` | Thor reports 30 GiB of swap, all free, measured inside `rc run -d thor:gpu0`, so the "zero swap" half of the box's `THIS BOX REBOOTS INSTEAD OF OOM-KILLING` warning is stale. `vm.overcommit_memory=1` is unchanged and the three 2026-08-11 reboots were observed, so the hazard stands and is not relaxed; unresolved are whether the swap is the host's or a container view, whether it changes the failure mode at all, and when it appeared | bug | diff --git a/.agents/specs/server-concurrency-failures.md b/.agents/specs/server-concurrency-failures.md index 9126071be..f17b3ccfd 100644 --- a/.agents/specs/server-concurrency-failures.md +++ b/.agents/specs/server-concurrency-failures.md @@ -441,3 +441,41 @@ loud failure, which is the protection #577 asked for and did not get. refuted as that cause: the pool was 36 against an offered concurrency of 8. It is a real mirror gap and needs its own change, because the next person raising `--max-num-seqs` for throughput silently raises the HTTP ceiling with it. +- [#1737](https://github.com/mudler/vllm.cpp/issues/1737) — **a request the + scheduler can never admit hangs forever with zero bytes, and our advertised + `max_model_len` is one block larger than the servable KV.** Found on + 2026-08-22 while chasing the sharper stall the 2026-08-22 comment on + [#931](https://github.com/mudler/vllm.cpp/issues/931) reported on `dgx:gpu0`, + which that reporter could not pursue because the box is bounded by + [#1647](https://github.com/mudler/vllm.cpp/issues/1647). **#931's own cause is + fixed and closed; this is a second defect with the same client-visible + signature**, and it reproduces OFF-GPU: a CPU build of `vllm-server` on + `facebook/opt-125m` with `--num-blocks 4 --block-size 16` and one + non-streaming `/v1/completions` request of 6 prompt + 56 output tokens + returns `curl: (28) ... with 0 bytes received` after 150 s, while `/health` + answers 200 and `/metrics` reads `num_requests_waiting 1`, + `num_requests_running 0`, `kv_cache_usage_perc 0.0` and + `num_preemptions_total 1`. `VT_SERVER_VERBOSE=1` shows 48,285 consecutive + `core-step end model_executed=0` heartbeats — the CPU equivalent of the + `0 %, 11.43 W` GPU reading in the #931 comment. **Mechanism, measured:** the + auto-fit advertises `max_model_len = num_blocks x block_size` because + `ResolveMaxModelLen` (`model_loader.cpp:1508`) reconstructs `available` from + the full block count, while only `(num_blocks - 1) x block_size` is + allocatable — block 0 is the reserved null block (`block_pool.cpp:53-57`, + `:453`). A request inside the advertised context is therefore admitted, + decodes until the pool is exhausted, preempts ITSELF (it is the only request + in flight), and can never be re-admitted. **Two halves, and they separate.** + The silent-wedge half is shared with vLLM, whose v1 scheduler breaks out of + the same loop and leaves the request waiting (`sched/scheduler.py:934-938` at + the pinned `555967922`), so changing it is a DIVERGENCE and needs a decision + rather than a patch. The off-by-one half is ours. Neither is fixed here: the + arithmetic change tightens an assertion + `tests/vllm/entrypoints/test_loaded_engine_dense.cpp:635-660` currently pins + (`params.num_blocks = 1` asserting `max_model_len == 32`), which the in-flow + rule excludes and which needs its own row, spec and fresh review. **It is NOT + established that this is what the #931 comment hit** — that report used a + 22-token request against a 27B checkpoint, nowhere near any advertised + context. What transfers is the SHAPE and a one-command discriminator that + needs no new code: restart that server with `VT_SERVER_VERBOSE=1` and read the + heartbeat during the stall, because `model_executed=0` against a non-zero + `unfinished` count puts the search on the KV pool rather than on `AsyncLLM`. From dac4a99d45b9c3de314d4840759eb3c9b79da271 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Sun, 23 Aug 2026 07:36:35 +0000 Subject: [PATCH 2/3] record(SERVER-STALL-1737): the off-by-one is upstream's too, so the proposed fix is a divergence (#1737) The record this commit corrects calls the advertised-context overhang "ours" and proposes correcting `ResolveMaxModelLen` to `(num_blocks - 1)`. Read at the exact parity pin `5559679229bc`, upstream carries the same arithmetic end to end: `kv_cache_utils.py:2144-2158` reconstructs `available_memory` from the FULL block count under `num_gpu_blocks_override`, `kv_cache_coordinator.py:90-95` builds `BlockPool(num_gpu_blocks=kv_cache_config.num_blocks)` with no `+ 1`, and `block_pool.py:188-191` pops block 0 as the null block. Upstream's allocatable capacity is `num_blocks - 1` against an advertised `max_model_len` that assumes `num_blocks`, which is the overhang the record attributes to us. The self-preemption half matches at `sched/scheduler.py:607-609`. Both halves are mirrored behaviour, so subtracting the null block is a decision about diverging from the reference rather than an implementation task, and the next reader must not pick it up as one. The hang itself reproduces independently off-GPU, on a second machine and a second build: opt-125m on a CPU build, `--num-blocks 4 --block-size 16`, one non-streaming `/v1/completions` of 6 prompt + 56 output tokens returns `curl: (28) ... with 0 bytes received` at `http_status=000 size=0` while `/health` answers 200, against a control at `--num-blocks 64` that returns 200 in 13.2 s. One hypothesis is recorded as falsified rather than dropped. Upstream's serving layer hard clamps `max_tokens` to `max_model_len - input_length` (`api_utils.py:170-206` feeding `completion/protocol.py:358` unconditionally) and we never do, so an over-long `max_tokens` was predicted to wedge the same way. It does not: measured at `--max-model-len 64` with `max_tokens 5000`, HTTP 200 with `finish_reason="length"`. Our engine stops at `max_model_len` by another route, so the mechanism differs while the client-visible result matches. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/server-concurrency-failures.md | 53 +++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-) diff --git a/.agents/specs/server-concurrency-failures.md b/.agents/specs/server-concurrency-failures.md index f17b3ccfd..44f960c8a 100644 --- a/.agents/specs/server-concurrency-failures.md +++ b/.agents/specs/server-concurrency-failures.md @@ -468,7 +468,8 @@ loud failure, which is the protection #577 asked for and did not get. The silent-wedge half is shared with vLLM, whose v1 scheduler breaks out of the same loop and leaves the request waiting (`sched/scheduler.py:934-938` at the pinned `555967922`), so changing it is a DIVERGENCE and needs a decision - rather than a patch. The off-by-one half is ours. Neither is fixed here: the + rather than a patch. **The off-by-one half was recorded here as "ours", and + that is WITHDRAWN — see the correction below.** Neither is fixed here: the arithmetic change tightens an assertion `tests/vllm/entrypoints/test_loaded_engine_dense.cpp:635-660` currently pins (`params.num_blocks = 1` asserting `max_model_len == 32`), which the in-flow @@ -479,3 +480,53 @@ loud failure, which is the protection #577 asked for and did not get. needs no new code: restart that server with `VT_SERVER_VERBOSE=1` and read the heartbeat during the stall, because `model_executed=0` against a non-zero `unfinished` count puts the search on the KV pool rather than on `AsyncLLM`. +- **CORRECTION to the entry above, 2026-08-23: the off-by-one is NOT ours, and + the fix it proposes is a DIVERGENCE from vLLM.** The entry above calls the + advertised-context overhang "ours" and proposes correcting `ResolveMaxModelLen` + to `(num_blocks - 1)`. Read at the exact parity pin `5559679229bc` in a local + checkout of the oracle, upstream carries the identical arithmetic: + `kv_cache_utils.py:2144-2158` reconstructs `available_memory = override x + bytes_per_block` from the FULL block count whenever `num_gpu_blocks_override` + is set, which is byte-for-byte what `model_loader.cpp:1505-1507` does; + `kv_cache_coordinator.py:90-95` then builds `BlockPool(num_gpu_blocks= + kv_cache_config.num_blocks, ...)` with NO `+ 1`; and `block_pool.py:188-191` + pops block 0 out of the free queue as the null block exactly as + `block_pool.cpp:53-57` does. Upstream's allocatable capacity is therefore + `num_blocks - 1` while its advertised `max_model_len` assumes `num_blocks`, + which is the same one-block overhang. The self-preemption half matches too: + `sched/scheduler.py:607-609` preempts the sole request, breaks, and leaves it + waiting. **BOTH halves are mirrored upstream behaviour**, so subtracting the + null block is a product decision about diverging from the reference, not an + in-flow bug fix, and it must not be taken as an implementation task. The + `## Owed` entry above is left in place rather than rewritten, because the + shape of the error — reading our own code against a hang and concluding the + arithmetic was ours without reading the oracle's — is the lesson. + +- **The hang REPRODUCES independently off-GPU, and one hypothesis was falsified + by measurement rather than dropped.** Fresh CPU build (`-DVLLM_CPP_CUDA=OFF`, + RelWithDebInfo, x86_64) on `facebook/opt-125m`, three legs, same binary: + a control at `--num-blocks 64 --block-size 16` with `max_tokens 16` returns + **HTTP 200** in 13.2 s with `finish_reason="length"`; the wedge at + `--num-blocks 4 --block-size 16` with `max_tokens 56` returns + `curl: (28) Operation timed out after 90002 milliseconds with 0 bytes + received`, `http_status=000 size=0`, while `/health` answers 200 throughout + and the server logs only its 11 startup lines plus + `INFO auto-fit max_model_len: reduced from 2048 to 64 to fit the KV cache + (4 blocks x 16 tokens)`. That confirms the entry above on a second machine and + a second build. **The falsified hypothesis:** upstream's serving layer hard + clamps `max_tokens` to `max_model_len - input_length` — `get_max_tokens` + (`entrypoints/serve/utils/api_utils.py:170-206`) returns the `min` over the + model headroom, the request value and the operator ceiling, and + `completion/serving.py:163-181` feeds it to `to_sampling_params`, which at + `completion/protocol.py:358` assigns it UNCONDITIONALLY rather than as a + fallback — while `serving_completion.cpp:241` and `serving_chat.cpp:767` call + `to_sampling_params()` with no argument at all and our `InputProcessor` fills + `max_tokens` only when it is unset (`input_processor.cpp:268-271`, `:304`, + `:340`). The predicted consequence was that an over-long `max_tokens` would + wedge the same way. **It does not.** Measured at `--num-blocks 8 --block-size + 16 --max-model-len 64` with `max_tokens 5000`: **HTTP 200** in 62.8 s with + `finish_reason="length"` and 58 generated tokens. Our engine stops at + `max_model_len` by another route, so the mechanism differs from upstream while + the client-visible result matches, and there is no defect here. The adjacent + operator-ceiling arm of that same upstream function is separately open as + [#544](https://github.com/mudler/vllm.cpp/issues/544). From f24dbeae2b054a2303d84d883ddb648b3a922603 Mon Sep 17 00:00:00 2001 From: Ettore Di Giacinto Date: Sun, 23 Aug 2026 08:26:03 +0000 Subject: [PATCH 3/3] record(SERVER-STALL-1737): the wedged server also ignores SIGTERM, so a supervisor restart hangs with it (#1737) Observed while reaping the reproduction's processes, which is the only reason it was seen at all: the wedged server had been sent SIGTERM by the harness after the client's 90 s timeout, had logged `server: shutting down on signal 15`, and was still alive 53 minutes later with 23 threads and ~5.2% average CPU. It needed SIGKILL. The two healthy legs of the same harness exited on the identical signal, so this is a one-variable contrast rather than a property of the shutdown path in general. The shutdown path was not traced and n is 1 on each arm; what is established is that the signal was received and acknowledged and the process still never terminated. It is recorded because it raises the cost of the wedge above a hung client: a supervisor that restarts on SIGTERM hangs with it. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --- .agents/specs/server-concurrency-failures.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/.agents/specs/server-concurrency-failures.md b/.agents/specs/server-concurrency-failures.md index 44f960c8a..7012ca8d3 100644 --- a/.agents/specs/server-concurrency-failures.md +++ b/.agents/specs/server-concurrency-failures.md @@ -530,3 +530,18 @@ loud failure, which is the protection #577 asked for and did not get. the client-visible result matches, and there is no defect here. The adjacent operator-ceiling arm of that same upstream function is separately open as [#544](https://github.com/mudler/vllm.cpp/issues/544). + +- **The wedge also survives SIGTERM, which the record above does not mention.** + Observed 2026-08-23 on the same CPU reproduction. After the client's 90 s + timeout the harness sent SIGTERM to the wedged server; it logged + `server: shutting down on signal 15` and then **did not exit**. Measured 53 + minutes later: still alive, `State: S`, 23 threads, ~5.2% average CPU since + launch, and it needed SIGKILL. The two healthy legs of the same harness — the + `--num-blocks 64` control and the `--max-model-len 64` / `max_tokens 5000` leg + — both exited on the identical SIGTERM, so the harness returned from its + `wait` promptly and left no process behind. That is a one-variable contrast + with n=1 on each arm, not a controlled experiment, and the shutdown path was + not traced; what is established is that the signal was received and + acknowledged and the process still never terminated. It raises the + operational cost of the wedge above a hung client: a supervisor that restarts + on SIGTERM will hang too, and only a SIGKILL escalation clears it.