fix(SERVE-CHAT-TEMPLATE): teach the vendored Jinja engine is undefined, and let a request's chat_template_kwargs reach the template (#1681)

.agents/issue-index.md

@@ -619,6 +619,7 @@ rather than merged. `scripts/check-agent-record.py` gates both.
 | [#1683](https://github.com/mudler/vllm.cpp/issues/1683) | `VT-CONV1D-TIME-BLOCK` | **The 11.48x scaling curve was measured on arm C, which does not ship, and the 4.11x ratio is composed across two jobs, one of which the row declares defective.** Found by the fresh review of [PR #1678](https://github.com/mudler/vllm.cpp/pull/1678) at head `e1396fc73`. `.agents/specs/vt-conv1d-time-block.md` §2b's scaling table carries the column header `arm C` and §2b defines arm C as `cf9296496`, "B + the conv decomposition, blocked UNCONDITIONALLY"; the shipped tree is arm D (`0f738d6ec`), which added the `out_channels * kernel <= in_len` condition in `06ba79d1b` and was measured at exactly ONE operating point -- 86 latents, 14 threads (§2c). The 1/2/4/8/14 sweep that produces 11.48x was never run on the shipped arm, and the number was nevertheless attributed to it in four places: the PR title, spec §9, `docs/STATUS.md` and `docs/BENCHMARKS.md`. `.agents/benchmark-record.md` got it right -- it prints the `arm C` header and names C as unconditional -- so the forensic record knew and the public projections dropped the label. The error is very likely CONSERVATIVE (at a 20-latent window the condition declines the two b0 shapes where C measured 0.82x/0.89x, so D >= C) but that is an inference, and the row's own standard is that an unmeasured quantity is reported as unmeasured. SEPARATELY, arm D's median 3.4989 s comes from job `214f5f70` (300 s settle, seven alternated rounds) while the 14.3895 s denominator comes from job `3ca07477`, of which the spec itself says the rounds "ran at `uptime` load 8.84, the decaying residue of three back-to-back builds in the same lease" and records that as "a defect in the SCHEDULE of the job rather than as a result" -- same boot id and worker, but a different job, not alternated against D and not under the settle §2c exists to provide, and whether arm A's 86-latent leg fell inside that window is not stated and cannot be recovered from the tree because no job log is committed. The 1.067x B-against-D figure beside it IS properly paired and is unaffected. FIXED IN FLOW as far as labelling goes -- all four public sites now name the curve as arm C's, say the shipped arm has one measured point, and name the denominator's contention. NOT fixed as a measurement: it needs one `thor:gpu0` lease, one boot id, both arms built inside it with distinct binary sha256, a settle with `uptime` on both sides, and 1/2/4/8/14 swept with the arms ALTERNATED at each count, which produces the shipped arm's own curve and a paired A-against-D ratio in one job. Owned by [`vt-conv1d-time-block.md`](specs/vt-conv1d-time-block.md) `## Owed` | verification |
 | [#1684](https://github.com/mudler/vllm.cpp/issues/1684) | `VT-CONV1D-TIME-BLOCK` | **No per-MODEL suite exercises a `blocks > 1` shape, so a defect confined to the time-blocked axis reds the op's suite and nothing else.** Measured by the fresh review of [PR #1678](https://github.com/mudler/vllm.cpp/pull/1678) rather than argued: mutation M7b sign-flips every output cell of `Conv1dKernel` when `blocks > 1` -- exactly and only the axis the row adds -- and at head `e1396fc73` EIGHT of the ten suites stayed GREEN (`test_vocoder1d`, `test_bigvgan`, `test_minimax_music3_acoustic`, `test_ltx2_vae`, `test_minimax_h3`, `test_indextts2_pipeline`, `test_indextts2_family`, `test_ops_conv1d_depthwise`), with only `test_ops_conv1d_general` and `test_host_parallel` red. All four audio consumers reach `vt::Conv1d`; every one of them does so at SINGLE-BLOCK shapes only. The row disclosed the gap in §6c and compensated with a geometry gate that asserted `blocks > 1` on the vocoder's shapes -- but that gate read SIX hard-coded shape constants transcribed by hand from `minimax_music3_loader.h:253-265`, so a loader change would have left it green while the production shapes collapsed to one block. PARTLY FIXED IN FLOW, both halves: the geometry gate now DERIVES its shapes by walking `MiniMaxMusic3VocoderConfig` and `kVocoderResidualDilations`/`kVocoderResidualUnits`, and the derivation is proved by mutation -- setting `upsampling_ratios{1,1,1,1}` reds the derived gate (1 case / 16 assertions, rc 1) and left the hand-transcribed one at 14 cases / 19 615 assertions / `SUCCESS!`, rc 0, completely blind; and `tests/vllm/models/test_vocoder1d.cpp` gained `vocoder1d Conv1d is exact ACROSS a time block boundary`, which enters through `vllm::vocoder1d::Conv1d` -- the body all four audio models call -- at 32 channels, kernel 7, 10 000 positions, asserts the block length is shorter than the output length so the case cannot silently become single-block, and reds under M7b at the repaired head (1 case / 4 assertions, rc 1). STILL OPEN: the four MODEL suites reach the provider at single-block shapes only, so M7b still leaves them green. Closing it means lengthening each consumer's reduced-dimension fixture until its convolutions cross a block boundary, which moves those fixtures' goldens -- a fixture change per model, not a test addition. Owned by [`vt-conv1d-time-block.md`](specs/vt-conv1d-time-block.md) `## Owed` | verification |
 | [#1685](https://github.com/mudler/vllm.cpp/issues/1685) | `SPEC-DFLASH2` | **The DFlash2 speed run's denominator declares `TRITON_ATTN` while five of its layers resolve `FlashAttentionBackend`, which `FA-CONSTRAINT.txt` says cannot target sm_12x.** Observed by the run that produced the first DFlash2 speed ratio (`dgx:gpu0`, `rc` job `ec9cf6cd-0aaf-4323-806d-6a12da2bd08f`, 2026-08-22, gate tree `d25730fbb`, `GATE_RC=0`, ours/vLLM = 0.8016987337853048 on `output_throughput_tok_s`). The gate did NOT refuse; this is a question about what the denominator IS, not a gate failure. `evidence/vllm-arm.json` records the scalar `attention_backend: TRITON_ATTN` from `attention_backend_source: read_back_from_engine`, and beside it the per-group census read off `...model_runner.attn_groups`: `GDNAttentionBackend` 48 layers, `TritonAttentionBackend` 16, and `FlashAttentionBackend` 5 -- the five being `model.layers.64-68.self_attn.attn`, i.e. the DFlash2 draft's sliding-window layers. `/workspace/oracle-dflash2/FA-CONSTRAINT.txt` records `FA_USABLE=0` for sm_12x from [#1456](https://github.com/mudler/vllm.cpp/issues/1456), where a real source build at `CUDA_ARCHS=12.0` emitted `sm_80` for `_vllm_fa2_C` and `sm_75` for `_vllm_fa3_C`, and `TRITON_ATTN` is the declared backend on this box BECAUSE of that. The engine nevertheless loaded, generated 64 tokens on every one of its 20 legs, and produced a clean 83-sample clock window. Three readings and none established: upstream intends the draft's sliding-window layers to select FA independently of the engine-wide choice; the label over-describes what executes and those layers fall back at runtime; or FA genuinely runs on sm_121 through driver JIT of the emitted `sm_80` PTX, in which case #1456's CONCLUSION -- not its measurement -- needs re-reading. This is the THIRD independent observation of FA resolving on this box against `FA_USABLE=0`. Settled by reading the pinned wheel's own selection path for sliding-window and draft layers and tracing whether those five layers dispatch FA kernels or fall back; either outcome is cheap and changes what the scalar should say. Recorded under `## Owed` O26 residual 1 of `.agents/specs/dflash2-spec-decode.md` and in the 2026-08-22 entry of `.agents/benchmark-record.md`. Related: [#1658](https://github.com/mudler/vllm.cpp/issues/1658) (why the per-group map is recorded beside the scalar at all: one string cannot describe 48 + 16 + 5) and [#1673](https://github.com/mudler/vllm.cpp/issues/1673) (the other open caveat on the same ratio). Evidence: `/mnt/nas_share/rc/dflash2-1673/out-n1673b/evidence/{vllm-arm.json,clock-vllm.json}` and `out-n1673b/m-gate.log`. | bug |
+| [#1681](https://github.com/mudler/vllm.cpp/issues/1681) | `SERVE-CHAT-TEMPLATE` | `POST /v1/chat/completions` answers HTTP 500 for the whole Qwen3.8 family because the vendored minja Jinja engine implements twelve of Jinja2's built-in tests and `undefined` is not one of them, so `{%- if enable_thinking is undefined or enable_thinking is true %}` throws at row 46 of the checkpoint's own template. `is true` was already present, so the first term was the only break. Fixed in flow together with the second half of the same defect: `enable_thinking` was set unconditionally by `apply_chat_template`, so even with `undefined` implemented the variable could never be undefined and the Qwen3.8 default would have been thinking-OFF against upstream's thinking-ON, and `ChatCompletionRequest` carried no `chat_template_kwargs` at all, so the `{"chat_template_kwargs":{"enable_thinking":false}}` body both competitor arms of [#1574](https://github.com/mudler/vllm.cpp/issues/1574) were measured with was silently ignored. Spec [`chat-template-jinja-undefined.md`](specs/chat-template-jinja-undefined.md), whose `## Owed` carries the twelve of Jinja2's thirty canonical built-in tests that stay unimplemented, each with the reason: nine need a grammar change because minja parses the right side of `is` as a bare identifier, `filter` and `test` need a name registry minja does not have, and `callable` can never be handed a callable because `BinaryOpExpr::do_evaluate` defers every binary operation whose left operand is one. No chat template of any checkpoint in `docs/USAGE.md` uses any of the twelve | bug |
 | [#1625](https://github.com/mudler/vllm.cpp/issues/1625) | `BACKEND-TENSTORRENT-HOST-FREE-FORWARD` | **Captured Tenstorrent decode hangs deterministically on the first MULTI-request run, while every single-request captured leg and the whole host-free eager path work.** Reproduced twice on the P150 at tree `b86e3705f` (main `52e328789` + the R5 flip): `VT_DUMP_IDS=1 test_qwen3_paged_engine` (16 sequential requests) stalls ~10 s into stepping with one tt-metal worker spinning at 100% and the main thread blocked, killed after 11 min; the last device log line is the allocator warning `Allocating device buffers is unsafe due to the existence of an active trace` (allocator.cpp:123). `VT_TT_RECAPTURE_EVERY=8` (live traces destroyed every 8 replays) hangs IDENTICALLY at the same point, ruling out a per-trace replay-count cap, and `VLLM_CPP_CUDAGRAPH=0` (host-free eager, no capture) completes the same gate in 35 s at 125/125 assertions and 10.94/10.95/11.06 tok/s warm. NOT FIXED IN FLOW: the mechanism is undiagnosed (eager-alloc-around-live-trace across the request boundary is the recorded hypothesis class, qwen3.cpp Step() comment, but the RECAPTURE_EVERY result narrows it), so #1604 lands with `support_static_graph_mode()` declined by default on TT (opt-in `VT_TT_DECODE_CAPTURE`); flipping capture back on by default is owned by this issue | bug |
 | [#1626](https://github.com/mudler/vllm.cpp/issues/1626) | `BACKEND-TENSTORRENT-HOST-FREE-FORWARD` | **`test_mistral_paged_engine.cpp:102`'s `RunGate` takes `const char* label` and streams it into doctest MESSAGE/REQUIRE_MESSAGE, so every label renders as `1`.** The #1604 R5 dump run printed `1 dumped our token ids -> ...` and the anchor-drift REQUIRE printed `logged: 1 anchor drift prompt[3] ...`. Same defect class #1508 fixed in the Qwen3 gate (doctest `MessageBuilder` has no `const char*` overload); the Mistral copy never got the `const std::string&` fix. FIXED IN FLOW by the same #1604 change (one-line parameter change, call site converts implicitly) | bug |
 | [#1627](https://github.com/mudler/vllm.cpp/issues/1627) | `BACKEND-TENSTORRENT-HOST-FREE-FORWARD` | **The Tenstorrent backend has no `SupportsAsyncSampledTokenReadback` override, so async scheduling resolves OFF on TT and `test_qwen3_dense_async_serving` FATALs on every cached checkpoint** (3 FATAL / 5 checkpoint-absent skip on the P150) at the anti-vacuous-pass guard `REQUIRE(loaded->async_scheduling_enabled())` (`test_qwen3_dense_async_serving.cpp:124`). Mechanism: `runner_supports_async()` derives from `vt::Backend::SupportsAsyncSampledTokenReadback()` (`runner.cpp:109-112`, default false at `backend.h:186`), overridden only by CPU and CUDA. PRE-EXISTING: zero hits under `src/vt/tenstorrent/` at base `52e328789`, and the R5 flip commits touch none of the resolution path — captured-vs-eager decode mode is orthogonal. NOT FIXED IN FLOW: enabling it needs a device-mirrored sampled-id design against the tt-metal allocator (CUDA's `async_device_mirror` equivalent) plus the #323-class guard re-proven on device — its own spec and gates; owned by this issue | bug |