common: force draft model to SPLIT_MODE_NONE when target uses SPLIT_MODE_TENSOR

[sunagent]

Summary

Two commits that together make DFlash/DSpark work under --split-mode tensor on multi-GPU setups. This is the complete fix chain (both are needed; plus upstream PR ggml-org#26636 for the shared-tensor device backends).

Commit 1: draft model forced to SPLIT_MODE_NONE

common_base_params_to_speculative() (common/speculative.cpp) - when the target uses tensor split, the draft context no longer inherits it. DFlash/DSpark drafters share tensors with the target (tok_embd, output.weight) which are split along axis 0 under tensor parallelism; per-row ops (GET_ROWS, TOP_K, candidate selection) in the draft graph cannot consume axis-0-split inputs:

GGML_ASSERT(src_ss[0].axis != GGML_BACKEND_SPLIT_AXIS_0) failed ggml-backend-meta.cpp:543

Commit 2: output.weight MIRRORED only for DFlash/DSpark

llama-model.cpp + llama.h + llama-model.h + common.cpp - the target's output.weight (~2.4 GB) stays unsplit (MIRRORED on every device) so the draft's per-row logits ops see the full vocabulary. The option is wired to the enabled speculative types: mirrored only for DFlash/DSpark, split as before for MTP / plain decoding (saving ~1.2 GB VRAM per GPU vs always-mirrored).

Why both

• Without commit 1: the draft scheduler inherits tensor split and the draft graph breaks on shared tensors.
• Without commit 2: output.weight is vocab-split, logits are partial, and the DFlash selector cannot pick global top-k.
• Plus upstream PR llama : add backends of the other model to the context ggml-org/llama.cpp#26636 (add backends of the other model to the context) is required for the shared tensors to be reachable on all devices - complementary, not replaced by these commits.

Tested

Hardware: 2x RTX 5060 Ti 16GB (sm_120, no NVLink), Ryzen 5 5600
Model: Qwen3.8-27B NVFP4 (tensor split) + incoai Qwen3.8-27B-DFlash2-Q4_K_M drafter

Metric	Result
Startup with tensor split	clean, no GGML_ASSERT
Continuous-load TG	51.6-58 t/s (n_max=2, 150k ctx, q4_0 KV)
Draft acceptance	86-90%
GPU utilization	GPU0 87-91% / GPU1 71-80%
MTP regression check	200k ctx at 14.78 GB/GPU - identical to upstream behavior, no regression

[youyoulyz]

I implemented an independent fix for the same crash — materializing draft-local copies of the shared tok_embd/output inside the draft model at load time instead of mirroring the target's output.weight — so I could cross-check this PR on a second rig. The implementation lives on https://github.com/youyoulyz/llama.cpp/tree/dflash2-tp-optimizations (commit `speculative: make DFlash/DSpark drafts compatible with -sm tensor`). Side-by-side results (2× RTX 3080 20GB, Qwen3.8-27B-Q4_K_M + DFlash2 draft, `-sm tensor`, code-gen prompt, 300 tok/req, temp 0, n_max=2):

config	t/s	acceptance	mean len
dflash (this PR, MIRRORED output)	85.2	0.84	2.7
dflash (local-copy fix, dflash2-tp-optimizations)	82.1	0.83	2.65
mtp (both setups)	86.5 / 86.6	0.80 / 0.77	—

The two approaches are performance-equivalent within noise, so consider this a confirmation of the approach rather than a competing claim. Three suggestions if you keep iterating on this PR:

1. NVFP4 scale tensors are not covered. `mirror_output_weight` only affects `output.weight`. On NVFP4 targets the drafter also borrows `output.scale` / `output.input_scale` via `ctx_other`, and those follow the same AXIS_1 split → the same "pre-allocated tensor in a buffer (Meta())" abort. Q4 targets don't hit it (no scale tensors), so it won't show in the current test matrix, but the fix will need it for NVFP4 models.

2. Draft device selection. `SPLIT_MODE_NONE` pins the drafter to `main_gpu` (default 0). Under `-sm tensor` the target split is balanced by free memory at load time, so GPU 0 isn't necessarily the one with headroom. The local-copy branch picks the device with the most free memory instead (`ggml_backend_dev_memory` loop), which also keeps the drafter off a card that ends up carrying most of the target.

3. VRAM trade-off (optional alternative). Mirroring `output.weight` costs ~1.2 GB per GPU (~2.4 GB total on 2 cards) even though only the drafter needs it unsplit — the target's own TP graph is fine with AXIS_1. The local-copy approach keeps the target fully split and adds only the draft-side copy (~0.4 GB for Q4, one-time gather at draft load via the meta buffer's `get_tensor`, which reconstructs the full tensor across devices). Trade-off is code size: ~40 lines vs ~200 lines; the local-copy version also adds no public API change (`llama.h` stays untouched).

One more data point from the same runs: draft-side acceptance is heavily workload-dependent — 0.83 on code-gen vs 0.62 on prose with the identical setup. That explains most of the spread in the full matrix between workloads, and it's worth noting in the PR description if you cite the acceptance numbers.

[dan-and]

I would rather like to see split mode layer than none. @sunagent would split model layer work on your setup?