feat!: upgrade llama.cpp from b10731 to b10797

CHANGELOG.md

@@ -58,6 +58,38 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by
   where the backend cannot provide it, `OFF` disables it.
 
 ### Changed
+- **llama.cpp `b10731` → `b10797`.** No project-source change: every header move in the range is
+  additive or a **widening** const-qualification, and the server wire contract is byte-identical
+  (request-field set, `set_hard_limits` bounds and response keys all verified mechanically, which is
+  the check that catches the contract-behind-a-stable-signature breaks a header diff cannot see).
+  All 8 local patches still apply.
+
+  Two upstream changes are worth knowing even though nothing broke:
+
+  1. **`preserve_reasoning` now defaults to enabled** (llama.cpp #28174). `common_params_parse_ex`
+     sets it when the caller did not, where it previously followed the chat template's own default.
+     On a template advertising `supports_preserve_reasoning`, the full history now carries reasoning
+     traces instead of only the last assistant message — better continuity, **more prompt tokens**.
+     It applies to every entry point that parses argv, so `NativeServer` in both modes and
+     `LlamaModel`'s own parameter parse. Pass `--no-reasoning-preserve` to restore the old behaviour.
+  2. **`data:` URLs are now accepted for `input_audio` and `input_video`**, not only images
+     (llama.cpp #27735) — so an OpenAI-style request may inline base64 audio/video the same way it
+     could already inline an image.
+
+  Also picked up: a fix for **Qwen3-TTS-0.6b** (llama.cpp #28231), where an F16 `ffn_down` overflowed
+  on intermediate peaks past the 65504 ceiling and turned the residual into NaN. That is on the
+  `TextToSpeech` path this project ships.
+
+  The final `b10792` → `b10797` step touches **nothing** the project links against — not one file
+  under `common/`, `include/`, `tools/server/`, `tools/mtmd/` or `ggml/include/`. It carries a
+  **big-endian correctness fix** that does matter to a platform this project ships in the default
+  JAR: `ggml-cpu`'s s390x `q5_1` path used an uninitialized `v_acc` accumulator (llama.cpp #28332),
+  so `Q5_1`-quantised inference on IBM Z could return garbage. There is also an upstream build
+  change (#28278) that moves `LLAMA_VERSION`/`LLAMA_COMMIT` from compile definitions into a
+  generated `llama-version.h`; it does not reach `getLlamaCppBuildInfo()`, which reads
+  `llama_build_info()` from `common/build-info.cpp` instead — verified by the smoke test that
+  cross-checks the pin against the linked binary.
+
 - **llama.cpp `b10682` → `b10731`.** One project-source change came out of it, and it is the kind a
   header diff does not surface: upstream renamed `--tensor-read-lazy` to `-lzm` / `--lazy-mode`
   (env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`) **with no alias**. The binding emitted

CLAUDE.md

@@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI.
 
-Current llama.cpp pinned version: **b10731**
+Current llama.cpp pinned version: **b10797**
 
 ## Upgrading CUDA Version
 
@@ -490,7 +490,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi
 ships no UI):
 ```bash
 # needs node/npm + network; embed.cpp is plain C++17 (no npm)
-git clone --depth 1 --branch b10731 https://github.com/ggml-org/llama.cpp /tmp/lc
+git clone --depth 1 --branch b10797 https://github.com/ggml-org/llama.cpp /tmp/lc
 ( cd /tmp/lc/tools/ui && npm ci && npm run build \
   && ( cd dist && find . -type f -not -path './_gzip/*' \
        | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \
@@ -530,7 +530,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend:
 - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored
   as the repo secret **`DEPOT_TOKEN`**.
 
-Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10731`), the
+Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10797`), the
 ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the
 *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's
 per-branch `actions/cache`), so every branch builds incrementally; a `b<nnnn>` version bump
@@ -1451,7 +1451,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson"
 
 #### Upstream source location (in CMake build tree)
 
-llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10731`.
+llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10797`.
 
 **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely
 by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the

README.md

@@ -11,7 +11,7 @@
 **Build:**  
 ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational)  
 ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey)  
-[![llama.cpp b10731](https://img.shields.io/badge/llama.cpp-%23b10731-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10731)  
+[![llama.cpp b10797](https://img.shields.io/badge/llama.cpp-%23b10797-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10797)  
 [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/)  
 ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162)  
 [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev)  

docs/history/llama-cpp-breaking-changes.md

@@ -686,3 +686,7 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r
 | b10679–b10682 | `ggml/src/ggml-metal/ggml-metal-tuning.cpp` (**#27932: flash-attention vec tunings for M1 Max**), `ggml/src/ggml-vulkan/ggml-vulkan.cpp` + three `vulkan-shaders/*.comp` (**#27925: `mul_mat_id` pads K rather than N**), `scripts/snapdragon/{build,sdk,setup-sdk}.py` + `docs/backend/snapdragon/windows.md` (**#27903: Windows SDK setup**), `tests/test-backend-ops.cpp` | **No project-source change, and the range cannot require one.** Ten files, 575 insertions / 59 deletions, 53 KB of diff — well under the runbook's 100 KiB chunking threshold, so this was taken as a single step. **Not one changed file is on the priority review list**: the delta is confined to `ggml/src/` backend implementations (Metal tuning tables, Vulkan matmul shaders), the Snapdragon build scripts (never compiled here), documentation, and an upstream test (`LLAMA_BUILD_TESTS` is OFF for a FetchContent subproject). Nothing under `common/`, `include/`, `tools/server/`, `tools/mtmd/` or `ggml/include/` moved, so the request-field set, its bounds, the emitted response keys, the `getMetrics()` class of silent contract break and the whole `mtmd_helper::gen_audio` surface are all provably out of scope rather than merely checked. The two ggml changes are backend-internal: neither `ggml_type` nor any `ggml-backend.h` type is touched, so the Metal and Vulkan classifier artifacts pick the work up by rebuilding, with no wiring change. |
 | b10679–b10682 | patches + upstream verification | **Zero intersection with the patch set, established mechanically rather than by re-running the applier and hoping.** The 42 files the eight patches touch were intersected against the range's changed-file list: empty. No patch context can therefore have moved, and `0001` stays at its 37-file form. Confirmed by a real build: fresh `cmake -B build-b10682 -DBUILD_TESTING=ON` through the FetchContent `PATCH_COMMAND`, which is fail-loud and pins its stamp to the checked-out commit — a fresh build directory is mandatory for exactly that reason. Configure reported `ggml commit: 5ea1b124e`, Release build clean, **`ctest` 520/520**, Java **1476 run / 0 failures / 17 model-gated skips**, `NativeLibraryLoadSmokeTest` **4/4 with 0 skipped** — including the pin cross-check against the freshly linked `libjllama.so`. `mvn spotless:apply` produced no changes. **One diagnostic defect surfaced during this verification and was fixed.** The first (incremental) run of that cross-check failed with *"Linked build-info `b10682-5ea1b124e` must start with the pinned tag `b10679-`"* even though every pin site already read b10682. Cause: `LLAMA_CPP_VERSION` is a `static final String`, i.e. a **compile-time constant that javac inlines into every referencing class** — including the test itself. An incremental build recompiles the constant but not the test class, whose own source did not change, so the stale literal survived in its constant pool (verified with `strings` on the `.class`: the test bytecode was from the previous build and still carried `b10679-`). A `clean` build is green. The assertion message named only the drift cause, i.e. the one that did **not** apply, so it now names both and tells the reader which is which. CI cannot hit this — it always builds from a clean checkout. |
 | b10682–b10731 | `common/arg.cpp` (**#the `--tensor-read-lazy` option renamed to `-lzm` / `--lazy-mode`, env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`, with no alias for the old spelling**), `common/speculative.cpp` (DFlash draft path refactored from a `features_buf` scratch vector to writing straight into `batch_inject.embd`; `llama_n_batch` → `llama_n_ubatch`), `ggml/include/ggml.h` + `ggml-backend.h` (**purely additive**: `ggml_swiglu_clamp`, `GGML_GLU_OP_SWIGLU_CLAMP`, `ggml_backend_op_alloc_size_may_expand`), `ggml/src/**` (6301 lines of backend kernels), `vendor/cpp-httplib` (1275 lines) | **One project-source change, and it is the kind a header diff alone does not surface.** `ModelParameters.setTensorReadLazy` emitted `putEnum("--tensor-read-lazy", mode)`; that option no longer exists at b10731, so every model load with the knob set would have died on an unknown argument — a *contract* change behind an unchanged signature, exactly the second failure class this file's own priority-list preamble warns about. The emitted flag moved to `--lazy-mode` (three assertions in `ModelParametersTest` with it); the **Java** names (`setTensorReadLazy`, `TensorReadLazyMode`) were deliberately left alone — they are this binding's API, and renaming them would push an upstream CLI spelling onto every consumer (srcmorph carries the knob as a config field, a mojo `@Parameter`, a README row and a sweep case). Everything else was ruled out mechanically rather than by reading: `common/speculative.h` is **byte-unchanged** (only the `.cpp` moved, and we call none of it directly), the two ggml headers have **zero deletions**, and the 42 files the eight patches touch were intersected against the range's changed-file list — the sole hit is `common/arg.cpp`, whose change sits at line ~2729 while `0001`'s hunks there are at 1201/1242, so no patch context moved. Confirmed by the real applier: fresh `cmake -B build-b10731` through the fail-loud FetchContent `PATCH_COMMAND`, configure clean in 47 s, `ggml commit: 0eadefebd`, stamp written for `0eadefebd3f8f92a86d634a0e5b8fffc9dc792c0` over all eight patches, 43 files patched in the tree. |
+| b10731–b10792 | `common/common.h` (**additive**: `common_params::preserve_reasoning_specified`; one whitespace realignment on `common_grammar grammar`), `common/arg.cpp` (**behaviour change, not a signature change** — see below), `common/log.h` (**additive**: `common_log_get_verbosity(ggml_log_level)`), `ggml/include/ggml.h` (**additive**: `ggml_flash_attn_ext_set_n_kv_max`), `tools/mtmd/mtmd.h` (**additive**: `mtmd_input_part` + `mtmd_tokenize_from_parts()`, #28250; `mtmd_tokenize`'s `bitmaps` parameter re-typed `const mtmd_bitmap **` → `const mtmd_bitmap * const *`, and `mtmd_tokenize`/`mtmd_bitmap_init_lazy` take a `const mtmd_context *`), `tools/mtmd/mtmd-helper.h` (**const-widening only**, #28307/#28310: `mtmd_helper_support_video`, `mtmd_helper_bitmap_init_from_file`, `mtmd_helper_bitmap_init_from_buf`, `mtmd_helper_video_init`, `mtmd_helper_video_init_from_buf`, `mtmd_helper_model_can_chat`), `tools/mtmd/CMakeLists.txt` (+`models/deepseek4v.cpp`, #28133/#28154), `tools/server/server-common.cpp` + `server-context.cpp` (both internal), `tools/mtmd/clip.cpp` (#28231) | **No project-source change.** Every header move is additive or a **widening** const-qualification, so existing call sites still compile: the project's only direct mtmd calls are `mtmd_helper_bitmap_init_from_file` + `mtmd_helper_init_opt_default` (`tts_engine.cpp`) and upstream's own `tokenize_input_prompts` (`jllama.cpp`), none of which changed shape. It never calls `mtmd_tokenize` itself, and the re-typed parameter would have been a legal qualification conversion anyway (`const T **` → `const T * const *` adds const at one level with nothing above it). `mtmd_helper::gen_audio` — the whole TTS surface — is **untouched** in the range. The **server contract was verified mechanically**, not just by header diff (the failure class that produced the `getMetrics()` and `repeat_last_n` breaks): `server-schema.cpp`'s request-field set, its `set_hard_limits` bounds, and `server-task.cpp`'s response keys in **both** emit forms are byte-identical between the two tags — zero added, zero removed, zero re-bounded. Two upstream changes are worth knowing even though nothing broke: **(1) `preserve_reasoning` now defaults to enabled** (#28174) — `common_params_parse_ex` writes `default_template_kwargs["preserve_reasoning"] = "true"` when the caller did not set it, where it previously followed the chat template's own default, so on a template advertising `supports_preserve_reasoning` the full history now carries reasoning traces and uses more tokens; `--no-reasoning-preserve` restores the old behaviour, and the new `preserve_reasoning_specified` flag is how upstream distinguishes "unset" from "explicitly false". This reaches every entry point that parses argv, i.e. `NativeServer` in both modes and `LlamaModel`'s own parameter parse. **(2)** `handle_media` now accepts `data:` URLs for `input_audio`/`input_video`, not only `input_image` (#27735), and raises `std::invalid_argument` rather than `std::runtime_error` — both inside a `static` function, so no surface the project links against moved. |
+| b10731–b10792 | patches + upstream verification | **All 8 patches still apply, and 3 of their 42 target files were in the range** — `common/arg.cpp` and `tools/mtmd/mtmd-cli.cpp` (both `0001`) and `tools/server/server-context.cpp` (`0002`/`0003`/`0010`) — so this was not a no-op intersection and had to be proven by running the applier, not by reasoning about it. Fresh `rm -rf build && cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path: configure clean, stamp written at `head c5a5535e6ebc2e74ab0e3f1b249d7d989aba26a2`, all eight hashes recorded. `0010` was additionally re-checked **by hand** against the rule in `CLAUDE.md` (the fail-loud applier can detect "does not apply" but never "upstream already fixed this, drop me"): `get_res_model_info()` in b10792 still emits `{"vocab_type", meta.model_vocab_type}` with no cast, so the patch stays. `0001` likewise: `common_params_parse_main` still appears nowhere in `b10792:common/arg.h`, and the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override is still in `arg.cpp` — upstream has not adopted [#26416](https://github.com/ggml-org/llama.cpp/issues/26416). **Chunking note:** the runbook's 100 KiB-per-step rule was **not** applied to this range, deliberately. The full `b10731...b10792` diff is 4.19 MB, but `docs/ops/Hexagon.csv` alone is 19 792 generated lines and the GPU backends (`ggml-opencl`, `ggml-hexagon`, `ggml-metal`, `ggml-cuda`, `ggml-sycl`) account for most of the rest — both outside the review surface the rule exists to bound (`ggml/src/**` is on the "safe to skip" list). Across `common/`, `include/`, `tools/server/`, `tools/mtmd/` and `ggml/include/` the range is **35 files, +758 / −175**, reviewable in one pass; chunking it would have produced ~44 commits for one bump without adding a single line of review. The size figures are recorded here so the deviation is auditable rather than silent. |
+| b10792–b10797 | `ggml/src/ggml-cpu/arch/s390/quants.c` (**#28332: `q5_1` used an uninitialized `v_acc`**), `ggml/src/ggml-sycl/**` (#27610 rms_norm+mul+add / add+add fusion, #26863 `GGML_SYCL_ENABLE_MKL_FA` made a global), `src/llama-hparams.{h,cpp}` + `src/llama-model.cpp` + `src/llama-model-loader.cpp` (**#28323: new internal `n_expert_used_max()`**), `src/CMakeLists.txt` + new `src/llama-version.h.in` (**#28278: stop rebuilding the whole source tree on every commit**), `src/llama.cpp`, `tests/test-backend-ops.cpp`, `docs/backend/SYCL.md` | **No project-source change, and this time the review surface is literally empty**: the range touches **zero** files under `common/`, `include/`, `tools/server/`, `tools/mtmd/`, `ggml/include/` or the top-level `CMakeLists.txt`. Server contract re-checked anyway and byte-identical (request fields, response keys). Two entries are still worth recording. **(1)** #28332 is a **big-endian correctness fix on a platform this project ships in the default JAR**: `ggml_vec_dot_q5_1_q8_1`'s s390x path accumulated into an uninitialized `v_acc`, so `Q5_1` inference on IBM Z could return garbage. The `build-linux-s390x` job cross-builds and runs the C++ suite under `qemu-user`, but that suite covers *our* endian-sensitive layer (`tts_wav.hpp`, the JSON/token transforms), not ggml's kernels — so this is precisely the class of bug the job cannot see and upstream has to fix. **(2)** #28278 moves `LLAMA_VERSION`/`LLAMA_COMMIT` out of `target_compile_definitions(llama PRIVATE ...)` into a `configure_file`-generated `src/llama-version.h`. That is the one change in the range that could plausibly have reached the project, since `NativeLibraryLoadSmokeTest.nativeBuildInfoMatchesPinnedVersionConstant` asserts the pin against the linked binary — but `jllama.cpp` reads `llama_build_info()` from `common/build-info.cpp`, a different mechanism, and the test passes at b10797. Verified rather than reasoned about. |
+| b10792–b10797 | patches + upstream verification | **Zero intersection with the patch set, established mechanically.** The 42 files the eight patches touch were intersected against the range's 18 changed files: empty, so no patch context can have moved. Confirmed by running the applier for real anyway: fresh `rm -rf build && cmake -B build -DBUILD_TESTING=ON`, configure clean, stamp at `head 832fd6f1741ad9f66fb2a785002396700666595a`, all eight hashes recorded. **This step needed no chunking decision**: the full `b10792...b10797` diff is 46 KB across 5 commits, comfortably inside the runbook's 100 KiB threshold, so it is a straight bump by the rule rather than in spite of it. |

llama/CMakeLists.txt

@@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE)
 FetchContent_Declare(
 	llama.cpp
 	GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git
-	GIT_TAG        b10731
+	GIT_TAG        b10797
 	PATCH_COMMAND  ${CMAKE_COMMAND}
 		-DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches
 		-DLLAMA_SRC=<SOURCE_DIR>