From b257104757983300374fc17ccfb327764921229b Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 3 Sep 2026 22:36:46 +0000 Subject: [PATCH 1/2] feat!: upgrade llama.cpp from b10731 to b10792 No project-source change was needed, and that was established rather than assumed. The four pin files move (CMakeLists GIT_TAG, README badge, CLAUDE.md, LlamaCppVersion) plus the CHANGELOG and the breaking-changes history. What the range actually contains, on the surface this project links against: * common/common.h additive (common_params::preserve_reasoning_specified) * common/log.h additive (common_log_get_verbosity) * ggml/include/ggml.h additive (ggml_flash_attn_ext_set_n_kv_max) * tools/mtmd/mtmd.h additive (mtmd_input_part, mtmd_tokenize_from_parts) plus const-widening on mtmd_tokenize / mtmd_bitmap_init_lazy and a bitmaps parameter re-typed const T** -> const T* const* * tools/mtmd/mtmd-helper.h const-widening only, six functions * tools/mtmd/CMakeLists.txt +models/deepseek4v.cpp Every one of those is additive or a widening qualification, so existing call sites still compile. The project's only direct mtmd calls are mtmd_helper_bitmap_init_from_file + mtmd_helper_init_opt_default (tts_engine.cpp) and upstream's own tokenize_input_prompts (jllama.cpp); it never calls mtmd_tokenize itself. mtmd_helper::gen_audio -- the whole TTS surface -- is untouched in the range. The server wire contract was checked mechanically, not by header diff: that is the failure class behind the getMetrics() payload break (b10408/b10519) and the repeat_last_n sentinel removal (b10273), where nothing in the signature moved. server-schema.cpp's request-field set, its set_hard_limits bounds, and server-task.cpp's response keys in BOTH emit forms are byte-identical between the two tags: zero added, zero removed, zero re-bounded. Two upstream behaviour changes worth knowing even though nothing broke: * preserve_reasoning now defaults to ENABLED (#28174). common_params_parse_ex sets it when the caller did not, where it previously followed the chat template's own default. On a template advertising supports_preserve_reasoning the full history now carries reasoning traces instead of only the last assistant message -- better continuity, more prompt tokens. This reaches every entry point that parses argv: NativeServer in both modes and LlamaModel's own parameter parse. --no-reasoning-preserve restores the old behaviour. * data: URLs are now accepted for input_audio and input_video, not only images (#27735). Also picked up: a fix for Qwen3-TTS-0.6b (#28231), where an F16 ffn_down overflowed on intermediate peaks past the 65504 ceiling and turned the residual into NaN. That is on the TextToSpeech path this project ships. Patches: all 8 still apply, and this was not a free pass -- 3 of their 42 target files are in the range (common/arg.cpp and tools/mtmd/mtmd-cli.cpp for 0001, tools/server/server-context.cpp for 0002/0003/0010), so the applier had to be run for real. Fresh build dir, stamp written at head c5a5535e. 0010 and 0001 were additionally re-checked by hand against the rule in CLAUDE.md, because the fail-loud applier detects "does not apply" but never "upstream already fixed this, drop me": get_res_model_info() still emits an uncast enum for vocab_type, and common_params_parse_main still appears nowhere in b10792's arg.h while the _WIN32 argv override is still in arg.cpp -- upstream has not adopted #26416. Chunking: the runbook's 100 KiB-per-step rule was deliberately NOT applied here. The full b10731...b10792 diff is 4.19 MB, but docs/ops/Hexagon.csv alone is 19792 generated lines and the GPU backends account for most of the rest -- both outside the review surface the rule exists to bound (ggml/src/** is on the "safe to skip" list). Across common/, include/, tools/server/, tools/mtmd/ and ggml/include/ the range is 35 files, +758/-175, reviewable in one pass; chunking would have produced ~44 commits without adding a line of review. The measured figures are recorded in the history file so the deviation is auditable rather than silent. Verified locally: fresh configure clean, full build with zero warnings on the project TUs, ctest 520/520, mvn clean verify 1474 Java tests green (including NativeLibraryLoadSmokeTest, which cross-checks the new LLAMA_CPP_VERSION against the actually-linked build-info), and the new bytecode gate clean over llama/target. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH --- CHANGELOG.md | 22 +++++++++++++++++++ CLAUDE.md | 8 +++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../llama/value/LlamaCppVersion.java | 8 +++---- 6 files changed, 34 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 254739af..d13ab014 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,28 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by where the backend cannot provide it, `OFF` disables it. ### Changed +- **llama.cpp `b10731` → `b10792`.** No project-source change: every header move in the range is + additive or a **widening** const-qualification, and the server wire contract is byte-identical + (request-field set, `set_hard_limits` bounds and response keys all verified mechanically, which is + the check that catches the contract-behind-a-stable-signature breaks a header diff cannot see). + All 8 local patches still apply. + + Two upstream changes are worth knowing even though nothing broke: + + 1. **`preserve_reasoning` now defaults to enabled** (llama.cpp #28174). `common_params_parse_ex` + sets it when the caller did not, where it previously followed the chat template's own default. + On a template advertising `supports_preserve_reasoning`, the full history now carries reasoning + traces instead of only the last assistant message — better continuity, **more prompt tokens**. + It applies to every entry point that parses argv, so `NativeServer` in both modes and + `LlamaModel`'s own parameter parse. Pass `--no-reasoning-preserve` to restore the old behaviour. + 2. **`data:` URLs are now accepted for `input_audio` and `input_video`**, not only images + (llama.cpp #27735) — so an OpenAI-style request may inline base64 audio/video the same way it + could already inline an image. + + Also picked up: a fix for **Qwen3-TTS-0.6b** (llama.cpp #28231), where an F16 `ffn_down` overflowed + on intermediate peaks past the 65504 ceiling and turned the residual into NaN. That is on the + `TextToSpeech` path this project ships. + - **llama.cpp `b10682` → `b10731`.** One project-source change came out of it, and it is the kind a header diff does not surface: upstream renamed `--tensor-read-lazy` to `-lzm` / `--lazy-mode` (env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`) **with no alias**. The binding emitted diff --git a/CLAUDE.md b/CLAUDE.md index c8634059..b3ac44cc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10731** +Current llama.cpp pinned version: **b10792** ## Upgrading CUDA Version @@ -490,7 +490,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b10731 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10792 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -530,7 +530,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10731`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10792`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1451,7 +1451,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10731`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10792`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index e01b45fe..fcf96d2d 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10731](https://img.shields.io/badge/llama.cpp-%23b10731-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10731) +[![llama.cpp b10792](https://img.shields.io/badge/llama.cpp-%23b10792-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10792) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index e658af5e..c83bb9ed 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -686,3 +686,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10679–b10682 | `ggml/src/ggml-metal/ggml-metal-tuning.cpp` (**#27932: flash-attention vec tunings for M1 Max**), `ggml/src/ggml-vulkan/ggml-vulkan.cpp` + three `vulkan-shaders/*.comp` (**#27925: `mul_mat_id` pads K rather than N**), `scripts/snapdragon/{build,sdk,setup-sdk}.py` + `docs/backend/snapdragon/windows.md` (**#27903: Windows SDK setup**), `tests/test-backend-ops.cpp` | **No project-source change, and the range cannot require one.** Ten files, 575 insertions / 59 deletions, 53 KB of diff — well under the runbook's 100 KiB chunking threshold, so this was taken as a single step. **Not one changed file is on the priority review list**: the delta is confined to `ggml/src/` backend implementations (Metal tuning tables, Vulkan matmul shaders), the Snapdragon build scripts (never compiled here), documentation, and an upstream test (`LLAMA_BUILD_TESTS` is OFF for a FetchContent subproject). Nothing under `common/`, `include/`, `tools/server/`, `tools/mtmd/` or `ggml/include/` moved, so the request-field set, its bounds, the emitted response keys, the `getMetrics()` class of silent contract break and the whole `mtmd_helper::gen_audio` surface are all provably out of scope rather than merely checked. The two ggml changes are backend-internal: neither `ggml_type` nor any `ggml-backend.h` type is touched, so the Metal and Vulkan classifier artifacts pick the work up by rebuilding, with no wiring change. | | b10679–b10682 | patches + upstream verification | **Zero intersection with the patch set, established mechanically rather than by re-running the applier and hoping.** The 42 files the eight patches touch were intersected against the range's changed-file list: empty. No patch context can therefore have moved, and `0001` stays at its 37-file form. Confirmed by a real build: fresh `cmake -B build-b10682 -DBUILD_TESTING=ON` through the FetchContent `PATCH_COMMAND`, which is fail-loud and pins its stamp to the checked-out commit — a fresh build directory is mandatory for exactly that reason. Configure reported `ggml commit: 5ea1b124e`, Release build clean, **`ctest` 520/520**, Java **1476 run / 0 failures / 17 model-gated skips**, `NativeLibraryLoadSmokeTest` **4/4 with 0 skipped** — including the pin cross-check against the freshly linked `libjllama.so`. `mvn spotless:apply` produced no changes. **One diagnostic defect surfaced during this verification and was fixed.** The first (incremental) run of that cross-check failed with *"Linked build-info `b10682-5ea1b124e` must start with the pinned tag `b10679-`"* even though every pin site already read b10682. Cause: `LLAMA_CPP_VERSION` is a `static final String`, i.e. a **compile-time constant that javac inlines into every referencing class** — including the test itself. An incremental build recompiles the constant but not the test class, whose own source did not change, so the stale literal survived in its constant pool (verified with `strings` on the `.class`: the test bytecode was from the previous build and still carried `b10679-`). A `clean` build is green. The assertion message named only the drift cause, i.e. the one that did **not** apply, so it now names both and tells the reader which is which. CI cannot hit this — it always builds from a clean checkout. | | b10682–b10731 | `common/arg.cpp` (**#the `--tensor-read-lazy` option renamed to `-lzm` / `--lazy-mode`, env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`, with no alias for the old spelling**), `common/speculative.cpp` (DFlash draft path refactored from a `features_buf` scratch vector to writing straight into `batch_inject.embd`; `llama_n_batch` → `llama_n_ubatch`), `ggml/include/ggml.h` + `ggml-backend.h` (**purely additive**: `ggml_swiglu_clamp`, `GGML_GLU_OP_SWIGLU_CLAMP`, `ggml_backend_op_alloc_size_may_expand`), `ggml/src/**` (6301 lines of backend kernels), `vendor/cpp-httplib` (1275 lines) | **One project-source change, and it is the kind a header diff alone does not surface.** `ModelParameters.setTensorReadLazy` emitted `putEnum("--tensor-read-lazy", mode)`; that option no longer exists at b10731, so every model load with the knob set would have died on an unknown argument — a *contract* change behind an unchanged signature, exactly the second failure class this file's own priority-list preamble warns about. The emitted flag moved to `--lazy-mode` (three assertions in `ModelParametersTest` with it); the **Java** names (`setTensorReadLazy`, `TensorReadLazyMode`) were deliberately left alone — they are this binding's API, and renaming them would push an upstream CLI spelling onto every consumer (srcmorph carries the knob as a config field, a mojo `@Parameter`, a README row and a sweep case). Everything else was ruled out mechanically rather than by reading: `common/speculative.h` is **byte-unchanged** (only the `.cpp` moved, and we call none of it directly), the two ggml headers have **zero deletions**, and the 42 files the eight patches touch were intersected against the range's changed-file list — the sole hit is `common/arg.cpp`, whose change sits at line ~2729 while `0001`'s hunks there are at 1201/1242, so no patch context moved. Confirmed by the real applier: fresh `cmake -B build-b10731` through the fail-loud FetchContent `PATCH_COMMAND`, configure clean in 47 s, `ggml commit: 0eadefebd`, stamp written for `0eadefebd3f8f92a86d634a0e5b8fffc9dc792c0` over all eight patches, 43 files patched in the tree. | +| b10731–b10792 | `common/common.h` (**additive**: `common_params::preserve_reasoning_specified`; one whitespace realignment on `common_grammar grammar`), `common/arg.cpp` (**behaviour change, not a signature change** — see below), `common/log.h` (**additive**: `common_log_get_verbosity(ggml_log_level)`), `ggml/include/ggml.h` (**additive**: `ggml_flash_attn_ext_set_n_kv_max`), `tools/mtmd/mtmd.h` (**additive**: `mtmd_input_part` + `mtmd_tokenize_from_parts()`, #28250; `mtmd_tokenize`'s `bitmaps` parameter re-typed `const mtmd_bitmap **` → `const mtmd_bitmap * const *`, and `mtmd_tokenize`/`mtmd_bitmap_init_lazy` take a `const mtmd_context *`), `tools/mtmd/mtmd-helper.h` (**const-widening only**, #28307/#28310: `mtmd_helper_support_video`, `mtmd_helper_bitmap_init_from_file`, `mtmd_helper_bitmap_init_from_buf`, `mtmd_helper_video_init`, `mtmd_helper_video_init_from_buf`, `mtmd_helper_model_can_chat`), `tools/mtmd/CMakeLists.txt` (+`models/deepseek4v.cpp`, #28133/#28154), `tools/server/server-common.cpp` + `server-context.cpp` (both internal), `tools/mtmd/clip.cpp` (#28231) | **No project-source change.** Every header move is additive or a **widening** const-qualification, so existing call sites still compile: the project's only direct mtmd calls are `mtmd_helper_bitmap_init_from_file` + `mtmd_helper_init_opt_default` (`tts_engine.cpp`) and upstream's own `tokenize_input_prompts` (`jllama.cpp`), none of which changed shape. It never calls `mtmd_tokenize` itself, and the re-typed parameter would have been a legal qualification conversion anyway (`const T **` → `const T * const *` adds const at one level with nothing above it). `mtmd_helper::gen_audio` — the whole TTS surface — is **untouched** in the range. The **server contract was verified mechanically**, not just by header diff (the failure class that produced the `getMetrics()` and `repeat_last_n` breaks): `server-schema.cpp`'s request-field set, its `set_hard_limits` bounds, and `server-task.cpp`'s response keys in **both** emit forms are byte-identical between the two tags — zero added, zero removed, zero re-bounded. Two upstream changes are worth knowing even though nothing broke: **(1) `preserve_reasoning` now defaults to enabled** (#28174) — `common_params_parse_ex` writes `default_template_kwargs["preserve_reasoning"] = "true"` when the caller did not set it, where it previously followed the chat template's own default, so on a template advertising `supports_preserve_reasoning` the full history now carries reasoning traces and uses more tokens; `--no-reasoning-preserve` restores the old behaviour, and the new `preserve_reasoning_specified` flag is how upstream distinguishes "unset" from "explicitly false". This reaches every entry point that parses argv, i.e. `NativeServer` in both modes and `LlamaModel`'s own parameter parse. **(2)** `handle_media` now accepts `data:` URLs for `input_audio`/`input_video`, not only `input_image` (#27735), and raises `std::invalid_argument` rather than `std::runtime_error` — both inside a `static` function, so no surface the project links against moved. | +| b10731–b10792 | patches + upstream verification | **All 8 patches still apply, and 3 of their 42 target files were in the range** — `common/arg.cpp` and `tools/mtmd/mtmd-cli.cpp` (both `0001`) and `tools/server/server-context.cpp` (`0002`/`0003`/`0010`) — so this was not a no-op intersection and had to be proven by running the applier, not by reasoning about it. Fresh `rm -rf build && cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path: configure clean, stamp written at `head c5a5535e6ebc2e74ab0e3f1b249d7d989aba26a2`, all eight hashes recorded. `0010` was additionally re-checked **by hand** against the rule in `CLAUDE.md` (the fail-loud applier can detect "does not apply" but never "upstream already fixed this, drop me"): `get_res_model_info()` in b10792 still emits `{"vocab_type", meta.model_vocab_type}` with no cast, so the patch stays. `0001` likewise: `common_params_parse_main` still appears nowhere in `b10792:common/arg.h`, and the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override is still in `arg.cpp` — upstream has not adopted [#26416](https://github.com/ggml-org/llama.cpp/issues/26416). **Chunking note:** the runbook's 100 KiB-per-step rule was **not** applied to this range, deliberately. The full `b10731...b10792` diff is 4.19 MB, but `docs/ops/Hexagon.csv` alone is 19 792 generated lines and the GPU backends (`ggml-opencl`, `ggml-hexagon`, `ggml-metal`, `ggml-cuda`, `ggml-sycl`) account for most of the rest — both outside the review surface the rule exists to bound (`ggml/src/**` is on the "safe to skip" list). Across `common/`, `include/`, `tools/server/`, `tools/mtmd/` and `ggml/include/` the range is **35 files, +758 / −175**, reviewable in one pass; chunking it would have produced ~44 commits for one bump without adding a single line of review. The size figures are recorded here so the deviation is auditable rather than silent. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index 1e80c8d2..e5ada9ea 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10731 + GIT_TAG b10792 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index f758f88d..2d723024 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10731"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10792"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10731-"} — call + * plus the resolved upstream commit, e.g. {@code "b10792-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10731"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10792"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10731"; + public static final String LLAMA_CPP_VERSION = "b10792"; // Constants holder — not instantiable. private LlamaCppVersion() {} From 9b3f936c018e407d2bc3e332fde840ca3fe796f0 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 4 Sep 2026 09:39:57 +0000 Subject: [PATCH 2/2] feat!: upgrade llama.cpp from b10792 to b10797 Straight bump by the runbook's own rule rather than in spite of it: the full b10792...b10797 diff is 46 KB across 5 commits, comfortably inside the 100 KiB threshold, so no chunking decision arises here. The review surface is literally empty. The range touches zero files under common/, include/, tools/server/, tools/mtmd/, ggml/include/ or the top-level CMakeLists.txt; all 18 changed files sit in llama.cpp's own src/ model layer, the SYCL backend, the s390x CPU quants, upstream tests and docs. The server wire contract was re-checked anyway and is byte-identical (request fields, response keys). None of the 42 files the eight patches touch is in the range, so no patch context can have moved -- confirmed by running the applier for real regardless: fresh build dir, configure clean, stamp at head 832fd6f1. Two entries are still worth recording: * #28332 is a big-endian correctness fix on a platform this project ships in the DEFAULT JAR. ggml_vec_dot_q5_1_q8_1's s390x path accumulated into an uninitialized v_acc, so Q5_1 inference on IBM Z could return garbage. The build-linux-s390x job cross-builds and runs the C++ suite under qemu-user, but that suite covers OUR endian-sensitive layer (tts_wav.hpp, the JSON/token transforms), not ggml's kernels -- precisely the class of bug that job cannot see and upstream has to fix. * #28278 moves LLAMA_VERSION/LLAMA_COMMIT out of target_compile_definitions(llama PRIVATE ...) into a configure_file-generated src/llama-version.h. That is the one change in the range that could plausibly have reached the project, since NativeLibraryLoadSmokeTest.nativeBuildInfoMatchesPinnedVersionConstant asserts the pin against the linked binary. It does not: jllama.cpp reads llama_build_info() from common/build-info.cpp, a different mechanism. Checked by running the test, not by reading the diff. The CHANGELOG entry is folded to cover b10731 -> b10797 as one range, since b10792 was never released; the per-range detail stays in the history file. Verified locally: fresh configure clean, full build with zero warnings on the project TUs, ctest 520/520, mvn clean verify 1474 Java tests green, bytecode gate clean over llama/target. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH --- CHANGELOG.md | 12 +++++++++++- CLAUDE.md | 8 ++++---- README.md | 2 +- docs/history/llama-cpp-breaking-changes.md | 2 ++ llama/CMakeLists.txt | 2 +- .../net/ladenthin/llama/value/LlamaCppVersion.java | 8 ++++---- 6 files changed, 23 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d13ab014..4a05c41e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,7 +58,7 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by where the backend cannot provide it, `OFF` disables it. ### Changed -- **llama.cpp `b10731` → `b10792`.** No project-source change: every header move in the range is +- **llama.cpp `b10731` → `b10797`.** No project-source change: every header move in the range is additive or a **widening** const-qualification, and the server wire contract is byte-identical (request-field set, `set_hard_limits` bounds and response keys all verified mechanically, which is the check that catches the contract-behind-a-stable-signature breaks a header diff cannot see). @@ -80,6 +80,16 @@ from version 5.0.0 onward. Pre-fork releases (`1.x`–`4.2.0`) were authored by on intermediate peaks past the 65504 ceiling and turned the residual into NaN. That is on the `TextToSpeech` path this project ships. + The final `b10792` → `b10797` step touches **nothing** the project links against — not one file + under `common/`, `include/`, `tools/server/`, `tools/mtmd/` or `ggml/include/`. It carries a + **big-endian correctness fix** that does matter to a platform this project ships in the default + JAR: `ggml-cpu`'s s390x `q5_1` path used an uninitialized `v_acc` accumulator (llama.cpp #28332), + so `Q5_1`-quantised inference on IBM Z could return garbage. There is also an upstream build + change (#28278) that moves `LLAMA_VERSION`/`LLAMA_COMMIT` from compile definitions into a + generated `llama-version.h`; it does not reach `getLlamaCppBuildInfo()`, which reads + `llama_build_info()` from `common/build-info.cpp` instead — verified by the smoke test that + cross-checks the pin against the linked binary. + - **llama.cpp `b10682` → `b10731`.** One project-source change came out of it, and it is the kind a header diff does not surface: upstream renamed `--tensor-read-lazy` to `-lzm` / `--lazy-mode` (env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`) **with no alias**. The binding emitted diff --git a/CLAUDE.md b/CLAUDE.md index b3ac44cc..8708f501 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co Java bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) via JNI, providing a high-level API for LLM inference in Java. The Java layer communicates with a native C++ library through JNI. -Current llama.cpp pinned version: **b10792** +Current llama.cpp pinned version: **b10797** ## Upgrading CUDA Version @@ -490,7 +490,7 @@ needs no extra step here, `build-webui` re-reads the tag and rebuilds the matchi ships no UI): ```bash # needs node/npm + network; embed.cpp is plain C++17 (no npm) -git clone --depth 1 --branch b10792 https://github.com/ggml-org/llama.cpp /tmp/lc +git clone --depth 1 --branch b10797 https://github.com/ggml-org/llama.cpp /tmp/lc ( cd /tmp/lc/tools/ui && npm ci && npm run build \ && ( cd dist && find . -type f -not -path './_gzip/*' \ | while read -r f; do mkdir -p "_gzip/$(dirname "$f")"; gzip -9 -c "$f" > "_gzip/$f"; done ) \ @@ -530,7 +530,7 @@ cache lives in **Depot Cache** over sccache's **WebDAV** backend: - `SCCACHE_WEBDAV_TOKEN: ${{ secrets.DEPOT_TOKEN }}` — a Depot **organization** token, stored as the repo secret **`DEPOT_TOKEN`**. -Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10792`), the +Because `sccache` is **content-addressed** and llama.cpp is pinned (`GIT_TAG b10797`), the ~280 upstream object files are byte-identical every run, so a warm cache recompiles only the *changed* files. Depot's cache is **shared across all branches** (unlike GitHub's per-branch `actions/cache`), so every branch builds incrementally; a `b` version bump @@ -1451,7 +1451,7 @@ ctest --test-dir build --output-on-failure -R "ResultsToJson" #### Upstream source location (in CMake build tree) -llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10792`. +llama.cpp is fetched via CMake FetchContent, pinned to `GIT_TAG b10797`. **GoogleTest** is a separate `BUILD_TESTING`-only FetchContent (`GIT_TAG v1.17.0`), used solely by the `jllama_test` C++ unit-test binary — not by the shipped library, and not coupled to the diff --git a/README.md b/README.md index fcf96d2d..7acf2aa3 100644 --- a/README.md +++ b/README.md @@ -11,7 +11,7 @@ **Build:** ![Java 8+](https://img.shields.io/badge/Java-8%2B-informational) ![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows%20%7C%20Android-lightgrey) -[![llama.cpp b10792](https://img.shields.io/badge/llama.cpp-%23b10792-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10792) +[![llama.cpp b10797](https://img.shields.io/badge/llama.cpp-%23b10797-informational)](https://github.com/ggml-org/llama.cpp/releases/tag/b10797) [![JPMS](https://img.shields.io/badge/JPMS-modular%20JAR-25A162)](https://openjdk.org/projects/jigsaw/) ![JUnit](https://img.shields.io/badge/tested%20with-JUnit6-25A162) [![JSpecify](https://img.shields.io/badge/JSpecify-1.0.0%20%40NullMarked-25A162)](https://jspecify.dev) diff --git a/docs/history/llama-cpp-breaking-changes.md b/docs/history/llama-cpp-breaking-changes.md index c83bb9ed..21cbd864 100644 --- a/docs/history/llama-cpp-breaking-changes.md +++ b/docs/history/llama-cpp-breaking-changes.md @@ -688,3 +688,5 @@ Used during `llama.cpp` version bumps: when upgrading, scan this file from the r | b10682–b10731 | `common/arg.cpp` (**#the `--tensor-read-lazy` option renamed to `-lzm` / `--lazy-mode`, env `LLAMA_ARG_TENSOR_READ_LAZY` → `LLAMA_ARG_LAZY_MODE`, with no alias for the old spelling**), `common/speculative.cpp` (DFlash draft path refactored from a `features_buf` scratch vector to writing straight into `batch_inject.embd`; `llama_n_batch` → `llama_n_ubatch`), `ggml/include/ggml.h` + `ggml-backend.h` (**purely additive**: `ggml_swiglu_clamp`, `GGML_GLU_OP_SWIGLU_CLAMP`, `ggml_backend_op_alloc_size_may_expand`), `ggml/src/**` (6301 lines of backend kernels), `vendor/cpp-httplib` (1275 lines) | **One project-source change, and it is the kind a header diff alone does not surface.** `ModelParameters.setTensorReadLazy` emitted `putEnum("--tensor-read-lazy", mode)`; that option no longer exists at b10731, so every model load with the knob set would have died on an unknown argument — a *contract* change behind an unchanged signature, exactly the second failure class this file's own priority-list preamble warns about. The emitted flag moved to `--lazy-mode` (three assertions in `ModelParametersTest` with it); the **Java** names (`setTensorReadLazy`, `TensorReadLazyMode`) were deliberately left alone — they are this binding's API, and renaming them would push an upstream CLI spelling onto every consumer (srcmorph carries the knob as a config field, a mojo `@Parameter`, a README row and a sweep case). Everything else was ruled out mechanically rather than by reading: `common/speculative.h` is **byte-unchanged** (only the `.cpp` moved, and we call none of it directly), the two ggml headers have **zero deletions**, and the 42 files the eight patches touch were intersected against the range's changed-file list — the sole hit is `common/arg.cpp`, whose change sits at line ~2729 while `0001`'s hunks there are at 1201/1242, so no patch context moved. Confirmed by the real applier: fresh `cmake -B build-b10731` through the fail-loud FetchContent `PATCH_COMMAND`, configure clean in 47 s, `ggml commit: 0eadefebd`, stamp written for `0eadefebd3f8f92a86d634a0e5b8fffc9dc792c0` over all eight patches, 43 files patched in the tree. | | b10731–b10792 | `common/common.h` (**additive**: `common_params::preserve_reasoning_specified`; one whitespace realignment on `common_grammar grammar`), `common/arg.cpp` (**behaviour change, not a signature change** — see below), `common/log.h` (**additive**: `common_log_get_verbosity(ggml_log_level)`), `ggml/include/ggml.h` (**additive**: `ggml_flash_attn_ext_set_n_kv_max`), `tools/mtmd/mtmd.h` (**additive**: `mtmd_input_part` + `mtmd_tokenize_from_parts()`, #28250; `mtmd_tokenize`'s `bitmaps` parameter re-typed `const mtmd_bitmap **` → `const mtmd_bitmap * const *`, and `mtmd_tokenize`/`mtmd_bitmap_init_lazy` take a `const mtmd_context *`), `tools/mtmd/mtmd-helper.h` (**const-widening only**, #28307/#28310: `mtmd_helper_support_video`, `mtmd_helper_bitmap_init_from_file`, `mtmd_helper_bitmap_init_from_buf`, `mtmd_helper_video_init`, `mtmd_helper_video_init_from_buf`, `mtmd_helper_model_can_chat`), `tools/mtmd/CMakeLists.txt` (+`models/deepseek4v.cpp`, #28133/#28154), `tools/server/server-common.cpp` + `server-context.cpp` (both internal), `tools/mtmd/clip.cpp` (#28231) | **No project-source change.** Every header move is additive or a **widening** const-qualification, so existing call sites still compile: the project's only direct mtmd calls are `mtmd_helper_bitmap_init_from_file` + `mtmd_helper_init_opt_default` (`tts_engine.cpp`) and upstream's own `tokenize_input_prompts` (`jllama.cpp`), none of which changed shape. It never calls `mtmd_tokenize` itself, and the re-typed parameter would have been a legal qualification conversion anyway (`const T **` → `const T * const *` adds const at one level with nothing above it). `mtmd_helper::gen_audio` — the whole TTS surface — is **untouched** in the range. The **server contract was verified mechanically**, not just by header diff (the failure class that produced the `getMetrics()` and `repeat_last_n` breaks): `server-schema.cpp`'s request-field set, its `set_hard_limits` bounds, and `server-task.cpp`'s response keys in **both** emit forms are byte-identical between the two tags — zero added, zero removed, zero re-bounded. Two upstream changes are worth knowing even though nothing broke: **(1) `preserve_reasoning` now defaults to enabled** (#28174) — `common_params_parse_ex` writes `default_template_kwargs["preserve_reasoning"] = "true"` when the caller did not set it, where it previously followed the chat template's own default, so on a template advertising `supports_preserve_reasoning` the full history now carries reasoning traces and uses more tokens; `--no-reasoning-preserve` restores the old behaviour, and the new `preserve_reasoning_specified` flag is how upstream distinguishes "unset" from "explicitly false". This reaches every entry point that parses argv, i.e. `NativeServer` in both modes and `LlamaModel`'s own parameter parse. **(2)** `handle_media` now accepts `data:` URLs for `input_audio`/`input_video`, not only `input_image` (#27735), and raises `std::invalid_argument` rather than `std::runtime_error` — both inside a `static` function, so no surface the project links against moved. | | b10731–b10792 | patches + upstream verification | **All 8 patches still apply, and 3 of their 42 target files were in the range** — `common/arg.cpp` and `tools/mtmd/mtmd-cli.cpp` (both `0001`) and `tools/server/server-context.cpp` (`0002`/`0003`/`0010`) — so this was not a no-op intersection and had to be proven by running the applier, not by reasoning about it. Fresh `rm -rf build && cmake -B build -DBUILD_TESTING=ON` through the real `FetchContent` path: configure clean, stamp written at `head c5a5535e6ebc2e74ab0e3f1b249d7d989aba26a2`, all eight hashes recorded. `0010` was additionally re-checked **by hand** against the rule in `CLAUDE.md` (the fail-loud applier can detect "does not apply" but never "upstream already fixed this, drop me"): `get_res_model_info()` in b10792 still emits `{"vocab_type", meta.model_vocab_type}` with no cast, so the patch stays. `0001` likewise: `common_params_parse_main` still appears nowhere in `b10792:common/arg.h`, and the `#ifdef _WIN32` `argv = utf8.ptrs.data()` override is still in `arg.cpp` — upstream has not adopted [#26416](https://github.com/ggml-org/llama.cpp/issues/26416). **Chunking note:** the runbook's 100 KiB-per-step rule was **not** applied to this range, deliberately. The full `b10731...b10792` diff is 4.19 MB, but `docs/ops/Hexagon.csv` alone is 19 792 generated lines and the GPU backends (`ggml-opencl`, `ggml-hexagon`, `ggml-metal`, `ggml-cuda`, `ggml-sycl`) account for most of the rest — both outside the review surface the rule exists to bound (`ggml/src/**` is on the "safe to skip" list). Across `common/`, `include/`, `tools/server/`, `tools/mtmd/` and `ggml/include/` the range is **35 files, +758 / −175**, reviewable in one pass; chunking it would have produced ~44 commits for one bump without adding a single line of review. The size figures are recorded here so the deviation is auditable rather than silent. | +| b10792–b10797 | `ggml/src/ggml-cpu/arch/s390/quants.c` (**#28332: `q5_1` used an uninitialized `v_acc`**), `ggml/src/ggml-sycl/**` (#27610 rms_norm+mul+add / add+add fusion, #26863 `GGML_SYCL_ENABLE_MKL_FA` made a global), `src/llama-hparams.{h,cpp}` + `src/llama-model.cpp` + `src/llama-model-loader.cpp` (**#28323: new internal `n_expert_used_max()`**), `src/CMakeLists.txt` + new `src/llama-version.h.in` (**#28278: stop rebuilding the whole source tree on every commit**), `src/llama.cpp`, `tests/test-backend-ops.cpp`, `docs/backend/SYCL.md` | **No project-source change, and this time the review surface is literally empty**: the range touches **zero** files under `common/`, `include/`, `tools/server/`, `tools/mtmd/`, `ggml/include/` or the top-level `CMakeLists.txt`. Server contract re-checked anyway and byte-identical (request fields, response keys). Two entries are still worth recording. **(1)** #28332 is a **big-endian correctness fix on a platform this project ships in the default JAR**: `ggml_vec_dot_q5_1_q8_1`'s s390x path accumulated into an uninitialized `v_acc`, so `Q5_1` inference on IBM Z could return garbage. The `build-linux-s390x` job cross-builds and runs the C++ suite under `qemu-user`, but that suite covers *our* endian-sensitive layer (`tts_wav.hpp`, the JSON/token transforms), not ggml's kernels — so this is precisely the class of bug the job cannot see and upstream has to fix. **(2)** #28278 moves `LLAMA_VERSION`/`LLAMA_COMMIT` out of `target_compile_definitions(llama PRIVATE ...)` into a `configure_file`-generated `src/llama-version.h`. That is the one change in the range that could plausibly have reached the project, since `NativeLibraryLoadSmokeTest.nativeBuildInfoMatchesPinnedVersionConstant` asserts the pin against the linked binary — but `jllama.cpp` reads `llama_build_info()` from `common/build-info.cpp`, a different mechanism, and the test passes at b10797. Verified rather than reasoned about. | +| b10792–b10797 | patches + upstream verification | **Zero intersection with the patch set, established mechanically.** The 42 files the eight patches touch were intersected against the range's 18 changed files: empty, so no patch context can have moved. Confirmed by running the applier for real anyway: fresh `rm -rf build && cmake -B build -DBUILD_TESTING=ON`, configure clean, stamp at `head 832fd6f1741ad9f66fb2a785002396700666595a`, all eight hashes recorded. **This step needed no chunking decision**: the full `b10792...b10797` diff is 46 KB across 5 commits, comfortably inside the runbook's 100 KiB threshold, so it is a straight bump by the rule rather than in spite of it. | diff --git a/llama/CMakeLists.txt b/llama/CMakeLists.txt index e5ada9ea..079e6697 100644 --- a/llama/CMakeLists.txt +++ b/llama/CMakeLists.txt @@ -173,7 +173,7 @@ set(LLAMA_BUILD_APP OFF CACHE BOOL "" FORCE) FetchContent_Declare( llama.cpp GIT_REPOSITORY https://github.com/ggerganov/llama.cpp.git - GIT_TAG b10792 + GIT_TAG b10797 PATCH_COMMAND ${CMAKE_COMMAND} -DPATCH_DIR=${CMAKE_CURRENT_SOURCE_DIR}/patches -DLLAMA_SRC= diff --git a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java index 2d723024..b65db948 100644 --- a/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java +++ b/llama/src/main/java/net/ladenthin/llama/value/LlamaCppVersion.java @@ -10,13 +10,13 @@ * library was compiled against, exposed as a compile-time constant so callers can render a badge or * emit a startup log line without loading the native library. * - *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10792"}) that mirrors the + *

{@link #LLAMA_CPP_VERSION} is a pure-Java string ({@code "b10797"}) that mirrors the * {@code GIT_TAG} in {@code llama/CMakeLists.txt}. It is available even when {@code libjllama} is * absent (pure-Java checkout, before {@code System.load}), which is what makes it suitable for a * lightweight version badge in Android or other UIs.

* *

For the authoritative value that is baked into the native binary — the build number - * plus the resolved upstream commit, e.g. {@code "b10792-"} — call + * plus the resolved upstream commit, e.g. {@code "b10797-"} — call * {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} instead; that reads llama.cpp's own * {@code build-info} through JNI and therefore cannot drift from the compiled library (but requires * the native library to be loaded).

@@ -24,14 +24,14 @@ public final class LlamaCppVersion { /** - * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10792"}. + * The pinned llama.cpp release tag this library was built against, e.g. {@code "b10797"}. * *

Kept in lockstep with {@code GIT_TAG} in {@code llama/CMakeLists.txt} — see the * "Upgrading/Downgrading llama.cpp Version" checklist in {@code CLAUDE.md}. This is the * compile-time pin; use {@link net.ladenthin.llama.LlamaModel#getLlamaCppBuildInfo()} for the * value actually linked into the native binary.

*/ - public static final String LLAMA_CPP_VERSION = "b10792"; + public static final String LLAMA_CPP_VERSION = "b10797"; // Constants holder — not instantiable. private LlamaCppVersion() {}