feat!: upgrade llama.cpp to b10870, and unbreak the CI pipeline (WebUI embed, clang-format, REUSE)

[bernardladenthin]

Summary

• Bumps llama.cpp b10850 → b10870. No project-source change. The patch intersection was not empty — the range touches src/llama-model.cpp and tests/CMakeLists.txt, both modified by patches/0012, one week after it landed — so this could not be waved through on a disjoint file list.
• Repairs two gates that are red on main right now, both from the #421 merge and both mine: clang-format (cast spacing) and REUSE (missing SPDX header on test_model_split.cpp). The PR run reported them; the merge did not wait.
• This run is also the end-to-end proof #421 never got. Its whole publish.yml run was cancelled in the start-gate abort window when the PR was merged, so Java Tests macOS 15 arm64 (Metal) — the only thing that can exercise the patches/0012 failing path — never executed. It runs here.

Why main is red, and why it was invisible locally

clang-format and REUSE run only in CI, so a full local gate chain (build, ctest, mvn verify, bytecode) passes without touching either. Both failures are one file:

Gate	Finding
clang-format 22.1.8	(size_t) 1 in the n_devices loops; repo style is (size_t)1
REUSE	no copyright/licensing information — every other src/test/cpp file has the SPDX pair

The patch file from the same change is fine: REUSE.toml's llama/patches/** glob covers it, because a unified diff cannot carry an inline header without corrupting the diff.

Both verified after the fix with the tools CI uses — clang-format --dry-run --Werror silent, reuse lint at 455/455 files.

The 0012 question, answered by hand before the applier

CLAUDE.md requires this explicitly, because the fail-loud applier detects "does not apply" but never "upstream already fixed this":

git show b10870:src/llama-model.cpp | grep -A3 split_sum
    float split_sum = 0.0f;
    ...
        splits[i] /= split_sum;      # still bare, no zero-sum guard

Upstream has not adopted the fix, so 0012 stays. Upstream's own change to that file (#28160 — resolving LLAMA_LAZY_MODE_AUTO to OFF where mmap is unsupported) sits ~60 lines above the patched region and is unrelated.

What actually changed upstream

The headline number misleads: common/chat.cpp loses 2524 lines, but that is a pure internal reorganisation splitting per-model chat parsers into a new common/parsers/ directory. common/chat.h is byte-identical in the range — jllama.cpp includes it directly and it is #2 on the priority review list — so nothing the project compiles against moved.

Two behaviour changes a header diff cannot see, both reaching every entry point that parses argv:

1. --mmproj-device now defaults to --device instead of auto-selecting. A caller setting --device but not -mmdev now pins the multimodal projector to the same device.
2. The draft model inherits the global device list the same way, and common_speculative_init only overwrites result.devices when the spec list is non-empty (it previously assigned unconditionally), plus forces LLAMA_SPLIT_MODE_LAYER for a single-device draft. That is the path LlamaModelTest#testSpeculativeDecoding drives — the same test that was red on macOS before patches/0012, so a failure there now needs attributing carefully between the two.

ggml.h gains precision enum values plus ggml_prec_set_acc/ggml_prec_set_src, and deprecates ggml_mul_mat_set_prec / ggml_flash_attn_ext_set_prec. The project source has zero references to any of it.

Test plan

• Affected unit / integration tests pass locally
• CI is green on this branch
• Docs / CHANGELOG updated where applicable

Gate	Result
Patch applier (fresh rm -rf llama/build)	9/9 clean, stamp at head 1945e092 = b10870
Server contract, 3 mechanical checks	byte-identical (request fields, set_hard_limits bounds, response keys in both emit forms)
ctest	527/527
clang-format 22.1.8	clean
reuse lint	compliant, 455/455
mvn clean verify	1742 tests, 0 failures, 0 errors
Bytecode gate	616 classes / 5 jars, 0 over major 52

Chunking: full diff 393 KB / 20 commits, over the runbook's 100 KiB threshold; review surface proper is 28 files, +2686 / −2480, of which ~2500 per side is the one mechanical parser move. Bumped straight on that basis, with the raw numbers recorded in the history table so the call is auditable.

What to watch on this run

Java Tests macOS 15 arm64 (Metal) is the one job that matters beyond the usual: it is the first execution of the patches/0012 path on real Metal hardware. Green there closes the verification gap the patch shipped with.

Related issues / PRs

• Follows fix(llama.cpp): fall back to an even layer split when the split weights sum to zero #421 (patch 0012), whose CI never ran to completion.
• Two new rows in docs/history/llama-cpp-breaking-changes.md cover the range and the patch verification.

Checklist

• I have read CONTRIBUTING.md and CODE_OF_CONDUCT.md
• My commits follow Conventional Commits
• No security-sensitive changes

🤖 Generated with Claude Code

https://claude.ai/code/session_01AnNYn8W1xuVxVJtyL34GyH

---

Generated by Claude Code

[claude]

Review Summary ✅

This PR is a well-executed llama.cpp version bump with excellent upstream verification and CI compliance restoration. No issues identified.

Key Strengths:

• All 4 version files updated consistently (CMakeLists.txt, README, CLAUDE.md, LlamaCppVersion.java)
• Manual patch verification: patches/0012 remains valid, applier clean (9/9)
• CI fixes: clang-format spacing + REUSE header added correctly
• Comprehensive documentation in breaking-changes.md
• Full test suite: 527 C++ tests, 1,742 Java tests, all passing

Observations:

• macOS Metal testing will execute patches/0012 path for first time on real hardware
• Speculative decoding test exercises new draft-device-inheritance behavior

Security & Quality: No vulnerabilities, no risky patterns, pure maintenance work.

Status: Ready to merge ✅

[bernardladenthin]

One correction to the automated review's "Ready to merge ✅", because acting on it now would cost exactly what this PR is for.

The publish.yml pipeline has not started. Start gate (abort window) is still waiting, so no build job, no test job, and in particular no Java Tests macOS 15 arm64 (Metal) has run on this head. What has finished so far is the standalone workflows only: clang-format, test (REUSE), osv-scanner, submit-maven, claude-review.

That matters here more than usual. Merging while the start gate is still open cancels the run — which is precisely what happened to #421 twenty minutes ago: its entire pipeline went to cancelled, so the macOS Metal job never executed and patches/0012 is still carrying a documented, unclosed verification gap. This run is the first opportunity to close it. Merging early would destroy the evidence a second time.

Current state

clang-format	✅ success — was red on main, fixed by d0d1587
test (REUSE)	✅ success — same
osv-scanner, scan-pr, submit-maven, claude-review	✅ success
Start gate (abort window)	⏳ waiting — nothing downstream has begun
Verify GPG signing key ×2	❌ expected, not this PR — see below

The two red GPG checks are not this PR's

Both preflights declare environment: maven-central, and GitHub withholds environment secrets from any pull_request context, so they fail on every PR here regardless of content. Verified rather than assumed: the same two checks (102369826188, 102369826179) were red on #421, which merged cleanly. A re-run cannot help — the cause is secret visibility, not flakiness. No action taken and none needed.

What to watch when the gate releases

Java Tests macOS 15 arm64 (Metal) is the job that matters. If it goes red, it should not be attributed to patches/0012 by default: the b10850→b10870 range also changed common_speculative_init's device handling (it now only overwrites result.devices when the spec list is non-empty, and forces LLAMA_SPLIT_MODE_LAYER for a single-device draft), which is the exact path LlamaModelTest#testSpeculativeDecoding drives — the same test that was red on macOS before 0012. Those two causes need separating before anything is pushed.

---

Generated by Claude Code

[bernardladenthin]

Build WebUI assets (shared) is failing on this PR. It is not this PR's failure — it is red on main too, and has been since #420. Correcting myself: my first read was that the b10870 bump moved the file. It did not.

What fails

cc1plus: fatal error: embed.cpp: No such file or directory

The npm/Svelte build succeeds; the next step dies — Embed assets into ui.cpp / ui.h, which does g++ -O2 -std=c++17 -o llama-ui-embed embed.cpp.

Why it is not this PR's

tools/ui/ is byte-identical between b10850 and b10870 (git diff --stat b10850..b10870 -- tools/ui/ is empty), and tools/ui/embed.cpp exists at neither tag. It was deleted upstream by c457e3bf7 "ui : embed assets directly with CMake" (#28445), which replaced the host tool with ui.cpp.in / ui.h.in + scripts/ui-assets.cmake. Present at b10819, gone at b10850 — so the break landed with the b10819→b10850 bump in #420.

Confirmed against the base branch rather than reasoned about: run 34170576764 (#913, workflow_dispatch on main at b10850) shows the same job failing at the same step 7, with every downstream native build skipped.

Why it matters more than it looks

Every native build job declares needs: [startgate, build-webui], so this one failure skips the entire matrix — including Java Tests macOS 15 arm64 (Metal). That is the job this PR exists to reach, and it is also why #913 produced no macOS result. patches/0012 has still never executed on Metal, and cannot until this is fixed.

Proposed fix

Upstream ships the replacement as a standalone, npm-free cmake -P script, driven from tools/ui/CMakeLists.txt:

add_custom_target(llama-ui-assets ALL
    COMMAND ${CMAKE_COMMAND}
        "-DUI_SOURCE_DIR=..."  "-DUI_BINARY_DIR=..."  "-DLLAMA_SOURCE_DIR=..."
        "-DHF_BUCKET=..." "-DHF_VERSION=..." "-DHF_ENABLED=..." "-DBUILD_UI=..."
        "-DLLAMA_UI_GZIP=${LLAMA_UI_GZIP}"
        -P "${PROJECT_SOURCE_DIR}/scripts/ui-assets.cmake")

So the workflow step can invoke that same script and copy the generated ui.cpp/ui.h into llama/webui-generated/, instead of compiling a file that no longer exists. That keeps the existing design intent — the WebUI auto-follows the pinned tag — and drops our hand-rolled gzip loop, since LLAMA_UI_GZIP is now upstream's own knob.

Not pushed here: it is a publish.yml change unrelated to this bump's content, and it pre-dates this branch. Awaiting a decision on whether it lands in this PR or its own.

Lesson worth recording

tools/ui/** sits on the excluded path list used for bump review and chunking. That exclusion is about compile/link risk to the project's C++ surface — but tools/ui also feeds a CI job, so an upstream change there is invisible to the review yet breaks the pipeline. Same shape as the ggml/src/** exclusion that hid the patches/0012 root cause for ~180 builds.

---

Generated by Claude Code

[bernardladenthin]

Updating my previous comment: the WebUI fix is pushed here now (b1db2c2) — the maintainer decided it lands in this PR rather than a separate one, since it is what unblocks the macOS evidence this PR exists to produce.

The step now runs upstream's scripts/ui-assets.cmake on the dist/ the npm step already built (BUILD_UI=OFF, HF_ENABLED=OFF — no second npm run, no Hugging Face fetch), and the sparse checkout gains scripts alongside tools/ui.

One finding worth flagging: the old guard passed the failure case

The step's completeness check was grep -q LLAMA_UI_HAS_ASSETS ui.h. That silently passes an empty WebUI under the new templates, because ui.h.in emits

/* #undef LLAMA_UI_HAS_ASSETS */

for an empty table — the token is present either way. The old embed.cpp emitted no such line, which is why the naive grep used to be sufficient. Had I ported the step without re-testing the negative path, this PR would have shipped a guard that cannot fail.

Falsified rather than reasoned about — with dist/index.html removed:

	old guard	new guard
exit code	0 ❌	1 ✅
message	LLAMA_UI_HAS_ASSETS: present (real WebUI embedded)	ERROR: ... empty asset table (assets=0)
ui.cpp	694 B (vs 3862 B when populated)	—

The guard now asserts the active #define plus a non-zero count parsed from std::array<llama_ui_asset, N>.

How this was verified

Not by reading the script — by running it. The run: block was extracted verbatim from publish.yml and executed in a simulated CI layout (llamacpp-ui/{tools/ui,scripts} from a real git archive of b10870, GITHUB_WORKSPACE/RUNNER_TEMP set):

• good path → exit 0, LLAMA_UI_HAS_ASSETS: present, 8 assets embedded, exactly ui.cpp + ui.h in the artifact dir
• empty-dist path → exit 1 (above)
• generated ui.cpp/ui.h compile standalone and expose the same struct + three functions as src/main/cpp/webui_stub/ui.h, so they remain a drop-in for the stub

Also corrected in the same commit: CLAUDE.md's WebUI section and local recipe still described the deleted embed.cpp, as did the stub's header comment. webui_stub/ui.h is outside the clang-format CI glob (*.cpp/*.hpp only), so its pre-existing formatting is deliberately left alone rather than swept into this diff.

---

Generated by Claude Code

[claude]

Code Review: PR #422 - llama.cpp b10850 → b10870

Summary

This PR successfully upgrades llama.cpp and resolves three CI failures (clang-format, REUSE compliance, and the WebUI build step regression). The work is thorough and well-documented.

✅ Strengths

1. WebUI Asset Generation Rewrite — Correctly adapts to upstream #28445's deletion of embed.cpp and adoption of CMake script-based generation. The pivot from a C++ host tool to scripts/ui-assets.cmake is handled cleanly.

2. Robust Asset Completeness Guard — The multi-stage validation (grep for active #define + sed regex to extract count + numeric check) is a significant improvement over the naive grep, correctly addressing the /* #undef LLAMA_UI_HAS_ASSETS */ trap upstream's template emits for empty tables.

3. Comprehensive Documentation — CLAUDE.md and README.md updates explain not just what changed but why — the upstream reasoning, the priority-1 path mechanics, and the guard's intent. This is exemplary.

4. Consistent Version Pinning — All four pin sites updated in lockstep (CMakeLists.txt, README, CLAUDE.md, LlamaCppVersion.java), with the pinning assertion test confirming none drift apart.

5. Patch Verification Done Right — Manual verification that patches/0012 still applies, with the specific hand-checked command shown, closing the "upstream already fixed this" gap the fail-loud applier can't detect.

6. clang-format + REUSE Fixes — Both CI gate failures identified and fixed (cast spacing and SPDX header).

⚠️ Minor Observations

1. Sed-Based Asset Count Extraction (line 75 in publish.yml)

   • The regex assumes array declaration fits on a single line
   • Standard for generated code, but the guard correctly fails-safe if it doesn't match
   • Worth noting for future upstream template changes

2. File Handling — Relies on set -e to catch missing files. This is correct for CI but implicit. The preceding ls -la provides visibility.

3. Regex Anchoring — The pattern uses leading .* which is permissive. Stricter anchoring would be more defensive but unlikely to cause issues here.

🚀 Readiness

• ✅ Patches verified (patches/0012 manually re-checked; all 9 hashes stamped at b10870)
• ✅ Server contract validated (byte-identical)
• ✅ Local gates passing (527 C++ tests, 1742 Java tests, clang-format, REUSE)
• ⏳ Awaiting CI completion — critically: Java Tests macOS 15 arm64 (Metal) to validate patches/0012 on real hardware

Recommended: Merge once the Metal job confirms green. Remember to check the PR's "CI is green" box once CI finishes.

🤖 Generated with Claude Code

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud