Skip to content

Bump llama.cpp to 9581154a2, release v0.8.71 - #128

Merged
nyo16 merged 1 commit into
masterfrom
bump-llama-9581154a2
Nov 17, 2026
Merged

nyo16 merged 1 commit into
masterfrom
bump-llama-9581154a2

Conversation

@nyo16

@nyo16 nyo16 commented Nov 17, 2026

Copy link
Copy Markdown
Owner

Bumps llama.cpp from 9daa0aa22 (b12361+7) to 9581154a2 (b12447+16), 95 commits. Version → v0.8.71.

NIF / API

  • #32306 deprecates the llama_batch API (llama_batch_init, llama_batch_get_one, llama_batch_free, llama_decode, llama_encode). The deprecated calls still work through llama_batch_compat, but the NIF now uses only llama_batch_ext + llama_process and builds with no deprecation warnings:
    • TokenBatch (llama_nif.h): one llama_batch_ext per context, sized to n_batch, plus the per-entry output flags decode_tracked records for sampler_sample_at's bounds.
    • decode_append replaces llama_batch_get_one in decode, generate and generate_tokens.
    • MTP prefill/verify/rollback fill the target's decode batch and the common_batch given to common_speculative_process from one call (mtp_push).
  • Behaviour: a batch NIF handed more than n_batch entries (batch_eval, batch_eval_sample, decode_batch, embed_decode, embed_batch_decode, an MTP verify) now returns {:error, "... batch exceeds n_batch (N tokens)"}. Before this it aborted the VM in llama_context::decode (GGML_ASSERT(n_tokens_all <= n_batch)). Bad token ids and seq ids inside a batch are named in the error instead of failed with code: -1. generate/generate_tokens request logits for the last prompt token only. New regression test in test/nif_guards_test.exs.
  • #32016 splits DFlash into a driver per head. :dflash still runs DFlash, DFlash 2 and LiliCorr drafters (picked from the drafter's metadata); :dspark refuses any non-DSpark drafter. MTP docs updated.
  • llama_model_default_params / llama_context_default_params unchanged. src/llama-vocab.cpp only adds the zonos3/inkling2 pre-tokenizers (tokenizer_st_partition untouched); tools/server/server-decision.* untouched. decision.cpp is now "Ported at" 9581154a2, no hunk to port.
  • ggml headers, ggml-rpc and ggml-cpu/CMakeLists.txt untouched: RPC protocol stays 10.0.0, GGML_OP_COUNT 99. Upstream defects Add Jinja chat templates, penalty params, and OpenAI response format #1–Bump v0.4.4: update llama.cpp submodule to b8198 #3 in the release guide re-checked as a source diff: all still present.

Verification (M1 Max, Metal)

  • mix test on a default build (442 passed) and on an LLAMA_RPC=1 build (442 passed), plus --include rpc_live against a local RPC.Server worker (443 passed).
  • :smoke :embeddings :slow :tools :decision :decision_clef (617 passed), :mtp with Qwen3.6-27B (454 passed), :mtp_sidecar with Qwen3.8-27B (448 passed, 6 skipped: no Gemma E4B pair), test/vm_exit_test.exs (3 passed, exit 0).
  • scripts/decision_compare.exs: ALL MATCH on 9 models (tinylaya, tinyopenjev, Clef Flash, d1-3B, d1-omni-600M, d1-2B, vllm-decision2 Kai, vllm-d3-edge, h2o-lightning-4b) against llama-server at the same commit.
  • mix format --check-formatted and mix dialyzer: clean.
  • Hex source build: mix hex.build, unpacked, LLAMA_BACKEND=cpu mix compile clones 9581154a2250… (= submodule) and builds the NIF with no C++ warnings. (mix deps.get needed HEX_OFFLINE=1 with the project's deps copied in: the local Hex API-key/OAuth exchange is failing, unrelated to this change.)
  • Not run: :dflash (no DFlash drafter on disk), DGX Spark.

95 commits from 9daa0aa22 (b12361+7) to b12447+16. #32306 deprecates the
llama_batch API; the NIF moves every decode to llama_batch_ext and
llama_process and builds without deprecation warnings. decision.cpp is
"Ported at" 9581154a2 with no hunk to port. decision_compare.exs: ALL MATCH
on 9 models against llama-server.

- TokenBatch (llama_nif.h): one llama_batch_ext per context, sized to
  n_batch, plus the per-entry output flags decode_tracked records.
  decode_append replaces llama_batch_get_one in decode/generate/
  generate_tokens; the MTP path fills its decode batch and the common_batch
  for common_speculative_process from one call.
- A batch past n_batch now returns {:error, ...} instead of aborting the VM
  in llama_context::decode; bad token and seq ids are named in the error.
- MTP docs: :dflash runs DFlash, DFlash 2 and LiliCorr drafters (#32016);
  :dspark refuses any non-DSpark drafter.
- RPC protocol stays 10.0.0; upstream defects #1-#3 re-checked as a source
  diff.
@nyo16
nyo16 merged commit 2742f01 into master Nov 17, 2026
9 checks passed
@nyo16
nyo16 deleted the bump-llama-9581154a2 branch November 17, 2026 10:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant