Repository navigation
Bump llama.cpp to 9581154a2, release v0.8.71 - #128
Merged
Merged
Conversation
95 commits from 9daa0aa22 (b12361+7) to b12447+16. #32306 deprecates the
llama_batch API; the NIF moves every decode to llama_batch_ext and
llama_process and builds without deprecation warnings. decision.cpp is
"Ported at" 9581154a2 with no hunk to port. decision_compare.exs: ALL MATCH
on 9 models against llama-server.
- TokenBatch (llama_nif.h): one llama_batch_ext per context, sized to
n_batch, plus the per-entry output flags decode_tracked records.
decode_append replaces llama_batch_get_one in decode/generate/
generate_tokens; the MTP path fills its decode batch and the common_batch
for common_speculative_process from one call.
- A batch past n_batch now returns {:error, ...} instead of aborting the VM
in llama_context::decode; bad token and seq ids are named in the error.
- MTP docs: :dflash runs DFlash, DFlash 2 and LiliCorr drafters (#32016);
:dspark refuses any non-DSpark drafter.
- RPC protocol stays 10.0.0; upstream defects #1-#3 re-checked as a source
diff.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bumps llama.cpp from
9daa0aa22(b12361+7) to9581154a2(b12447+16), 95 commits. Version → v0.8.71.NIF / API
llama_batchAPI (llama_batch_init,llama_batch_get_one,llama_batch_free,llama_decode,llama_encode). The deprecated calls still work throughllama_batch_compat, but the NIF now uses onlyllama_batch_ext+llama_processand builds with no deprecation warnings:TokenBatch(llama_nif.h): onellama_batch_extper context, sized ton_batch, plus the per-entry output flagsdecode_trackedrecords forsampler_sample_at's bounds.decode_appendreplacesllama_batch_get_oneindecode,generateandgenerate_tokens.common_batchgiven tocommon_speculative_processfrom one call (mtp_push).n_batchentries (batch_eval,batch_eval_sample,decode_batch,embed_decode,embed_batch_decode, an MTP verify) now returns{:error, "... batch exceeds n_batch (N tokens)"}. Before this it aborted the VM inllama_context::decode(GGML_ASSERT(n_tokens_all <= n_batch)). Bad token ids and seq ids inside a batch are named in the error instead offailed with code: -1.generate/generate_tokensrequest logits for the last prompt token only. New regression test intest/nif_guards_test.exs.:dflashstill runs DFlash, DFlash 2 and LiliCorr drafters (picked from the drafter's metadata);:dsparkrefuses any non-DSpark drafter.MTPdocs updated.llama_model_default_params/llama_context_default_paramsunchanged.src/llama-vocab.cpponly adds thezonos3/inkling2pre-tokenizers (tokenizer_st_partitionuntouched);tools/server/server-decision.*untouched.decision.cppis now "Ported at"9581154a2, no hunk to port.ggml-rpcandggml-cpu/CMakeLists.txtuntouched: RPC protocol stays 10.0.0,GGML_OP_COUNT99. Upstream defects Add Jinja chat templates, penalty params, and OpenAI response format #1–Bump v0.4.4: update llama.cpp submodule to b8198 #3 in the release guide re-checked as a source diff: all still present.Verification (M1 Max, Metal)
mix teston a default build (442 passed) and on anLLAMA_RPC=1build (442 passed), plus--include rpc_liveagainst a localRPC.Serverworker (443 passed).:smoke :embeddings :slow :tools :decision :decision_clef(617 passed),:mtpwith Qwen3.6-27B (454 passed),:mtp_sidecarwith Qwen3.8-27B (448 passed, 6 skipped: no Gemma E4B pair),test/vm_exit_test.exs(3 passed, exit 0).scripts/decision_compare.exs: ALL MATCH on 9 models (tinylaya, tinyopenjev, Clef Flash, d1-3B, d1-omni-600M, d1-2B, vllm-decision2 Kai, vllm-d3-edge, h2o-lightning-4b) againstllama-serverat the same commit.mix format --check-formattedandmix dialyzer: clean.mix hex.build, unpacked,LLAMA_BACKEND=cpu mix compileclones9581154a2250…(= submodule) and builds the NIF with no C++ warnings. (mix deps.getneededHEX_OFFLINE=1with the project's deps copied in: the local Hex API-key/OAuth exchange is failing, unrelated to this change.):dflash(no DFlash drafter on disk), DGX Spark.