Repository navigation
Bump llama.cpp to a691b2345, release v0.8.68 - #124
Merged
Merged
Conversation
172 commits from 785322d17 (b12032+1) to b12205. No NIF change was needed for the bump; decision.cpp is "Ported at" a691b2345 with no hunk to port. decision_compare.exs: ALL MATCH on 9 models against llama-server. - MTP.init/2 drops the DFlash/DSpark device check: upstream #31635 gives the draft context the target's backends, verified on two Sparks. - MTP.init/2 warns when a DFlash target's output device is an RPC device: since #31506 the drafter's features live there (7.9 tok/s with ["CUDA0", "RPC0"], 32.5 with ["RPC0", "CUDA0"]). - Fix the macOS LLAMA_RPC=1 build (SOCK_CLOEXEC, since v0.8.67). - Document the :tensor-over-RPC dirty-scheduler stack overflow (+sssdcpu 256) and the re-measured -sm tensor numbers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
llama.cpp
785322d17→a691b2345(b12205), 172 commits. Upstream versions: llama.cpp 0.9.0, ggml 0.29.0.API: the bump needed no NIF change.
llama.haddsmoe_cache_cpu_share(default0; the binding does not set the MoE cache).common.hdropscommon_get_path_from_envand changescommon_prompt_checkpoint's buffers; the NIF uses neither.ggml.hdropsggml_rope_yarn_corr_dims(not called).ggml-backend.handllama-ext.honly add.llama_model_default_paramsis unchanged. RPC protocol stays at 8.0.1 andGGML_OP_COUNTat 102.Decision port:
llama-vocab.cppandserver-decision.*are untouched; "Ported at" is nowa691b2345.scripts/decision_compare.exsagainstllama-serverat the same commit (Metal,-np 1): ALL MATCH on tinylaya, tinyopenjev, d1-2B, d1-3B, d1-omni-600M, vllm-decision2-Kai, vllm-d3-edge, h2o-lightning-4b and Clef Flash.Binding changes:
MTP.init/2no longer refuses a DFlash/DSpark drafter that is missing one of the target's devices. Upstream #31635 is the same change as thedraft-ctx-other-devicesfork branch. On two Sparks, a drafter onCUDA0alone gives the same text and rate as one loaded on both devices.MTP.init/2warns when a DFlash target's last device is an RPC device. Since #31506 the drafter reads its features from the target's output device. Qwen3.8-27B NVFP4 split over two Sparks::devices["CUDA0", "RPC0"], v0.8.67["CUDA0", "RPC0"], v0.8.68["RPC0", "CUDA0"], v0.8.68The docs and
scripts/spark/e2e.exsnow list the local device last.Fix: an
LLAMA_RPC=1build did not compile on macOS since v0.8.67 (SOCK_CLOEXEC).Known issue (documented, not coded around):
split_mode: :tensorwith an RPC device segfaults the VM on the first decode. Upstream's recursive RPC graph serialization overflows the default 320 KB stack of a dirty CPU scheduler.+sssdcpu 256fixes it;:layeris unaffected. Upstreamllama-bench(gpt-oss-120b, two Sparks):-sm tensor15.2 tok/s against-sm layer50.4, so the advice to use:layeracross hosts stands.Notable upstream changes:
LLAMA_KV_ROWS_DISABLEturns it off.Verified:
LLAMA_RPC=1build;:smoke,:embeddings(Qwen3-Embedding, embeddinggemma-2),:slow,:mtp,:tools,:decision,:decision_clef;:mtp_sidecar(Qwen3.8-27B) andvm_exit_test.exs;--strictand dialyzer clean;a691b2345and compiles.verify-build-flags.shclean)::dflashand bge-small;vm_exit_test.exs,:rpc_liveande2e.exs two-node(12/12);bench/spark_spec.exsunchanged: plain 14.21, MTP 26.55, DFlash 44.24.:mtp_cancelis still the known race (2/4 abort, record updated).Fork branches:
rpc-add-server-no-abortandrpc-client-spin-waitstill apply cleanly.cuda-mmq-src1-reuseconflicts with #31296 and #31047.Upstream defects #1 and #3 are unchanged, and #2 has been re-measured; the re-check is in
docs/release-guide.md.