Repository navigation
Conversation
Add myModules.home.llama-cpp home-manager module, enabled on the darwin host. Vendors the v0.6.0 package from nixpkgs master into packages/llama-cpp since nixpkgs-unstable still ships b10273 (pre-Clef, no /v1/systemone API). v0.6.0 is the minimum for Clef text+vision decision models and the newest stable release. Metal is enabled by default on aarch64-darwin (GGML_METAL=TRUE, embedded library, llama-server built). Drop the vendored package and default back to pkgs.llama-cpp once nixpkgs-unstable >= 0.6.0. Also bumps the nixpkgs flake input (Sep 30 -> Oct 7).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds llama.cpp to the darwin system via a new home-manager module:
modules/home/llama-cpp.nix(new):myModules.home.llama-cpp.enable+packageoverride option, installs intohome.packagespackages/llama-cpp/package.nix(new): pinned v0.6.0 vendored from nixpkgs masterhosts/darwin/home.nix, imported viamodules/home/default.nixnixpkgsflake input (Sep 30 → Oct 7)Why pinned, not
pkgs.llama-cppnixpkgs-unstable(even after this bump) still ships b10273 (Aug 4) — pre-Clef, no/v1/systemoneAPI. v0.6.0 is the minimum for Clef text+vision decision models (#29831/#29969) and the newest stable (nightlies are atb11476+, unreleased). nixpkgsmasteralready has v0.6.0; drop the vendored package and default back topkgs.llama-cpponcenixpkgs-unstable >= 0.6.0.Metal is on by default on
aarch64-darwin(GGML_METAL=TRUE, embedded library,llama-serverbuilt).Verified
0.6.0,aarch64-darwinsupportedalejandra+deadnixpre-commit hooks passFollow-up
./rebuild.shon darwin (first build compiles Metal backends + Web UI, takes a while), thenllama cli -hf …/llama serve -hf …version/tag/hashin the vendored package