Skip to content

Qualify pinned Granite 4.2 3B and MiniCPM5 2B GGUF models - #41

Closed
pirumpi wants to merge 2 commits into
mainfrom
feat/compact-llama-models
Closed

pirumpi wants to merge 2 commits into
mainfrom
feat/compact-llama-models

Conversation

@pirumpi

@pirumpi pirumpi commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Implements the scoped Q4_K_M catalog/pull path for #32 and #37. This is intentionally draft and does not close the issues yet.

Research date: 2026-09-18. Official sources: Granite card, Granite GGUF, MiniCPM card, and MiniCPM GGUF. Field reports used only as discovery: MiniCPM release discussion and small-model benchmarking.

Current HF metadata shows Granite Q4 bytes/hash changed from the September 2 issue; this PR pins revision c40945d71cd90f249a56985e8155551a9188dc30 and verified full-file SHA-256 e0406663965846ae22a403456eb826ccce5f450840491f71952f18a7cb78e7d5. MiniCPM pins revision 2079a22f3beaa4e306449978533478fe0522f4b3 and verified full-file SHA-256 ec2d5801640099e97d8d7e8003ad4d81f336e757811f03a26173dddf386602fd.

Both exact files were downloaded and hashed. On this 96 GiB Apple Silicon Mac, bundled llama.cpp b10603 successfully loaded and answered with CPU and Metal at 8K context. Granite handled Spanish/system messages, thinking on/off, and a basic tool-call round trip; MiniCPM handled Chinese/system messages and multi-turn chat with thinking off. Tapioca's own pull, offline cache reuse, serve and chat paths passed for MiniCPM; cached pull and serve/chat passed for Granite. Go suite, model-watch tests, catalog validation and checksum passed.

Still required before merging/closing: Windows/Linux CPU/Vulkan model-load and chat tests, memory/latency measurements on smaller hardware, streaming/cancellation checks, and quality comparison. Windows host 192.168.86.11 was unreachable over SSH today. Do not advertise structured tools or long-context support from this PR.

@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pirumpi

pirumpi commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #43, rebuilt on released v0.12.0/main so it preserves the newer Granite and pinned-download implementation. This draft branch conflicted with main and should not be merged.

@pirumpi pirumpi closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant