Skip to content

Add Spark-X2.5-4B support via llama.cpp #38

Open

Description

@pirumpi

Proposal

Research date: 2026-09-16. Tapioca checkout: efe7a55d6e4641daecf7b35c9e04201c96c29f78.

Qualify Spark-X2.5-4B Q4_K_M as an optional local text/chat model through llama.cpp. This is an implementation proposal, not current Tapioca support. The existing b10603 bundle predates this architecture; a tested runtime upgrade is part of this issue. Do not change default model recommendations.

Official identity and access

  • Official model card, configuration, and official GGUF repository.
  • Public release was discoverable September 1, 2026; the Hub repositories were created August 24 (HF) and August 28 (GGUF), which are not necessarily release dates. The current GGUF revision was updated September 15. The official card does not give a definitive launch date.
  • Both official repositories are public and ungated, with Apache-2.0 licensing. Preserve the license and applicable notices; no account/token approval is required for the selected download.
  • Dense Spark2_5ForCausalLM / spark2_5, about 4.1B parameters; 36 layers, 16 attention heads, 4 KV heads, head dimension 256. Hybrid attention repeats three 512-token sliding-window layers then one full-attention layer. Fused QKV, sigmoid head gates, GELU FFN, and different full/sliding RoPE dimensions. Text only.

Exact initial artifact

  • Repository: XHToken/Spark-X2.5-4B-GGUF
  • Revision: 902d865994943ab9235670e24f01846ee06091f2
  • File: Spark-X2.5-4B-Q4_K_M.gguf
  • Size: 2,600,224,352 bytes (about 2.42 GiB)
  • Hub-reported SHA-256: adfcfa19a4ed6a5985da8bf565fe15f8e1a7e131d79bae2d19d48d1c40109428
  • Pinned download

Today's bounded range read returned HTTP 206 and verified GGUF v3, 290 tensors, general.architecture=spark2_5, tokenizer.ggml.pre=spark2_5, the 36-layer attention pattern, and full/sliding RoPE dimensions 64/256. This is not a complete checksum verification or inference test; qualification must download and hash the whole file.

Runtime and platform path

Upstream PR #27868 merged September 6 at 3ad1ba7336986d98592d3e28cafd1a406715351f. It adds model loading, tokenizer/conversion support and architecture tests without requiring a new GGML backend/operator. Its reference-model tests cover the 1.7B model on CPU/A100, not this 4B Q4 artifact or Tapioca's other platforms.

llama.cpp v0.4.1, released September 14, includes this merge and points to b10964 binaries. GitHub ancestry confirms the Spark merge is 225 commits ahead of b10603, not included in the current pin. b10964 lists matching macOS ARM64, Windows x64 Vulkan/ARM64 CPU, and Linux x64/ARM64 Vulkan archives.

Qualification targets: CPU fallback, macOS Apple Silicon Metal, Windows x64 Vulkan, Linux x64/ARM64 Vulkan, and Windows ARM64 CPU. Advertise only targets that pass. CUDA exists upstream but is not a shipped Tapioca language-model bundle. MLX/MLX-VLM would require a separate artifact/runtime qualification; Diffusers, ONNX, DirectML, image/audio/video and mobile are outside this issue.

Hardware guidance

Start testing at 8 GiB system/unified memory, 16 GiB preferred, with an 8K context and bounded output. These are conservative qualification targets, not measured minimums. Reserve memory beyond the 2.42 GiB weights for KV cache, compute buffers, the application and OS. Measure GPU VRAM and host RSS separately.

From the official dimensions, FP16 K+V storage is 4,096 bytes per layer/token. Allocating all 36 layers at 8K would be about 1.125 GiB before overhead; bounded sliding-window allocation can reduce this, but actual runtime allocation must be measured. The metadata's 1,048,576-token context is not a consumer-hardware promise. Do not advertise 1M operation.

Implementation and acceptance tests

  1. Qualify and pin b10964 (or a specifically verified successor), verify archive integrity/layout, and test clean runtime replacement rather than silently reusing a stale bundle. Audit server flags and protocol changes; v0.4.1 also changes load-mode options and chat/schema handling. Tapioca currently launches with --jinja, --ctx-size, and --n-gpu-layers.
  2. Add an optional exact-artifact entry to the built-in and public catalogs, including integrity metadata, license, conservative context, and measured hardware guidance. Keep basic chat as the initial advertised capability.
  3. Verify fresh download/full SHA-256, interrupted download, bad checksum rejection and offline cache reuse.
  4. Exercise model loading, system messages, English/Chinese multi-turn chat, streaming, cancellation, EOS termination, and reasoning on/off. Compare Q4 quality against an official higher-precision artifact before recommending it over current small models.
  5. Test tool-call serialization/parsing separately. Keep structured tools unadvertised until complete round trips work; model-generated tool syntax alone is insufficient.
  6. Record exact hardware, memory and latency for every claimed CPU/Metal/Vulkan bundle. Test 8K first, then longer contexts deliberately. No platform is qualified by today's header inspection.
  7. Run all five freshness checks plus affected server/app/desktop/runtime tests and existing-model regressions after the runtime change. Regenerate the catalog checksum only after validation.

Risks and field evidence

Main risks are cross-platform graph correctness, quantization quality, chat-template/reasoning behavior, and regressions from upgrading the shared runtime. The upstream architecture blocker is resolved, but product qualification remains outstanding.

Reddit is non-authoritative: September 1 release thread reports mixed language/coding results and questions long-context utility; September 16 agent-task comparison describes incomplete task execution. Use these to shape tests, not as proof of quality or a Tapioca defect.

Duplicate review searched all open/closed issues for Spark, spark2_5 and llama.cpp, and inspected the related compact-model issues #32 and #37 plus historical #29. They do not track this model. No product code was changed during this review.

Activity

  1. pirumpi commented on Sep 18, 2026

    @pirumpi
    ContributorAuthor

    Implementation is in draft PR #42 (stacked on #41): pinned official Q4_K_M GGUF with size/SHA checks, llama.cpp b10964 CLI and desktop bundles, docs, and tests. Apple Silicon CPU/Metal chat and a full Tapioca pull/serve smoke passed; existing Phi-4 Mini and GLM-4.7-Flash Q8 were regression-smoked. I am leaving this issue open until Windows/Linux Vulkan, low-memory, tool-use, and broad CI qualification are complete.

  2. pirumpi commented on Sep 18, 2026

    @pirumpi
    ContributorAuthor

    Current implementation moved from obsolete stacked #42 to draft PR #43 on released v0.12.0/main. Spark Q4_K_M and llama.cpp b10964 passed Mac CPU/Metal chat and Tapioca pull/serve smoke; release bundles are building in CI. Windows/Linux Vulkan, low-memory and tool-use qualification remain, so this issue stays open.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions