Repository navigation
Windows CUDA release version missing #27
Description
The link to the windows CUDA version in the releases is giving a 404
Reacted by Nika Asatiani and Pasha Khosravi
Reacted by Jeremy, Pasha Khosravi, Copilot and ...
Metadata
Metadata
Assignees
Labels
No labels
Activity
fuzzballb commentedon Apr 22, 2026
I build it from source and it works perfectly
Running Ternary Bonsai with CUDA on Windows
Requirements: Windows, NVIDIA GPU (Maxwell/GTX 900 series or newer), ~5 GB free disk space
STEP 1 - Install prerequisites
Visual Studio 2022 Community: https://visualstudio.microsoft.com/
During install, select "Desktop development with C++" workload
CMake: https://cmake.org/download/
Git: https://git-scm.com/
CUDA Toolkit 12.4: https://developer.nvidia.com/cuda-12-4-1-download-archive?target_os=Windows&target_arch=x86_64&target_version=11&target_type=exe_network
During install, select Custom and UNCHECK the driver — keep only the Toolkit
STEP 2 - Download models
Get whichever .gguf you want from PrismML's Hugging Face page:
Ternary-Bonsai-1.7B-Q2_0.gguf (~0.4 GB) — fastest
Ternary-Bonsai-4B-Q2_0.gguf (~0.9 GB) — best balance
Place them somewhere easy, e.g. C:\Ternary\
STEP 3 - Download CUDA runtime DLLs
From https://github.com/PrismML-Eng/llama.cpp/releases/tag/prism-b8846-d104cf1
download: cudart-llama-bin-win-cuda-12.4-x64.zip
Extract it to C:\Ternary\cudart\
STEP 4 - Build llama.cpp from source
Open: Start Menu -> "Developer Command Prompt for VS 2022"
Run these commands:
cd C:\Ternary
git clone https://github.com/PrismML-Eng/llama.cpp.git llama.cpp-src
cd llama.cpp-src
set CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.4
cmake -B build -G Ninja -DGGML_CUDA=ON -DCMAKE_CUDA_COMPILER="%CUDA_PATH%\bin\nvcc.exe" -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j 4
This will take 10-20 minutes.
STEP 5 - Copy DLLs
Copy these three files from C:\Ternary\cudart\ into C:\Ternary\llama.cpp-src\build\bin:
cublas64_12.dll
cublasLt64_12.dll
cudart64_12.dll
STEP 6 - Run
C:\Ternary\llama.cpp-src\build\bin\llama-cli.exe -m C:\Ternary\Ternary-Bonsai-4B-Q2_0.gguf -c 2048 -n -1 --conversation -ngl 99 -t 4
Expected speeds (GTX 930MX, 2 GB VRAM):
1.7B model -> ~15 tokens/sec
4B model -> ~7 tokens/sec
Faster GPUs with more VRAM will do significantly better.
The 4B model is recommended as the best balance of speed and quality.
vinodkrishna221 commentedon Apr 22, 2026
i am also having this same issue
khosravipasha commentedon Apr 22, 2026
hmm seems windows has the wrong link, I see the artifact in the list, Need to see what went wrong the release is automated with a github action
https://github.com/PrismML-Eng/llama.cpp/releases/download/prism-b8846-d104cf1/cudart-llama-bin-win-cuda-12.4-x64.zip
https://github.com/PrismML-Eng/llama.cpp/releases/download/prism-b8846-d104cf1/llama-prism-b1-d104cf1-bin-win-cuda-12.4-x64.zip
Copilot commentedon Apr 22, 2026
The agent encountered an error and was unable to start working on this issue: The service is temporarily unavailable. Please try again later. (Request id: D3F8:9E1E6:15AC77:15D7B5:69E8F4EF)
khosravipasha commentedon Apr 22, 2026
On the https://github.com/PrismML-Eng/Bonsai-demo this is handled, not sure why windows one gets the wrong name.
Need to take a closer look, in the meantime can use the windows installer there (should handle everything automatically)
https://github.com/PrismML-Eng/Bonsai-demo/blob/main/setup.ps1
docs: final investigation summary + upstream tracking PrismML-Eng#23 P…
feat: asymmetric K/V support + q8_0 × turbo FA kernel instantiations
docs: final investigation summary + upstream tracking PrismML-Eng#23 P…
feat: asymmetric K/V support + q8_0 × turbo FA kernel instantiations
docs: final investigation summary + upstream tracking PrismML-Eng#23 P…
feat: asymmetric K/V support + q8_0 × turbo FA kernel instantiations
Merge pull request PrismML-Eng#27 from charlie12345/agent/promote-exp…