You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, thanks for the great work on Bonsai-8B and the Q1_0 quantization.
I'd like to ask about plans for running Q1_0 models in the browser via WebGPU.
Current state (as far as I can see):
The Q1_0 custom kernels exist for CUDA (.cu), Metal (.metal), and Vulkan (GLSL/SPIR-V), plus CPU.
Upstream ggml-org/llama.cpp has an active WebGPU backend in development (multiple PRs merged in recent weeks, including Intel subgroup matrix support).
However, the WebGPU backend upstream only targets standard quant types; there is no WGSL implementation of the Q1_0 kernel yet.
Questions:
Is a WGSL port of the Q1_0 kernel on your roadmap, or under consideration?
If not planned internally, would you be open to a community PR once the upstream WebGPU backend stabilizes?
Any preference between waiting for upstream WebGPU to mature vs. an earlier experimental port?
Running Bonsai-8B purely in-browser (no server, no install) would be a great showcase of the 1-bit quantization story — curious where this sits in your plans.
Yeah WebGPU backend in llama.cpp for Q1_0 would be amazing. Currently we are not actively working on it.
Of course would love a PR (can do here, or main llama.cpp or both :) )
I have not played around with the webgpu backend so don't know how good it is
Closing this now, happy to check out a webgpu backend for 1-bit and ternary models in llama.cpp.
Won't have the bandwith near future to work on ourselves, but PRs are welcome (probably best to go to main llama.cpp)
Activity
khosravipasha commentedon Apr 16, 2026
Thanks :)
Yeah WebGPU backend in llama.cpp for Q1_0 would be amazing. Currently we are not actively working on it.
Of course would love a PR (can do here, or main llama.cpp or both :) )
I have not played around with the webgpu backend so don't know how good it is
One a related note Huggingface team made a WebGPU demo you can try here:
I believe its using their transformer.js library.
https://huggingface.co/spaces/webml-community/bonsai-webgpu
khosravipasha commentedon Apr 19, 2026
Closing this now, happy to check out a webgpu backend for 1-bit and ternary models in llama.cpp.
Won't have the bandwith near future to work on ourselves, but PRs are welcome (probably best to go to main llama.cpp)
For now can check these two demos:
khosravipasha commentedon May 5, 2026
Just noticed a webgpu kernel was merged in main llama.cpp here: ggml-org#22374
Have not picked up the changes in our fork yet, will do sometime this week.