Dawn backend via cffi that runs natively and in Pyodide (Emdawnwebgpu) by hmaarrfk · Pull Request #840 · pygfx/wgpu-py

hmaarrfk · 2026-10-01T03:09:03Z

Note

This pull request was written with the help of an AI assistant (Claude, by Anthropic): the code, the benchmarks and this description. The numbers below were measured on the machines and CI runs that are named next to them.

claude

Draft. This shows that one set of compiled Dawn bindings runs wgpu-py both natively (Dawn) and in Pyodide. In Pyodide it uses the browser's own WebGPU (via Dawn's Emdawnwebgpu), so it is hardware accelerated. It is meant for the discussion, and as a comparable alternative to #839, not as a merge proposal as-is.

xref: #838

Demo and evidence

Live demo: https://www.markharfouche.com/wgpu-py/ (needs a browser with WebGPU, e.g. recent Chrome or Edge; the demo only uses the async API, so it does not need JSPI). It loads Pyodide and the two wheels, prints the adapter info, squares 1M floats with a compute shader, and draws a rotating triangle on a <canvas>. The page is deployed by this PR's workflow, on the fork only.
CI: run 36813360910 on the fork, all jobs green.
- Pyodide in Node.js: navigator.gpu comes from the webgpu npm package, with lavapipe. 73 test cases pass from 7 existing wgpu-py test files (compute, render, render-to-texture, textures). They run with pytest inside Pyodide.
- Pyodide in headless Chrome, on SwiftShader: the same tests pass (72; one test is excluded, see below).
- Native Dawn from conda, on lavapipe: 172 test cases pass from 14 test files, including Add an experimental Dawn backend (generated Cython bindings) #839's tests/test_dawn_backend.py (copied unchanged) and tests/test_dawn_cffi.py. An informational run of the whole suite gives 193 passed; the failures are wgpu-native specific tests, Dawn's limits and feature names, and test shaders that Tint rejects (fixed in Add @interpolate(flat) to integer varyings in test shaders #841).
- The per-call benchmark runs as an informational step, natively and in Pyodide (Node).
Real GPU, locally: headless Chrome 153 on an Intel Arc iGPU (--use-angle=vulkan). The adapter reports vendor='intel' architecture='xe-lpg' backend=WebGPU, the compute results are correct, and the triangle renders on the canvas. The same compute runs natively through the same bindings on Vulkan.

Learnings from #839, applied here (still cffi)

#839 showed that the per-call cost is mostly in the Python wrapper, not in the binding. This PR keeps cffi (API mode) and applies that:

Hot pass encoder calls (draw, set_pipeline, set_bind_group, set_vertex_buffer, set_viewport, dispatch_workgroups, ...) call lib directly. Dawn defers encoder errors to finish(), so they need no error-checking wrapper. set_bind_group() without dynamic offsets allocates nothing. Passes and bundles no longer keep objects alive from Python, because Dawn holds its own references.
No GIL release for encoder calls. In API mode, cffi releases the GIL and saves/restores errno around every call. That is about 15 ns of a ~50 ns empty call. build_cffi.py post-processes the C code that cffi generates, so that only the wgpuRenderPassEncoder*, wgpuComputePassEncoder* and wgpuRenderBundleEncoder* wrappers keep the GIL. These never block or call back into Python. All other functions (waits, present, submit, creation) still release it. Add an experimental Dawn backend (generated Cython bindings) #839 makes the same trade-off.
Validation errors at the call site. Natively, Dawn reports an error synchronously through the uncaptured-error callback. The wrapper of every other call now checks a list after the call (it used a thread-local capture stack before) and raises the error there. In Pyodide this works too when the WebGPU implementation reports the error during the call (as Node's webgpu does); otherwise the error is logged.
Sync waits block in wgpuInstanceWaitAny(), with the GIL released, instead of polling with sleeps.
API that Add an experimental Dawn backend (generated Cython bindings) #839 has: error scopes, real compilation info, the device-lost promise, promise.then() natively (a light thread schedules process_events() in the event loop, as in Add an experimental Dawn backend (generated Cython bindings) #839), and RuntimeError for released objects. tests/test_dawn_backend.py from Add an experimental Dawn backend (generated Cython bindings) #839 passes unchanged.
Shared files, kept identical to Add an experimental Dawn backend (generated Cython bindings) #839 so that the PRs don't conflict: auto.py and _register_backend() (honour an already-loaded backend and WGPUPY_BACKEND; raise when a second, different backend is registered), conftest.py, and tools/bench_backends.py. auto.py adds 4 lines that pick Dawn in Pyodide when wgpu_dawn is installed. Without the auto.py change, import wgpu.backends.dawn followed by wgpu.utils.get_default_device() silently loaded wgpu-native.
Codegen: the report lists API methods that a backend does not implement (as in Add an experimental Dawn backend (generated Cython bindings) #839). The struct-check validation now parses the backend that is being patched; it always parsed the wgpu-native backend.

Before/after: `tools/bench_backends.py` (from #839)

ns per call, median of 3 runs. Intel Arc iGPU (Arrow Lake), Linux, Python 3.14, Dawn v20260929.200159, same machine and session for all columns. "Before" is this PR at 3ad9a6d, "after" is this PR now. The #839 column is origin/dawn at 7ebd4b0, built locally with tools/build_dawn.py.

native	before	after	Cython #839	wgpu-native
`render_pass.draw(3)`	257	71	46	314
`render_pass.set_bind_group(0, bg)`	424	128	114	448
`render_pass.set_viewport(...)`	327	94	74	352
`compute_pass.dispatch_workgroups(1)`	494	274	239	474
`queue.write_buffer(64 bytes)`	2,919	2,471	2,171	2,151
`device.create_bind_group(2 entries)`	5,901	3,305	2,198	6,676
frame: encoder + pass + 10 draws + submit	86,200	39,400	37,700	50,200

Pyodide in Node 26 (`webgpu` npm, same iGPU)	before	after
`render_pass.draw(3)`	733	354
`render_pass.set_bind_group(0, bg)`	1,277	672
`render_pass.set_viewport(...)`	1,199	468
`compute_pass.dispatch_workgroups(1)`	1,192	610
`queue.write_buffer(64 bytes)`	5,782	4,636
`device.create_bind_group(2 entries)`	20,200	13,200
frame: encoder + pass + 10 draws + submit	81,400	89,100

Natively, draw went from 5.6x to 1.5x the Cython cost. A raw cffi call to wgpuRenderPassEncoderDraw (no wrapper at all) measures about 56 ns here, so the Python method now adds about 10–15 ns. The rest of the gap to Cython is in cffi's generic call wrapper: argument tuple, PyArg_UnpackTuple, per-argument conversion.
The Pyodide frame row is noisy and depends on what ran before it. When it runs alone, or right after create_bind_group, after is 10–15% faster than before (65 vs 76 µs). In the full sequence above it was 9% slower. A frame in Pyodide is dominated by the JS side (Emdawnwebgpu and the browser's WebGPU).
I did not measure Add an experimental Dawn backend (generated Cython bindings) #839 in Pyodide with this script. An earlier experiment, with a different script (Add an experimental Dawn backend (generated Cython bindings) #839 built for Pyodide with this PR's glue), measured draw at about 262 ns.

Adapter enumeration (and pygfx on lavapipe)

webgpu.h can only request the "best" adapter. So on a machine with a GPU, Dawn never returns lavapipe, even with force_fallback_adapter. Natively, wgpu_dawn now includes a small C++ helper that calls dawn::native::Instance::EnumerateAdapters (exported by libwebgpu_dawn). enumerate_adapters_sync() therefore returns every adapter, for all backends, de-duplicated and in the documented backend order. Here that is the Intel iGPU and lavapipe, both via Vulkan.

On Vulkan, info["vendor"] is now the driver name, as with wgpu-native. Dawn puts it at the start of the description; its own vendor field is a normalized name ("intel", "mesa"). With this:

pygfx's PYGFX_WGPU_ADAPTER_NAME=llvmpipe selects lavapipe, and its is_lavapipe check (vendor == "llvmpipe") passes.
pygfx's screenshot comparisons now run with the Dawn backend instead of being skipped. With pygfx at aa03f57 (a branch with Tint-compatible WGSL), examples/tests/test_examples.py -k "lavapipe or compare" gives 41 passed and 13 failed with WGPUPY_BACKEND=dawn. wgpu-native gives the same 41 passed and the same 13 failures on this machine (Mesa 26.0.8 / LLVM 21, versus the CI's reference images).

How Emdawnwebgpu gets into a Pyodide extension

Pyodide extensions are Emscripten side modules, and side modules cannot contain an Emscripten JS library. Half of Emdawnwebgpu is such a library (library_webgpu.js), so it is loaded in three steps:

Build time: dawn_bindings/tools/gen_emdawn_glue.py links a throw-away program with --use-port=emdawnwebgpu, using Pyodide's Emscripten and settings. It then cuts the expanded JS library out of the result. That file ships in the wheel.
The extension: webgpu.cpp, the C++ half of Emdawnwebgpu, is compiled into the extension.
Import time: an EM_JS function evaluates the JS inside Pyodide's module scope, which is how Emscripten's dynamic linker evaluates EM_JS bodies. The JS then registers its functions in wasmImports. The extension's wgpu* imports are lazy stubs, so they resolve to these functions on first call.

This needs no Pyodide changes. It is independent of the binding technology.

Binding strategy: cffi in API mode, from one generated cdef

One cdef for both targets: dawn_bindings/tools/gen_cdef.py generates the cdef from Emdawnwebgpu's webgpu.h.
- At Dawn v20260929.200159, Emdawnwebgpu's header is a strict subset of native Dawn's header. All 607 declarations are identical. Native Dawn only adds enum members and Dawn-specific APIs (gen_cdef.py diff).
- So one cdef, and one compiled source, serves both targets. The native build adds the surface structs and the adapter-enumeration helper, and is compiled as C++20.
- In API mode, the C compiler checks the cdef against each target's real header, and constants such as WGPU_STRLEN (32-bit on wasm32) are taken from that header.
The Python backend (wgpu/backends/dawn) started as a copy of the wgpu-native backend, and codegen validates it against Dawn's header (149 C calls, 86 structs).
Callbacks use cffi extern "Python": one static entry point per callback type. ffi.callback is not possible in wasm.
Why cffi: it gives maximum reuse of the existing backend and codegen, and it is already a dependency.

Async model

Native: callbacks use AllowProcessEvents. A sync wait blocks in wgpuInstanceWaitAny on the promise's future. await polls wgpuInstanceProcessEvents with a backoff. For .then(), a light thread schedules process_events() in the event loop while promises are pending; Dawn is never called from that thread.
Pyodide: callbacks use AllowSpontaneous. They run from the JS event loop when the browser's promise resolves, and they resolve the GPUPromise.
- await is plain asyncio.
- The sync API (request_adapter_sync, map_sync, queue.read_buffer, compute_with_buffers, ...) uses JSPI through pyodide.ffi.run_sync. This works in Chrome and in Node 26, when the code is run via e.g. runPythonAsync.

Version constraints

Emdawnwebgpu: Dawn v20260929.200159, the same release as the native dawn conda package. Its remote port needs Emscripten 4.0.10 or later; this PR uses the package zip.
Pyodide: 314.0.7 (Python 3.14, Emscripten 5.0.3), with pyodide-build 0.39.1.
- The glue must be generated with the Emscripten of the target Pyodide.
- Pyodide 0.29.x uses Emscripten 4.0.9.
Build flag: pyodide build --exports pyinit is required. The default export mode tries to export the EM_JS symbol as a function, and the link fails.

What works, what doesn't

Works on both targets:

adapter and device, buffers and mapping, textures and samplers, bind groups
compute and render pipelines, including async creation natively
the sync and async APIs
the canvas in the browser

Natively also: validation errors raised at the call site, error scopes, compilation info, the device-lost promise, and enumeration of all adapters.

Not supported:

wgpu-native extras: GLSL, pipeline statistics, polygon mode, native limits and features, reports
SPIR-V in the browser
In the browser, validation errors usually arrive asynchronously through the uncapturederror event, so they are logged rather than raised from the failing call. This is why test_view_formats is excluded in Chrome.

Known differences natively:

Dawn's limits, for example maxBindGroups is 4.
Tint is stricter than naga. Some test shaders need changes, for example @interpolate(flat) on integer vertex outputs (Add @interpolate(flat) to integer varyings in test shaders #841).
Native surfaces (glfw and friends) compile but are not tested here.

Relation to #839

The two PRs now share the backend selection, conftest.py, the benchmark and test_dawn_backend.py, with identical content. They differ in the binding layer:

Add an experimental Dawn backend (generated Cython bindings) #839 (Cython): fastest per call, natively (draw 46 vs 71 ns here).
This PR (cffi): reuses the existing backend and codegen. It is within about 1.1–1.5x of Cython on the hot calls, and within 5% on a whole frame. It runs in Pyodide from the same bindings, with true adapter enumeration natively.

The Pyodide pieces (the Emdawnwebgpu glue, bindings from the common header subset, JSPI and spontaneous callbacks in the browser) work with either binding layer. Earlier I built #839's Cython backend for Pyodide with this PR's glue (locally, without changing that branch), and it passed the same 73 tests in Node.

Layout

dawn_bindings/: the wgpu-dawn package (cdef, build script, glue, generators), test runners, the demo, and a README with the details.
wgpu/backends/dawn/: the backend. Select it with WGPUPY_BACKEND=dawn. In Pyodide it is the default when wgpu_dawn is installed. wgpu-py still installs and works without it.
codegen/dawn_patcher.py: mappings and validation against the Dawn cdef.
tools/bench_backends.py: the per-call benchmark (from Add an experimental Dawn backend (generated Cython bindings) #839).
.github/workflows/dawn-pyodide.yml: the CI described above.

https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> Add dawn_bindings/, a small separately-built package (wgpu-dawn) with cffi API-mode (out-of-line) bindings to webgpu.h. The same cdef is compiled against native Dawn (libwebgpu_dawn) and, with pyodide build, against Emdawnwebgpu, Dawn's webgpu.h implementation on top of the browser's navigator.gpu. - tools/gen_cdef.py generates the cdef from Emdawnwebgpu's webgpu.h and can diff it against Dawn's native header. At Dawn v20260929.200159 Emdawnwebgpu's header is a strict subset of the native one (607 common declarations with identical structs and signatures; native only adds enum members and Dawn-specific APIs), so one cdef serves both. - Callbacks use cffi extern "Python" entry points (one per callback type), since runtime-generated callbacks are not possible in wasm. - Pyodide extensions are Emscripten side modules, which cannot carry Emscripten JS libraries. tools/gen_emdawn_glue.py runs Emscripten's jsifier over Emdawnwebgpu's JS library once at build time, and emdawn_glue.cpp evaluates the result inside Pyodide's module scope via an EM_JS function and registers it in wasmImports, where the side module's lazy import stubs pick it up. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> Add wgpu/backends/dawn, based on a copy of the wgpu-native backend, on top of the wgpu_dawn bindings. The same Python code runs on native Dawn and on Emdawnwebgpu in Pyodide. - Callbacks are dispatched through a registry keyed by userdata1. - Natively, callbacks use AllowProcessEvents and GPUPromise waits by polling wgpuInstanceProcessEvents (no poll thread). In the browser, callbacks are spontaneous (from the JS event loop); await is plain asyncio, and sync_wait uses JSPI via pyodide.ffi.run_sync. - Buffer reads/writes use wgpuBuffer{Read,Write}MappedRange. - wgpu-native specific features (GLSL, pipeline statistics, primitive extras, native limits, reports) are removed or raise. - Select with WGPUPY_BACKEND=dawn; in Pyodide it is the default when wgpu_dawn is installed. - codegen writes backends/dawn/_mappings.py from the Dawn cdef and validates the C calls and structs in backends/dawn/_api.py. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> - write_mapped with a size that is not a multiple of 4 falls back to wgpuBufferGetMappedRange, since WriteMappedRange needs aligned sizes. - Request the ShaderSourceSPIRV instance feature natively. - renderutils.py no longer imports the wgpu_native backend explicitly, so the render tests can run against another backend. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> - dawn_bindings/tests: scripts (compute async/sync, triangle) that run natively and in Pyodide, plus runners for Pyodide in Node.js (navigator.gpu from the webgpu npm package) and in headless Chrome, which can also run the wgpu-py test files with pytest inside Pyodide. - dawn_bindings/demo: a static page that loads Pyodide and the wheels and runs a compute shader and a rotating triangle on the browser's WebGPU. - .github/workflows/dawn.yml: builds the Pyodide wheel, tests it in Node (lavapipe) and Chrome (SwiftShader), builds and tests natively against Dawn from conda (lavapipe), and deploys the demo to the fork's Pages. - dawn_bindings/README.md: design, async model and version constraints. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> - The surface sType values are already in the shared WGPUSType enum, so only declare the ones that are missing (none at this Dawn version). - Rename the workflow to dawn-pyodide.yml so it does not clash with the workflow of the Cython-based Dawn backend PR. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> The npm pyodide package does not ship the package wheels (cffi), so the browser test loads Pyodide and its packages from jsdelivr. The automatic backend selection in Pyodide now only falls back to the js_webgpu stub when wgpu_dawn is not installed, instead of hiding its import errors. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> Adopt the backend selection changes of the Cython Dawn backend PR (pygfx#839), with identical content, so that both PRs can share them: * auto.py uses a backend that is already registered (e.g. after `import wgpu.backends.dawn`), so that `wgpu.gpu.request_adapter_sync()` no longer loads (and silently switches to) wgpu-native in that case. It also honours the WGPUPY_BACKEND env var. * _register_backend() raises if a second, different backend is registered. * conftest.py loads the backend from WGPUPY_BACKEND before collection. * tools/bench_backends.py: the per-call overhead benchmark from pygfx#839. auto.py additionally selects the Dawn backend in Pyodide when the compiled wgpu_dawn package is installed. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> Performance, keeping cffi (API mode) as the binding layer: * build_cffi.py post-processes the C code that cffi generates, so that the wrappers of wgpuRenderPassEncoder*/wgpuComputePassEncoder*/ wgpuRenderBundleEncoder* functions keep the GIL and skip the errno save/restore. These calls are cheap, never block, and never call back into Python. All other functions still release the GIL. * Hot pass encoder methods (draw, set_pipeline, set_bind_group, set_vertex_buffer, set_viewport, dispatch_workgroups, ...) call lib directly: Dawn defers encoder errors to finish(), so no error-checking wrapper is needed. set_bind_group() without dynamic offsets does no allocation. Objects are no longer kept alive from Python in passes and bundles; Dawn holds its own references. * The error-checking wrapper for other calls is now a call plus a list truthiness test (was a thread-local capture stack). Validation errors that Dawn reports during a call are still raised at the call site natively, and in Pyodide when the WebGPU implementation reports them during the call. * Sync waits block in wgpuInstanceWaitAny() (GIL released) on the promise's future, instead of polling with sleeps. * create_bind_group() writes entries directly into a C array, and write_buffer() avoids an address round trip. API, matching the Cython backend's tests (tests/test_dawn_backend.py, copied unchanged from pygfx#839, passes): * push_error_scope() / pop_error_scope_async(). * get_compilation_info_async() returns real messages. The base GPUCompilationMessage/GPUCompilationInfo now store their values. * The device-lost promise (_get_lost_async). * promise.then() works natively: a light thread schedules process_events() in the event loop while promises are pending. * Released objects raise RuntimeError instead of cffi's TypeError. * process_events is exported from wgpu.backends.dawn. Adapters: * enumerate_adapters_sync() returns all adapters natively, for all backends, including CPU adapters like lavapipe. webgpu.h can only request the "best" adapter, so wgpu_dawn now includes a small C++ helper that calls dawn::native::Instance::EnumerateAdapters (the native build is now C++20). * On Vulkan, info["vendor"] is the driver name, as with wgpu-native (Dawn puts it at the start of the description). This lets downstream code that detects lavapipe with info["vendor"] == "llvmpipe" work unchanged. Fixes: create_render_bundle_encoder() with a depth_stencil_format. Codegen: the struct-check validation now parses the backend being patched (it always parsed the wgpu-native backend), and the report lists API methods that a backend does not implement. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> * Native job: also run tests/test_dawn_backend.py, tests/test_dawn_cffi.py, the buffer and occlusion query tests, an informational run of the full suite (some test shaders are rejected by Tint and are being fixed separately), and the per-call overhead benchmark. Install a C++ compiler for the adapter-enumeration helper. * Pyodide (Node) job: run the benchmark as an informational step. run_node.mjs only awaits main() if it is a coroutine function, so it can run plain scripts like tools/bench_backends.py. * Document the overhead work, the error model and adapter enumeration in dawn_bindings/README.md. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> The same _api.pyx now also compiles for Pyodide, against Emdawnwebgpu (Dawn's webgpu.h on top of the browser's WebGPU), so one backend runs natively and in the browser. Adapted from the cffi-based work in pygfx#840. Codegen: * Vendor Emdawnwebgpu's webgpu.h (same Dawn release, v20260929.200159) as wgpu/resources/emdawn_webgpu.h. It is a subset of Dawn's native header. * The main block of _webgpu.pxd is now generated from it, so it is valid for both targets. The few native-only declarations that the backend uses (logging callback, wgpuDeviceTick, native surface sources) are listed in NATIVE_ONLY and declared in a separate block, from the generated dawn_native_only.h. That header includes webgpu.h natively, and with Emscripten defines them (copied from Dawn's header) with no-op functions. * The codegen checks that the common declarations and enum values are the same in both headers. Build (tools/build_dawn.py): when cross-compiling for Emscripten (e.g. with `pyodide build`), download the pinned Emdawnwebgpu package (sha256-checked), generate the JS glue with gen_emdawn_glue.py (from pygfx#840: link a throw-away program with --use-port=emdawnwebgpu, extract the expanded JS library), and compile Emdawnwebgpu's webgpu.cpp plus emdawn_glue.cpp into the extension. At import, the extension evals the glue in Pyodide's module scope and puts its functions in wasmImports. The glue generator also fixes a bug in Emdawnwebgpu's wgpuBufferGetMappedRange (uninitialized memory instead of the buffer contents). The hatch hook builds it with WGPU_PY_BUILD_DAWN=1 and adds Cython/setuptools as build dependencies; a Pyodide wheel is built with `WGPU_PY_BUILD_NOARCH=1 WGPU_PY_BUILD_DAWN=1 pyodide build --exports pyinit`. Backend (_api.pyx), with _IS_EMSCRIPTEN: * Callbacks use AllowSpontaneous, so the JS event loop calls them. Natively the model is unchanged (AllowProcessEvents, TimedWaitAny for sync waits, an event pump for then()). * sync_wait() suspends with JSPI (pyodide.ffi.run_sync) and waits for the promise's async event; awaiting is plain asyncio. * request_adapter/request_device do not block in the browser. * Validation errors are still raised at the call site when reported during the call (Dawn in Node.js does that); otherwise (Chrome) they are logged from the event loop instead of being raised by a later, unrelated call. * Surfaces are <canvas> elements (CSS selector); the browser presents. * Mapped ranges: copying is done with wgpuBufferRead/WriteMappedRange natively. In the browser, the whole mapped range is fetched once, because Emdawnwebgpu copies each accessed range and the browser does not allow overlapping ranges. * enumerate_adapters returns the browser's adapter. In Pyodide, the Dawn backend is selected automatically if it is built. tools/dawn_pyodide has runners for Node.js (webgpu npm package) and headless Chrome, and example scripts that also run natively. In headless Chrome, the tests that expect a validation error to be raised by the failing call do not pass, because the browser reports it later. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> Modeled on the workflow of pygfx#840, without the GitHub Pages deployment: * pyodide-build: build the wgpu wheel with the Dawn extension for Pyodide (Pyodide 314.0.7, pyodide-build 0.39.1, Pyodide's Emscripten). * pyodide-node: run the example scripts and the wgpu-py tests that apply in Pyodide in Node.js, with navigator.gpu from the `webgpu` npm package (Dawn) on lavapipe. Also run the call-overhead benchmark. * pyodide-chrome: the same in headless Chrome (SwiftShader), with Pyodide from the CDN. Uploads a screenshot of the canvas and the demo page (a static site with the wheel) as an artifact. The native jobs now install a C++ compiler (for dawn_native_extras.cpp), print the enumerated adapters, and also run the example scripts. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> Port what pygfx#840 gained that is not specific to cffi: * wgpu/_classes.py: the base GPUCompilationMessage and GPUCompilationInfo now store their values (same change as in pygfx#840), so the Dawn backend's subclasses no longer duplicate that code. * codegen: the struct-check validation parses the backend being patched, instead of always parsing the wgpu-native backend, and the report lists API methods that a (Python) backend does not implement. * tests/test_dawn_backend.py: the tests that pygfx#840 added in test_dawn_cffi.py: encoder errors raised at finish(), set_bind_group with dynamic offsets (list, numpy, start/length), and a render bundle with a depth_stencil_format (pygfx#839 did not have pygfx#840's bug there; the test now also executes the bundle in a pass with a depth attachment). Also check that enumerated adapters are unique and can create a device. The dynamic-offsets test found a bug in this backend: set_bind_group() tested `if dynamic_offsets_data:`, which drops a numpy array [0] (and raises for longer arrays). It now checks the length. auto.py: in Pyodide, detect the Dawn extension by its file instead of with find_spec(), which imported (and so registered) the backend as a side effect. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

<details><summary>Claude's draft</summary> * tools/dawn_pyodide/demo: a demo page (adapted from pygfx#840's): it installs the wgpu wheel with the Cython Dawn backend in Pyodide, prints the adapter info, runs a compute shader on 1M floats and checks the result, and renders a rotating triangle to a <canvas>, through the browser's WebGPU. * New `pages` job (pushes to `dawn` on the fork only, after the Pyodide jobs pass): deploys that demo to the fork's GitHub Pages. It also builds the demo of the cffi variant from the dawn-pyodide branch (pygfx#840) and puts it under /cffi/, so both stay reachable on one site. If that build fails, a placeholder is deployed at /cffi/ and the main demo still deploys. * The native benchmark step now downloads wgpu-native and runs `bench_backends.py --all`, for a Dawn vs wgpu-native table (informational). The Node and Chrome benchmark steps were already there. * Deselect test_encoder_errors_raise_at_finish in Chrome: like the other call-site error tests, the browser reports the error asynchronously. Resume this Claude session: ``` cd /home/mark/git/feedstock/staged-recipes claude --resume 2cab6db9-a6ea-4976-a0e6-f4153fe5a651 ``` </details> Claude-Session: https://claude.ai/code/session_01QMLpZTQYYCu2K7EaWNnkEG

hmaarrfk added 9 commits September 30, 2026 22:50

hmaarrfk force-pushed the dawn-pyodide branch from 0371b00 to 4927018 Compare October 1, 2026 04:03

hmaarrfk mentioned this pull request Oct 1, 2026

Add an experimental Dawn backend (generated Cython bindings) #839

Closed

hmaarrfk closed this Oct 1, 2026

Navigation Menu

Repository navigation

Dawn backend via cffi that runs natively and in Pyodide (Emdawnwebgpu) - #840

Dawn backend via cffi that runs natively and in Pyodide (Emdawnwebgpu)#840
hmaarrfk wants to merge 9 commits into
pygfx:mainfrom
hmaarrfk:dawn-pyodide

hmaarrfk commented Oct 1, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

hmaarrfk commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Demo and evidence

Learnings from #839, applied here (still cffi)

Before/after: tools/bench_backends.py (from #839)

Adapter enumeration (and pygfx on lavapipe)

How Emdawnwebgpu gets into a Pyodide extension

Binding strategy: cffi in API mode, from one generated cdef

Async model

Version constraints

What works, what doesn't

Relation to #839

Layout

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

hmaarrfk commented Oct 1, 2026 •

edited

Loading

Before/after: `tools/bench_backends.py` (from #839)