Skip to content

Add Linux support: PulseAudio system audio, WebRTC AEC, Vulkan whisper.cpp - #214

Open
Mathjk wants to merge 1 commit into
ppXD:mainfrom
Mathjk:linux-port
Open

Mathjk wants to merge 1 commit into
ppXD:mainfrom
Mathjk:linux-port

Conversation

@Mathjk

@Mathjk Mathjk commented Sep 28, 2026

Copy link
Copy Markdown

Summary

Adds Linux support to Wisp, following the existing platform-crate architecture
(wisp-loopback on Windows, wisp-screencapture on macOS):

  • crates/wisp-pulseaudio (new): captures the default PulseAudio sink's
    .monitor source on a dedicated capture thread (PulseAudio objects are
    thread-affine), feeding the shared 16 kHz mono f32 frame channel. Works on
    PulseAudio proper and on PipeWire via pipewire-pulse. Includes a readiness
    timeout, a 250 ms heartbeat so next_frame never blocks indefinitely on
    silent output, panic-safe teardown (catch_unwind closes the channel),
    DONT_MOVE so PA module-rescue cannot rebind the stream, and unit tests for
    the pure helpers (frame decoding, sink-monitor naming, wait logic).
  • App wiring: open_system_capture() returns the monitor source on Linux;
    wisp-aec (WebRTC AEC3) is enabled for Linux the same way as macOS, with the
    existing passthrough fallback.
  • whisper.cpp: the whisper-vulkan cargo feature now also applies to
    Linux, building the vendored ggml Vulkan backend and linking the system
    Vulkan loader. family_runnable / accelerator reporting treat Vulkan as the
    Linux GPU path, matching Windows. Core ML encoder availability is now
    macOS-only (it could never work on Linux builds).
  • Packaging: tauri.linux.conf.json bundles the staged sherpa/onnxruntime
    shared libraries plus libvulkan.so.1 under linux-runtime/, declares the
    real runtime Depends: for the deb (webkit2gtk, gtk, pulse, alsa,
    libvulkan1, ...), and bundles deb + appimage targets. build.rs stages the
    runtime libs and links with --disable-new-dtags so the binary gets
    DT_RPATH=$ORIGIN:$ORIGIN/../lib/Wisp (RUNPATH would not propagate to
    transitive deps of the bundled libs).
  • CI/release: a linux-app job on ubuntu-22.04 compiles the app in
    every PR (default and whisper-vulkan builds), and the release matrix gains
    an Ubuntu 22.04 leg producing .deb + .AppImage. Two non-obvious toolchain
    details are handled: jammy's archive has no glslc (pulled from LunarG's
    official repo) and its meson 0.61 is older than the vendored
    webrtc-audio-processing meson_version >= 0.63 requirement (installed via
    pip instead).

Verification (Ubuntu 24.04, GNOME/X11, PipeWire via pipewire-pulse)

  • cargo check, clippy, fmt clean for the workspace and the new crate;
    8/8 new unit tests pass; 51/51 wisp-models tests pass with
    --all-features.
  • Live capture test: silent desktop delivers heartbeat frames (~250 ms
    cadence) without blocking; a 440 Hz tone played through the default sink is
    captured via the .monitor source (measured frames match wall time at
    16 kHz); stop + restart capture works without hangs.
  • End-to-end: built .deb + .AppImage, launched the app, downloaded a
    model, recorded a meeting with synthesized speech playing through the
    speakers -- the system-audio monitor path transcribed it in real time and
    attributed it to "them".
  • Built artifacts were inspected: correct Depends:, bundled
    sherpa/onnxruntime/vulkan libs, working $ORIGIN-relative RPATH. The
    release build targets Ubuntu 22.04 to keep the glibc floor low; the full
    pipeline (including the vendored ggml Vulkan build and the WebRTC AEC meson
    build) was verified inside an ubuntu:22.04 container.

Notes

  • Default-sink changes mid-recording are resolved once at capture start, same
    as the WASAPI loopback behavior.
  • The binary also retains ggml's CPU fallback when no Vulkan device is
    present.

…r.cpp

- crates/wisp-pulseaudio: new Linux AudioSource capturing the default
  sink's .monitor via libpulse-binding (works on PulseAudio proper and
  PipeWire's pipewire-pulse). Dedicated capture thread, bounded frame
  channel, clean shutdown -- same lifecycle as the WASAPI loopback crate.
- app: open_system_capture() for Linux, wisp-pulseaudio + wisp-aec +
  wisp-engine-whisper-cpp cfg deps; mic-only fallback unchanged.
- wisp-engine-whisper-cpp: extend the whisper-vulkan feature to Linux
  (ggml-vulkan + CPU fallback); family_runnable/test cfg gates updated.
- Packaging: tauri.linux.conf.json stages linux-runtime/*.so into the
  bundle, build.rs emits $ORIGIN rpaths and stages all sherpa/onnxruntime
  shared libs; deb + appimage bundle targets enabled.
- release.yml: ubuntu-22.04 job with the system deps the port needs
  (webkit2gtk, libpulse, meson/ninja for webrtc-sys, Vulkan/SPIR-V tools,
  libclang for bindgen).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant