Skip to content

Commit d43e36b

Browse files
Yalhashclaude
andcommitted
Add Parakeet v3 and Qwen3-ASR as selectable speech engines
asr.engine picks the local recognizer: whisper (default), parakeet or qwen3. Neither new engine needs NVIDIA hardware: - Parakeet TDT 0.6B v3 via onnx-asr on ONNX Runtime: int8 on the CPU by default; uses CUDA or DirectML automatically when that onnxruntime build is installed. 25 European languages. It does not report the language, so the transcript carries the locked/configured one. - Qwen3-ASR (0.6B default, 1.7B option) via qwen-asr on PyTorch: float32 on the CPU, bfloat16 on a CUDA GPU. 30 languages, language detected. Both are optional extras (uv sync --extra parakeet / --extra qwen3) and are imported only when their model loads; a missing package is a ConfigError naming the install command. Settings > Speech gains an engine selector that shows only the chosen engine's options. Tests: adapter tests against fake libraries, wiring/config tests, a settings-dialog test, and slow real-model tests that skip when the extras are not installed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0178pC5pfGXepbgU7JEpqVUh
1 parent 5cdf637 commit d43e36b

14 files changed

Lines changed: 688 additions & 18 deletions

File tree

‎README.md‎

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,10 +4,12 @@ A small always-on-top window that listens to the other side of a call, transcrib
44
locally, translates it into your language, and suggests replies in theirs when you
55
press a hotkey.
66

7-
* **Local speech recognition** with [faster-whisper](https://github.com/SYSTRAN/faster-whisper).
7+
* **Local speech recognition**: [faster-whisper](https://github.com/SYSTRAN/faster-whisper)
8+
by default, or [Parakeet v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) or
9+
[Qwen3-ASR](https://github.com/QwenLM/Qwen3-ASR) (see [Speech engines](#speech-engines)).
810
Audio never leaves your machine.
911
* **Sentence-aware**: voice-activity detection (Silero VAD) cuts audio at the speaker's
10-
pauses, not every N seconds, so Whisper and the translator see whole sentences.
12+
pauses, not every N seconds, so the recognizer and the translator see whole sentences.
1113
* **Translation** (OpenAI or Google) and **reply suggestions** (OpenAI) are opt-in.
1214
Only text is sent, and only once you enable them.
1315
* **Hidden from screen sharing on Windows 10 2004+**: the window stays visible to you
@@ -29,6 +31,23 @@ uv run polyglot # the real thing
2931

3032
On first real run Whisper downloads its model (`base` is about 150 MB).
3133

34+
### Speech engines
35+
36+
Settings → Speech → Speech engine picks the local recognizer (`asr.engine` in
37+
`config.toml`). All three run without an NVIDIA GPU; install the optional ones first:
38+
39+
| Engine | Install | Languages | Runs on | Notes |
40+
| --- | --- | --- | --- | --- |
41+
| Whisper (default) | included | 99 | CPU, CUDA | Model size under "Whisper model". |
42+
| Parakeet v3 | `uv sync --extra parakeet` | 25 European (incl. French) | CPU (int8), CUDA, DirectML | Fastest and most accurate on a CPU. ~670 MB download. Does not report the language: set "Their language" so translation and suggestions know it. |
43+
| Qwen3-ASR | `uv sync --extra qwen3` | 30, plus Chinese dialects | CPU (float32), CUDA | 0.6B suits a CPU; 1.7B wants a GPU. Pulls in PyTorch (~1 GB). Detects the language. |
44+
45+
Both extras together: `uv sync --extra parakeet --extra qwen3`. The model downloads on the
46+
first run with that engine. "Run on: Automatic" uses a GPU when the engine's runtime can
47+
use one and the CPU otherwise. For Parakeet on an AMD or Intel GPU, replace ONNX Runtime
48+
with its DirectML build (`uv pip uninstall onnxruntime && uv pip install
49+
onnxruntime-directml`, redone after each `uv sync`); Automatic then picks it up.
50+
3251
### Hotkeys (system-wide)
3352

3453
| Action | Default |
@@ -130,7 +149,7 @@ uv run polyglot --headless # listen live, print to the termi
130149
```bash
131150
uv sync # dev tools included
132151
uv run pytest # fast suite: no models, devices or network
133-
uv run pytest -m slow # real Whisper tiny + Silero (needs espeak-ng)
152+
uv run pytest -m slow # real Whisper tiny + Silero (needs espeak-ng); Parakeet/Qwen3 too if installed
134153
uv run ruff check . && uv run ruff format .
135154
uv run mypy
136155
```

‎config.example.toml‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,10 +19,17 @@ min_utterance_ms = 300
1919
partial_interval_ms = 0
2020

2121
[asr]
22-
model = "base"
23-
device = "auto"
22+
engine = "whisper" # whisper | parakeet | qwen3 (parakeet/qwen3: uv sync --extra <name>)
23+
device = "auto" # auto | cpu | cuda (auto: a GPU the engine can use, else the CPU)
24+
# Whisper
25+
model = "base" # tiny | base | small | medium | large-v3 | large-v3-turbo
2426
compute_type = "auto" # auto | int8 | float16 | float32 | ... (auto: int8 on CPU, float16 on GPU)
2527
beam_size = 5
28+
# Parakeet v3: 25 European languages, fast on a CPU (ONNX Runtime)
29+
parakeet_model = "nemo-parakeet-tdt-0.6b-v3"
30+
parakeet_quantization = "int8" # int8 | none
31+
# Qwen3-ASR: 30 languages (PyTorch). 0.6B suits a CPU; 1.7B is more accurate but wants a GPU.
32+
qwen3_model = "Qwen/Qwen3-ASR-0.6B"
2633
lock_after = 2
2734
lock_min_probability = 0.8
2835
queue_size = 8

‎docs/ARCHITECTURE.md‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ src/polyglot/
5151
adapters/
5252
audio/ wasapi_loopback, sounddevice_source, playback (WAV/array), resample (soxr)
5353
vad/ silero (onnxruntime), energy
54-
asr/ faster_whisper
54+
asr/ faster_whisper, parakeet (onnx-asr), qwen3 (qwen-asr); picked by asr.engine
5555
translate/ openai, google
5656
suggest/ openai (JSON-schema structured output)
5757
http.py stdlib HTTPS client + status -> typed error mapping
@@ -246,3 +246,5 @@ skipped. Whenever the window is not hidden, the footer says "Visible in screen s
246246
| Suggest requests supersede each other (generation counter) | One request at a time; the newest press wins |
247247
| OpenAI model per feature, from a built-in list plus the account's live list (`adapters/openai_models.py`) | Model lineups change faster than releases; the dialog gets the list through an injected catalog, so the UI never imports an adapter |
248248
| Request options chosen per model family, with a one-step fallback on HTTP 400 naming a parameter | Older models take `temperature`; reasoning models reject it and take `reasoning_effort`, whose accepted values differ by family |
249+
| Three local ASR engines behind the one `Transcriber` port; Parakeet and Qwen3 are optional extras, imported only when their model loads | Whisper covers the most languages; Parakeet (ONNX Runtime) is faster and more accurate on a CPU for European languages; Qwen3-ASR covers more languages than Parakeet. None needs NVIDIA hardware, and the default install stays small |
250+
| Parakeet reports the requested language back; Qwen3's reported language counts as confident (it gives no score) | Parakeet has no language output; the lock rule (`asr.lock_after` agreeing utterances) still guards Qwen3's guesses |

‎pyproject.toml‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,12 @@ dependencies = [
2121
"pynput>=1.7",
2222
]
2323

24+
[project.optional-dependencies]
25+
# Alternative speech engines (asr.engine). Whisper stays installed either way: Silero VAD
26+
# ships inside faster-whisper. Neither engine needs NVIDIA hardware.
27+
parakeet = ["onnx-asr[hub]>=0.12"] # ONNX Runtime: any CPU; DirectML/CUDA GPUs if installed
28+
qwen3 = ["qwen-asr>=0.0.6"] # PyTorch: CPU, or a CUDA GPU
29+
2430
[project.scripts]
2531
polyglot = "polyglot.__main__:main"
2632

@@ -51,7 +57,7 @@ pythonpath = ["src"]
5157
addopts = "-ra --strict-markers -m 'not slow'"
5258
timeout = 60
5359
markers = [
54-
"slow: needs real models (Whisper/Silero); deselected by default, run with -m slow",
60+
"slow: needs real models (Whisper/Silero, Parakeet/Qwen3 if installed); deselected by default, run with -m slow",
5561
"ui: needs Qt (runs offscreen)",
5662
]
5763
qt_api = "pyqt6"
Lines changed: 141 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,141 @@
1+
"""Local speech recognition with NVIDIA Parakeet TDT v3, run through ONNX Runtime.
2+
3+
``onnx-asr`` runs the exported model on ONNX Runtime, so no NVIDIA hardware or NeMo/PyTorch
4+
is needed: it runs on any CPU (int8 by default), on CUDA when ``onnxruntime-gpu`` is
5+
installed, and on any DirectX 12 GPU (AMD, Intel) when ``onnxruntime-directml`` is.
6+
7+
Parakeet v3 covers 25 European languages and picks the language itself. It does not
8+
report which one it heard, so the transcript carries the language the pipeline asked
9+
for (the locked or configured one), or none.
10+
"""
11+
12+
from __future__ import annotations
13+
14+
import logging
15+
from typing import Any
16+
17+
import numpy as np
18+
19+
from polyglot.core.errors import ConfigError, TranscriptionFailed
20+
from polyglot.core.models import SAMPLE_RATE, Transcript
21+
from polyglot.core.ports import RawAudio
22+
23+
log = logging.getLogger(__name__)
24+
25+
DEFAULT_MODEL = "nemo-parakeet-tdt-0.6b-v3"
26+
27+
#: ISO 639-1 codes of the 25 languages Parakeet TDT 0.6B v3 transcribes.
28+
LANGUAGES = frozenset(
29+
[
30+
"bg",
31+
"hr",
32+
"cs",
33+
"da",
34+
"nl",
35+
"en",
36+
"et",
37+
"fi",
38+
"fr",
39+
"de",
40+
"el",
41+
"hu",
42+
"it",
43+
"lv",
44+
"lt",
45+
"mt",
46+
"pl",
47+
"pt",
48+
"ro",
49+
"sk",
50+
"sl",
51+
"es",
52+
"sv",
53+
"ru",
54+
"uk",
55+
]
56+
)
57+
58+
_CPU, _CUDA, _DML = "CPUExecutionProvider", "CUDAExecutionProvider", "DmlExecutionProvider"
59+
60+
61+
def choose_providers(device: str, available: list[str]) -> list[str]:
62+
"""ONNX Runtime providers for a ``device`` setting, fastest first, CPU always last.
63+
64+
``auto`` uses a GPU provider when the installed onnxruntime build has one (CUDA, then
65+
DirectML). ``cuda`` insists on CUDA, so a missing build is an error, not a slow
66+
surprise."""
67+
if device == "cpu":
68+
return [_CPU]
69+
if device == "cuda":
70+
if _CUDA not in available:
71+
raise ConfigError(
72+
"Parakeet on CUDA needs the onnxruntime-gpu package; "
73+
"choose Automatic or CPU instead"
74+
)
75+
return [_CUDA, _CPU]
76+
gpu = [p for p in (_CUDA, _DML) if p in available]
77+
return [*gpu[:1], _CPU]
78+
79+
80+
class ParakeetTranscriber:
81+
def __init__(
82+
self,
83+
model: str = DEFAULT_MODEL,
84+
*,
85+
device: str = "auto",
86+
quantization: str | None = "int8",
87+
download_root: str | None = None,
88+
) -> None:
89+
self.model_name, self.device = model, device
90+
self.quantization = quantization or None
91+
self.download_root = download_root
92+
self._model: Any = None
93+
self.effective_device: str | None = None # the provider ONNX Runtime uses, after load()
94+
95+
def load(self) -> None:
96+
if self._model is not None:
97+
return
98+
try:
99+
import onnx_asr # lazy: optional dependency (uv sync --extra parakeet)
100+
import onnxruntime
101+
except ImportError as exc:
102+
raise ConfigError(
103+
"Parakeet needs the onnx-asr package (uv sync --extra parakeet)"
104+
) from exc
105+
providers = choose_providers(self.device, list(onnxruntime.get_available_providers()))
106+
log.info(
107+
"loading Parakeet %s (%s) with %s",
108+
self.model_name,
109+
self.quantization or "full precision",
110+
providers[0],
111+
)
112+
try:
113+
self._model = onnx_asr.load_model(
114+
self.model_name,
115+
self.download_root,
116+
quantization=self.quantization,
117+
providers=providers,
118+
)
119+
except Exception as exc:
120+
raise TranscriptionFailed(
121+
f"could not load Parakeet model {self.model_name!r}: {exc}"
122+
) from exc
123+
self.effective_device = providers[0].removesuffix("ExecutionProvider").lower()
124+
log.info("Parakeet %s loaded on %s", self.model_name, self.effective_device)
125+
126+
def transcribe(self, audio: RawAudio, language: str | None) -> Transcript:
127+
if self._model is None:
128+
raise TranscriptionFailed("model not loaded")
129+
try:
130+
result = self._model.recognize(
131+
np.ascontiguousarray(audio, dtype=np.float32), sample_rate=SAMPLE_RATE
132+
)
133+
except Exception as exc:
134+
raise TranscriptionFailed(f"transcription failed: {exc}") from exc
135+
text = str(getattr(result, "text", result) or "").strip()
136+
# No language detection to report: echo what the pipeline already decided.
137+
return Transcript(
138+
text=text,
139+
language=language,
140+
language_probability=1.0 if language else 0.0,
141+
)

‎src/polyglot/adapters/asr/qwen3.py‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
"""Local speech recognition with Qwen3-ASR (0.6B or 1.7B), via the ``qwen-asr`` package.
2+
3+
Runs on PyTorch: on a CUDA GPU when one is available, otherwise on the CPU in float32
4+
(no NVIDIA hardware needed; the 0.6B model is the practical size on a CPU). Qwen3-ASR
5+
covers 30 languages plus Chinese dialects, and reports the language it heard.
6+
"""
7+
8+
from __future__ import annotations
9+
10+
import logging
11+
from typing import Any
12+
13+
import numpy as np
14+
15+
from polyglot.core import languages
16+
from polyglot.core.errors import ConfigError, TranscriptionFailed
17+
from polyglot.core.models import SAMPLE_RATE, Transcript
18+
from polyglot.core.ports import RawAudio
19+
20+
log = logging.getLogger(__name__)
21+
22+
DEFAULT_MODEL = "Qwen/Qwen3-ASR-0.6B"
23+
MODELS = ("Qwen/Qwen3-ASR-0.6B", "Qwen/Qwen3-ASR-1.7B")
24+
25+
#: Language code -> the name Qwen3-ASR takes and reports. Its 30 languages.
26+
LANGUAGES: dict[str, str] = {
27+
"zh": "Chinese", "en": "English", "yue": "Cantonese", "ar": "Arabic", "de": "German",
28+
"fr": "French", "es": "Spanish", "pt": "Portuguese", "id": "Indonesian",
29+
"it": "Italian", "ko": "Korean", "ru": "Russian", "th": "Thai", "vi": "Vietnamese",
30+
"ja": "Japanese", "tr": "Turkish", "hi": "Hindi", "ms": "Malay", "nl": "Dutch",
31+
"sv": "Swedish", "da": "Danish", "fi": "Finnish", "pl": "Polish", "cs": "Czech",
32+
"tl": "Filipino", "fa": "Persian", "el": "Greek", "hu": "Hungarian",
33+
"mk": "Macedonian", "ro": "Romanian",
34+
} # fmt: skip
35+
_CODES = {name.lower(): code for code, name in LANGUAGES.items()}
36+
_CODES["fil"] = "tl" # Filipino's ISO 639-2 code, should the model report it
37+
38+
39+
def language_name(code: str | None) -> str | None:
40+
"""The name to force ``code`` with, or None (auto-detect) if Qwen3-ASR lacks it."""
41+
if not code:
42+
return None
43+
code = code.replace("_", "-").lower()
44+
if code in ("fil", "zh-yue", "zh-hk"):
45+
code = "tl" if code == "fil" else "yue"
46+
return LANGUAGES.get(code) or LANGUAGES.get(languages.primary(code) or "")
47+
48+
49+
def language_code(reported: str | None) -> str | None:
50+
"""Qwen3-ASR's reported language ("French", sometimes "fr") as a code, or None."""
51+
if not reported:
52+
return None
53+
key = reported.strip().lower()
54+
if key in LANGUAGES:
55+
return key
56+
return _CODES.get(key)
57+
58+
59+
class Qwen3Transcriber:
60+
def __init__(
61+
self,
62+
model: str = DEFAULT_MODEL,
63+
*,
64+
device: str = "auto",
65+
max_new_tokens: int = 256,
66+
) -> None:
67+
self.model_name, self.device, self.max_new_tokens = model, device, max_new_tokens
68+
self._model: Any = None
69+
self.effective_device: str | None = None # "cpu" or "cuda:0", after load()
70+
71+
def load(self) -> None:
72+
if self._model is not None:
73+
return
74+
try:
75+
import torch # lazy: optional dependency (uv sync --extra qwen3)
76+
from qwen_asr import Qwen3ASRModel
77+
except ImportError as exc:
78+
raise ConfigError(
79+
"Qwen3-ASR needs the qwen-asr package (uv sync --extra qwen3)"
80+
) from exc
81+
has_cuda = bool(torch.cuda.is_available())
82+
if self.device == "cuda" and not has_cuda:
83+
raise ConfigError("no CUDA GPU is available to PyTorch; choose Automatic or CPU")
84+
on_gpu = has_cuda and self.device != "cpu"
85+
device_map = "cuda:0" if on_gpu else "cpu"
86+
# bfloat16 halves GPU memory; on a CPU it is slow or unsupported, so use float32.
87+
dtype = torch.bfloat16 if on_gpu else torch.float32
88+
log.info("loading Qwen3-ASR %s on %s", self.model_name, device_map)
89+
try:
90+
self._model = Qwen3ASRModel.from_pretrained(
91+
self.model_name,
92+
dtype=dtype,
93+
device_map=device_map,
94+
max_inference_batch_size=1, # one utterance at a time
95+
max_new_tokens=self.max_new_tokens,
96+
)
97+
except Exception as exc:
98+
raise TranscriptionFailed(
99+
f"could not load Qwen3-ASR model {self.model_name!r}: {exc}"
100+
) from exc
101+
self.effective_device = device_map
102+
log.info("Qwen3-ASR %s loaded on %s", self.model_name, device_map)
103+
104+
def transcribe(self, audio: RawAudio, language: str | None) -> Transcript:
105+
if self._model is None:
106+
raise TranscriptionFailed("model not loaded")
107+
forced = language_name(language)
108+
try:
109+
results = self._model.transcribe(
110+
audio=(np.ascontiguousarray(audio, dtype=np.float32), SAMPLE_RATE),
111+
language=forced,
112+
)
113+
result = results[0] if isinstance(results, (list, tuple)) else results
114+
except Exception as exc:
115+
raise TranscriptionFailed(f"transcription failed: {exc}") from exc
116+
text = str(getattr(result, "text", "") or "").strip()
117+
code = language_code(getattr(result, "language", None)) if text else None
118+
if forced and code is None:
119+
code = language # forced: the model did not need to say
120+
# Qwen3-ASR gives no confidence score. A reported language counts as confident;
121+
# the lock still waits for ``asr.lock_after`` utterances that agree.
122+
return Transcript(text=text, language=code, language_probability=1.0 if code else 0.0)

0 commit comments

Comments
 (0)