Skip to content
Merged
37 changes: 32 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@ A small always-on-top window that listens to the other side of a call, transcrib
locally, translates it into your language, and suggests replies in theirs when you
press a hotkey.

* **Local speech recognition** with [faster-whisper](https://github.com/SYSTRAN/faster-whisper).
* **Local speech recognition**: [faster-whisper](https://github.com/SYSTRAN/faster-whisper)
by default, or [Parakeet v3](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) or
[Qwen3-ASR](https://github.com/QwenLM/Qwen3-ASR) (see [Speech engines](#speech-engines)).
Audio never leaves your machine.
* **Sentence-aware**: voice-activity detection (Silero VAD) cuts audio at the speaker's
pauses, not every N seconds, so Whisper and the translator see whole sentences.
pauses, not every N seconds, so the recognizer and the translator see whole sentences.
* **Translation** (OpenAI or Google) and **reply suggestions** (OpenAI) are opt-in.
Only text is sent, and only once you enable them.
* **Hidden from screen sharing on Windows 10 2004+**: the window stays visible to you
Expand All @@ -29,6 +31,23 @@ uv run polyglot # the real thing

On first real run Whisper downloads its model (`base` is about 150 MB).

### Speech engines

Settings → Speech → Speech engine picks the local recognizer (`asr.engine` in
`config.toml`). All three run without an NVIDIA GPU; install the optional ones first:

| Engine | Install | Languages | Runs on | Notes |
| --- | --- | --- | --- | --- |
| Whisper (default) | included | 99 | CPU, CUDA | Model size under "Whisper model". |
| Parakeet v3 | `uv sync --extra parakeet` | 25 European (incl. French) | CPU (int8), CUDA, DirectML | Fastest and most accurate on a CPU. ~670 MB download. Does not report the language: set "Their language" so translation and suggestions know it. |
| Qwen3-ASR | `uv sync --extra qwen3` | 30, plus Chinese dialects | CPU (float32), CUDA | 0.6B suits a CPU; 1.7B wants a GPU. Pulls in PyTorch (~1 GB). Detects the language. |

Both extras together: `uv sync --extra parakeet --extra qwen3`. The model downloads on the
first run with that engine. "Run on: Automatic" uses a GPU when the engine's runtime can
use one and the CPU otherwise. For Parakeet on an AMD or Intel GPU, replace ONNX Runtime
with its DirectML build (`uv pip uninstall onnxruntime && uv pip install
onnxruntime-directml`, redone after each `uv sync`); Automatic then picks it up.

### Hotkeys (system-wide)

| Action | Default |
Expand Down Expand Up @@ -101,8 +120,16 @@ first: the playback would otherwise be picked up as the other person's audio.
Settings → About you lets suggestions talk about *you*: your name, role, skills, languages,
experience, hobbies and interests, how you want replies to sound, and what you want from the
conversation. Fill it in once; it is saved in `config.toml` (the `[profile]` section) and
used every time. Suggestions also see what was said earlier in the conversation, up to
`suggestions.history` utterances or `suggestions.history_chars` characters.
used every time. Suggestions also see the conversation around the sentence they answer:
up to `suggestions.history` utterances before it and, when a reply hotkey answers an
earlier sentence, up to `suggestions.history_after` after it, within
`suggestions.history_chars` characters in all (the nearest are kept first). The prompt
numbers the lines and marks the one to reply to.

Settings → Suggestions → Extra instructions adds your own rules to every request
(`suggestions.instructions`, up to 1000 characters), for example "Don't ask questions back
unless the conversation needs one." Like the profile, it is sent to OpenAI and applies to the
next suggestion without restarting listening.

Everything in your profile is sent to the AI provider with each suggestion request, so it only has
fields for things you would say out loud to the other person. There is deliberately no field
Expand Down Expand Up @@ -130,7 +157,7 @@ uv run polyglot --headless # listen live, print to the termi
```bash
uv sync # dev tools included
uv run pytest # fast suite: no models, devices or network
uv run pytest -m slow # real Whisper tiny + Silero (needs espeak-ng)
uv run pytest -m slow # real Whisper tiny + Silero (needs espeak-ng); Parakeet/Qwen3 too if installed
uv run ruff check . && uv run ruff format .
uv run mypy
```
Expand Down
17 changes: 14 additions & 3 deletions config.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,17 @@ min_utterance_ms = 300
partial_interval_ms = 0

[asr]
model = "base"
device = "auto"
engine = "whisper" # whisper | parakeet | qwen3 (parakeet/qwen3: uv sync --extra <name>)
device = "auto" # auto | cpu | cuda (auto: a GPU the engine can use, else the CPU)
# Whisper
model = "base" # tiny | base | small | medium | large-v3 | large-v3-turbo
compute_type = "auto" # auto | int8 | float16 | float32 | ... (auto: int8 on CPU, float16 on GPU)
beam_size = 5
# Parakeet v3: 25 European languages, fast on a CPU (ONNX Runtime)
parakeet_model = "nemo-parakeet-tdt-0.6b-v3"
parakeet_quantization = "int8" # int8 | none
# Qwen3-ASR: 30 languages (PyTorch). 0.6B suits a CPU; 1.7B is more accurate but wants a GPU.
qwen3_model = "Qwen/Qwen3-ASR-0.6B"
lock_after = 2
lock_min_probability = 0.8
queue_size = 8
Expand All @@ -40,8 +47,12 @@ model = "gpt-6-luna" # gpt-4o-mini and gpt-4.1-mini retire 2027-01-14 (replacem
timeout_s = 6.0
conversation_type = "meeting"
history = 6 # most utterances of earlier conversation to include
history_chars = 3000 # soft cap on their total length (the newest is always kept)
history_after = 4 # most utterances said after the sentence being answered (reply hotkeys)
history_chars = 3000 # soft cap on the context's total length (the nearest is always kept)
options = 2
# Extra instructions added to every suggestion request (up to 1000 characters), e.g.
# "Don't ask questions back unless the conversation needs one. Keep replies short."
instructions = ""

# What suggestions may say about you. Saved once, used every time. Sent to the AI provider with each
# suggestion request: public, shareable details only (no health, politics, religion, money).
Expand Down
4 changes: 3 additions & 1 deletion docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ src/polyglot/
adapters/
audio/ wasapi_loopback, sounddevice_source, playback (WAV/array), resample (soxr)
vad/ silero (onnxruntime), energy
asr/ faster_whisper
asr/ faster_whisper, parakeet (onnx-asr), qwen3 (qwen-asr); picked by asr.engine
translate/ openai, google
suggest/ openai (JSON-schema structured output)
http.py stdlib HTTPS client + status -> typed error mapping
Expand Down Expand Up @@ -246,3 +246,5 @@ skipped. Whenever the window is not hidden, the footer says "Visible in screen s
| Suggest requests supersede each other (generation counter) | One request at a time; the newest press wins |
| OpenAI model per feature, from a built-in list plus the account's live list (`adapters/openai_models.py`) | Model lineups change faster than releases; the dialog gets the list through an injected catalog, so the UI never imports an adapter |
| Request options chosen per model family, with a one-step fallback on HTTP 400 naming a parameter | Older models take `temperature`; reasoning models reject it and take `reasoning_effort`, whose accepted values differ by family |
| Three local ASR engines behind the one `Transcriber` port; Parakeet and Qwen3 are optional extras, imported only when their model loads | Whisper covers the most languages; Parakeet (ONNX Runtime) is faster and more accurate on a CPU for European languages; Qwen3-ASR covers more languages than Parakeet. None needs NVIDIA hardware, and the default install stays small |
| Parakeet reports the requested language back; Qwen3's reported language counts as confident (it gives no score) | Parakeet has no language output; the lock rule (`asr.lock_after` agreeing utterances) still guards Qwen3's guesses |
8 changes: 7 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ dependencies = [
"pynput>=1.7",
]

[project.optional-dependencies]
# Alternative speech engines (asr.engine). Whisper stays installed either way: Silero VAD
# ships inside faster-whisper. Neither engine needs NVIDIA hardware.
parakeet = ["onnx-asr[hub]>=0.12"] # ONNX Runtime: any CPU; DirectML/CUDA GPUs if installed
qwen3 = ["qwen-asr>=0.0.6"] # PyTorch: CPU, or a CUDA GPU

[project.scripts]
polyglot = "polyglot.__main__:main"

Expand Down Expand Up @@ -51,7 +57,7 @@ pythonpath = ["src"]
addopts = "-ra --strict-markers -m 'not slow'"
timeout = 60
markers = [
"slow: needs real models (Whisper/Silero); deselected by default, run with -m slow",
"slow: needs real models (Whisper/Silero, Parakeet/Qwen3 if installed); deselected by default, run with -m slow",
"ui: needs Qt (runs offscreen)",
]
qt_api = "pyqt6"
Expand Down
141 changes: 141 additions & 0 deletions src/polyglot/adapters/asr/parakeet.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
"""Local speech recognition with NVIDIA Parakeet TDT v3, run through ONNX Runtime.

``onnx-asr`` runs the exported model on ONNX Runtime, so no NVIDIA hardware or NeMo/PyTorch
is needed: it runs on any CPU (int8 by default), on CUDA when ``onnxruntime-gpu`` is
installed, and on any DirectX 12 GPU (AMD, Intel) when ``onnxruntime-directml`` is.

Parakeet v3 covers 25 European languages and picks the language itself. It does not
report which one it heard, so the transcript carries the language the pipeline asked
for (the locked or configured one), or none.
"""

from __future__ import annotations

import logging
from typing import Any

import numpy as np

from polyglot.core.errors import ConfigError, TranscriptionFailed
from polyglot.core.models import SAMPLE_RATE, Transcript
from polyglot.core.ports import RawAudio

log = logging.getLogger(__name__)

DEFAULT_MODEL = "nemo-parakeet-tdt-0.6b-v3"

#: ISO 639-1 codes of the 25 languages Parakeet TDT 0.6B v3 transcribes.
LANGUAGES = frozenset(
[
"bg",
"hr",
"cs",
"da",
"nl",
"en",
"et",
"fi",
"fr",
"de",
"el",
"hu",
"it",
"lv",
"lt",
"mt",
"pl",
"pt",
"ro",
"sk",
"sl",
"es",
"sv",
"ru",
"uk",
]
)

_CPU, _CUDA, _DML = "CPUExecutionProvider", "CUDAExecutionProvider", "DmlExecutionProvider"


def choose_providers(device: str, available: list[str]) -> list[str]:
"""ONNX Runtime providers for a ``device`` setting, fastest first, CPU always last.

``auto`` uses a GPU provider when the installed onnxruntime build has one (CUDA, then
DirectML). ``cuda`` insists on CUDA, so a missing build is an error, not a slow
surprise."""
if device == "cpu":
return [_CPU]
if device == "cuda":
if _CUDA not in available:
raise ConfigError(
"Parakeet on CUDA needs the onnxruntime-gpu package; "
"choose Automatic or CPU instead"
)
return [_CUDA, _CPU]
gpu = [p for p in (_CUDA, _DML) if p in available]
return [*gpu[:1], _CPU]


class ParakeetTranscriber:
def __init__(
self,
model: str = DEFAULT_MODEL,
*,
device: str = "auto",
quantization: str | None = "int8",
download_root: str | None = None,
) -> None:
self.model_name, self.device = model, device
self.quantization = quantization or None
self.download_root = download_root
self._model: Any = None
self.effective_device: str | None = None # the provider ONNX Runtime uses, after load()

def load(self) -> None:
if self._model is not None:
return
try:
import onnx_asr # lazy: optional dependency (uv sync --extra parakeet)
import onnxruntime
except ImportError as exc:
raise ConfigError(
"Parakeet needs the onnx-asr package (uv sync --extra parakeet)"
) from exc
providers = choose_providers(self.device, list(onnxruntime.get_available_providers()))
log.info(
"loading Parakeet %s (%s) with %s",
self.model_name,
self.quantization or "full precision",
providers[0],
)
try:
self._model = onnx_asr.load_model(
self.model_name,
self.download_root,
quantization=self.quantization,
providers=providers,
)
except Exception as exc:
raise TranscriptionFailed(
f"could not load Parakeet model {self.model_name!r}: {exc}"
) from exc
self.effective_device = providers[0].removesuffix("ExecutionProvider").lower()
log.info("Parakeet %s loaded on %s", self.model_name, self.effective_device)

def transcribe(self, audio: RawAudio, language: str | None) -> Transcript:
if self._model is None:
raise TranscriptionFailed("model not loaded")
try:
result = self._model.recognize(
np.ascontiguousarray(audio, dtype=np.float32), sample_rate=SAMPLE_RATE
)
except Exception as exc:
raise TranscriptionFailed(f"transcription failed: {exc}") from exc
text = str(getattr(result, "text", result) or "").strip()
# No language detection to report: echo what the pipeline already decided.
return Transcript(
text=text,
language=language,
language_probability=1.0 if language else 0.0,
)
Loading
Loading