Polyglot v1 is a pipeline of single-purpose stages around a pure-Python core. Imports point inward only, there is one thread bridge to the UI, and only the coordinator writes conversation state. This document describes what is built. The design rationale and the critique of v0 that motivated it are in the "Polyglot v1 — Architecture Design" doc.
(Sources: v1-layers.mmd, v1-pipeline.mmd; render with mmdc -i <file> -o <png>.)
| Layer | Package | May import | Knows about |
|---|---|---|---|
| Core | polyglot.core |
stdlib, numpy | nothing outside itself |
| Adapters | polyglot.adapters |
core + its own library (lazily) | one technology each |
| Platform | polyglot.platform |
core, ctypes | the OS |
| Config | polyglot.config |
core, pydantic, platformdirs | settings and secrets |
| Presentation | polyglot.ui |
core (presenter is Qt-free; ui.qt adds PyQt6) |
view model, widgets |
| Composition | wiring, controller, app, __main__ |
everything | which concrete class goes where |
The dependency rule is a test. tests/unit/test_architecture.py parses every module's
imports and fails the build if the core imports anything but the standard library and
numpy, if an adapter or the platform layer imports the UI, or if the UI reaches into
adapters. OS checks (sys.platform ==) outside polyglot.platform also fail it.
src/polyglot/
__main__.py CLI: GUI, --demo, --headless, --file, --list-devices
app.py Qt composition root: window, presenter, bridge, tray, hotkeys
wiring.py config -> adapters -> Pipeline (Qt-free); build_demo() for fakes
controller.py pipeline lifecycle; applies settings; rebuilds keeping the conversation
config.py AppConfig (pydantic), TOML load/save, env overrides, Secrets
logging_setup.py rotating file + stderr; no transcript text by default
selfcheck.py --self-check: loads every native dependency (used on packaged builds)
core/
models.py Utterance, Speaker, SpeechSegment, Transcript, Suggestion, ReplyOption, IdSource
errors.py ProviderTimeout, ProviderUnavailable, QuotaExceeded, AuthError, ...
events.py UtteranceUpdated, SuggestionsReady, PipelineStatus, StageError, EventBus
ports.py AudioSource, Resampler, VoiceActivityModel, Transcriber, Translator,
Suggester, WindowShield, SecretStore
ringbuffer.py preallocated SPSC audio ring buffer with overrun accounting
segmenter.py VAD endpointing state machine (pure)
session.py conversation state + language lock policy
profile.py UserProfile: public details the user shares with the suggester
context.py ContextBuilder: Session + UserProfile -> SuggestionContext
languages.py code <-> name
pipeline.py FrontEnd, AsrQueue, AsrWorker, Coordinator, Pipeline
adapters/
audio/ wasapi_loopback, sounddevice_source, playback (WAV/array), resample (soxr)
vad/ silero (onnxruntime), energy
asr/ faster_whisper, parakeet (onnx-asr), qwen3 (qwen-asr); picked by asr.engine
translate/ openai, google
suggest/ openai (JSON-schema structured output)
http.py stdlib HTTPS client + status -> typed error mapping
secrets.py keyring store, memory store
fakes.py a working fake for every port
dump.py optional per-utterance WAV dumps
platform/ WindowsShield (SetWindowDisplayAffinity), UnsupportedShield
ui/
presenter.py Presenter, RowVM, SuggestionVM, StatusVM, ViewPort, Commands
qt/ overlay, settings_dialog, bridge, shield_filter, hotkeys, theme.qss
Utterance is the unit of work: one continuous stretch of the other person's speech.
It is a frozen dataclass. Every change makes a new copy, so a snapshot handed to
another thread can never change underneath it.
| Status | Meaning |
|---|---|
LISTENING |
speech started, no text yet |
PARTIAL |
provisional text (only when partials are enabled) |
FINAL |
endpoint reached, text fixed; error set if translation failed |
TRANSLATED |
translation attached |
FAILED |
transcription failed or the utterance was dropped under overload |
Utterance IDs come from one monotonic IdSource and define spoken order. Results are
placed by ID, never by arrival time.
Every utterance has a Speaker: OTHER (the person being listened to) or USER. Only
OTHER utterances are ever replied to; USER turns (from the user's own microphone, see
"The user's microphone" below) are context for the next suggestion.
ContextBuilder (core, pure, immutable) is the one place that decides what the suggester
sees: the target utterance, a window of earlier utterances from both speakers (capped by
count, suggestions.history, and by length, suggestions.history_chars; cut from the
oldest end, the newest is always kept), the languages, the conversation type and the user's
UserProfile. The coordinator owns the builder and the Session, so state stays
single-writer; a Suggester only renders the context it is given. A rolling summary
would slot in here as one more derived input.
UserProfile is an allowlist of public details (name, role, location, summary, goals,
reply tone, languages, skills, experience, education, hobbies, interests, talking points).
It lives in the [profile] section of config.toml, so it persists, and a change is
applied to the running pipeline without a rebuild. A unit test pins the field list.
| Thread | Job | Must never |
|---|---|---|
| audio library callback | downmix and copy the block into the ring buffer | allocate per sample, log, touch disk |
polyglot-frontend |
resample to 16 kHz, cut 32 ms frames, VAD, segmenter, route | transcribe |
polyglot-frontend-user (only with the microphone on) |
the same, for the user's microphone | transcribe |
polyglot-asr (exactly one) |
load the model, transcribe segments in ID order | run in parallel with itself |
polyglot-net-* (4) |
translation and suggestion calls with timeouts | write state |
polyglot-coord |
own Session; apply results; dispatch; publish events |
block on I/O |
| Qt main thread | receive events through EventBridge; presenter; view |
(not part of the pipeline) |
| Rule | Default | Config |
|---|---|---|
| start: probability ≥ start threshold for N frames | 0.5 × 3 frames (~100 ms) | vad.start_threshold, vad.start_frames |
| pre-roll kept before the start | 300 ms | vad.pre_roll_ms |
| end: probability < end threshold for this long | 0.35 for 600 ms | vad.end_threshold, vad.end_silence_ms |
| force-cut at the quietest frame of the last 2 s | 15 s | vad.max_utterance_s |
| discard if less speech than | 300 ms | vad.min_utterance_ms |
| partial transcripts every | off | vad.partial_interval_ms |
Gap fill. WASAPI loopback delivers no data at all while nothing is playing. Without
help, an utterance whose speaker stops talking would never see its 600 ms of silence. The
front end therefore feeds real-time-equivalent silence while an utterance is open and no
audio has arrived for 250 ms (audio.gap_fill).
With audio.user_mic = true the pipeline runs a second, independent audio stream tagged
Speaker.USER: its own source (SoundDeviceSource), ring buffer, resampler, VAD instance
(VADs keep state) and segmenter, in a second front-end thread. Everything after the
segmenter is shared:
- One
IdSource(locked), so IDs still define spoken order across both streams. - One
AsrQueueand the one ASR worker; the segment carries itsspeakerthrough to theUtterance. Both speakers are transcribed with the same language hint (the other person's locked language), because the user is replying in it. - One coordinator and
Session. User utterances never move the language lock, are not translated, are never the target of a suggestion, and do not mark a suggestion stale. They do appear in the history the suggester sees, and in the window as "You" rows. - A microphone that fails (at start or later) is reported as a warning
(
StageError("audio", ...), "microphone: ...") and listening to the other person continues. The mic is skipped, with a warning, when the main input is the same device, and for replayed files.
Settings has a microphone test (ui/mic_test.py, Qt-free): record 5 s from the chosen
device on a worker thread, judge the level (AudioClip.level()), play it back. Recording and
playback come from a MicTester injected by the composition root (SoundDeviceMicTester),
so the UI never imports an adapter. It refuses to run while listening, because the playback
would be captured as the other person.
Known limitation: with speakers instead of headphones the microphone hears the other person too, and those lines would be tagged as the user's. There is no echo cancellation or cross-stream gating yet.
- Every result carries its utterance ID; the coordinator places it by ID.
- Nothing is dropped for being late. A late translation fills its own row.
- Suggestions run only on the hotkey, for the latest final utterance by the other person, with the last 6 final utterances (either speaker, within a length budget) and the user profile as context. A reply stays linked to the sentence it answers. If newer sentences arrive first, the presenter marks it "newer sentences since". Pressing the hotkey again supersedes an in-flight request: its result is ignored, since the newer request wins.
- Older partials are skipped (a newer partial or the final covers the same audio).
- Ring buffer: 30 s; on overrun the oldest audio is overwritten, counted, and a
FALLING_BEHINDstatus is published. - ASR queue: at most 8 waiting finals. When full, the oldest is dropped, marked
FAILED(overloaded), andFALLING_BEHINDis published. Latency stays bounded instead of growing for the whole session. - Network: hard timeouts (translation 3 s, suggestions 6 s) and one retry with jitter for retryable errors.
Pipeline.start/stop control listening (source, front end, ASR worker). The coordinator
and session outlive them, so suggestions and language commands keep working while
stopped. stop finishes and transcribes the utterance in progress. When a settings change
needs a new pipeline, the controller passes the old session and ids to the new one,
so the conversation continues and IDs keep increasing.
Model–View–Presenter with a passive view:
EventBridgeis the only thread bridge. It subscribes to the bus and re-emits each event as a queued Qt signal, so delivery happens on the GUI thread.Presenter(pure Python, unit-tested) keeps the view model, formats it (language names, error wording, "translating…", "Replying to: …") and turns clicks and hotkeys intoCommands.OverlayWindowdraws rows by ID, the suggestion panel, a status strip and the capture indicator. It emits intent signals and holds no logic beyond view state:FollowLock(pure,ui/follow.py) decides whether the transcript follows the newest sentence. It starts locked, unlocks when the user scrolls, and re-locks only on the lock button or hotkey. The window scrolls when the scroll range changes, i.e. after the layout has measured the new text.
| Concern | Windows | macOS | Linux |
|---|---|---|---|
| System audio | WASAPI loopback (PyAudioWPatch), follows the default output | BlackHole via SoundDeviceSource |
any input device |
| Hide from screen capture | SetWindowDisplayAffinity(WDA_EXCLUDEFROMCAPTURE) on every top-level window, on show and on handle change, read back with GetWindowDisplayAffinity |
not attempted (ScreenCaptureKit ignores NSWindow.sharingType on macOS 15+) |
not attempted |
On Windows builds before 19041 the same call would put a black box in the share, so it is skipped. Whenever the window is not hidden, the footer says "Visible in screen share".
- Ports raise typed errors (
core/errors.py). Failures show as failures (row note, status strip) and are never replaced with made-up text. - No modal dialogs mid-conversation; the only modal dialog is the quit confirmation while listening.
- Logs: module loggers, rotating file in the user log directory. INFO carries IDs and
timings (
utt=3 final asr_ms=412), never transcript text. - Translation and suggestions are off until enabled. Only text is sent to providers. Keys live in the OS keyring. Audio dumps are off by default and go to the user data directory.
| Suite | What | Needs |
|---|---|---|
tests/unit |
segmenter rules, ring buffer, session, events, AsrQueue, config, presenter, controller, architecture rule | nothing |
tests/pipeline |
the threaded pipeline end to end with fakes: pairing under out-of-order completion, single ASR worker, language lock, retries, overload, suggestions (supersede, late, failure), clear, stop/restart, gap fill, resampled stereo input, error reporting | nothing |
tests/adapters |
HTTP error mapping, OpenAI/Google/suggester request and response handling, WAV I/O, playback, resampling, VAD, Windows shield (fake user32), keyring store | soxr/Silero tests skip if absent |
tests/ui |
overlay rows and panels, bridge delivers on the GUI thread, shield filter covers dialogs, settings dialog, demo-mode app smoke test; real shield on Windows | PyQt6 + pytest-qt (offscreen) |
tests/integration (-m slow) |
Silero and Whisper tiny on espeak-synthesized speech, end to end |
faster-whisper, espeak-ng |
| Decision | Why |
|---|---|
HTTP via the standard library behind an HttpClient interface, not the OpenAI SDK or httpx |
Two REST calls do not justify the SDK's dependency tree; tests inject a fake transport |
| Config via plain pydantic models plus a small TOML writer, not pydantic-settings / tomli-w | The precedence rules are ~30 lines; fewer dependencies |
Resampling behind a Resampler port, built per source rate |
Keeps the core numpy-only and lets tests resample without soxr |
| Silero loaded from faster-whisper's bundled model, both v4 and v5 interfaces detected | No extra download; falls back to the energy VAD with a visible warning if it cannot load |
| Coordinator outlives start/stop | Suggestions and language commands work while not listening |
| Gap fill in the front end | Loopback delivers nothing during silence; utterances must still end |
| Suggest requests supersede each other (generation counter) | One request at a time; the newest press wins |
OpenAI model per feature, from a built-in list plus the account's live list (adapters/openai_models.py) |
Model lineups change faster than releases; the dialog gets the list through an injected catalog, so the UI never imports an adapter |
| Request options chosen per model family, with a one-step fallback on HTTP 400 naming a parameter | Older models take temperature; reasoning models reject it and take reasoning_effort, whose accepted values differ by family |
Three local ASR engines behind the one Transcriber port; Parakeet and Qwen3 are optional extras, imported only when their model loads |
Whisper covers the most languages; Parakeet (ONNX Runtime) is faster and more accurate on a CPU for European languages; Qwen3-ASR covers more languages than Parakeet. None needs NVIDIA hardware, and the default install stays small |
| Parakeet reports the requested language back; Qwen3's reported language counts as confident (it gives no score) | Parakeet has no language output; the lock rule (asr.lock_after agreeing utterances) still guards Qwen3's guesses |

