Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 103 additions & 0 deletions .github/workflows/agent-voice.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
name: agent-voice

# agent-voice is a standalone package (own bun.lock, not a workspace member), so
# it gets its own workflow instead of riding along with the monorepo CI.
#
# The Rust leg runs `cargo check`/`clippy`/`test` rather than a full Tauri
# bundle: building the app needs WebView2, CMake and LLVM, which are slower and
# unavailable on hosted runners. `cargo check --tests` still compiles the
# `tauri::generate_context!` macro, so configuration errors (capabilities,
# window labels, updater key) fail here.

on:
push:
branches: [main]
paths:
- 'Apps/agent-voice/**'
- '.github/workflows/agent-voice.yml'
pull_request:
paths:
- 'Apps/agent-voice/**'
- '.github/workflows/agent-voice.yml'

concurrency:
group: agent-voice-${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

defaults:
run:
working-directory: Apps/agent-voice

jobs:
typescript:
name: TypeScript (app + worker)
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v6

- uses: oven-sh/setup-bun@v2
with:
bun-version-file: .bun-version

- name: Install dependencies
run: bun install --frozen-lockfile

- name: Typecheck
run: bun run typecheck

- name: Test (frontend helpers + license worker)
run: bun test

- name: Build frontend
run: bun run build

rust:
name: Rust core
runs-on: ${{ matrix.os }}
timeout-minutes: 45
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
steps:
- uses: actions/checkout@v6

- uses: oven-sh/setup-bun@v2
with:
bun-version-file: .bun-version

- uses: dtolnay/rust-toolchain@stable
with:
components: rustfmt, clippy

- name: Install Linux build dependencies
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libwebkit2gtk-4.1-dev libappindicator3-dev \
librsvg2-dev patchelf libxdo-dev cmake clang libssl-dev

- name: Cache cargo
uses: Swatinem/rust-cache@v2
with:
workspaces: 'Apps/agent-voice/src-tauri -> target'

- name: Install frontend dependencies
run: bun install --frozen-lockfile

- name: Frontend build (tauri.conf.json points at ../dist)
working-directory: Apps/agent-voice
run: bun run build

- name: Formatting
working-directory: Apps/agent-voice/src-tauri
run: cargo fmt --check

- name: Clippy
working-directory: Apps/agent-voice/src-tauri
run: cargo clippy --all-targets -- -D warnings

- name: Tests
working-directory: Apps/agent-voice/src-tauri
run: cargo test
19 changes: 19 additions & 0 deletions Apps/agent-voice/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Paddle checkout overlay. Copy to `.env` and fill in — `.env` is gitignored.
#
# Client-side token from Paddle → Developer tools → Authentication. It is safe
# to ship in the bundle; it only authorises opening checkout.
VITE_PADDLE_TOKEN=

# Price id of the one-time Pro price (Paddle → Catalog → product → Prices).
VITE_PADDLE_PRICE_ID=

# `sandbox` (default) or `production`.
VITE_PADDLE_ENV=sandbox

# Build-time Rust knobs. These are *not* read from .env — pass them to cargo:
#
# AGENT_VOICE_WORKER_URL=https://licenses.example.com \
# AGENT_VOICE_ENTITLEMENT_PUBKEY="$(base64 -w0 entitlement.pub)" \
# cargo tauri build
#
# See docs/entitlement.md before rotating either one.
31 changes: 31 additions & 0 deletions Apps/agent-voice/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Logs
logs
*.log
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
lerna-debug.log*

node_modules
dist
dist-ssr
*.local

# Editor directories and files
.vscode/*
!.vscode/extensions.json
.idea
.DS_Store
*.suo
*.ntvs*
*.njsproj
*.sln
*.sw?

# Rust build output and generated Tauri schemas
src-tauri/target
src-tauri/gen/schemas

# Wrangler state
worker/.wrangler
3 changes: 3 additions & 0 deletions Apps/agent-voice/.vscode/extensions.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"recommendations": ["tauri-apps.tauri-vscode", "rust-lang.rust-analyzer"]
}
122 changes: 122 additions & 0 deletions Apps/agent-voice/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# agent-voice

Hold a hotkey. Speak. The text lands in whatever you were typing — locally,
with no audio ever leaving the machine.

Built for people who dictate to coding agents: hold `Ctrl/Cmd+Shift+Space`,
say the sentence, release, and it is pasted into the terminal, editor or chat
box that had focus. Transcription runs on whisper.cpp inside the app, so it
works on a plane, on an offline machine, and with source code you cannot send
to anyone.

## What it does

- **Push to talk** — global hotkey (remappable) with an always-on-top overlay
that shows the microphone level.
- **Local transcription** — whisper.cpp via `whisper-rs`. Models download once
and live in the app-data directory.
- **Pastes into the right window** — focus is captured before recording starts
and restored before the paste keystrokes.
- **Snippets and vocabulary** — say "new line", get a newline; teach it that
your project is "Khan" not "can".
- **AI cleanup (Pro)** — per-app modes that tidy the raw transcript using your
own API key or a local Ollama model.
- **Agents can ask you (Pro)** — an MCP server exposes `ask_user_by_voice`, so
Claude Code or Cursor can put a question on your overlay and read your spoken
answer back.

## Tiers

| | Free | Pro ($59 once) |
| --------------------------- | -------------------------- | --------------------------- |
| Minutes per day | 30, resets at UTC midnight | Unlimited |
| Models | Whisper base, small | + large-v3-turbo |
| Modes, snippets, vocabulary | Snippets and vocabulary | All, incl. AI cleanup modes |
| `ask_user_by_voice` (MCP) | — | Yes |

The free tier is not a trial: there is no expiry and no account.

## Install

```bash
bun install
bun run tauri dev # development
bun run tauri build # NSIS + MSI on Windows, DMG on macOS, AppImage/deb on Linux
```

On Windows the first build needs the [MSVC build tools](https://visualstudio.microsoft.com/downloads/#build-tools-for-visual-studio-2022),
and `whisper-rs` compiles with CMake. `clang` must be on `LIBCLANG_PATH` for
bindgen on every platform.

## Using it

1. Launch the app and hold the hotkey (default `Ctrl/Cmd+Shift+Space`).
2. The overlay appears at the bottom of the screen; speak.
3. Release. The transcript is pasted into the app you were in.

Pick a model under **Settings → Model**. Whisper base is a ~140 MB download;
large-v3-turbo is ~1.6 GB and noticeably better at code and proper nouns.

## Settings worth knowing

- **Hotkey** — any accelerator the global-shortcut plugin accepts. The new
combo is registered before the old one is dropped, so a bad combo never
leaves you without a working hotkey.
- **Language** — leave on auto-detect unless you switch between languages often.
- **Snippets / vocabulary** — matched on whole words, case-insensitively for
snippets. A trigger such as `c++` works too.

## MCP integration (Pro)

`agent-voice mcp` is a stdio MCP server that exposes one tool:

```
ask_user_by_voice({ prompt, timeout_ms? })
```

It talks to the running GUI over a loopback HTTP bridge (the port is published
in `bridge.json` next to the app data), so the app must be running. Point any
MCP client at the binary:

```jsonc
{
"mcpServers": {
"agent-voice": {
"command": "/path/to/agent-voice",
"args": ["mcp"],
},
},
}
```

Without a Pro license the server starts but advertises no tools, so a
misconfigured agent fails loudly instead of silently doing nothing.

## Privacy

Audio is captured only while the hotkey is held, written nowhere, and
transcribed in-process. The only network traffic the app generates is:

- downloading a model the first time you select it (from Hugging Face),
- `POST /activate` when you enter a license code,
- checking the update feed.

Telemetry is not collected.

## Development

```bash
bun run typecheck # app + worker
bun test # frontend helpers + worker (57 tests)
bun run build # frontend bundle
cd src-tauri && cargo test # Rust unit tests
```

The license backend is a Cloudflare Worker in [`worker/`](worker/README.md);
the token format and key rotation are documented in
[`docs/entitlement.md`](docs/entitlement.md).

## License

Source under the repository license. Whisper models are downloaded from
Hugging Face under their own licenses (MIT for most, see the model cards).
Loading
Loading