diff --git a/package.json b/package.json index 7a6e52f..c734c2e 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "vox", - "version": "3.1.0", + "version": "3.1.1", "description": "Agent-facing speech to text and text to speech over Alibaba Model Studio.", "repository": "https://github.com/celados/vox", "bin": { diff --git a/src/index.md b/src/index.md index b6cad98..785ebc5 100644 --- a/src/index.md +++ b/src/index.md @@ -6,206 +6,41 @@ description: Agent-facing contract for transcription, speaker identity, and spee # vox -File transcription and TTS through Alibaba Model Studio, with local MLX speaker identification. `vox hear` -uploads a file, transcribes it, and stores a **run**; markdown, subtitles, and -plain text are exports from that run. There is no realtime ASR and no -microphone capture for transcription. Files up to 12 hours are ordinary input. - -Do not invent commands. `vox @schema` is the surface. - -## First moves - -```bash -vox @schema -vox status -``` - -Provider keys need no setup. Commands that call a provider take an optional -`apiKey` field; omitted, the default chain runs: the env var -(`DASHSCOPE_API_KEY`, `MIMO_API_KEY`) if set, else the celados-agent vault -through latch. To use another key, pass a reference, never paste the key: -`apiKey: { ref: 'env:MY_KEY' }` or `{ ref: 'bw:///' }`. -`vox status` resolves both chains and shows, per provider, `available` or what -it `tried` — never key material. Stored runs, `export`, `session.*`, -`speaker.*` and cached `say` read no key. - -## Speech to text - -```bash -vox hear --file recording.wav -vox hear --file lecture.mp3 --lang zh -vox hear "{ file: 'lecture.mp3', lang: ['zh'] }" -vox hear "{ file: 'meeting.m4a', speakers: true, vocab: 'meeting' }" -vox hear "{ file: 'recording.wav', refresh: true }" - -vox session.list -vox session.list --file recording.wav -vox session.remove --sid a3f1c2 -vox session.remove --all - -vox export --sid a3f1c2 --format md -vox export --sid a3f1c2 --format srt --output out.srt -vox export --sid a3f1c2 --format txt -``` - -`--lang` is the cheapest quality win on known-language audio. `--vocab` only -biases domain terms; it cannot recover bad audio. Do not split a file to work -around length: splitting truncates words at every cut. - -Vendors: `fun` (default, Fun-ASR), `qwen`, or `mimo`. DashScope's `fun` and -`qwen` cap at **12 hours / 2 GB**. MiMo accepts only MP3/WAV with Base64 payloads -up to 10 MB, returns plain text without timestamps or speaker labels, and does -not support Vox vocabularies. Use it for short A/B samples: - -```bash -vox hear --file sample.mp3 --model mimo --lang zh -``` - -Expect roughly a minute per 45 minutes of audio from the DashScope file models. - -`hear` always writes a Markdown transcript under `~/.vox/runs//transcript.md` -and returns its path. Pass `output` to choose a destination. - -## Speaker identity (Apple Silicon) - -Prerequisites: `uv` and `ffmpeg` on PATH (`brew install uv ffmpeg`). ASR still -uses DashScope; voice matching runs locally. The first enrollment or identity -request lazily downloads the pinned model (~26 MB) and Python runtime dependencies. -Weights live in `~/.vox/models/`, runtime files in `~/.vox/runtime/`, and download -caches in `~/.vox/cache/uv` and `~/.vox/models/huggingface`. No PyTorch is required. - -Enroll one clean solo recording per person (3–30 seconds; longer files use the -first 30 seconds by default). `start` and `end` select seconds in an existing -recording. Both enrollment and matching remove long near-digital silence before -feature extraction; at least 3 seconds must remain. This is an energy gate, not -a speech/noise classifier. Keep enrollment and evaluation recordings separate when checking quality. - -```bash -vox speaker.enroll "{ id: 'ethan', name: 'Ethan', file: 'ethan.wav' }" -vox speaker.enroll "{ id: 'deniffer', name: 'Deniffer', file: 'deniffer.wav', start: 10, end: 30 }" -vox speaker.list -vox hear "{ file: 'meeting.m4a', identify: true, output: 'meeting.transcript.md' }" -``` - -`identify: true` automatically enables ASR speaker separation and matches against -all enrolled people. Limit candidates with `speakerIds: ['ethan', 'deniffer']`. -MiMo cannot identify speakers because it has no speaker labels. - -The Markdown includes timestamps and names. Anonymous IDs remain in the stored -JSON; names and decisions are saved separately as `identified.json`. Uncertain -matches keep `Speaker `, rather than forcing a name. IDs are local to each -recording; multiple IDs may match one person. Identity defaults (`threshold: 0.65`, -`margin: 0.1`) are starting points, not a calibrated accuracy guarantee. - -```bash -# Re-identify a stored run after enrollment/config changes; no ASR upload or charge. -vox speaker.identify "{ sid: 'a3f1c2', output: 'meeting.transcript.md' }" -vox speaker.config -vox speaker.config "{ threshold: 0.7, margin: 0.1 }" -vox speaker.enroll "{ id: 'ethan', name: 'Ethan', file: 'better.wav', replace: true }" -vox speaker.remove --id ethan -``` - -`export` uses the latest saved identity snapshot for that run, including for -SRT/VTT/TXT/JSON. Re-run `speaker.identify` to apply changes to the speaker library. -If identity fails after ASR, the anonymous ASR run remains stored; use -`session.list` and `speaker.identify` to resume. No meeting transcript is modified -outside vox's own run or the explicit output destination. - -Profiles and matching settings are in `~/.vox/speakers.json` (private permissions). -Source enrollment audio is not copied; only its normalized voiceprint is stored. -Treat these profiles as personal voice data. `voice.*` is still TTS voice cloning; -`speaker.*` is recognition identity. `VOX_HOME` overrides the state directory for -isolated tests. All speaker artifacts use the model version recorded with the -profiles; changing embedding models requires re-enrollment. - -## Result envelopes - -`hear` returns a YAML envelope. A short transcript is inlined as `text`. A long -one folds to `preview` plus `$hints` naming the `vox export` command. Do not -try to widen the envelope — run the export. - -`session.list` is the same records without `text` or `preview`. `export` -without `--output` prints the document itself (not a YAML wrapper). - -## Vocabularies - -A vocabulary is a YAML file you write directly. There is no create/edit command. - -```yaml -# ~/.vox/vocabulary/meeting.yaml -lang: zh -default_weight: 4 -words: - 百炼: 5 - Fun-ASR: 5 - 赛德克巴莱: -``` - -```bash -vox vocab.list -vox vocab.prune --dryRun -``` - -`vox hear --vocab meeting` syncs on demand. Editing the YAML changes the run -id, so the next `hear` re-recognizes. - -Account cap: **10 lists shared across models**. One vocabulary used with both -models consumes two slots. Words are at most 15 characters when non-ASCII. -Weight is 1–5; 50 is a super hotword that only `qwen` honours. - -## Text to speech - -```bash -vox say --text 'Hello world' --voice Cherry -vox say - <<'EOF' -voice: Cherry -text: | - First line. - Slow and clear. -EOF -vox say "{ text: { ref: 'file:script.txt' }, voice: 'Cherry' }" -vox say --text 'Save this' --output ~/Desktop/out.wav -vox voice.list -vox voice.record --file ~/sample.wav --name myvoice -vox voice.remove --voiceId -``` - -A script already on disk is `text: { ref: 'file:' }` (read verbatim, -relative to the working directory, `~/` is home) — don't paste it back. - -System voices: Cherry, Ethan, Chelsie, Serena (zh/en), Dylan (Beijing), Jada -(Shanghai), Sunny (Sichuan). `--output` writes a WAV and does not play. - -## Failures - -Domain refusals carry `error: DOMAIN_ERROR` and a matchable `code`. Every -failure exits 1 — branch on `code`, not the process status or the detail prose: - -`key_unavailable` · `audio_too_large` · `audio_unsupported` · -`vocab_not_found` · `vocab_quota_exceeded` · `vocab_model_mismatch` · -`vocab_index_corrupt` · `session_not_found` · `session_ambiguous` · -`invalid_usage` · `io_error` · `api_error` · `speaker_not_found` · -`speaker_labels_missing` · `speaker_runtime_unavailable` - -A key the chain cannot find fails as `INVALID_INPUT` on `apiKey` (commands that -always call the provider) or `key_unavailable` (commands that read the key only -when a provider is called); both name every reference tried. Report it to a -person rather than hunting for the key yourself. `session_not_found` → `vox session.list`. - -## Anti-patterns - -| Don't | Do instead | Why | -| --------------------------------------- | ------------------------------------- | ---------------------------------------------- | -| Split a long file into clips | Pass the whole file to `hear` | Cuts drop words and native sentence boundaries | -| Re-read `hear` output for the full hour | `vox export --sid --format md` | The envelope folds on purpose | -| Treat the envelope as a subtitle file | `export --format srt` or `txt` | Index and content never share stdout | -| Invent a create/edit vocab command | Write `~/.vox/vocabulary/.yaml` | The YAML is the source of truth | -| Put multi-line TTS in a JSON object | `vox say - <<'EOF'` with `text: \|` | Escaping burns tokens and breaks | -| Paste a script file into `text` | `text: { ref: 'file:script.txt' }` | Read as is; no copy in the transcript | -| Paste a key literal into `apiKey` | `apiKey: { ref: 'env:NAME' }` | Literals land in shell history and transcripts | +File transcription and TTS through Alibaba Model Studio, with local MLX speaker identification. `vox hear` uploads a file, transcribes it, and stores a **run**; markdown, subtitles and plain text are exports of that run. No realtime ASR, no microphone transcription. Preflight: `vox status`. + +Read the section for your task, then only that slice: `vox @schema .hear`, or several at once with `vox @schema '.{hear,export}'`. Deeper material: `vox @skill references/`. + +## Everywhere + +- Provider keys need no setup. A command's optional `apiKey` falls back to the env var (`DASHSCOPE_API_KEY`, `MIMO_API_KEY`), then the vault. To use another key pass a reference (`{ ref: 'env:MY_KEY' }`), never the literal: literals land in shell history and transcripts. A key nobody can find is for a person to fix; don't hunt for it. +- Failures exit 1 with `error: DOMAIN_ERROR` and a `code`. Branch on `code`, never on the exit status or detail prose. `session_not_found` → `vox session.list`. +- Stored runs, `export`, `session.*`, `speaker.*` and cached `say` read no key. `VOX_HOME` relocates `~/.vox` for isolated tests. + +## Transcribing — `.{hear,session}` + +- Pass `--lang` for known-language audio: the cheapest quality win. Pass the whole file; splitting to get under a limit truncates words at every cut. Expect about a minute per 45 minutes of audio. +- `fun` (default) and `qwen` cap at 12 hours / 2 GB. `mimo` takes only MP3/WAV up to 10 MB and returns plain text with no timestamps, speaker labels or vocabularies. Use it for short A/B samples. +- `hear` writes `~/.vox/runs//transcript.md` and returns a YAML envelope. A long transcript folds to `preview` plus a `$hints` export command; run that rather than re-reading the envelope. `session.list` is the same records without text. +- The run id hashes audio plus arguments, so repeating a call is a cache hit; `refresh: true` re-recognizes. + +## Exporting — `.export` + +`export` renders a stored run. Without `--output` stdout is the document itself, not a YAML wrapper. Use `srt`/`vtt`/`txt`/`json` for subtitles and plain text; the envelope is not a subtitle file. Exports use the latest saved speaker-identity snapshot. + +## Speaker identity — `.speaker` (and `hear --identify`) + +Local, Apple Silicon, needs `uv` and `ffmpeg`. Voiceprints are personal voice data: treat them accordingly. Enrollment rules, matching semantics and storage: `vox @skill references/speaker.md`. + +## Vocabularies — `.vocab` + +A vocabulary is a YAML file you write at `~/.vox/vocabulary/.yaml`; there is no create or edit command. `hear --vocab ` syncs on demand, and editing the file changes the run id. It only biases domain terms; it cannot rescue bad audio. File format and quota: `vox @skill references/vocab.md`. + +## Speaking — `.{say,voice,cache}` + +- Multi-line text goes as the whole input in YAML (`vox say - <<'EOF'` with `text: |`), not inside a JSON object. A script already on disk is `text: { ref: 'file:' }`; don't paste it back. +- System voices: Cherry, Ethan, Chelsie, Serena (zh/en), Dylan (Beijing), Jada (Shanghai), Sunny (Sichuan). `--output` writes a WAV and does not play. +- `voice.*` is TTS voice cloning; `speaker.*` is recognition identity. They are unrelated. ## Feedback -File issues at [celados/vox](https://github.com/celados/vox/issues) with the -command, the run id, and the YAML error document. +File issues at [celados/vox](https://github.com/celados/vox/issues) with the command, the run id and the YAML error document. diff --git a/src/main.test.ts b/src/main.test.ts index 47247b4..a591cf2 100644 --- a/src/main.test.ts +++ b/src/main.test.ts @@ -75,7 +75,7 @@ test("@skill prints the embedded guide", async () => { const { stdout, stderr, exitCode } = await run(["@skill"]); expect(exitCode, stderr).toBe(0); expect(stdout).toContain("vox @schema"); - expect(stdout).toContain("There is no realtime ASR"); + expect(stdout).toContain("No realtime ASR"); }); test("status reports each provider key without key material", async () => { diff --git a/src/references/speaker.md b/src/references/speaker.md new file mode 100644 index 0000000..9196201 --- /dev/null +++ b/src/references/speaker.md @@ -0,0 +1,29 @@ +--- +type: Guide +title: vox speaker identity +description: Enrollment, matching semantics and storage for vox speaker.* and hear --identify. +--- + +# Speaker identity (Apple Silicon) + +Prerequisites: `uv` and `ffmpeg` on PATH (`brew install uv ffmpeg`). ASR still uses DashScope; voice matching runs locally. The first enrollment or identity request lazily downloads the pinned model (~26 MB) and Python runtime dependencies. Weights live in `~/.vox/models/`, runtime files in `~/.vox/runtime/`, caches in `~/.vox/cache/uv` and `~/.vox/models/huggingface`. No PyTorch needed. + +Selectors: `vox @schema '.{speaker,hear}'`. + +## Enrolling + +Enroll one clean solo recording per person (3–30 seconds; longer files use the first 30 seconds). `start` and `end` select seconds in an existing recording. Enrollment and matching both strip long near-digital silence first, and at least 3 seconds must remain. That is an energy gate, not a speech/noise classifier. Keep enrollment and evaluation recordings separate when checking quality. Re-enroll with `replace: true`. + +## Matching + +`identify: true` on `hear` turns on ASR speaker separation and matches against all enrolled people; limit candidates with `speakerIds`. MiMo cannot identify speakers (no labels). + +The Markdown includes timestamps and names. Anonymous IDs stay in the stored JSON; names and decisions are saved separately as `identified.json`. Uncertain matches keep `Speaker ` rather than a forced name. IDs are local to each recording, and several IDs may match one person. `threshold: 0.65` and `margin: 0.1` are starting points, not a calibrated accuracy guarantee. + +## Re-identifying + +After enrollment or config changes, `speaker.identify` re-matches a stored run with no ASR upload or charge. `export` uses the latest saved identity snapshot, so re-run `speaker.identify` to apply library changes. If identity fails after ASR, the anonymous ASR run remains stored: resume with `session.list` and `speaker.identify`. Nothing outside vox's own run or the explicit `output` is modified. + +## Storage + +Profiles and settings: `~/.vox/speakers.json` (private permissions). Source enrollment audio is not copied; only the normalized voiceprint is stored. All artifacts use the model version recorded with the profiles; changing embedding models requires re-enrollment. diff --git a/src/references/vocab.md b/src/references/vocab.md new file mode 100644 index 0000000..93de186 --- /dev/null +++ b/src/references/vocab.md @@ -0,0 +1,24 @@ +--- +type: Guide +title: vox vocabularies +description: Hotword YAML format and account quota for vox vocab.* and hear --vocab. +--- + +# Vocabularies + +```yaml +# ~/.vox/vocabulary/meeting.yaml +lang: zh +default_weight: 4 +words: + 百炼: 5 + Fun-ASR: 5 + 赛德克巴莱: +``` + +Selector: `vox @schema .vocab`. + +- Account cap: 10 lists shared across models. One vocabulary used with both models consumes two slots. +- Non-ASCII words are at most 15 characters. Weight is 1–5; 50 is a super hotword that only `qwen` honours. +- MiMo does not support vocabularies. +- `vocab.prune --dryRun` first: it deletes server-side lists no local YAML claims. diff --git a/src/skill.embed.ts b/src/skill.embed.ts index 5a2820a..92baa02 100644 --- a/src/skill.embed.ts +++ b/src/skill.embed.ts @@ -1,8 +1,10 @@ import { pickFiles } from "@celados/argc/skill"; export function embedSkill(): Record { - const files = pickFiles(import.meta.dir, ["index.md"]); - const body = files["index.md"]; + const { "index.md": body, ...references } = pickFiles(import.meta.dir, [ + "index.md", + "references/**/*.md", + ]); if (body === undefined) throw new Error("Embedded skill body is missing index.md"); - return { "SKILL.md": body }; + return { "SKILL.md": body, ...references }; }