Skip to content

Import custom ONNX speech models - #211

Merged
ppXD merged 1 commit into
mainfrom
agent/import-custom-onnx-models
Jul 28, 2026
Merged

ppXD merged 1 commit into
mainfrom
agent/import-custom-onnx-models

Conversation

@ppXD

@ppXD ppXD commented Jul 28, 2026

Copy link
Copy Markdown
Owner

What changed

  • allow the local model picker to select .onnx speech models alongside GGML/GGUF
  • validate ONNX graphs with the bundled ONNX Runtime and read official sherpa-onnx model_type metadata
  • support SenseVoice, Paraformer, sherpa Whisper, and NeMo Parakeet bundles
  • automatically discover matching tokens, encoder/decoder/joiner graphs, and external .weights/.data files
  • run validation and large-file copying on a blocking worker so the UI stays responsive
  • persist imported model size and retain backward compatibility with existing custom-model registries
  • document custom ONNX imports in the English and Simplified Chinese READMEs

Why

Custom model import only accepted Whisper GGML/GGUF files even though many downloadable speech models are distributed as sherpa-compatible ONNX bundles. Accepting an arbitrary .onnx file by extension alone would be unsafe because ONNX also contains vision, VAD, embedding, and LLM graphs; Wisp now validates the graph and supports only ASR families it can actually execute.

Validation

  • Test First: the initial ONNX tests failed before implementation
  • app Rust tests: 28 passed
  • root workspace tests: all passed
  • new ONNX coverage: metadata mapping, real non-ASR rejection, single/pair/triplet companion discovery, missing-token failure, encoder metadata fallback, external weights
  • real-model smoke: downloaded the official 228 MB SenseVoice int8 ONNX + tokens, imported the bundle, initialized the Wisp sherpa recognizer, and completed one inference call
  • app clippy with -D warnings
  • root fmt/clippy with -D warnings
  • Svelte check: 0 errors (one pre-existing missing Node type warning)
  • production Vite build passed

@ppXD
ppXD merged commit 0298517 into main Jul 28, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant