Skip to content

feat: playground web validation: audio-studio, sherpa-onnx, moonshine - #523

Open
abretonc7s wants to merge 3 commits into
mainfrom
feat/manual-000177-validate-audio-web-builds
Open

abretonc7s wants to merge 3 commits into
mainfrom
feat/manual-000177-validate-audio-web-builds

Conversation

@abretonc7s

@abretonc7s abretonc7s commented Nov 7, 2026 •

Copy link
Copy Markdown
Collaborator

Validate audio-studio, sherpa-onnx.rn and moonshine.rn on web

Recorded web runs in Chrome of the playground and sherpa-voice, plus fixes for every failure they found.

Fixes

  • sherpa-onnx.rn
    • Web initTts names the model, tokens or lexicon URLs it couldn't fetch, including when none of them loads. It removes the files that did load, then their directory. Before, a missing lexicon crashed with memory access out of bounds, and a total download failure gave a bare Failed to load TTS model files.
    • Web FileSystem.removePath removes directories. isDirectory read stat.isDir, which Emscripten doesn't set, so directories were sent to unlink and stayed.
    • Non-VITS models are rejected with a clear error. The README documents VITS-only web TTS.
    • The web loader clears its 30 s backup timer, so Module loading timeout reached no longer fires after a successful load.
    • Tests: webTtsInit.test.ts, webFileSystem.test.ts, webCombinedLoader.test.ts.
  • sherpa-voice
    • On web, the ASR, speaker ID and audio tagging sample cards show the real size and duration instead of 0 Bytes / 0:00. getInfoAsync doesn't exist on web, so web fetches the file.
    • Test: fileUtils.test.ts.
  • playground
    • build:deps also builds the sherpa-onnx.rn and moonshine.rn lib/, which were stale.
    • Test: config/build-deps.test.ts.
  • agentic tooling
    • web-browser.mjs adds WEB_FAKE_AUDIO_FILE for fake mic capture from a WAV.
    • Two recipe tests no longer depend on slot env.
    • Four new web recipes. The sherpa recipes restore the route they started on. The Moonshine recipe waits for the stop to settle and checks the final transcript.

Evidence (web)

Flow Run Result
audio-studio: record, pause/resume, stop, playback, waveform and mel views, preview, decode, features, trim audio-studio-web-validation Pause Δ0 ms, resume +2 s, playback past 0.5 s, waveform 82,560 samples, mel 498 steps
sherpa ASR file + Live Mic web-asr-validation Both transcripts contain "my fellow Americans", card shows 343.83 KB / 0:11
sherpa TTS with the hosted lexicon web-tts-validation "Speech played successfully!"; the lexicon came from the Hugging Face URL (HTTP 200)
moonshine model load + live transcription moonshine-web-live-transcription Live interim and final transcript "And so, my fellow American."
TTS failure cleanup probe forcing 404s Errors name the failed URLs; no /wasm/tts-* directory left behind
All of the above in one session task recipe 20261107030405563-74735 177/177 in 84 s. sherpa-voice starts on a pushed route and returns to it; Metro then switches to the playground and the tab reloads. Playground load window: 0 console errors

The slot serves a local sherpa-onnx 2.0.1 WASM, asserted in the run, because the farm's copy is 1.3.1.

Checks:

  • sherpa-onnx.rn: 97/97 tests; typecheck clean.
  • playground: test:config 16/16.
  • sherpa-voice: fileUtils.test.ts 4/4 with jest CLI overrides (the app's yarn test needs @react-native/jest-preset, which isn't installed); tsc clean.
  • repo: check:deps passes; all 48 recipes validate.
  • ESLint: no new errors.

Limits

  • Native: iOS and Android weren't run.
    • The getFileSize native branch keeps the same getInfoAsync call and is unit-tested only.
    • build:deps changes the lib/ bundled on every platform.
    • Both need a device run before merge.
  • sherpa-voice unit test isn't run by CI: no CI job covers sherpa-voice, and its Jest setup needs @react-native/jest-preset. The web fix is also asserted by the sherpa ASR recipe.
  • Playground loads the published 2.0.0 glue from jsDelivr, so it gets the loader and removePath fixes only after a release.
  • Moonshine + Sherpa live mode stays native-only in the playground. The playground hook resolves native asset URIs, while the package parts run on web.

Screenshots/Recordings

Full evidence gallery and publication receipt

Publication approval

farm-automation · service · farm-automation · 2026-11-07T03:57:28.538Z
Package: d7d26c9e540f2ba5d02cd4d110960dc1b8b0ed7e47b65def89eec3bf02a3ee73. Visual receipts: pipeline-sha256:0ef5850080e20b601a9fdadc7cd77c653708aaf268c65851f715fa6447b3a2d2, pipeline-sha256:17a137dae9c3050fd443be893a758a5792909d0ab25ba44f249421735302c9dc, pipeline-sha256:392349c4f3ea8cff455c8ed5fe94b6f9d4fa1b061df1349558a9f2e32fa4768f, pipeline-sha256:4003074c45592979f325f0e3c14f8d520cd95e07eef20d855136a7364723b572, pipeline-sha256:5ed45b66b428a9dea536a79a2ced137779c2e3d00e2266cb2b3233435be084c9, pipeline-sha256:93231ecf693d763da4814954e0dbf0f6df8d4f14a540c2720442d15eca89994b, pipeline-sha256:94eca704ace2ee7828bb89a4460f41e6978476073f7e22beaf8951091178af24, pipeline-sha256:98785c9be59b12a9bd7bb7da7beaa04da055641a538041465eedbc61897b9c4a, pipeline-sha256:a05bedac10b3ba462a0de095f2f14738d709b8bb4448da7be0d01b6c425b0a96, pipeline-sha256:b7f29975d8b1165593dab5be370b9815944c875d806f0cda0765827ad29a2c43, pipeline-sha256:b8359e1355c22e9d7b6fc032881e9fddbfa62b2fe109a20a25e69806b66420f8, pipeline-sha256:e5955a7d27e36b45f174ba40a57bf93c18b987bad33e67092eae256c882c4793.

Description

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Performance improvement
  • Refactoring

Architecture Decision

Test-Driven Development Checklist

Note: TDD is recommended but not required. The testing framework is newly available, and test coverage is being improved incrementally.

Pre-Implementation (for new features - recommended)

  • Architecture document created/updated
  • API design reviewed
  • Test plan created with test cases
  • Tests written before implementation (when feasible)

Implementation

  • TypeScript interfaces/types defined
  • iOS implementation complete
  • Android implementation complete
  • Web implementation complete (if applicable)

Testing

  • Unit tests included (recommended)
  • iOS platform tests pass
  • Android platform tests pass
  • Integration tests included (when applicable)
  • Cross-platform consistency verified
  • Test coverage adequate for the change
  • Edge cases and error scenarios considered
  • Performance impact measured (if applicable)

Test Evidence

# Test output (if applicable)
./scripts/run_tests.sh all
...

Documentation

  • API documentation updated
  • Usage examples added (if new feature)
  • CHANGELOG.md updated
  • README updated (if needed)

Breaking Changes

Related Issues

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • Tests are included (recommended for new features and bug fixes)
  • Existing tests pass locally with my changes
  • Any dependent changes have been merged and published

Additional Notes

Visual judgments

Execution: d0acd43a-9e2e-49af-9aad-6c76b76694f3

Criterion Verdict Reviewed by
AC-2: AC-2: audio-studio on web: record, pause/resume, stop, playback, waveform/feature extraction the playground exposes. proven Verified by agent · Independent review
AC-3: AC-3: sherpa-onnx on web: model load and at least one ASR flow and one TTS flow the playground exposes. proven Verified by agent · Independent review
AC-4: AC-4: moonshine on web: model load and live transcription. proven Verified by agent · Independent review
sherpa-asr/sample-info: Selecting a bundled sample on web shows its real size and duration. proven Verified by agent · Independent review

…builds

- sherpa-onnx.rn: web initTts names unfetched model/tokens/lexicon URLs
  and frees loaded files instead of crashing out of bounds; rejects
  non-VITS models; README documents VITS-only web TTS
- sherpa-onnx.rn: web loader clears its 30 s backup timer once loading
  settles, so the timeout warning no longer fires after a good load
- sherpa-voice: sample cards measure files by fetch on web, where
  getInfoAsync is missing (was 0 Bytes / 0:00); add the jest preset
  jest-expo needs so yarn test runs
- playground: build:deps also builds sherpa-onnx.rn and moonshine.rn lib
- agentic: WEB_FAKE_AUDIO_FILE for fake mic capture, hermetic recipe
  tests, four web validation recipes
@abretonc7s abretonc7s changed the title chore: prepare farmslot publication pkg-ccbb7833-mw5vfv30 feat: playground web validation: audio-studio, sherpa-onnx, moonshine Nov 7, 2026
@abretonc7s
abretonc7s marked this pull request as ready for review November 7, 2026 03:58
@sonarqubecloud

sonarqubecloud Bot commented Nov 7, 2026

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant