Repository navigation
docs: show the app in the README with screenshots #15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,182 +1,141 @@ | ||
| # RunAnywhere web example | ||
|
|
||
| A browser app built on the RunAnywhere Web SDK. Models download to the browser | ||
| and run there via WebAssembly, with a WebGPU path where the browser and the | ||
| model support it. Prompts, audio, images, and documents stay on the device. | ||
|
|
||
| ## What works | ||
|
|
||
| | Surface | What it does | SDK entry point | | ||
| |---|---|---| | ||
| | Assistant | Streaming chat, tool calling, saved conversations | `RunAnywhere.llm.generateStream`, `RunAnywhere.llm.tools` | | ||
| | Talk | Full voice session (VAD, STT, LLM, TTS) | `RunAnywhere.voice.createSession` | | ||
| | Image & Live | Describe a photo or a live camera frame | `RunAnywhere.vlm.generateStream` | | ||
| | Transcribe | Batch and streaming speech to text | `RunAnywhere.stt.transcribe`, `.transcribeStream` | | ||
| | Read Aloud | Speak arbitrary text | `RunAnywhere.tts.speak` | | ||
| | Voice Activity | Streaming speech detection | `RunAnywhere.vad.detectStream` | | ||
| | Documents | RAG over `.txt`, `.md`, and `.json` files you drop in | `RunAnywhere.rag.open` | | ||
| | Solutions | Two packaged YAML pipelines: voice agent and document Q&A | `RunAnywhere.solutions.run` | | ||
| | Benchmarks | One prompt at three token budgets (50, 256, 512), charted | `RunAnywhere.llm.generateStream` | | ||
| | Downloads | Model registry, disk usage, storage folder | `RunAnywhere.storage`, `RunAnywhere.models` | | ||
| | Settings | Generation preferences, API credentials, Hugging Face token | `RunAnywhere.setHuggingFaceToken` | | ||
|
|
||
| Segmentation and Diarization have views and SDK calls wired, but no browser | ||
| engine registers those capabilities, so the catalog is empty for both and each | ||
| tab renders an unavailable placeholder. | ||
|
|
||
| ## Requirements | ||
|
|
||
| | Item | Minimum | | ||
| |---|---| | ||
| | Node.js | 22.12 (CI runs 24) | | ||
| | Browser | Chrome or Edge 86, Safari, Firefox | | ||
| | Cross-origin isolation | Required for `SharedArrayBuffer`. The dev server and `vercel.json` send COOP `same-origin` and COEP `require-corp`; `public/coi-serviceworker.js` covers hosts that cannot | | ||
| | Disk space | Hundreds of megabytes to a few gigabytes for downloaded models, held in OPFS | | ||
|
|
||
| ## Run it | ||
|
|
||
| ```bash | ||
| git clone https://github.com/RunanywhereAI/runanywhere-web.git | ||
| cd runanywhere-web | ||
|
|
||
| npm ci | ||
| npm run dev # http://localhost:3000 | ||
| ``` | ||
|
|
||
| `npm ci` pulls the SDK and its WASM artifacts. There is no separate WASM build | ||
| step and no Emscripten toolchain to install. | ||
|
|
||
| | Script | What it does | | ||
| |---|---| | ||
| | `npm run dev` | Vite dev server on `localhost:3000` (strict port) with COOP/COEP headers | | ||
| | `npm run build` | Production bundle into `dist/`, including the Emscripten `.js`/`.wasm` pairs | | ||
| | `npm run preview` | Serve the built `dist/` on `localhost:3000` | | ||
| | `npm run typecheck` | `tsc --noEmit` | | ||
| | `npm run lint` | ESLint over `src`, zero warnings tolerated | | ||
| | `npm run test` | Vitest over `src/**/*.test.ts` | | ||
| | `npm run release:build` | `build` followed by `release:verify` | | ||
| | `npm run release:verify` | Assert `dist/` holds every required runtime file | | ||
| | `npm run release:deploy` | Build, verify, and deploy a prebuilt static bundle to Vercel | | ||
|
|
||
| ## SDK packages | ||
|
|
||
| Everything comes from the npm registry. There are no `file:` links, no `paths` | ||
| aliases, and no monorepo checkout, so `npm install` is the only thing that | ||
| decides which SDK version the app runs against. Both the TypeScript modules and | ||
| every WASM artifact come out of `node_modules/@runanywhere/*`. | ||
|
|
||
| ```jsonc | ||
| "dependencies": { | ||
| "@runanywhere/proto-ts": "^0.20.19", | ||
| "@runanywhere/web": "^0.20.19", | ||
| "@runanywhere/web-llamacpp": "^0.20.19", | ||
| "@runanywhere/web-onnx": "^0.20.19" | ||
| } | ||
| ``` | ||
|
|
||
| | Package | Role | | ||
| |---|---| | ||
| | `@runanywhere/web` | SDK lifecycle and the inference facades. `@runanywhere/web/browser` adds `AudioCapture`, `AudioFileLoader`, and `VideoCapture` | | ||
| | `@runanywhere/web-llamacpp` | LLM and VLM backend registration, CPU and WebGPU builds | | ||
| | `@runanywhere/web-onnx` | Sherpa-ONNX backend registration for STT, TTS, VAD, and embeddings | | ||
| | `@runanywhere/proto-ts` | Generated protobuf types for models, events, errors, and modalities | | ||
| # RunAnywhere AI for the web | ||
|
|
||
| To try an unreleased SDK build, `npm install` a packed tarball or use | ||
| `npm link`. Do not reintroduce a source alias. | ||
| <p align="center"> | ||
| <img src="https://raw.githubusercontent.com/RunanywhereAI/runanywhere-sdks/main/docs/logo.svg" alt="RunAnywhere" width="120"/> | ||
| </p> | ||
|
|
||
| ## WASM artifacts | ||
| <p align="center"> | ||
| <a href="https://runanywhere-web-demo.vercel.app"> | ||
| <img src="https://img.shields.io/badge/Try%20it-in%20your%20browser-ff6900?style=for-the-badge&logo=googlechrome&logoColor=white" alt="Try it in your browser" /> | ||
| </a> | ||
| </p> | ||
|
|
||
| Five JS/WASM pairs ship across the three SDK packages. `vite.config.ts` copies | ||
| each canonical pair into `dist/assets/` next to Vite's hashed copy, because | ||
| Emscripten's pthread glue starts its workers from the original filename. | ||
| <p align="center"> | ||
| <img src="https://img.shields.io/badge/Runs-WebAssembly-654FF0?style=flat-square&logo=webassembly&logoColor=white" alt="WebAssembly" /> | ||
| <img src="https://img.shields.io/badge/Accelerated-WebGPU-005A9C?style=flat-square" alt="WebGPU" /> | ||
| <img src="https://img.shields.io/badge/TypeScript-5-3178C6?style=flat-square&logo=typescript&logoColor=white" alt="TypeScript" /> | ||
| <img src="https://img.shields.io/badge/License-RunAnywhere-blue?style=flat-square" alt="RunAnywhere License" /> | ||
| </p> | ||
|
|
||
| | Pair | Package | | ||
| |---|---| | ||
| | `racommons.{js,wasm}` | `@runanywhere/web` | | ||
| | `racommons-llamacpp.{js,wasm}` | `@runanywhere/web-llamacpp` | | ||
| | `racommons-llamacpp-webgpu.{js,wasm}` | `@runanywhere/web-llamacpp` | | ||
| | `racommons-onnx-sherpa.{js,wasm}` | `@runanywhere/web-onnx` | | ||
| | `racommons-onnx-sherpa-webgpu.{js,wasm}` | `@runanywhere/web-onnx` | | ||
| The RunAnywhere consumer app for the browser, written in TypeScript. | ||
|
|
||
| A production build fails naming the missing files rather than shipping a bundle | ||
| that only breaks after deployment. | ||
| There is nothing to install. Open the page, pick a model, and it downloads into your browser | ||
| and runs there through WebAssembly, with a WebGPU path where your browser and the model both | ||
| support it. Whatever you type, say, or upload stays in the tab. | ||
|
|
||
| ## Project layout | ||
| ## Try it | ||
|
|
||
| ``` | ||
| runanywhere-web/ | ||
| index.html Vite entry, plus the pre-paint theme script and boot screen | ||
| src/ | ||
| main.ts Boot: cross-origin isolation, SDK init, backend registration, catalog | ||
| app.ts Shell, drawer navigation, hash routing, the Advanced hub | ||
| views/ One file per surface: chat, vision, voice, transcribe, speak, vad, | ||
| segmentation, diarization, documents, storage, solutions, | ||
| benchmarks, settings | ||
| services/ Model catalog, engine availability, conversation store (IndexedDB), | ||
| Hugging Face client, markdown, formatting helpers | ||
| components/ Model selection sheet, dialogs, file drop, icons, shared notices | ||
| styles/ design-system.css is the only token layer | ||
| public/coi-serviceworker.js Cross-origin-isolation fallback | ||
| scripts/release.sh Static release verify, stage, and deploy | ||
| tests/ Manual browser test plan | ||
| vite.config.ts Dev/preview COOP-COEP headers, WASM copy plugin, chrome86 target | ||
| vercel.json COOP/COEP headers and SPA rewrites | ||
| ``` | ||
| **[runanywhere-web-demo.vercel.app](https://runanywhere-web-demo.vercel.app)** | ||
|
|
||
| Every surface has a URL fragment (`#/vision`, `#/benchmarks`), so a tab survives | ||
| a refresh, a pasted link, and the reload the isolation service worker performs | ||
| on Safari. | ||
| Works in Chrome or Edge 86 and newer, Safari, and Firefox. Models are held in OPFS on your | ||
| own disk, so the second visit starts instantly. | ||
|
|
||
| Views may import `@runanywhere/web` and `@runanywhere/web/browser`. They must | ||
| not reach into `@runanywhere/web/internal` or `@runanywhere/web/backend`, and | ||
| must not reimplement SDK routing, storage, or inference rules in UI code. See | ||
| `AGENTS.md`. | ||
| <!-- GIF slot: chat with tool calling, the voice session, and image understanding. | ||
| Waiting on the capture pass that follows the current app bug fixes. --> | ||
|
|
||
| ## Configuration | ||
| ## What it looks like | ||
|
|
||
| Settings holds an API key and base URL for the session only; neither is written | ||
| to storage. For a hosted deployment, Vite reads two build-time variables and | ||
| boots the SDK straight into the production environment: | ||
| Captured in a Chromium browser with WebGPU active, running PrismML Bonsai 1.7B through the | ||
| llama.cpp WASM build. | ||
|
|
||
| | Variable | Meaning | | ||
| | | | | ||
| |---|---| | ||
| | `VITE_RUNANYWHERE_API_KEY` | Publishable browser key. Never a server-side secret, since Vite inlines it into the bundle | | ||
| | `VITE_RUNANYWHERE_BASE_URL` | Production API origin | | ||
| |  |  | | ||
| | An answer generated in the browser tab. The badge in the corner shows WebGPU is in use. | The picker recommends a model for the machine and reports what the browser can do. | | ||
| |  |  | | ||
| | Talk assembles a speech-to-text, chat, text-to-speech and voice-detection model, then runs the conversation in the tab. | The lower-level surfaces: transcribe, read aloud, voice activity, diarization, segmentation, benchmarks, solutions. | | ||
| |  |  | | ||
| | What is on disk and what it costs. Browser storage is private to the site unless you pick a folder. | Sampling, the system prompt, and optional API credentials. | | ||
|
|
||
| The image files are in [`docs/screenshots/`](docs/screenshots). | ||
|
|
||
| ## What you can do | ||
|
|
||
| | | | | ||
| | --- | --- | | ||
| | **Assistant** | Streaming chat with tool calling and saved conversations | | ||
| | **Talk** | A full voice session: it listens, thinks, and answers out loud | | ||
| | **Image and Live** | Describe a photo, or a live camera frame | | ||
| | **Transcribe** | Batch and streaming speech to text | | ||
| | **Read aloud** | Speak any text you give it | | ||
| | **Documents** | Drop in `.txt`, `.md`, or `.json` and ask questions about them | | ||
| | **Solutions** | Two packaged pipelines: a voice agent and document Q&A | | ||
| | **Benchmarks** | One prompt at three token budgets, charted | | ||
| | **Downloads** | Model registry, disk usage, storage folder | | ||
|
|
||
| Segmentation and Diarization have views wired up, but no browser engine registers those | ||
| capabilities yet, so both tabs show an unavailable placeholder. | ||
|
|
||
| ## Build it yourself | ||
|
|
||
| Both must be set, or the app boots in the development environment. | ||
| ```bash | ||
| git clone https://github.com/RunanywhereAI/runanywhere-web.git | ||
| cd runanywhere-web | ||
|
|
||
| ## Continuous integration | ||
| npm ci | ||
| npm run dev # http://localhost:3000 | ||
| ``` | ||
|
|
||
| `.github/workflows/ci.yml` runs on every push to `main` and every pull request: | ||
| `ubuntu-latest` and Node 24, then `npm ci`, `typecheck`, `lint`, `test`, `build`. | ||
| `npm ci` pulls the SDK and every WASM artifact with it. There is no separate WASM build step | ||
| and no Emscripten toolchain to install. You need Node 22.12 or newer. | ||
|
|
||
| CI installs with `npm ci`, the same command `vercel.json` uses, so a | ||
| `package-lock.json` out of sync with `package.json` fails the gate instead of | ||
| breaking production. Commit the regenerated lock with any dependency change: | ||
| `npm install` would quietly repair the lock locally and hide the breakage. | ||
| The one thing that trips people up: the page has to be cross-origin isolated for | ||
| `SharedArrayBuffer`. The dev server and `vercel.json` both send COOP `same-origin` and COEP | ||
| `require-corp`, and `public/coi-serviceworker.js` covers hosts that cannot. | ||
|
|
||
| ## Troubleshooting | ||
| [`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md) has the scripts table, the WASM artifact list, | ||
| deployment configuration, and troubleshooting. | ||
|
|
||
| | Symptom | Fix | | ||
| |---|---| | ||
| | `npm ci` fails with `Missing: @runanywhere/… from lock file` | `package.json` and `package-lock.json` drifted. Run `npm install` and commit the refreshed lock | | ||
| | `SharedArrayBuffer is not defined` | The page is not cross-origin isolated. Serve with COOP `same-origin` and COEP `require-corp` | | ||
| | Build fails naming missing `racommons-*` files | The SDK packages did not install completely. Re-run `npm ci` | | ||
| | Model download stalls or workers hang | Hard-reload to clear a stale service worker, then recheck the COOP/COEP headers | | ||
| | A WebGPU model produces garbage | Switch that model to the CPU variant | | ||
| ## Architecture | ||
|
|
||
| ## Related | ||
| Four npm packages, no monorepo checkout, no source aliases. What `npm install` resolves is | ||
| what the app runs against, currently `0.20.24`. | ||
|
|
||
| | Resource | Link | | ||
| |---|---| | ||
| | iOS example | [github.com/RunanywhereAI/runanywhere-ios](https://github.com/RunanywhereAI/runanywhere-ios) | | ||
| | Android example | [github.com/RunanywhereAI/runanywhere-android](https://github.com/RunanywhereAI/runanywhere-android) | | ||
| | Electron example | [github.com/RunanywhereAI/runanywhere-electron](https://github.com/RunanywhereAI/runanywhere-electron) | | ||
| | SDK monorepo | [github.com/RunanywhereAI/runanywhere-sdks](https://github.com/RunanywhereAI/runanywhere-sdks) | | ||
| ``` | ||
| index.html → main.ts → app.ts | ||
| boot screen SDK init shell, routing, views | ||
| │ | ||
| ┌─────────────────┴──────────────────┐ | ||
| │ @runanywhere/web │ lifecycle + facades | ||
| └─────────────────┬──────────────────┘ | ||
| │ | ||
| ┌─────────────────┼──────────────────┐ | ||
| │ │ │ | ||
| web-llamacpp web-onnx proto-ts | ||
| LLM · VLM STT·TTS·VAD·embed generated types | ||
| CPU + WebGPU CPU + WebGPU | ||
| │ | ||
| ▼ | ||
| racommons.wasm, the same C++ core | ||
| that ships in the Swift, Kotlin, and Electron apps | ||
| ``` | ||
|
Comment on lines
+94
to
+111
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win Add Both fences trigger MD040 because they do not declare a language.
🧰 Tools🪛 markdownlint-cli2 (0.23.2)[warning] 94-94: Fenced code blocks should have a language specified (MD040, fenced-code-language) 📍 Affects 2 files
🤖 Prompt for AI AgentsSource: Linters/SAST tools |
||
|
|
||
| Views call `RunAnywhere.*` and nothing deeper. They may import `@runanywhere/web` and | ||
| `@runanywhere/web/browser`, never `/internal` or `/backend`, and they do not reimplement | ||
| routing, storage, or inference rules in UI code. | ||
|
|
||
| Every surface has a URL fragment (`#/vision`, `#/benchmarks`), so a tab survives a refresh, a | ||
| pasted link, and the reload the isolation service worker performs on Safari. | ||
|
|
||
| | Reference | | | ||
| | --- | --- | | ||
| | Which SDK call each view makes | [`docs/reference/sdk-surface-by-view.md`](docs/reference/sdk-surface-by-view.md) | | ||
| | WASM artifacts and how they are copied | [`docs/reference/wasm-artifacts.md`](docs/reference/wasm-artifacts.md) | | ||
| | Building, deploying, troubleshooting | [`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md) | | ||
| | Contributor conventions | [`AGENTS.md`](AGENTS.md) | | ||
|
|
||
| ## The other apps | ||
|
|
||
| | Platform | Repo | | ||
| | --- | --- | | ||
| | iOS and macOS, Swift | [runanywhere-ios](https://github.com/RunanywhereAI/runanywhere-ios) | | ||
| | Android, Kotlin | [runanywhere-android](https://github.com/RunanywhereAI/runanywhere-android) | | ||
| | Windows, Electron | [runanywhere-electron](https://github.com/RunanywhereAI/runanywhere-electron) | | ||
| | SDK monorepo | [runanywhere-sdks](https://github.com/RunanywhereAI/runanywhere-sdks) | | ||
| | Documentation | [docs.runanywhere.ai](https://docs.runanywhere.ai) | | ||
| | Discord | [discord.gg/N359FBbDVd](https://discord.gg/N359FBbDVd) | | ||
| | Email | founders@runanywhere.ai | | ||
|
|
||
| ## License | ||
|
|
||
| RunAnywhere License, based on Apache 2.0 with additional commercial-use terms. | ||
| See [LICENSE](LICENSE). | ||
| RunAnywhere License, Apache 2.0 based with additional commercial-use terms. See | ||
| [LICENSE](LICENSE). | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
Repository: RunanywhereAI/runanywhere-web
Length of output: 50385
🏁 Script executed:
Repository: RunanywhereAI/runanywhere-web
Length of output: 36100
🏁 Script executed:
Repository: RunanywhereAI/runanywhere-web
Length of output: 37889
🌐 Web query:
@runanywhere/web 0.20.19 RunAnywhere.initialize baseUrl telemetry inference local browser💡 Result:
The
@runanywhere/webSDK (version 0.20.19) provides a framework for on-device AI inference in the browser [1][2]. Initialization is a multi-step process that involves configuring the core SDK followed by registering specific inference backends [3][4]. To initialize the SDK, use the RunAnywhere.initialize method [3][4]: await RunAnywhere.initialize({ environment: SDKEnvironment.Development, debug: true, baseURL: 'https://your-api-base-url.com', // Optional: Base URL for API requests // acceleration: 'auto', // Optional: 'auto', 'webgpu', or 'cpu' }); The initialize method accepts an object with several configuration options: - environment: Specifies the SDK environment (e.g., SDKEnvironment.Development, SDKEnvironment.Staging, SDKEnvironment.Production) [3][4]. - debug: A boolean to enable or disable debug logging [3][4]. - baseURL: An optional string used as the base URL for API requests [3]. - acceleration: An optional preference for hardware acceleration, such as 'webgpu' or 'cpu' [3]. Following core initialization, you must register the required backend packages (e.g.,@runanywhere/web-llamacppor@runanywhere/web-onnx) to enable specific inference capabilities like LLMs, VLM, STT, or TTS [3][4][5]. Registration is idempotent and loads the necessary WebAssembly (WASM) binaries [3][4]. The SDK uses the browser's Origin Private File System (OPFS) for local storage, allowing models to persist across page reloads and browser restarts [3][5]. Telemetry and logging should be managed according to the SDK's environment settings, with care taken not to log sensitive credentials in production [6]. All components, including the core and backend packages, should be kept at compatible versions (e.g., 0.20.19) [1][7].Citations:
🏁 Script executed:
Repository: RunanywhereAI/runanywhere-web
Length of output: 50386
🏁 Script executed:
Repository: RunanywhereAI/runanywhere-web
Length of output: 50386
🏁 Script executed:
Repository: RunanywhereAI/runanywhere-web
Length of output: 12730
Scope the privacy statement to the execution path.
textto the architecture diagram fence to satisfy MD040.🤖 Prompt for AI Agents