diff --git a/README.md b/README.md index fdc842d1..84b5ceb0 100644 --- a/README.md +++ b/README.md @@ -1,182 +1,141 @@ -# RunAnywhere web example - -A browser app built on the RunAnywhere Web SDK. Models download to the browser -and run there via WebAssembly, with a WebGPU path where the browser and the -model support it. Prompts, audio, images, and documents stay on the device. - -## What works - -| Surface | What it does | SDK entry point | -|---|---|---| -| Assistant | Streaming chat, tool calling, saved conversations | `RunAnywhere.llm.generateStream`, `RunAnywhere.llm.tools` | -| Talk | Full voice session (VAD, STT, LLM, TTS) | `RunAnywhere.voice.createSession` | -| Image & Live | Describe a photo or a live camera frame | `RunAnywhere.vlm.generateStream` | -| Transcribe | Batch and streaming speech to text | `RunAnywhere.stt.transcribe`, `.transcribeStream` | -| Read Aloud | Speak arbitrary text | `RunAnywhere.tts.speak` | -| Voice Activity | Streaming speech detection | `RunAnywhere.vad.detectStream` | -| Documents | RAG over `.txt`, `.md`, and `.json` files you drop in | `RunAnywhere.rag.open` | -| Solutions | Two packaged YAML pipelines: voice agent and document Q&A | `RunAnywhere.solutions.run` | -| Benchmarks | One prompt at three token budgets (50, 256, 512), charted | `RunAnywhere.llm.generateStream` | -| Downloads | Model registry, disk usage, storage folder | `RunAnywhere.storage`, `RunAnywhere.models` | -| Settings | Generation preferences, API credentials, Hugging Face token | `RunAnywhere.setHuggingFaceToken` | - -Segmentation and Diarization have views and SDK calls wired, but no browser -engine registers those capabilities, so the catalog is empty for both and each -tab renders an unavailable placeholder. - -## Requirements - -| Item | Minimum | -|---|---| -| Node.js | 22.12 (CI runs 24) | -| Browser | Chrome or Edge 86, Safari, Firefox | -| Cross-origin isolation | Required for `SharedArrayBuffer`. The dev server and `vercel.json` send COOP `same-origin` and COEP `require-corp`; `public/coi-serviceworker.js` covers hosts that cannot | -| Disk space | Hundreds of megabytes to a few gigabytes for downloaded models, held in OPFS | - -## Run it - -```bash -git clone https://github.com/RunanywhereAI/runanywhere-web.git -cd runanywhere-web - -npm ci -npm run dev # http://localhost:3000 -``` - -`npm ci` pulls the SDK and its WASM artifacts. There is no separate WASM build -step and no Emscripten toolchain to install. - -| Script | What it does | -|---|---| -| `npm run dev` | Vite dev server on `localhost:3000` (strict port) with COOP/COEP headers | -| `npm run build` | Production bundle into `dist/`, including the Emscripten `.js`/`.wasm` pairs | -| `npm run preview` | Serve the built `dist/` on `localhost:3000` | -| `npm run typecheck` | `tsc --noEmit` | -| `npm run lint` | ESLint over `src`, zero warnings tolerated | -| `npm run test` | Vitest over `src/**/*.test.ts` | -| `npm run release:build` | `build` followed by `release:verify` | -| `npm run release:verify` | Assert `dist/` holds every required runtime file | -| `npm run release:deploy` | Build, verify, and deploy a prebuilt static bundle to Vercel | - -## SDK packages - -Everything comes from the npm registry. There are no `file:` links, no `paths` -aliases, and no monorepo checkout, so `npm install` is the only thing that -decides which SDK version the app runs against. Both the TypeScript modules and -every WASM artifact come out of `node_modules/@runanywhere/*`. - -```jsonc -"dependencies": { - "@runanywhere/proto-ts": "^0.20.19", - "@runanywhere/web": "^0.20.19", - "@runanywhere/web-llamacpp": "^0.20.19", - "@runanywhere/web-onnx": "^0.20.19" -} -``` - -| Package | Role | -|---|---| -| `@runanywhere/web` | SDK lifecycle and the inference facades. `@runanywhere/web/browser` adds `AudioCapture`, `AudioFileLoader`, and `VideoCapture` | -| `@runanywhere/web-llamacpp` | LLM and VLM backend registration, CPU and WebGPU builds | -| `@runanywhere/web-onnx` | Sherpa-ONNX backend registration for STT, TTS, VAD, and embeddings | -| `@runanywhere/proto-ts` | Generated protobuf types for models, events, errors, and modalities | +# RunAnywhere AI for the web -To try an unreleased SDK build, `npm install` a packed tarball or use -`npm link`. Do not reintroduce a source alias. +

+ RunAnywhere +

-## WASM artifacts +

+ + Try it in your browser + +

-Five JS/WASM pairs ship across the three SDK packages. `vite.config.ts` copies -each canonical pair into `dist/assets/` next to Vite's hashed copy, because -Emscripten's pthread glue starts its workers from the original filename. +

+ WebAssembly + WebGPU + TypeScript + RunAnywhere License +

-| Pair | Package | -|---|---| -| `racommons.{js,wasm}` | `@runanywhere/web` | -| `racommons-llamacpp.{js,wasm}` | `@runanywhere/web-llamacpp` | -| `racommons-llamacpp-webgpu.{js,wasm}` | `@runanywhere/web-llamacpp` | -| `racommons-onnx-sherpa.{js,wasm}` | `@runanywhere/web-onnx` | -| `racommons-onnx-sherpa-webgpu.{js,wasm}` | `@runanywhere/web-onnx` | +The RunAnywhere consumer app for the browser, written in TypeScript. -A production build fails naming the missing files rather than shipping a bundle -that only breaks after deployment. +There is nothing to install. Open the page, pick a model, and it downloads into your browser +and runs there through WebAssembly, with a WebGPU path where your browser and the model both +support it. Whatever you type, say, or upload stays in the tab. -## Project layout +## Try it -``` -runanywhere-web/ - index.html Vite entry, plus the pre-paint theme script and boot screen - src/ - main.ts Boot: cross-origin isolation, SDK init, backend registration, catalog - app.ts Shell, drawer navigation, hash routing, the Advanced hub - views/ One file per surface: chat, vision, voice, transcribe, speak, vad, - segmentation, diarization, documents, storage, solutions, - benchmarks, settings - services/ Model catalog, engine availability, conversation store (IndexedDB), - Hugging Face client, markdown, formatting helpers - components/ Model selection sheet, dialogs, file drop, icons, shared notices - styles/ design-system.css is the only token layer - public/coi-serviceworker.js Cross-origin-isolation fallback - scripts/release.sh Static release verify, stage, and deploy - tests/ Manual browser test plan - vite.config.ts Dev/preview COOP-COEP headers, WASM copy plugin, chrome86 target - vercel.json COOP/COEP headers and SPA rewrites -``` +**[runanywhere-web-demo.vercel.app](https://runanywhere-web-demo.vercel.app)** -Every surface has a URL fragment (`#/vision`, `#/benchmarks`), so a tab survives -a refresh, a pasted link, and the reload the isolation service worker performs -on Safari. +Works in Chrome or Edge 86 and newer, Safari, and Firefox. Models are held in OPFS on your +own disk, so the second visit starts instantly. -Views may import `@runanywhere/web` and `@runanywhere/web/browser`. They must -not reach into `@runanywhere/web/internal` or `@runanywhere/web/backend`, and -must not reimplement SDK routing, storage, or inference rules in UI code. See -`AGENTS.md`. + -## Configuration +## What it looks like -Settings holds an API key and base URL for the session only; neither is written -to storage. For a hosted deployment, Vite reads two build-time variables and -boots the SDK straight into the production environment: +Captured in a Chromium browser with WebGPU active, running PrismML Bonsai 1.7B through the +llama.cpp WASM build. -| Variable | Meaning | +| | | |---|---| -| `VITE_RUNANYWHERE_API_KEY` | Publishable browser key. Never a server-side secret, since Vite inlines it into the bundle | -| `VITE_RUNANYWHERE_BASE_URL` | Production API origin | +| ![Chat](docs/screenshots/01-chat.jpg) | ![Choosing a model](docs/screenshots/02-model-picker.jpg) | +| An answer generated in the browser tab. The badge in the corner shows WebGPU is in use. | The picker recommends a model for the machine and reports what the browser can do. | +| ![Talk](docs/screenshots/03-voice.jpg) | ![SDK utilities](docs/screenshots/04-advanced.jpg) | +| Talk assembles a speech-to-text, chat, text-to-speech and voice-detection model, then runs the conversation in the tab. | The lower-level surfaces: transcribe, read aloud, voice activity, diarization, segmentation, benchmarks, solutions. | +| ![Downloads](docs/screenshots/05-downloads.jpg) | ![Settings](docs/screenshots/06-settings.jpg) | +| What is on disk and what it costs. Browser storage is private to the site unless you pick a folder. | Sampling, the system prompt, and optional API credentials. | + +The image files are in [`docs/screenshots/`](docs/screenshots). + +## What you can do + +| | | +| --- | --- | +| **Assistant** | Streaming chat with tool calling and saved conversations | +| **Talk** | A full voice session: it listens, thinks, and answers out loud | +| **Image and Live** | Describe a photo, or a live camera frame | +| **Transcribe** | Batch and streaming speech to text | +| **Read aloud** | Speak any text you give it | +| **Documents** | Drop in `.txt`, `.md`, or `.json` and ask questions about them | +| **Solutions** | Two packaged pipelines: a voice agent and document Q&A | +| **Benchmarks** | One prompt at three token budgets, charted | +| **Downloads** | Model registry, disk usage, storage folder | + +Segmentation and Diarization have views wired up, but no browser engine registers those +capabilities yet, so both tabs show an unavailable placeholder. + +## Build it yourself -Both must be set, or the app boots in the development environment. +```bash +git clone https://github.com/RunanywhereAI/runanywhere-web.git +cd runanywhere-web -## Continuous integration +npm ci +npm run dev # http://localhost:3000 +``` -`.github/workflows/ci.yml` runs on every push to `main` and every pull request: -`ubuntu-latest` and Node 24, then `npm ci`, `typecheck`, `lint`, `test`, `build`. +`npm ci` pulls the SDK and every WASM artifact with it. There is no separate WASM build step +and no Emscripten toolchain to install. You need Node 22.12 or newer. -CI installs with `npm ci`, the same command `vercel.json` uses, so a -`package-lock.json` out of sync with `package.json` fails the gate instead of -breaking production. Commit the regenerated lock with any dependency change: -`npm install` would quietly repair the lock locally and hide the breakage. +The one thing that trips people up: the page has to be cross-origin isolated for +`SharedArrayBuffer`. The dev server and `vercel.json` both send COOP `same-origin` and COEP +`require-corp`, and `public/coi-serviceworker.js` covers hosts that cannot. -## Troubleshooting +[`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md) has the scripts table, the WASM artifact list, +deployment configuration, and troubleshooting. -| Symptom | Fix | -|---|---| -| `npm ci` fails with `Missing: @runanywhere/… from lock file` | `package.json` and `package-lock.json` drifted. Run `npm install` and commit the refreshed lock | -| `SharedArrayBuffer is not defined` | The page is not cross-origin isolated. Serve with COOP `same-origin` and COEP `require-corp` | -| Build fails naming missing `racommons-*` files | The SDK packages did not install completely. Re-run `npm ci` | -| Model download stalls or workers hang | Hard-reload to clear a stale service worker, then recheck the COOP/COEP headers | -| A WebGPU model produces garbage | Switch that model to the CPU variant | +## Architecture -## Related +Four npm packages, no monorepo checkout, no source aliases. What `npm install` resolves is +what the app runs against, currently `0.20.24`. -| Resource | Link | -|---|---| -| iOS example | [github.com/RunanywhereAI/runanywhere-ios](https://github.com/RunanywhereAI/runanywhere-ios) | -| Android example | [github.com/RunanywhereAI/runanywhere-android](https://github.com/RunanywhereAI/runanywhere-android) | -| Electron example | [github.com/RunanywhereAI/runanywhere-electron](https://github.com/RunanywhereAI/runanywhere-electron) | -| SDK monorepo | [github.com/RunanywhereAI/runanywhere-sdks](https://github.com/RunanywhereAI/runanywhere-sdks) | +``` + index.html → main.ts → app.ts + boot screen SDK init shell, routing, views + │ + ┌─────────────────┴──────────────────┐ + │ @runanywhere/web │ lifecycle + facades + └─────────────────┬──────────────────┘ + │ + ┌─────────────────┼──────────────────┐ + │ │ │ + web-llamacpp web-onnx proto-ts + LLM · VLM STT·TTS·VAD·embed generated types + CPU + WebGPU CPU + WebGPU + │ + ▼ + racommons.wasm, the same C++ core + that ships in the Swift, Kotlin, and Electron apps +``` + +Views call `RunAnywhere.*` and nothing deeper. They may import `@runanywhere/web` and +`@runanywhere/web/browser`, never `/internal` or `/backend`, and they do not reimplement +routing, storage, or inference rules in UI code. + +Every surface has a URL fragment (`#/vision`, `#/benchmarks`), so a tab survives a refresh, a +pasted link, and the reload the isolation service worker performs on Safari. + +| Reference | | +| --- | --- | +| Which SDK call each view makes | [`docs/reference/sdk-surface-by-view.md`](docs/reference/sdk-surface-by-view.md) | +| WASM artifacts and how they are copied | [`docs/reference/wasm-artifacts.md`](docs/reference/wasm-artifacts.md) | +| Building, deploying, troubleshooting | [`docs/DEVELOPMENT.md`](docs/DEVELOPMENT.md) | +| Contributor conventions | [`AGENTS.md`](AGENTS.md) | + +## The other apps + +| Platform | Repo | +| --- | --- | +| iOS and macOS, Swift | [runanywhere-ios](https://github.com/RunanywhereAI/runanywhere-ios) | +| Android, Kotlin | [runanywhere-android](https://github.com/RunanywhereAI/runanywhere-android) | +| Windows, Electron | [runanywhere-electron](https://github.com/RunanywhereAI/runanywhere-electron) | +| SDK monorepo | [runanywhere-sdks](https://github.com/RunanywhereAI/runanywhere-sdks) | +| Documentation | [docs.runanywhere.ai](https://docs.runanywhere.ai) | | Discord | [discord.gg/N359FBbDVd](https://discord.gg/N359FBbDVd) | -| Email | founders@runanywhere.ai | ## License -RunAnywhere License, based on Apache 2.0 with additional commercial-use terms. -See [LICENSE](LICENSE). +RunAnywhere License, Apache 2.0 based with additional commercial-use terms. See +[LICENSE](LICENSE). diff --git a/docs/DEVELOPMENT.md b/docs/DEVELOPMENT.md new file mode 100644 index 00000000..5cb1d3dd --- /dev/null +++ b/docs/DEVELOPMENT.md @@ -0,0 +1,146 @@ +# Development reference + +Detail moved out of the root README so it stays a consumer-facing page. Everything here +is about building and shipping the app, not about using it. + +## Requirements + +| Item | Minimum | +|---|---| +| Node.js | 22.12 (CI runs 24) | +| Browser | Chrome or Edge 86, Safari, Firefox | +| Cross-origin isolation | Required for `SharedArrayBuffer`. The dev server and `vercel.json` send COOP `same-origin` and COEP `require-corp`; `public/coi-serviceworker.js` covers hosts that cannot | +| Disk space | Hundreds of megabytes to a few gigabytes for downloaded models, held in OPFS | + +## Run it + +```bash +git clone https://github.com/RunanywhereAI/runanywhere-web.git +cd runanywhere-web + +npm ci +npm run dev # http://localhost:3000 +``` + +`npm ci` pulls the SDK and its WASM artifacts. There is no separate WASM build +step and no Emscripten toolchain to install. + +| Script | What it does | +|---|---| +| `npm run dev` | Vite dev server on `localhost:3000` (strict port) with COOP/COEP headers | +| `npm run build` | Production bundle into `dist/`, including the Emscripten `.js`/`.wasm` pairs | +| `npm run preview` | Serve the built `dist/` on `localhost:3000` | +| `npm run typecheck` | `tsc --noEmit` | +| `npm run lint` | ESLint over `src`, zero warnings tolerated | +| `npm run test` | Vitest over `src/**/*.test.ts` | +| `npm run release:build` | `build` followed by `release:verify` | +| `npm run release:verify` | Assert `dist/` holds every required runtime file | +| `npm run release:deploy` | Build, verify, and deploy a prebuilt static bundle to Vercel | + +## SDK packages + +Everything comes from the npm registry. There are no `file:` links, no `paths` +aliases, and no monorepo checkout, so `npm install` is the only thing that +decides which SDK version the app runs against. Both the TypeScript modules and +every WASM artifact come out of `node_modules/@runanywhere/*`. + +```jsonc +"dependencies": { + "@runanywhere/proto-ts": "^0.20.24", + "@runanywhere/web": "^0.20.24", + "@runanywhere/web-llamacpp": "^0.20.24", + "@runanywhere/web-onnx": "^0.20.24" +} +``` + +| Package | Role | +|---|---| +| `@runanywhere/web` | SDK lifecycle and the inference facades. `@runanywhere/web/browser` adds `AudioCapture`, `AudioFileLoader`, and `VideoCapture` | +| `@runanywhere/web-llamacpp` | LLM and VLM backend registration, CPU and WebGPU builds | +| `@runanywhere/web-onnx` | Sherpa-ONNX backend registration for STT, TTS, VAD, and embeddings | +| `@runanywhere/proto-ts` | Generated protobuf types for models, events, errors, and modalities | + +To try an unreleased SDK build, `npm install` a packed tarball or use +`npm link`. Do not reintroduce a source alias. + +## WASM artifacts + +Five JS/WASM pairs ship across the three SDK packages. `vite.config.ts` copies +each canonical pair into `dist/assets/` next to Vite's hashed copy, because +Emscripten's pthread glue starts its workers from the original filename. + +| Pair | Package | +|---|---| +| `racommons.{js,wasm}` | `@runanywhere/web` | +| `racommons-llamacpp.{js,wasm}` | `@runanywhere/web-llamacpp` | +| `racommons-llamacpp-webgpu.{js,wasm}` | `@runanywhere/web-llamacpp` | +| `racommons-onnx-sherpa.{js,wasm}` | `@runanywhere/web-onnx` | +| `racommons-onnx-sherpa-webgpu.{js,wasm}` | `@runanywhere/web-onnx` | + +A production build fails naming the missing files rather than shipping a bundle +that only breaks after deployment. + +## Project layout + +``` +runanywhere-web/ + index.html Vite entry, plus the pre-paint theme script and boot screen + src/ + main.ts Boot: cross-origin isolation, SDK init, backend registration, catalog + app.ts Shell, drawer navigation, hash routing, the Advanced hub + views/ One file per surface: chat, vision, voice, transcribe, speak, vad, + segmentation, diarization, documents, storage, solutions, + benchmarks, settings + services/ Model catalog, engine availability, conversation store (IndexedDB), + Hugging Face client, markdown, formatting helpers + components/ Model selection sheet, dialogs, file drop, icons, shared notices + styles/ design-system.css is the only token layer + public/coi-serviceworker.js Cross-origin-isolation fallback + scripts/release.sh Static release verify, stage, and deploy + tests/ Manual browser test plan + vite.config.ts Dev/preview COOP-COEP headers, WASM copy plugin, chrome86 target + vercel.json COOP/COEP headers and SPA rewrites +``` + +Every surface has a URL fragment (`#/vision`, `#/benchmarks`), so a tab survives +a refresh, a pasted link, and the reload the isolation service worker performs +on Safari. + +Views may import `@runanywhere/web` and `@runanywhere/web/browser`. They must +not reach into `@runanywhere/web/internal` or `@runanywhere/web/backend`, and +must not reimplement SDK routing, storage, or inference rules in UI code. See +`AGENTS.md`. + +## Configuration + +Settings holds an API key and base URL for the session only; neither is written +to storage. For a hosted deployment, Vite reads two build-time variables and +boots the SDK straight into the production environment: + +| Variable | Meaning | +|---|---| +| `VITE_RUNANYWHERE_API_KEY` | Publishable browser key. Never a server-side secret, since Vite inlines it into the bundle | +| `VITE_RUNANYWHERE_BASE_URL` | Production API origin | + +Both must be set, or the app boots in the development environment. + +## Continuous integration + +`.github/workflows/ci.yml` runs on every push to `main` and every pull request: +`ubuntu-latest` and Node 24, then `npm ci`, `typecheck`, `lint`, `test`, `build`. + +CI installs with `npm ci`, the same command `vercel.json` uses, so a +`package-lock.json` out of sync with `package.json` fails the gate instead of +breaking production. Commit the regenerated lock with any dependency change: +`npm install` would quietly repair the lock locally and hide the breakage. + +## Troubleshooting + +| Symptom | Fix | +|---|---| +| `npm ci` fails with `Missing: @runanywhere/… from lock file` | `package.json` and `package-lock.json` drifted. Run `npm install` and commit the refreshed lock | +| `SharedArrayBuffer is not defined` | The page is not cross-origin isolated. Serve with COOP `same-origin` and COEP `require-corp` | +| Build fails naming missing `racommons-*` files | The SDK packages did not install completely. Re-run `npm ci` | +| Model download stalls or workers hang | Hard-reload to clear a stale service worker, then recheck the COOP/COEP headers | +| A WebGPU model produces garbage | Switch that model to the CPU variant | + diff --git a/docs/screenshots/01-chat.jpg b/docs/screenshots/01-chat.jpg new file mode 100644 index 00000000..5d1566a3 Binary files /dev/null and b/docs/screenshots/01-chat.jpg differ diff --git a/docs/screenshots/02-model-picker.jpg b/docs/screenshots/02-model-picker.jpg new file mode 100644 index 00000000..046a88d3 Binary files /dev/null and b/docs/screenshots/02-model-picker.jpg differ diff --git a/docs/screenshots/03-voice.jpg b/docs/screenshots/03-voice.jpg new file mode 100644 index 00000000..06969e3a Binary files /dev/null and b/docs/screenshots/03-voice.jpg differ diff --git a/docs/screenshots/04-advanced.jpg b/docs/screenshots/04-advanced.jpg new file mode 100644 index 00000000..49f2e63b Binary files /dev/null and b/docs/screenshots/04-advanced.jpg differ diff --git a/docs/screenshots/05-downloads.jpg b/docs/screenshots/05-downloads.jpg new file mode 100644 index 00000000..36cdd68b Binary files /dev/null and b/docs/screenshots/05-downloads.jpg differ diff --git a/docs/screenshots/06-settings.jpg b/docs/screenshots/06-settings.jpg new file mode 100644 index 00000000..78cf518b Binary files /dev/null and b/docs/screenshots/06-settings.jpg differ