diff --git a/CHANGELOG.md b/CHANGELOG.md index a56d5118..1a498207 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,16 +1,18 @@ -## Unreleased +## 0.8.20 + +* Added `code_assets` 2.x compatibility while retaining support for 1.x native + asset toolchains. * Updated the default llama.cpp native runtime pin to `leehack/llamadart-native@b10514`, adding BailingMoE3, GraniteSWA/GraniteMoeSWA, speculators-format DSpark checkpoints, and current - upstream multimodal/backend fixes and performance improvements. Regenerated - matching Dart FFI bindings, refreshed the `llamadart_llama_cpp_flutter` - Apple SwiftPM checksum, and aligned current native-runtime documentation. + upstream multimodal/backend improvements. Matching Dart FFI bindings and the + `llamadart_llama_cpp_flutter` Apple SwiftPM artifact were refreshed. * Updated the default WebGPU bridge assets to `v0.1.37`, embedding llama.cpp `b10514` to restore native/Web parity. The bridge provisions an explicit 1 MiB - Wasm stack for both wasm32 and memory64, preventing the `b10514` graph - parameter growth from aborting Qwen3-ASR memory64 context construction. + Wasm stack for wasm32 and memory64, preventing Qwen3-ASR memory64 context + construction from overflowing the default stack. * Improved Web microphone transcription by warming up browser capture before showing the recording-ready state, trimming the warmup silence, and @@ -18,30 +20,23 @@ inference. * Added logical batch-size (`n_batch`) and micro-batch-size (`n_ubatch`) - controls for llama.cpp/WebGPU models in the Flutter chat example. Android - Auto now probes the packaged Vulkan device before memory planning so capable - models do not fall back to CPU merely because the dynamic GPU module was not - registered yet. - -* Updated the native LiteRT-LM runtime to `v0.16.0-native.2` and added an - experimental dedicated `SpeechToTextEngine.liteRtLm` path with runtime - capability discovery, worker-isolated CPU inference, bounded mono 16 kHz - float PCM input, partial/final transcript events, finalization, and - cooperative cancellation. The low-level synchronous ASR session remains - available for advanced integrations. The Apple companion packages the iOS - Gemma constraint provider and Metal accelerator/sampler plugins required by - the published runtime. + controls for llama.cpp/WebGPU models in the Flutter chat example. + +* Improved Android Auto backend selection by probing the packaged Vulkan device + before memory planning, avoiding unnecessary CPU fallback on capable models. + +* Updated the native LiteRT-LM runtime to `v0.16.0-native.2`. The Apple companion + packages the iOS Gemma constraint provider and Metal accelerator/sampler + plugins required by the published runtime. + +* Added an experimental `SpeechToTextEngine.liteRtLm` path with capability + discovery, worker-isolated CPU inference, bounded mono 16 kHz float PCM, + partial/final transcript events, finalization, and cancellation. * Added experimental live dictation to the native Flutter chat example for - chat models, including generic audio-chat models. The app offers the - recommended 54 MB Moonshine Tiny sidecar and an optional higher-capacity, - heavier 615 MB Parakeet TDT 0.6B sidecar with checksum-pinned assets, - persistent selection, explicit download progress, cancellation, and retry. - It captures mono 16 kHz PCM, uses the public worker-isolated streaming speech - API, renders confirmed and pending text, and returns the finalized transcript - to the editable composer. A persisted settings switch explains and enables - or disables the optional workflow. The first path is CPU-only, - English-only, capped at five minutes, and unavailable on Linux and Web. + chat models using selectable Moonshine Tiny and Parakeet TDT 0.6B sidecars. + Live dictation is CPU-only, English-only, capped at five minutes, and + unavailable on Linux and Web. * Improved Flutter chat example model downloads with bounded retries for transient network failures, safe resume after truncated responses, and a @@ -52,44 +47,26 @@ and stopped streaming responses from pulling users away from chat history. * Added an experimental typed `TextToSpeechEngine` for native llama.cpp and - WebGPU bridge assets `v0.1.33+` with Qwen3-TTS models, including capability - discovery, language and optional speaker - reference input, cancellable progress, complete 24 kHz PCM output, and WAV - encoding. The Flutter chat example adds a checksum-pinned Qwen3-TTS preset - with file or microphone speaker references, automatic playback, replay, and - WAV save controls. Web accepts byte-backed speaker references and complete - PCM/WAV output, with a bounded example-app utterance length for browser - stability. The example pins bridge assets `v0.1.37`, retaining the `v0.1.34` - recovery that retries a failed worker WebGPU synthesis once on CPU; current - LiteRT-LM artifacts fail explicitly as unsupported. Apple companion builds - now discover the native TTS wrapper ABI from the embedded llama framework. + WebGPU with Qwen3-TTS models, including capability discovery, speaker + references, cancellable progress, complete 24 kHz PCM output, and WAV + encoding. The Flutter chat example adds synthesis, playback, replay, and WAV + save controls. Apple apps discover the TTS ABI in the embedded llama + framework; current LiteRT-LM artifacts fail explicitly as unsupported. * Added an experimental typed `SpeechToTextEngine` with an explicit Qwen3-ASR adapter profile for whole-file llama.cpp transcription. The Flutter chat - example includes a SHA-256-verified Qwen3-ASR 0.6B preset plus separate - **Transcribe Audio** and foreground microphone recording flows on native and - WebGPU. Web requires validated bridge assets `v0.1.30+`, pins `v0.1.37` (with - the short-speech recovery introduced in `v0.1.32` and the `b10514` memory64 - stack fix) to - recover short speech that would otherwise terminate empty, accepts WAV bytes - only, and remains separate from native LiteRT-LM live dictation. The native - flow has been exercised on a physical Pixel with CPU inference and in the - iOS Simulator. + example includes a checksum-pinned Qwen3-ASR 0.6B preset plus file and + microphone transcription on native and WebGPU. Web accepts WAV bytes only; + native LiteRT-LM live dictation remains a separate implementation. * Added **Ask with voice** to the native Flutter chat example for Gemma 4 E2B - through both the LiteRT-LM direct-media bundle and the audio-capable GGUF + - projector path. The app sends a microphone recording through normal - multimodal chat so the model can answer the spoken request; it remains - separate from typed speech-to-text and is code-supported on Android, iOS, - macOS, and Windows, with Linux recording and Web excluded. Current packaged - microphone validation covers LiteRT-LM on macOS; llama.cpp audio input is - experimental and has engine-level Metal evidence only. Android, iOS, and - Windows device validation remains outstanding. + through LiteRT-LM direct media and audio-capable GGUF + projector paths. It + sends a microphone recording through multimodal chat and remains separate + from typed speech-to-text. * Added experimental, opt-in native llama.cpp DSpark speculative decoding - through `SpeculativeDecodingConfig.draftDspark(...)`, including exact - `draft-dspark` mapping, external draft-model validation, typed unsupported - failures, and benchmark coverage. + through `SpeculativeDecodingConfig.draftDspark(...)` with a compatible + external draft model. ## 0.8.19 diff --git a/README.md b/README.md index 47de70a1..8707ceef 100644 --- a/README.md +++ b/README.md @@ -55,7 +55,7 @@ For Dart or Flutter apps: ```yaml dependencies: - llamadart: ^0.8.19 + llamadart: ^0.8.20 ``` Flutter iOS/macOS apps that should link Apple XCFrameworks through Swift @@ -63,9 +63,9 @@ Package Manager should also add the runtime companion packages they need: ```yaml dependencies: - llamadart: ^0.8.19 - llamadart_llama_cpp_flutter: ^0.0.13 # GGUF / llama.cpp - llamadart_litert_lm_flutter: ^0.0.9 # Apple .litertlm / LiteRT-LM targets + llamadart: ^0.8.20 + llamadart_llama_cpp_flutter: ^0.0.14 # GGUF / llama.cpp + llamadart_litert_lm_flutter: ^0.0.10 # Apple .litertlm / LiteRT-LM targets ``` The LiteRT-LM companion manifest includes the complete iOS SwiftPM runtime diff --git a/example/chat_app/pubspec.lock b/example/chat_app/pubspec.lock index 97abd3c5..83db03ca 100644 --- a/example/chat_app/pubspec.lock +++ b/example/chat_app/pubspec.lock @@ -413,21 +413,21 @@ packages: path: "../.." relative: true source: path - version: "0.8.19" + version: "0.8.20" llamadart_litert_lm_flutter: dependency: "direct main" description: path: "../../packages/llamadart_litert_lm_flutter" relative: true source: path - version: "0.0.9" + version: "0.0.10" llamadart_llama_cpp_flutter: dependency: "direct main" description: path: "../../packages/llamadart_llama_cpp_flutter" relative: true source: path - version: "0.0.13" + version: "0.0.14" logging: dependency: transitive description: diff --git a/packages/llamadart_litert_lm_flutter/CHANGELOG.md b/packages/llamadart_litert_lm_flutter/CHANGELOG.md index 2ca04b83..b7a3a510 100644 --- a/packages/llamadart_litert_lm_flutter/CHANGELOG.md +++ b/packages/llamadart_litert_lm_flutter/CHANGELOG.md @@ -1,10 +1,11 @@ -## Unreleased +## 0.0.10 * Updated Apple SwiftPM native pin to `leehack/litert-lm-native@v0.16.0-native.2` and packaged the iOS Gemma constraint provider and Metal accelerator/sampler plugins required by that runtime. * Kept macOS on the complete hook-managed runtime bundle instead of linking an incomplete SwiftPM subset. +* Updated the install example for `llamadart` 0.8.20. ## 0.0.9 diff --git a/packages/llamadart_litert_lm_flutter/README.md b/packages/llamadart_litert_lm_flutter/README.md index 719b278c..a313dc45 100644 --- a/packages/llamadart_litert_lm_flutter/README.md +++ b/packages/llamadart_litert_lm_flutter/README.md @@ -15,8 +15,8 @@ also request an x86_64 Simulator slice must exclude x86_64 for LiteRT-LM builds. ```yaml dependencies: - llamadart: ^0.8.19 - llamadart_litert_lm_flutter: ^0.0.9 + llamadart: ^0.8.20 + llamadart_litert_lm_flutter: ^0.0.10 ``` This package has no runtime Dart API of its own. Import `package:llamadart` diff --git a/packages/llamadart_litert_lm_flutter/pubspec.yaml b/packages/llamadart_litert_lm_flutter/pubspec.yaml index 4686043c..0ddcaee3 100644 --- a/packages/llamadart_litert_lm_flutter/pubspec.yaml +++ b/packages/llamadart_litert_lm_flutter/pubspec.yaml @@ -1,6 +1,6 @@ name: llamadart_litert_lm_flutter description: Flutter Apple SwiftPM runtime companion package for llamadart LiteRT-LM support. -version: 0.0.9 +version: 0.0.10 repository: https://github.com/leehack/llamadart/tree/main/packages/llamadart_litert_lm_flutter homepage: https://github.com/leehack/llamadart issue_tracker: https://github.com/leehack/llamadart/issues diff --git a/packages/llamadart_llama_cpp_flutter/CHANGELOG.md b/packages/llamadart_llama_cpp_flutter/CHANGELOG.md index 33bdbc0b..236d70cb 100644 --- a/packages/llamadart_llama_cpp_flutter/CHANGELOG.md +++ b/packages/llamadart_llama_cpp_flutter/CHANGELOG.md @@ -1,4 +1,4 @@ -## Unreleased +## 0.0.14 * Updated Apple SwiftPM native pin to `leehack/llamadart-native@b10514`. diff --git a/packages/llamadart_llama_cpp_flutter/README.md b/packages/llamadart_llama_cpp_flutter/README.md index b3c7a9a4..1328c089 100644 --- a/packages/llamadart_llama_cpp_flutter/README.md +++ b/packages/llamadart_llama_cpp_flutter/README.md @@ -9,8 +9,8 @@ core package's native-assets fallback. ```yaml dependencies: - llamadart: ^0.8.19 - llamadart_llama_cpp_flutter: ^0.0.13 + llamadart: ^0.8.20 + llamadart_llama_cpp_flutter: ^0.0.14 ``` This package has no runtime Dart API of its own. Import `package:llamadart` diff --git a/packages/llamadart_llama_cpp_flutter/pubspec.yaml b/packages/llamadart_llama_cpp_flutter/pubspec.yaml index d2f23848..01e70402 100644 --- a/packages/llamadart_llama_cpp_flutter/pubspec.yaml +++ b/packages/llamadart_llama_cpp_flutter/pubspec.yaml @@ -1,6 +1,6 @@ name: llamadart_llama_cpp_flutter description: Flutter Apple SwiftPM runtime companion package for llamadart llama.cpp and GGUF support. -version: 0.0.13 +version: 0.0.14 repository: https://github.com/leehack/llamadart/tree/main/packages/llamadart_llama_cpp_flutter homepage: https://github.com/leehack/llamadart issue_tracker: https://github.com/leehack/llamadart/issues diff --git a/pubspec.yaml b/pubspec.yaml index bfac731b..0b6307fe 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -1,6 +1,6 @@ name: llamadart description: Dart and Flutter local LLM inference with llama.cpp GGUF and LiteRT-LM across native platforms and web. -version: 0.8.19 +version: 0.8.20 homepage: https://github.com/leehack/llamadart repository: https://github.com/leehack/llamadart issue_tracker: https://github.com/leehack/llamadart/issues @@ -29,7 +29,7 @@ dependencies: http: ^1.1.0 crypto: ^3.0.0 web: ^1.0.0 - code_assets: ^1.0.0 + code_assets: '>=1.0.0 <3.0.0' hooks: '>=1.0.0 <3.0.0' logging: ^1.3.0 dinja: ^1.0.0 diff --git a/website/docs/changelog/recent-releases.md b/website/docs/changelog/recent-releases.md index f41deaff..0f70e948 100644 --- a/website/docs/changelog/recent-releases.md +++ b/website/docs/changelog/recent-releases.md @@ -7,52 +7,49 @@ For canonical full release notes, use: - [`CHANGELOG.md`](https://github.com/leehack/llamadart/blob/main/CHANGELOG.md) -## Unreleased +## 0.8.20 - Updated WebGPU bridge assets to `v0.1.37` (llama.cpp `b10514`), restoring native/Web parity and provisioning an explicit 1 MiB Wasm stack for wasm32 - and memory64 so `b10514` graph-parameter growth does not abort Qwen3-ASR - memory64 context construction. + and memory64 so Qwen3-ASR memory64 context construction does not overflow the + default stack. - Improved Web microphone transcription with browser-capture warmup trimming and early short, silent, and unsupported PCM WAV diagnostics. - Added logical and micro-batch controls for llama.cpp/WebGPU models to the - Flutter chat example, and made Android Auto probe the packaged Vulkan device - before choosing GPU offload. + Flutter chat example. -- Updated the native LiteRT-LM runtime to `v0.16.0-native.2` and added an - experimental dedicated `SpeechToTextEngine.liteRtLm` path with bounded mono - 16 kHz float PCM input, partial/final transcript events, worker-isolated CPU - inference, backpressure, and cancellation. The Apple companion also packages - the iOS Gemma constraint provider and Metal plugins required by the published - runtime. +- Made Android Auto probe the packaged Vulkan device before choosing GPU + offload, avoiding unnecessary CPU fallback on capable models. + +- Updated the native LiteRT-LM runtime to `v0.16.0-native.2`; the Apple companion + packages the iOS Gemma constraint provider and Metal plugins required by the + published runtime. + +- Added an experimental `SpeechToTextEngine.liteRtLm` path with bounded mono + 16 kHz float PCM, partial/final transcript events, worker-isolated CPU + inference, backpressure, and cancellation. - Added experimental live English dictation to native Flutter chat models, including generic audio-chat models, using selectable checksum-pinned Moonshine Tiny (recommended, 54 MB) and Parakeet TDT 0.6B (optional, 615 MB) - LiteRT sidecars. The UI reports determinate download progress and supports - cancellation/retry; the public worker-isolated streaming STT API produces - confirmed/pending text and editable composer finalization. A persisted - settings switch explains and enables or disables the optional workflow. - Audio-chat models retain - **Ask with voice** as a separate action. + LiteRT sidecars. Live dictation is CPU-only, English-only, capped at five + minutes, and unavailable on Linux and Web. Audio-chat models retain **Ask + with voice** as a separate action. - Improved Flutter chat example model downloads with bounded retries for transient network failures, safe resume after truncated responses, and a - distinct integrity-verification state after transfer reaches 100%. + distinct integrity-verification state after transfer reaches 100%. The + redesigned onboarding and Lab surfaces preserve model-card position while + downloads reorder and keep streaming responses from pulling users away from + chat history. - Added experimental typed Qwen3-TTS synthesis on native llama.cpp and WebGPU - bridge assets `v0.1.33+`, including - capability discovery, cancellation, complete PCM/WAV output, and a dedicated - synthesize workflow with file or microphone speaker references, automatic - playback, replay, and WAV export in the Flutter chat example. Web accepts - byte-backed speaker references and complete PCM/WAV output, with a bounded - example-app utterance length for browser stability. The example pins bridge - assets `v0.1.37`, retaining the `v0.1.34` recovery that retries a failed - worker WebGPU synthesis once on CPU; Apple companion builds discover the - native TTS wrapper ABI from the embedded llama framework, while current - LiteRT-LM artifacts remain unsupported. + with capability discovery, cancellation, speaker references, complete + PCM/WAV output, and synthesis/playback/export controls in the Flutter chat + example. Apple apps discover the TTS ABI in the embedded llama framework; + current LiteRT-LM artifacts remain unsupported. - Updated the native llama.cpp runtime to `b10514`, adding BailingMoE3, GraniteSWA/GraniteMoeSWA, speculators-format DSpark checkpoints, and current @@ -60,12 +57,9 @@ For canonical full release notes, use: FFI bindings and the Apple SwiftPM artifact checksum were refreshed. - Added an experimental typed Qwen3-ASR whole-file transcription workflow, a - SHA-256-verified Qwen3-ASR 0.6B native-and-Web chat-app preset, and foreground - microphone recording. WebGPU requires validated bridge assets `v0.1.30+`, - pins `v0.1.37` with the `v0.1.32` short-speech recovery and the `b10514` - memory64 stack fix, - and accepts WAV bytes only; native LiteRT-LM live dictation remains a - separate implementation. + checksum-pinned Qwen3-ASR 0.6B native-and-Web chat-app preset, and file and + microphone transcription. Web accepts WAV bytes only; native LiteRT-LM live + dictation remains a separate implementation. - Added **Ask with voice** to the native Flutter chat example for Gemma 4 E2B LiteRT-LM and audio-capable GGUF models. It sends a short microphone recording diff --git a/website/docs/getting-started/installation.md b/website/docs/getting-started/installation.md index 6b68998e..f0a50f9c 100644 --- a/website/docs/getting-started/installation.md +++ b/website/docs/getting-started/installation.md @@ -27,7 +27,7 @@ In Xcode, set `IPHONEOS_DEPLOYMENT_TARGET = 16.4` or ```yaml dependencies: - llamadart: ^0.8.19 + llamadart: ^0.8.20 ``` For Flutter iOS/macOS apps that should link Apple XCFrameworks through Swift @@ -35,9 +35,9 @@ Package Manager, also add the runtime companion packages you need: ```yaml dependencies: - llamadart: ^0.8.19 - llamadart_llama_cpp_flutter: ^0.0.13 # GGUF / llama.cpp - llamadart_litert_lm_flutter: ^0.0.9 # Apple .litertlm / LiteRT-LM targets + llamadart: ^0.8.20 + llamadart_llama_cpp_flutter: ^0.0.14 # GGUF / llama.cpp + llamadart_litert_lm_flutter: ^0.0.10 # Apple .litertlm / LiteRT-LM targets ``` The companion packages are published independently from the `packages/`