Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 36 additions & 59 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,42 @@
## Unreleased
## 0.8.20

* Added `code_assets` 2.x compatibility while retaining support for 1.x native
asset toolchains.

* Updated the default llama.cpp native runtime pin to
`leehack/llamadart-native@b10514`, adding BailingMoE3,
GraniteSWA/GraniteMoeSWA, speculators-format DSpark checkpoints, and current
upstream multimodal/backend fixes and performance improvements. Regenerated
matching Dart FFI bindings, refreshed the `llamadart_llama_cpp_flutter`
Apple SwiftPM checksum, and aligned current native-runtime documentation.
upstream multimodal/backend improvements. Matching Dart FFI bindings and the
`llamadart_llama_cpp_flutter` Apple SwiftPM artifact were refreshed.

* Updated the default WebGPU bridge assets to `v0.1.37`, embedding llama.cpp
`b10514` to restore native/Web parity. The bridge provisions an explicit 1 MiB
Wasm stack for both wasm32 and memory64, preventing the `b10514` graph
parameter growth from aborting Qwen3-ASR memory64 context construction.
Wasm stack for wasm32 and memory64, preventing Qwen3-ASR memory64 context
construction from overflowing the default stack.

* Improved Web microphone transcription by warming up browser capture before
showing the recording-ready state, trimming the warmup silence, and
rejecting too-short, silent, or unsupported PCM WAV captures before
inference.

* Added logical batch-size (`n_batch`) and micro-batch-size (`n_ubatch`)
controls for llama.cpp/WebGPU models in the Flutter chat example. Android
Auto now probes the packaged Vulkan device before memory planning so capable
models do not fall back to CPU merely because the dynamic GPU module was not
registered yet.

* Updated the native LiteRT-LM runtime to `v0.16.0-native.2` and added an
experimental dedicated `SpeechToTextEngine.liteRtLm` path with runtime
capability discovery, worker-isolated CPU inference, bounded mono 16 kHz
float PCM input, partial/final transcript events, finalization, and
cooperative cancellation. The low-level synchronous ASR session remains
available for advanced integrations. The Apple companion packages the iOS
Gemma constraint provider and Metal accelerator/sampler plugins required by
the published runtime.
controls for llama.cpp/WebGPU models in the Flutter chat example.

* Improved Android Auto backend selection by probing the packaged Vulkan device
before memory planning, avoiding unnecessary CPU fallback on capable models.

* Updated the native LiteRT-LM runtime to `v0.16.0-native.2`. The Apple companion
packages the iOS Gemma constraint provider and Metal accelerator/sampler
plugins required by the published runtime.

* Added an experimental `SpeechToTextEngine.liteRtLm` path with capability
discovery, worker-isolated CPU inference, bounded mono 16 kHz float PCM,
partial/final transcript events, finalization, and cancellation.

* Added experimental live dictation to the native Flutter chat example for
chat models, including generic audio-chat models. The app offers the
recommended 54 MB Moonshine Tiny sidecar and an optional higher-capacity,
heavier 615 MB Parakeet TDT 0.6B sidecar with checksum-pinned assets,
persistent selection, explicit download progress, cancellation, and retry.
It captures mono 16 kHz PCM, uses the public worker-isolated streaming speech
API, renders confirmed and pending text, and returns the finalized transcript
to the editable composer. A persisted settings switch explains and enables
or disables the optional workflow. The first path is CPU-only,
English-only, capped at five minutes, and unavailable on Linux and Web.
chat models using selectable Moonshine Tiny and Parakeet TDT 0.6B sidecars.
Live dictation is CPU-only, English-only, capped at five minutes, and
unavailable on Linux and Web.

* Improved Flutter chat example model downloads with bounded retries for
transient network failures, safe resume after truncated responses, and a
Expand All @@ -52,44 +47,26 @@
and stopped streaming responses from pulling users away from chat history.

* Added an experimental typed `TextToSpeechEngine` for native llama.cpp and
WebGPU bridge assets `v0.1.33+` with Qwen3-TTS models, including capability
discovery, language and optional speaker
reference input, cancellable progress, complete 24 kHz PCM output, and WAV
encoding. The Flutter chat example adds a checksum-pinned Qwen3-TTS preset
with file or microphone speaker references, automatic playback, replay, and
WAV save controls. Web accepts byte-backed speaker references and complete
PCM/WAV output, with a bounded example-app utterance length for browser
stability. The example pins bridge assets `v0.1.37`, retaining the `v0.1.34`
recovery that retries a failed worker WebGPU synthesis once on CPU; current
LiteRT-LM artifacts fail explicitly as unsupported. Apple companion builds
now discover the native TTS wrapper ABI from the embedded llama framework.
WebGPU with Qwen3-TTS models, including capability discovery, speaker
references, cancellable progress, complete 24 kHz PCM output, and WAV
encoding. The Flutter chat example adds synthesis, playback, replay, and WAV
save controls. Apple apps discover the TTS ABI in the embedded llama
framework; current LiteRT-LM artifacts fail explicitly as unsupported.

* Added an experimental typed `SpeechToTextEngine` with an explicit Qwen3-ASR
adapter profile for whole-file llama.cpp transcription. The Flutter chat
example includes a SHA-256-verified Qwen3-ASR 0.6B preset plus separate
**Transcribe Audio** and foreground microphone recording flows on native and
WebGPU. Web requires validated bridge assets `v0.1.30+`, pins `v0.1.37` (with
the short-speech recovery introduced in `v0.1.32` and the `b10514` memory64
stack fix) to
recover short speech that would otherwise terminate empty, accepts WAV bytes
only, and remains separate from native LiteRT-LM live dictation. The native
flow has been exercised on a physical Pixel with CPU inference and in the
iOS Simulator.
example includes a checksum-pinned Qwen3-ASR 0.6B preset plus file and
microphone transcription on native and WebGPU. Web accepts WAV bytes only;
native LiteRT-LM live dictation remains a separate implementation.

* Added **Ask with voice** to the native Flutter chat example for Gemma 4 E2B
through both the LiteRT-LM direct-media bundle and the audio-capable GGUF +
projector path. The app sends a microphone recording through normal
multimodal chat so the model can answer the spoken request; it remains
separate from typed speech-to-text and is code-supported on Android, iOS,
macOS, and Windows, with Linux recording and Web excluded. Current packaged
microphone validation covers LiteRT-LM on macOS; llama.cpp audio input is
experimental and has engine-level Metal evidence only. Android, iOS, and
Windows device validation remains outstanding.
through LiteRT-LM direct media and audio-capable GGUF + projector paths. It
sends a microphone recording through multimodal chat and remains separate
from typed speech-to-text.

* Added experimental, opt-in native llama.cpp DSpark speculative decoding
through `SpeculativeDecodingConfig.draftDspark(...)`, including exact
`draft-dspark` mapping, external draft-model validation, typed unsupported
failures, and benchmark coverage.
through `SpeculativeDecodingConfig.draftDspark(...)` with a compatible
external draft model.

## 0.8.19

Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,17 +55,17 @@ For Dart or Flutter apps:

```yaml
dependencies:
llamadart: ^0.8.19
llamadart: ^0.8.20
```

Flutter iOS/macOS apps that should link Apple XCFrameworks through Swift
Package Manager should also add the runtime companion packages they need:

```yaml
dependencies:
llamadart: ^0.8.19
llamadart_llama_cpp_flutter: ^0.0.13 # GGUF / llama.cpp
llamadart_litert_lm_flutter: ^0.0.9 # Apple .litertlm / LiteRT-LM targets
llamadart: ^0.8.20
llamadart_llama_cpp_flutter: ^0.0.14 # GGUF / llama.cpp
llamadart_litert_lm_flutter: ^0.0.10 # Apple .litertlm / LiteRT-LM targets
```

The LiteRT-LM companion manifest includes the complete iOS SwiftPM runtime
Expand Down
6 changes: 3 additions & 3 deletions example/chat_app/pubspec.lock
Original file line number Diff line number Diff line change
Expand Up @@ -413,21 +413,21 @@ packages:
path: "../.."
relative: true
source: path
version: "0.8.19"
version: "0.8.20"
llamadart_litert_lm_flutter:
dependency: "direct main"
description:
path: "../../packages/llamadart_litert_lm_flutter"
relative: true
source: path
version: "0.0.9"
version: "0.0.10"
llamadart_llama_cpp_flutter:
dependency: "direct main"
description:
path: "../../packages/llamadart_llama_cpp_flutter"
relative: true
source: path
version: "0.0.13"
version: "0.0.14"
logging:
dependency: transitive
description:
Expand Down
3 changes: 2 additions & 1 deletion packages/llamadart_litert_lm_flutter/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
## Unreleased
## 0.0.10

* Updated Apple SwiftPM native pin to `leehack/litert-lm-native@v0.16.0-native.2`
and packaged the iOS Gemma constraint provider and Metal accelerator/sampler
plugins required by that runtime.
* Kept macOS on the complete hook-managed runtime bundle instead of linking an
incomplete SwiftPM subset.
* Updated the install example for `llamadart` 0.8.20.

## 0.0.9

Expand Down
4 changes: 2 additions & 2 deletions packages/llamadart_litert_lm_flutter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ also request an x86_64 Simulator slice must exclude x86_64 for LiteRT-LM builds.

```yaml
dependencies:
llamadart: ^0.8.19
llamadart_litert_lm_flutter: ^0.0.9
llamadart: ^0.8.20
llamadart_litert_lm_flutter: ^0.0.10
```

This package has no runtime Dart API of its own. Import `package:llamadart`
Expand Down
2 changes: 1 addition & 1 deletion packages/llamadart_litert_lm_flutter/pubspec.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: llamadart_litert_lm_flutter
description: Flutter Apple SwiftPM runtime companion package for llamadart LiteRT-LM support.
version: 0.0.9
version: 0.0.10
repository: https://github.com/leehack/llamadart/tree/main/packages/llamadart_litert_lm_flutter
homepage: https://github.com/leehack/llamadart
issue_tracker: https://github.com/leehack/llamadart/issues
Expand Down
2 changes: 1 addition & 1 deletion packages/llamadart_llama_cpp_flutter/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## Unreleased
## 0.0.14

* Updated Apple SwiftPM native pin to `leehack/llamadart-native@b10514`.

Expand Down
4 changes: 2 additions & 2 deletions packages/llamadart_llama_cpp_flutter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ core package's native-assets fallback.

```yaml
dependencies:
llamadart: ^0.8.19
llamadart_llama_cpp_flutter: ^0.0.13
llamadart: ^0.8.20
llamadart_llama_cpp_flutter: ^0.0.14
```

This package has no runtime Dart API of its own. Import `package:llamadart`
Expand Down
2 changes: 1 addition & 1 deletion packages/llamadart_llama_cpp_flutter/pubspec.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: llamadart_llama_cpp_flutter
description: Flutter Apple SwiftPM runtime companion package for llamadart llama.cpp and GGUF support.
version: 0.0.13
version: 0.0.14
repository: https://github.com/leehack/llamadart/tree/main/packages/llamadart_llama_cpp_flutter
homepage: https://github.com/leehack/llamadart
issue_tracker: https://github.com/leehack/llamadart/issues
Expand Down
4 changes: 2 additions & 2 deletions pubspec.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: llamadart
description: Dart and Flutter local LLM inference with llama.cpp GGUF and LiteRT-LM across native platforms and web.
version: 0.8.19
version: 0.8.20
homepage: https://github.com/leehack/llamadart
repository: https://github.com/leehack/llamadart
issue_tracker: https://github.com/leehack/llamadart/issues
Expand Down Expand Up @@ -29,7 +29,7 @@ dependencies:
http: ^1.1.0
crypto: ^3.0.0
web: ^1.0.0
code_assets: ^1.0.0
code_assets: '>=1.0.0 <3.0.0'
hooks: '>=1.0.0 <3.0.0'
logging: ^1.3.0
dinja: ^1.0.0
Expand Down
62 changes: 28 additions & 34 deletions website/docs/changelog/recent-releases.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,65 +7,59 @@ For canonical full release notes, use:

- [`CHANGELOG.md`](https://github.com/leehack/llamadart/blob/main/CHANGELOG.md)

## Unreleased
## 0.8.20

- Updated WebGPU bridge assets to `v0.1.37` (llama.cpp `b10514`), restoring
native/Web parity and provisioning an explicit 1 MiB Wasm stack for wasm32
and memory64 so `b10514` graph-parameter growth does not abort Qwen3-ASR
memory64 context construction.
and memory64 so Qwen3-ASR memory64 context construction does not overflow the
default stack.

- Improved Web microphone transcription with browser-capture warmup trimming
and early short, silent, and unsupported PCM WAV diagnostics.

- Added logical and micro-batch controls for llama.cpp/WebGPU models to the
Flutter chat example, and made Android Auto probe the packaged Vulkan device
before choosing GPU offload.
Flutter chat example.

- Updated the native LiteRT-LM runtime to `v0.16.0-native.2` and added an
experimental dedicated `SpeechToTextEngine.liteRtLm` path with bounded mono
16 kHz float PCM input, partial/final transcript events, worker-isolated CPU
inference, backpressure, and cancellation. The Apple companion also packages
the iOS Gemma constraint provider and Metal plugins required by the published
runtime.
- Made Android Auto probe the packaged Vulkan device before choosing GPU
offload, avoiding unnecessary CPU fallback on capable models.

- Updated the native LiteRT-LM runtime to `v0.16.0-native.2`; the Apple companion
packages the iOS Gemma constraint provider and Metal plugins required by the
published runtime.

- Added an experimental `SpeechToTextEngine.liteRtLm` path with bounded mono
16 kHz float PCM, partial/final transcript events, worker-isolated CPU
inference, backpressure, and cancellation.

- Added experimental live English dictation to native Flutter chat models,
including generic audio-chat models, using selectable checksum-pinned
Moonshine Tiny (recommended, 54 MB) and Parakeet TDT 0.6B (optional, 615 MB)
LiteRT sidecars. The UI reports determinate download progress and supports
cancellation/retry; the public worker-isolated streaming STT API produces
confirmed/pending text and editable composer finalization. A persisted
settings switch explains and enables or disables the optional workflow.
Audio-chat models retain
**Ask with voice** as a separate action.
LiteRT sidecars. Live dictation is CPU-only, English-only, capped at five
minutes, and unavailable on Linux and Web. Audio-chat models retain **Ask
with voice** as a separate action.

- Improved Flutter chat example model downloads with bounded retries for
transient network failures, safe resume after truncated responses, and a
distinct integrity-verification state after transfer reaches 100%.
distinct integrity-verification state after transfer reaches 100%. The
redesigned onboarding and Lab surfaces preserve model-card position while
downloads reorder and keep streaming responses from pulling users away from
chat history.

- Added experimental typed Qwen3-TTS synthesis on native llama.cpp and WebGPU
bridge assets `v0.1.33+`, including
capability discovery, cancellation, complete PCM/WAV output, and a dedicated
synthesize workflow with file or microphone speaker references, automatic
playback, replay, and WAV export in the Flutter chat example. Web accepts
byte-backed speaker references and complete PCM/WAV output, with a bounded
example-app utterance length for browser stability. The example pins bridge
assets `v0.1.37`, retaining the `v0.1.34` recovery that retries a failed
worker WebGPU synthesis once on CPU; Apple companion builds discover the
native TTS wrapper ABI from the embedded llama framework, while current
LiteRT-LM artifacts remain unsupported.
with capability discovery, cancellation, speaker references, complete
PCM/WAV output, and synthesis/playback/export controls in the Flutter chat
example. Apple apps discover the TTS ABI in the embedded llama framework;
current LiteRT-LM artifacts remain unsupported.

- Updated the native llama.cpp runtime to `b10514`, adding BailingMoE3,
GraniteSWA/GraniteMoeSWA, speculators-format DSpark checkpoints, and current
upstream multimodal/backend fixes and performance improvements. Matching Dart
FFI bindings and the Apple SwiftPM artifact checksum were refreshed.

- Added an experimental typed Qwen3-ASR whole-file transcription workflow, a
SHA-256-verified Qwen3-ASR 0.6B native-and-Web chat-app preset, and foreground
microphone recording. WebGPU requires validated bridge assets `v0.1.30+`,
pins `v0.1.37` with the `v0.1.32` short-speech recovery and the `b10514`
memory64 stack fix,
and accepts WAV bytes only; native LiteRT-LM live dictation remains a
separate implementation.
checksum-pinned Qwen3-ASR 0.6B native-and-Web chat-app preset, and file and
microphone transcription. Web accepts WAV bytes only; native LiteRT-LM live
dictation remains a separate implementation.

- Added **Ask with voice** to the native Flutter chat example for Gemma 4 E2B
LiteRT-LM and audio-capable GGUF models. It sends a short microphone recording
Expand Down
8 changes: 4 additions & 4 deletions website/docs/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,17 @@ In Xcode, set `IPHONEOS_DEPLOYMENT_TARGET = 16.4` or

```yaml
dependencies:
llamadart: ^0.8.19
llamadart: ^0.8.20
```

For Flutter iOS/macOS apps that should link Apple XCFrameworks through Swift
Package Manager, also add the runtime companion packages you need:

```yaml
dependencies:
llamadart: ^0.8.19
llamadart_llama_cpp_flutter: ^0.0.13 # GGUF / llama.cpp
llamadart_litert_lm_flutter: ^0.0.9 # Apple .litertlm / LiteRT-LM targets
llamadart: ^0.8.20
llamadart_llama_cpp_flutter: ^0.0.14 # GGUF / llama.cpp
llamadart_litert_lm_flutter: ^0.0.10 # Apple .litertlm / LiteRT-LM targets
```

The companion packages are published independently from the `packages/`
Expand Down