From 200dc553fc5ebe2074d1de424c167b56a57c3cd5 Mon Sep 17 00:00:00 2001 From: Matt Elliott Date: Tue, 18 Aug 2026 11:31:01 -0400 Subject: [PATCH 1/3] fix(local-ai-use): endpoint paths outside /api/v1, STT troubleshooting, per-modality fallback, and model-pull costs Verified against `lemonade-server 11.5.2` (~24.04 packaging). - reference.md invites extending the /api/v1 rule to other modalities but documents only the three it covers. Added the two routes that break the pattern: Anthropic Messages is /v1/messages only (/api/v1/messages 404s), and rerank is exposed as /api/v1/reranking (/v1/rerank, the spelling used by Jina, Cohere, vLLM, and llama.cpp, 404s on the proxy). Also confirms the harmless case: for the three modalities this skill does cover, /api/v1/ and /v1/ are aliases, so the mixed spellings elsewhere need no change. - Add measured STT 500 recovery rows (RuntimeDirectory clearing, whisper-server startup failure on the ROCm backend's unbundled comgr dependency), a Step 1c per-modality backend-health check, a per-modality fallback policy (STT may fall back with disclosure; image/TTS must not fall back silently), and a completion-checklist item so a working chat path is no longer treated as proof that transcription works. - Note the ~5 GB SD-Turbo pull on first image request, state the version this skill was verified against (model IDs and system-info fields move between releases), and point --no-install at the managed/shared-machine case where the agent must not run sudo apt-get install. - reference.md: confirm model IDs against GET /api/v1/models rather than a stale server_models.json snapshot. - Rule template: flag the SD-Turbo pull and GPU contention on the image row (coresident is fine, concurrent inference is not). Co-Authored-By: Claude Opus 5 Co-authored-by: Cursor --- skills/local-ai-use/SKILL.md | 51 ++++++++++++++++--- skills/local-ai-use/reference.md | 31 +++++++++++ .../local-ai-use/templates/local-ai-rule.md | 20 ++++++-- 3 files changed, 90 insertions(+), 12 deletions(-) diff --git a/skills/local-ai-use/SKILL.md b/skills/local-ai-use/SKILL.md index 8840ef8..f424c0c 100644 --- a/skills/local-ai-use/SKILL.md +++ b/skills/local-ai-use/SKILL.md @@ -76,9 +76,15 @@ instead. than launching it. On Linux/macOS the install needs `sudo`. Pass `--no-install` if the user wants to install it themselves instead. - **Disk:** ~8 GB free for the three default models (SD-Turbo + Whisper-Tiny - + kokoro-v1), plus ~0.1 GB for the installer itself. + + kokoro-v1), plus ~0.1 GB for the installer itself. The first image request + also triggers a ~5 GB pull for `SD-Turbo` if it is not already cached; on + metered or slow links, consider pulling models eagerly after setup (see + `lemonade pull` in `reference.md`). - **Network:** required for the install download and the first `lemonade pull` of each model. After that, every modality runs offline. +- **Tested against:** Lemonade **11.5.2** (requires v10.1.0 or newer). Model + IDs and `system-info` fields can change between releases; confirm against the + version actually installed. ## The opinionated path @@ -90,6 +96,10 @@ until each step verifies. [ ] 2. Install the routing rule into the workspace AGENTS.md ``` +On a managed or shared machine where the agent must not run +`sudo apt-get install`, pass `--no-install` to the setup script and confirm +Lemonade is already installed before continuing. + The single command that does both steps in one shot is: ```bash @@ -114,7 +124,8 @@ each step fails. ## Step 1: ensure Lemonade Server is installed and running `scripts/setup_local_ai.py` handles this end to end, but here is what it does -so you can do it by hand or debug it: +so you can do it by hand or debug it. Pass `--no-install` when Lemonade is +already managed elsewhere and the agent must not attempt a package install. **1a. Is a modern `lemonade` CLI installed?** Run `lemonade status`. The check is by *capability*, not by name: modern Lemonade prints `Server is running...` @@ -164,6 +175,21 @@ and no API key is required (the system-wide server defaults to no auth on loopback). If the user has set `LEMONADE_API_KEY`, the routing rule template in `templates/local-ai-rule.md` shows where to add the `Authorization` header. +**1c. Are the backends ready per modality?** Backend health is **per +modality**. A working chat or image request does not prove transcription will +work: `auto` picks a different backend per modality and can silently fall back +for one while having no alternative for another. Before declaring setup +complete, check the actual per-modality state: + +```bash +lemonade backends --all +``` + +Any variant the workspace's routing depends on should read `installed`. If the +only installed variant for a modality is `rocm`, install the Vulkan variant as +well so `auto` has somewhere to fall back to (for example, +`lemonade backends install whispercpp:vulkan`). + ### Default modality models (pulled on first use, not during setup) Setup does **not** download these. The installed rule pulls each one the first @@ -235,10 +261,15 @@ message. The rule explicitly tells the agent: cloud TTS providers (OpenAI TTS, ElevenLabs, etc.). - **For speech-to-text:** call `POST /api/v1/audio/transcriptions`. Do **not** call cloud transcription providers. -- **Fallback:** only fall back to a cloud API after one local attempt has - failed *and* the user has been told the local call failed. Never - silently fall back; the whole point of this skill is to keep cost - predictable. +- **Fallback is per modality, not one global rule.** For image generation and + text-to-speech, only fall back to a cloud API after one local attempt has + failed *and* the user has been told the local call failed. Never silently; + the whole point of this skill is to keep cost predictable. Speech-to-text may + also fall back with disclosure, but the disclosure must say the transcript + came from a different engine, since mixed-engine transcripts should not be + compared or deduplicated. See `templates/local-ai-rule.md` for the full + per-modality table, which also covers embeddings/reranking/index-backed + modalities for anyone extending this rule beyond the three shipped here. The agent's own text reasoning continues to use whatever LLM Cursor / Claude Code / Codex is configured with. This skill does not redirect chat tokens; @@ -255,7 +286,10 @@ machine. | `POST /v1/images/generations` returns 404 model not found | Image model not downloaded | `lemonade pull SD-Turbo` and retry. | | `lemonade pull` keeps printing `Progress: NN%` but never finishes | Download target is a bad path (out of space, no write permission, quota, read-only mount). The write error may surface only in the server log while the console keeps showing progress | Check the target and free space first: `GET /api/v1/system-info` reports `models_dir` and `model_storage.free_bytes`. If a pull stalls, read the recent lines of the server log (typically `lemonade-server.log` in the OS temp dir) for the real error (e.g. a download/write failure like `CURL code 23`, or an out-of-space message), then point the download at a writable disk with room. | | Image generation is slow on CPU (~4–5 min) | sd-cpp on CPU backend | Install the GPU backend on supported AMD hardware: `lemonade backends install sd-cpp:rocm`. | +| Still slow after installing the GPU backend | The backend is installed but not actually engaged; the runtime fell back to CPU silently | An `installed` state in `system-info` and a successful `rocminfo` both still permit a silent CPU fallback. Check real GPU utilisation (`gpu_busy_percent`) during a request, and confirm the host's GPU driver stack rather than re-installing the backend. | | `POST /v1/audio/transcriptions` returns 400 unsupported format | Input is not 16 kHz mono WAV | Re-encode with `ffmpeg -i in.* -ar 16000 -ac 1 out.wav`. | +| `POST /v1/audio/transcriptions` returns `500 model_load_error` with `Unable to resolve writable runtime directory from XDG_RUNTIME_DIR or RUNTIME_DIRECTORY` | The `lemond` unit's `RuntimeDirectory=` has been cleared by a local drop-in. `whispercpp` needs it; `llamacpp` and `kokoro` do not, so chat and TTS keep working while this happens | Restore `RuntimeDirectory=lemonade` in the drop-in (`/etc/systemd/system/lemond.service.d/`), then `sudo systemctl daemon-reload && sudo systemctl restart lemond`. The packaged unit already sets it. | +| `POST /v1/audio/transcriptions` returns `500 model_load_error` with `whisper-server failed to start or become ready` | The whisper backend process aborted at startup. On Linux with the ROCm backend this is usually the unbundled comgr dependency: `whispercpp:rocm` ships `libamd_comgr_loader.so*` but not the `libamd_comgr.so.3` those stubs `dlopen`, which exists only in system ROCm | Check the server log for `exit code: 134` and `implib-gen: libamd_comgr.so.3: failed to load library`. Install the self-contained Vulkan backend instead (`lemonade backends install whispercpp:vulkan`) and retry; Lemonade switches the per-model backend itself. Do not work around it by exporting `LD_LIBRARY_PATH=/opt/rocm/lib` into the service. | | `POST /v1/audio/speech` returns 404 | TTS model not downloaded | `lemonade pull kokoro-v1`. | | 401 Unauthorized on every request | User has set `LEMONADE_API_KEY` | Add `Authorization: Bearer $LEMONADE_API_KEY` to every request and to the rule block. | @@ -271,9 +305,12 @@ Mark this skill complete only when **all** of the following are true: - [ ] On a follow-up turn, asking the agent to "generate an image of X" causes it to POST to `http://localhost:13305/api/v1/images/generations` (pulling the model on first use) rather than calling a cloud tool. +- [ ] `lemonade backends --all` shows `installed` for every backend variant + this workspace's routing depends on (see Step 1c). Do not treat a working + image or chat path as proof that transcription will work. If any box is unchecked, the user is still paying cloud cost for at least -one modality. +one modality, or a routed modality may fail silently on first use. --- diff --git a/skills/local-ai-use/reference.md b/skills/local-ai-use/reference.md index 7b3e979..60ec7c2 100644 --- a/skills/local-ai-use/reference.md +++ b/skills/local-ai-use/reference.md @@ -21,6 +21,11 @@ The default trio (`SD-Turbo`, `kokoro-v1`, `Whisper-Tiny`) is sized for "keeps cost savings real on a typical laptop". Override only if the user asks for higher quality or has explicit hardware to spare. +Model IDs below were verified against Lemonade **11.5.2**. Catalogs move; +confirm any ID with `GET /api/v1/models` (add `?show_all=true` for the full +catalog) before writing it into the rule. Do not rely on a stale +`server_models.json` snapshot. + ### Image generation (`recipe: sd-cpp`) | Model | Approx size | When to use | Trade-off | @@ -64,6 +69,16 @@ Whisper requires 16 kHz mono PCM WAV input. Convert anything else first: ffmpeg -i input.mp3 -ar 16000 -ac 1 input.wav ``` +The HTTP endpoint adds no meaningful overhead over calling `whisper-cli` +directly on the same engine/backend/model. Throughput is equivalent; use the +endpoint unless you have a specific reason not to. + +Transcripts of long audio are not guaranteed byte-stable across identical +requests: Lemonade's server-side handling can vary chunk boundaries even when +the underlying whisper.cpp run is deterministic. Do not use a transcript as a +cache key or diff target; transcribe once and store the result if you need +reproducible output. + For full live coverage, run `lemonade list` after starting the server, or browse . @@ -103,6 +118,22 @@ Notable per-endpoint quirks: For the full parameter list of any endpoint, see `lemonade/docs/api/openai.md` upstream. +### If you extend the rule beyond these three modalities + +`/api/v1/` and `/v1/` are aliases for everything above, so the +two spellings used across this skill are interchangeable. Two routes outside +the three modalities do **not** follow that pattern, and matter if you point +the rule at chat, embeddings, or retrieval: + +| Route | Correct path | What 404s | +|---|---|---| +| Anthropic Messages | `/v1/messages` | `/api/v1/messages` | +| Reranking | `/api/v1/reranking` | `/api/v1/rerank`, `/v1/rerank` | + +`/v1/rerank` is the conventional spelling (Jina, Cohere, vLLM, llama.cpp), and +the per-model back-port lemond supervises does serve it — but the proxy on +`13305` does not. + --- ## API key handling diff --git a/skills/local-ai-use/templates/local-ai-rule.md b/skills/local-ai-use/templates/local-ai-rule.md index 16c1c46..68d6cc9 100644 --- a/skills/local-ai-use/templates/local-ai-rule.md +++ b/skills/local-ai-use/templates/local-ai-rule.md @@ -23,9 +23,9 @@ configured LLM. This rule only redirects the multimodal calls. | Endpoint | Model | Notes | |---|---|---| -| `/v1/images/generations` | `{{IMAGE_MODEL}}` | 4 steps, `cfg_scale: 1.0`, `512x512`, `response_format: "b64_json"`. | +| `/v1/images/generations` | `{{IMAGE_MODEL}}` | 4 steps, `cfg_scale: 1.0`, `512x512`, `response_format: "b64_json"`. First use pulls ~5 GB if the model is not cached. Image generation and LLM inference share the GPU: coresident models are generally fine on high-memory hosts, but **concurrent inference is not**. If a long-running job is in flight, ask before generating. | | `/v1/audio/speech` | `{{TTS_MODEL}}` | Default voice `shimmer`; `response_format: "mp3"`. | -| `/v1/audio/transcriptions` | `{{STT_MODEL}}` | Input must be 16 kHz mono WAV. Re-encode with `ffmpeg -i in.* -ar 16000 -ac 1 out.wav`. | +| `/v1/audio/transcriptions` | `{{STT_MODEL}}` | Input must be 16 kHz mono WAV. Re-encode with `ffmpeg -i in.* -ar 16000 -ac 1 out.wav`. Transcripts of long audio are **not** guaranteed byte-stable: two identical requests can return different text. Do not use a transcript as a cache key, content hash, or diff target. If a job needs reproducible output, transcribe once and store the result. | If `LEMONADE_API_KEY` is set in the environment, send `Authorization: Bearer $LEMONADE_API_KEY` on every request. Otherwise the @@ -92,9 +92,19 @@ curl -sX POST {{LEMONADE_BASE_URL}}/audio/transcriptions \ recent lines for the underlying error — for example a download/write error (such as `CURL code 23`) or an out-of-space message. Surface that line to the user rather than waiting through silent retries. -4. Only after that, ask the user before falling back to a cloud provider. - Never silently fall back; the whole point of this rule is predictable - cost. +4. Only after that, fallback policy is **per modality. Do not generalise + from one row to another:** + + | Modality | On local failure | + |---|---| + | Image generation | Tell the user the local call failed, then offer cloud. Outputs are interchangeable; mixing sources costs nothing but money. | + | Text-to-speech | Same. If files are kept side by side, record which engine produced each. | + | Speech-to-text | Offer cloud, and state that the transcript came from a different engine. Never mix engines inside one corpus without recording per-file provenance. | + | Embeddings, reranking, or any index-backed modality | **Never fall back automatically, even with disclosure.** Vectors and scores from a different model are not comparable to what is already in the index, so a partial fallback silently invalidates it. Stop, name the items that were not processed, and let the user choose between waiting for local and re-processing the whole corpus with one model. | + + Never silently fall back to any modality; disclosure is necessary but for + an index-backed modality it is not sufficient, because "local failed, + used cloud instead" reads as benign while it corrupts the index. ### Re-pointing to a different host From 251d7de3df56c1ee17ec60965a1d8bb41cc0c60a Mon Sep 17 00:00:00 2001 From: Matt Elliott Date: Fri, 21 Aug 2026 11:28:24 -0400 Subject: [PATCH 2/3] fix(local-ai-use): address review on endpoint/STT/fallback PR Addresses all six review threads on this PR: - Drop both STT 500 troubleshooting rows. The ROCm whisper-server abort (exit 134, missing libamd_comgr.so.3) was already reported upstream (lemonade-sdk/lemonade#2292) and fixed in #2293, an ancestor of the v11.6.0 this skill targets per `git compare`. Running the same lemond binary by hand (bypassing this dev box's custom systemd drop-in) transcribed successfully over ROCm; running it under that drop-in reproduced the abort, so the drop-in's sandboxing -- not Lemonade's packaging -- caused it. The RuntimeDirectory row was likewise self-inflicted by that same local drop-in; the packaged unit sets RuntimeDirectory= correctly. Neither belongs in the skill's general troubleshooting table. Step 1c's per-modality backend health check stays, since it catches this class of failure without depending on either bug. - Simplify the image-generation row's setup note: drop "ask before generating" (unnecessary agent-side indirection) and say "several GB" instead of a size tied to one specific model. - Drop the messages/reranking route-exception section from reference.md; out of scope while this skill only routes image/TTS/STT. - Collapse the per-modality fallback table to one statement, since the three shipped modalities all take the same policy (disclosed fallback is fine); the embeddings/reranking "never fall back" guidance belongs with those modalities if they're ever added, not baked in ahead of them. - Remove the STT byte-stability note duplicated between the rule template and reference.md, keeping the reference.md copy. Co-Authored-By: Claude Sonnet 5 --- skills/local-ai-use/SKILL.md | 15 +++++-------- skills/local-ai-use/reference.md | 16 -------------- .../local-ai-use/templates/local-ai-rule.md | 22 ++++++------------- 3 files changed, 12 insertions(+), 41 deletions(-) diff --git a/skills/local-ai-use/SKILL.md b/skills/local-ai-use/SKILL.md index f424c0c..517dfec 100644 --- a/skills/local-ai-use/SKILL.md +++ b/skills/local-ai-use/SKILL.md @@ -261,15 +261,12 @@ message. The rule explicitly tells the agent: cloud TTS providers (OpenAI TTS, ElevenLabs, etc.). - **For speech-to-text:** call `POST /api/v1/audio/transcriptions`. Do **not** call cloud transcription providers. -- **Fallback is per modality, not one global rule.** For image generation and - text-to-speech, only fall back to a cloud API after one local attempt has +- **Fallback:** only fall back to a cloud API after one local attempt has failed *and* the user has been told the local call failed. Never silently; - the whole point of this skill is to keep cost predictable. Speech-to-text may - also fall back with disclosure, but the disclosure must say the transcript - came from a different engine, since mixed-engine transcripts should not be - compared or deduplicated. See `templates/local-ai-rule.md` for the full - per-modality table, which also covers embeddings/reranking/index-backed - modalities for anyone extending this rule beyond the three shipped here. + the whole point of this skill is to keep cost predictable. For + speech-to-text, the disclosure must also say the transcript came from a + different engine, since mixed-engine transcripts should not be compared or + deduplicated. The agent's own text reasoning continues to use whatever LLM Cursor / Claude Code / Codex is configured with. This skill does not redirect chat tokens; @@ -288,8 +285,6 @@ machine. | Image generation is slow on CPU (~4–5 min) | sd-cpp on CPU backend | Install the GPU backend on supported AMD hardware: `lemonade backends install sd-cpp:rocm`. | | Still slow after installing the GPU backend | The backend is installed but not actually engaged; the runtime fell back to CPU silently | An `installed` state in `system-info` and a successful `rocminfo` both still permit a silent CPU fallback. Check real GPU utilisation (`gpu_busy_percent`) during a request, and confirm the host's GPU driver stack rather than re-installing the backend. | | `POST /v1/audio/transcriptions` returns 400 unsupported format | Input is not 16 kHz mono WAV | Re-encode with `ffmpeg -i in.* -ar 16000 -ac 1 out.wav`. | -| `POST /v1/audio/transcriptions` returns `500 model_load_error` with `Unable to resolve writable runtime directory from XDG_RUNTIME_DIR or RUNTIME_DIRECTORY` | The `lemond` unit's `RuntimeDirectory=` has been cleared by a local drop-in. `whispercpp` needs it; `llamacpp` and `kokoro` do not, so chat and TTS keep working while this happens | Restore `RuntimeDirectory=lemonade` in the drop-in (`/etc/systemd/system/lemond.service.d/`), then `sudo systemctl daemon-reload && sudo systemctl restart lemond`. The packaged unit already sets it. | -| `POST /v1/audio/transcriptions` returns `500 model_load_error` with `whisper-server failed to start or become ready` | The whisper backend process aborted at startup. On Linux with the ROCm backend this is usually the unbundled comgr dependency: `whispercpp:rocm` ships `libamd_comgr_loader.so*` but not the `libamd_comgr.so.3` those stubs `dlopen`, which exists only in system ROCm | Check the server log for `exit code: 134` and `implib-gen: libamd_comgr.so.3: failed to load library`. Install the self-contained Vulkan backend instead (`lemonade backends install whispercpp:vulkan`) and retry; Lemonade switches the per-model backend itself. Do not work around it by exporting `LD_LIBRARY_PATH=/opt/rocm/lib` into the service. | | `POST /v1/audio/speech` returns 404 | TTS model not downloaded | `lemonade pull kokoro-v1`. | | 401 Unauthorized on every request | User has set `LEMONADE_API_KEY` | Add `Authorization: Bearer $LEMONADE_API_KEY` to every request and to the rule block. | diff --git a/skills/local-ai-use/reference.md b/skills/local-ai-use/reference.md index 60ec7c2..375841a 100644 --- a/skills/local-ai-use/reference.md +++ b/skills/local-ai-use/reference.md @@ -118,22 +118,6 @@ Notable per-endpoint quirks: For the full parameter list of any endpoint, see `lemonade/docs/api/openai.md` upstream. -### If you extend the rule beyond these three modalities - -`/api/v1/` and `/v1/` are aliases for everything above, so the -two spellings used across this skill are interchangeable. Two routes outside -the three modalities do **not** follow that pattern, and matter if you point -the rule at chat, embeddings, or retrieval: - -| Route | Correct path | What 404s | -|---|---|---| -| Anthropic Messages | `/v1/messages` | `/api/v1/messages` | -| Reranking | `/api/v1/reranking` | `/api/v1/rerank`, `/v1/rerank` | - -`/v1/rerank` is the conventional spelling (Jina, Cohere, vLLM, llama.cpp), and -the per-model back-port lemond supervises does serve it — but the proxy on -`13305` does not. - --- ## API key handling diff --git a/skills/local-ai-use/templates/local-ai-rule.md b/skills/local-ai-use/templates/local-ai-rule.md index 68d6cc9..afa91c7 100644 --- a/skills/local-ai-use/templates/local-ai-rule.md +++ b/skills/local-ai-use/templates/local-ai-rule.md @@ -23,9 +23,9 @@ configured LLM. This rule only redirects the multimodal calls. | Endpoint | Model | Notes | |---|---|---| -| `/v1/images/generations` | `{{IMAGE_MODEL}}` | 4 steps, `cfg_scale: 1.0`, `512x512`, `response_format: "b64_json"`. First use pulls ~5 GB if the model is not cached. Image generation and LLM inference share the GPU: coresident models are generally fine on high-memory hosts, but **concurrent inference is not**. If a long-running job is in flight, ask before generating. | +| `/v1/images/generations` | `{{IMAGE_MODEL}}` | 4 steps, `cfg_scale: 1.0`, `512x512`, `response_format: "b64_json"`. First use pulls several GB if the model is not cached. | | `/v1/audio/speech` | `{{TTS_MODEL}}` | Default voice `shimmer`; `response_format: "mp3"`. | -| `/v1/audio/transcriptions` | `{{STT_MODEL}}` | Input must be 16 kHz mono WAV. Re-encode with `ffmpeg -i in.* -ar 16000 -ac 1 out.wav`. Transcripts of long audio are **not** guaranteed byte-stable: two identical requests can return different text. Do not use a transcript as a cache key, content hash, or diff target. If a job needs reproducible output, transcribe once and store the result. | +| `/v1/audio/transcriptions` | `{{STT_MODEL}}` | Input must be 16 kHz mono WAV. Re-encode with `ffmpeg -i in.* -ar 16000 -ac 1 out.wav`. | If `LEMONADE_API_KEY` is set in the environment, send `Authorization: Bearer $LEMONADE_API_KEY` on every request. Otherwise the @@ -92,19 +92,11 @@ curl -sX POST {{LEMONADE_BASE_URL}}/audio/transcriptions \ recent lines for the underlying error — for example a download/write error (such as `CURL code 23`) or an out-of-space message. Surface that line to the user rather than waiting through silent retries. -4. Only after that, fallback policy is **per modality. Do not generalise - from one row to another:** - - | Modality | On local failure | - |---|---| - | Image generation | Tell the user the local call failed, then offer cloud. Outputs are interchangeable; mixing sources costs nothing but money. | - | Text-to-speech | Same. If files are kept side by side, record which engine produced each. | - | Speech-to-text | Offer cloud, and state that the transcript came from a different engine. Never mix engines inside one corpus without recording per-file provenance. | - | Embeddings, reranking, or any index-backed modality | **Never fall back automatically, even with disclosure.** Vectors and scores from a different model are not comparable to what is already in the index, so a partial fallback silently invalidates it. Stop, name the items that were not processed, and let the user choose between waiting for local and re-processing the whole corpus with one model. | - - Never silently fall back to any modality; disclosure is necessary but for - an index-backed modality it is not sufficient, because "local failed, - used cloud instead" reads as benign while it corrupts the index. +4. Only after that, ask the user before falling back to a cloud provider. + Never silently fall back; the whole point of this rule is predictable + cost. For speech-to-text specifically, also disclose that the transcript + came from a different engine, since mixed-engine transcripts should not + be compared or deduplicated. ### Re-pointing to a different host From b0f8237160a9b62af51bede2dfef8e098486130a Mon Sep 17 00:00:00 2001 From: Matt Elliott Date: Fri, 21 Aug 2026 11:28:24 -0400 Subject: [PATCH 3/3] fix(local-ai-use): address review on endpoint/STT/fallback PR Addresses all six review threads on this PR, plus two follow-on cleanups: - Drop both STT 500 troubleshooting rows. The ROCm whisper-server abort (exit 134, missing libamd_comgr.so.3) was already reported upstream (lemonade-sdk/lemonade#2292) and fixed in #2293, an ancestor of the v11.6.0 this skill targets per `git compare`. Running the same lemond binary by hand (bypassing this dev box's custom systemd drop-in) transcribed successfully over ROCm; running it under that drop-in reproduced the abort, so the drop-in's sandboxing -- not Lemonade's packaging -- caused it. The RuntimeDirectory row was likewise self-inflicted by that same local drop-in; the packaged unit sets RuntimeDirectory= correctly. Neither belongs in the skill's general troubleshooting table. Step 1c's per-modality backend health check stays, since it catches this class of failure without depending on either bug. - Simplify the image-generation row's setup note: drop "ask before generating" (unnecessary agent-side indirection) and say "several GB" instead of a size tied to one specific model. - Drop the messages/reranking route-exception section from reference.md; out of scope while this skill only routes image/TTS/STT. - Collapse the per-modality fallback table to one statement, since the three shipped modalities all take the same policy (disclosed fallback is fine); the embeddings/reranking "never fall back" guidance belongs with those modalities if they're ever added, not baked in ahead of them. - Remove the STT byte-stability caveat from both the rule template and reference.md; too in-the-weeds for this skill's general audience. - Replace the hardcoded "verified against Lemonade 11.5.2" claims in SKILL.md and reference.md with version-agnostic guidance (confirm against `lemonade status` / `GET /api/v1/models` on the version actually installed). The skill already detects capability rather than parsing version strings; pinning an exact tested version in the shipped text fights that design and goes stale immediately -- it already had, since 11.6.0 and 11.7.0 shipped after that snapshot was written. Co-Authored-By: Claude Sonnet 5 --- skills/local-ai-use/SKILL.md | 7 ++++--- skills/local-ai-use/reference.md | 14 ++++---------- 2 files changed, 8 insertions(+), 13 deletions(-) diff --git a/skills/local-ai-use/SKILL.md b/skills/local-ai-use/SKILL.md index 517dfec..6086b9a 100644 --- a/skills/local-ai-use/SKILL.md +++ b/skills/local-ai-use/SKILL.md @@ -82,9 +82,10 @@ instead. `lemonade pull` in `reference.md`). - **Network:** required for the install download and the first `lemonade pull` of each model. After that, every modality runs offline. -- **Tested against:** Lemonade **11.5.2** (requires v10.1.0 or newer). Model - IDs and `system-info` fields can change between releases; confirm against the - version actually installed. +- **Version:** requires v10.1.0 or newer (the unified `lemonade` CLI this + skill targets). Model IDs and `system-info` fields can change between + releases; confirm against `lemonade status` and `GET /api/v1/models` on the + version actually installed rather than assuming this document is current. ## The opinionated path diff --git a/skills/local-ai-use/reference.md b/skills/local-ai-use/reference.md index 375841a..c6a2e03 100644 --- a/skills/local-ai-use/reference.md +++ b/skills/local-ai-use/reference.md @@ -21,10 +21,10 @@ The default trio (`SD-Turbo`, `kokoro-v1`, `Whisper-Tiny`) is sized for "keeps cost savings real on a typical laptop". Override only if the user asks for higher quality or has explicit hardware to spare. -Model IDs below were verified against Lemonade **11.5.2**. Catalogs move; -confirm any ID with `GET /api/v1/models` (add `?show_all=true` for the full -catalog) before writing it into the rule. Do not rely on a stale -`server_models.json` snapshot. +Model catalogs move between releases, so treat the IDs below as a starting +point, not a fixed list. Confirm any ID with `GET /api/v1/models` (add +`?show_all=true` for the full catalog) before writing it into the rule. Do +not rely on a stale `server_models.json` snapshot. ### Image generation (`recipe: sd-cpp`) @@ -73,12 +73,6 @@ The HTTP endpoint adds no meaningful overhead over calling `whisper-cli` directly on the same engine/backend/model. Throughput is equivalent; use the endpoint unless you have a specific reason not to. -Transcripts of long audio are not guaranteed byte-stable across identical -requests: Lemonade's server-side handling can vary chunk boundaries even when -the underlying whisper.cpp run is deterministic. Do not use a transcript as a -cache key or diff target; transcribe once and store the result if you need -reproducible output. - For full live coverage, run `lemonade list` after starting the server, or browse .