Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 31 additions & 1 deletion docs/admin-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -405,7 +405,7 @@ Local AI engine. Powers document classification, voice transcription, and text-t

First-run setup takes 10 to 20 minutes:

1. Asks managed oMLX (default, recommended) or external OpenAI-compatible endpoint.
1. Asks you to confirm the local engine. To use an AI server somewhere else instead, answer no and see [AI on another machine](#ai-on-another-machine-or-a-hosted-provider) below.
2. Taps and trusts the `jundot/omlx` Homebrew tap, then installs oMLX (see below).
3. Installs `cmake` and `ffmpeg` if missing.
4. Clones and builds whisper.cpp with Metal support (1 to 2 minutes of compilation).
Expand Down Expand Up @@ -464,11 +464,40 @@ Useful commands:
```bash
./stack ai models
./stack ai download <model-id>
./stack ai connect <url>
./stack setup ai
```

To switch LLM models: edit `[ai] default` in `stack.toml` (alternatives are listed as commented lines), then `./stack setup ai`.

#### AI on another machine, or a hosted provider

Everything that uses AI talks to one OpenAI-compatible address, `[ai] openai_url`. The engine `./stack up ai` installs is one option. If your models run on another computer in the house (oMLX, Ollama, LM Studio, anything that speaks the OpenAI API and listens on the network), or you use a hosted provider, skip `./stack up ai` and point the stack there:

```bash
./stack ai connect 192.168.1.20:11434 # Ollama on another machine
./stack ai connect https://api.example.com --key sk-... # a hosted provider
./stack ai connect local # back to the engine on this Mac
```

A second Mac running famstack's AI stacklet in port mode serves both chat and voice to the network, oMLX on port 42060 and Whisper on 42062. Its API key is its own `[ai] openai_key`, `local` unless changed:

```bash
./stack ai connect 192.168.1.20:42060 --key local --whisper 192.168.1.20:42062
```

The command checks that the server answers, picks a model it has (`--model <id>` when it lists several), writes `[ai]` in `stack.toml`, and names the running stacklets to restart. It installs nothing and restarts nothing. `./stack list` then shows the AI stacklet as `remote`, with the server's name, and as degraded when that server stops answering.

**Voice messages** have their own address, because not every AI server can transcribe:

- `--whisper <url>` names a speech server. It gets every voice message, whatever the AI server offers. A second Mac running the AI stacklet serves one on port 42062.
- Without one, the AI server transcribes, if it has a speech-to-text model loaded. The command looks for one and warns when there is none.
- `--whisper ai` removes the speech server again.

**What leaves the house.** With a hosted provider, the text of your documents, notes, chat questions and voice messages is sent to that provider and processed by whoever runs it. The command warns whenever an address is outside your home network; machines on your LAN or your Tailscale network count as home.

**Switching back.** On a remote setup, `./stack up ai` asks before it switches the stack to the engine on this Mac; no leaves everything as it was. After switching to a remote server, `./stack down ai` stops a local engine that is still running and frees its memory.

### ChatAI (`chatai`) optional

Open WebUI: a ChatGPT-style interface for your local LLM.
Expand Down Expand Up @@ -672,6 +701,7 @@ Key things to know:

```bash
./stack ai models # available AI models in the backend
./stack ai connect <url> # use an AI server on another machine, or `local`
./stack config # show resolved configuration
./stack config --secrets # include generated passwords
./stack version # print version
Expand Down
3 changes: 3 additions & 0 deletions docs/agent/ops.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ If a precondition is missing, `./stack` prints exactly what to do. Don't improvi
| `./stack host` | yes | no | disk / memory / uptime |
| `./stack config [--secrets]` | yes | no | prints resolved config |
| `./stack ai models` | yes | no | lists installed AI models |
| `./stack ai connect <url>` | yes | no | points `[ai]` at an AI server elsewhere, or `local`; checks it, installs nothing, restarts nothing. Warns when the address is outside the home network |
| `./stack setup ai` | yes | no | re-runs AI install (model swap path) |

**Output contract:** every command returns JSON when piped or when `--json` is passed. Force human output with `--pretty`. Exit code 0 == success.
Expand Down Expand Up @@ -102,6 +103,8 @@ Port collisions: do not silently rebind. Surface them. The user's fix is "stop t
| AI install fails at whisper.cpp build | `xcode-select -p` | Xcode CLT missing. `xcode-select --install`. |
| `brew: command not found` after install | PATH on Apple Silicon | Add `eval "$(/opt/homebrew/bin/brew shellenv)"` to `~/.zshrc`. |
| LLM OOM / very slow | RAM tier | Edit `[ai] default` to smaller model, then `./stack setup ai`. |
| `stack list` shows ai as `remote` | `[ai] provider` in `stack.toml` | The stack uses an AI server elsewhere (`stack ai connect`). `stack up ai` asks before switching back to this Mac. |
| Voice messages fail on a remote AI setup | `./stack ai connect <url>` output, "Voice messages" line | The AI server lists no speech-to-text model. Pass `--whisper <url>` for a speech server. |
| Disk full | `./stack host` | Likely the photo library. Move `data_dir` to external SSD. |
| Element warns "browser not supported" | n/a | Click Continue. The check is outdated; Element works in every modern browser. |
| Wiki pages stale after filings | `docker logs stack-memory-curator` | Curator debounces (~3 min quiet) before rebuilding; topic pages wait for the nightly sweep. Manual override: `./stack memory wiki`. |
Expand Down
25 changes: 24 additions & 1 deletion docs/stack-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,7 +196,9 @@ Available template variables:
| `{ai_openai_url}` | Derived from `stack.toml` → `[ai].openai_url` |
| `{ai_openai_url_docker}` | Same, rewritten for container access via `host.docker.internal` |
| `{ai_openai_key}` | `stack.toml` → `[ai].openai_key` |
| `{ai_whisper_url_docker}` | Derived from `stack.toml` → `[ai].whisper_url` |
| `{ai_whisper_url_docker}` | Derived from `stack.toml` → `[ai].whisper_url`, or the AI server when unset |
| `{ai_whisper_key}` | `stack.toml` → `[ai].whisper_key` with a dedicated speech server, otherwise the AI key |
| `{ai_whisper_model}` | `stack.toml` → `[ai].whisper_model`, default `whisper-1` |
| `{ai_default_model}` | `stack.toml` → `[ai].default` |
| `{ai_tts_voice}` | Derived from `[ai].language` |
| `{messages_server_name}` | `stack.toml` → `[messages].server_name` |
Expand Down Expand Up @@ -664,6 +666,21 @@ The context object (`ctx`) provides:
| `ctx["http_post"]` | `callable` | `http_post(url, body, content_type=..., headers=...)` → parsed JSON. Form-encoded by default. |
| `ctx["http_get"]` | `callable` | `http_get(url, headers=...)` → parsed JSON. Pass auth explicitly: `headers={"Authorization": "Bearer ..."}`. |

A hook that raises stops the command and is reported as a failure. A
hook that asked the admin and got a no raises `stack.hooks.Cancelled`
instead: `stack up` stops with `{"cancelled": <message>}`, prints the
message once, exits 1, and does not tell the family room that the
stacklet failed to start.

```python
from stack.hooks import Cancelled
from stack.prompt import confirm

def run(ctx):
if not confirm("Replace the remote AI server with this Mac?", default=False):
raise Cancelled("Cancelled. Nothing changed.")
```

Example `hooks/on_install.py` (system work with `ctx.shell()`):

```python
Expand Down Expand Up @@ -725,6 +742,12 @@ The convention:
Files starting with `_` are private helpers (e.g., `_matrix.py`), not
exposed as commands.

A command that records a choice in `stack.toml` calls
`config["set_cfg"](section, key, value)`, the plugin-side twin of a
hook's `ctx.cfg(key, value)`. It changes that one line in that section
and leaves the admin's comments and other sections as they are.
`stack ai connect` uses it to write `[ai]`.

---

## Docker Compose Conventions
Expand Down
11 changes: 8 additions & 3 deletions lib/stack/ai/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -449,9 +449,13 @@ class Transcriber:
archivist vs scribe) without touching call sites later.
"""

def __init__(self, client: AsyncOpenAI, *, namespace: str | None = None):
def __init__(self, client: AsyncOpenAI, *, namespace: str | None = None,
model: str = _DEFAULT_WHISPER_MODEL):
self._client = client
self.namespace = namespace
# The speech model to ask for. OpenAI's is `whisper-1`; other
# servers name theirs and refuse any other name.
self.model = model

@classmethod
def from_env(cls, *, namespace: str | None = None,
Expand Down Expand Up @@ -480,7 +484,8 @@ def from_env(cls, *, namespace: str | None = None,
# on *some* key — same trick as LLM.from_env.
key = os.environ.get("WHISPER_KEY", "") or "not-needed"
client = AsyncOpenAI(base_url=url, api_key=key, max_retries=max_retries)
return cls(client, namespace=namespace)
return cls(client, namespace=namespace,
model=os.environ.get("WHISPER_MODEL", "") or _DEFAULT_WHISPER_MODEL)

async def transcribe(self, audio: bytes, *, filename: str = "voice.ogg",
model: str | None = None,
Expand Down Expand Up @@ -616,7 +621,7 @@ async def _stt(self, audio: bytes, *, filename: str,
"""
try:
return await self._client.audio.transcriptions.create(
model=model or _DEFAULT_WHISPER_MODEL,
model=model or self.model,
file=(filename, audio),
**params,
**({"prompt": vocabulary} if vocabulary.strip() else {}),
Expand Down
48 changes: 48 additions & 0 deletions lib/stack/ai/probe.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@
from __future__ import annotations

import dataclasses
import ipaddress
import json
import socket
import ssl
import urllib.error
import urllib.parse
import urllib.request

# LAN endpoints often serve self-signed TLS; the AI endpoint is trusted by
Expand Down Expand Up @@ -49,3 +52,48 @@ def probe(url: str, key: str = "", *, timeout: float = 3.0) -> ProbeResult:
return ProbeResult(reachable=False)
except Exception:
return ProbeResult(reachable=False)


def transcribes(url: str, key: str = "", *, timeout: float = 3.0) -> bool:
"""Whether ``{url}/audio/transcriptions`` exists.

Asked with an empty POST, so nothing is uploaded and nothing is
transcribed. A server with the endpoint rejects the request for the
missing file (400, 415 or 422); one without it answers 404 or 405.
"""
req = urllib.request.Request(
f"{url.rstrip('/')}/audio/transcriptions", data=b"", method="POST",
headers={"Authorization": f"Bearer {key}"} if key else {})
try:
with urllib.request.urlopen(req, timeout=timeout, context=_SSL):
return True
except urllib.error.HTTPError as e:
return e.code in (400, 415, 422)
except Exception:
return False


# Tailscale hands out addresses from the carrier-grade NAT range, which
# Python does not count as private.
_TAILNET = ipaddress.ip_network("100.64.0.0/10")


def stays_home(url: str) -> bool:
"""Whether every address the host of ``url`` resolves to is on the
home network: loopback, private, link-local or a Tailscale tailnet.

Anything else is a server run by someone else, and what the stack
sends it (document text, notes, voice) leaves the house. A name
that does not resolve is not assumed to be at home.
"""
host = urllib.parse.urlsplit(url).hostname or ""
try:
infos = socket.getaddrinfo(host, None)
except OSError:
return False
for info in infos:
ip = ipaddress.ip_address(info[4][0].split("%")[0])
if not (ip.is_loopback or ip.is_private or ip.is_link_local
or ip in _TAILNET):
return False
return bool(infos)
15 changes: 7 additions & 8 deletions lib/stack/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,7 @@ def up(self, stacklet_id: str) -> dict:
return self._up_all()

result = self.stack.up(stacklet_id)
if "error" in result:
if "error" in result or "cancelled" in result:
return result

stacklet = self.stack._find_stacklet(stacklet_id)
Expand Down Expand Up @@ -409,7 +409,7 @@ def _up_ordered(self, include: set[str]) -> dict:
errors: list[dict] = []
for sid in order:
result = self.up(sid)
if "error" in result:
if "error" in result or "cancelled" in result:
errors.append({"stacklet": sid, "result": result})
else:
started.append(sid)
Expand Down Expand Up @@ -854,6 +854,9 @@ def handle_up(stck, args):
name = stacklet.get("name", args.stacklet) if stacklet else args.stacklet
print(f"\n Bringing up {TEAL}{name}{RESET}...\n", file=sys.stderr)
result = cli.up(args.stacklet)
if "cancelled" in result:
print(f" {DIM}{result['cancelled']}{RESET}", file=sys.stderr)
sys.exit(1)
if result.get("ok"):
print_up_success(result, stck)
_notify_up(stck, result)
Expand Down Expand Up @@ -1326,11 +1329,7 @@ def handle_setup(stck, args):


def handle_install(stck, args):
full = getattr(args, "full", False)
if full:
from .installer import wizard
else:
from .installer_v2 import wizard
from .installer import wizard

# Save/restore terminal in case a crash leaves it in raw mode
saved_term = None
Expand Down Expand Up @@ -1892,7 +1891,7 @@ def main():
parser.add_argument("--version", action="store_true")

sub = parser.add_subparsers(dest="command")
p = sub.add_parser("install"); p.add_argument("--full", action="store_true")
p = sub.add_parser("install")
p = sub.add_parser("uninstall")
p.add_argument("--yes", action="store_true")
p.add_argument(
Expand Down
11 changes: 11 additions & 0 deletions lib/stack/hooks.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,15 @@
from pathlib import Path


class Cancelled(Exception):
"""Raised by a hook when the admin answered no.

It stops the command the way an error does, but it is not one:
nothing is reported as broken and nobody is told a stacklet failed.
The message is what the admin sees, once.
"""


class StackContext:
"""Everything a hook needs. No artificial restrictions.

Expand Down Expand Up @@ -220,6 +229,8 @@ def _run_python(self, path: Path, ctx) -> bool:
if hasattr(mod, "run"):
mod.run(ctx)
return True
except Cancelled:
raise
except Exception as e:
if hasattr(ctx, "stack") and hasattr(ctx.stack, "output"):
ctx.stack.output.error(f"Hook {path.name} error: {e}")
Expand Down
Loading
Loading