Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Tier 1 of the release gate (docs/agent/dev.md, "Releases"): the checks
# that need no Docker and no macOS. Runs on every push/PR; on version
# that need no Docker and no macOS, including that every commit subject
# parses as a changelog entry. Runs on every push/PR; on version
# tags it additionally enforces that the tag, the CLI VERSION, and
# pyproject.toml all agree. Tiers 2 (integration suite) and 3 (fresh
# install, ai stacklet) stay manual — see docs/cleanup-backlog.md.
Expand Down Expand Up @@ -61,6 +62,21 @@ jobs:
print("Versions agree:", ", ".join(f"{s}={v}" for s, v in versions.items()))
EOF

- name: Commit convention
# The changelog is generated from these subjects, so one that
# does not parse is an entry missing from the release notes.
# Squash merging makes the PR title the subject that lands, so
# it is checked too.
if: github.event_name == 'pull_request'
env:
PR_TITLE: ${{ github.event.pull_request.title }}
BASE: ${{ github.event.pull_request.base.sha }}
HEAD: ${{ github.event.pull_request.head.sha }}
run: |
git fetch --no-tags --depth=50 origin "$BASE" "$HEAD" 2>/dev/null || true
python3 tools/commit-lint --title "$PR_TITLE"
python3 tools/commit-lint --range "$BASE..$HEAD"

- name: Framework tests
run: uv run --extra test pytest tests/framework/ --ignore=tests/framework/test_config_to_container.py -q

Expand Down
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,11 @@ tests/integration/eval/runs/
.claude/skills/*
!.claude/skills/famstack-lsp/

# Extension stacklets live in ~/<product>-extensions, not here. This is
# the obvious place to try anyway, so keep anything dropped here out of
# git rather than in someone's next commit.
/extensions/

# Internal development notes
impl/

Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ Apply to every role, every session.
3. **Never `git push` without explicit human approval.** Every push, every branch, every time.
4. **Never commit to `main`.** Feature branches only.
5. **No `Co-Authored-By:` trailers** in commit messages.
6. **Semantic commit prefixes:** `feat:`, `fix:`, `docs:`, `refactor:`, `chore:`, `test:`, `ci:`, `style:`.
6. **Commit subjects are the changelog.** `<type>(<scope>): <imperative summary>`, then a body that says what and why, in plain technical English. Types, scopes and the `Upgrade:` footer are in [docs/agent/dev.md](docs/agent/dev.md#commits-the-subject-is-the-changelog).
7. **Never read or copy a user's production family vault.** Reference paths only; fabricate test data.
8. **Destructive ops need confirmation.** `stack destroy`, `stack uninstall`, `rm -rf`, `git reset --hard`, force-push.
9. **Announce actions before running them.** No silent long running or integration test runs, scripts, or background commands.
Expand Down
18 changes: 15 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,13 @@ One entry point, [Homebrew](https://brew.sh), which brings the rest:

```bash
brew install uv
uv sync --extra test
tools/init-repo
```

`tools/init-repo` is the one-time setup for a clone, and is safe to re-run. It checks the two things that otherwise fail confusingly hours later (an old Python, a missing `uv`), creates the test virtualenv, points git at the repo's hooks, and installs the language server binary. Everything it does is local to your clone.

The hooks are the part git cannot do for you: a fresh clone must not be able to run code, so no repo can enable its own hooks on clone. `make test-unit` and `make typecheck` also enable them, so in practice they are on before your first commit either way.

`uv sync` creates `.venv` in the repo root. Nothing needs activating: `uv run` and `uvx` find it, and the type checker is pointed at it by `pyproject.toml`. The `test` extra declares every dependency the test suite needs, including the bot runtime libraries that tests import directly.

The CLI itself depends only on the standard library. Stacklet containers carry their own runtime dependencies, which is why `.venv` will not contain all of them: `nanobot` and `fastapi`, for example, exist only inside their images.
Expand All @@ -31,7 +35,7 @@ Neither is a merge gate, and the type checker is not clean today. Run it on what

## Language server

Agents working in this repo get a language server, so they can ask for a definition or every reference to a symbol instead of grepping for a name and hoping it is unique. Install the binary once:
Agents working in this repo get a language server, so they can ask for a definition or every reference to a symbol instead of grepping for a name and hoping it is unique. `tools/init-repo` installs the binary; by hand it is:

```bash
uv tool install basedpyright
Expand Down Expand Up @@ -69,4 +73,12 @@ A bare `make typecheck` reports on `lib`, `stacklets`, `tools` and `hooks`. `tes

## Tests, style, commits

All three live in [docs/agent/dev.md](docs/agent/dev.md), which is the canonical reference and stays shorter than a duplicate here would. The short version: `make test-unit` before every commit, module tests over unit tests, semantic commit prefixes, feature branches only, never push without asking.
All three live in [docs/agent/dev.md](docs/agent/dev.md), which is the canonical reference and stays shorter than a duplicate here would. The short version: the hooks lint every commit for you, `make test-unit` before a push or when a piece of work is done, module tests over unit tests, feature branches only, never push without asking. Commit subjects are the changelog famstack.dev renders, so they follow `<type>(<scope>): <what the product does now>` and every one has to parse.

The hooks check ruff on staged files and the commit subject before a commit exists, rather than in CI. `tools/init-repo` turns them on, as do `make test-unit` and `make typecheck`. To do only that:

```bash
make hooks # or: git config core.hooksPath hooks
```

An explicit `core.hooksPath` is never overwritten, so pointing it elsewhere, or nowhere, is respected.
85 changes: 66 additions & 19 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,34 +1,81 @@
# famstack development tasks
# famstack development tasks.
#
# One name per lane. The aliases that used to exist (unit, fast, test,
# demo, lifecycle, e2e, smoke, test-all, test-integration) are gone: they
# were eighteen names for six jobs, and `test-all` ran only the lifecycle
# lane, which is worse than not existing.
#
# `make` on its own prints the table.

PYTEST = uv run --extra test pytest

.DEFAULT_GOAL := help

help:
@printf '%s\n' \
"" \
" Linting is automatic: the hooks run ruff on staged files and check" \
" the commit subject. Nothing to remember per commit." \
"" \
" Run the cheapest lane that proves what you changed." \
"" \
" LANE TIME NEEDS EXCLUSIVE RUN IT WHEN" \
" make lint <1s nothing no the whole tree, when you want it" \
" make test-unit ~75s nothing no before a push, or when a piece of work is done" \
" make test-lifecycle ~6m Docker yes lifecycle, env, compose, ports, health" \
" make test-demo ~4m demo instance yes it must work against the running bots" \
" make test-smoke ? test-owned rig yes quick cross-service check" \
" make test-e2e 20-30m uninstalled yes end of a branch, before review" \
"" \
" make typecheck types across the shipped code, not a gate" \
" make hooks point git at the repo's hooks" \
"" \
" Exclusive means it owns fixed container names and ports, so exactly" \
" one run at a time on this Mac. ? means nobody has timed it yet;" \
" if you run one, write the number down. Details: tests/README.md" \
""

# The whole tree, the way CI runs it. Per commit the hook already covers
# the staged files, which is faster and is why nothing here is per commit.
lint: hooks
uvx ruff check .

# Type check the shipped code. Not a gate: read what it says about the files
# you touched. `uvx basedpyright <paths>` narrows it further.
typecheck types:
typecheck: hooks
-uvx basedpyright

# Fast unit tests: no Docker. Run before every commit.
test-unit unit fast test:
$(PYTEST) tests/framework tests/stacklets -v --ignore=tests/framework/test_config_to_container.py
# Point git at the repo's own hooks. Git will not do this on clone, by
# design: a fresh clone must not be able to run code. The nearest honest
# thing is to do it the first time someone uses the repo's tooling, which
# is what the lanes below depend on.
#
# Only when unset, so an explicit choice (an absolute path, a different
# directory, deliberately no hooks) is never overwritten.
hooks:
@tools/init-repo --hooks-only

# Live demo-rig tests: uses the already-running Simpsons demo instance.
test-demo demo-rig demo:
tests/integration/stacktests demo-rig
# Offline. No Docker, no live services, nothing shared. Fast enough to run
# when a piece of work is finished, too slow to run on every commit.
test-unit: hooks
$(PYTEST) tests/framework tests/stacklets -v --ignore=tests/framework/test_config_to_container.py

# Docker lifecycle tests: validates stack up/down/destroy and container env.
test-lifecycle container-lifecycle lifecycle:
# Docker lifecycle: up, down, destroy, env rendering, container environment.
# Owns its own throwaway instance, but on fixed names and ports.
test-lifecycle:
$(PYTEST) tests/framework/test_config_to_container.py -v

# Managed container e2e tests: starts the test rig and runs every e2e test.
test-e2e container-e2e e2e:
tests/integration/stacktests e2e
# Against the demo instance already running in this checkout. Tests create
# unique data and clean up after themselves; they never reset the instance.
test-demo:
tests/integration/stacktests demo-rig

# Quick managed-rig e2e subset.
test-smoke smoke:
# A small subset of the managed-rig e2e lane, for a quick answer.
test-smoke:
tests/integration/stacktests smoke

# Backwards-compatible names.
test-all: test-lifecycle
test-integration: test-e2e
# The full managed rig: seeds a test-owned instance and runs every e2e test.
test-e2e:
tests/integration/stacktests e2e

.PHONY: typecheck types test-unit unit fast test test-demo demo-rig demo test-lifecycle container-lifecycle lifecycle test-e2e container-e2e e2e test-smoke smoke test-all test-integration
.PHONY: help hooks lint typecheck test-unit test-lifecycle test-demo test-smoke test-e2e
7 changes: 6 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,4 +21,9 @@
- [ADR-007](adr/adr-007-port-convention.md) 42xxx port convention
- [ADR-008](adr/adr-008-convention-based-bot-runner.md) Convention-based bot runner
- [ADR-009](adr/adr-009-managed-ai-provider.md) Managed AI with oMLX as default
- [RFC-001](adr/rfc-001-dashboard-and-companion-apps.md) Dashboard and companion apps
- [ADR-010](adr/adr-010-event-pipeline.md) Event pipeline
- [ADR-011](adr/adr-011-vault-brain-projection.md) Vault to brain projection
- [ADR-012](adr/adr-012-nanobot-fork.md) nanobot fork
- [ADR-013](adr/adr-013-stacklet-locations-and-stages.md) Stacklet locations and stages
- [RFC-001](adr/rfc-001-dashboard-and-companion-apps.md) Dashboard and companion apps
- [RFC-002](adr/rfc-002-installation-instances-and-updates.md) Installation, instances and updates
152 changes: 146 additions & 6 deletions docs/admin-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -828,16 +828,156 @@ Stash the tarball on a USB stick or in 1Password.

Container images update themselves nightly at 3am via Watchtower. You do nothing for that.

For the famstack code itself:
The famstack code is a git checkout, and a release is a tag. Updating means moving your checkout to a newer tag.

```bash
git pull
./stack restart <stacklet> # for any stacklet whose code changed
./stack update
```

If `git pull` shows changes to `lib/stack/` or to a stacklet you are running, restart that stacklet. If in doubt, restart everything.
That is the whole thing from v0.3.0-beta.4 onward. It fetches the releases, shows what the jump changes, moves the checkout, and sets your local edits aside and puts them back.

Pre-1.0 caveat: occasionally a release changes a config schema or env var. Release notes will say so. Read them before pulling.
It does not restart anything. New code on disk is not new code running: a stacklet picks up a release when its containers are recreated, and when your family loses a service for thirty seconds is your call, not a side effect of updating. So the command works out which restarts the release actually earns and prints them:

```
✓ Updated to v0.3.0-beta.4

Restart to pick it up
./stack restart docs
./stack doctor
```

```bash
./stack update --dry-run # show the plan, change nothing
./stack update v0.3.0-beta.4 # a specific release instead of the newest
./stack update --yes # skip the confirmation
```

Every form fetches tags first, including `--dry-run`: without it the answer
would only be as fresh as your last fetch. It opens with the version and,
once it knows, the move it is proposing:

```
famstack 0.3.0-beta.3
Fetching releases...

Update v0.3.0-beta.3 → v0.3.0-beta.4
```

A stacklet is named only if the release changed files inside it and it is running, so a documentation release asks for nothing. If the release changed anything under `lib/`, every running stacklet is named, because that is the code they all share.

### Updating by hand

`stack update` ships in v0.3.0-beta.4. Moving *to* that release, or moving between older ones, is the same steps typed out:

```bash
cd ~/famstack
git fetch --tags
git tag | tail -5 # what is available
git checkout v0.3.0-beta.4 # the one you want
./stack doctor
```

Then restart what the release touched. This prints the list:

```bash
git diff --name-only v0.3.0-beta.3 v0.3.0-beta.4 -- stacklets/ | cut -d/ -f2 | sort -u
```

Restart any of those you run. If the same command against `lib/` prints anything, the framework itself changed and everything wants a restart:

```bash
./stack down all && ./stack up all
```

Nothing in the update touches your instance. `stack.toml`, `users.toml`, `.stack/`, `~/famstack-data/` and `~/famstack-extensions/` are all outside git's reach, so your config, secrets, data and your own stacklets come through a tag switch untouched.

### If you have edited files in the checkout

`stack update` handles this for you. It lists the files it is about to set aside, warns in advance when one of them is a file the release also changes, and puts them back afterwards.

If they collide, it winds the whole update back: you end up on the release you started from, with your edits exactly where they were and nothing left in the stash. Then it tells you how to go ahead deliberately:

```
⚠ Your edits collide with v0.3.0-beta.4. Nothing changed.
Back on v0.3.0-beta.3 with your edits where they were.
The collision is in:
• stacklets/docs/docker-compose.yml
To take the release anyway, deal with that file first:
git checkout -- <file> drop your version for the release's
git stash keep it, then pop and merge by hand
then run ./stack update again.
```

A half-applied update is worse than no update: a compose file with conflict markers in it does not parse, so the services you restart next fail for a reason that has nothing to do with the release. What follows is the same work done by hand, which is what an older release needs.

git refuses to move while one of your edits is in the way:

```
error: Your local changes to the following files would be overwritten by checkout:
stacklets/docs/docker-compose.yml
Please commit your changes or stash them before you switch branches.
Aborting
```

Set them aside, switch, put them back:

```bash
git stash
git checkout v0.3.0-beta.3
git stash pop
```

If the release did not touch the same file, `git stash pop` re-applies your edit and you are done.

If the release changed the same lines, it reports a conflict and **keeps your stash**, so nothing is lost:

```
CONFLICT (content): Merge conflict in stacklets/docs/docker-compose.yml
The stash entry is kept in case you need it again.
```

From there, look at what your edit was, then keep it or drop it:

```bash
git stash show -p # your change

# keep it: edit the file to merge both sides, then
git add stacklets/docs/docker-compose.yml && git stash drop

# or take the release's version and give up your edit
git checkout HEAD -- stacklets/docs/docker-compose.yml && git stash drop
```

Editing a shipped stacklet means doing this on every update. There is no override mechanism: a stacklet in `~/famstack-extensions/` can add to what famstack ships, not replace it. If your change is one other families would want, an issue or a pull request costs you less in the long run.

### Knowing where you are

```bash
git describe --tags # v0.3.0-beta.3, or a tag plus commits past it
```

Checking out a tag leaves you in "detached HEAD". That is normal and nothing to repair, but it does mean `git pull` no longer works:

```
$ git pull
You are not currently on a branch.
Please specify which branch you want to merge with.
```

Do not take git's suggestion to run `git pull origin main`. It succeeds, and it quietly moves you off releases onto unreleased development code while `./stack version` carries on reporting the last release number. Use `git fetch --tags` and `git checkout <tag>`.

If you cloned and never checked out a tag, you are on `main`, which is development. `git pull` there gives you unreleased work rather than the next release. `./stack update` moves you onto the newest release and says so before it does: your branch is left where it is, and `git switch main` goes back to it. If `main` already contains the newest release, which it will if you pulled recently, it tells you that instead of moving you backwards.

**On a fork**, `origin` is your copy and has none of the project's releases in it. `./stack update` fetches tags from every remote, so add the project as one:

```bash
git remote add upstream https://github.com/famstack-dev/famstack.git
./stack update
```

Without it there is nothing to update to, and the command says which remotes it looked at.

Pre-1.0 caveat: occasionally a release changes a config schema or env var. Release notes say so. Read them before updating.

### Paperless-ngx 3.x (docs stacklet)

Expand All @@ -848,7 +988,7 @@ This release moves the `docs` stacklet from Paperless-ngx 2.20.15 to 3.0.4.
```bash
./stack down docs
tar -czf ~/docs-pre-3x-backup.tar.gz -C ~/famstack-data docs
git pull
git fetch --tags && git checkout v0.3.0-beta.3
./stack up docs
```

Expand Down
Loading
Loading