build: lay out tests and scripts by lane, stash the demo around e2e

.claude/skills/famstack-lsp/hooks/session-start.sh

@@ -7,7 +7,7 @@
 # server answers from a partial index instead of waiting.
 
 if ! command -v basedpyright-langserver >/dev/null 2>&1; then
-    echo "famstack-lsp: basedpyright-langserver is not on PATH, so there is no language server this session. Install it with 'uv tool install basedpyright' (or run tools/init-repo), then /reload-plugins. Until then, use 'uvx basedpyright <paths>' for diagnostics and grep for navigation."
+    echo "famstack-lsp: basedpyright-langserver is not on PATH, so there is no language server this session. Install it with 'uv tool install basedpyright' (or run script/setup), then /reload-plugins. Until then, use 'uvx basedpyright <paths>' for diagnostics and grep for navigation."
     exit 0
 fi
 

.github/workflows/ci.yml

@@ -1,8 +1,7 @@
 # Tier 1 of the release gate (docs/agent/dev.md, "Releases"): the checks
-# that need no Docker and no macOS, including that every commit subject
-# parses as a changelog entry. Runs on every push/PR; on version
-# tags it additionally enforces that the tag, the CLI VERSION, and
-# pyproject.toml all agree. Tiers 2 (integration suite) and 3 (fresh
+# that need no Docker and no macOS, run by script/cibuild. Runs on every
+# push/PR; on version tags it additionally enforces that the tag, the CLI
+# VERSION, and pyproject.toml all agree. Tiers 2 (integration suite) and 3 (fresh
 # install, ai stacklet) stay manual — see docs/cleanup-backlog.md.
 
 name: CI
@@ -12,6 +11,8 @@ on:
     branches: [main]
     tags: ["v*"]
   pull_request:
+    # `edited` so a description changed after opening is checked again.
+    types: [opened, synchronize, reopened, edited]
 
 jobs:
   checks:
@@ -21,64 +22,13 @@ jobs:
 
       - uses: astral-sh/setup-uv@v5
 
-      - name: Lint (ruff)
-        run: uvx ruff check .
-
-      - name: Lockfile in sync with pyproject.toml
-        run: uv lock --check
-
-      - name: Version consistency
+      # Everything the job checks lives in script/cibuild, so it runs
+      # the same on a laptop. The PR values are empty on a push.
+      - name: script/cibuild
         env:
           GITHUB_REF: ${{ github.ref }}
-        run: |
-          uv run python - <<'EOF'
-          import os, re, sys, tomllib
-
-          def normalize(v: str) -> str:
-              """PEP-440-ish: v0.3.0-beta.1 / 0.3.0b1 -> 0.3.0b1"""
-              v = v.strip().lstrip("v").lower()
-              v = re.sub(r"[-.]?beta[-.]?", "b", v)
-              v = re.sub(r"[-.]?alpha[-.]?", "a", v)
-              v = re.sub(r"[-.]?rc[-.]?", "rc", v)
-              return v
-
-          cli = re.search(
-              r'^VERSION = "([^"]+)"',
-              open("lib/stack/cli.py").read(), re.M,
-          ).group(1)
-          pyproject = tomllib.load(open("pyproject.toml", "rb"))["project"]["version"]
-
-          versions = {"lib/stack/cli.py": cli, "pyproject.toml": pyproject}
-          ref = os.environ.get("GITHUB_REF", "")
-          if ref.startswith("refs/tags/v"):
-              versions["tag"] = ref.removeprefix("refs/tags/")
-
-          normalized = {src: normalize(v) for src, v in versions.items()}
-          if len(set(normalized.values())) != 1:
-              print("Version mismatch:")
-              for src, v in versions.items():
-                  print(f"  {src}: {v} (normalized: {normalized[src]})")
-              sys.exit(1)
-          print("Versions agree:", ", ".join(f"{s}={v}" for s, v in versions.items()))
-          EOF
-
-      - name: Commit convention
-        # The changelog is generated from these subjects, so one that
-        # does not parse is an entry missing from the release notes.
-        # Squash merging makes the PR title the subject that lands, so
-        # it is checked too.
-        if: github.event_name == 'pull_request'
-        env:
           PR_TITLE: ${{ github.event.pull_request.title }}
+          PR_BODY: ${{ github.event.pull_request.body }}
           BASE: ${{ github.event.pull_request.base.sha }}
           HEAD: ${{ github.event.pull_request.head.sha }}
-        run: |
-          git fetch --no-tags --depth=50 origin "$BASE" "$HEAD" 2>/dev/null || true
-          python3 tools/commit-lint --title "$PR_TITLE"
-          python3 tools/commit-lint --range "$BASE..$HEAD"
-
-      - name: Framework tests
-        run: uv run --extra test pytest tests/framework/ --ignore=tests/framework/test_config_to_container.py -q
-
-      - name: Stacklet tests
-        run: uv run --extra test pytest tests/stacklets/ -q
+        run: script/cibuild

.gitignore

@@ -3,8 +3,8 @@ stack.toml
 config.sh
 users.toml
 # …except the test instance's config, which is checked in.
-!tests/integration/instance/stack.toml
-!tests/integration/instance/users.toml
+!tests/e2e/instance/stack.toml
+!tests/e2e/instance/users.toml
 
 # Runtime state — generated by stack CLI
 .stack/
@@ -44,7 +44,7 @@ htmlcov/
 
 # Pipeline-eval per-run artifacts (OCR text, classification JSON,
 # scorecards) — written by `stacktests eval`, never committed.
-tests/integration/eval/runs/
+tests/e2e/eval/runs/
 
 # AI assistant. Local settings and transcripts stay out, but the LSP plugin
 # is repo configuration: it makes agents read this codebase with a language

CONTRIBUTING.md

@@ -10,10 +10,10 @@ One entry point, [Homebrew](https://brew.sh), which brings the rest:
 
 ```bash
 brew install uv
-tools/init-repo
+script/setup
 ```
 
-`tools/init-repo` is the one-time setup for a clone, and is safe to re-run. It checks the two things that otherwise fail confusingly hours later (an old Python, a missing `uv`), creates the test virtualenv, points git at the repo's hooks, and installs the language server binary. Everything it does is local to your clone.
+`script/setup` is the one-time setup for a clone, and is safe to re-run. It checks the two things that otherwise fail confusingly hours later (an old Python, a missing `uv`), creates the test virtualenv, points git at the repo's hooks, and installs the language server binary. Everything it does is local to your clone.
 
 The hooks are the part git cannot do for you: a fresh clone must not be able to run code, so no repo can enable its own hooks on clone. `make test-unit` and `make typecheck` also enable them, so in practice they are on before your first commit either way.
 
@@ -35,7 +35,7 @@ Neither is a merge gate, and the type checker is not clean today. Run it on what
 
 ## Language server
 
-Agents working in this repo get a language server, so they can ask for a definition or every reference to a symbol instead of grepping for a name and hoping it is unique. `tools/init-repo` installs the binary; by hand it is:
+Agents working in this repo get a language server, so they can ask for a definition or every reference to a symbol instead of grepping for a name and hoping it is unique. `script/setup` installs the binary; by hand it is:
 
 ```bash
 uv tool install basedpyright
@@ -75,7 +75,7 @@ A bare `make typecheck` reports on `lib`, `stacklets`, `tools` and `hooks`. `tes
 
 All three live in [docs/agent/dev.md](docs/agent/dev.md), which is the canonical reference and stays shorter than a duplicate here would. The short version: the hooks lint every commit for you, `make test-unit` before a push or when a piece of work is done, module tests over unit tests, feature branches only, never push without asking. Commit subjects are the changelog famstack.dev renders, so they follow `<type>(<scope>): <what the product does now>` and every one has to parse.
 
-The hooks check ruff on staged files and the commit subject before a commit exists, rather than in CI. `tools/init-repo` turns them on, as do `make test-unit` and `make typecheck`. To do only that:
+The hooks check ruff on staged files and the commit subject before a commit exists, rather than in CI. `script/setup` turns them on, as do `make test-unit` and `make typecheck`. To do only that:
 
 ```bash
 make hooks                  # or: git config core.hooksPath hooks

Makefile

@@ -1,81 +1,33 @@
-# famstack development tasks.
-#
-# One name per lane. The aliases that used to exist (unit, fast, test,
-# demo, lifecycle, e2e, smoke, test-all, test-integration) are gone: they
-# were eighteen names for six jobs, and `test-all` ran only the lifecycle
-# lane, which is worse than not existing.
-#
-# `make` on its own prints the table.
-
-PYTEST = uv run --extra test pytest
+# Aliases for the scripts in script/, which hold the logic. `make` on
+# its own prints the lane table. Details: script/test help.
 
 .DEFAULT_GOAL := help
 
 help:
-	@printf '%s\n' \
-	  "" \
-	  "  Linting is automatic: the hooks run ruff on staged files and check" \
-	  "  the commit subject. Nothing to remember per commit." \
-	  "" \
-	  "  Run the cheapest lane that proves what you changed." \
-	  "" \
-	  "  LANE                TIME    NEEDS          EXCLUSIVE  RUN IT WHEN" \
-	  "  make lint           <1s     nothing        no         the whole tree, when you want it" \
-	  "  make test-unit      ~75s    nothing        no         before a push, or when a piece of work is done" \
-	  "  make test-lifecycle ~6m     Docker         yes        lifecycle, env, compose, ports, health" \
-	  "  make test-demo      ~4m     demo instance  yes        it must work against the running bots" \
-	  "  make test-smoke     ?       test-owned rig yes        quick cross-service check" \
-	  "  make test-e2e       20-30m  uninstalled    yes        end of a branch, before review" \
-	  "" \
-	  "  make typecheck              types across the shipped code, not a gate" \
-	  "  make hooks                  point git at the repo's hooks" \
-	  "" \
-	  "  Exclusive means it owns fixed container names and ports, so exactly" \
-	  "  one run at a time on this Mac. ? means nobody has timed it yet;" \
-	  "  if you run one, write the number down. Details: tests/README.md" \
-	  ""
+	@script/test help
 
-# The whole tree, the way CI runs it. Per commit the hook already covers
-# the staged files, which is faster and is why nothing here is per commit.
-lint: hooks
-	uvx ruff check .
+hooks:
+	@script/setup --hooks-only
 
-# Type check the shipped code. Not a gate: read what it says about the files
-# you touched. `uvx basedpyright <paths>` narrows it further.
-typecheck: hooks
-	-uvx basedpyright
+lint:
+	@script/test lint
 
-# Point git at the repo's own hooks. Git will not do this on clone, by
-# design: a fresh clone must not be able to run code. The nearest honest
-# thing is to do it the first time someone uses the repo's tooling, which
-# is what the lanes below depend on.
-#
-# Only when unset, so an explicit choice (an absolute path, a different
-# directory, deliberately no hooks) is never overwritten.
-hooks:
-	@tools/init-repo --hooks-only
+typecheck:
+	@script/test typecheck
 
-# Offline. No Docker, no live services, nothing shared. Fast enough to run
-# when a piece of work is finished, too slow to run on every commit.
-test-unit: hooks
-	$(PYTEST) tests/framework tests/stacklets -v --ignore=tests/framework/test_config_to_container.py
+test-unit:
+	@script/test unit
 
-# Docker lifecycle: up, down, destroy, env rendering, container environment.
-# Owns its own throwaway instance, but on fixed names and ports.
-test-lifecycle:
-	$(PYTEST) tests/framework/test_config_to_container.py -v
+test-integration:
+	@script/test integration
 
-# Against the demo instance already running in this checkout. Tests create
-# unique data and clean up after themselves; they never reset the instance.
 test-demo:
-	tests/integration/stacktests demo-rig
+	@script/test demo
 
-# A small subset of the managed-rig e2e lane, for a quick answer.
 test-smoke:
-	tests/integration/stacktests smoke
+	@script/test smoke
 
-# The full managed rig: seeds a test-owned instance and runs every e2e test.
 test-e2e:
-	tests/integration/stacktests e2e
+	@script/test e2e
 
-.PHONY: help hooks lint typecheck test-unit test-lifecycle test-demo test-smoke test-e2e
+.PHONY: help hooks lint typecheck test-unit test-integration test-demo test-smoke test-e2e

docs/adr/adr-011-vault-brain-projection.md

@@ -101,5 +101,5 @@ agent fell back to grep on its `vault/` mount, which is the brain.
 
 Cost: a page filed seconds ago is findable after the next mirror tick,
 not at once. Read-your-writes is tested for todos only
-(`tests/integration/test_demo_rig_e2e.py`), so no stated invariant changes.
+(`tests/e2e/test_demo_rig_e2e.py`), so no stated invariant changes.
 

docs/adr/adr-012-nanobot-fork.md

@@ -53,7 +53,7 @@ supposed to have replaced it has a third state, and that third state is silent.
 
 ### 2. The shim tests cannot catch the thing that actually breaks shims
 
-`tests/stacklets/test_agent_runtime_shims.py` asserts every patch is attached,
+`tests/unit/stacklets/test_agent_runtime_shims.py` asserts every patch is attached,
 which is real value: it catches *our* mistakes. It cannot catch upstream
 moving a symbol, because it runs against a stub nanobot this repo hand-writes.
 The stub still has the old symbol, so the lane stays green while production is

docs/adr/rfc-002-installation-instances-and-updates.md

@@ -168,7 +168,7 @@ any compose project named `stack-*` to a stacklet id. A second instance would
 report the first instance's containers as its own, so `stack list` lies before
 anything collides.
 
-This is why `tests/integration/stacktests` takes the dev instance down rather
+This is why `tests/e2e/stacktests` takes the dev instance down rather
 than running beside it, and why the operator docs say two instances cannot run
 on one Mac.
 

docs/adr/rfc-003-agentic-development-harness.md

@@ -121,7 +121,7 @@ its own containers, both reachable.
 
 ### 5. An instance that says which machine it is (an afternoon)
 
-The rig already has half of this. `tests/integration/stacktests` refuses to
+The rig already has half of this. `tests/e2e/stacktests` refuses to
 run over a real stack unless `.stack/.test-instance` is present, and the
 comment above it says why: "running the rig over someone's real stack errors
 out instead of overwriting it". That sentinel protects one script.

docs/agent/dev.md

@@ -211,9 +211,10 @@ Test runner: **`uv run --extra test pytest`**. The `test` extra in `pyproject.to
 ```bash
 make                 # the lane table: cost, what each needs, when to run it
 make lint            # <1s, the whole tree. The hook already covers staged files.
-make test-unit       # ~75s, offline, no Docker. Before a push, or when a
+make test-unit       # ~50s, offline, no Docker. Before a push, or when a
                      #   piece of work is done. Not per commit.
-make test-lifecycle  # ~6m, Docker. Lifecycle, env rendering, compose, health.
+make test-integration # ~6m, Docker, APFS. Lifecycle, env rendering, compose,
+                     #   health, and the backup engine on real disk images.
 make test-demo       # against the running demo instance.
 make test-smoke      # quick managed-rig e2e subset.
 make test-e2e        # full managed-rig e2e suite.
@@ -231,17 +232,17 @@ Testing rules:
 - **Do not write tests that mirror the implementation.** A test written next to the code it covers proves the two agree, not that either is right. Assert against something external: a spec, a real service's response, an invariant we promise. If a test could only fail when someone changes their mind, delete it.
 - **Behavioural TDD: RED then GREEN.** Write the failing test that captures the behaviour you want; make it pass with the smallest change; then refactor.
 - **Blackbox at the module boundary.** Test what a module promises through its public surface. Mock only external interfaces (network endpoints, the LLM), and only when truly required.
-- **Prefer a real model over a stubbed one.** `tests/integration/stacktests ai local` points the rig at a self-hosted endpoint: real answers, no cost per call, only slower. A green run against a stub proves the wiring, not the behaviour.
+- **Prefer a real model over a stubbed one.** `tests/e2e/stacktests ai local` points the rig at a self-hosted endpoint: real answers, no cost per call, only slower. A green run against a stub proves the wiring, not the behaviour.
 - **Seed through the front door.** See [AGENTS.md non-negotiable 11](../../AGENTS.md). To populate an instance with documents, run `tools/family-docs/ingest.py`: it posts each rendered demo document into the Matrix `documents` room as the family member who would plausibly have sent it, so the archivist picks it up and the whole pipeline runs. Uploading to `POST /api/documents/post_document/` instead is faster and produces the wrong corpus - bare OCR text with a filename for a title, no tags, correspondent, document type, summary note, or vault entry. Search, classification, and mirror behaviour all read differently against that, so measurements taken on it are worthless. The same rule holds for every stacklet: drive the user-facing surface, not the service behind it.
-- **Docker integration tests when warranted.** If a change crosses a container boundary or depends on a real service's behaviour, add a test under `tests/integration/`.
+- **Docker integration tests when warranted.** If a change crosses a container boundary or depends on a real service's behaviour, add a test under `tests/e2e/`.
 - **Tests run against real stacklets and real hooks.** No parallel test-only compose files.
 - **Use real Synapse via the `messages` stacklet.** No handwritten Matrix mocks.
 - **Allowed mocks: loggers only.** Real imports catch real errors.
 - **Established deps over handwritten fakes:** `pytest-httpserver` for HTTP, real services for the rest.
 - **Test helpers do one thing.** Add a parameter only when a second test needs it - not preemptively.
-- **`tests/integration/eval/` is opt-in** (slow, real model). Excluded from `pytest tests/` by `norecursedirs`.
+- **`tests/e2e/eval/` is opt-in** (slow, real model). Excluded from `pytest tests/` by `norecursedirs`.
 - **Write tests before fixing.** No duct tape.
-- **The rig is shared, not off-limits.** This repo root is the Simpsons dev instance, not anyone's real famstack, so agents may run the rig lanes. Ports are fixed, so exactly one run at a time: check nothing else is mid-run before starting. `tests/integration/stacktests help` lists which subcommands are autonomous, shared, or destructive.
+- **The rig is shared, not off-limits.** This repo root is the Simpsons dev instance, not anyone's real famstack, so agents may run the rig lanes. Ports are fixed, so exactly one run at a time: check nothing else is mid-run before starting. `tests/e2e/stacktests help` lists which subcommands are autonomous, shared, or destructive.
 
 ## Code style
 
@@ -385,12 +386,12 @@ subject is not: it appears in plain-text contexts too, including
 | `Upgrade: <what the admin must do>` | Renders as **Action required** at the top of the release. Any update needing a human step: a backup, a config edit, a one-way migration. |
 | `BREAKING CHANGE: <what breaks>` | Same section, for something that breaks an existing setup rather than asking for a step. |
 | `Refs: FAM-12` | Links the tracker card. |
-| `Co-Authored-By:` | Never. Project rule. |
+| `Co-Authored-By:` | Never. Project rule, enforced: `tools/commit-lint` rejects the trailer, a "Generated with Claude Code" line, its link and an Anthropic noreply address, in commit messages and in PR descriptions. |
 
 `tools/commit-lint` enforces the header, and is the same check in every
 place that matters: the `commit-msg` hook rejects it before the commit
 exists, CI rejects the PR title and every commit in the PR, and the
-release gate runs it over the range being tagged. `tools/init-repo`
+release gate runs it over the range being tagged. `script/setup`
 turns the hooks on, as do `make test-unit` and `make typecheck`; git
 cannot do it on clone, by design.
 

docs/design-notes.md

@@ -48,7 +48,7 @@ the same one.** Half a relation is worse than none, because the send
 side looks right in the client while routing silently degrades.
 
 Also why it survived a release: the intent spec that would have caught it
-(`tests/integration/test_room_modes_e2e.py`) is marked
+(`tests/e2e/test_room_modes_e2e.py`) is marked
 `xfail(strict=False)`, which is green when broken and green when fixed.
 A non-strict xfail is not coverage. Making it `strict=True`, skipping it
 with a reason, or deleting it are all better; the action is FAM-6.