From b7383e92ff68b6193d8413b2d5fa4a0b7992822c Mon Sep 17 00:00:00 2001 From: Arthur Date: Sat, 26 Sep 2026 23:58:50 +0200 Subject: [PATCH 1/2] feat(memory)!: bring the wiki up to date now with wiki update The curator regenerates the pages a filing touched after a quiet window, and rebuilds the whole wiki once a night. Checking that a filing reached the wiki meant waiting for one or the other. `stack memory wiki` is now a noun with verbs: - `wiki update` asks the curator for the pages the latest filings touched and waits until they are written. - `wiki update --all` asks it for the nightly rebuild now (diary, source reconcile, every page) and waits for it. - `wiki update --home`, `--member`, `--topic`, `--topics` and `--dry-run` regenerate those pages directly, as the bare command did. - `wiki clean` is unchanged; a bare `wiki` lists the verbs. While waiting, the command prints each step and each page the curator writes or skips. The curator stays the brain's only writer: the command leaves a trigger and waits for the record the curator writes when done. BREAKING CHANGE: `stack memory wiki` no longer regenerates pages on its own. Use `stack memory wiki update --all` for every page, or `stack memory wiki update --home` and the other page flags as before. --- docs/admin-guide.md | 6 +- docs/agent/ops.md | 2 +- stacklets/memory/bot/cli/wiki.py | 25 ++- stacklets/memory/bot/curator.py | 95 ++++++--- stacklets/memory/cli/_curator.py | 145 ++++++++++++++ stacklets/memory/cli/sync.py | 80 ++------ stacklets/memory/cli/wiki.py | 100 +++++++--- stacklets/memory/lib.py | 106 +++++++++- stacklets/memory/seeds/index.md | 2 +- tests/driver/README.md | 2 +- tests/driver/commands.py | 4 +- tests/unit/stacklets/test_memory_curator.py | 30 +++ tests/unit/stacklets/test_memory_cycles.py | 207 ++++++++++++++++++++ 13 files changed, 665 insertions(+), 139 deletions(-) create mode 100644 stacklets/memory/cli/_curator.py create mode 100644 tests/unit/stacklets/test_memory_cycles.py diff --git a/docs/admin-guide.md b/docs/admin-guide.md index c3ee312f..9aaca458 100644 --- a/docs/admin-guide.md +++ b/docs/admin-guide.md @@ -562,7 +562,7 @@ The persistent layer of the household brain: a curated knowledge vault, rendered Setup seeds the vault with three things: the classification ontology (the topics and document types the archivist files against), household facts, and a hand-curated correspondents layer with aliases, so "Springfield Insurance" and "Springfield Ins. Co." resolve to the same page instead of becoming duplicates. -The wiki maintains itself. A curator sidecar (`stack-memory-curator`) watches the vault: when new filings settle it regenerates the pages of the family members involved plus the home page (a couple of LLM calls, a few minutes after the burst), and once a night it rebuilds everything — topic pages, cross-references, the lot — while the GPU has nothing better to do. `./stack memory wiki` stays available as the manual trigger, and `[memory]` in `stack.toml` holds the knobs (see [Configuration](#configuration)). Edits happen in Forgejo (every wiki page links to its source), so the commit log doubles as the household's learning history. +The wiki maintains itself. A curator sidecar (`stack-memory-curator`) watches the vault: when new filings settle it regenerates the pages of the family members involved plus the home page (a couple of LLM calls, a few minutes after the burst), and once a night it rebuilds everything — topic pages, cross-references, the lot — while the GPU has nothing better to do. To bring the wiki up to date now instead of waiting, `./stack memory wiki update` regenerates what the latest filings touched and `./stack memory wiki update --all` runs the whole nightly rebuild; both return when the pages are written. `--member`, `--topic` and `--home` regenerate just those pages, and `[memory]` in `stack.toml` holds the knobs (see [Configuration](#configuration)). Edits happen in Forgejo (every wiki page links to its source), so the commit log doubles as the household's learning history. **The family diary.** Everything posted in `#Memories` becomes a diary entry. Each night the curator transcribes the new recordings, has the AI read each entry (a title, a summary, the people, topics from the ontology, facts), writes one card per entry to the vault under `family/diary/entries/`, and rebuilds the diary pages in the wiki from those cards. At 07:30 local time the archivist posts each new card as a quiet notice in the thread under its memory, and one short notice with the count (`diary_job_at` in the archivist's `bot.toml` sets the time; it catches up after a night the Mac slept through). A family member corrects a card by replying in its thread, typed or spoken; the correction is committed to the vault under their name, and a corrected or hand-edited card is never overwritten by a later run. The archivist says nothing else in `#Memories`. The user guide has the family's side: [The Memories Room](user-guide.md#the-memories-room). @@ -575,7 +575,9 @@ Useful commands: ```bash ./stack memory search # full-text query over the vault -./stack memory wiki # regenerate home, member, and topic pages with the LLM +./stack memory wiki update # bring the latest filings into the wiki, and wait +./stack memory wiki update --all # the nightly rebuild now: diary, sources, every page +./stack memory wiki update --member homer # regenerate one page ./stack memory ontology # sync the vault ontology with the shipped seed ./stack memory lookup # which topic or doctype does a term resolve to? ./stack memory correspondents # inspect the correspondent layer diff --git a/docs/agent/ops.md b/docs/agent/ops.md index 2afdf650..321c87eb 100644 --- a/docs/agent/ops.md +++ b/docs/agent/ops.md @@ -114,7 +114,7 @@ Port collisions: do not silently rebind. Surface them. The user's fix is "stop t | Voice messages fail on a remote AI setup | `./stack ai connect ` output, "Voice messages" line | The AI server lists no speech-to-text model. Pass `--whisper ` for a speech server. | | Disk full | `./stack host` | Likely the photo library. Move `data_dir` to external SSD. | | Element warns "browser not supported", then never loads | Which address is open? | Port mode over the LAN IP: plain HTTP is not a secure context. On the server Mac open `http://localhost:42030`; other computers use the Element desktop app with `http://:42031`. | -| Wiki pages stale after filings | `docker logs stack-memory-curator` | Curator debounces (~3 min quiet) before rebuilding; topic pages wait for the nightly sweep. Manual override: `./stack memory wiki`. | +| Wiki pages stale after filings | `docker logs stack-memory-curator` | Curator debounces (~3 min quiet) before rebuilding; topic pages wait for the nightly sweep. Manual override: `./stack memory wiki update` (what the last filings touched) or `./stack memory wiki update --all` (everything, as at night). | | Wiki shows old content entirely | `docker logs stack-memory-curator` | The curator owns the vault `git pull` (the wiki container is a read-only view). Curator down/stuck → vault and wiki go stale together. | | Curator logs "waiting for vault" | `./stack status` → memory, code | Vault not cloned yet — the memory install hooks own the initial clone. | diff --git a/stacklets/memory/bot/cli/wiki.py b/stacklets/memory/bot/cli/wiki.py index efc6b784..ae4765c3 100644 --- a/stacklets/memory/bot/cli/wiki.py +++ b/stacklets/memory/bot/cli/wiki.py @@ -1,17 +1,22 @@ -"""`stack memory wiki` — regenerate the family wiki's entry pages. +"""`stack memory wiki update` — regenerate the family wiki's entry pages. Walks the memory vault, pulls every document's `> [!summary]` callout, and asks the LLM to compose the browsable pages a family lands on: - stack memory wiki home + members + topics (apply) - stack memory wiki --home just the household home page - stack memory wiki --member homer just Homer's page - stack memory wiki --topic camping just one topic's page - stack memory wiki --topics every topic page, no home/members - stack memory wiki --dry-run preview to stdout, no writes - stack memory wiki clean delete every generated page (asks first) - stack memory wiki clean --dry-run list the pages clean would delete - stack memory wiki clean --yes skip the confirmation (scripted rebuild) + (no flags) home + members + topics: the curator's nightly pass + --home just the household home page + --member homer just Homer's page + --topic camping just one topic's page + --topics every topic page, no home/members + --dry-run preview to stdout, no writes + clean delete every generated page (asks first) + clean --dry-run list the pages clean would delete + clean --yes skip the confirmation (scripted rebuild) + +These are the arguments this module takes inside the bot-runner. Admins +reach the page flags as `stack memory wiki update --member homer` and +`clean` as `stack memory wiki clean`; a bare `stack memory wiki update` +asks the curator instead (host side, `cli/wiki.py`). `--member` and `--topic` repeat and combine with `--home`: any selection flag switches from the full sweep to "generate exactly this diff --git a/stacklets/memory/bot/curator.py b/stacklets/memory/bot/curator.py index 86da9ffc..423009c3 100644 --- a/stacklets/memory/bot/curator.py +++ b/stacklets/memory/bot/curator.py @@ -28,7 +28,7 @@ Failure shape: a failed rebuild keeps the SHA where it was and retries after another quiet window — the trigger is never lost, and -`stack memory wiki` always works as the manual override. If the +`stack memory wiki update` always works as the manual override. If the curator is down, the vault (and therefore the wiki) goes stale together — one component, one failure story. Disabling `wiki_auto_rebuild` stops the rebuilds, NOT the pull: the wiki must @@ -43,6 +43,7 @@ from __future__ import annotations import asyncio +import json import os import subprocess import sys @@ -61,11 +62,15 @@ from memory.lib import ( # noqa: E402 MIRROR_SHA_NAME, MIRROR_TRIGGER_NAME, + NIGHTLY_RUN_NAME, + NIGHTLY_TRIGGER_NAME, + REBUILT_SHA_NAME, PRESERVE_LOCAL, RESET_LOCAL, SyncResult, _parse_frontmatter, authenticated_remote, + report_progress, brain_remote_url, is_auth_failure, reconcile_with_remote, @@ -403,14 +408,14 @@ def _fm(reader, path: str) -> dict: return ops -def consume_trigger(state_dir: Path) -> bool: - """Consume a pending mirror-now trigger. True when one was there. +def consume_trigger(state_dir: Path, name: str = TRIGGER_NAME) -> bool: + """Consume a pending trigger (mirror-now by default). True when one was there. Deletion is the consumption: a concurrent second consumer loses the unlink race and correctly reports no trigger. """ try: - (state_dir / TRIGGER_NAME).unlink() + (state_dir / name).unlink() return True except OSError: return False @@ -764,7 +769,7 @@ def _reconcile(self) -> SyncResult: # ── Rebuild ────────────────────────────────────────────────────────────── -async def compile_diary() -> bool: +async def compile_diary(state_dir: Path | None = None) -> bool: """One diary pass over the memories room, on the nightly sweep. The compiler re-reads the whole room every time, because a reply or @@ -774,19 +779,26 @@ async def compile_diary() -> bool: the nightly pays for recordings and readings that are genuinely new and reuses the rest. """ - return await _run_command("diary", [], "compiling diary") + return await _run_command("diary", [], "compiling the diary", state_dir) -async def rebuild(selection: list[str]) -> bool: +async def rebuild(selection: list[str], state_dir: Path | None = None) -> bool: """One wiki generation pass via the CLI entrypoint — the same code - path `stack memory wiki` execs, in a subprocess so a wedged LLM - call dies with the child instead of inside this loop.""" - label = " ".join(selection) if selection else "(full sweep)" - return await _run_command("wiki", selection, f"rebuilding wiki: {label}") + path `stack memory wiki update` execs, in a subprocess so a wedged + LLM call dies with the child instead of inside this loop.""" + label = " ".join(selection) if selection else "every page" + return await _run_command("wiki", selection, f"regenerating wiki pages: {label}", state_dir) -async def _run_command(command: str, selection: list[str], what: str) -> bool: - logger.info("[curator] {}", what) +async def _run_command(command: str, selection: list[str], what: str, + state_dir: Path | None = None) -> bool: + """Run one entrypoint command in a child process; True when it succeeded. + + The child's lines (one per page: published, unchanged, skipped) go to + the log and, with a `state_dir`, to the progress file a waiting + `stack memory wiki update` prints from, as they come. + """ + _progress(state_dir, what) # Starting the child and waiting on it are separate failure modes, and # only the second one has a child to kill. Keeping them in one block # left `proc.kill()` reachable on a path where `proc` was never bound. @@ -800,10 +812,19 @@ async def _run_command(command: str, selection: list[str], what: str) -> bool: logger.warning("[curator] {} failed to start: {}", command, e) return False + lines: list[str] = [] + + async def follow() -> None: + assert proc.stdout is not None + async for raw in proc.stdout: + line = raw.decode(errors="replace").rstrip() + lines.append(line) + if line.strip() and not line.startswith("{"): + _progress(state_dir, line.strip()[:200]) + await proc.wait() + try: - out_bytes, _ = await asyncio.wait_for( - proc.communicate(), timeout=REBUILD_TIMEOUT_SECS, - ) + await asyncio.wait_for(follow(), timeout=REBUILD_TIMEOUT_SECS) except TimeoutError: proc.kill() logger.warning("[curator] {} timed out after {}s", @@ -813,16 +834,26 @@ async def _run_command(command: str, selection: list[str], what: str) -> bool: logger.warning("[curator] {} failed: {}", command, e) return False - output = out_bytes.decode(errors="replace").strip() if proc.returncode != 0: - tail = "\n".join(output.splitlines()[-5:]) + tail = "\n".join(lines[-5:]) logger.warning("[curator] {} rc={}: {}", command, proc.returncode, tail) return False - published = sum(1 for ln in output.splitlines() if ln.startswith("published ")) + published = sum(1 for ln in lines if ln.startswith("published ")) logger.info("[curator] {} done — {} page(s) published", command, published) return True +def _progress(state_dir: Path | None, what: str, *, fresh: bool = False) -> None: + """One step, to the log and, when someone may be waiting, the progress file.""" + logger.info("[curator] {}", what) + if state_dir is None: + return + try: + report_progress(state_dir, what, fresh=fresh) + except OSError as e: + logger.warning("[curator] could not record progress: {}", e) + + # ── Main loop ──────────────────────────────────────────────────────────── async def main() -> None: @@ -832,7 +863,7 @@ async def main() -> None: brain_dir = Path(os.environ.get("BRAIN_REPO_DIR", "/data/memory/brain")) shared_bucket = os.environ.get("SHARED_BUCKET", "family") state_dir = Path(os.environ.get("CURATOR_STATE_DIR", "/data/memory/curator")) - sha_file = state_dir / "last-rebuilt-sha" + sha_file = state_dir / REBUILT_SHA_NAME mirror_file = state_dir / MIRROR_SHA_NAME nightly_file = state_dir / "last-nightly-date" @@ -872,7 +903,7 @@ def _read(path: Path) -> str: return path.read_text(encoding="utf-8").strip() if path.exists() else "" # First boot starts from the current HEAD: existing content is the - # operator's to backfill (`stack memory wiki`), not something to + # operator's to backfill (`stack memory wiki update --all`), not something to # surprise a fresh install with. From then on the SHA survives # restarts, so a rebuild interrupted mid-flight is simply redone. last = _read(sha_file) @@ -976,20 +1007,29 @@ async def mirror_reconcile() -> bool: # date is recorded even on failure: one attempt per night, the # incremental path and the manual CLI cover the gap. A full # source reconcile precedes the regen so brain self-heals any - # drift the incremental mirror missed. - if nightly_due(NIGHTLY, _read(nightly_file), time.localtime()): - _write(nightly_file, time.strftime("%Y-%m-%d", time.localtime())) + # drift the incremental mirror missed. `stack memory wiki update --all` + # asks for the same sweep now; that does not count as tonight's. + asked = consume_trigger(state_dir, NIGHTLY_TRIGGER_NAME) + if asked or nightly_due(NIGHTLY, _read(nightly_file), time.localtime()): + if not asked: + _write(nightly_file, time.strftime("%Y-%m-%d", time.localtime())) + _progress(state_dir, "rebuilding the whole wiki", fresh=True) + _progress(state_dir, "reconciling the source vault with the brain") if await mirror_reconcile(): mirror_sha = _write(mirror_file, head) # Diary first: it writes pages into the same working tree, # and the commit below should carry both nights' work. - await compile_diary() - if await rebuild([]): + await compile_diary(state_dir) + rebuilt = await rebuild([], state_dir) + if rebuilt: # Generation wrote pages into brain's working tree; commit # and push them (one commit alongside the reconcile). await brain.commit_push(f"{BRAIN_COMMIT_PREFIX} nightly rebuild") last = _write(sha_file, head) debounce.reset() + # The record a waiting `stack memory wiki update --all` reads. + _write(state_dir / NIGHTLY_RUN_NAME, + json.dumps({"at": time.time(), "ok": rebuilt, "asked": asked})) continue # ── Incremental: persons + home, debounced ──────────────────── @@ -1015,7 +1055,8 @@ async def mirror_reconcile() -> bool: debounce.reset() continue - if await rebuild(selection): + _progress(state_dir, "regenerating the pages new filings touched", fresh=True) + if await rebuild(selection, state_dir): # Commit + push the regenerated pages the wiki CLI wrote into # brain's working tree. await brain.commit_push(f"{BRAIN_COMMIT_PREFIX} incremental rebuild") diff --git a/stacklets/memory/cli/_curator.py b/stacklets/memory/cli/_curator.py new file mode 100644 index 00000000..dbeed64b --- /dev/null +++ b/stacklets/memory/cli/_curator.py @@ -0,0 +1,145 @@ +"""Asking the curator for its work now, and waiting until it is done. + +The curator is the only writer of the brain projection (ADR-011), so no +command mirrors or regenerates anything itself. Each drops a trigger file +the curator's tick loop watches and waits for the record the curator +leaves when it has finished. A timeout is not a lost request: the trigger +stays until the curator is free to take it. + +Three depths, each including the one before: + + mirrored the source vault's HEAD is in the brain projection + pages the wiki pages that change touched are regenerated + everything the nightly rebuild: diary, source reconcile, every page + +While it waits, a command prints what the curator reports doing: each +step, and each page it writes or skips. +""" + +from __future__ import annotations + +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) +from lib import ( # noqa: E402 + curator_state_dir_for, + progress_since, + request_mirror, + request_nightly, + vault_local_head, + vault_path_for, + vault_remote_head, + wait_for_mirror, + wait_for_nightly, + wait_for_rebuilt, +) + +# Longer than a write's own few seconds: a person at a terminal asked for +# this and would rather wait than re-run it. +MIRROR_WAIT_SECS = 40.0 +# Regenerating the touched pages is a few LLM calls per member. +PAGES_WAIT_SECS = 600.0 +# Everything is an LLM call per page plus the diary; half an hour covers +# a family's worth of pages on a slow model. +EVERYTHING_WAIT_SECS = 1800.0 +POLL_INTERVAL = 0.75 + + +class Vault: + """The source vault and the curator's state dir of one instance. + + Resolved from the plugin config; `error` says why it cannot be used. + + vault = Vault.of(config) + if vault.error: + return {"error": vault.error} + """ + + def __init__(self, memory: Path, state_dir: Path, error: str = ""): + self.memory, self.state_dir, self.error = memory, state_dir, error + + @classmethod + def of(cls, config: dict) -> "Vault": + data_dir = config.get("data_dir") + if not data_dir: + return cls(Path(), Path(), "stack data_dir not configured") + memory = vault_path_for(Path(data_dir)) + state_dir = curator_state_dir_for(Path(data_dir)) + if not (memory / ".git").exists(): + return cls(memory, state_dir, f"memory vault not cloned at {memory}") + return cls(memory, state_dir) + + def target_head(self) -> str: + """The commit the curator has to reach. + + Prefers the remote's HEAD: a hand edit pushed from Obsidian is + exactly the change an operator runs this to bring through, and it + is not in the local clone yet. Falls back to the local HEAD when + Forgejo is unreachable; the curator pulls before mirroring, so a + local target is still a valid floor. + """ + return vault_remote_head(self.memory) or vault_local_head(self.memory) or "" + + +class Echo: + """Prints the curator's progress lines recorded since a moment, once each. + + Handed to a wait as its `on_poll`, so the lines appear while it waits. + + echo = Echo(vault.state_dir, time.time()) + wait_for_rebuilt(..., on_poll=echo) + """ + + def __init__(self, state_dir: Path, since: float): + self.state_dir, self.since, self.shown = state_dir, since, 0 + + def __call__(self) -> None: + steps = progress_since(self.state_dir, self.since) + if len(steps) < self.shown: + self.shown = 0 # the curator started a new piece of work + for step in steps[self.shown:]: + print(f" {step}", flush=True) + self.shown = len(steps) + + +def mirrored(vault: Vault, target: str) -> tuple[str | None, str]: + request_mirror(vault.state_dir) + sha = wait_for_mirror(vault.state_dir, vault.memory, target, + timeout=MIRROR_WAIT_SECS, interval=POLL_INTERVAL) + if sha is None: + return None, (f"curator did not mirror {target[:10]} within {int(MIRROR_WAIT_SECS)}s " + "- is the memory stacklet running (or busy rebuilding the wiki)?") + print(f"Brain projection current at {sha[:10]} (source {target[:10]})") + return sha, "" + + +def pages(vault: Vault, target: str, since: float) -> tuple[str | None, str]: + """Wait for the touched pages, printing what the curator did since `since`.""" + echo = Echo(vault.state_dir, since) + sha = wait_for_rebuilt(vault.state_dir, vault.memory, target, + timeout=PAGES_WAIT_SECS, interval=POLL_INTERVAL, on_poll=echo) + echo() + if sha is None: + return None, (f"curator did not regenerate the wiki pages for {target[:10]} within " + f"{int(PAGES_WAIT_SECS)}s - is [memory] wiki_auto_rebuild off, or the AI down?") + print(f"Wiki pages current at {sha[:10]}") + return sha, "" + + +def everything(vault: Vault) -> tuple[int | None, str]: + """The nightly rebuild, now; how many seconds it took.""" + asked = request_nightly(vault.state_dir) + print("Asked the curator to rebuild the whole wiki; waiting until it has finished...") + echo = Echo(vault.state_dir, asked) + record = wait_for_nightly(vault.state_dir, asked, timeout=EVERYTHING_WAIT_SECS, on_poll=echo) + echo() + if record is None: + return None, (f"the curator did not finish a rebuild within {int(EVERYTHING_WAIT_SECS)}s " + "- is the memory stacklet running, and [memory] wiki_auto_rebuild on?") + took = round(record["at"] - asked) + if not record.get("ok"): + return None, (f"the rebuild ran ({took}s) but regenerating the wiki pages failed; " + "see ./stack logs memory") + print(f"Wiki rebuilt in {took}s") + return took, "" diff --git a/stacklets/memory/cli/sync.py b/stacklets/memory/cli/sync.py index 271654a0..3b8ee9c2 100644 --- a/stacklets/memory/cli/sync.py +++ b/stacklets/memory/cli/sync.py @@ -2,20 +2,15 @@ Every write already asks for this on its own way out (`propagate_write`, called from the write seam) and waits about five seconds. This command is -the same two steps with an operator's patience instead of an agent's: it -waits long enough to sit through a nightly sweep, and it says out loud -how far the projection got. +the same two steps with an operator's patience instead of an agent's, and +it says out loud how far the projection got. -The curator is the ONLY writer of the brain projection (one-writer -invariant, ADR-011). This command therefore mirrors nothing itself: it -drops a `mirror-now` trigger file the curator's tick loop watches, then -waits until the curator records the current memory HEAD as mirrored. +It stops at the mirror. `stack memory wiki update` goes on to the pages. -Failure shape: if the curator is down (memory stacklet stopped) or busy -past the wait cap (a nightly LLM sweep can hold a cycle for minutes), -this times out with the lag printed and a nonzero exit. The trigger file -survives either way; the curator consumes it on its next free tick, so -the request is never lost. +The curator is the only writer of the brain projection (ADR-011), so this +mirrors nothing itself: it asks the curator and waits (`_curator.py`). If +the curator is down or busy past the wait, this times out with a nonzero +exit, and the curator still takes the request on its next free tick. """ from __future__ import annotations @@ -23,61 +18,20 @@ import sys from pathlib import Path -sys.path.insert(0, str(Path(__file__).resolve().parent.parent)) -from lib import ( # noqa: E402 - curator_state_dir_for, - request_mirror, - vault_local_head, - vault_path_for, - vault_remote_head, - wait_for_mirror, -) +sys.path.insert(0, str(Path(__file__).resolve().parent)) +from _curator import Vault, mirrored # noqa: E402 HELP = "Mirror memory source into the brain projection now" -# Wait cap per the test-loop rule: polls are cheap, caps are hard. Far -# longer than a write's own wait, because a person at a terminal asked -# for this by hand and would rather wait than re-run it. -WAIT_SECS = 40.0 -POLL_INTERVAL = 0.75 - - -def target_head(memory: Path) -> str: - """The commit the mirror has to reach. - - Prefers the remote's HEAD: a hand edit pushed from Obsidian is - exactly the change an operator runs this to bring through, and it is - not in the local clone yet. Falls back to the local HEAD when Forgejo - is unreachable - the curator pulls before mirroring, so a local-only - target is still a valid best-effort floor. - """ - return vault_remote_head(memory) or vault_local_head(memory) or "" - def run(args, stacklet, config): - data_dir = config.get("data_dir") - if not data_dir: - return {"error": "stack data_dir not configured"} - - memory = vault_path_for(Path(data_dir)) - state_dir = curator_state_dir_for(Path(data_dir)) - - if not (memory / ".git").exists(): - return {"error": f"memory vault not cloned at {memory}"} - - target = target_head(memory) + vault = Vault.of(config) + if vault.error: + return {"error": vault.error} + target = vault.target_head() if not target: return {"error": "cannot resolve memory HEAD"} - - request_mirror(state_dir) - mirrored = wait_for_mirror( - state_dir, memory, target, timeout=WAIT_SECS, interval=POLL_INTERVAL, - ) - if mirrored is None: - return {"error": ( - f"curator did not mirror {target[:10]} within {int(WAIT_SECS)}s " - "- is the memory stacklet running (or busy in a nightly sweep)?" - )} - - print(f"Brain projection current at {mirrored[:10]} (source {target[:10]})") - return {"mirrored": mirrored, "target": target} + sha, err = mirrored(vault, target) + if err: + return {"error": err} + return {"mirrored": sha, "target": target} diff --git a/stacklets/memory/cli/wiki.py b/stacklets/memory/cli/wiki.py index 3414e954..972d1dbb 100644 --- a/stacklets/memory/cli/wiki.py +++ b/stacklets/memory/cli/wiki.py @@ -1,41 +1,91 @@ -"""stack memory wiki — regenerate the family wiki's entry pages. +"""stack memory wiki - the family wiki: bring it up to date, or clear it. -Walks the source vault, asks the LLM to compose the household home -page, a personal page for every household member, and a topic page -for every topic folder (bootstrapped from `Thema:` / `Topic:` Matrix -rooms); writes the result into the brain projection working copy. The -curator commits and pushes brain and the wiki container picks the -change up within seconds. The memory (source) repo is never written. + stack memory wiki list these verbs + stack memory wiki update the latest filings into the wiki, and wait + stack memory wiki update --all the whole wiki, as at night: diary, sources, every page + stack memory wiki update --home just the household home page + stack memory wiki update --member homer just one member's page + stack memory wiki update --topic camping just one topic's page + stack memory wiki update --topics every topic page, no home or members + stack memory wiki update --dry-run preview home, member and topic pages, write nothing + stack memory wiki clean delete every generated page (asks first) -Apply by default; `--dry-run` is the opt-in preview that streams the -generated pages to stdout without writing anywhere. - - stack memory wiki # home + members + topics (apply) - stack memory wiki --home # just the household home page - stack memory wiki --member homer # just one member's page - stack memory wiki --topic camping # just one topic's page - stack memory wiki --topics # every topic page, no home/members - stack memory wiki --dry-run # preview, no writes +The curator keeps the wiki current on its own: it regenerates what a +filing touched once the filings settle, and rebuilds everything once a +night. `update` and `update --all` ask it for that work now and wait +until it is done (`_curator.py`), so the curator stays the brain's only +writer. The page flags regenerate the pages named, straight away, in the +bot-runner (`bot/cli/wiki.py`); the curator commits what they write. Updates use a splice contract: the LLM-generated body lives inside ` ... ` markers in -each page. Everything outside the markers — a welcome line, custom -frontmatter, hand-written notes — survives every regen. - -The command runs inside `stack-core-bot-runner` (the bot-runner has -the LLM client and the rendered env); the host wrapper is a thin -docker-exec. +each page. Everything outside the markers, a welcome line, custom +frontmatter, hand-written notes, survives every regeneration. """ -HELP = "Regenerate the family wiki's home, member, and topic pages" +from __future__ import annotations import sys +import time from pathlib import Path sys.path.insert(0, str(Path(__file__).parent)) +import _curator # noqa: E402 from _common import dispatch # noqa: E402 +HELP = "The family wiki: update it (the latest filings, a page, or all of it), or clean it" + +VERBS = { + "update": "bring the latest filings into the wiki; --all rebuilds all of it, " + "--home/--member/--topic/--topics regenerate those pages", + "clean": "delete every generated page (asks first)", +} + +# Flags that name pages to regenerate directly, rather than asking the curator. +PAGE_FLAGS = ("--home", "--member", "--topic", "--topics", "--dry-run", "--dry") + def run(args, stacklet, config): - argv = sys.argv[3:] - return dispatch("wiki", *argv) + verb, rest = (args[0], list(args[1:])) if args else ("", []) + if verb == "update": + return _update(rest, config) + if verb == "clean": + return dispatch("wiki", "clean", *rest) + if verb: + return {"error": f"unknown wiki command {verb!r}; use one of: {', '.join(VERBS)}"} + return {"commands": VERBS} + + +def _update(flags: list[str], config: dict) -> dict: + pages_named = [f for f in flags if f in PAGE_FLAGS] + if "--all" in flags and pages_named: + return {"error": f"--all rebuilds every page; drop {' '.join(pages_named)} or --all"} + if pages_named: + return dispatch("wiki", *flags) + vault = _curator.Vault.of(config) + if vault.error: + return {"error": vault.error} + if "--all" in flags: + return _everything(vault) + return _latest(vault) + + +def _latest(vault: _curator.Vault) -> dict: + target = vault.target_head() + if not target: + return {"error": "cannot resolve memory HEAD"} + asked = time.time() + mirrored, err = _curator.mirrored(vault, target) + if err: + return {"error": err} + rebuilt, err = _curator.pages(vault, target, since=asked) + if err: + return {"error": err} + return {"mirrored": mirrored, "rebuilt": rebuilt, "target": target} + + +def _everything(vault: _curator.Vault) -> dict: + took, err = _curator.everything(vault) + if err: + return {"error": err} + return {"ok": True, "seconds": took} diff --git a/stacklets/memory/lib.py b/stacklets/memory/lib.py index c060c747..a4a02398 100644 --- a/stacklets/memory/lib.py +++ b/stacklets/memory/lib.py @@ -711,6 +711,14 @@ def _preserve_and_reset(git, local_head: str, remote_head: str) -> SyncResult: MIRROR_TRIGGER_NAME = "mirror-now" # The file the curator writes after each successful projection. MIRROR_SHA_NAME = "last-mirrored-sha" +# The memory commit the wiki pages were last regenerated up to. +REBUILT_SHA_NAME = "last-rebuilt-sha" +# Asks the curator for its nightly sweep now, whatever the time. +NIGHTLY_TRIGGER_NAME = "nightly-now" +# When the last nightly sweep finished, and whether its rebuild succeeded. +NIGHTLY_RUN_NAME = "last-nightly-run" +# What the curator is doing: one timestamped line per step and per page. +PROGRESS_NAME = "progress" # How long a write waits for its own change to become visible. Short on # purpose: the curator slices its sleep by the second and a mirror is @@ -753,26 +761,110 @@ def mirrored_contains(memory: Path, target: str, mirrored: str) -> bool: def wait_for_mirror(state_dir: Path, memory: Path, target: str, *, timeout: float = MIRROR_WAIT_SECS, - interval: float = MIRROR_POLL_INTERVAL) -> Optional[str]: + interval: float = MIRROR_POLL_INTERVAL, + on_poll: Callable[[], None] | None = None) -> Optional[str]: """Poll `last-mirrored-sha` until it contains `target`. Returns the mirrored sha, or None on timeout. The condition is checked once before the clock is consulted, so an already-current mirror succeeds even with a zero timeout. """ - sha_file = Path(state_dir) / MIRROR_SHA_NAME + return _wait_for_sha(Path(state_dir) / MIRROR_SHA_NAME, memory, target, timeout, interval, on_poll) + + +def wait_for_rebuilt(state_dir: Path, memory: Path, target: str, *, + timeout: float, interval: float = MIRROR_POLL_INTERVAL, + on_poll: Callable[[], None] | None = None) -> Optional[str]: + """Poll `last-rebuilt-sha` until the wiki pages are regenerated up to `target`. + + The curator records it after regenerating the pages a change touched, + and also when a change touched none, so it always moves on. + """ + return _wait_for_sha(Path(state_dir) / REBUILT_SHA_NAME, memory, target, timeout, interval, on_poll) + + +def _wait_for_sha(sha_file: Path, memory: Path, target: str, timeout: float, + interval: float, on_poll: Callable[[], None] | None) -> Optional[str]: deadline = time.monotonic() + timeout while True: - mirrored = "" - if sha_file.exists(): - mirrored = sha_file.read_text(encoding="utf-8").strip() - if mirrored_contains(memory, target, mirrored): - return mirrored + if on_poll: + on_poll() + recorded = sha_file.read_text(encoding="utf-8").strip() if sha_file.exists() else "" + if mirrored_contains(memory, target, recorded): + return recorded if time.monotonic() >= deadline: return None time.sleep(interval) +# ── The nightly sweep, on request ──────────────────────────────────────── +# Tonight's sweep (diary, full source reconcile, every wiki page) can be +# asked for now. The curator stays the one writer: this drops a trigger, +# wakes its tick, and waits for the run record the sweep leaves behind. + +def request_nightly(state_dir: Path) -> float: + """Ask for the nightly sweep now; the time asked, to wait against.""" + state_dir.mkdir(parents=True, exist_ok=True) + asked = time.time() + (state_dir / NIGHTLY_TRIGGER_NAME).write_text(str(asked), encoding="utf-8") + request_mirror(state_dir) # wakes the tick within a second + return asked + + +def last_nightly_run(state_dir: Path) -> Optional[dict]: + """The last finished sweep: {"at": epoch seconds, "ok": bool}, or None.""" + record = Path(state_dir) / NIGHTLY_RUN_NAME + try: + return json.loads(record.read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + + +def wait_for_nightly(state_dir: Path, asked: float, *, timeout: float, + interval: float = 2.0, + on_poll: Callable[[], None] | None = None) -> Optional[dict]: + """Wait for a sweep that finished after `asked`; its record, or None.""" + deadline = time.monotonic() + timeout + while True: + if on_poll: + on_poll() + run = last_nightly_run(state_dir) + if run and run.get("at", 0) >= asked: + return run + if time.monotonic() >= deadline: + return None + time.sleep(interval) + + +# ── What the curator is doing ──────────────────────────────────────────── +# A command waiting on the curator would otherwise sit silent for minutes. +# The curator appends one line per step and per page it writes, stamped +# with the time; a waiting command prints the lines stamped after it asked. +# The file starts over with each piece of work, so it stays small. + +def report_progress(state_dir: Path, what: str, *, fresh: bool = False) -> None: + """Record one step of the curator's work; `fresh` starts a new piece of work.""" + with open(Path(state_dir) / PROGRESS_NAME, "w" if fresh else "a", encoding="utf-8") as log: + log.write(f"{time.time()!r} {what}\n") + + +def progress_since(state_dir: Path, since: float) -> list[str]: + """The steps the curator recorded at or after `since`, oldest first.""" + try: + lines = (Path(state_dir) / PROGRESS_NAME).read_text(encoding="utf-8").splitlines() + except OSError: + return [] + steps = [] + for line in lines: + stamp, _, what = line.partition(" ") + try: + if float(stamp) >= since: + steps.append(what) + except ValueError: + continue + return steps + + def propagate_write(data_dir: Path, *, timeout: float = MIRROR_WAIT_SECS, interval: float = MIRROR_POLL_INTERVAL) -> bool: diff --git a/stacklets/memory/seeds/index.md b/stacklets/memory/seeds/index.md index 3ba164ab..efd1f000 100644 --- a/stacklets/memory/seeds/index.md +++ b/stacklets/memory/seeds/index.md @@ -23,7 +23,7 @@ the documents you file and the notes you save, and refreshes it every night. To build it now instead of waiting, run: ```sh -stack memory wiki +stack memory wiki update --all ``` diff --git a/tests/driver/README.md b/tests/driver/README.md index 89098082..af0a197c 100644 --- a/tests/driver/README.md +++ b/tests/driver/README.md @@ -70,7 +70,7 @@ member is their user name (`marge`). `--json` on `as`, `read`, `answer` and Some of the chain from a message to the wiki runs on a timer: the curator regenerates pages after a quiet window following a filing, and sweeps everything once a night. `driver cycle` runs that work now and returns when it -is done, through `stack memory sync --pages` and `stack memory nightly`. In +is done, through `stack memory wiki update` and `stack memory wiki update --all`. In the protocol it is a `CYCLE` line, so a reader tells what the family did from what the stack did on its own: diff --git a/tests/driver/commands.py b/tests/driver/commands.py index cb1fe937..f88019e6 100644 --- a/tests/driver/commands.py +++ b/tests/driver/commands.py @@ -81,9 +81,9 @@ def stack(args, instance: Instance) -> None: # runs it now and waits until it is done. CYCLES = { "curator": ("the curator regenerates the pages the last filing touched " - "(on its own after a quiet window)", ("memory", "sync", "--pages")), + "(on its own after a quiet window)", ("memory", "wiki", "update")), "nightly": ("the curator's nightly sweep: diary, source reconcile, every page " - "(on its own at 03:30)", ("memory", "nightly")), + "(on its own at 03:30)", ("memory", "wiki", "update", "--all")), } diff --git a/tests/unit/stacklets/test_memory_curator.py b/tests/unit/stacklets/test_memory_curator.py index 34b110d8..773a7174 100644 --- a/tests/unit/stacklets/test_memory_curator.py +++ b/tests/unit/stacklets/test_memory_curator.py @@ -427,6 +427,12 @@ def test_consumes_and_reports_pending_trigger(self, tmp_path): def test_no_trigger_is_false(self, tmp_path): assert consume_trigger(tmp_path) is False + def test_a_named_trigger_is_consumed_on_its_own(self, tmp_path): + # `stack memory wiki update --all` drops its own trigger next to mirror-now. + (tmp_path / "nightly-now").write_text("now", encoding="utf-8") + assert consume_trigger(tmp_path) is False + assert consume_trigger(tmp_path, "nightly-now") is True + class TestSleepUntilTick: async def test_pending_trigger_wakes_immediately(self, tmp_path): @@ -493,3 +499,27 @@ async def test_a_child_that_cannot_start_is_false_not_a_crash( ): monkeypatch.setattr(curator.sys, "executable", "/nonexistent/python") assert await curator.rebuild([]) is False + + async def test_every_page_the_child_reports_reaches_the_progress_file( + self, tmp_path, monkeypatch, + ): + # An admin waiting on `stack memory wiki update` sees each page as + # the child writes it, not a count once the whole pass is over. + from memory.lib import progress_since + script = tmp_path / "pages.py" + script.write_text( + "import sys\n" + "print('published family/home.md', file=sys.stderr, flush=True)\n" + "print('unchanged homer/about.md, skipped', file=sys.stderr, flush=True)\n", + encoding="utf-8", + ) + monkeypatch.setattr(curator, "ENTRYPOINT", str(script)) + started = time.time() + + assert await curator.rebuild(["--home"], tmp_path) is True + + assert progress_since(tmp_path, started) == [ + "regenerating wiki pages: --home", + "published family/home.md", + "unchanged homer/about.md, skipped", + ] diff --git a/tests/unit/stacklets/test_memory_cycles.py b/tests/unit/stacklets/test_memory_cycles.py new file mode 100644 index 00000000..a257e8f4 --- /dev/null +++ b/tests/unit/stacklets/test_memory_cycles.py @@ -0,0 +1,207 @@ +"""Bringing the wiki up to date now, from the CLI. + +The curator works on its own clock: after a filing it waits for a quiet +window before regenerating the pages the change touched, and once a night it +sweeps everything (diary, source reconcile, every page). A person checking a +filing, or an agent driving a scenario, cannot wait for either. So each has a +command that asks for it now and waits until it is done: + +- `stack memory wiki update`: the cycle after a filing, pages included. +- `stack memory wiki update --all`: tonight's rebuild, the whole wiki. + +`stack memory sync` stops earlier, at the mirror. + +The curator stays the brain's only writer (ADR-011): a command drops a +trigger and waits for the record the curator leaves. Here a thread plays the +curator, answering exactly that protocol: triggers in, record files out. +""" + +from __future__ import annotations + +import importlib.util +import json +import subprocess +import sys +import threading +import time +from pathlib import Path + +_MEMORY_DIR = Path(__file__).resolve().parents[3] / "stacklets" / "memory" +sys.path.insert(0, str(_MEMORY_DIR)) + +from lib import ( # noqa: E402 + MIRROR_SHA_NAME, + MIRROR_TRIGGER_NAME, + NIGHTLY_RUN_NAME, + NIGHTLY_TRIGGER_NAME, + REBUILT_SHA_NAME, + curator_state_dir_for, + progress_since, + report_progress, + request_nightly, + vault_path_for, + wait_for_nightly, +) + + +def _command(name: str): + spec = importlib.util.spec_from_file_location(f"memory_cli_{name}", _MEMORY_DIR / "cli" / f"{name}.py") + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module + + +def _vault(tmp_path: Path) -> tuple[Path, str]: + """A data dir whose memory clone holds one commit; its HEAD.""" + data_dir = tmp_path / "data" + vault = vault_path_for(data_dir) + vault.mkdir(parents=True) + run = lambda *a: subprocess.run(["git", "-C", str(vault), *a], check=True, capture_output=True, text=True) # noqa: E731 + run("init", "-q") + run("-c", "user.email=t@local", "-c", "user.name=t", "commit", "-q", "--allow-empty", "-m", "filed") + return data_dir, run("rev-parse", "HEAD").stdout.strip() + + +def _curator(state: Path, answer, delay: float = 0.05) -> threading.Timer: + timer = threading.Timer(delay, answer) + timer.start() + return timer + + +class TestTheCycleAfterAFiling: + + def test_update_waits_until_the_touched_pages_are_regenerated(self, tmp_path): + data_dir, head = _vault(tmp_path) + state = curator_state_dir_for(data_dir) + + def curator(): + (state / MIRROR_SHA_NAME).write_text(head) + time.sleep(0.1) + (state / REBUILT_SHA_NAME).write_text(head) + + timer = _curator(state, curator) + result = _command("wiki").run(["update"], {}, {"data_dir": str(data_dir)}) + timer.join() + + assert result == {"mirrored": head, "rebuilt": head, "target": head} + + def test_pages_that_never_come_are_named_as_such(self, tmp_path, monkeypatch): + data_dir, head = _vault(tmp_path) + state = curator_state_dir_for(data_dir) + state.mkdir(parents=True) + (state / MIRROR_SHA_NAME).write_text(head) + wiki = _command("wiki") + monkeypatch.setattr(sys.modules["_curator"], "PAGES_WAIT_SECS", 0.1) + + result = wiki.run(["update"], {}, {"data_dir": str(data_dir)}) + + assert "wiki pages" in result["error"] + + def test_sync_stops_at_the_mirror(self, tmp_path): + data_dir, head = _vault(tmp_path) + state = curator_state_dir_for(data_dir) + state.mkdir(parents=True) + (state / MIRROR_SHA_NAME).write_text(head) + + result = _command("sync").run([], {}, {"data_dir": str(data_dir)}) + + assert result == {"mirrored": head, "target": head} + + +class TestTheWholeWikiNow: + + def test_asking_wakes_the_curator_and_names_the_sweep(self, tmp_path): + state = tmp_path / "curator" + + request_nightly(state) + + assert (state / NIGHTLY_TRIGGER_NAME).exists() + assert (state / MIRROR_TRIGGER_NAME).exists() # the tick wakes within a second + + def test_a_sweep_that_finished_before_asking_does_not_count(self, tmp_path): + state = tmp_path / "curator" + state.mkdir() + (state / NIGHTLY_RUN_NAME).write_text(json.dumps({"at": time.time() - 3600, "ok": True})) + + assert wait_for_nightly(state, time.time(), timeout=0.1, interval=0.02) is None + + def test_update_all_waits_for_the_rebuild_and_says_how_long_it_took(self, tmp_path, monkeypatch): + data_dir, _ = _vault(tmp_path) + state = curator_state_dir_for(data_dir) + + def curator(): + (state / NIGHTLY_RUN_NAME).write_text(json.dumps({"at": time.time(), "ok": True, "asked": True})) + + wiki = _command("wiki") + monkeypatch.setattr(sys.modules["_curator"], "wait_for_nightly", + lambda s, asked, timeout, on_poll: wait_for_nightly( + s, asked, timeout=2, interval=0.02, on_poll=on_poll)) + timer = _curator(state, curator) + result = wiki.run(["update", "--all"], {}, {"data_dir": str(data_dir)}) + timer.join() + + assert result["ok"] is True and result["seconds"] >= 0 + + def test_a_sweep_whose_pages_failed_is_an_error(self, tmp_path, monkeypatch): + data_dir, _ = _vault(tmp_path) + state = curator_state_dir_for(data_dir) + + def curator(): + (state / NIGHTLY_RUN_NAME).write_text(json.dumps({"at": time.time(), "ok": False})) + + wiki = _command("wiki") + monkeypatch.setattr(sys.modules["_curator"], "wait_for_nightly", + lambda s, asked, timeout, on_poll: wait_for_nightly( + s, asked, timeout=2, interval=0.02, on_poll=on_poll)) + timer = _curator(state, curator) + result = wiki.run(["update", "--all"], {}, {"data_dir": str(data_dir)}) + timer.join() + + assert "failed" in result["error"] + + +class TestTheWikiCommand: + """`wiki` is a noun: its verbs say what happens to the wiki.""" + + def test_bare_wiki_lists_its_verbs(self): + assert set(_command("wiki").run([], {}, {})["commands"]) == {"update", "clean"} + + def test_an_unknown_verb_names_the_known_ones(self): + error = _command("wiki").run(["rebuild"], {}, {})["error"] + assert "update" in error and "clean" in error + + def test_all_and_a_page_selection_contradict_each_other(self): + error = _command("wiki").run(["update", "--all", "--member", "homer"], {}, {})["error"] + assert "--all" in error and "--member" in error + + +class TestProgressWhileWaiting: + """A command waiting on the curator prints what it does, as it does it.""" + + def test_update_prints_each_page_the_curator_reports(self, tmp_path, capsys): + data_dir, head = _vault(tmp_path) + state = curator_state_dir_for(data_dir) + + def curator(): + (state / MIRROR_SHA_NAME).write_text(head) + report_progress(state, "regenerating the pages new filings touched", fresh=True) + report_progress(state, "published homer/about.md") + time.sleep(0.8) # one poll of the waiting command sees the lines first + (state / REBUILT_SHA_NAME).write_text(head) + + timer = _curator(state, curator) + _command("wiki").run(["update"], {}, {"data_dir": str(data_dir)}) + timer.join() + + out = capsys.readouterr().out + assert out.count("published homer/about.md") == 1 + assert out.index("regenerating") < out.index("published homer/about.md") + + def test_steps_from_before_the_request_are_not_shown(self, tmp_path): + state = tmp_path / "curator" + state.mkdir() + report_progress(state, "yesterday's rebuild") + asked = time.time() + report_progress(state, "today's rebuild") + + assert progress_since(state, asked) == ["today's rebuild"] From 50955117274900a9be5216968f6c0aeab4a7b8df Mon Sep 17 00:00:00 2001 From: Arthur Date: Sun, 27 Sep 2026 14:10:07 +0200 Subject: [PATCH 2/2] docs: set the rules stack commands follow Every stacklet adds commands, and without written rules each one names its commands and flags its own way. `docs/stack-reference.md` now lists them under CLI Commands: name what the admin wants, not the component or the schedule; nouns with verbs; one meaning per flag; apply by default; wait with progress on background work; return a dict. The CLI module header and the engineer guide point to them. --- docs/agent/dev.md | 2 +- docs/stack-reference.md | 26 ++++++++++++++++++++++++++ lib/stack/cli.py | 5 +++++ 3 files changed, 32 insertions(+), 1 deletion(-) diff --git a/docs/agent/dev.md b/docs/agent/dev.md index 357d5e03..28f4e9ac 100644 --- a/docs/agent/dev.md +++ b/docs/agent/dev.md @@ -132,7 +132,7 @@ Shell hooks (`.sh`) receive env vars: all rendered vars plus `FAMSTACK_DATA_DIR` ## CLI plugin contract -Files under `stacklets//cli/.py` (excluding `_*.py` and `post_setup.py`) become `./stack `. +Files under `stacklets//cli/.py` (excluding `_*.py` and `post_setup.py`) become `./stack `. Naming, verbs and flags follow the command rules in [../stack-reference.md § CLI Commands](../stack-reference.md#command-rules); check a new or changed command against them. ```python HELP = "One-line description for --help" diff --git a/docs/stack-reference.md b/docs/stack-reference.md index c89fbec9..a642ed1f 100644 --- a/docs/stack-reference.md +++ b/docs/stack-reference.md @@ -810,6 +810,32 @@ The convention: Files starting with `_` are private helpers (e.g., `_matrix.py`), not exposed as commands. +### Command rules + +Every stacklet adds commands, so these keep them reading as one CLI. +Check a new command, or a changed one, against them. + +1. **Name the thing or the action, as an admin sees it.** Never the + component that does the work (`curator`, `archivist`) and never when + it usually runs (`nightly`): those change, the admin's intent does not. +2. **A thing with more than one action is a noun with verbs.** + `stack memory wiki update`, `stack memory wiki clean`, + `stack messages room list`. The bare noun lists its verbs, and the + noun leaves room for a later verb such as `show`. +3. **Flags narrow or modify, and one word means one thing everywhere.** + `--all` everything, `--dry-run` preview and write nothing, `--yes` + skip the confirmation, `--as ` act as that family member. + `--json` belongs to the framework. Flags that contradict each other + are refused in one line that names both. +4. **A command that changes something applies by default.** `--dry-run` + is the opt-in preview, never the other way round. +5. **A command that asks a background worker waits until the work is + done**, prints each step and each item while it waits, and on timeout + says how far it got. The request itself survives the timeout. +6. **A command returns a dict**, `{"error": ...}` on failure, which exits + non-zero. With `--json` that dict is all of stdout; what the command + prints for people goes to stderr. + A command that records a choice in `stack.toml` calls `config["set_cfg"](section, key, value)`, the plugin-side twin of a hook's `ctx.cfg(key, value)`. It changes that one line in that section diff --git a/lib/stack/cli.py b/lib/stack/cli.py index df444fbd..241472ce 100644 --- a/lib/stack/cli.py +++ b/lib/stack/cli.py @@ -11,6 +11,11 @@ All framework logic lives in the Stack class. Docker operations use the docker module. This file is the glue. + +Stacklet commands (`stacklets//cli/.py`) follow the command +rules in docs/stack-reference.md § CLI Commands: named for what the +admin wants, nouns with verbs, one meaning per flag, apply by default, +wait with progress, return a dict. """ from __future__ import annotations