From 2712e8339b1bce8174622afba58e136c9f7dd253 Mon Sep 17 00:00:00 2001 From: tweinmann Date: Wed, 28 Oct 2026 21:18:52 +0000 Subject: [PATCH 1/3] Propose phase 2: overview and budget (v0.2) Proposal, delta specs, design and tasks for siding ui, the memory budget, gc, lint, the label dev.siding.gitdir and just build-darwin. Docker-outside-of-Docker is dropped: devcontainers are Docker-in-Docker only. Co-Authored-By: Claude Opus 5.5 --- .../.openspec.yaml | 1 - .../phase-2-overview-and-budget/design.md | 168 ++++++++++++++++++ .../phase-2-overview-and-budget/proposal.md | 62 +++++-- .../specs/agent-skill/spec.md | 14 ++ .../specs/cli-contract/spec.md | 41 +++++ .../specs/compose-lint/spec.md | 59 ++++++ .../specs/dev-environment/spec.md | 36 ++++ .../specs/memory-budget/spec.md | 57 ++++++ .../specs/overview/spec.md | 52 ++++++ .../specs/proxy/spec.md | 50 ++++++ .../specs/stack-gc/spec.md | 73 ++++++++ .../specs/stack-identity/spec.md | 22 +++ .../specs/stack-lifecycle/spec.md | 29 +++ .../specs/stack-list/spec.md | 24 +++ .../phase-2-overview-and-budget/tasks.md | 64 +++++++ 15 files changed, 735 insertions(+), 17 deletions(-) create mode 100644 openspec/changes/phase-2-overview-and-budget/design.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/agent-skill/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/cli-contract/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/compose-lint/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/dev-environment/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/memory-budget/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/overview/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/stack-gc/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/stack-identity/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/stack-lifecycle/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/specs/stack-list/spec.md create mode 100644 openspec/changes/phase-2-overview-and-budget/tasks.md diff --git a/openspec/changes/phase-2-overview-and-budget/.openspec.yaml b/openspec/changes/phase-2-overview-and-budget/.openspec.yaml index 0d31f8b..1bb6456 100644 --- a/openspec/changes/phase-2-overview-and-budget/.openspec.yaml +++ b/openspec/changes/phase-2-overview-and-budget/.openspec.yaml @@ -1,3 +1,2 @@ schema: spec-driven created: 2026-10-28 -skip_specs: true diff --git a/openspec/changes/phase-2-overview-and-budget/design.md b/openspec/changes/phase-2-overview-and-budget/design.md new file mode 100644 index 0000000..768563a --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/design.md @@ -0,0 +1,168 @@ +# Design + +## Context + +v0.1 (archived as `2026-10-28-phase-1-stacks-and-routing`) has every building block this phase +reuses: `ps` collects stacks and memory from the Docker API, `down` knows how to remove a stack +by project name and how to guard its volumes, `internal/stack` derives routes from the +normalised compose config, and `cli.Options` holds every seam to the machine. Motivation and +scope: see proposal.md; behaviour: see the delta specs. Constraints from CLAUDE.md that shape +this phase: no background service, Docker is the database (only a losable cache under +`~/.siding`), never lose work, the compose file is never edited. + +## Goals / Non-Goals + +**Goals:** +- One machine-wide view and one machine-wide limit, built on what `ps` already reads. +- `gc` and `down --volumes` share one notion of "merged", tested against real git repositories. +- Every new command fully testable with fakes; `just smoke-stacks` covers `ui`, the budget + refusal and `gc` against the inner daemon. + +**Non-Goals:** +- Actions in the web page (start, stop, gc): the page is read-only; actions stay in the CLI. +- A queue instead of refusing, learned memory peaks, a parallel build limit: v0.3. +- Removing git worktrees or branches. +- Docker-outside-of-Docker, in this repository's devcontainer or as a supported setup for + siding inside a user's devcontainer. + +## Decisions + +### D1. `siding ui` in the foreground (maintainer's decision) +`siding ui` serves the page while it runs and removes its route when it stops. It keeps the +guardrail "no background service". Alternatives, asked: a container `siding-ui` next to the +proxy (always there, but a background service and a published Linux image of siding, since the +macOS binary cannot run in a container); a static HTML file (no server, but no live data and +no `siding.localhost`). + +### D2. How the proxy reaches `siding ui` +Traefik's file provider watches `~/.siding/proxy//routes/`, mounted read-only into +`siding-proxy`. `siding ui` writes `ui.yml` there (atomically, via rename) with a router for +`Host(\`siding.localhost\`)` to `http://host.docker.internal:`, and removes it on exit. +The address `host.docker.internal` depends on where the daemon runs, which `setup` decides +once: +- The gateway of the network `siding` is an address of this machine (Docker Engine on Linux, + and the devcontainer's inner daemon): `extra_hosts: host.docker.internal:`, and + `siding ui` listens on that gateway and on `127.0.0.1`. +- Otherwise the daemon runs in a VM (Docker Desktop, OrbStack): `extra_hosts: + host.docker.internal:host-gateway`, and `siding ui` listens on `127.0.0.1`, which these VMs + forward to. + +Tried in the inner daemon before this proposal: `host-gateway` resolves to the default +bridge's gateway, not the proxy network's, and gave 502; the pinned gateway gave 200, and +removing the file gave 404 within two seconds. The VM case is tested by hand on the Mac +(task 9.3), which is what `just build-darwin` is for. Never listening on `0.0.0.0` keeps the +page off the LAN. Alternatives: a dummy container with Traefik labels that forwards to the +host (one more container, still needs the same address); the Traefik API (needs a second +published port). + +### D3. The page +Server-rendered with `html/template`, refreshed by ``, +no JavaScript and no assets outside the binary. The handlers call the same functions as +`ps --json` and `gc --dry-run --json`, so `/api/stacks` and `/api/gc` are those documents byte +for byte. Port: `--port`, else a free one from the kernel; `ui` prints both addresses. +Alternative: a JavaScript front end polling the JSON endpoints (nicer, but a build step and a +second code base for a read-only table). + +### D4. Budget on the proxy (maintainer's decision) +The label `dev.siding.budget` (bytes) on `siding-proxy`, written into the proxy's compose file +by `setup --budget`; `setup` without `--budget` carries the running proxy's label forward; +`--budget default` drops it. Without a label the budget is 75 % of `docker info`'s `MemTotal` +(the VM's memory on Docker Desktop). Sizes parse as `[k|m|g]` with binary units, or plain +bytes. Alternatives, asked: refuse only when the use already reached the budget (a new stack +may overshoot it by gigabytes); a configuration file under `~/.siding` (changes the guardrail +that `~/.siding` is only a losable cache). The new label goes into the conventions in CLAUDE.md +and `openspec/config.yaml`. + +### D5. The estimate and the check +`up` checks after reading the config and before writing the override: use = memory of the +running containers of every siding stack except this project (the same stats `ps` reads), +estimate = per service and replica (`deploy.replicas`, `scale`), `deploy.resources.limits.memory`, +else `mem_limit`, else 256 MiB. Refuse when use + estimate > budget. The 256 MiB default is a +guess that v0.3 replaces with learned peaks. `--force` skips the refusal and warns. + +### D6. setup recreates through Compose +Instead of comparing port and image itself, `setup` always rewrites the proxy's compose file +and runs `docker compose up -d --wait`; Compose recreates the container only when the file +differs. `setup` compares the container ID before and after to report `created`, `updated` +or `unchanged`. This covers the budget label and the new file provider, and a v0.1 proxy is +upgraded by the next `setup` on the port it had. Alternative: a label with a hash of the +configuration (one more label to keep in step with what Compose already compares). + +### D7. "Merged" (maintainer's decision for gc's scope; the shared definition decided here) +`internal/git` gains one function for both commands: merged = the tip is an ancestor of the +default branch, or the upstream branch is gone (`%(upstream:track)` is `[gone]`, which a squash +merge with a deleted remote branch leaves after `git fetch --prune`). `gc` adds two conditions: +not the default branch itself, and the default branch has moved past the tip. Without them a +branch an agent just created, which has no commit of its own, would count as merged and its +running stack would be removed. `down --volumes` keeps v0.1's rule plus the squash case: +removing the data of a branch without own commits loses no work. Alternatives: only ancestry +(squash merges always need `--force`); the GitHub API (a network call and a token). + +### D8. `gc` (maintainer's decision: stacks only, never worktrees) +A pure planning function takes the stacks of this machine (`dev.siding.machine` equals ours) +and git facts per stack and returns actions; the command fetches the facts, prints the plan, +asks, and removes with `docker compose -p down --remove-orphans [--volumes]`. For a +stack whose worktree is gone, git runs as `git --git-dir=`. The label goes +on containers only: a new label on networks makes Compose recreate them (seen in Phase 1), +which would stop every v0.1 stack at its first v0.2 `up`; on containers it costs one recreate +of the containers, no data. Stacks without the label are treated as unmerged. Asking reads +`Options.Stdin` when `Options.IsTerminal()` is true; otherwise `--yes` or `--dry-run` is +required (`NotInteractive`). Alternatives, asked: also `git worktree remove` for merged, clean +worktrees; only stacks whose worktree is gone. + +### D9. `lint` +The rules and severities are those of the spec `compose-lint`. Lint reads the same normalised +config as `up`, with the project siding would use (from git when the directory is a worktree, +else the directory's slug), so a fixed name is one that differs from `_`. The +worktree for `HostPath` is the git top level, else the compose file's directory. The parser in +`internal/compose` gains `container_name`, bind mounts, healthchecks, memory limits, replicas +and labels of volumes. Lint errors exit 1 with a `LintReport` (like a Failed verdict), not with +an `Error` document: the command worked, the file has problems. Alternatives: a rule set that +fixes things (would edit the user's compose file); only errors, no warnings (hides why a +verdict may be weaker than expected). + +### D10. Devcontainers: Docker-in-Docker only (maintainer's decision) +This repository's devcontainer stays Docker-in-Docker, and siding inside any devcontainer is +supported with Docker-in-Docker only. Docker-outside-of-Docker gives code in the container the +host's daemon, which is root on the host, the opposite of why people put agents into +devcontainers; it also breaks bind mounts unless the workspace has the same path inside and +out, and needs siding to probe through `host.docker.internal`. Trying siding on real projects +happens on the Mac with `just build-darwin` (both architectures, `CGO_ENABLED=0`). What siding +creates there is product use, not development footprint: `hack/nuke.sh` leaves it alone. +Alternatives, discussed: a second devcontainer configuration with the host's socket, used only +to try siding while tests refuse it; tests against the host daemon with the prefix +`siding-test-`. The roadmap line and the architecture in `openspec/config.yaml` change with it; +the README gets a short "siding in a devcontainer" section with what Docker-in-Docker means +there (one view and one budget per container, one forwarded port per container, own image +cache). + +### D11. Error codes and exit 3 +`NoCapacity` is the first error that does not exit 1: `Execute` maps it to exit 3, with the +usual `Error` document. `NotInteractive` is a normal error (exit 1). + +## Risks / Trade-offs + +- [The VM case of D2 is untested in CI] → a manual test on the Mac in the tasks; the error + when the page cannot be reached names `siding setup` and the address `siding ui` listens on. +- [Two `up`s at the same moment both pass the budget check] → accepted for v0.2; the queue of + v0.3 serialises starts. +- [The 256 MiB default is wrong for big services] → users set `mem_limit`; `lint` could suggest + it later; v0.3 learns peaks. +- [`setup --budget` recreates the proxy] → routing pauses for the seconds Traefik needs to + start; stacks keep running (the spec `proxy` already requires it). +- [A killed `siding ui` leaves its route file] → the proxy answers 502 for `siding.localhost`; + the next `ui` replaces the file. +- [`dev.siding.gitdir` recreates v0.1 containers once] → containers only, volumes stay; noted + in the README's upgrade line. +- [A branch whose upstream was deleted without a merge counts as merged] → its worktree must be + clean, so only committed work is affected, and it is still in the local branch, which gc never + deletes. + +## Migration Plan + +- Run `siding setup` once after upgrading: it recreates the proxy with the file provider and + host alias on its current port. `up`, `status` and the budget work with a v0.1 proxy; + `siding ui` asks for `siding setup`. +- The first `up` of an existing stack recreates its containers once (new label); networks and + volumes are untouched. +- Rollback: siding v0.1 works with a v0.2 proxy (it only reads the port). diff --git a/openspec/changes/phase-2-overview-and-budget/proposal.md b/openspec/changes/phase-2-overview-and-budget/proposal.md index 4710beb..007a600 100644 --- a/openspec/changes/phase-2-overview-and-budget/proposal.md +++ b/openspec/changes/phase-2-overview-and-budget/proposal.md @@ -1,29 +1,59 @@ # Proposal -> Draft. This change holds the roadmap text only and is marked `skip_specs: true`. -> `/opsx:propose` turns it into a full change: it writes the delta specs, `design.md` and -> `tasks.md`, revises this proposal, and removes `skip_specs` from `.openspec.yaml`. - ## Why -With several stacks running, the human needs one view of what runs, what it costs and what can -go, and the machine needs a limit so it stays usable. +v0.1 answers "does my branch run?" for one stack at a time. With several agents in parallel the +human needs the whole picture: what runs, what it costs, what can go after a merge. And the +machine needs a limit, so that the fifth stack does not make the first four unusable. ## What Changes -Roadmap v0.2: - -- Web overview at `siding.localhost`. -- `siding gc`: git-aware cleanup that never deletes unmerged or dirty work. -- `siding lint`: conventions that make a compose file ready for siding (berth-ready). -- Memory budget: `up` refuses when the budget is exhausted, exit 3 NoCapacity. -- Devcontainer: Docker-outside-of-Docker as an alternative to Docker-in-Docker. +- `siding ui`: a read-only overview at `http://siding.localhost`, served by siding in the + foreground while the command runs (no background service). It shows every stack with its + state, address and memory, the budget, and what `gc` would remove. +- Memory budget: `siding setup --budget ` stores the budget as a label on + `siding-proxy` (default: 75 % of Docker's memory). `siding up` refuses with exit 3 + (`NoCapacity`) when the other stacks' memory plus the new stack's estimate exceed it; + `--force` starts it anyway. `siding ps` shows the budget. +- `siding gc [--dry-run] [--yes]`: removes stacks that can go: whose branch is merged and + worktree clean (containers, networks, volumes), and whose worktree is gone (containers and + networks; volumes only when the branch is merged). It never touches git worktrees or + unmerged or dirty work, and asks before removing unless `--yes`. +- `siding lint`: checks a compose file for what breaks parallel stacks (fixed container, + network or volume names, shared host paths, ambiguous ports, missing healthchecks), as a + `LintReport` with error, warning and info findings. +- A new label `dev.siding.gitdir` (the repository's common git directory) on every stack, so + `gc` can tell whether the branch of a deleted worktree is merged. +- Development: `just build-darwin` builds a macOS binary to try siding on the Mac against + Docker Desktop. Docker-outside-of-Docker is dropped from the roadmap: the devcontainer stays + Docker-in-Docker only, and siding inside any devcontainer is supported with + Docker-in-Docker only. ## Capabilities -To be decided at `/opsx:propose`. +### New Capabilities +- `overview`: `siding ui`, the web page at `siding.localhost` and its JSON endpoints. +- `memory-budget`: the budget, how a new stack is estimated, and the refusal with exit 3. +- `stack-gc`: `siding gc` and what may and may not be removed. +- `compose-lint`: `siding lint` and its rules. + +### Modified Capabilities +- `cli-contract`: exit 3 for `NoCapacity` with an `Error` document; new error codes + `NoCapacity` and `NotInteractive`. +- `proxy`: `setup --budget`, and the proxy's route to `siding ui`. +- `stack-identity`: the label `dev.siding.gitdir`. +- `stack-list`: `ps` shows the budget. +- `stack-lifecycle`: `down --volumes` accepts a squash-merged branch (upstream deleted), as `gc` does. +- `agent-skill`: the skill explains exit 3. +- `dev-environment`: `just build-darwin`, siding used on the host is not development + footprint, Docker-in-Docker only. ## Impact -New commands and a small web server; the devcontainer and the spec `dev-environment` -(footprint, safety rules) for Docker-outside-of-Docker. +- New commands `ui`, `gc` and `lint` in `internal/cli`; packages for the budget, gc planning, + lint rules and the web page (Go `net/http`, `html/template`, no JavaScript framework). +- `siding-proxy` gains Traefik's file provider and a host alias; existing proxies are + recreated by the next `siding setup`. +- `cli.Options` gains stdin and a terminal check (for `gc`'s question) and a listener. +- CLAUDE.md and `openspec/config.yaml`: the new label, the budget label, the roadmap without + Docker-outside-of-Docker, devcontainers with Docker-in-Docker only. diff --git a/openspec/changes/phase-2-overview-and-budget/specs/agent-skill/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/agent-skill/spec.md new file mode 100644 index 0000000..4ab5dc1 --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/agent-skill/spec.md @@ -0,0 +1,14 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: What the skill teaches +The skill SHALL tell the agent to run `siding up --wait --json` after a change that affects the +running app, to read the verdict and exit code, to fix the named service from the failure's +reason, hint and log tail, to report the address when Ready, and on exit 3 (`NoCapacity`) to +stop and tell the user. It MUST NOT tell the agent to use `--force`, `--volumes`, `siding gc` +or `siding share`. + +#### Scenario: The commands in the skill +- **WHEN** the printed skill is read +- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1, 2 and 3, and none of `--force`, `--volumes`, `gc` or `share` diff --git a/openspec/changes/phase-2-overview-and-budget/specs/cli-contract/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/cli-contract/spec.md new file mode 100644 index 0000000..5537e1f --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/cli-contract/spec.md @@ -0,0 +1,41 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Error document +A command that fails SHALL exit 1, or 3 for `NoCapacity`. With `--json` it SHALL print a +document of kind `Error` with a stable `code` and a human-readable `message`; without `--json` +it SHALL print `error: ` on stderr, followed by a hint line when there is one. + +#### Scenario: An unknown flag with --json +- **WHEN** `siding version --json --nope` runs +- **THEN** it exits 1 and prints an `Error` document with `code` `InvalidArguments`, even though flag parsing failed before `--json` was read + +#### Scenario: A failure without --json +- **WHEN** `siding up` runs outside a git repository +- **THEN** it exits 1 and prints `error: …` and a hint on stderr, and nothing on stdout + +#### Scenario: The error codes +- **WHEN** a command fails +- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch`, `DirtyWorktree`, `NoCapacity`, `NotInteractive`, or `Internal` for an unexpected error + +#### Scenario: No capacity +- **WHEN** `siding up --json` refuses because the stack does not fit into the memory budget +- **THEN** it prints an `Error` document with `code` `NoCapacity` and exits 3 + +### Requirement: Exit codes +siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), 3 when +`up` refuses for lack of memory (NoCapacity), and 130 without a message when interrupted. + +#### Scenario: A verdict decides the exit code +- **WHEN** `siding status --wait` ends with the verdict Failed +- **THEN** it exits 1, and with Ready it exits 0 + +#### Scenario: Interrupted +- **WHEN** the user presses Ctrl-C during `siding logs -f` +- **THEN** siding stops docker compose and exits 130 without an error message + +#### Scenario: Over budget +- **WHEN** `siding up --wait` refuses because the memory budget is exhausted +- **THEN** it exits 3 without starting a container diff --git a/openspec/changes/phase-2-overview-and-budget/specs/compose-lint/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/compose-lint/spec.md new file mode 100644 index 0000000..c8c8598 --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/compose-lint/spec.md @@ -0,0 +1,59 @@ +# Spec Delta + +## Purpose + +Defines `siding lint`, which tells a user before the first `up` what in their compose file +breaks parallel stacks or weakens the verdict, without changing the file. + +## ADDED Requirements + +### Requirement: Lint reads, never writes +`siding lint [--file]` SHALL check the compose file siding would use, through the configuration +Compose prints, and MUST NOT change any file. It works in any directory with a compose file, +inside a git worktree or not, and needs no running proxy. + +#### Scenario: Outside a repository +- **WHEN** `siding lint` runs in a directory with `compose.yaml` that is not a git worktree +- **THEN** it checks that file + +### Requirement: Findings +Every finding SHALL have a rule, a severity (`error`, `warning` or `info`), the service or +resource it is about, a message and a hint. `lint` SHALL exit 1 when there is an error and 0 +otherwise, and with `--json` print one `LintReport` document with the file and its findings. + +#### Scenario: A clean file +- **WHEN** the compose file has no finding +- **THEN** `siding lint` prints that it found nothing and exits 0 + +#### Scenario: Errors as JSON +- **WHEN** `siding lint --json` finds an error +- **THEN** it prints a `LintReport` with the finding and exits 1 + +### Requirement: Errors +`lint` SHALL report as errors what makes two stacks of the same file collide or `up` refuse: +`ContainerName` (a fixed `container_name`), `FixedNetworkName` (an own network with a fixed +`name`), `FixedVolumeName` (an own volume with a fixed `name`, unless it carries +`dev.siding.shared`), and `InvalidRoute` and `AmbiguousPort` as `up` reports them. + +#### Scenario: A fixed container name +- **WHEN** `db` sets `container_name: shop-db` +- **THEN** lint reports `ContainerName` for `db` with the hint to remove it, since a second stack could not start + +#### Scenario: A volume shared on purpose +- **WHEN** the volume `cache` has `name: pip-cache` and the label `dev.siding.shared: "true"` +- **THEN** lint reports `FixedVolumeName` as info, not as an error + +### Requirement: Warnings and info +`lint` SHALL warn about `HostPath` (a bind mount of an absolute path outside the worktree, which +every stack shares), `NetworkMode` (a service siding cannot route) and `NoHealthcheck` (a +service without a healthcheck and without a route path, of which the verdict only knows that it +runs), and report as info `PublishedPort` (ports siding resets) and `NoRoot` (HTTP services but +no `dev.siding.route: "/"`). + +#### Scenario: A host path +- **WHEN** `db` mounts `/var/lib/shop:/data` +- **THEN** lint warns `HostPath` for `db`, because every branch's stack would write into the same directory + +#### Scenario: A published database port +- **WHEN** `db` publishes `5432:5432` +- **THEN** lint reports `PublishedPort` as info, with the hint that siding resets it and how to reach the database diff --git a/openspec/changes/phase-2-overview-and-budget/specs/dev-environment/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/dev-environment/spec.md new file mode 100644 index 0000000..4714f16 --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/dev-environment/spec.md @@ -0,0 +1,36 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: A Docker daemon of its own +The devcontainer SHALL run its own Docker daemon (Docker-in-Docker), which also hosts every +stack siding's tests create, and MUST NOT mount or reach the host's Docker socket. +Docker-in-Docker is the only devcontainer setup; there is no variant with the host's socket. + +#### Scenario: docker inside the container +- **WHEN** `docker info` runs inside the devcontainer +- **THEN** it reports the container's own host name, not the Docker Desktop VM or the host + +#### Scenario: No Docker-outside-of-Docker +- **WHEN** someone looks for a devcontainer configuration of this repository +- **THEN** there is only `.devcontainer/devcontainer.json`, and no configuration mounts `/var/run/docker.sock` from the host + +## ADDED Requirements + +### Requirement: macOS binaries +`just build-darwin` SHALL build static siding binaries for macOS, `bin/siding-darwin-arm64` and +`bin/siding-darwin-amd64`, in the devcontainer, so siding can be tried on the Mac against +Docker Desktop without installing Go there. + +#### Scenario: Try siding on the Mac +- **WHEN** `just build-darwin` has run in the devcontainer +- **THEN** `bin/siding-darwin-arm64 version` runs on an Apple silicon Mac and reports `darwin/arm64` + +### Requirement: siding used on the host +What a human creates by running siding on the host (`siding-proxy`, the network `siding`, their +stacks, `~/.siding`) SHALL count as use of the product, not as development footprint: +`hack/nuke.sh` MUST NOT remove it, and it is removed with siding's own commands. + +#### Scenario: nuke.sh and a real proxy +- **WHEN** the maintainer ran `siding setup` on the Mac and then runs `hack/nuke.sh` +- **THEN** `siding-proxy` and the network `siding` on the host stay, and `docker compose -p siding down` removes them diff --git a/openspec/changes/phase-2-overview-and-budget/specs/memory-budget/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/memory-budget/spec.md new file mode 100644 index 0000000..99bafb3 --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/memory-budget/spec.md @@ -0,0 +1,57 @@ +# Spec Delta + +## Purpose + +Defines the memory budget that keeps the machine usable with many stacks: where it is kept, how +the memory of a new stack is estimated before it starts, and how `up` refuses when it would not +fit. + +## ADDED Requirements + +### Requirement: Budget on the proxy +The budget SHALL be kept as the label `dev.siding.budget` (bytes) on `siding-proxy`, set with +`siding setup --budget ` (`512m`, `12g`, or bytes) and removed with `--budget default`. +Without the label the budget SHALL be 75 % of the memory the Docker daemon reports. + +#### Scenario: Set a budget +- **WHEN** `siding setup --budget 12g` runs +- **THEN** `siding-proxy` carries `dev.siding.budget=12884901888`, and running stacks keep running + +#### Scenario: Default +- **WHEN** the proxy has no budget label and Docker reports 16 GiB of memory +- **THEN** the budget is 12 GiB + +#### Scenario: setup keeps the budget +- **WHEN** `siding setup` runs again without `--budget` +- **THEN** the budget set before stays + +### Requirement: Estimate of a stack +The estimate of a stack SHALL be the sum over its services of the memory limit +(`deploy.resources.limits.memory`, else `mem_limit`), or 256 MiB for a service without a +limit; a service scaled to several replicas counts once per replica. + +#### Scenario: Limits and defaults +- **WHEN** a stack has `db` with `mem_limit: 1g`, `web` with no limit and `api` with `deploy.resources.limits.memory: 512m` +- **THEN** its estimate is 1 GiB + 256 MiB + 512 MiB + +### Requirement: up refuses over budget +`siding up` SHALL refuse with `NoCapacity` and exit 3, before starting anything, when the +memory in use by every other siding stack on the machine plus the estimate of this stack +exceeds the budget. The message MUST name the budget, the use and the estimate. `--force` +starts it anyway. + +#### Scenario: Over budget +- **WHEN** the budget is 4 GiB, other stacks use 3.5 GiB and the new stack's estimate is 768 MiB +- **THEN** `siding up` exits 3 with `NoCapacity` and the hint names `siding gc`, `siding down` in another worktree, `siding setup --budget` and `--force` + +#### Scenario: Forced +- **WHEN** `siding up --force` runs in the same situation +- **THEN** the stack starts, and a warning on stderr says the budget is exceeded + +#### Scenario: The stack itself does not count twice +- **WHEN** `siding up` runs again for a stack that already runs +- **THEN** only the other stacks' memory and this stack's estimate count, not its current use as well + +#### Scenario: Within budget +- **WHEN** the other stacks and the estimate fit into the budget +- **THEN** `siding up` behaves as without a budget diff --git a/openspec/changes/phase-2-overview-and-budget/specs/overview/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/overview/spec.md new file mode 100644 index 0000000..0f82ac1 --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/overview/spec.md @@ -0,0 +1,52 @@ +# Spec Delta + +## Purpose + +Defines `siding ui`, the one page where a human sees every stack on the machine, what it costs +against the budget, and what could go after a merge, served only while the command runs. + +## ADDED Requirements + +### Requirement: Overview in the foreground +`siding ui` SHALL serve a read-only overview until it is stopped, at `http://siding.localhost` +through the proxy and at the local address it prints. When it stops, the route MUST go away and +nothing of it MUST keep running. It SHALL NOT change any stack. + +#### Scenario: Start and stop +- **WHEN** `siding ui` runs and the user presses Ctrl-C +- **THEN** while it ran `http://siding.localhost` showed the overview, and afterwards the proxy answers 404 for `siding.localhost` and no process of siding is left + +#### Scenario: No proxy +- **WHEN** `siding ui` runs while `siding-proxy` is not running or lacks the overview route +- **THEN** it fails with `ProxyNotRunning` and the hint `siding setup` + +### Requirement: Only this machine +The overview SHALL listen only on addresses of this machine that the proxy can reach (the +loopback interface, or the proxy network's gateway when the Docker daemon runs on this +machine), on `--port` or a free port, and MUST NOT listen on all interfaces. + +#### Scenario: Not on the network +- **WHEN** `siding ui` runs on a Linux machine with a LAN address +- **THEN** a request to that LAN address and the overview's port is refused + +### Requirement: What the page shows +The page SHALL show every stack as `siding ps` lists it (repository, branch, state, address as +a link, memory, worktree), the memory in use against the budget, and the stacks `siding gc` +would remove, with the reason. It SHALL refresh itself every 5 seconds. + +#### Scenario: Two stacks and a merged branch +- **WHEN** stacks run for `main` and for a merged, clean `feature/login` +- **THEN** the page lists both with links to their addresses, the total against the budget, and `feature/login` under "can go" with the reason `Merged` + +### Requirement: JSON endpoints +The overview SHALL answer `GET /api/stacks` with the `StackList` document of `siding ps --json` +and `GET /api/gc` with the `GC` document of `siding gc --dry-run --json`, so the page and +scripts read the same data. Every other method MUST be refused with 405. + +#### Scenario: Same document as ps +- **WHEN** `GET http://siding.localhost/api/stacks` is requested +- **THEN** the body is a `StackList` document equal to what `siding ps --json` prints at that moment + +#### Scenario: No actions +- **WHEN** `POST http://siding.localhost/api/gc` is requested +- **THEN** the overview answers 405 and removes nothing diff --git a/openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md new file mode 100644 index 0000000..b5be84c --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md @@ -0,0 +1,50 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Setup starts the proxy +`siding setup` SHALL create the network `siding` if it is missing and run the Compose project +`siding` with one container `siding-proxy`: Traefik with the Docker provider on the network +`siding`, restarting unless stopped. It SHALL print the proxy's address and budget, with +`--json` as a `Setup` document. + +#### Scenario: First setup +- **WHEN** `siding setup` runs on a machine without the proxy +- **THEN** the network `siding` and the running container `siding-proxy` exist, and it prints `http://*.localhost` as the address + +#### Scenario: JSON +- **WHEN** `siding setup --json` runs +- **THEN** it prints a `Setup` document with the proxy's container name, network, image, port and budget in bytes + +### Requirement: Setup is idempotent +`siding setup` SHALL change nothing when the proxy already runs as requested, and SHALL +recreate only the proxy container when its configuration differs (port, image, budget, or the +proxy configuration of a newer siding). Running stacks MUST keep running and be routed again +once the proxy is back. + +#### Scenario: Second run +- **WHEN** `siding setup` runs twice +- **THEN** the second run reports the proxy as already running and recreates nothing + +#### Scenario: Port change with running stacks +- **WHEN** a stack runs and `siding setup --port 8080` follows `siding setup` +- **THEN** the stack's containers are not restarted and it answers on port 8080 + +#### Scenario: A proxy from siding v0.1 +- **WHEN** `siding-proxy` was created by siding v0.1 and `siding setup` of v0.2 runs +- **THEN** the proxy is recreated with the route to the overview, on the port it had + +## ADDED Requirements + +### Requirement: Route to the overview +The proxy SHALL route `siding.localhost` to `siding ui` while it runs, through a route file +that `siding ui` writes into `~/.siding/proxy//routes/` and removes when it stops. The +proxy SHALL reach the machine siding runs on as `host.docker.internal`. + +#### Scenario: A route file +- **WHEN** `siding ui` has written its route file +- **THEN** the proxy sends `Host: siding.localhost` to `siding ui` within a few seconds, without a restart + +#### Scenario: A route left behind +- **WHEN** `siding ui` was killed and its route file is still there +- **THEN** the proxy answers 502 for `siding.localhost`, and the next `siding ui` replaces the file diff --git a/openspec/changes/phase-2-overview-and-budget/specs/stack-gc/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/stack-gc/spec.md new file mode 100644 index 0000000..bcc12c5 --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/stack-gc/spec.md @@ -0,0 +1,73 @@ +# Spec Delta + +## Purpose + +Defines `siding gc`, which removes the stacks that can go after a merge or a deleted worktree, +and never anything that could hold unmerged or uncommitted work. + +## ADDED Requirements + +### Requirement: Merged +A branch SHALL count as merged for `gc` when it is not the default branch, its worktree has no +changes, and either its tip is contained in the default branch while the default branch has +moved past it, or its upstream branch is gone (deleted after a merge, a squash merge included). + +#### Scenario: Merged and the default branch moved on +- **WHEN** `feature/login` was merged into `main`, `main` has a commit after it, and the worktree is clean +- **THEN** `feature/login` counts as merged + +#### Scenario: A fresh branch +- **WHEN** an agent just created `feature/new` from `main`, without a commit of its own +- **THEN** it does not count as merged, and gc leaves its stack alone + +#### Scenario: Squash merge +- **WHEN** the PR of `feature/x` was squash-merged and the remote branch deleted, and `git fetch --prune` ran +- **THEN** `feature/x` counts as merged, although its commits are not in `main` + +#### Scenario: The default branch +- **WHEN** a stack runs for `main` +- **THEN** gc never removes it + +### Requirement: What gc removes +`siding gc` SHALL consider the stacks of this machine and remove: for a merged branch, the +stack's containers, networks and volumes (reason `Merged`); for a stack whose worktree no +longer exists, its containers and networks, and its volumes only if the branch is merged +(reason `WorktreeGone`). It MUST NOT touch git worktrees or branches. + +#### Scenario: A deleted worktree with an unmerged branch +- **WHEN** the worktree of `feature/wip` was deleted, but the branch has commits `main` lacks +- **THEN** gc removes its containers and networks, keeps its volumes, and reports them as kept + +#### Scenario: A stack from an older siding +- **WHEN** a stack without the label `dev.siding.gitdir` has lost its worktree +- **THEN** gc removes its containers and networks and keeps its volumes, since it cannot tell whether the branch is merged + +#### Scenario: Work in progress +- **WHEN** a worktree has uncommitted changes or its branch is not merged +- **THEN** gc removes nothing of its stack and lists it as kept with the reason `Dirty` or `Unmerged` + +#### Scenario: Worktrees stay +- **WHEN** gc removes the stack of a merged branch +- **THEN** the worktree directory and the branch still exist + +### Requirement: Ask before removing +`siding gc` SHALL list what it would remove and ask before removing it. `--yes` removes without +asking, and `--dry-run` only lists. Without a terminal, or with `--json`, it MUST fail with +`NotInteractive` unless `--yes` or `--dry-run` is given. + +#### Scenario: Interactive +- **WHEN** `siding gc` runs in a terminal and the user answers anything but `y` +- **THEN** nothing is removed + +#### Scenario: An agent runs gc +- **WHEN** `siding gc --json` runs without `--yes` or `--dry-run` +- **THEN** it fails with `NotInteractive` and removes nothing + +### Requirement: GC document +With `--json`, gc SHALL print one `GC` document: `dryRun`, the stacks it removes (or would) +with project, repository, branch, worktree, reason, and the containers, networks and volumes +removed and kept, and the stacks it keeps with their reason. + +#### Scenario: Dry run as JSON +- **WHEN** `siding gc --dry-run --json` runs with one merged stack and one dirty one +- **THEN** it prints a `GC` document with `dryRun: true`, the merged stack under `stacks` with reason `Merged`, and the dirty one under `kept` with reason `Dirty` diff --git a/openspec/changes/phase-2-overview-and-budget/specs/stack-identity/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/stack-identity/spec.md new file mode 100644 index 0000000..2cd8f35 --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/stack-identity/spec.md @@ -0,0 +1,22 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Labels +siding SHALL label every container, network and volume of a stack with `dev.siding.repo`, +`dev.siding.branch` (the branch name, not the slug), `dev.siding.worktree` (absolute path) and +`dev.siding.machine`, and its containers with `dev.siding.gitdir` (the repository's common git +directory) and with `dev.siding.agent` when `--agent` or `SIDING_AGENT` names one. External +networks and volumes MUST NOT be relabelled. + +#### Scenario: Labels on a running stack +- **WHEN** `siding up` has run in `/src/shop-login` on branch `feature/login` +- **THEN** every container, the project's networks and its named volumes carry `dev.siding.repo=shop`, `dev.siding.branch=feature/login` and `dev.siding.worktree=/src/shop-login` + +#### Scenario: An agent name +- **WHEN** `SIDING_AGENT=claude siding up` runs +- **THEN** the containers also carry `dev.siding.agent=claude` + +#### Scenario: The git directory +- **WHEN** `siding up` has run in the linked worktree `/src/shop-login` of the repository `/src/shop` +- **THEN** its containers carry `dev.siding.gitdir=/src/shop/.git`, and its networks and volumes do not, so a stack of siding v0.1 keeps its networks on its next `up` diff --git a/openspec/changes/phase-2-overview-and-budget/specs/stack-lifecycle/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/stack-lifecycle/spec.md new file mode 100644 index 0000000..89d9a7a --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/stack-lifecycle/spec.md @@ -0,0 +1,29 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Removing data keeps work +`siding down --volumes` SHALL also remove the stack's volumes, but MUST refuse with +`UnmergedBranch` when the branch is neither merged into the default branch nor has a deleted +upstream branch, and with `DirtyWorktree` when `git status` lists changes, unless `--force` is +given. + +#### Scenario: Unmerged branch +- **WHEN** `siding down --volumes` runs on a branch with commits that the default branch lacks +- **THEN** it fails with `UnmergedBranch`, removes nothing, and the hint names `--force` + +#### Scenario: Dirty worktree +- **WHEN** the worktree has uncommitted changes and `siding down --volumes` runs +- **THEN** it fails with `DirtyWorktree` and removes nothing + +#### Scenario: Forced +- **WHEN** `siding down --volumes --force` runs on an unmerged branch +- **THEN** the containers, networks and volumes of the stack are removed + +#### Scenario: No default branch +- **WHEN** siding finds neither `origin/HEAD` nor a local `main` or `master` +- **THEN** the branch counts as unmerged + +#### Scenario: Squash-merged +- **WHEN** the branch's PR was squash-merged, its remote branch deleted and `git fetch --prune` ran +- **THEN** `siding down --volumes` removes the volumes without `--force` diff --git a/openspec/changes/phase-2-overview-and-budget/specs/stack-list/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/stack-list/spec.md new file mode 100644 index 0000000..7aa17f6 --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/specs/stack-list/spec.md @@ -0,0 +1,24 @@ +# Spec Delta + +## MODIFIED Requirements + +### Requirement: Memory in use +`ps` SHALL show each stack's memory in use, the sum over its running containers as the Docker +API reports it, and the total over all stacks against the memory budget. + +#### Scenario: Two stacks +- **WHEN** one stack uses 300 MiB and another 500 MiB +- **THEN** `siding ps` shows 300 MiB and 500 MiB for them and 800 MiB in total + +#### Scenario: The budget +- **WHEN** the budget is 12 GiB and the stacks use 800 MiB +- **THEN** the total line of `siding ps` reads `800 MiB of 12.0 GiB` + +### Requirement: StackList document +With `--json`, `ps` SHALL print one `StackList` document with one entry per stack: project, +repository, branch, worktree, machine, agent, state, address, memory in bytes, and +`worktreeMissing` when the worktree is gone; and the total memory and the budget in bytes. + +#### Scenario: No stacks +- **WHEN** `siding ps --json` runs and no stack exists +- **THEN** it prints a `StackList` document without stacks, with the budget, and exits 0 diff --git a/openspec/changes/phase-2-overview-and-budget/tasks.md b/openspec/changes/phase-2-overview-and-budget/tasks.md new file mode 100644 index 0000000..bd0c9ff --- /dev/null +++ b/openspec/changes/phase-2-overview-and-budget/tasks.md @@ -0,0 +1,64 @@ +# Tasks + +## 1. Foundations + +- [ ] 1.1 Add `Stdin` and `IsTerminal` to `cli.Options` (wired in `cmd/siding/main.go`, fakes in the test env); verify `just test` passes and no package-level variable is a seam +- [ ] 1.2 Add the error codes `NoCapacity` (exit 3 in `Execute`) and `NotInteractive` (D11); verify with the error-code table test and a test that `NoCapacity` exits 3 with an `Error` document +- [ ] 1.3 Extend `internal/compose` with `container_name`, bind mounts, healthchecks, memory limits (`mem_limit`, `deploy.resources.limits.memory`), replicas (`deploy.replicas`, `scale`), and names and labels of top-level networks and volumes; verify with a second captured config in `internal/compose/testdata/` and table assertions + +## 2. Merged and the git directory + +- [ ] 2.1 Extend `internal/git` with the shared merged check (ancestor of the default branch, or upstream `[gone]`), runnable with `--git-dir` for a missing worktree, and the gc conditions (not the default branch, default branch moved past the tip) (D7); verify with table tests against temporary repositories including a clone whose upstream branch was deleted and pruned, a fresh branch without own commits, and a deleted worktree +- [ ] 2.2 Let `down --volumes` accept a branch whose upstream is gone; verify with a fake-based test and the end-to-end guard case +- [ ] 2.3 Put `dev.siding.gitdir` on containers only (D8); verify with the override golden (`just golden`) and a test that networks and volumes do not carry it +- [ ] 2.4 Add `dev.siding.gitdir` and `dev.siding.budget` to the label conventions in CLAUDE.md and `openspec/config.yaml`; verify `openspec validate --all --strict` passes + +## 3. Memory budget + +- [ ] 3.1 Implement size parsing (`512m`, `12g`, bytes, `default`) and the estimate of a project (D5); verify with table tests including replicas and both limit forms +- [ ] 3.2 Rework `setup` to always run Compose and compare container IDs (D6), and add `--budget` with the label carried forward; verify with fake-based tests for created, updated, unchanged, a budget change, `--budget default`, a v0.1 proxy, and the updated `setup.json` golden +- [ ] 3.3 Make `up` refuse with `NoCapacity` (exit 3) when use of other stacks plus estimate exceeds the budget, and start anyway with `--force` and a warning; verify with fake-based tests for over, within, forced, and the stack not counting twice +- [ ] 3.4 Show the budget in `ps` (total line, `budgetBytes` in `StackList`); verify with the updated text test and `stacklist.json` golden +- [ ] 3.5 Document the budget in the README (`setup --budget`, the estimate, `mem_limit`, exit 3, `--force`); verify the commands run as written in the devcontainer + +## 4. gc + +- [ ] 4.1 Implement the pure gc plan (D8): `Merged`, `WorktreeGone` with and without a merged branch, stacks without `dev.siding.gitdir`, kept stacks with `Dirty` or `Unmerged`, other machines ignored, the default branch never; verify with a table test per scenario of the spec `stack-gc` +- [ ] 4.2 Implement `siding gc [--dry-run] [--yes] [--json]` with the question on a terminal and `NotInteractive` otherwise, removing by project name; verify with fake-based tests (answer y, answer n, no terminal, `--json` without `--yes`) and `gc.json` and `gc-dryrun.json` goldens +- [ ] 4.3 Document `siding gc` in the README; verify the shown commands run as written + +## 5. lint + +- [ ] 5.1 Implement the lint rules and severities of the spec `compose-lint` (D9); verify with a captured config that triggers every rule and one that triggers none +- [ ] 5.2 Implement `siding lint [--file] [--json]` with exit 1 on errors and the `LintReport` document; verify with fake-based tests, a text test and a `lintreport.json` golden +- [ ] 5.3 Document `siding lint` in the README with the rules in one table; verify the table lists every rule the code has + +## 6. Overview + +- [ ] 6.1 Extend the proxy's compose file with the file provider on `~/.siding/proxy//routes` and the host alias decided by `setup` (D2: the network's gateway when it is a local address, else `host-gateway`); verify with fake-based tests for both cases and the proxy compose file content +- [ ] 6.2 Implement the route file of `siding ui` (atomic write, removal on exit, replacing a stale one) and the listen addresses of D2; verify with unit tests in a temporary home +- [ ] 6.3 Implement the server: the page (D3) with stacks, budget and gc candidates, `GET /api/stacks` and `GET /api/gc` returning the same documents as `ps --json` and `gc --dry-run --json`, 405 for other methods; verify with `httptest` tests and a golden of the rendered page with fixed data +- [ ] 6.4 Implement `siding ui [--port]` (ProxyNotRunning without the route support, prints both addresses, Ctrl-C ends with 130 after removing the route); verify with a fake-based test that the route file exists while it runs and is gone after cancel +- [ ] 6.5 Document `siding ui` in the README; verify the shown command runs as written in the devcontainer + +## 7. Agent skill + +- [ ] 7.1 Teach exit 3 in the skill (stop and tell the user) and keep `--force`, `--volumes`, `gc` and `share` out of it; verify with the updated skill test and `just golden` + +## 8. Devcontainer and docs + +- [ ] 8.1 Add `just build-darwin` (arm64 and amd64, `CGO_ENABLED=0`); verify both binaries are Mach-O for their architecture (`file bin/siding-darwin-*`) +- [ ] 8.2 Make Docker-in-Docker the only devcontainer setup in CLAUDE.md, the README ("siding in a devcontainer" with its limits) and `openspec/config.yaml` (architecture and roadmap without Docker-outside-of-Docker, also in the draft `phase-5-remote-host` if it mentions it); verify `openspec validate --all --strict` passes +- [ ] 8.3 Document in CLAUDE.md's safety rules that siding used on the host is product use and that `hack/nuke.sh` leaves it alone; verify `hack/nuke.sh` contains no name of a product object (`siding-proxy`, `siding`) + +## 9. End-to-end and integration + +- [ ] 9.1 Extend `internal/e2e`: a budget too small refuses with exit 3 and `--force` starts; `gc --dry-run --json` lists a merged stack and keeps a dirty one, `gc --yes` removes the merged one with its volumes; `siding ui` answers through the test proxy at `siding.localhost` and the route is gone after cancel; verify with `just smoke-stacks` and that no `siding-test-*` object is left +- [ ] 9.2 Run the flow by hand in the devcontainer: `setup --port 8080 --budget 2g`, two stacks, `ps`, `siding ui` in the host browser at `http://siding.localhost:8080`, `lint`, `gc`; record the result in the phase summary +- [ ] 9.3 Run the flow on the Mac with `bin/siding-darwin-arm64` against Docker Desktop, including `siding ui` (the VM case of D2) and port 80; record the result and fix what differs +- [ ] 9.4 Update the status in CLAUDE.md, the README and `openspec/config.yaml` to v0.2; verify `just test` passes + +## Workflow follow-up + +- Stop and show the result; wait for the maintainer's approval of Phase 2. +- `/opsx:archive` merges the deltas into `openspec/specs/`. From aac48799324df315a2f8b1184735127d6f19bf6b Mon Sep 17 00:00:00 2001 From: tweinmann Date: Wed, 28 Oct 2026 22:06:51 +0000 Subject: [PATCH 2/3] Implement phase 2: overview and budget (v0.2) siding ui serves a read-only overview at siding.localhost while it runs; the memory budget (setup --budget, a label on siding-proxy) makes up refuse with exit 3; gc removes the stacks of merged branches and deleted worktrees and never touches worktrees or unmerged work; lint checks a compose file for what breaks parallel stacks; teardown removes the proxy and leaves stacks running; just build-darwin builds siding for the Mac. down --volumes accepts squash-merged branches. Containers carry dev.siding.gitdir. Devcontainers are Docker-in-Docker only. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 26 +- README.md | 81 ++++++- cmd/siding/main.go | 34 ++- internal/budget/budget.go | 74 ++++++ internal/budget/budget_test.go | 53 +++++ internal/cli/cli.go | 20 +- internal/cli/cli_test.go | 10 +- internal/cli/down.go | 11 +- internal/cli/down_test.go | 13 + internal/cli/env_test.go | 129 ++++++++-- internal/cli/gc.go | 223 ++++++++++++++++++ internal/cli/gc_test.go | 125 ++++++++++ internal/cli/json.go | 85 ++++++- internal/cli/lint.go | 79 +++++++ internal/cli/lint_test.go | 71 ++++++ internal/cli/ps.go | 12 +- internal/cli/ps_test.go | 8 +- internal/cli/setup.go | 126 +++++++--- internal/cli/setup_test.go | 98 +++++++- internal/cli/skill/SKILL.md | 3 + internal/cli/skill_test.go | 4 +- internal/cli/teardown.go | 130 ++++++++++ internal/cli/teardown_test.go | 80 +++++++ internal/cli/testdata/json/gc-dryrun.json | 55 +++++ internal/cli/testdata/json/gc.json | 54 +++++ internal/cli/testdata/json/lintreport.json | 90 +++++++ internal/cli/testdata/json/setup.json | 3 +- internal/cli/testdata/json/skill.json | 2 +- .../cli/testdata/json/stacklist-empty.json | 3 +- internal/cli/testdata/json/stacklist.json | 3 +- .../cli/testdata/json/teardown-dryrun.json | 7 + internal/cli/testdata/json/teardown.json | 6 + internal/cli/testdata/skill/SKILL.md | 3 + internal/cli/testdata/ui/page.html | 71 ++++++ internal/cli/ui.go | 185 +++++++++++++++ internal/cli/ui/page.html | 64 +++++ internal/cli/ui_test.go | 149 ++++++++++++ internal/cli/up.go | 50 +++- internal/cli/up_test.go | 61 +++++ internal/compose/compose.go | 35 ++- internal/compose/config.go | 88 ++++++- internal/compose/config_test.go | 65 +++++ internal/compose/testdata/lint/compose.yaml | 35 +++ internal/compose/testdata/lint/config.json | 119 ++++++++++ internal/docker/docker.go | 63 +++++ internal/e2e/e2e_test.go | 141 ++++++++++- internal/fail/fail.go | 4 + internal/gc/gc.go | 82 +++++++ internal/gc/gc_test.go | 105 +++++++++ internal/git/git.go | 106 +++++++-- internal/git/git_test.go | 102 +++++--- internal/lint/lint.go | 158 +++++++++++++ internal/lint/lint_test.go | 82 +++++++ internal/overview/route.go | 95 ++++++++ internal/overview/route_test.go | 57 +++++ internal/proxy/proxy.go | 37 ++- internal/stack/identity.go | 20 +- internal/stack/identity_test.go | 7 + internal/stack/routing_test.go | 2 +- internal/stack/testdata/override/routing.yml | 7 + justfile | 7 + .../phase-2-overview-and-budget/design.md | 15 ++ .../phase-2-overview-and-budget/proposal.md | 5 +- .../specs/proxy/spec.md | 23 ++ .../phase-2-overview-and-budget/tasks.md | 67 +++--- openspec/config.yaml | 26 +- 66 files changed, 3625 insertions(+), 229 deletions(-) create mode 100644 internal/budget/budget.go create mode 100644 internal/budget/budget_test.go create mode 100644 internal/cli/gc.go create mode 100644 internal/cli/gc_test.go create mode 100644 internal/cli/lint.go create mode 100644 internal/cli/lint_test.go create mode 100644 internal/cli/teardown.go create mode 100644 internal/cli/teardown_test.go create mode 100644 internal/cli/testdata/json/gc-dryrun.json create mode 100644 internal/cli/testdata/json/gc.json create mode 100644 internal/cli/testdata/json/lintreport.json create mode 100644 internal/cli/testdata/json/teardown-dryrun.json create mode 100644 internal/cli/testdata/json/teardown.json create mode 100644 internal/cli/testdata/ui/page.html create mode 100644 internal/cli/ui.go create mode 100644 internal/cli/ui/page.html create mode 100644 internal/cli/ui_test.go create mode 100644 internal/compose/testdata/lint/compose.yaml create mode 100644 internal/compose/testdata/lint/config.json create mode 100644 internal/gc/gc.go create mode 100644 internal/gc/gc_test.go create mode 100644 internal/lint/lint.go create mode 100644 internal/lint/lint_test.go create mode 100644 internal/overview/route.go create mode 100644 internal/overview/route_test.go diff --git a/CLAUDE.md b/CLAUDE.md index 557edd3..6f8e3e6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,8 +17,11 @@ changing architecture. Phase 0 approved (2026-10-28). Phase 1 (v0.1) approved and archived (2026-10-28, `openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing`): `setup`, `up [--wait]`, `status [--wait]`, `ps`, `logs`, `open`, `down [--volumes]`, `skill`, routing by host and path -through `siding-proxy`, `just smoke-stacks`. Next: Phase 2 (`phase-2-overview-and-budget`, -v0.2), starting with `/opsx:propose`. +through `siding-proxy`, `just smoke-stacks`. Phase 2 (`phase-2-overview-and-budget`, v0.2) +implemented, waiting for approval (2026-10-28), not yet archived: `siding ui` at +`siding.localhost`, the memory budget (`setup --budget`, `up` exits 3), `gc`, `lint`, +`teardown`, `just build-darwin`; tested by hand in the devcontainer and on the Mac with Docker Desktop. +After approval: `/opsx:archive`, then Phase 3 (`phase-3-snapshots-and-queue`, v0.3). ## Working agreements @@ -63,6 +66,9 @@ v0.2), starting with `/opsx:propose`. - Removable. Uninstalling siding leaves plain Compose projects that keep running. - Not an agent manager, not a sandbox for the agent, not Kubernetes, not production, not a hot-reload tool (Compose `develop.watch` is passed through). +- In a devcontainer, Docker-in-Docker only: no Docker-outside-of-Docker, neither for this + repository's devcontainer nor as a supported setup for users, because the host's Docker socket + is root on the host. ## Dev environment @@ -83,6 +89,7 @@ requests and on `main`, all inside the devcontainer image. | `just test` | devcontainer | level-1 checks, as in CI: gofmt, vet, `go test -race`, golangci-lint, `openspec validate --all --strict` | | `just golden` | devcontainer | rewrite golden files after an intended change | | `just build` | devcontainer | build a static `bin/siding` (linux) | +| `just build-darwin` | devcontainer | build `bin/siding-darwin-arm64` and `-amd64`, to try siding on the Mac against Docker Desktop (product use, not development) | | `just smoke-docker` | devcontainer | inner daemon: compose project with two services and healthchecks, `ports: !reset []`, Traefik on `siding` routing `demo.main.localhost` at `/` and `/api` | | `just smoke-stacks` | devcontainer | inner daemon: siding's own commands end to end (`internal/e2e`, build tag `docker`): setup, `up --wait`, routing, a Failed verdict with log tail, `ps`, `down`, the guard of `down --volumes`; also checks golden overrides with `docker compose config` | | `bin/siding setup --port 8080` | devcontainer | the real `siding-proxy` on the forwarded port, to try siding by hand | @@ -101,6 +108,10 @@ requests and on `main`, all inside the devcontainer image. (devcontainer, its images and volumes). Delete them by exact name — never by prefix, never with `prune`. New host objects go into that spec, `hack/nuke.sh` and (for volumes) `devcontainer.json` together, in one change. +- Running siding on the host (e.g. `bin/siding-darwin-arm64` from `just build-darwin`) is use + of the product, not development: what it creates there (`siding-proxy`, the network + `siding`, the user's stacks, `~/.siding`) is not development footprint. `hack/nuke.sh` never + removes it; siding's own commands and `docker compose -p siding down` do. - Tests create stacks only as Compose projects with the prefix `siding-test-` and remove them by their exact name. The smoke tests' proxies listen on 18080, never on 8080; the end-to-end tests run their own proxy `siding-test-e2e-proxy` on the network `siding-test-e2e`, so a @@ -118,10 +129,13 @@ requests and on `main`, all inside the devcontainer image. - Module `github.com/tweinmann/siding`, binary `siding`, API version `siding.dev/v1alpha1`. - Labels on every container, network and volume siding creates or adopts: `dev.siding.repo`, `dev.siding.branch`, `dev.siding.worktree` (absolute path), `dev.siding.machine` (stable - machine id); on containers also `dev.siding.agent` (optional; never on networks or volumes, - whose labels must not change between two `up`s). Service labels: `dev.siding.route` - (`/`, `/`, `false`), `dev.siding.port` (the HTTP port of a service with several), - `dev.siding.seed` (`volume|script|none`). Volume label `dev.siding.shared`. + machine id); on containers also `dev.siding.gitdir` (the repository's common git directory) + and `dev.siding.agent` (optional), never on networks or volumes, whose labels must not change + between two `up`s. Service labels: `dev.siding.route` (`/`, `/`, `false`), + `dev.siding.port` (the HTTP port of a service with several), `dev.siding.seed` + (`volume|script|none`). Volume label `dev.siding.shared`. On `siding-proxy`: + `dev.siding.budget` (the memory budget in bytes) and `dev.siding.host-alias` (what + `host.docker.internal` resolves to in the proxy). - Compose project `-`, host `..localhost`, proxy `siding-proxy` on the network `siding`, override `.siding/override.yml`. - Docker state is read through the Docker Engine API (Go SDK); lifecycle operations invoke diff --git a/README.md b/README.md index e2280f0..03eeb49 100644 --- a/README.md +++ b/README.md @@ -3,8 +3,9 @@ siding gives every git worktree its own Docker Compose stack, at its own `*.localhost` address, and tells you and your coding agents whether it actually runs and what it costs. -> **Pre-alpha (v0.1).** Stacks per worktree, routing through one local proxy and the verdict -> of `siding status` work; expect rough edges and changes before v1. There are no releases +> **Pre-alpha (v0.2).** Stacks per worktree, routing through one local proxy, the verdict of +> `siding status`, the overview `siding ui`, a memory budget, `gc` and `lint` work; expect +> rough edges and changes before v1. There are no releases > yet: build it from source (see below). The plan is in [openspec/changes/](openspec/changes) > (one change per phase), what siding does today in [openspec/specs/](openspec/specs). @@ -17,6 +18,7 @@ siding needs Docker with Compose 2.24 or later, and git. Your compose file stays ```sh siding setup # Traefik as siding-proxy on 127.0.0.1:80 siding setup --port 8080 # if port 80 is taken, or inside siding's devcontainer +siding teardown # remove the proxy again; stacks keep running, --dry-run lists it ``` Every stack is reached through this one proxy at `http://..localhost`. Browsers @@ -57,6 +59,51 @@ answers through the proxy. A failure names the service, the reason, a hint and i lines. Exit codes: 0 Ready, 1 Failed, 2 TimedOut or still starting. Every command prints one JSON document with `--json`, for scripts and agents. +### Before the first up: lint + +```sh +siding lint # exits 1 when it finds an error; --json for a LintReport +``` + +`lint` checks the compose file siding would use and never changes it. + +| Rule | Severity | Finds | +|---|---|---| +| `ContainerName` | error | a fixed `container_name`: a second stack cannot start | +| `FixedNetworkName` | error | an own network with a fixed `name`, shared by every stack | +| `FixedVolumeName` | error (info with `dev.siding.shared`) | an own volume with a fixed `name`, so every stack shares its data | +| `InvalidRoute`, `AmbiguousPort` | error | route labels `siding up` would refuse | +| `HostPath` | warning | a bind mount of a path outside the worktree, shared by every stack | +| `NetworkMode` | warning | `network_mode`: siding cannot route the service | +| `NoHealthcheck` | warning | no healthcheck and no route: Ready only means it started | +| `PublishedPort` | info | published ports, which siding resets | +| `NoRoot` | info | HTTP services, but none at `/` | + +### Memory budget + +```sh +siding setup --budget 12g # memory for all stacks together; "default" is 75 % of Docker's +siding up --force # start even if the stack does not fit +``` + +Before `up` starts a stack, siding adds the memory the other stacks use now to what this +stack will need: per service its `mem_limit` (or `deploy.resources.limits.memory`), or +256 MiB without a limit. If that exceeds the budget, `up` refuses with exit code 3 and says +how much is missing. Setting `mem_limit` on big services makes the estimate right. `siding ps` +shows the total against the budget. + +### Overview + +```sh +siding ui # http://siding.localhost while it runs; Ctrl-C stops it +``` + +The page lists every stack with its address and memory, the total against the budget, and what +`siding gc` would remove. It refreshes itself and is read-only; `/api/stacks` and `/api/gc` +return the documents of `siding ps --json` and `siding gc --dry-run --json`. It listens on this +machine only, and nothing of it keeps running after Ctrl-C. After upgrading siding, run +`siding setup` once, so the proxy knows the route. + ### Logs, address, and taking it down ```sh @@ -67,6 +114,35 @@ siding down --volumes # also the data; refused while the branch is unmerg # worktree dirty, unless --force; --dry-run lists it first ``` +### After a merge: gc + +```sh +siding gc --dry-run # what can go, and what stays and why +siding gc # asks, then removes; --yes for scripts +``` + +`gc` removes the stacks of merged branches whose worktree is clean, with their volumes, and +the containers of stacks whose worktree you deleted (their volumes only if the branch is +merged). A branch counts as merged when the default branch has moved past it, or when its +upstream branch is gone, as after a squash-merged pull request and `git fetch --prune`. `gc` +never deletes worktrees or branches, and never the stack of the default branch or of a branch +without commits of its own. + +### In a devcontainer + +siding works inside a devcontainer that runs its own Docker daemon (Docker-in-Docker), as this +repository's does. Then: + +- `siding ps`, `siding ui` and the budget see the stacks of that container, not the whole + machine; for one view of everything, run siding on the machine itself; +- forward the proxy's port (`forwardPorts` with `requireLocalPort`) and run + `siding setup --port` with it; each devcontainer needs its own port; +- the container has its own image cache. + +Mounting the host's Docker socket into the devcontainer (Docker-outside-of-Docker) is not +supported: it gives everything in the container root on the host, the opposite of why agents +run in devcontainers, and bind mounts break unless the paths match inside and out. + ### For coding agents `siding skill` prints a Claude Code skill that tells an agent to run `siding up --wait --json` @@ -90,6 +166,7 @@ on your machine and siding's tests never touch your Docker containers. | `just test` | formatting, vet, unit tests with `-race`, golangci-lint, OpenSpec validation (as in CI) | | `just golden` | rewrite golden files after an intended output change | | `just build` | build `bin/siding` | + | `just build-darwin` | build `bin/siding-darwin-arm64` and `-amd64`, to run siding on your Mac | | `just smoke-docker` | Compose, `ports: !reset []` and Traefik routing against the container's own daemon | | `just smoke-stacks` | siding's own commands end to end against the container's own daemon | diff --git a/cmd/siding/main.go b/cmd/siding/main.go index 778f220..a277ccb 100644 --- a/cmd/siding/main.go +++ b/cmd/siding/main.go @@ -13,6 +13,7 @@ import ( "github.com/tweinmann/siding/internal/compose" "github.com/tweinmann/siding/internal/docker" "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/overview" "github.com/tweinmann/siding/internal/probe" "github.com/tweinmann/siding/internal/proxy" "github.com/tweinmann/siding/internal/stack" @@ -34,20 +35,29 @@ func main() { } ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) opts := cli.Options{ - Build: cli.CurrentBuild(version), - Dir: dir, - Home: home, - Getenv: os.Getenv, - OSMachineID: stack.OSMachineID, - Git: git.Client{}, - Compose: &compose.CLI{}, - Docker: &docker.Client{}, - Probe: probe.HTTP{Header: stack.ServiceHeader}, - Browser: cli.SystemBrowser{Getenv: os.Getenv}, - Clock: cli.SystemClock{}, - Proxy: proxy.Default(), + Build: cli.CurrentBuild(version), + Dir: dir, + Home: home, + Getenv: os.Getenv, + OSMachineID: stack.OSMachineID, + Stdin: os.Stdin, + IsLocalAddress: overview.IsLocalAddress, + IsTerminal: stdinIsTerminal, + Git: git.Client{}, + Compose: &compose.CLI{}, + Docker: &docker.Client{}, + Probe: probe.HTTP{Header: stack.ServiceHeader}, + Browser: cli.SystemBrowser{Getenv: os.Getenv}, + Clock: cli.SystemClock{}, + Proxy: proxy.Default(), } code := cli.Execute(ctx, cli.New(opts), os.Args[1:], os.Stdout, os.Stderr) stop() os.Exit(code) } + +// stdinIsTerminal reports whether stdin is a terminal, i.e. a human can answer a question. +func stdinIsTerminal() bool { + fi, err := os.Stdin.Stat() + return err == nil && fi.Mode()&os.ModeCharDevice != 0 +} diff --git a/internal/budget/budget.go b/internal/budget/budget.go new file mode 100644 index 0000000..2882f95 --- /dev/null +++ b/internal/budget/budget.go @@ -0,0 +1,74 @@ +// Package budget decides whether another stack fits into the memory the machine has for siding. +// Everything here is a pure function; the numbers come from Docker and the compose config. +package budget + +import ( + "fmt" + "strconv" + "strings" + + "github.com/tweinmann/siding/internal/compose" +) + +const ( + // ServiceDefault is the estimate of a service without a memory limit. It is a guess, until + // siding learns real peaks (v0.3). + ServiceDefault int64 = 256 << 20 + // DefaultShare is the part of Docker's memory that is the budget when none is set. + DefaultShare = 0.75 +) + +// Default is the budget without a label: 75 % of the memory the Docker daemon reports. +func Default(dockerMemory int64) int64 { + return int64(float64(dockerMemory) * DefaultShare) +} + +// ParseSize reads a size as bytes or with the binary suffix k, m or g: 512m, 12g, 1073741824. +func ParseSize(s string) (int64, error) { + s = strings.ToLower(strings.TrimSpace(s)) + shift := 0 + switch { + case strings.HasSuffix(s, "k"): + shift = 10 + case strings.HasSuffix(s, "m"): + shift = 20 + case strings.HasSuffix(s, "g"): + shift = 30 + } + if shift > 0 { + s = s[:len(s)-1] + } + n, err := strconv.ParseInt(s, 10, 64) + if err != nil || n <= 0 || n > (1<<62)>>shift { + return 0, fmt.Errorf("not a size: use bytes or a number with k, m or g, such as 512m or 12g") + } + return n << shift, nil +} + +// Estimate is the memory a project will need: per service and replica its memory limit, or +// ServiceDefault without one. +func Estimate(p *compose.Project) int64 { + var sum int64 + for _, s := range p.Services { + per := s.MemoryLimit + if per <= 0 { + per = ServiceDefault + } + replicas := s.Replicas + if replicas < 1 { + replicas = 1 + } + sum += per * int64(replicas) + } + return sum +} + +// Check is the outcome of fitting a stack into the budget. +type Check struct { + Budget int64 + Used int64 // memory of every other stack + Estimate int64 // of this stack +} + +// Fits reports whether the stack fits. +func (c Check) Fits() bool { return c.Used+c.Estimate <= c.Budget } diff --git a/internal/budget/budget_test.go b/internal/budget/budget_test.go new file mode 100644 index 0000000..91ac3c8 --- /dev/null +++ b/internal/budget/budget_test.go @@ -0,0 +1,53 @@ +package budget + +import ( + "testing" + + "github.com/tweinmann/siding/internal/compose" +) + +const mib = 1 << 20 + +func TestParseSize(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + in string + want int64 + }{ + {"512m", 512 * mib}, {"12g", 12 << 30}, {"12G", 12 << 30}, {"64k", 64 << 10}, {"1073741824", 1 << 30}, {" 2g ", 2 << 30}, + {"", 0}, {"0", 0}, {"-1g", 0}, {"12gb", 0}, {"lots", 0}, {"99999999999g", 0}, + } { + got, err := ParseSize(tt.in) + if (tt.want == 0) != (err != nil) || got != tt.want { + t.Errorf("ParseSize(%q) = %d, %v; want %d", tt.in, got, err, tt.want) + } + } +} + +func TestEstimate(t *testing.T) { + t.Parallel() + p := &compose.Project{Services: map[string]compose.Service{ + "db": {MemoryLimit: 1024 * mib, Replicas: 1}, + "web": {Replicas: 1}, + "api": {MemoryLimit: 512 * mib, Replicas: 1}, + "worker": {MemoryLimit: 100 * mib, Replicas: 3}, + "old": {}, // a project parsed before replicas existed counts once + }} + want := int64(1024+256+512+300+256) * mib + if got := Estimate(p); got != want { + t.Errorf("Estimate = %d MiB, want %d MiB", got/mib, want/mib) + } +} + +func TestCheck(t *testing.T) { + t.Parallel() + if !(Check{Budget: 4096 * mib, Used: 3000 * mib, Estimate: 1096 * mib}).Fits() { + t.Error("exactly the budget must fit") + } + if (Check{Budget: 4096 * mib, Used: 3584 * mib, Estimate: 768 * mib}).Fits() { + t.Error("over the budget must not fit") + } + if got := Default(16 << 30); got != 12<<30 { + t.Errorf("Default(16 GiB) = %d", got) + } +} diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 4a33737..f226ed1 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -26,7 +26,7 @@ const ( exitOK = 0 exitFailed = 1 exitTimedOut = 2 - // exitNoCapacity is reserved for the memory budget (v0.2). + // exitNoCapacity: up refused because the stack does not fit into the memory budget. exitNoCapacity = 3 // exitInterrupted ends a command stopped by Ctrl-C or SIGTERM, without a message, as // shells do. @@ -45,6 +45,11 @@ type Options struct { Getenv func(string) string // OSMachineID returns the operating system's machine id, or "". OSMachineID func() string + // IsLocalAddress reports whether an IP address belongs to this machine. + IsLocalAddress func(ip string) bool + // Stdin is where questions are answered; IsTerminal reports whether a human is there. + Stdin io.Reader + IsTerminal func() bool Git Git Compose Compose @@ -60,7 +65,7 @@ type Options struct { type Git interface { Worktree(ctx context.Context, dir string) (git.Worktree, error) Dirty(ctx context.Context, top string) (bool, error) - Merged(ctx context.Context, top, branch string) (merged bool, base string, err error) + MergeState(ctx context.Context, r git.Repo, branch string) (git.MergeState, error) } // Compose runs docker compose. @@ -77,6 +82,10 @@ type Docker interface { Networks(ctx context.Context, labels map[string]string) ([]docker.Network, error) Volumes(ctx context.Context, labels map[string]string) ([]docker.Volume, error) EnsureNetwork(ctx context.Context, name string, labels map[string]string) (bool, error) + MemTotal(ctx context.Context) (int64, error) + NetworkGateway(ctx context.Context, name string) (string, error) + NetworkUsers(ctx context.Context, name string) (exists bool, users []string, err error) + RemoveNetwork(ctx context.Context, name string) error Logs(ctx context.Context, id string, tail int) ([]string, error) Memory(ctx context.Context, id string) (uint64, error) } @@ -121,6 +130,10 @@ func New(opts Options) *cobra.Command { newLogsCommand(a), newOpenCommand(a), newDownCommand(a), + newGCCommand(a), + newLintCommand(a), + newUICommand(a), + newTeardownCommand(a), newSkillCommand(), ) return root @@ -168,6 +181,9 @@ func Execute(ctx context.Context, root *cobra.Command, args []string, stdout, st _, _ = fmt.Fprintf(stderr, "hint: %s\n", e.Hint) } } + if e.Code == fail.NoCapacity { + return exitNoCapacity + } return exitFailed } diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index a202261..3a66dad 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -113,7 +113,7 @@ func TestErrorCodes(t *testing.T) { fail.InvalidArguments, fail.NotAGitRepository, fail.NoBranch, fail.ComposeFileNotFound, fail.ComposeFailed, fail.DockerUnavailable, fail.ProxyNotRunning, fail.PortInUse, fail.InvalidRoute, fail.AmbiguousPort, fail.SlugCollision, fail.StackNotFound, - fail.UnmergedBranch, fail.DirtyWorktree, fail.Internal, + fail.UnmergedBranch, fail.DirtyWorktree, fail.NoCapacity, fail.NotInteractive, fail.Internal, } for _, code := range codes { t.Run(code, func(t *testing.T) { @@ -127,8 +127,12 @@ func TestErrorCodes(t *testing.T) { root.AddCommand(sub) var out, errOut bytes.Buffer - if got := Execute(context.Background(), root, []string{"boom"}, &out, &errOut); got != exitFailed { - t.Errorf("exit code %d", got) + wantCode := exitFailed + if code == fail.NoCapacity { + wantCode = exitNoCapacity + } + if got := Execute(context.Background(), root, []string{"boom"}, &out, &errOut); got != wantCode { + t.Errorf("exit code %d, want %d", got, wantCode) } if out.String() != "" || errOut.String() != "error: it broke\nhint: try again\n" { t.Errorf("text: stdout %q, stderr %q", out.String(), errOut.String()) diff --git a/internal/cli/down.go b/internal/cli/down.go index 484e245..61971a4 100644 --- a/internal/cli/down.go +++ b/internal/cli/down.go @@ -10,6 +10,7 @@ import ( "github.com/tweinmann/siding/internal/compose" "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/git" "github.com/tweinmann/siding/internal/stack" ) @@ -85,17 +86,17 @@ func (a *app) keepsWork(ctx context.Context, st existing) error { return fail.New(fail.DirtyWorktree, "the worktree %s has uncommitted changes; its volumes stay", st.wt.Top). WithHint("commit or stash them first, or remove the data anyway with --force") } - merged, base, err := a.opts.Git.Merged(ctx, st.wt.Top, st.wt.Branch) + m, err := a.opts.Git.MergeState(ctx, git.Repo{Dir: st.wt.Top}, st.wt.Branch) if err != nil { return err } - if !merged { + if !m.Merged() { into := "no default branch was found (origin/HEAD, main or master)" - if base != "" { - into = "it is not merged into " + strings.TrimPrefix(strings.TrimPrefix(base, "refs/remotes/"), "refs/heads/") + if m.Base != "" { + into = "it is not merged into " + m.BaseName() } return fail.New(fail.UnmergedBranch, "the branch %s may hold unmerged work: %s; its volumes stay", st.wt.Branch, into). - WithHint("merge it first, or remove the data anyway with --force (a squash merge also needs --force)") + WithHint("merge it first, or remove the data anyway with --force (after a squash merge, git fetch --prune makes it count as merged)") } return nil } diff --git a/internal/cli/down_test.go b/internal/cli/down_test.go index cbeee1e..6bec782 100644 --- a/internal/cli/down_test.go +++ b/internal/cli/down_test.go @@ -108,6 +108,19 @@ func TestDown(t *testing.T) { }) } + t.Run("squash-merged: upstream gone", func(t *testing.T) { + t.Parallel() + e := downEnv(t, "feat") + origin := filepath.Join(e.base, "origin.git") + e.git("init", "-q", "--bare", origin) + e.git("remote", "add", "origin", origin) + e.git("push", "-q", "-u", "origin", "feat") + e.git("push", "-q", "origin", "--delete", "feat") // the PR was merged and its branch deleted + if code, stdout, _ := e.run("down", "--volumes", "--json"); code != exitOK || !strings.Contains(stdout, "shop-feat_data") { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) + t.Run("no stack", func(t *testing.T) { t.Parallel() e := newEnv(t) diff --git a/internal/cli/env_test.go b/internal/cli/env_test.go index 3cd23fb..6f51d52 100644 --- a/internal/cli/env_test.go +++ b/internal/cli/env_test.go @@ -3,11 +3,14 @@ package cli import ( "bytes" "context" + "fmt" "io" "maps" "os" "os/exec" "path/filepath" + "regexp" + "strconv" "strings" "sync" "testing" @@ -64,7 +67,7 @@ func newEnv(t *testing.T) *env { t: t, base: base, repo: filepath.Join(base, "shop"), - docker: &fakeDocker{logs: map[string][]string{}, memory: map[string]uint64{}}, + docker: &fakeDocker{logs: map[string][]string{}, memory: map[string]uint64{}, memTotal: 16 << 30}, compose: &fakeCompose{config: testConfig}, probe: &fakeProbe{answers: map[string]probe.Response{}}, browser: &fakeBrowser{}, @@ -85,17 +88,60 @@ func newEnv(t *testing.T) *env { Home: filepath.Join(base, "home"), Getenv: func(k string) string { return e.envVars[k] }, OSMachineID: func() string { return "not-a-real-machine-id" }, - Git: git.Client{}, - Compose: e.compose, - Docker: e.docker, - Probe: e.probe, - Browser: e.browser, - Clock: e.clock, - Proxy: proxy.Default(), + Stdin: strings.NewReader(""), + IsTerminal: func() bool { return false }, + // The daemon runs on this machine, as in the devcontainer. + IsLocalAddress: func(ip string) bool { return ip == "172.19.0.1" }, + Git: git.Client{}, + Compose: e.compose, + Docker: e.docker, + Probe: e.probe, + Browser: e.browser, + Clock: e.clock, + Proxy: proxy.Default(), } + e.compose.onProxy = e.composeProxy return e } +var publishedPort = regexp.MustCompile(`"127\.0\.0\.1:(\d+):80"`) +var budgetLabel = regexp.MustCompile(`dev\.siding\.budget: "(\d+)"`) +var hostAliasLabel = regexp.MustCompile(`dev\.siding\.host-alias: "([^"]+)"`) + +// composeProxy does what docker compose up does for the proxy's project: it recreates the +// container, with a new ID, only when the compose file changed or the container is gone. +func (e *env) composeProxy(r composeRun) { + if r.args[0] == "down" { + e.docker.mu.Lock() + e.docker.proxy, e.docker.proxyFile = nil, "" + e.docker.mu.Unlock() + return + } + data, err := os.ReadFile(r.inv.Files[0]) + if err != nil { + e.t.Error(err) + return + } + e.docker.mu.Lock() + defer e.docker.mu.Unlock() + if e.docker.proxy != nil && e.docker.proxy.State == "running" && e.docker.proxyFile == string(data) { + return + } + e.docker.proxyFile = string(data) + e.docker.sidingNetwork = true + e.docker.proxyGen++ + port, _ := strconv.Atoi(publishedPort.FindStringSubmatch(string(data))[1]) + labels := map[string]string{} + if m := budgetLabel.FindStringSubmatch(string(data)); m != nil { + labels[stack.LabelBudget] = m[1] + } + if m := hostAliasLabel.FindStringSubmatch(string(data)); m != nil { + labels[stack.LabelHostAlias] = m[1] + } + e.docker.proxy = &docker.Container{ID: fmt.Sprintf("proxy-%d", e.docker.proxyGen), Name: "siding-proxy", Image: proxy.Image, + State: "running", Labels: labels, Ports: []docker.Port{{HostIP: "127.0.0.1", HostPort: port, ContainerPort: 80}}} +} + func (e *env) git(args ...string) { e.t.Helper() cmd := exec.Command("git", append([]string{"-C", e.repo}, args...)...) @@ -163,15 +209,22 @@ func container(id stack.Identity, service, state, health string, exit int) docke } type fakeDocker struct { - mu sync.Mutex - pingErr error - proxy *docker.Container - containers []docker.Container - networks []docker.Network - volumes []docker.Volume - logs map[string][]string - memory map[string]uint64 - ensured []string + mu sync.Mutex + pingErr error + memTotal int64 + proxy *docker.Container + proxyFile string // the compose file the proxy was created from + proxyGen int + // sidingNetwork: the network siding exists. Its users are the proxy and every running stack + // container. + sidingNetwork bool + removedNetworks []string + containers []docker.Container + networks []docker.Network + volumes []docker.Volume + logs map[string][]string + memory map[string]uint64 + ensured []string } func (d *fakeDocker) add(c docker.Container) { @@ -257,6 +310,40 @@ func (d *fakeDocker) EnsureNetwork(_ context.Context, name string, _ map[string] return true, nil } +func (d *fakeDocker) MemTotal(context.Context) (int64, error) { return d.memTotal, nil } + +func (d *fakeDocker) NetworkGateway(context.Context, string) (string, error) { + return "172.19.0.1", nil +} + +func (d *fakeDocker) NetworkUsers(_ context.Context, name string) (bool, []string, error) { + d.mu.Lock() + defer d.mu.Unlock() + if name != "siding" || !d.sidingNetwork { + return false, nil, nil + } + var users []string + if d.proxy != nil { + users = append(users, d.proxy.Name) + } + for _, c := range d.containers { + if c.State == "running" { + users = append(users, c.Name) + } + } + return true, users, nil +} + +func (d *fakeDocker) RemoveNetwork(_ context.Context, name string) error { + d.mu.Lock() + defer d.mu.Unlock() + d.removedNetworks = append(d.removedNetworks, name) + if name == "siding" { + d.sidingNetwork = false + } + return nil +} + func (d *fakeDocker) Logs(_ context.Context, id string, tail int) ([]string, error) { d.mu.Lock() defer d.mu.Unlock() @@ -283,8 +370,9 @@ type fakeCompose struct { config string runs []composeRun runErr error - // onRun, if set, runs after each recorded run, e.g. to start containers. - onRun func(composeRun) + // onRun, if set, runs after each recorded run, e.g. to start containers; onProxy instead + // for runs of the proxy's project. + onRun, onProxy func(composeRun) } func (c *fakeCompose) Config(context.Context, compose.Invocation) ([]byte, error) { @@ -296,6 +384,9 @@ func (c *fakeCompose) Run(_ context.Context, inv compose.Invocation, args []stri r := composeRun{inv: inv, args: args} c.runs = append(c.runs, r) onRun, err := c.onRun, c.runErr + if inv.Project == "siding" { + onRun = c.onProxy + } c.mu.Unlock() if err != nil { return err diff --git a/internal/cli/gc.go b/internal/cli/gc.go new file mode 100644 index 0000000..7a69779 --- /dev/null +++ b/internal/cli/gc.go @@ -0,0 +1,223 @@ +package cli + +import ( + "bufio" + "context" + "errors" + "fmt" + "io/fs" + "os" + "sort" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/gc" + "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/stack" +) + +func newGCCommand(a *app) *cobra.Command { + var asJSON, dryRun, yes bool + cmd := &cobra.Command{ + Use: "gc", + Short: "Remove the stacks that can go: merged branches and deleted worktrees", + Long: `Remove the stacks of this machine that can go: + - a merged branch with a clean worktree: containers, networks and volumes; + - a deleted worktree: containers and networks, and the volumes only if the branch is merged. +A branch counts as merged when the default branch has moved past its tip, or when its upstream +branch is gone (as after a squash merge and git fetch --prune). gc never removes git +worktrees or branches, and never touches unmerged or uncommitted work. It asks first; --yes +removes without asking.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + ctx := cmd.Context() + doc, actions, err := a.gcPlan(ctx) + if err != nil { + return err + } + if dryRun || len(actions) == 0 { + doc.DryRun = dryRun + return printGC(cmd, doc, asJSON) + } + if !yes { + if asJSON || a.opts.IsTerminal == nil || !a.opts.IsTerminal() { + return fail.New(fail.NotInteractive, "siding gc asks before removing, and nobody can answer here"). + WithHint("list what it would remove with --dry-run, and remove it with --yes") + } + doc.DryRun = true + if err := printGC(cmd, doc, false); err != nil { + return err + } + doc.DryRun = false + _, _ = fmt.Fprint(cmd.ErrOrStderr(), "Remove these? [y/N] ") + answer, _ := bufio.NewReader(a.opts.Stdin).ReadString('\n') + if a := strings.ToLower(strings.TrimSpace(answer)); a != "y" && a != "yes" { + _, err := fmt.Fprintln(cmd.OutOrStdout(), "Nothing was removed.") + return err + } + } + for _, act := range actions { + args := []string{"down", "--remove-orphans"} + if act.Volumes { + args = append(args, "--volumes") + } + // By project name, from a directory that exists: the worktree may be gone. + inv := compose.Invocation{Dir: a.opts.Home, Project: act.Project} + if err := a.opts.Compose.Run(ctx, inv, args, cmd.ErrOrStderr(), cmd.ErrOrStderr()); err != nil { + return err + } + } + return printGC(cmd, doc, asJSON) + }, + } + cmd.Flags().BoolVar(&dryRun, "dry-run", false, "list what would be removed, remove nothing") + cmd.Flags().BoolVar(&yes, "yes", false, "remove without asking") + registerJSON(cmd, &asJSON) + return cmd +} + +// gcPlan gathers what Docker and git know about every stack and plans gc. The overview uses it +// too, for its "can go" list. +func (a *app) gcPlan(ctx context.Context) (gcDocument, []gc.Action, error) { + doc := gcDocument{header: newHeader(kindGC)} + machine, err := a.machineID() + if err != nil { + return doc, nil, err + } + if err := a.opts.Docker.Ping(ctx); err != nil { + return doc, nil, err + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelRepo: ""}) + if err != nil { + return doc, nil, err + } + projects := map[string]stack.Identity{} + for _, c := range cs { + if _, ok := projects[c.Labels[stack.LabelProject]]; !ok { + projects[c.Labels[stack.LabelProject]] = stack.FromLabels(c.Labels) + } + } + var facts []gc.Facts + for project, id := range projects { + facts = append(facts, a.gcFacts(ctx, project, id)) + } + actions, kept := gc.Plan(facts, machine) + for _, act := range actions { + id := projects[act.Project] + s := gcStack{Project: act.Project, Repo: id.Repo, Branch: id.Branch, Worktree: id.Worktree, Reason: act.Reason} + byProject := map[string]string{stack.LabelProject: act.Project} + containers, err := a.opts.Docker.Containers(ctx, byProject) + if err != nil { + return doc, nil, err + } + networks, err := a.opts.Docker.Networks(ctx, byProject) + if err != nil { + return doc, nil, err + } + volumes, err := a.opts.Docker.Volumes(ctx, byProject) + if err != nil { + return doc, nil, err + } + s.Containers, s.Networks = containerNames(containers), networkNames(networks) + if act.Volumes { + s.Volumes = volumeNames(volumes) + } else { + s.KeptVolumes = volumeNames(volumes) + } + doc.Stacks = append(doc.Stacks, s) + } + for _, k := range kept { + id := projects[k.Project] + doc.Kept = append(doc.Kept, gcKept{Project: k.Project, Repo: id.Repo, Branch: id.Branch, Worktree: id.Worktree, Reason: k.Reason}) + } + return doc, actions, nil +} + +// gcFacts asks the file system and git about one stack. When git cannot answer, the merge state +// stays unknown, and gc keeps the stack's data. +func (a *app) gcFacts(ctx context.Context, project string, id stack.Identity) gc.Facts { + f := gc.Facts{Project: project, Branch: id.Branch, Worktree: id.Worktree, Machine: id.Machine} + repo := git.Repo{GitDir: id.GitDir} + if _, err := os.Stat(id.Worktree); !errors.Is(err, fs.ErrNotExist) { + f.WorktreeExists = true + dirty, err := a.opts.Git.Dirty(ctx, id.Worktree) + f.Dirty = dirty || err != nil // when in doubt, it holds work + repo = git.Repo{Dir: id.Worktree} + } + if repo.Dir == "" && repo.GitDir == "" { + return f + } + if m, err := a.opts.Git.MergeState(ctx, repo, id.Branch); err == nil { + f.Merge, f.MergeKnown = m, true + } + return f +} + +func containerNames(cs []docker.Container) []string { + var out []string + for _, c := range cs { + out = append(out, c.Name) + } + sort.Strings(out) + return out +} + +func networkNames(ns []docker.Network) []string { + var out []string + for _, n := range ns { + out = append(out, n.Name) + } + sort.Strings(out) + return out +} + +func volumeNames(vs []docker.Volume) []string { + var out []string + for _, v := range vs { + out = append(out, v.Name) + } + sort.Strings(out) + return out +} + +func printGC(cmd *cobra.Command, doc gcDocument, asJSON bool) error { + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + var b strings.Builder + if len(doc.Stacks) == 0 { + b.WriteString("Nothing to remove.\n") + } + verb := "Removed" + if doc.DryRun { + verb = "Would remove" + } + for _, s := range doc.Stacks { + fmt.Fprintf(&b, "%s %s (%s, %s):\n", verb, s.Project, s.Branch, s.Reason) + for _, list := range []struct { + kind string + names []string + }{{"container", s.Containers}, {"network", s.Networks}, {"volume", s.Volumes}} { + for _, n := range list.names { + fmt.Fprintf(&b, " %s %s\n", list.kind, n) + } + } + for _, v := range s.KeptVolumes { + fmt.Fprintf(&b, " keeps volume %s: the branch may hold unmerged work\n", v) + } + } + if len(doc.Kept) > 0 { + b.WriteString("Kept:\n") + var rows [][]string + for _, k := range doc.Kept { + rows = append(rows, []string{k.Project, k.Branch, k.Reason}) + } + b.WriteString(table(" ", rows)) + } + _, err := fmt.Fprint(cmd.OutOrStdout(), b.String()) + return err +} diff --git a/internal/cli/gc_test.go b/internal/cli/gc_test.go new file mode 100644 index 0000000..eaaaada --- /dev/null +++ b/internal/cli/gc_test.go @@ -0,0 +1,125 @@ +package cli + +import ( + "os" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/docker" + "github.com/tweinmann/siding/internal/stack" + "github.com/tweinmann/siding/internal/testutil" +) + +// gcEnv has five stacks: main (the default branch), done (merged, main moved on, clean), wip +// (dirty), old (worktree deleted, branch merged) and one of another machine. +func gcEnv(t *testing.T) *env { + t.Helper() + e := newEnv(t) + e.git("branch", "done") + e.git("branch", "old") + e.git("commit", "-q", "--allow-empty", "-m", "main moved on") + e.git("worktree", "add", "-q", filepath.Join(e.base, "shop-done"), "done") + e.git("worktree", "add", "-q", "-b", "wip", filepath.Join(e.base, "shop-wip")) + if err := os.WriteFile(filepath.Join(e.base, "shop-wip", "notes.txt"), []byte("unsaved"), 0o644); err != nil { + t.Fatal(err) + } + + add := func(branch, worktree, machine string) stack.Identity { + id := e.id() + id.Branch, id.Worktree, id.GitDir = branch, worktree, filepath.Join(e.repo, ".git") + if machine != "" { + id.Machine = machine + } + e.docker.add(container(id, "web", "running", "", 0)) + project := map[string]string{stack.LabelProject: id.Project()} + e.docker.networks = append(e.docker.networks, docker.Network{Name: id.Project() + "_default", Labels: project}) + e.docker.volumes = append(e.docker.volumes, docker.Volume{Name: id.Project() + "_data", Labels: project}) + return id + } + add("main", e.repo, "") + add("done", filepath.Join(e.base, "shop-done"), "") + add("wip", filepath.Join(e.base, "shop-wip"), "") + add("old", filepath.Join(e.base, "shop-old"), "") // never existed: its worktree is gone + add("remote", filepath.Join(e.base, "elsewhere"), "another-machine") + return e +} + +func TestGC(t *testing.T) { + t.Parallel() + + t.Run("dry run", func(t *testing.T) { + t.Parallel() + e := gcEnv(t) + code, stdout, _ := e.run("gc", "--dry-run", "--json") + if code != exitOK || len(e.compose.recorded()) != 0 { + t.Fatalf("exit %d, compose runs %d: %s", code, len(e.compose.recorded()), stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "gc-dryrun.json"), []byte(stdout)) + }) + + t.Run("yes", func(t *testing.T) { + t.Parallel() + e := gcEnv(t) + code, stdout, _ := e.run("gc", "--yes", "--json") + if code != exitOK { + t.Fatalf("exit %d: %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "gc.json"), []byte(stdout)) + var got []string + for _, r := range e.compose.recorded() { + got = append(got, r.inv.Project+" "+strings.Join(r.args, " ")) + } + want := []string{"shop-done down --remove-orphans --volumes", "shop-old down --remove-orphans --volumes"} + if !slices.Equal(got, want) { + t.Errorf("compose ran %q, want %q", got, want) + } + for _, dir := range []string{"shop-done", "shop-wip"} { + if _, err := os.Stat(filepath.Join(e.base, dir)); err != nil { + t.Errorf("gc touched the worktree %s: %v", dir, err) + } + } + }) + + t.Run("asks on a terminal", func(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + answer string + runs int + }{{"y\n", 2}, {"n\n", 0}, {"\n", 0}} { + e := gcEnv(t) + e.opts.IsTerminal = func() bool { return true } + e.opts.Stdin = strings.NewReader(tt.answer) + code, stdout, stderr := e.run("gc") + if code != exitOK || len(e.compose.recorded()) != tt.runs || !strings.Contains(stderr, "Remove these? [y/N]") || + !strings.Contains(stdout, "Would remove shop-done (done, Merged)") { + t.Errorf("answer %q: exit %d, compose runs %d, stdout %q, stderr %q", tt.answer, code, len(e.compose.recorded()), stdout, stderr) + } + } + }) + + t.Run("nobody to ask", func(t *testing.T) { + t.Parallel() + e := gcEnv(t) + if code, _, stderr := e.run("gc"); code != exitFailed || !strings.Contains(stderr, "--yes") { + t.Errorf("no terminal: exit %d, stderr %q", code, stderr) + } + e.opts.IsTerminal = func() bool { return true } + if code, stdout, _ := e.run("gc", "--json"); code != exitFailed || !strings.Contains(stdout, `"code": "NotInteractive"`) { + t.Errorf("--json: exit %d, stdout %s", code, stdout) + } + if len(e.compose.recorded()) != 0 { + t.Error("gc removed something without being asked") + } + }) + + t.Run("nothing to remove", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.stackUp(e.id(), "web") + if code, stdout, _ := e.run("gc"); code != exitOK || !strings.HasPrefix(stdout, "Nothing to remove.") { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) +} diff --git a/internal/cli/json.go b/internal/cli/json.go index be95652..c6f070d 100644 --- a/internal/cli/json.go +++ b/internal/cli/json.go @@ -18,16 +18,19 @@ import ( const documentAPIVersion = "siding.dev/v1alpha1" const ( - kindAddress = "Address" - kindDown = "Down" - kindError = "Error" - kindLogs = "Logs" - kindSetup = "Setup" - kindSkill = "Skill" - kindStack = "Stack" - kindStackList = "StackList" - kindStatus = "Status" - kindVersion = "Version" + kindAddress = "Address" + kindDown = "Down" + kindError = "Error" + kindGC = "GC" + kindLintReport = "LintReport" + kindLogs = "Logs" + kindSetup = "Setup" + kindSkill = "Skill" + kindStack = "Stack" + kindStackList = "StackList" + kindStatus = "Status" + kindTeardown = "Teardown" + kindVersion = "Version" ) // jsonFlag is the name of the flag, looked up again by Execute to report an error as a document. @@ -106,6 +109,8 @@ type setupDocument struct { URL string `json:"url"` // State is created, updated or unchanged. State string `json:"state"` + // BudgetBytes is the memory budget for all stacks. + BudgetBytes int64 `json:"budgetBytes"` } // stackDocument is what up prints without --wait. @@ -158,6 +163,7 @@ type stackListDocument struct { header Stacks []stackEntry `json:"stacks,omitempty"` MemoryBytes uint64 `json:"memoryBytes,omitempty"` + BudgetBytes int64 `json:"budgetBytes"` } type stackEntry struct { @@ -209,3 +215,62 @@ type skillDocument struct { Name string `json:"name"` Text string `json:"text"` } + +// gcDocument is what gc prints, and what the overview's /api/gc returns. +type gcDocument struct { + header + DryRun bool `json:"dryRun,omitempty"` + Stacks []gcStack `json:"stacks,omitempty"` + Kept []gcKept `json:"kept,omitempty"` +} + +type gcStack struct { + Project string `json:"project"` + Repo string `json:"repo"` + Branch string `json:"branch"` + Worktree string `json:"worktree"` + // Reason is Merged or WorktreeGone. + Reason string `json:"reason"` + Containers []string `json:"containers,omitempty"` + Networks []string `json:"networks,omitempty"` + Volumes []string `json:"volumes,omitempty"` + KeptVolumes []string `json:"keptVolumes,omitempty"` +} + +type gcKept struct { + Project string `json:"project"` + Repo string `json:"repo"` + Branch string `json:"branch"` + Worktree string `json:"worktree"` + // Reason is Dirty, Unmerged or DefaultBranch. + Reason string `json:"reason"` +} + +// lintDocument is what lint --json prints. +type lintDocument struct { + header + File string `json:"file"` + Findings []lintFinding `json:"findings,omitempty"` +} + +type lintFinding struct { + Rule string `json:"rule"` + // Severity is error, warning or info. + Severity string `json:"severity"` + Subject string `json:"subject"` + Message string `json:"message"` + Hint string `json:"hint,omitempty"` +} + +// teardownDocument is what teardown prints. +type teardownDocument struct { + header + DryRun bool `json:"dryRun,omitempty"` + // Container and Network are what teardown removes (or would). + Container string `json:"container,omitempty"` + Network string `json:"network,omitempty"` + // KeptNetwork is the proxy network when stacks still use it. + KeptNetwork string `json:"keptNetwork,omitempty"` + // Stacks keep running as plain Compose projects. + Stacks []string `json:"stacks,omitempty"` +} diff --git a/internal/cli/lint.go b/internal/cli/lint.go new file mode 100644 index 0000000..f142b3c --- /dev/null +++ b/internal/cli/lint.go @@ -0,0 +1,79 @@ +package cli + +import ( + "fmt" + "path/filepath" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/lint" + "github.com/tweinmann/siding/internal/stack" +) + +func newLintCommand(a *app) *cobra.Command { + var asJSON bool + var file string + cmd := &cobra.Command{ + Use: "lint", + Short: "Check a compose file for what breaks parallel stacks", + Long: `Check the compose file siding would use for what breaks parallel stacks (fixed container, +network or volume names, routes up would refuse) as errors, for what weakens the verdict or is +shared between stacks as warnings, and for what siding changes as info. lint never changes a +file. It exits 1 when there is an error.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + ctx := cmd.Context() + // The project and worktree siding would use; outside a worktree, the directory. + top, project := a.opts.Dir, stack.Slug(filepath.Base(a.opts.Dir)) + if wt, err := a.opts.Git.Worktree(ctx, a.opts.Dir); err == nil { + top = wt.Top + project = stack.Identity{Repo: wt.RepoName(), Branch: wt.Branch}.Project() + } + path, p, err := a.loadProject(ctx, top, file, project) + if err != nil { + return err + } + findings := lint.Check(p, project, top) + doc := lintDocument{header: newHeader(kindLintReport), File: path} + for _, f := range findings { + doc.Findings = append(doc.Findings, lintFinding(f)) + } + if asJSON { + err = printJSON(cmd.OutOrStdout(), doc) + } else { + err = printLintText(cmd, doc) + } + if err != nil { + return err + } + if lint.HasErrors(findings) { + return exitCode(exitFailed) + } + return nil + }, + } + cmd.Flags().StringVarP(&file, "file", "f", "", "compose file (default: compose.yaml or another name Compose looks for)") + registerJSON(cmd, &asJSON) + return cmd +} + +func printLintText(cmd *cobra.Command, doc lintDocument) error { + var b strings.Builder + if len(doc.Findings) == 0 { + fmt.Fprintf(&b, "%s: no findings\n", doc.File) + } + count := map[string]int{} + for _, f := range doc.Findings { + count[f.Severity]++ + fmt.Fprintf(&b, "%-7s %s, %s: %s\n", f.Severity, f.Rule, f.Subject, f.Message) + if f.Hint != "" { + fmt.Fprintf(&b, " hint: %s\n", f.Hint) + } + } + if len(doc.Findings) > 0 { + fmt.Fprintf(&b, "%s: %d errors, %d warnings, %d info\n", doc.File, count[lint.Error], count[lint.Warning], count[lint.Info]) + } + _, err := fmt.Fprint(cmd.OutOrStdout(), b.String()) + return err +} diff --git a/internal/cli/lint_test.go b/internal/cli/lint_test.go new file mode 100644 index 0000000..3121f88 --- /dev/null +++ b/internal/cli/lint_test.go @@ -0,0 +1,71 @@ +package cli + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestLint(t *testing.T) { + t.Parallel() + lintConfig, err := os.ReadFile(filepath.Join("..", "compose", "testdata", "lint", "config.json")) + if err != nil { + t.Fatal(err) + } + + t.Run("errors as JSON", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.compose.config = string(lintConfig) + code, stdout, _ := e.run("lint", "--json") + if code != exitFailed { + t.Fatalf("exit %d: %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "lintreport.json"), []byte(stdout)) + }) + + t.Run("text", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.compose.config = string(lintConfig) + _, stdout, _ := e.run("lint") + for _, want := range []string{ + "error ContainerName, service db: db sets container_name \"shop-db\", so a second stack of this file cannot start\n hint: remove container_name", + "info NoRoot, stack:", + "/src/shop/compose.yaml: 3 errors, 6 warnings, 3 info\n", + } { + if !strings.Contains(stdout, want) { + t.Errorf("the text lacks %q:\n%s", want, stdout) + } + } + }) + + t.Run("clean", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.compose.config = `{"name": "shop-main", "services": {"web": {"expose": ["80"], "labels": {"dev.siding.route": "/"}, + "networks": {"default": null}, "healthcheck": {"test": ["CMD", "true"]}}}, "networks": {"default": {"name": "shop-main_default"}}}` + if code, stdout, _ := e.run("lint"); code != exitOK || stdout != "/src/shop/compose.yaml: no findings\n" { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) + + t.Run("outside a repository", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + dir := filepath.Join(e.base, "plain") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(dir, "compose.yml"), []byte("services: {}\n"), 0o644); err != nil { + t.Fatal(err) + } + e.opts.Dir = dir + if code, stdout, stderr := e.run("lint", "--json"); !strings.Contains(stdout, `"file": "/src/plain/compose.yml"`) || code == exitFailed && strings.Contains(stdout, `"kind": "Error"`) { + t.Errorf("exit %d, stdout %s, stderr %s", code, stdout, stderr) + } + }) +} diff --git a/internal/cli/ps.go b/internal/cli/ps.go index 47dc6b8..11217f1 100644 --- a/internal/cli/ps.go +++ b/internal/cli/ps.go @@ -49,6 +49,13 @@ func (a *app) ps(ctx context.Context) (stackListDocument, error) { return doc, err } port := a.shownPort(ctx) + proxyContainer, err := a.opts.Docker.Container(ctx, a.opts.Proxy.Container) + if err != nil { + return doc, err + } + if doc.BudgetBytes, err = a.budget(ctx, proxyContainer); err != nil { + return doc, err + } projects := map[string][]docker.Container{} for _, c := range cs { p := c.Labels[stack.LabelProject] @@ -118,7 +125,7 @@ func (a *app) memory(ctx context.Context, cs []docker.Container) (map[string]uin func printPsText(cmd *cobra.Command, doc stackListDocument) error { if len(doc.Stacks) == 0 { - _, err := fmt.Fprintln(cmd.OutOrStdout(), "No stacks. Start one with siding up in a worktree.") + _, err := fmt.Fprintf(cmd.OutOrStdout(), "No stacks. Start one with siding up in a worktree. Memory budget %s.\n", bytesText(uint64(doc.BudgetBytes))) return err } rows := [][]string{{"REPO", "BRANCH", "STATE", "URL", "MEMORY", "WORKTREE"}} @@ -129,8 +136,7 @@ func printPsText(cmd *cobra.Command, doc stackListDocument) error { } rows = append(rows, []string{s.Repo, s.Branch, s.State, s.URL, bytesText(s.MemoryBytes), worktree}) } - rows = append(rows, []string{"total", "", "", "", bytesText(doc.MemoryBytes), ""}) - _, err := fmt.Fprint(cmd.OutOrStdout(), table("", rows)) + _, err := fmt.Fprintf(cmd.OutOrStdout(), "%stotal: %s of %s\n", table("", rows), bytesText(doc.MemoryBytes), bytesText(uint64(doc.BudgetBytes))) return err } diff --git a/internal/cli/ps_test.go b/internal/cli/ps_test.go index aa2e10f..522c253 100644 --- a/internal/cli/ps_test.go +++ b/internal/cli/ps_test.go @@ -38,10 +38,10 @@ func TestPs(t *testing.T) { testutil.Golden(t, filepath.Join("testdata", "json", "stacklist.json"), []byte(stdout)) _, text, _ := e.run("ps") - want := `REPO BRANCH STATE URL MEMORY WORKTREE -shop feature/login partial http://shop.feature-login.localhost:8080/ 500 MiB /src/shop-login (missing) -shop main running http://shop.main.localhost:8080/ 300 MiB /src/shop -total 800 MiB + want := `REPO BRANCH STATE URL MEMORY WORKTREE +shop feature/login partial http://shop.feature-login.localhost:8080/ 500 MiB /src/shop-login (missing) +shop main running http://shop.main.localhost:8080/ 300 MiB /src/shop +total: 800 MiB of 12.0 GiB ` if text != want { t.Errorf("text:\n%s\nwant:\n%s", text, want) diff --git a/internal/cli/setup.go b/internal/cli/setup.go index 6fc003e..234a716 100644 --- a/internal/cli/setup.go +++ b/internal/cli/setup.go @@ -1,15 +1,19 @@ package cli import ( + "context" "errors" "fmt" "os" "path/filepath" + "strconv" "strings" "github.com/spf13/cobra" + "github.com/tweinmann/siding/internal/budget" "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" "github.com/tweinmann/siding/internal/fail" "github.com/tweinmann/siding/internal/proxy" "github.com/tweinmann/siding/internal/stack" @@ -17,19 +21,20 @@ import ( func newSetupCommand(a *app) *cobra.Command { var asJSON bool - var port int + var f setupFlags cmd := &cobra.Command{ Use: "setup", Short: "Start the shared proxy siding-proxy on 127.0.0.1", Long: `Start the shared proxy siding-proxy: Traefik on the network "siding", published on -127.0.0.1 only. Without --port it keeps the port of a running proxy, and uses 80 otherwise. -Running setup again changes nothing unless the port or the image differs.`, +127.0.0.1 only. Without --port it keeps the port of a running proxy, and uses 80 otherwise; +without --budget it keeps the budget set before. Running setup again recreates the proxy only +when something differs.`, Args: noArgs, RunE: func(cmd *cobra.Command, _ []string) error { - if port < 0 || port > 65535 { - return fail.New(fail.InvalidArguments, "--port must be between 1 and 65535, got %d", port) + if f.port < 0 || f.port > 65535 { + return fail.New(fail.InvalidArguments, "--port must be between 1 and 65535, got %d", f.port) } - doc, err := a.setup(cmd, port) + doc, err := a.setup(cmd, f) if err != nil { return err } @@ -37,16 +42,23 @@ Running setup again changes nothing unless the port or the image differs.`, return printJSON(cmd.OutOrStdout(), doc) } verb := map[string]string{"created": "started", "updated": "restarted", "unchanged": "already runs"}[doc.State] - _, err = fmt.Fprintf(cmd.OutOrStdout(), "%s %s on %s\n", doc.Container, verb, address("*.localhost", doc.Port, "/")) + _, err = fmt.Fprintf(cmd.OutOrStdout(), "%s %s on %s, memory budget %s\n", doc.Container, verb, + address("*.localhost", doc.Port, "/"), bytesText(uint64(doc.BudgetBytes))) return err }, } - cmd.Flags().IntVar(&port, "port", 0, "port on 127.0.0.1 for the proxy (default: the running proxy's, else 80)") + cmd.Flags().IntVar(&f.port, "port", 0, "port on 127.0.0.1 for the proxy (default: the running proxy's, else 80)") + cmd.Flags().StringVar(&f.budget, "budget", "", "memory budget for all stacks, e.g. 12g; \"default\" for 75% of Docker's memory (default: the budget set before)") registerJSON(cmd, &asJSON) return cmd } -func (a *app) setup(cmd *cobra.Command, port int) (setupDocument, error) { +type setupFlags struct { + port int + budget string +} + +func (a *app) setup(cmd *cobra.Command, f setupFlags) (setupDocument, error) { ctx := cmd.Context() cfg := a.opts.Proxy if err := a.opts.Docker.Ping(ctx); err != nil { @@ -60,51 +72,99 @@ func (a *app) setup(cmd *cobra.Command, port int) (setupDocument, error) { if err != nil { return setupDocument{}, err } - currentPort := 0 - if current != nil { - for _, p := range current.Ports { - if p.ContainerPort == proxyContainerPort { - currentPort = p.HostPort - } + settings := proxy.Settings{Port: f.port, Machine: machine} + if settings.Port == 0 { + settings.Port = cfg.DefaultPort + if p := hostPort(current); current != nil && current.State == "running" && p != 0 { + settings.Port = p } } - if port == 0 { - port = cfg.DefaultPort - if current != nil && current.State == "running" && currentPort != 0 { - port = currentPort + switch f.budget { + case "": + if current != nil { + settings.Budget, _ = strconv.ParseInt(current.Labels[stack.LabelBudget], 10, 64) + } + case "default": + default: + if settings.Budget, err = budget.ParseSize(f.budget); err != nil { + return setupDocument{}, fail.New(fail.InvalidArguments, "--budget %q: %v", f.budget, err) } - } - doc := setupDocument{header: newHeader(kindSetup), Container: cfg.Container, Network: cfg.Network, - Image: cfg.Image, Port: port, URL: address("*.localhost", port, "/")} - - if current != nil && current.State == "running" && currentPort == port && current.Image == cfg.Image { - doc.State = "unchanged" - return doc, nil - } - doc.State = "created" - if current != nil { - doc.State = "updated" } if _, err := a.opts.Docker.EnsureNetwork(ctx, cfg.Network, map[string]string{stack.LabelMachine: machine}); err != nil { return setupDocument{}, err } + // Where the proxy finds siding ui (design D2): on the network's gateway when the daemon + // runs on this machine, through Docker's host-gateway when it runs in a VM. + settings.HostAlias = proxy.HostGateway + gateway, err := a.opts.Docker.NetworkGateway(ctx, cfg.Network) + if err != nil { + return setupDocument{}, err + } + if gateway != "" && a.opts.IsLocalAddress != nil && a.opts.IsLocalAddress(gateway) { + settings.HostAlias = gateway + } dir := filepath.Join(a.opts.Home, ".siding", "proxy", cfg.Project) - if err := os.MkdirAll(dir, 0o755); err != nil { + settings.RoutesDir = filepath.Join(dir, "routes") + if err := os.MkdirAll(settings.RoutesDir, 0o755); err != nil { return setupDocument{}, fmt.Errorf("write the proxy's compose file: %w", err) } file := filepath.Join(dir, "compose.yml") - if err := os.WriteFile(file, proxy.ComposeFile(cfg, port, machine), 0o644); err != nil { + if err := os.WriteFile(file, proxy.ComposeFile(cfg, settings), 0o644); err != nil { return setupDocument{}, fmt.Errorf("write the proxy's compose file: %w", err) } + // Compose recreates the container only when the file differs, so it decides whether the + // proxy is up to date; the container's ID tells what happened (design D6). inv := compose.Invocation{Dir: dir, Project: cfg.Project, Files: []string{file}} err = a.opts.Compose.Run(ctx, inv, []string{"up", "-d", "--wait", "--wait-timeout", "60", "--remove-orphans"}, cmd.ErrOrStderr(), cmd.ErrOrStderr()) if err != nil { - return setupDocument{}, portInUse(err, port) + return setupDocument{}, portInUse(err, settings.Port) + } + after, err := a.opts.Docker.Container(ctx, cfg.Container) + if err != nil { + return setupDocument{}, err + } + doc := setupDocument{header: newHeader(kindSetup), Container: cfg.Container, Network: cfg.Network, + Image: cfg.Image, Port: settings.Port, URL: address("*.localhost", settings.Port, "/"), State: "updated"} + switch { + case current == nil: + doc.State = "created" + case after != nil && after.ID == current.ID && current.State == "running": + doc.State = "unchanged" + } + if doc.BudgetBytes, err = a.budget(ctx, after); err != nil { + return setupDocument{}, err } return doc, nil } +// hostPort is the port on the host a proxy container publishes, or 0. +func hostPort(c *docker.Container) int { + if c == nil { + return 0 + } + for _, p := range c.Ports { + if p.ContainerPort == proxyContainerPort { + return p.HostPort + } + } + return 0 +} + +// budget is the memory budget: the proxy's label, else 75 % of Docker's memory. +func (a *app) budget(ctx context.Context, proxyContainer *docker.Container) (int64, error) { + if proxyContainer != nil { + if b, err := strconv.ParseInt(proxyContainer.Labels[stack.LabelBudget], 10, 64); err == nil && b > 0 { + return b, nil + } + } + total, err := a.opts.Docker.MemTotal(ctx) + if err != nil { + return 0, err + } + return budget.Default(total), nil +} + // portInUse turns Compose's "address already in use" into PortInUse. func portInUse(err error, port int) error { var e *fail.Error diff --git a/internal/cli/setup_test.go b/internal/cli/setup_test.go index 0d7a1f3..c2eae29 100644 --- a/internal/cli/setup_test.go +++ b/internal/cli/setup_test.go @@ -1,6 +1,7 @@ package cli import ( + "encoding/json" "os" "path/filepath" "slices" @@ -11,6 +12,19 @@ import ( "github.com/tweinmann/siding/internal/testutil" ) +func setupDoc(t *testing.T, e *env, args ...string) setupDocument { + t.Helper() + code, stdout, stderr := e.run(append([]string{"setup", "--json"}, args...)...) + if code != exitOK { + t.Fatalf("setup %v: exit %d, %s %s", args, code, stdout, stderr) + } + var doc setupDocument + if err := json.Unmarshal([]byte(stdout), &doc); err != nil { + t.Fatal(err) + } + return doc +} + func TestSetup(t *testing.T) { t.Parallel() @@ -39,37 +53,99 @@ func TestSetup(t *testing.T) { t.Errorf("the proxy's compose file lacks %s:\n%s", want, data) } } + if strings.Contains(string(data), "dev.siding.budget") { + t.Error("a default budget must not be a label") + } if !strings.HasPrefix(runs[0].inv.Files[0], e.opts.Home+"/.siding/proxy/") { t.Errorf("compose file %s is not under ~/.siding", runs[0].inv.Files[0]) } }) - t.Run("already running", func(t *testing.T) { + t.Run("second setup changes nothing", func(t *testing.T) { t.Parallel() e := newEnv(t) - e.proxyUp(80) + setupDoc(t, e) code, stdout, _ := e.run("setup") - if code != exitOK || stdout != "siding-proxy already runs on http://*.localhost/\n" || len(e.compose.recorded()) != 0 { - t.Errorf("exit %d, stdout %q, compose runs %d", code, stdout, len(e.compose.recorded())) + if code != exitOK || stdout != "siding-proxy already runs on http://*.localhost/, memory budget 12.0 GiB\n" { + t.Errorf("exit %d, stdout %q", code, stdout) } }) t.Run("keeps the running port without --port", func(t *testing.T) { t.Parallel() e := newEnv(t) - e.proxyUp(8080) - if code, stdout, _ := e.run("setup"); code != exitOK || !strings.Contains(stdout, "already runs on http://*.localhost:8080/") { - t.Errorf("exit %d, stdout %q", code, stdout) + setupDoc(t, e, "--port", "8080") + if doc := setupDoc(t, e); doc.State != "unchanged" || doc.Port != 8080 { + t.Errorf("%+v", doc) } }) t.Run("another port recreates the proxy", func(t *testing.T) { t.Parallel() e := newEnv(t) - e.proxyUp(80) - code, stdout, _ := e.run("setup", "--port", "8080") - if code != exitOK || stdout != "siding-proxy restarted on http://*.localhost:8080/\n" || len(e.compose.recorded()) != 1 { - t.Errorf("exit %d, stdout %q", code, stdout) + setupDoc(t, e) + if doc := setupDoc(t, e, "--port", "8080"); doc.State != "updated" || doc.Port != 8080 || doc.URL != "http://*.localhost:8080/" { + t.Errorf("%+v", doc) + } + }) + + t.Run("budget", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + if doc := setupDoc(t, e, "--budget", "12g"); doc.BudgetBytes != 12<<30 { + t.Errorf("set: %+v", doc) + } + if got := e.docker.proxy.Labels["dev.siding.budget"]; got != "12884901888" { + t.Errorf("label %q", got) + } + if doc := setupDoc(t, e); doc.State != "unchanged" || doc.BudgetBytes != 12<<30 { + t.Errorf("kept: %+v", doc) + } + e.docker.memTotal = 8 << 30 + if doc := setupDoc(t, e, "--budget", "default"); doc.State != "updated" || doc.BudgetBytes != 6<<30 { + t.Errorf("default: %+v", doc) + } + if code, stdout, _ := e.run("setup", "--budget", "lots", "--json"); code != exitFailed || !strings.Contains(stdout, `"code": "InvalidArguments"`) { + t.Errorf("bad size: exit %d, %s", code, stdout) + } + }) + + t.Run("a proxy of siding v0.1", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(8080) // created by v0.1: no compose file of this siding + if doc := setupDoc(t, e); doc.State != "updated" || doc.Port != 8080 { + t.Errorf("%+v", doc) + } + }) + + t.Run("route to the overview", func(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + name string + local bool + alias string + }{ + {name: "daemon on this machine", local: true, alias: "172.19.0.1"}, + {name: "daemon in a VM", local: false, alias: "host-gateway"}, + } { + e := newEnv(t) + e.opts.IsLocalAddress = func(string) bool { return tt.local } + setupDoc(t, e) + data, err := os.ReadFile(filepath.Join(e.opts.Home, ".siding", "proxy", "siding", "compose.yml")) + if err != nil { + t.Fatal(err) + } + routes := filepath.Join(e.opts.Home, ".siding", "proxy", "siding", "routes") + for _, want := range []string{"--providers.file.directory=/etc/siding/routes", "--providers.file.watch=true", + `"host.docker.internal:` + tt.alias + `"`, `"` + routes + `:/etc/siding/routes:ro"`, `dev.siding.host-alias: "` + tt.alias + `"`} { + if !strings.Contains(string(data), want) { + t.Errorf("%s: the compose file lacks %s", tt.name, want) + } + } + if fi, err := os.Stat(routes); err != nil || !fi.IsDir() { + t.Errorf("%s: no routes directory: %v", tt.name, err) + } } }) diff --git a/internal/cli/skill/SKILL.md b/internal/cli/skill/SKILL.md index 2f6d9c6..c6f6924 100644 --- a/internal/cli/skill/SKILL.md +++ b/internal/cli/skill/SKILL.md @@ -31,6 +31,9 @@ Read its `verdict` and the exit code: listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady` means it is still starting. Read the hint and the logs; if it is only slow, run `siding status --wait --json --timeout 5m`. +- **3, `NoCapacity`**: the stack does not fit into the machine's memory budget; nothing was + started. Stop and tell the user, with the `message` of the `Error` document. Freeing memory + or raising the budget is the user's decision. An `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`, `message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`. diff --git a/internal/cli/skill_test.go b/internal/cli/skill_test.go index e94cd55..f938dd5 100644 --- a/internal/cli/skill_test.go +++ b/internal/cli/skill_test.go @@ -21,13 +21,13 @@ func TestSkill(t *testing.T) { if !strings.HasPrefix(text, "---\nname: siding\ndescription: ") { t.Errorf("not a skill named siding:\n%s", text[:min(len(text), 80)]) } - for _, want := range []string{"siding up --wait --json", "siding status --json", "siding logs --json", "**0, `Ready`**", "**1, `Failed`**", "**2, `TimedOut`**"} { + for _, want := range []string{"siding up --wait --json", "siding status --json", "siding logs --json", "**0, `Ready`**", "**1, `Failed`**", "**2, `TimedOut`**", "**3, `NoCapacity`**"} { if !strings.Contains(text, want) { t.Errorf("the skill does not mention %q", want) } } // Only a human removes data, forces anything, or exposes a stack. - for _, never := range []string{"--force", "--volumes", "share"} { + for _, never := range []string{"--force", "--volumes", "share", "gc"} { if strings.Contains(text, never) { t.Errorf("the skill mentions %q", never) } diff --git a/internal/cli/teardown.go b/internal/cli/teardown.go new file mode 100644 index 0000000..9c7e822 --- /dev/null +++ b/internal/cli/teardown.go @@ -0,0 +1,130 @@ +package cli + +import ( + "errors" + "fmt" + "io/fs" + "os" + "path/filepath" + "slices" + "sort" + "strings" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/stack" +) + +func newTeardownCommand(a *app) *cobra.Command { + var asJSON, dryRun bool + cmd := &cobra.Command{ + Use: "teardown", + Short: "Remove siding-proxy and its network; stacks keep running", + Long: `Remove what siding setup created: siding-proxy, its files under ~/.siding/proxy/, and the +network "siding" unless a stack still uses it. Stacks are never touched: they keep running as +plain Compose projects, without the proxy. Remove them first with siding down or siding gc.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + doc, err := a.teardown(cmd, dryRun) + if err != nil { + return err + } + if asJSON { + return printJSON(cmd.OutOrStdout(), doc) + } + return printTeardownText(cmd, doc) + }, + } + cmd.Flags().BoolVar(&dryRun, "dry-run", false, "list what would be removed, remove nothing") + registerJSON(cmd, &asJSON) + return cmd +} + +func (a *app) teardown(cmd *cobra.Command, dryRun bool) (teardownDocument, error) { + ctx := cmd.Context() + cfg := a.opts.Proxy + doc := teardownDocument{header: newHeader(kindTeardown), DryRun: dryRun} + if err := a.opts.Docker.Ping(ctx); err != nil { + return doc, err + } + proxyContainer, err := a.opts.Docker.Container(ctx, cfg.Container) + if err != nil { + return doc, err + } + if proxyContainer != nil { + doc.Container = cfg.Container + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelRepo: ""}) + if err != nil { + return doc, err + } + for _, c := range cs { + if p := c.Labels[stack.LabelProject]; !slices.Contains(doc.Stacks, p) { + doc.Stacks = append(doc.Stacks, p) + } + } + sort.Strings(doc.Stacks) + exists, users, err := a.opts.Docker.NetworkUsers(ctx, cfg.Network) + if err != nil { + return doc, err + } + users = slices.DeleteFunc(users, func(n string) bool { return n == cfg.Container }) + switch { + case exists && len(users) == 0: + doc.Network = cfg.Network + case exists: + doc.KeptNetwork = cfg.Network + } + if dryRun { + return doc, nil + } + + dir := filepath.Join(a.opts.Home, ".siding", "proxy", cfg.Project) + if doc.Container != "" { + // By project name: the compose file may be gone, or come from another siding. + inv := compose.Invocation{Dir: a.opts.Home, Project: cfg.Project} + if err := a.opts.Compose.Run(ctx, inv, []string{"down", "--remove-orphans"}, cmd.ErrOrStderr(), cmd.ErrOrStderr()); err != nil { + return doc, err + } + } + if err := os.RemoveAll(dir); err != nil && !errors.Is(err, fs.ErrNotExist) { + return doc, fmt.Errorf("remove %s: %w", dir, err) + } + // ~/.siding/proxy goes too once no other proxy (a test's) has files there. + _ = os.Remove(filepath.Dir(dir)) + if doc.Network != "" { + if err := a.opts.Docker.RemoveNetwork(ctx, doc.Network); err != nil { + return doc, err + } + } + return doc, nil +} + +func printTeardownText(cmd *cobra.Command, doc teardownDocument) error { + var b strings.Builder + verb := "Removed" + if doc.DryRun { + verb = "Would remove" + } + if doc.Container == "" && doc.Network == "" { + b.WriteString("Nothing to remove: siding-proxy does not run here.\n") + } + if doc.Container != "" { + fmt.Fprintf(&b, "%s container %s\n", verb, doc.Container) + } + if doc.Network != "" { + fmt.Fprintf(&b, "%s network %s\n", verb, doc.Network) + } + if doc.KeptNetwork != "" { + fmt.Fprintf(&b, "Kept network %s: stacks still use it\n", doc.KeptNetwork) + } + if len(doc.Stacks) > 0 { + fmt.Fprintf(&b, "These stacks keep running, without the proxy: %s\nRemove them with siding down in their worktree, or siding gc.\n", strings.Join(doc.Stacks, ", ")) + } + if doc.DryRun { + b.WriteString("Dry run: nothing was removed.\n") + } + _, err := fmt.Fprint(cmd.OutOrStdout(), b.String()) + return err +} diff --git a/internal/cli/teardown_test.go b/internal/cli/teardown_test.go new file mode 100644 index 0000000..71fe128 --- /dev/null +++ b/internal/cli/teardown_test.go @@ -0,0 +1,80 @@ +package cli + +import ( + "os" + "path/filepath" + "slices" + "strings" + "testing" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestTeardown(t *testing.T) { + t.Parallel() + + t.Run("nothing runs but the proxy", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + setupDoc(t, e) + machineID := filepath.Join(e.opts.Home, ".siding", "machine-id") + if err := os.WriteFile(machineID, []byte("0123456789abcdef\n"), 0o644); err != nil { + t.Fatal(err) + } + code, stdout, _ := e.run("teardown", "--json") + if code != exitOK { + t.Fatalf("exit %d: %s", code, stdout) + } + testutil.Golden(t, filepath.Join("testdata", "json", "teardown.json"), []byte(stdout)) + if e.docker.proxy != nil || !slices.Equal(e.docker.removedNetworks, []string{"siding"}) { + t.Errorf("proxy %v, removed networks %v", e.docker.proxy, e.docker.removedNetworks) + } + if _, err := os.Stat(filepath.Join(e.opts.Home, ".siding", "proxy")); !os.IsNotExist(err) { + t.Errorf("~/.siding/proxy is still there: %v", err) + } + if _, err := os.Stat(machineID); err != nil { + t.Errorf("the machine id is gone: %v", err) + } + }) + + t.Run("a stack still runs", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + setupDoc(t, e) + e.stackUp(e.id(), "web") + code, stdout, _ := e.run("teardown") + want := "Removed container siding-proxy\nKept network siding: stacks still use it\n" + + "These stacks keep running, without the proxy: shop-main\nRemove them with siding down in their worktree, or siding gc.\n" + if code != exitOK || stdout != want { + t.Errorf("exit %d, stdout:\n%s\nwant:\n%s", code, stdout, want) + } + if len(e.docker.removedNetworks) != 0 { + t.Errorf("removed networks %v", e.docker.removedNetworks) + } + for _, r := range e.compose.recorded() { + if r.inv.Project != "siding" { + t.Errorf("teardown touched the stack %s", r.inv.Project) + } + } + }) + + t.Run("dry run", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + setupDoc(t, e) + runs := len(e.compose.recorded()) + code, stdout, _ := e.run("teardown", "--dry-run", "--json") + if code != exitOK || len(e.compose.recorded()) != runs || e.docker.proxy == nil || len(e.docker.removedNetworks) != 0 { + t.Fatalf("exit %d, the dry run removed something", code) + } + testutil.Golden(t, filepath.Join("testdata", "json", "teardown-dryrun.json"), []byte(stdout)) + }) + + t.Run("no proxy", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + if code, stdout, _ := e.run("teardown"); code != exitOK || !strings.HasPrefix(stdout, "Nothing to remove") { + t.Errorf("exit %d, stdout %q", code, stdout) + } + }) +} diff --git a/internal/cli/testdata/json/gc-dryrun.json b/internal/cli/testdata/json/gc-dryrun.json new file mode 100644 index 0000000..16906d5 --- /dev/null +++ b/internal/cli/testdata/json/gc-dryrun.json @@ -0,0 +1,55 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "GC", + "dryRun": true, + "stacks": [ + { + "project": "shop-done", + "repo": "shop", + "branch": "done", + "worktree": "/src/shop-done", + "reason": "Merged", + "containers": [ + "shop-done-web-1" + ], + "networks": [ + "shop-done_default" + ], + "volumes": [ + "shop-done_data" + ] + }, + { + "project": "shop-old", + "repo": "shop", + "branch": "old", + "worktree": "/src/shop-old", + "reason": "WorktreeGone", + "containers": [ + "shop-old-web-1" + ], + "networks": [ + "shop-old_default" + ], + "volumes": [ + "shop-old_data" + ] + } + ], + "kept": [ + { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "reason": "DefaultBranch" + }, + { + "project": "shop-wip", + "repo": "shop", + "branch": "wip", + "worktree": "/src/shop-wip", + "reason": "Dirty" + } + ] +} diff --git a/internal/cli/testdata/json/gc.json b/internal/cli/testdata/json/gc.json new file mode 100644 index 0000000..8bd7912 --- /dev/null +++ b/internal/cli/testdata/json/gc.json @@ -0,0 +1,54 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "GC", + "stacks": [ + { + "project": "shop-done", + "repo": "shop", + "branch": "done", + "worktree": "/src/shop-done", + "reason": "Merged", + "containers": [ + "shop-done-web-1" + ], + "networks": [ + "shop-done_default" + ], + "volumes": [ + "shop-done_data" + ] + }, + { + "project": "shop-old", + "repo": "shop", + "branch": "old", + "worktree": "/src/shop-old", + "reason": "WorktreeGone", + "containers": [ + "shop-old-web-1" + ], + "networks": [ + "shop-old_default" + ], + "volumes": [ + "shop-old_data" + ] + } + ], + "kept": [ + { + "project": "shop-main", + "repo": "shop", + "branch": "main", + "worktree": "/src/shop", + "reason": "DefaultBranch" + }, + { + "project": "shop-wip", + "repo": "shop", + "branch": "wip", + "worktree": "/src/shop-wip", + "reason": "Dirty" + } + ] +} diff --git a/internal/cli/testdata/json/lintreport.json b/internal/cli/testdata/json/lintreport.json new file mode 100644 index 0000000..65ee1d4 --- /dev/null +++ b/internal/cli/testdata/json/lintreport.json @@ -0,0 +1,90 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "LintReport", + "file": "/src/shop/compose.yaml", + "findings": [ + { + "rule": "ContainerName", + "severity": "error", + "subject": "service db", + "message": "db sets container_name \"shop-db\", so a second stack of this file cannot start", + "hint": "remove container_name; siding names containers per stack" + }, + { + "rule": "FixedNetworkName", + "severity": "error", + "subject": "network shared", + "message": "the network shared has the fixed name \"everyone\", which every stack of this file would share", + "hint": "remove name:, or declare it external if stacks are meant to share it" + }, + { + "rule": "FixedVolumeName", + "severity": "error", + "subject": "volume pgdata", + "message": "the volume pgdata has the fixed name \"shop-pgdata\", so every stack of this file would share its data", + "hint": "remove name:, or label it dev.siding.shared: \"true\" if stacks are meant to share it" + }, + { + "rule": "HostPath", + "severity": "warning", + "subject": "service db", + "message": "db mounts /var/lib/shop from the host, and every branch's stack writes into the same directory", + "hint": "mount a path inside the worktree, or a named volume" + }, + { + "rule": "HostPath", + "severity": "warning", + "subject": "service db", + "message": "db mounts /src/shop/init from the host, and every branch's stack writes into the same directory", + "hint": "mount a path inside the worktree, or a named volume" + }, + { + "rule": "NoHealthcheck", + "severity": "warning", + "subject": "service db", + "message": "db has no healthcheck and no route path; the verdict only knows that it runs", + "hint": "add a healthcheck, so Ready means the service works, not only that it started" + }, + { + "rule": "NetworkMode", + "severity": "warning", + "subject": "service tools", + "message": "tools uses network_mode \"host\", so siding cannot route it", + "hint": "remove network_mode to make the service reachable through the proxy" + }, + { + "rule": "NoHealthcheck", + "severity": "warning", + "subject": "service tools", + "message": "tools has no healthcheck and no route path; the verdict only knows that it runs", + "hint": "add a healthcheck, so Ready means the service works, not only that it started" + }, + { + "rule": "NoHealthcheck", + "severity": "warning", + "subject": "service worker", + "message": "worker has no healthcheck and no route path; the verdict only knows that it runs", + "hint": "add a healthcheck, so Ready means the service works, not only that it started" + }, + { + "rule": "PublishedPort", + "severity": "info", + "subject": "service db", + "message": "db publishes 5432; siding resets published ports", + "hint": "reach it through its route, or with docker compose -p shop-main exec db" + }, + { + "rule": "FixedVolumeName", + "severity": "info", + "subject": "volume cache", + "message": "the volume cache has the fixed name \"pip-cache\" and is shared on purpose (dev.siding.shared)" + }, + { + "rule": "NoRoot", + "severity": "info", + "subject": "stack", + "message": "no service is the root of the stack's address; it answers 404 at /", + "hint": "give the main service dev.siding.route: \"/\"" + } + ] +} diff --git a/internal/cli/testdata/json/setup.json b/internal/cli/testdata/json/setup.json index 9c0f34c..5a9b417 100644 --- a/internal/cli/testdata/json/setup.json +++ b/internal/cli/testdata/json/setup.json @@ -6,5 +6,6 @@ "image": "traefik:v3.7.16", "port": 80, "url": "http://*.localhost/", - "state": "created" + "state": "created", + "budgetBytes": 12884901888 } diff --git a/internal/cli/testdata/json/skill.json b/internal/cli/testdata/json/skill.json index e5a492d..ed127e5 100644 --- a/internal/cli/testdata/json/skill.json +++ b/internal/cli/testdata/json/skill.json @@ -2,5 +2,5 @@ "apiVersion": "siding.dev/v1alpha1", "kind": "Skill", "name": "siding", - "text": "---\nname: siding\ndescription: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done.\n---\n\n# Check that the app runs with siding\n\nsiding runs this git worktree's compose file as its own stack, reachable through a local proxy\nat `http://..localhost`, and tells you whether it actually runs. Every command\nbelow prints one JSON document with `--json`.\n\n## After a change that affects the running app\n\nRun:\n\n```sh\nsiding up --wait --json\n```\n\nIt starts or updates the stack and waits, three minutes at most, for one `Status` document.\nRead its `verdict` and the exit code:\n\n- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service\n answers through the proxy. Report the address in `stack.url` (and a service's `url` if it\n matters) together with your result.\n- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`,\n `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the\n cause in that service, then run `siding up --wait --json` again.\n- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that\n was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it\n listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady`\n means it is still starting. Read the hint and the logs; if it is only slow, run\n `siding status --wait --json --timeout 5m`.\n\nAn `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`,\n`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`.\n\n## More detail\n\n- `siding status --json`: the current verdict, without waiting (exit 2 while starting).\n- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per\n service. Values from `.env` and the services' environment appear as `***`.\n- `siding ps --json`: every stack on this machine, with state and memory.\n\n## Rules\n\n- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file\n or the service labels `dev.siding.route` and `dev.siding.port` instead.\n- Leave stopping and removing stacks to the user; `siding down` only removes containers, and\n removing data is the user's decision.\n- Do not paste log lines into places outside this conversation.\n" + "text": "---\nname: siding\ndescription: Start this worktree's Docker Compose stack with siding and wait for the verdict, to check that a change to the app actually runs. Use after changing code, Dockerfiles, compose files or configuration that the running app depends on, and before reporting such work as done.\n---\n\n# Check that the app runs with siding\n\nsiding runs this git worktree's compose file as its own stack, reachable through a local proxy\nat `http://..localhost`, and tells you whether it actually runs. Every command\nbelow prints one JSON document with `--json`.\n\n## After a change that affects the running app\n\nRun:\n\n```sh\nsiding up --wait --json\n```\n\nIt starts or updates the stack and waits, three minutes at most, for one `Status` document.\nRead its `verdict` and the exit code:\n\n- **0, `Ready`**: every service runs, every healthcheck is healthy and every routed service\n answers through the proxy. Report the address in `stack.url` (and a service's `url` if it\n matters) together with your result.\n- **1, `Failed`**: `failure` names the `service`, the `reason` (`Exited`, `Unhealthy`,\n `Restarting`, `Missing`), a `hint` and the last log lines of that service in `logs`. Fix the\n cause in that service, then run `siding up --wait --json` again.\n- **2, `TimedOut`**: the stack was not ready in time. `failure` names the first service that\n was not ready: `NotAnswering` means it runs but does not answer through the proxy (often it\n listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady`\n means it is still starting. Read the hint and the logs; if it is only slow, run\n `siding status --wait --json --timeout 5m`.\n- **3, `NoCapacity`**: the stack does not fit into the machine's memory budget; nothing was\n started. Stop and tell the user, with the `message` of the `Error` document. Freeing memory\n or raising the budget is the user's decision.\n\nAn `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`,\n`message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`.\n\n## More detail\n\n- `siding status --json`: the current verdict, without waiting (exit 2 while starting).\n- `siding logs --json` or `siding logs --json --tail 200`: the last log lines per\n service. Values from `.env` and the services' environment appear as `***`.\n- `siding ps --json`: every stack on this machine, with state and memory.\n\n## Rules\n\n- Do not edit `.siding/override.yml`; siding rewrites it on every `up`. Change the compose file\n or the service labels `dev.siding.route` and `dev.siding.port` instead.\n- Leave stopping and removing stacks to the user; `siding down` only removes containers, and\n removing data is the user's decision.\n- Do not paste log lines into places outside this conversation.\n" } diff --git a/internal/cli/testdata/json/stacklist-empty.json b/internal/cli/testdata/json/stacklist-empty.json index e5eea1c..e536dcb 100644 --- a/internal/cli/testdata/json/stacklist-empty.json +++ b/internal/cli/testdata/json/stacklist-empty.json @@ -1,4 +1,5 @@ { "apiVersion": "siding.dev/v1alpha1", - "kind": "StackList" + "kind": "StackList", + "budgetBytes": 12884901888 } diff --git a/internal/cli/testdata/json/stacklist.json b/internal/cli/testdata/json/stacklist.json index 8028677..efe7450 100644 --- a/internal/cli/testdata/json/stacklist.json +++ b/internal/cli/testdata/json/stacklist.json @@ -25,5 +25,6 @@ "memoryBytes": 314572800 } ], - "memoryBytes": 838860800 + "memoryBytes": 838860800, + "budgetBytes": 12884901888 } diff --git a/internal/cli/testdata/json/teardown-dryrun.json b/internal/cli/testdata/json/teardown-dryrun.json new file mode 100644 index 0000000..685361a --- /dev/null +++ b/internal/cli/testdata/json/teardown-dryrun.json @@ -0,0 +1,7 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Teardown", + "dryRun": true, + "container": "siding-proxy", + "network": "siding" +} diff --git a/internal/cli/testdata/json/teardown.json b/internal/cli/testdata/json/teardown.json new file mode 100644 index 0000000..f7665df --- /dev/null +++ b/internal/cli/testdata/json/teardown.json @@ -0,0 +1,6 @@ +{ + "apiVersion": "siding.dev/v1alpha1", + "kind": "Teardown", + "container": "siding-proxy", + "network": "siding" +} diff --git a/internal/cli/testdata/skill/SKILL.md b/internal/cli/testdata/skill/SKILL.md index 2f6d9c6..c6f6924 100644 --- a/internal/cli/testdata/skill/SKILL.md +++ b/internal/cli/testdata/skill/SKILL.md @@ -31,6 +31,9 @@ Read its `verdict` and the exit code: listens on 127.0.0.1 instead of 0.0.0.0, or on another port than the one routed), `NotReady` means it is still starting. Read the hint and the logs; if it is only slow, run `siding status --wait --json --timeout 5m`. +- **3, `NoCapacity`**: the stack does not fit into the machine's memory budget; nothing was + started. Stop and tell the user, with the `message` of the `Error` document. Freeing memory + or raising the budget is the user's decision. An `Error` document (exit 1, `kind: Error`) means siding could not act at all; its `code`, `message` and `hint` say why. `ProxyNotRunning`: ask the user to run `siding setup`. diff --git a/internal/cli/testdata/ui/page.html b/internal/cli/testdata/ui/page.html new file mode 100644 index 0000000..ced5d3a --- /dev/null +++ b/internal/cli/testdata/ui/page.html @@ -0,0 +1,71 @@ + + + + + + +siding + + + +

siding

+

Every stack on this machine. Refreshes every 5 seconds; read-only, act with the siding CLI.

+ +

Memory

+

800 MiB of 12.0 GiB (6 %)

+
+ +

Stacks

+ + + + + + + + + + + + + + + + +
RepositoryBranchStateAddressMemoryWorktree
shopfeature/loginpartialhttp://shop.feature-login.localhost:8080/500 MiB/src/shop-login (missing)
shopmainrunninghttp://shop.main.localhost:8080/300 MiB/src/shop
+ + +

Can go

+ + + + + + + + + + + + + + +
StackBranchReasonRemoves
shop-donedoneMerged1 containers, 1 networks, 1 volumes
shop-oldoldWorktreeGone1 containers, 1 networks, 1 volumes
+

Remove them with siding gc.

+ + + diff --git a/internal/cli/ui.go b/internal/cli/ui.go new file mode 100644 index 0000000..c0209d7 --- /dev/null +++ b/internal/cli/ui.go @@ -0,0 +1,185 @@ +package cli + +import ( + "bytes" + "context" + _ "embed" + "errors" + "fmt" + "html/template" + "net" + "net/http" + "path/filepath" + "strconv" + "time" + + "github.com/spf13/cobra" + + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/overview" + "github.com/tweinmann/siding/internal/stack" +) + +//go:embed ui/page.html +var pageHTML string + +var pageTemplate = template.Must(template.New("page").Funcs(template.FuncMap{ + "bytes": bytesText, +}).Parse(pageHTML)) + +type pageData struct { + Stacks stackListDocument + GC gcDocument + Used, Budget string + // Percent is the memory in use against the budget; Bar is the same, at most 100. + Percent, Bar int +} + +func renderPage(stacks stackListDocument, gc gcDocument) ([]byte, error) { + d := pageData{Stacks: stacks, GC: gc, Used: bytesText(stacks.MemoryBytes), Budget: bytesText(uint64(stacks.BudgetBytes))} + if stacks.BudgetBytes > 0 { + d.Percent = int(stacks.MemoryBytes * 100 / uint64(stacks.BudgetBytes)) + } + d.Bar = min(d.Percent, 100) + var b bytes.Buffer + if err := pageTemplate.Execute(&b, d); err != nil { + return nil, err + } + return b.Bytes(), nil +} + +// overviewHandler serves the page and its JSON endpoints, all read-only: the page at /, and at +// /api/stacks and /api/gc the documents of ps --json and gc --dry-run --json. The mux answers +// other methods with 405 and other paths with 404. +func (a *app) overviewHandler() http.Handler { + mux := http.NewServeMux() + mux.HandleFunc("GET /{$}", func(w http.ResponseWriter, r *http.Request) { + stacks, err := a.ps(r.Context()) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + gc, err := a.gcDryRun(r.Context()) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + page, err := renderPage(stacks, gc) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + w.Header().Set("Content-Type", "text/html; charset=utf-8") + _, _ = w.Write(page) + }) + serveJSON := func(get func(context.Context) (any, error)) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + doc, err := get(r.Context()) + if err != nil { + http.Error(w, err.Error(), http.StatusInternalServerError) + return + } + w.Header().Set("Content-Type", "application/json") + _ = printJSON(w, doc) + } + } + mux.HandleFunc("GET /api/stacks", serveJSON(func(ctx context.Context) (any, error) { return a.ps(ctx) })) + mux.HandleFunc("GET /api/gc", serveJSON(func(ctx context.Context) (any, error) { return a.gcDryRun(ctx) })) + return mux +} + +// gcDryRun is the document of gc --dry-run --json. +func (a *app) gcDryRun(ctx context.Context) (gcDocument, error) { + doc, _, err := a.gcPlan(ctx) + doc.DryRun = true + return doc, err +} + +func newUICommand(a *app) *cobra.Command { + var asJSON bool + var port int + cmd := &cobra.Command{ + Use: "ui", + Short: "Serve the overview at http://siding.localhost while it runs", + Long: `Serve a read-only overview of every stack, the memory budget and what gc would remove, at +http://siding.localhost through siding-proxy, until Ctrl-C. It listens on this machine only and +removes its route when it stops; nothing of it keeps running.`, + Args: noArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + return a.ui(cmd, port, asJSON) + }, + } + cmd.Flags().IntVar(&port, "port", 0, "local port (default: a free one)") + registerJSON(cmd, &asJSON) + return cmd +} + +func (a *app) ui(cmd *cobra.Command, port int, asJSON bool) error { + ctx := cmd.Context() + if err := a.opts.Docker.Ping(ctx); err != nil { + return err + } + proxyPort, err := a.proxyPort(ctx) + if err != nil { + return err + } + proxyContainer, err := a.opts.Docker.Container(ctx, a.opts.Proxy.Container) + if err != nil { + return err + } + alias := proxyContainer.Labels[stack.LabelHostAlias] + if alias == "" { + return fail.New(fail.ProxyNotRunning, "%s has no route to the overview yet", a.opts.Proxy.Container). + WithHint("update it with siding setup") + } + + // The first address picks the port; the others take the same one. + var listeners []net.Listener + defer func() { + for _, l := range listeners { + _ = l.Close() + } + }() + for _, host := range overview.ListenAddrs(alias) { + l, err := net.Listen("tcp", net.JoinHostPort(host, strconv.Itoa(port))) + if err != nil { + return fail.New(fail.PortInUse, "siding ui cannot listen on %s:%d: %v", host, port, err). + WithHint("choose another port with --port, or run siding setup if the proxy network changed") + } + listeners = append(listeners, l) + port = l.Addr().(*net.TCPAddr).Port + } + + routes := filepath.Join(a.opts.Home, ".siding", "proxy", a.opts.Proxy.Project, "routes") + removeRoute, err := overview.WriteRoute(routes, port) + if err != nil { + return err + } + defer func() { _ = removeRoute() }() + + url := address(overview.Host, proxyPort, "/") + if asJSON { + if err := printJSON(cmd.OutOrStdout(), addressDocument{header: newHeader(kindAddress), URL: url}); err != nil { + return err + } + } else if _, err := fmt.Fprintf(cmd.OutOrStdout(), "siding ui at %s (and http://127.0.0.1:%d/); Ctrl-C stops it\n", url, port); err != nil { + return err + } + + srv := &http.Server{Handler: a.overviewHandler(), ReadHeaderTimeout: 10 * time.Second} + errs := make(chan error, len(listeners)) + for _, l := range listeners { + go func(l net.Listener) { errs <- srv.Serve(l) }(l) + } + select { + case <-ctx.Done(): + case err := <-errs: + if !errors.Is(err, http.ErrServerClosed) { + return fmt.Errorf("siding ui: %w", err) + } + } + shutdownCtx, cancel := context.WithTimeout(context.Background(), 2*time.Second) + defer cancel() + _ = srv.Shutdown(shutdownCtx) + return ctx.Err() +} diff --git a/internal/cli/ui/page.html b/internal/cli/ui/page.html new file mode 100644 index 0000000..64fc2c4 --- /dev/null +++ b/internal/cli/ui/page.html @@ -0,0 +1,64 @@ + + + + + + +siding + + + +

siding

+

Every stack on this machine. Refreshes every 5 seconds; read-only, act with the siding CLI.

+ +

Memory

+

{{.Used}} of {{.Budget}} ({{.Percent}} %)

+
+ +

Stacks

+{{if .Stacks.Stacks}} + + +{{range .Stacks.Stacks}} + + + + + +{{end}} +
RepositoryBranchStateAddressMemoryWorktree
{{.Repo}}{{.Branch}}{{.State}}{{.URL}}{{bytes .MemoryBytes}}{{.Worktree}}{{if .WorktreeMissing}} (missing){{end}}
+{{else}} +

No stacks. Start one with siding up in a worktree.

+{{end}} + +

Can go

+{{if .GC.Stacks}} + + +{{range .GC.Stacks}} + + + + +{{end}} +
StackBranchReasonRemoves
{{.Project}}{{.Branch}}{{.Reason}}{{len .Containers}} containers, {{len .Networks}} networks, {{len .Volumes}} volumes{{if .KeptVolumes}}; keeps {{len .KeptVolumes}} volumes{{end}}
+

Remove them with siding gc.

+{{else}} +

Nothing. Stacks of merged branches and deleted worktrees show up here.

+{{end}} + + diff --git a/internal/cli/ui_test.go b/internal/cli/ui_test.go new file mode 100644 index 0000000..5478952 --- /dev/null +++ b/internal/cli/ui_test.go @@ -0,0 +1,149 @@ +package cli + +import ( + "bytes" + "context" + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "regexp" + "strings" + "testing" + "time" + + "github.com/tweinmann/siding/internal/testutil" +) + +func TestOverviewHandler(t *testing.T) { + t.Parallel() + e := gcEnv(t) + e.proxyUp(80) + a := &app{opts: e.opts} + srv := httptest.NewServer(a.overviewHandler()) + t.Cleanup(srv.Close) + + get := func(method, path string) (int, string) { + t.Helper() + req, _ := http.NewRequest(method, srv.URL+path, nil) + resp, err := http.DefaultClient.Do(req) + if err != nil { + t.Fatal(err) + } + defer func() { _ = resp.Body.Close() }() + body, _ := io.ReadAll(resp.Body) + return resp.StatusCode, e.clean(string(body)) + } + + for path, args := range map[string][]string{"/api/stacks": {"ps", "--json"}, "/api/gc": {"gc", "--dry-run", "--json"}} { + code, body := get(http.MethodGet, path) + _, want, _ := e.run(args...) + if code != 200 || body != want { + t.Errorf("GET %s: %d\n%s\nwant the output of siding %s:\n%s", path, code, body, strings.Join(args, " "), want) + } + } + if code, body := get(http.MethodGet, "/"); code != 200 || !strings.Contains(body, "shop-done") || !strings.Contains(body, `http-equiv="refresh"`) { + t.Errorf("GET /: %d\n%s", code, body) + } + for _, m := range []string{http.MethodPost, http.MethodDelete} { + if code, _ := get(m, "/api/gc"); code != http.StatusMethodNotAllowed { + t.Errorf("%s /api/gc: %d, want 405", m, code) + } + } + if code, _ := get(http.MethodGet, "/nope"); code != http.StatusNotFound { + t.Errorf("GET /nope: %d", code) + } + if len(e.compose.recorded()) != 0 { + t.Error("the overview ran docker compose") + } +} + +// The page for the documents of the golden files stacklist.json and gc-dryrun.json. +func TestRenderPage(t *testing.T) { + t.Parallel() + var stacks stackListDocument + var gc gcDocument + for file, doc := range map[string]any{"stacklist.json": &stacks, "gc-dryrun.json": &gc} { + data, err := os.ReadFile(filepath.Join("testdata", "json", file)) + if err != nil { + t.Fatal(err) + } + if err := json.Unmarshal(data, doc); err != nil { + t.Fatal(err) + } + } + page, err := renderPage(stacks, gc) + if err != nil { + t.Fatal(err) + } + testutil.Golden(t, filepath.Join("testdata", "ui", "page.html"), page) + + empty, err := renderPage(stackListDocument{BudgetBytes: 1 << 30}, gcDocument{}) + if err != nil || !strings.Contains(string(empty), "No stacks.") || !strings.Contains(string(empty), "0 MiB of 1.0 GiB (0 %)") { + t.Errorf("empty page: %v\n%s", err, empty) + } +} + +func TestUI(t *testing.T) { + t.Parallel() + + t.Run("serves until cancelled", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.opts.IsLocalAddress = func(string) bool { return false } // loopback only, as with Docker Desktop + setupDoc(t, e) + route := filepath.Join(e.opts.Home, ".siding", "proxy", "siding", "routes", "ui.yml") + + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan int, 1) + var out, errOut bytes.Buffer + go func() { done <- Execute(ctx, New(e.opts), []string{"ui"}, &out, &errOut) }() + + var data []byte + for i := 0; i < 200 && len(data) == 0; i++ { + time.Sleep(10 * time.Millisecond) + data, _ = os.ReadFile(route) + } + m := regexp.MustCompile(`host\.docker\.internal:(\d+)`).FindSubmatch(data) + if m == nil { + cancel() + t.Fatalf("no route file: %q", data) + } + resp, err := http.Get("http://127.0.0.1:" + string(m[1]) + "/api/stacks") + if err != nil || resp.StatusCode != 200 { + cancel() + t.Fatalf("GET /api/stacks: %v %v", resp, err) + } + _ = resp.Body.Close() + + cancel() + if code := <-done; code != exitInterrupted || errOut.Len() != 0 { + t.Errorf("exit %d, stderr %q", code, errOut.String()) + } + if _, err := os.Stat(route); !os.IsNotExist(err) { + t.Errorf("the route survived: %v", err) + } + if !strings.Contains(out.String(), "siding ui at http://siding.localhost/ (and http://127.0.0.1:"+string(m[1])+"/)") { + t.Errorf("stdout %q", out.String()) + } + }) + + t.Run("a proxy of siding v0.1", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + e.proxyUp(80) + if code, _, stderr := e.run("ui"); code != exitFailed || !strings.Contains(stderr, "siding setup") { + t.Errorf("exit %d, stderr %q", code, stderr) + } + }) + + t.Run("no proxy", func(t *testing.T) { + t.Parallel() + e := newEnv(t) + if code, stdout, _ := e.run("ui", "--json"); code != exitFailed || !strings.Contains(stdout, `"code": "ProxyNotRunning"`) { + t.Errorf("exit %d, stdout %s", code, stdout) + } + }) +} diff --git a/internal/cli/up.go b/internal/cli/up.go index 0b4ac85..86678d3 100644 --- a/internal/cli/up.go +++ b/internal/cli/up.go @@ -1,14 +1,19 @@ package cli import ( + "context" "fmt" + "io" "os" "path/filepath" + "slices" "strings" "github.com/spf13/cobra" + "github.com/tweinmann/siding/internal/budget" "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/docker" "github.com/tweinmann/siding/internal/fail" "github.com/tweinmann/siding/internal/stack" ) @@ -46,6 +51,7 @@ With --wait, up ends like siding status --wait, with its verdict and exit code.` cmd.Flags().StringVarP(&f.file, "file", "f", "", "compose file (default: compose.yaml or another name Compose looks for)") cmd.Flags().StringVar(&f.repo, "repo", "", "repository name (default: the main worktree's directory name)") cmd.Flags().StringVar(&f.agent, "agent", "", "agent name for the label dev.siding.agent (default: $SIDING_AGENT)") + cmd.Flags().BoolVar(&f.force, "force", false, "start the stack even if it does not fit into the memory budget") w.register(cmd) registerJSON(cmd, &asJSON) return cmd @@ -53,9 +59,48 @@ With --wait, up ends like siding status --wait, with its verdict and exit code.` type upFlags struct { file, repo, agent string + force bool compose []string } +// checkBudget refuses with NoCapacity when the memory of every other stack plus the estimate of +// this one exceeds the budget; with force it only warns. +func (a *app) checkBudget(ctx context.Context, project string, p *compose.Project, force bool, stderr io.Writer) error { + proxyContainer, err := a.opts.Docker.Container(ctx, a.opts.Proxy.Container) + if err != nil { + return err + } + b, err := a.budget(ctx, proxyContainer) + if err != nil { + return err + } + cs, err := a.opts.Docker.Containers(ctx, map[string]string{stack.LabelRepo: ""}) + if err != nil { + return err + } + // This stack's current use does not count: its estimate replaces it. + others := slices.DeleteFunc(cs, func(c docker.Container) bool { return c.Labels[stack.LabelProject] == project }) + memory, err := a.memory(ctx, others) + if err != nil { + return err + } + check := budget.Check{Budget: b, Estimate: budget.Estimate(p)} + for _, m := range memory { + check.Used += int64(m) + } + if check.Fits() { + return nil + } + msg := fmt.Sprintf("%s needs about %s, and the other stacks use %s of the memory budget %s", + project, bytesText(uint64(check.Estimate)), bytesText(uint64(check.Used)), bytesText(uint64(check.Budget))) + if force { + warnf(stderr, "%s; starting it anyway (--force)", msg) + return nil + } + return fail.New(fail.NoCapacity, "%s", msg). + WithHint("free memory with siding gc or siding down in another worktree, raise it with siding setup --budget, or start anyway with --force") +} + func (a *app) up(cmd *cobra.Command, f upFlags, w waitFlags, asJSON bool) error { ctx := cmd.Context() stderr := cmd.ErrOrStderr() @@ -71,7 +116,7 @@ func (a *app) up(cmd *cobra.Command, f upFlags, w waitFlags, asJSON bool) error if repo == "" { repo = wt.RepoName() } - id := stack.Identity{Repo: repo, Branch: wt.Branch, Worktree: wt.Top, Machine: machine, Agent: f.agent} + id := stack.Identity{Repo: repo, Branch: wt.Branch, Worktree: wt.Top, Machine: machine, Agent: f.agent, GitDir: wt.CommonDir} project := id.Project() if err := a.opts.Docker.Ping(ctx); err != nil { @@ -108,6 +153,9 @@ func (a *app) up(cmd *cobra.Command, f upFlags, w waitFlags, asJSON bool) error for _, w := range warnings { warnf(stderr, "%s", w) } + if err := a.checkBudget(ctx, project, p, f.force, stderr); err != nil { + return err + } override, err := writeOverride(wt.Top, stack.Override(p, id, routes, a.opts.Proxy.Network)) if err != nil { return err diff --git a/internal/cli/up_test.go b/internal/cli/up_test.go index abe7133..3dd415f 100644 --- a/internal/cli/up_test.go +++ b/internal/cli/up_test.go @@ -1,6 +1,7 @@ package cli import ( + "fmt" "os" "os/exec" "path/filepath" @@ -161,3 +162,63 @@ Wait for the verdict with: siding status --wait } }) } + +func TestUpBudget(t *testing.T) { + t.Parallel() + const mib = 1 << 20 + // testConfig has four services without limits: an estimate of 1 GiB. + budgetEnv := func(t *testing.T, budgetMiB uint64) *env { + e := newEnv(t) + e.proxyUp(80) + e.docker.proxy.Labels["dev.siding.budget"] = fmt.Sprint(budgetMiB * mib) + other := e.id() + other.Branch, other.Worktree = "feature/other", "/src/other" + e.stackUp(other, "web") + e.docker.memory["shop-feature-other-web-1"] = 3584 * mib + return e + } + + t.Run("over budget", func(t *testing.T) { + t.Parallel() + e := budgetEnv(t, 4096) + code, stdout, _ := e.run("up", "--json") + if code != exitNoCapacity || !strings.Contains(stdout, `"code": "NoCapacity"`) || len(e.compose.recorded()) != 0 { + t.Fatalf("exit %d, compose runs %d, stdout %s", code, len(e.compose.recorded()), stdout) + } + for _, want := range []string{"1.0 GiB", "3.5 GiB", "4.0 GiB", "siding gc", "--budget", "--force"} { + if !strings.Contains(stdout, want) { + t.Errorf("the error does not name %q: %s", want, stdout) + } + } + if _, err := os.Stat(filepath.Join(e.repo, ".siding")); err == nil { + t.Error("up wrote the override although it refused") + } + }) + + t.Run("forced", func(t *testing.T) { + t.Parallel() + e := budgetEnv(t, 4096) + code, _, stderr := e.run("up", "--force") + if code != exitOK || !strings.Contains(stderr, "warning:") || !strings.Contains(stderr, "--force") || len(e.compose.recorded()) != 1 { + t.Errorf("exit %d, stderr %q", code, stderr) + } + }) + + t.Run("within budget", func(t *testing.T) { + t.Parallel() + e := budgetEnv(t, 8192) + if code, _, stderr := e.run("up"); code != exitOK || stderr != "" { + t.Errorf("exit %d, stderr %q", code, stderr) + } + }) + + t.Run("the stack itself does not count twice", func(t *testing.T) { + t.Parallel() + e := budgetEnv(t, 4608) // 3584 + 1024 fits exactly + e.stackUp(e.id(), "web") + e.docker.memory["shop-main-web-1"] = 900 * mib + if code, stdout, _ := e.run("up", "--json"); code != exitOK { + t.Errorf("exit %d: %s", code, stdout) + } + }) +} diff --git a/internal/compose/compose.go b/internal/compose/compose.go index 043092c..7961e1b 100644 --- a/internal/compose/compose.go +++ b/internal/compose/compose.go @@ -7,6 +7,7 @@ import ( "fmt" "io" "os/exec" + "regexp" "slices" "strings" "sync" @@ -128,13 +129,45 @@ func composeError(what, output string, err error) error { if !errors.As(err, &exit) { return fail.New(fail.ComposeFailed, "%s: %v", what, err) } - output = strings.TrimSpace(output) + output = errorLines(output) if output == "" { return fail.New(fail.ComposeFailed, "%s failed with exit code %d", what, exit.ExitCode()) } return fail.New(fail.ComposeFailed, "%s failed: %s", what, output) } +// progress matches Compose's progress lines, such as "Container shop-db-1 Creating". +var progress = regexp.MustCompile(`^(Container|Network|Volume|Image)\s+\S+\s+\S+\s*$`) + +// errorLines keeps what explains a failure in Compose's stderr: no progress lines, the lines +// that say "error" when there are any, each message once. +func errorLines(output string) string { + var lines, errs []string + seen := map[string]bool{} + for _, l := range strings.Split(output, "\n") { + l = strings.TrimSpace(l) + if l == "" || progress.MatchString(l) { + continue + } + // Compose repeats a service's error with a "service:: " prefix. + if _, rest, ok := strings.Cut(l, " "); ok && strings.HasPrefix(l, "service:") { + l = rest + } + if seen[l] { + continue + } + seen[l] = true + lines = append(lines, l) + if strings.Contains(strings.ToLower(l), "error") { + errs = append(errs, l) + } + } + if len(errs) > 0 { + lines = errs + } + return strings.Join(lines, "\n") +} + // tailBuffer keeps the last max lines written to it. type tailBuffer struct { max int diff --git a/internal/compose/config.go b/internal/compose/config.go index 753f6b5..cb5a579 100644 --- a/internal/compose/config.go +++ b/internal/compose/config.go @@ -62,17 +62,31 @@ func (p *Project) ServiceNames() []string { // Service is one service of a Project. type Service struct { // Ports are the distinct container ports from ports (targets) and expose, ascending. - Ports []int + Ports []int + // Published are the ports the compose file publishes on the host. + Published []int Labels map[string]string Networks []string // keys of the service's networks, sorted NetworkMode string Environment map[string]string + + ContainerName string + // Binds are the absolute host paths of the service's bind mounts. + Binds []string + // Healthcheck is set when the compose file gives the service a healthcheck. One from the + // image is not visible here. + Healthcheck bool + // MemoryLimit is deploy.resources.limits.memory, else mem_limit, in bytes; 0 for none. + MemoryLimit int64 + // Replicas is deploy.replicas, else scale, else 1. + Replicas int } // Resource is a top-level network or volume. type Resource struct { Name string External bool + Labels map[string]string } type rawProject struct { @@ -84,9 +98,29 @@ type rawProject struct { type rawService struct { Ports []struct { - Target int `json:"target"` - Protocol string `json:"protocol"` + Target int `json:"target"` + Published string `json:"published"` + Protocol string `json:"protocol"` } `json:"ports"` + ContainerName string `json:"container_name"` + Volumes []struct { + Type string `json:"type"` + Source string `json:"source"` + } `json:"volumes"` + Healthcheck *struct { + Test []string `json:"test"` + Disable bool `json:"disable"` + } `json:"healthcheck"` + MemLimit string `json:"mem_limit"` + Scale *int `json:"scale"` + Deploy *struct { + Replicas *int `json:"replicas"` + Resources struct { + Limits struct { + Memory string `json:"memory"` + } `json:"limits"` + } `json:"resources"` + } `json:"deploy"` Expose []json.RawMessage `json:"expose"` Labels map[string]string `json:"labels"` Networks map[string]json.RawMessage `json:"networks"` @@ -95,8 +129,9 @@ type rawService struct { } type rawResource struct { - Name string `json:"name"` - External bool `json:"external"` + Name string `json:"name"` + External bool `json:"external"` + Labels map[string]string `json:"labels"` } // ParseConfig reads the output of docker compose config --format json. @@ -107,7 +142,8 @@ func ParseConfig(data []byte) (*Project, error) { } p := &Project{Name: raw.Name, Services: map[string]Service{}, Networks: map[string]Resource{}, Volumes: map[string]Resource{}} for name, rs := range raw.Services { - s := Service{Labels: rs.Labels, NetworkMode: rs.NetworkMode, Environment: map[string]string{}} + s := Service{Labels: rs.Labels, NetworkMode: rs.NetworkMode, Environment: map[string]string{}, + ContainerName: rs.ContainerName, Replicas: 1} if s.Labels == nil { s.Labels = map[string]string{} } @@ -116,6 +152,34 @@ func ParseConfig(data []byte) (*Project, error) { if port.Protocol == "" || port.Protocol == "tcp" { ports[port.Target] = true } + if n, err := strconv.Atoi(port.Published); err == nil { + s.Published = append(s.Published, n) + } + } + for _, v := range rs.Volumes { + if v.Type == "bind" { + s.Binds = append(s.Binds, v.Source) + } + } + if h := rs.Healthcheck; h != nil && !h.Disable && len(h.Test) > 0 && h.Test[0] != "NONE" { + s.Healthcheck = true + } + limit := rs.MemLimit + if rs.Deploy != nil && rs.Deploy.Resources.Limits.Memory != "" { + limit = rs.Deploy.Resources.Limits.Memory + } + if limit != "" { + n, err := strconv.ParseInt(limit, 10, 64) + if err != nil { + return nil, fmt.Errorf("service %s: memory limit %q", name, limit) + } + s.MemoryLimit = n + } + switch { + case rs.Deploy != nil && rs.Deploy.Replicas != nil: + s.Replicas = *rs.Deploy.Replicas + case rs.Scale != nil: + s.Replicas = *rs.Scale } for _, e := range rs.Expose { expanded, err := parseExpose(e) @@ -146,14 +210,22 @@ func ParseConfig(data []byte) (*Project, error) { p.Services[name] = s } for k, r := range raw.Networks { - p.Networks[k] = Resource(r) + p.Networks[k] = resource(r) } for k, r := range raw.Volumes { - p.Volumes[k] = Resource(r) + p.Volumes[k] = resource(r) } return p, nil } +func resource(r rawResource) Resource { + labels := r.Labels + if labels == nil { + labels = map[string]string{} + } + return Resource{Name: r.Name, External: r.External, Labels: labels} +} + // parseExpose expands one expose entry ("3000", "3000/tcp", "9000-9001", or a number) into // TCP ports. func parseExpose(raw json.RawMessage) ([]int, error) { diff --git a/internal/compose/config_test.go b/internal/compose/config_test.go index 68152f1..073081c 100644 --- a/internal/compose/config_test.go +++ b/internal/compose/config_test.go @@ -5,6 +5,7 @@ import ( "os" "path/filepath" "reflect" + "strings" "testing" "github.com/tweinmann/siding/internal/fail" @@ -124,3 +125,67 @@ func codeOf(err error) string { } return "" } + +// testdata/lint/config.json is captured like testdata/routing/config.json, from +// testdata/lint/compose.yaml, with the capturing directory replaced by /src/shop. +func TestParseConfigDetails(t *testing.T) { + t.Parallel() + data, err := os.ReadFile(filepath.Join("testdata", "lint", "config.json")) + if err != nil { + t.Fatal(err) + } + p, err := ParseConfig(data) + if err != nil { + t.Fatal(err) + } + const mib = 1 << 20 + tests := []struct { + name string + containerName string + binds []string + published []int + healthcheck bool + memoryLimit int64 + replicas int + }{ + {name: "web", healthcheck: true, memoryLimit: 512 * mib, replicas: 2}, + {name: "db", containerName: "shop-db", binds: []string{"/var/lib/shop", "/src/shop/init"}, published: []int{5432}, memoryLimit: 1024 * mib, replicas: 1}, + {name: "worker", replicas: 3}, + {name: "tools", replicas: 1}, + } + for _, want := range tests { + s := p.Services[want.name] + if s.ContainerName != want.containerName || !reflect.DeepEqual(s.Binds, want.binds) || !reflect.DeepEqual(s.Published, want.published) || + s.Healthcheck != want.healthcheck || s.MemoryLimit != want.memoryLimit || s.Replicas != want.replicas { + t.Errorf("%s: %+v\nwant %+v", want.name, s, want) + } + } + if got := p.Volumes["cache"]; got.Name != "pip-cache" || got.Labels["dev.siding.shared"] != "true" { + t.Errorf("volume cache %+v", got) + } + if got := p.Volumes["pgdata"]; got.Name != "shop-pgdata" || len(got.Labels) != 0 { + t.Errorf("volume pgdata %+v", got) + } + if got := p.Networks["shared"]; got.Name != "everyone" { + t.Errorf("network shared %+v", got) + } +} + +func TestErrorLines(t *testing.T) { + t.Parallel() + out := ` Container demo-db Creating + service:db:1 Error response from daemon: Conflict. The container name "/demo-db" is already in use. +Error response from daemon: Conflict. The container name "/demo-db" is already in use. +` + if got, want := errorLines(out), `Error response from daemon: Conflict. The container name "/demo-db" is already in use.`; got != want { + t.Errorf("got %q, want %q", got, want) + } + pull := " traefik Pulling \n traefik Error Get \"https://registry-1.docker.io/v2/\": net/http: TLS handshake timeout\n" + + "Error response from daemon: Get \"https://registry-1.docker.io/v2/\": net/http: TLS handshake timeout\n" + if got := errorLines(pull); strings.Contains(got, "Pulling") || !strings.Contains(got, "TLS handshake timeout") { + t.Errorf("a failed pull: %q", got) + } + if got := errorLines(" Network x_default Creating\nno such service: nope\n"); got != "no such service: nope" { + t.Errorf("without an error line: %q", got) + } +} diff --git a/internal/compose/testdata/lint/compose.yaml b/internal/compose/testdata/lint/compose.yaml new file mode 100644 index 0000000..f005d99 --- /dev/null +++ b/internal/compose/testdata/lint/compose.yaml @@ -0,0 +1,35 @@ +# Triggers every lint rule except the routing errors, and every field the parser reads. +services: + web: + image: busybox:1.37.0 + expose: ["3000"] + labels: {dev.siding.route: "/app"} + deploy: + replicas: 2 + resources: {limits: {memory: 512m}} + healthcheck: {test: ["CMD", "true"]} + db: + image: busybox:1.37.0 + container_name: shop-db + ports: ["5432:5432"] + mem_limit: 1g + volumes: + - /var/lib/shop:/data + - ./init:/docker-entrypoint-initdb.d + - pgdata:/pgdata + - cache:/cache + healthcheck: {test: ["NONE"]} + worker: + image: busybox:1.37.0 + scale: 3 + networks: [shared] + tools: + image: busybox:1.37.0 + network_mode: host +volumes: + pgdata: {name: shop-pgdata} + cache: + name: pip-cache + labels: {dev.siding.shared: "true"} +networks: + shared: {name: everyone} diff --git a/internal/compose/testdata/lint/config.json b/internal/compose/testdata/lint/config.json new file mode 100644 index 0000000..d8b1858 --- /dev/null +++ b/internal/compose/testdata/lint/config.json @@ -0,0 +1,119 @@ +{ + "name": "shop-main", + "networks": { + "default": { + "name": "shop-main_default", + "ipam": {} + }, + "shared": { + "name": "everyone", + "ipam": {} + } + }, + "services": { + "db": { + "container_name": "shop-db", + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "mem_limit": "1073741824", + "networks": { + "default": null + }, + "volumes": [ + { + "type": "bind", + "source": "/var/lib/shop", + "target": "/data", + "bind": {} + }, + { + "type": "bind", + "source": "/src/shop/init", + "target": "/docker-entrypoint-initdb.d", + "bind": {} + }, + { + "type": "volume", + "source": "pgdata", + "target": "/pgdata", + "volume": {} + }, + { + "type": "volume", + "source": "cache", + "target": "/cache", + "volume": {} + } + ], + "healthcheck": { + "test": [ + "NONE" + ] + }, + "ports": [ + { + "mode": "ingress", + "target": 5432, + "published": "5432", + "protocol": "tcp" + } + ] + }, + "tools": { + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "network_mode": "host" + }, + "web": { + "deploy": { + "replicas": 2, + "resources": { + "limits": { + "memory": "536870912" + } + }, + "placement": {} + }, + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "labels": { + "dev.siding.route": "/app" + }, + "networks": { + "default": null + }, + "expose": [ + "3000" + ], + "healthcheck": { + "test": [ + "CMD", + "true" + ] + } + }, + "worker": { + "scale": 3, + "command": null, + "entrypoint": null, + "image": "busybox:1.37.0", + "networks": { + "shared": null + } + } + }, + "volumes": { + "cache": { + "name": "pip-cache", + "labels": { + "dev.siding.shared": "true" + } + }, + "pgdata": { + "name": "shop-pgdata" + } + } +} \ No newline at end of file diff --git a/internal/docker/docker.go b/internal/docker/docker.go index 26b192d..a8ae065 100644 --- a/internal/docker/docker.go +++ b/internal/docker/docker.go @@ -116,6 +116,19 @@ func labelFilters(labels map[string]string) client.Filters { return f } +// MemTotal returns the memory the Docker daemon has, the VM's on Docker Desktop. +func (c *Client) MemTotal(ctx context.Context) (int64, error) { + api, err := c.connect() + if err != nil { + return 0, err + } + res, err := api.Info(ctx, client.InfoOptions{}) + if err != nil { + return 0, unavailable(err) + } + return res.Info.MemTotal, nil +} + // Containers returns every container, running or not, that carries all the given labels; an // empty value matches any value. func (c *Client) Containers(ctx context.Context, labels map[string]string) ([]Container, error) { @@ -237,6 +250,56 @@ func (c *Client) EnsureNetwork(ctx context.Context, name string, labels map[stri return true, nil } +// NetworkGateway returns the IPv4 gateway of a network, "" when it has none. +func (c *Client) NetworkGateway(ctx context.Context, name string) (string, error) { + api, err := c.connect() + if err != nil { + return "", err + } + res, err := api.NetworkInspect(ctx, name, client.NetworkInspectOptions{}) + if err != nil { + return "", fmt.Errorf("inspect network %s: %w", name, err) + } + for _, cfg := range res.Network.IPAM.Config { + if cfg.Gateway.Is4() { + return cfg.Gateway.String(), nil + } + } + return "", nil +} + +// NetworkUsers returns the names of the containers attached to a network, and false when the +// network does not exist. +func (c *Client) NetworkUsers(ctx context.Context, name string) (exists bool, users []string, err error) { + api, err := c.connect() + if err != nil { + return false, nil, err + } + res, err := api.NetworkInspect(ctx, name, client.NetworkInspectOptions{}) + if cerrdefs.IsNotFound(err) { + return false, nil, nil + } + if err != nil { + return false, nil, unavailable(err) + } + for _, e := range res.Network.Containers { + users = append(users, e.Name) + } + return true, users, nil +} + +// RemoveNetwork removes a network by its exact name. +func (c *Client) RemoveNetwork(ctx context.Context, name string) error { + api, err := c.connect() + if err != nil { + return err + } + if _, err := api.NetworkRemove(ctx, name, client.NetworkRemoveOptions{}); err != nil { + return fmt.Errorf("remove network %s: %w", name, err) + } + return nil +} + // Volumes returns the volumes that carry all the given labels. func (c *Client) Volumes(ctx context.Context, labels map[string]string) ([]Volume, error) { api, err := c.connect() diff --git a/internal/e2e/e2e_test.go b/internal/e2e/e2e_test.go index 3a28e4c..03db8c0 100644 --- a/internal/e2e/e2e_test.go +++ b/internal/e2e/e2e_test.go @@ -23,6 +23,7 @@ import ( "github.com/tweinmann/siding/internal/compose" "github.com/tweinmann/siding/internal/docker" "github.com/tweinmann/siding/internal/git" + "github.com/tweinmann/siding/internal/overview" "github.com/tweinmann/siding/internal/probe" "github.com/tweinmann/siding/internal/proxy" "github.com/tweinmann/siding/internal/stack" @@ -93,14 +94,22 @@ func newMachine(t *testing.T) *machine { // siding runs one command in dir and returns exit code and stdout; stderr goes to the log. func (m *machine) siding(dir string, args ...string) (int, string) { m.t.Helper() + return m.sidingCtx(context.Background(), dir, args...) +} + +func (m *machine) sidingCtx(ctx context.Context, dir string, args ...string) (int, string) { + m.t.Helper() + // A machine id of its own: gc acts on every stack of "this machine", and must only ever see + // the tests' stacks, never one a developer started in the same daemon. opts := cli.Options{ - Dir: dir, Home: m.home, Getenv: os.Getenv, OSMachineID: stack.OSMachineID, + Dir: dir, Home: m.home, Getenv: os.Getenv, OSMachineID: func() string { return "siding-test-e2e" }, + IsLocalAddress: overview.IsLocalAddress, Stdin: strings.NewReader(""), IsTerminal: func() bool { return false }, Git: git.Client{}, Compose: &compose.CLI{}, Docker: &docker.Client{}, Probe: probe.HTTP{Header: stack.ServiceHeader}, Browser: noBrowser{}, Clock: cli.SystemClock{}, Proxy: testProxy, } var out, errOut bytes.Buffer - code := cli.Execute(context.Background(), cli.New(opts), args, &out, &errOut) + code := cli.Execute(ctx, cli.New(opts), args, &out, &errOut) m.t.Logf("siding %s: exit %d\n%s", strings.Join(args, " "), code, errOut.String()) return code, out.String() } @@ -306,4 +315,132 @@ func TestStacks(t *testing.T) { t.Errorf("the volume survived --force: %q", out) } }) + + t.Run("down --volumes after a squash merge", func(t *testing.T) { + squash := repo(t, "squash", shopCompose, "", "feature/s") + origin := filepath.Join(t.TempDir(), "origin.git") + gitIn(t, squash, "init", "-q", "--bare", origin) + gitIn(t, squash, "remote", "add", "origin", origin) + gitIn(t, squash, "push", "-q", "-u", "origin", "feature/s") + gitIn(t, squash, "push", "-q", "origin", "--delete", "feature/s") + if code, out := m.siding(squash, "up", "--repo", "siding-test-squash", "--wait", "--json"); code != 0 { + t.Fatalf("up: exit %d\n%s", code, out) + } + if code, out := m.siding(squash, "down", "--volumes", "--json"); code != 0 || !strings.Contains(out, "siding-test-squash-feature-s_data") { + t.Errorf("exit %d\n%s", code, out) + } + }) + + t.Run("budget", func(t *testing.T) { + budgeted := repo(t, "budget", shopCompose, "", "feature/b") + // The stack's estimate is 512 MiB (two services without limits). + if code, out := m.siding(t.TempDir(), "setup", "--budget", "64m", "--json"); code != 0 { + t.Fatalf("setup --budget: exit %d\n%s", code, out) + } + code, out := m.siding(budgeted, "up", "--repo", "siding-test-budget", "--wait", "--json") + if code != 3 || decode(t, out).Code != "NoCapacity" { + t.Errorf("over budget: exit %d\n%s", code, out) + } + if out := run(t, "", "docker", "ps", "-aq", "--filter", "label=com.docker.compose.project=siding-test-budget-feature-b"); strings.TrimSpace(out) != "" { + t.Errorf("up started containers although it refused: %q", out) + } + if code, out := m.siding(budgeted, "up", "--repo", "siding-test-budget", "--force", "--wait", "--json"); code != 0 { + t.Errorf("--force: exit %d\n%s", code, out) + } + if code, out := m.siding(t.TempDir(), "setup", "--budget", "default", "--json"); code != 0 { + t.Fatalf("setup --budget default: exit %d\n%s", code, out) + } + }) + + t.Run("gc", func(t *testing.T) { + merged := repo(t, "gcm", shopCompose, "", "feature/m") + unmerged := repo(t, "gcu", shopCompose, "", "feature/u") + for dir, name := range map[string]string{merged: "siding-test-gcm", unmerged: "siding-test-gcu"} { + if code, out := m.siding(dir, "up", "--repo", name, "--wait", "--json"); code != 0 { + t.Fatalf("up %s: exit %d\n%s", name, code, out) + } + } + // Merge feature/m into main, move main on, and go back: a clean, merged worktree. + gitIn(t, merged, "switch", "-q", "main") + gitIn(t, merged, "merge", "-q", "--ff-only", "feature/m") + gitIn(t, merged, "commit", "-q", "--allow-empty", "-m", "after the merge") + gitIn(t, merged, "switch", "-q", "feature/m") + + code, out := m.siding(t.TempDir(), "gc", "--dry-run", "--json") + if code != 0 || !strings.Contains(out, `"project": "siding-test-gcm-feature-m"`) || !strings.Contains(out, `"reason": "Merged"`) { + t.Fatalf("dry run: exit %d\n%s", code, out) + } + if !strings.Contains(out, `"project": "siding-test-gcu-feature-u"`) || !strings.Contains(out, `"reason": "Unmerged"`) { + t.Errorf("the unmerged stack is not kept: %s", out) + } + if code, out := m.siding(t.TempDir(), "gc", "--json"); code != 1 || decode(t, out).Code != "NotInteractive" { + t.Errorf("without --yes: exit %d\n%s", code, out) + } + if code, out := m.siding(t.TempDir(), "gc", "--yes", "--json"); code != 0 || !strings.Contains(out, "siding-test-gcm-feature-m_data") { + t.Fatalf("--yes: exit %d\n%s", code, out) + } + if out := run(t, "", "docker", "volume", "ls", "-q", "--filter", "name=siding-test-gcm-feature-m_data"); strings.TrimSpace(out) != "" { + t.Errorf("the merged stack's volume survived: %q", out) + } + if _, err := os.Stat(merged); err != nil { + t.Errorf("gc removed the worktree: %v", err) + } + if out := run(t, "", "docker", "ps", "-q", "--filter", "label=com.docker.compose.project=siding-test-gcu-feature-u"); strings.TrimSpace(out) == "" { + t.Error("gc removed the unmerged stack") + } + }) + + t.Run("ui", func(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + done := make(chan int, 1) + go func() { + code, _ := m.sidingCtx(ctx, t.TempDir(), "ui") + done <- code + }() + var code int + for i := 0; i < 40 && code != 200; i++ { + time.Sleep(250 * time.Millisecond) + code, _, _ = get(t, "siding.localhost", "/api/stacks") + } + if code != 200 { + t.Errorf("siding.localhost through the proxy: %d", code) + } + cancel() + if got := <-done; got != 130 { + t.Errorf("ui ended with %d, want 130", got) + } + for i := 0; i < 40 && code != 404; i++ { + time.Sleep(250 * time.Millisecond) + code, _, _ = get(t, "siding.localhost", "/") + } + if code != 404 { + t.Errorf("after ui stopped: %d, want 404", code) + } + }) + + t.Run("teardown", func(t *testing.T) { + kept := repo(t, "keep", shopCompose, "", "feature/k") + if code, out := m.siding(kept, "up", "--repo", "siding-test-keep", "--wait", "--json"); code != 0 { + t.Fatalf("up: exit %d\n%s", code, out) + } + code, out := m.siding(t.TempDir(), "teardown", "--json") + if code != 0 || !strings.Contains(out, `"keptNetwork": "siding-test-e2e"`) || !strings.Contains(out, "siding-test-keep-feature-k") { + t.Fatalf("with a stack: exit %d\n%s", code, out) + } + if out := run(t, "", "docker", "ps", "-aq", "--filter", "name=^siding-test-e2e-proxy$"); strings.TrimSpace(out) != "" { + t.Errorf("the proxy survived: %q", out) + } + if out := run(t, "", "docker", "ps", "-q", "--filter", "label=com.docker.compose.project=siding-test-keep-feature-k"); strings.TrimSpace(out) == "" { + t.Error("teardown stopped the stack") + } + if code, out := m.siding(kept, "down", "--json"); code != 0 { + t.Fatalf("down: exit %d\n%s", code, out) + } + if code, out := m.siding(t.TempDir(), "teardown", "--json"); code != 0 || !strings.Contains(out, `"network": "siding-test-e2e"`) { + t.Fatalf("without a stack: exit %d\n%s", code, out) + } + if out := run(t, "", "docker", "network", "ls", "-q", "--filter", "name=^siding-test-e2e$"); strings.TrimSpace(out) != "" { + t.Errorf("the network survived: %q", out) + } + }) } diff --git a/internal/fail/fail.go b/internal/fail/fail.go index f853cf3..60de4df 100644 --- a/internal/fail/fail.go +++ b/internal/fail/fail.go @@ -55,6 +55,10 @@ const ( UnmergedBranch = "UnmergedBranch" // DirtyWorktree: removing data refused because the worktree has changes. DirtyWorktree = "DirtyWorktree" + // NoCapacity: the stack does not fit into the memory budget. The only code that exits 3. + NoCapacity = "NoCapacity" + // NotInteractive: a command would ask a question, but nobody can answer it. + NotInteractive = "NotInteractive" // Internal: an unexpected error without a code of its own. Internal = "Internal" ) diff --git a/internal/gc/gc.go b/internal/gc/gc.go new file mode 100644 index 0000000..6c59ade --- /dev/null +++ b/internal/gc/gc.go @@ -0,0 +1,82 @@ +// Package gc decides which stacks can go: a pure function from what Docker and git know about +// each stack to the actions siding gc takes. It never removes git worktrees or branches, and +// never anything that could hold unmerged or uncommitted work. +package gc + +import ( + "sort" + + "github.com/tweinmann/siding/internal/git" +) + +// Reasons a stack goes. Part of the --json contract. +const ( + Merged = "Merged" + WorktreeGone = "WorktreeGone" +) + +// Reasons a stack stays. Part of the --json contract. +const ( + Dirty = "Dirty" + Unmerged = "Unmerged" + DefaultBranch = "DefaultBranch" +) + +// Facts are what siding found out about one stack. +type Facts struct { + Project string + Branch string + Worktree string + // Machine is the stack's dev.siding.machine; gc only acts on this machine's stacks. + Machine string + // WorktreeExists is false when the worktree directory is gone. + WorktreeExists bool + // Dirty: the worktree has changes. Only meaningful while it exists. + Dirty bool + // Merge is the branch's merge state; MergeKnown is false when git could not be asked (no + // dev.siding.gitdir label, or the git directory is gone too). + Merge git.MergeState + MergeKnown bool +} + +// Action removes a stack. +type Action struct { + Project string + Reason string + // Volumes: also remove the volumes, and with them the data. + Volumes bool +} + +// Kept is a stack gc leaves alone. +type Kept struct { + Project string + Reason string +} + +// Plan returns what gc removes and what it keeps of the stacks of machine, each sorted by +// project. Stacks of other machines are neither removed nor listed. +func Plan(stacks []Facts, machine string) (actions []Action, kept []Kept) { + for _, f := range stacks { + if f.Machine != machine { + continue + } + merged := f.MergeKnown && f.Merge.MergedForGC() + switch { + case !f.WorktreeExists: + // The worktree is gone: nothing can start these containers again. The data goes + // only when git confirms the branch's work is merged. + actions = append(actions, Action{Project: f.Project, Reason: WorktreeGone, Volumes: merged}) + case f.Dirty: + kept = append(kept, Kept{Project: f.Project, Reason: Dirty}) + case f.MergeKnown && f.Merge.IsBase: + kept = append(kept, Kept{Project: f.Project, Reason: DefaultBranch}) + case merged: + actions = append(actions, Action{Project: f.Project, Reason: Merged, Volumes: true}) + default: + kept = append(kept, Kept{Project: f.Project, Reason: Unmerged}) + } + } + sort.Slice(actions, func(i, j int) bool { return actions[i].Project < actions[j].Project }) + sort.Slice(kept, func(i, j int) bool { return kept[i].Project < kept[j].Project }) + return actions, kept +} diff --git a/internal/gc/gc_test.go b/internal/gc/gc_test.go new file mode 100644 index 0000000..d36fac1 --- /dev/null +++ b/internal/gc/gc_test.go @@ -0,0 +1,105 @@ +package gc + +import ( + "reflect" + "testing" + + "github.com/tweinmann/siding/internal/git" +) + +func TestPlan(t *testing.T) { + t.Parallel() + base := "refs/heads/main" + mergedState := git.MergeState{Base: base, Exists: true, Ancestor: true} + tests := []struct { + name string + facts Facts + action *Action + kept *Kept + }{ + { + name: "merged and clean", + facts: Facts{Project: "p", WorktreeExists: true, Merge: mergedState, MergeKnown: true}, + action: &Action{Project: "p", Reason: Merged, Volumes: true}, + }, + { + name: "squash-merged: upstream gone", + facts: Facts{Project: "p", WorktreeExists: true, Merge: git.MergeState{Base: base, Exists: true, UpstreamGone: true}, MergeKnown: true}, + action: &Action{Project: "p", Reason: Merged, Volumes: true}, + }, + { + name: "merged but dirty", + facts: Facts{Project: "p", WorktreeExists: true, Dirty: true, Merge: mergedState, MergeKnown: true}, + kept: &Kept{Project: "p", Reason: Dirty}, + }, + { + name: "unmerged", + facts: Facts{Project: "p", WorktreeExists: true, Merge: git.MergeState{Base: base, Exists: true}, MergeKnown: true}, + kept: &Kept{Project: "p", Reason: Unmerged}, + }, + { + name: "a fresh branch without own commits", + facts: Facts{Project: "p", WorktreeExists: true, Merge: git.MergeState{Base: base, Exists: true, Ancestor: true, AtBase: true}, MergeKnown: true}, + kept: &Kept{Project: "p", Reason: Unmerged}, + }, + { + name: "the default branch", + facts: Facts{Project: "p", WorktreeExists: true, Merge: git.MergeState{Base: base, Exists: true, Ancestor: true, AtBase: true, IsBase: true}, MergeKnown: true}, + kept: &Kept{Project: "p", Reason: DefaultBranch}, + }, + { + name: "worktree gone, branch merged", + facts: Facts{Project: "p", Merge: mergedState, MergeKnown: true}, + action: &Action{Project: "p", Reason: WorktreeGone, Volumes: true}, + }, + { + name: "worktree gone, branch unmerged", + facts: Facts{Project: "p", Merge: git.MergeState{Base: base, Exists: true}, MergeKnown: true}, + action: &Action{Project: "p", Reason: WorktreeGone}, + }, + { + name: "worktree gone, a stack of siding v0.1 without the gitdir label", + facts: Facts{Project: "p"}, + action: &Action{Project: "p", Reason: WorktreeGone}, + }, + { + name: "merge state unknown although the worktree exists", + facts: Facts{Project: "p", WorktreeExists: true}, + kept: &Kept{Project: "p", Reason: Unmerged}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + tt.facts.Machine = "m" + actions, kept := Plan([]Facts{tt.facts}, "m") + var gotAction *Action + var gotKept *Kept + if len(actions) == 1 { + gotAction = &actions[0] + } + if len(kept) == 1 { + gotKept = &kept[0] + } + if len(actions)+len(kept) != 1 || !reflect.DeepEqual(gotAction, tt.action) || !reflect.DeepEqual(gotKept, tt.kept) { + t.Errorf("actions %+v, kept %+v; want %+v, %+v", actions, kept, tt.action, tt.kept) + } + }) + } +} + +func TestPlanSorted(t *testing.T) { + t.Parallel() + actions, kept := Plan([]Facts{{Project: "b", Machine: "m"}, {Project: "a", Machine: "m"}, {Project: "d", Machine: "m", WorktreeExists: true}, {Project: "c", Machine: "m", WorktreeExists: true}}, "m") + if actions[0].Project != "a" || kept[0].Project != "c" { + t.Errorf("not sorted: %+v %+v", actions, kept) + } +} + +func TestPlanOtherMachines(t *testing.T) { + t.Parallel() + actions, kept := Plan([]Facts{{Project: "remote", Machine: "other"}}, "m") + if len(actions)+len(kept) != 0 { + t.Errorf("a stack of another machine: %+v %+v", actions, kept) + } +} diff --git a/internal/git/git.go b/internal/git/git.go index a7076fb..33276ec 100644 --- a/internal/git/git.go +++ b/internal/git/git.go @@ -38,11 +38,15 @@ type Client struct { } func (c Client) run(ctx context.Context, dir string, args ...string) (string, error) { + return c.exec(ctx, append([]string{"-C", dir}, args...)) +} + +func (c Client) exec(ctx context.Context, args []string) (string, error) { bin := c.Binary if bin == "" { bin = "git" } - cmd := exec.CommandContext(ctx, bin, append([]string{"-C", dir}, args...)...) + cmd := exec.CommandContext(ctx, bin, args...) var out, errOut bytes.Buffer cmd.Stdout = &out cmd.Stderr = &errOut @@ -110,16 +114,34 @@ func (c Client) Dirty(ctx context.Context, top string) (bool, error) { return strings.TrimSpace(out) != "", nil } +// Repo names a repository for commands that need no worktree: the worktree's directory, or, +// when the worktree is gone, the common git directory. +type Repo struct { + Dir string + GitDir string +} + +func (r Repo) args(rest ...string) []string { + if r.GitDir != "" { + return append([]string{"--git-dir=" + r.GitDir}, rest...) + } + return append([]string{"-C", r.Dir}, rest...) +} + +func (c Client) git(ctx context.Context, r Repo, args ...string) (string, error) { + return c.exec(ctx, r.args(args...)) +} + // DefaultBranch returns the ref the repository merges into: what origin/HEAD points to, else // a local main, else a local master. It returns "" when there is none. -func (c Client) DefaultBranch(ctx context.Context, top string) (string, error) { - if out, err := c.run(ctx, top, "symbolic-ref", "--quiet", "refs/remotes/origin/HEAD"); err == nil { +func (c Client) DefaultBranch(ctx context.Context, r Repo) (string, error) { + if out, err := c.git(ctx, r, "symbolic-ref", "--quiet", "refs/remotes/origin/HEAD"); err == nil { return strings.TrimSpace(out), nil } else if exitCode(err) < 0 { return "", err } for _, ref := range []string{"refs/heads/main", "refs/heads/master"} { - if _, err := c.run(ctx, top, "rev-parse", "--verify", "--quiet", ref); err == nil { + if _, err := c.git(ctx, r, "rev-parse", "--verify", "--quiet", ref); err == nil { return ref, nil } else if exitCode(err) < 0 { return "", err @@ -128,20 +150,76 @@ func (c Client) DefaultBranch(ctx context.Context, top string) (string, error) { return "", nil } -// Merged reports whether the tip of branch is contained in the default branch, and which ref -// that is. Without a default branch it reports false. -func (c Client) Merged(ctx context.Context, top, branch string) (merged bool, base string, err error) { - base, err = c.DefaultBranch(ctx, top) - if err != nil || base == "" { - return false, "", err +// MergeState is what git knows about whether a branch's work has arrived in the default branch. +type MergeState struct { + // Base is the default branch's ref, "" when there is none. + Base string + // Exists is false when the branch itself is gone. + Exists bool + // Ancestor: the branch's tip is contained in Base. + Ancestor bool + // AtBase: the branch's tip is Base's tip, so Base has not moved past it. + AtBase bool + // UpstreamGone: the branch tracked a remote branch that no longer exists, as after a merged + // pull request whose branch was deleted and pruned (a squash merge included). + UpstreamGone bool + // IsBase: the branch is the default branch itself. + IsBase bool +} + +// Merged is the rule of down --volumes: the branch's commits are in the default branch, or its +// upstream is gone. +func (m MergeState) Merged() bool { return m.Exists && (m.Ancestor || m.UpstreamGone) } + +// MergedForGC is the stricter rule of gc: not the default branch, and either the default branch +// has moved past a contained tip or the upstream is gone. A branch without commits of its own, +// which an agent may just have created, does not count. +func (m MergeState) MergedForGC() bool { + return m.Exists && !m.IsBase && ((m.Ancestor && !m.AtBase) || m.UpstreamGone) +} + +// BaseName is the default branch's short name, e.g. main for refs/remotes/origin/main. +func (m MergeState) BaseName() string { + name := strings.TrimPrefix(m.Base, "refs/heads/") + if rest, ok := strings.CutPrefix(name, "refs/remotes/"); ok { + _, name, _ = strings.Cut(rest, "/") + } + return name +} + +// MergeState returns the merge state of branch. +func (c Client) MergeState(ctx context.Context, r Repo, branch string) (MergeState, error) { + var m MergeState + ref := "refs/heads/" + branch + tip, err := c.git(ctx, r, "rev-parse", "--verify", "--quiet", ref) + switch { + case exitCode(err) > 0: + return m, nil // the branch is gone + case err != nil: + return m, err + } + m.Exists = true + track, err := c.git(ctx, r, "for-each-ref", "--format=%(upstream:track)", ref) + if err != nil { + return m, err + } + m.UpstreamGone = strings.TrimSpace(track) == "[gone]" + if m.Base, err = c.DefaultBranch(ctx, r); err != nil || m.Base == "" { + return m, err + } + m.IsBase = m.BaseName() == branch + baseTip, err := c.git(ctx, r, "rev-parse", "--verify", "--quiet", m.Base) + if err != nil { + return m, err } - _, err = c.run(ctx, top, "merge-base", "--is-ancestor", "refs/heads/"+branch, base) + m.AtBase = strings.TrimSpace(tip) == strings.TrimSpace(baseTip) + _, err = c.git(ctx, r, "merge-base", "--is-ancestor", ref, m.Base) switch code := exitCode(err); { case err == nil: - return true, base, nil + m.Ancestor = true case code == 1: - return false, base, nil default: - return false, base, err + return m, err } + return m, nil } diff --git a/internal/git/git_test.go b/internal/git/git_test.go index d97d4dc..1cd0b7e 100644 --- a/internal/git/git_test.go +++ b/internal/git/git_test.go @@ -113,46 +113,74 @@ func TestDirty(t *testing.T) { } } -func TestMerged(t *testing.T) { +func TestMergeState(t *testing.T) { t.Parallel() ctx := context.Background() - t.Run("merged and unmerged against main", func(t *testing.T) { - t.Parallel() - dir := newRepo(t) - sh(t, dir, "branch", "done") - sh(t, dir, "switch", "-q", "-c", "wip") - sh(t, dir, "commit", "-q", "--allow-empty", "-m", "wip") - for branch, want := range map[string]bool{"done": true, "wip": false, "main": true} { - merged, base, err := Client{}.Merged(ctx, dir, branch) - if err != nil || merged != want || base != "refs/heads/main" { - t.Errorf("%s: merged %v, base %q, err %v; want %v against refs/heads/main", branch, merged, base, err, want) - } - } - }) + // shop: main, "fresh" at main's tip, "done" merged with main one commit further, "wip" + // with a commit of its own. + dir := newRepo(t) + sh(t, dir, "branch", "done") + sh(t, dir, "commit", "-q", "--allow-empty", "-m", "after done") + sh(t, dir, "branch", "fresh") + sh(t, dir, "switch", "-q", "-c", "wip") + sh(t, dir, "commit", "-q", "--allow-empty", "-m", "wip") + sh(t, dir, "switch", "-q", "main") - t.Run("origin/HEAD wins", func(t *testing.T) { - t.Parallel() - origin := newRepo(t) - clone := filepath.Join(t.TempDir(), "clone") - sh(t, filepath.Dir(clone), "clone", "-q", origin, clone) - merged, base, err := Client{}.Merged(ctx, clone, "main") - if err != nil || !merged || base != "refs/remotes/origin/main" { - t.Errorf("merged %v, base %q, err %v; want true against refs/remotes/origin/main", merged, base, err) - } - }) + // clone: "feat" tracked a remote branch that was squash-merged and deleted, then pruned. + origin := newRepo(t) + sh(t, origin, "switch", "-q", "-c", "feat") + sh(t, origin, "commit", "-q", "--allow-empty", "-m", "feat") + sh(t, origin, "switch", "-q", "main") + clone := filepath.Join(t.TempDir(), "clone") + sh(t, filepath.Dir(clone), "clone", "-q", origin, clone) + sh(t, clone, "switch", "-q", "feat") + sh(t, clone, "switch", "-q", "main") + sh(t, origin, "branch", "-q", "-D", "feat") + sh(t, clone, "fetch", "-q", "--prune") - t.Run("no default branch", func(t *testing.T) { - t.Parallel() - dir := filepath.Join(t.TempDir(), "r") - if err := os.MkdirAll(dir, 0o755); err != nil { - t.Fatal(err) - } - sh(t, dir, "init", "-q", "-b", "trunk") - sh(t, dir, "commit", "-q", "--allow-empty", "-m", "first") - merged, base, err := Client{}.Merged(ctx, dir, "trunk") - if err != nil || merged || base != "" { - t.Errorf("merged %v, base %q, err %v; want unmerged without a base", merged, base, err) - } - }) + // gone: the worktree of a merged branch was deleted without git; only its git dir is left. + gone := newRepo(t) + wt := filepath.Join(t.TempDir(), "gone-wt") + sh(t, gone, "worktree", "add", "-q", "-b", "old", wt) + sh(t, gone, "commit", "-q", "--allow-empty", "-m", "main moved on") + if err := os.RemoveAll(wt); err != nil { + t.Fatal(err) + } + + noBase := filepath.Join(t.TempDir(), "r") + if err := os.MkdirAll(noBase, 0o755); err != nil { + t.Fatal(err) + } + sh(t, noBase, "init", "-q", "-b", "trunk") + sh(t, noBase, "commit", "-q", "--allow-empty", "-m", "first") + + tests := []struct { + name string + repo Repo + branch string + merged, forGC bool + base string + }{ + {name: "a fresh branch is merged but not for gc", repo: Repo{Dir: dir}, branch: "fresh", merged: true, base: "refs/heads/main"}, + {name: "merged and main moved on", repo: Repo{Dir: dir}, branch: "done", merged: true, forGC: true, base: "refs/heads/main"}, + {name: "own commits", repo: Repo{Dir: dir}, branch: "wip", base: "refs/heads/main"}, + {name: "the default branch", repo: Repo{Dir: dir}, branch: "main", merged: true, base: "refs/heads/main"}, + {name: "upstream gone after a squash merge", repo: Repo{Dir: clone}, branch: "feat", merged: true, forGC: true, base: "refs/remotes/origin/main"}, + {name: "the worktree is gone", repo: Repo{GitDir: filepath.Join(gone, ".git")}, branch: "old", merged: true, forGC: true, base: "refs/heads/main"}, + {name: "the branch is gone", repo: Repo{Dir: dir}, branch: "nope"}, + {name: "no default branch", repo: Repo{Dir: noBase}, branch: "trunk"}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + m, err := Client{}.MergeState(ctx, tt.repo, tt.branch) + if err != nil { + t.Fatal(err) + } + if m.Merged() != tt.merged || m.MergedForGC() != tt.forGC || m.Base != tt.base { + t.Errorf("%+v: merged %v, for gc %v; want %v, %v, base %q", m, m.Merged(), m.MergedForGC(), tt.merged, tt.forGC, tt.base) + } + }) + } } diff --git a/internal/lint/lint.go b/internal/lint/lint.go new file mode 100644 index 0000000..c17f808 --- /dev/null +++ b/internal/lint/lint.go @@ -0,0 +1,158 @@ +// Package lint finds what in a compose file breaks parallel stacks or weakens siding's verdict. +// It reads the normalised configuration and never changes a file. +package lint + +import ( + "errors" + "fmt" + "path/filepath" + "sort" + "strings" + + "github.com/tweinmann/siding/internal/compose" + "github.com/tweinmann/siding/internal/fail" + "github.com/tweinmann/siding/internal/stack" +) + +// Severities, worst first. +const ( + Error = "error" + Warning = "warning" + Info = "info" +) + +// Rules. Part of the --json contract. +const ( + ContainerName = "ContainerName" + FixedNetworkName = "FixedNetworkName" + FixedVolumeName = "FixedVolumeName" + InvalidRoute = fail.InvalidRoute + AmbiguousPort = fail.AmbiguousPort + HostPath = "HostPath" + NetworkMode = "NetworkMode" + NoHealthcheck = "NoHealthcheck" + PublishedPort = "PublishedPort" + NoRoot = "NoRoot" +) + +// Finding is one problem. +type Finding struct { + Rule string + Severity string + // Subject is what the finding is about: "service web", "volume data", or "stack". + Subject string + Message string + Hint string +} + +// Check lints p, the configuration of project, whose worktree is the directory every relative +// bind mount lives in. +func Check(p *compose.Project, project, worktree string) []Finding { + var out []Finding + add := func(rule, severity, subject, hint, format string, args ...any) { + out = append(out, Finding{Rule: rule, Severity: severity, Subject: subject, Message: fmt.Sprintf(format, args...), Hint: hint}) + } + + routes, _, err := stack.Routes(p) + var routeErr *fail.Error + if errors.As(err, &routeErr) { + add(routeErr.Code, Error, "stack", routeErr.Hint, "%s", routeErr.Message) + } + routed := map[string]string{} + for _, r := range routes { + routed[r.Service] = r.Path + } + + for _, name := range p.ServiceNames() { + s := p.Services[name] + subject := "service " + name + if s.ContainerName != "" { + add(ContainerName, Error, subject, "remove container_name; siding names containers per stack", + "%s sets container_name %q, so a second stack of this file cannot start", name, s.ContainerName) + } + for _, src := range s.Binds { + if !within(src, worktree) { + add(HostPath, Warning, subject, "mount a path inside the worktree, or a named volume", + "%s mounts %s from the host, and every branch's stack writes into the same directory", name, src) + } + } + if s.NetworkMode != "" { + add(NetworkMode, Warning, subject, "remove network_mode to make the service reachable through the proxy", + "%s uses network_mode %q, so siding cannot route it", name, s.NetworkMode) + } + if !s.Healthcheck && routed[name] == "" { + add(NoHealthcheck, Warning, subject, "add a healthcheck, so Ready means the service works, not only that it started", + "%s has no healthcheck and no route path; the verdict only knows that it runs", name) + } + if len(s.Published) > 0 { + add(PublishedPort, Info, subject, fmt.Sprintf("reach it through its route, or with docker compose -p %s exec %s", project, name), + "%s publishes %s; siding resets published ports", name, joinPorts(s.Published)) + } + } + for _, key := range sortedKeys(p.Networks) { + n := p.Networks[key] + if !n.External && n.Name != project+"_"+key { + add(FixedNetworkName, Error, "network "+key, "remove name:, or declare it external if stacks are meant to share it", + "the network %s has the fixed name %q, which every stack of this file would share", key, n.Name) + } + } + for _, key := range sortedKeys(p.Volumes) { + v := p.Volumes[key] + if v.External || v.Name == project+"_"+key { + continue + } + if v.Labels[stack.LabelShared] != "" && v.Labels[stack.LabelShared] != "false" { + add(FixedVolumeName, Info, "volume "+key, "", + "the volume %s has the fixed name %q and is shared on purpose (%s)", key, v.Name, stack.LabelShared) + continue + } + add(FixedVolumeName, Error, "volume "+key, fmt.Sprintf("remove name:, or label it %s: \"true\" if stacks are meant to share it", stack.LabelShared), + "the volume %s has the fixed name %q, so every stack of this file would share its data", key, v.Name) + } + if len(routes) > 0 && routeErr == nil { + root := false + for _, r := range routes { + root = root || r.Path == "/" + } + if !root { + add(NoRoot, Info, "stack", fmt.Sprintf("give the main service %s: \"/\"", stack.LabelRoute), + "no service is the root of the stack's address; it answers 404 at /") + } + } + + rank := map[string]int{Error: 0, Warning: 1, Info: 2} + sort.SliceStable(out, func(i, j int) bool { return rank[out[i].Severity] < rank[out[j].Severity] }) + return out +} + +// HasErrors reports whether a finding is an error. +func HasErrors(fs []Finding) bool { + for _, f := range fs { + if f.Severity == Error { + return true + } + } + return false +} + +func within(path, dir string) bool { + rel, err := filepath.Rel(dir, path) + return err == nil && rel != ".." && !strings.HasPrefix(rel, "../") +} + +func joinPorts(ps []int) string { + s := make([]string, len(ps)) + for i, p := range ps { + s[i] = fmt.Sprint(p) + } + return strings.Join(s, ", ") +} + +func sortedKeys(m map[string]compose.Resource) []string { + keys := make([]string, 0, len(m)) + for k := range m { + keys = append(keys, k) + } + sort.Strings(keys) + return keys +} diff --git a/internal/lint/lint_test.go b/internal/lint/lint_test.go new file mode 100644 index 0000000..057a12a --- /dev/null +++ b/internal/lint/lint_test.go @@ -0,0 +1,82 @@ +package lint + +import ( + "os" + "path/filepath" + "reflect" + "testing" + + "github.com/tweinmann/siding/internal/compose" +) + +func TestCheckEveryRule(t *testing.T) { + t.Parallel() + data, err := os.ReadFile(filepath.Join("..", "compose", "testdata", "lint", "config.json")) + if err != nil { + t.Fatal(err) + } + p, err := compose.ParseConfig(data) + if err != nil { + t.Fatal(err) + } + type key struct{ rule, severity, subject string } + var got []key + for _, f := range Check(p, "shop-main", "/src/shop") { + if f.Message == "" { + t.Errorf("%s without a message", f.Rule) + } + got = append(got, key{f.Rule, f.Severity, f.Subject}) + } + want := []key{ + {ContainerName, Error, "service db"}, + {FixedNetworkName, Error, "network shared"}, + {FixedVolumeName, Error, "volume pgdata"}, + {HostPath, Warning, "service db"}, // /var/lib/shop; ./init is inside the worktree + {NoHealthcheck, Warning, "service db"}, + {NetworkMode, Warning, "service tools"}, + {NoHealthcheck, Warning, "service tools"}, + {NoHealthcheck, Warning, "service worker"}, + {PublishedPort, Info, "service db"}, + {FixedVolumeName, Info, "volume cache"}, // shared on purpose + {NoRoot, Info, "stack"}, + } + if !reflect.DeepEqual(got, want) { + t.Errorf("findings\n%v\nwant\n%v", got, want) + } + if !HasErrors(Check(p, "shop-main", "/src/shop")) { + t.Error("HasErrors is false") + } +} + +func TestCheckClean(t *testing.T) { + t.Parallel() + p := &compose.Project{ + Services: map[string]compose.Service{ + "web": {Ports: []int{3000}, Labels: map[string]string{"dev.siding.route": "/"}, Networks: []string{"default"}, Healthcheck: true}, + "db": {Labels: map[string]string{}, Networks: []string{"default"}, Healthcheck: true, Binds: []string{"/src/shop/init"}}, + }, + Networks: map[string]compose.Resource{"default": {Name: "shop-main_default"}, "ext": {Name: "shared", External: true}}, + Volumes: map[string]compose.Resource{"data": {Name: "shop-main_data"}}, + } + if fs := Check(p, "shop-main", "/src/shop"); len(fs) != 0 { + t.Errorf("findings %+v", fs) + } +} + +func TestCheckRoutes(t *testing.T) { + t.Parallel() + for _, tt := range []struct { + name string + svc compose.Service + rule string + }{ + {name: "invalid", svc: compose.Service{Ports: []int{80}, Labels: map[string]string{"dev.siding.route": "api"}, Healthcheck: true}, rule: InvalidRoute}, + {name: "ambiguous", svc: compose.Service{Ports: []int{3000, 9229}, Labels: map[string]string{"dev.siding.route": "/"}, Healthcheck: true}, rule: AmbiguousPort}, + } { + p := &compose.Project{Services: map[string]compose.Service{"web": tt.svc}} + fs := Check(p, "shop-main", "/src/shop") + if len(fs) != 1 || fs[0].Rule != tt.rule || fs[0].Severity != Error { + t.Errorf("%s: %+v", tt.name, fs) + } + } +} diff --git a/internal/overview/route.go b/internal/overview/route.go new file mode 100644 index 0000000..47e711b --- /dev/null +++ b/internal/overview/route.go @@ -0,0 +1,95 @@ +// Package overview holds what siding ui needs besides the page: the route file through which +// siding-proxy reaches it, and the addresses it listens on. +package overview + +import ( + "errors" + "fmt" + "io/fs" + "net" + "os" + "path/filepath" + + "github.com/tweinmann/siding/internal/proxy" +) + +// Host is the overview's address on the proxy. +const Host = "siding.localhost" + +// RouteFileName is the route file's name in the proxy's routes directory. +const RouteFileName = "ui.yml" + +// Route is the Traefik dynamic configuration that sends Host to siding ui on port. +func Route(port int) []byte { + return fmt.Appendf(nil, `# Written by siding ui while it runs; removed when it stops. +http: + routers: + siding-ui: + rule: "Host(`+"`%s`"+`)" + entrypoints: [web] + service: siding-ui + services: + siding-ui: + loadBalancer: + servers: + - url: "http://host.docker.internal:%d" +`, Host, port) +} + +// WriteRoute writes the route file into dir, replacing one a killed siding ui left behind, and +// returns the function that removes it. The file appears at once (write and rename), so the +// proxy never reads half of it. +func WriteRoute(dir string, port int) (remove func() error, err error) { + path := filepath.Join(dir, RouteFileName) + tmp, err := os.CreateTemp(dir, ".ui-*.yml") + if err != nil { + return nil, fmt.Errorf("write the overview's route: %w", err) + } + if _, err := tmp.Write(Route(port)); err != nil { + _ = tmp.Close() + _ = os.Remove(tmp.Name()) + return nil, fmt.Errorf("write the overview's route: %w", err) + } + if err := tmp.Close(); err != nil { + _ = os.Remove(tmp.Name()) + return nil, fmt.Errorf("write the overview's route: %w", err) + } + if err := os.Chmod(tmp.Name(), 0o644); err != nil { + _ = os.Remove(tmp.Name()) + return nil, err + } + if err := os.Rename(tmp.Name(), path); err != nil { + _ = os.Remove(tmp.Name()) + return nil, fmt.Errorf("write the overview's route: %w", err) + } + return func() error { + if err := os.Remove(path); err != nil && !errors.Is(err, fs.ErrNotExist) { + return err + } + return nil + }, nil +} + +// IsLocalAddress reports whether ip is assigned to an interface of this machine. +func IsLocalAddress(ip string) bool { + addrs, err := net.InterfaceAddrs() + if err != nil { + return false + } + for _, a := range addrs { + if n, ok := a.(*net.IPNet); ok && n.IP.String() == ip { + return true + } + } + return false +} + +// ListenAddrs are the addresses siding ui listens on: the loopback interface, and the proxy +// network's gateway when the proxy reaches this machine there. Never all interfaces. +func ListenAddrs(hostAlias string) []string { + addrs := []string{"127.0.0.1"} + if hostAlias != "" && hostAlias != proxy.HostGateway && hostAlias != "127.0.0.1" { + addrs = append(addrs, hostAlias) + } + return addrs +} diff --git a/internal/overview/route_test.go b/internal/overview/route_test.go new file mode 100644 index 0000000..b32e746 --- /dev/null +++ b/internal/overview/route_test.go @@ -0,0 +1,57 @@ +package overview + +import ( + "os" + "path/filepath" + "reflect" + "strings" + "testing" +) + +func TestWriteRoute(t *testing.T) { + t.Parallel() + dir := t.TempDir() + // A route a killed siding ui left behind. + if err := os.WriteFile(filepath.Join(dir, RouteFileName), []byte("stale"), 0o644); err != nil { + t.Fatal(err) + } + remove, err := WriteRoute(dir, 4711) + if err != nil { + t.Fatal(err) + } + data, err := os.ReadFile(filepath.Join(dir, RouteFileName)) + if err != nil { + t.Fatal(err) + } + for _, want := range []string{"rule: \"Host(`siding.localhost`)\"", `url: "http://host.docker.internal:4711"`, "entrypoints: [web]"} { + if !strings.Contains(string(data), want) { + t.Errorf("route lacks %s:\n%s", want, data) + } + } + entries, _ := os.ReadDir(dir) + if len(entries) != 1 { + t.Errorf("temporary files left: %v", entries) + } + if err := remove(); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Join(dir, RouteFileName)); !os.IsNotExist(err) { + t.Errorf("the route is still there: %v", err) + } + if err := remove(); err != nil { + t.Errorf("removing twice: %v", err) + } +} + +func TestListenAddrs(t *testing.T) { + t.Parallel() + for alias, want := range map[string][]string{ + "172.19.0.1": {"127.0.0.1", "172.19.0.1"}, + "host-gateway": {"127.0.0.1"}, + "": {"127.0.0.1"}, + } { + if got := ListenAddrs(alias); !reflect.DeepEqual(got, want) { + t.Errorf("ListenAddrs(%q) = %v, want %v", alias, got, want) + } + } +} diff --git a/internal/proxy/proxy.go b/internal/proxy/proxy.go index 5c65ccd..0edcb20 100644 --- a/internal/proxy/proxy.go +++ b/internal/proxy/proxy.go @@ -26,11 +26,35 @@ func Default() Config { return Config{Project: "siding", Container: "siding-proxy", Network: "siding", Image: Image, DefaultPort: 80} } +// RoutesMount is where the proxy reads route files that siding ui writes. +const RoutesMount = "/etc/siding/routes" + +// HostGateway is the host alias when the Docker daemon runs in a VM: Docker's own name for the +// machine outside it. +const HostGateway = "host-gateway" + +// Settings are what one setup chooses for the proxy. +type Settings struct { + Port int + Machine string + // Budget is the memory budget in bytes, 0 for the default. + Budget int64 + // RoutesDir is the directory on this machine with the route files of siding ui. + RoutesDir string + // HostAlias is what host.docker.internal resolves to in the proxy: the proxy network's + // gateway when that is an address of this machine, else HostGateway (design D2). + HostAlias string +} + // ComposeFile renders the proxy's compose file. The constraint keeps Traefik to containers // siding labelled for this proxy's network, so foreign Traefik labels on the same daemon are // never exposed. -func ComposeFile(c Config, port int, machine string) []byte { +func ComposeFile(c Config, s Settings) []byte { constraint := fmt.Sprintf("LabelRegex(`dev.siding.repo`, `.+`) && Label(`traefik.docker.network`, `%s`)", c.Network) + labels := fmt.Sprintf(" dev.siding.machine: %q\n dev.siding.host-alias: %q\n", s.Machine, s.HostAlias) + if s.Budget > 0 { + labels += fmt.Sprintf(" dev.siding.budget: \"%d\"\n", s.Budget) + } var b strings.Builder fmt.Fprintf(&b, `# Written by siding setup. Do not edit: siding setup rewrites it. services: @@ -44,15 +68,19 @@ services: - --providers.docker.exposedbydefault=false - --providers.docker.network=%s - %q + - --providers.file.directory=%s + - --providers.file.watch=true - --ping=true ports: - "127.0.0.1:%d:80" + extra_hosts: + - "host.docker.internal:%s" volumes: - /var/run/docker.sock:/var/run/docker.sock:ro + - %q networks: [proxy] labels: - dev.siding.machine: %q - healthcheck: +%s healthcheck: test: ["CMD", "traefik", "healthcheck", "--ping"] interval: 2s timeout: 2s @@ -61,6 +89,7 @@ networks: proxy: name: %q external: true -`, c.Image, c.Container, c.Network, "--providers.docker.constraints="+constraint, port, machine, c.Network) +`, c.Image, c.Container, c.Network, "--providers.docker.constraints="+constraint, RoutesMount, s.Port, s.HostAlias, + s.RoutesDir+":"+RoutesMount+":ro", labels, c.Network) return []byte(b.String()) } diff --git a/internal/stack/identity.go b/internal/stack/identity.go index 7e7f9a3..e472aae 100644 --- a/internal/stack/identity.go +++ b/internal/stack/identity.go @@ -16,8 +16,13 @@ const ( LabelWorktree = "dev.siding.worktree" LabelMachine = "dev.siding.machine" LabelAgent = "dev.siding.agent" - LabelRoute = "dev.siding.route" - LabelPort = "dev.siding.port" + LabelGitDir = "dev.siding.gitdir" + LabelBudget = "dev.siding.budget" + LabelShared = "dev.siding.shared" + // LabelHostAlias, on siding-proxy, is what host.docker.internal resolves to in it. + LabelHostAlias = "dev.siding.host-alias" + LabelRoute = "dev.siding.route" + LabelPort = "dev.siding.port" // LabelProject is Compose's own label naming a container's project. LabelProject = "com.docker.compose.project" @@ -62,6 +67,9 @@ type Identity struct { Worktree string // absolute path of the worktree Machine string // stable machine id Agent string // optional agent name + // GitDir is the repository's common git directory, so gc can still ask git about the branch + // when the worktree is gone. + GitDir string } // RepoSlug is the slug of the repository name. @@ -90,9 +98,14 @@ func (id Identity) Labels() map[string]string { } } -// ContainerLabels are the labels of the stack's containers: Labels plus the agent, if any. +// ContainerLabels are the labels of the stack's containers: Labels plus the git directory and +// the agent, if any. Those two stay off networks and volumes: they are new in v0.2 or change +// between ups, and Compose recreates a network whose labels changed. func (id Identity) ContainerLabels() map[string]string { l := id.Labels() + if id.GitDir != "" { + l[LabelGitDir] = id.GitDir + } if id.Agent != "" { l[LabelAgent] = id.Agent } @@ -107,5 +120,6 @@ func FromLabels(l map[string]string) Identity { Worktree: l[LabelWorktree], Machine: l[LabelMachine], Agent: l[LabelAgent], + GitDir: l[LabelGitDir], } } diff --git a/internal/stack/identity_test.go b/internal/stack/identity_test.go index 20d6f10..78758cf 100644 --- a/internal/stack/identity_test.go +++ b/internal/stack/identity_test.go @@ -77,6 +77,13 @@ func TestIdentity(t *testing.T) { if _, ok := id.Labels()[LabelAgent]; ok { t.Error("networks and volumes must not carry the agent label") } + id.GitDir = "/src/shop/.git" + if got := id.ContainerLabels()[LabelGitDir]; got != "/src/shop/.git" { + t.Errorf("gitdir label = %q", got) + } + if _, ok := id.Labels()[LabelGitDir]; ok { + t.Error("networks and volumes must not carry the gitdir label") + } if got := FromLabels(id.ContainerLabels()); got != id { t.Errorf("FromLabels = %+v, want %+v", got, id) } diff --git a/internal/stack/routing_test.go b/internal/stack/routing_test.go index 21a98b3..b4d9052 100644 --- a/internal/stack/routing_test.go +++ b/internal/stack/routing_test.go @@ -134,7 +134,7 @@ func TestRoutes(t *testing.T) { // compose.yaml next to it) and the override siding writes for it (override.yml). func TestOverride(t *testing.T) { t.Parallel() - id := Identity{Repo: "shop", Branch: "main", Worktree: "/src/shop", Machine: "0123456789abcdef", Agent: "claude"} + id := Identity{Repo: "shop", Branch: "main", Worktree: "/src/shop", Machine: "0123456789abcdef", Agent: "claude", GitDir: "/src/shop/.git"} for _, dir := range []string{"routing"} { t.Run(dir, func(t *testing.T) { t.Parallel() diff --git a/internal/stack/testdata/override/routing.yml b/internal/stack/testdata/override/routing.yml index 9c8b29e..df9b6d7 100644 --- a/internal/stack/testdata/override/routing.yml +++ b/internal/stack/testdata/override/routing.yml @@ -5,6 +5,7 @@ services: labels: "dev.siding.agent": "claude" "dev.siding.branch": "main" + "dev.siding.gitdir": "/src/shop/.git" "dev.siding.machine": "0123456789abcdef" "dev.siding.repo": "shop" "dev.siding.worktree": "/src/shop" @@ -17,6 +18,7 @@ services: labels: "dev.siding.agent": "claude" "dev.siding.branch": "main" + "dev.siding.gitdir": "/src/shop/.git" "dev.siding.machine": "0123456789abcdef" "dev.siding.repo": "shop" "dev.siding.worktree": "/src/shop" @@ -40,6 +42,7 @@ services: labels: "dev.siding.agent": "claude" "dev.siding.branch": "main" + "dev.siding.gitdir": "/src/shop/.git" "dev.siding.machine": "0123456789abcdef" "dev.siding.repo": "shop" "dev.siding.worktree": "/src/shop" @@ -59,6 +62,7 @@ services: labels: "dev.siding.agent": "claude" "dev.siding.branch": "main" + "dev.siding.gitdir": "/src/shop/.git" "dev.siding.machine": "0123456789abcdef" "dev.siding.repo": "shop" "dev.siding.worktree": "/src/shop" @@ -75,6 +79,7 @@ services: labels: "dev.siding.agent": "claude" "dev.siding.branch": "main" + "dev.siding.gitdir": "/src/shop/.git" "dev.siding.machine": "0123456789abcdef" "dev.siding.repo": "shop" "dev.siding.worktree": "/src/shop" @@ -86,6 +91,7 @@ services: labels: "dev.siding.agent": "claude" "dev.siding.branch": "main" + "dev.siding.gitdir": "/src/shop/.git" "dev.siding.machine": "0123456789abcdef" "dev.siding.repo": "shop" "dev.siding.worktree": "/src/shop" @@ -106,6 +112,7 @@ services: labels: "dev.siding.agent": "claude" "dev.siding.branch": "main" + "dev.siding.gitdir": "/src/shop/.git" "dev.siding.machine": "0123456789abcdef" "dev.siding.repo": "shop" "dev.siding.worktree": "/src/shop" diff --git a/justfile b/justfile index 46cdd02..77bb5b7 100644 --- a/justfile +++ b/justfile @@ -33,6 +33,13 @@ golden: build: CGO_ENABLED=0 go build -trimpath -o bin/siding ./cmd/siding +# Build static macOS binaries into bin/ (siding-darwin-arm64, siding-darwin-amd64), to try +# siding on the Mac against Docker Desktop; bin/ is in the repository folder, so on the Mac too +build-darwin: + for arch in arm64 amd64; do \ + GOOS=darwin GOARCH=$arch CGO_ENABLED=0 go build -trimpath -o bin/siding-darwin-$arch ./cmd/siding; \ + done + # Inner daemon: a compose project with a healthcheck, ports reset by an override, Traefik routing smoke-docker: hack/smoke/docker.sh diff --git a/openspec/changes/phase-2-overview-and-budget/design.md b/openspec/changes/phase-2-overview-and-budget/design.md index 768563a..4d7559f 100644 --- a/openspec/changes/phase-2-overview-and-budget/design.md +++ b/openspec/changes/phase-2-overview-and-budget/design.md @@ -47,6 +47,11 @@ once: host.docker.internal:host-gateway`, and `siding ui` listens on `127.0.0.1`, which these VMs forward to. +`setup` stores its choice as the label `dev.siding.host-alias` on `siding-proxy` (the gateway's +address, or `host-gateway`), so `siding ui` reads from Docker where to listen, and a proxy +without the label (from v0.1) is one that needs `siding setup`. Whether the gateway belongs to +this machine is a seam in `cli.Options` (`IsLocalAddress`). + Tried in the inner daemon before this proposal: `host-gateway` resolves to the default bridge's gateway, not the proxy network's, and gave 502; the pinned gateway gave 200, and removing the file gave 404 within two seconds. The VM case is tested by hand on the Mac @@ -140,6 +145,16 @@ cache). `NoCapacity` is the first error that does not exit 1: `Execute` maps it to exit 3, with the usual `Error` document. `NotInteractive` is a normal error (exit 1). +### D12. `siding teardown` (added during apply, at the maintainer's request) +The counterpart of `setup`: `docker compose -p siding down --remove-orphans` from the proxy's +directory, then that directory, then the network `siding` through the API, but only when no +container is attached to it. It never touches stacks; they keep running as plain Compose +projects (the guardrail "removable"), and teardown lists them with the hint `siding down` or +`siding gc`. `~/.siding/machine-id` stays, so stacks started later carry the same machine id. +Alternatives: refuse while stacks exist (forces removing work to uninstall the proxy); also +remove the stacks (would remove data and break "never lose work"); remove all of `~/.siding` +(changes the machine id of stacks that still run). + ## Risks / Trade-offs - [The VM case of D2 is untested in CI] → a manual test on the Mac in the tasks; the error diff --git a/openspec/changes/phase-2-overview-and-budget/proposal.md b/openspec/changes/phase-2-overview-and-budget/proposal.md index 007a600..b1a00bd 100644 --- a/openspec/changes/phase-2-overview-and-budget/proposal.md +++ b/openspec/changes/phase-2-overview-and-budget/proposal.md @@ -22,6 +22,9 @@ machine needs a limit, so that the fifth stack does not make the first four unus - `siding lint`: checks a compose file for what breaks parallel stacks (fixed container, network or volume names, shared host paths, ambiguous ports, missing healthchecks), as a `LintReport` with error, warning and info findings. +- `siding teardown [--dry-run]`: the counterpart of `setup`; removes `siding-proxy`, its files + and the network `siding` when unused, and leaves every stack running as a plain Compose + project. - A new label `dev.siding.gitdir` (the repository's common git directory) on every stack, so `gc` can tell whether the branch of a deleted worktree is merged. - Development: `just build-darwin` builds a macOS binary to try siding on the Mac against @@ -40,7 +43,7 @@ machine needs a limit, so that the fifth stack does not make the first four unus ### Modified Capabilities - `cli-contract`: exit 3 for `NoCapacity` with an `Error` document; new error codes `NoCapacity` and `NotInteractive`. -- `proxy`: `setup --budget`, and the proxy's route to `siding ui`. +- `proxy`: `setup --budget`, the proxy's route to `siding ui`, and `siding teardown`. - `stack-identity`: the label `dev.siding.gitdir`. - `stack-list`: `ps` shows the budget. - `stack-lifecycle`: `down --volumes` accepts a squash-merged branch (upstream deleted), as `gc` does. diff --git a/openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md b/openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md index b5be84c..6dcda7a 100644 --- a/openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md +++ b/openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md @@ -48,3 +48,26 @@ proxy SHALL reach the machine siding runs on as `host.docker.internal`. #### Scenario: A route left behind - **WHEN** `siding ui` was killed and its route file is still there - **THEN** the proxy answers 502 for `siding.localhost`, and the next `siding ui` replaces the file + +### Requirement: Teardown removes the proxy +`siding teardown` SHALL remove `siding-proxy`, its files under `~/.siding/proxy/`, and the +network `siding` unless a container still uses it. It MUST NOT touch any stack: running stacks +keep running as plain Compose projects, and teardown names them. `--dry-run` removes nothing; +with `--json` it prints a `Teardown` document. + +#### Scenario: Nothing runs but the proxy +- **WHEN** `siding teardown` runs and no stack exists +- **THEN** `siding-proxy`, the network `siding` and `~/.siding/proxy/` are gone, and `~/.siding/machine-id` stays + +#### Scenario: A stack still runs +- **WHEN** `siding teardown` runs while the stack `shop-main` runs +- **THEN** the proxy is removed, the network `siding` stays because `shop-main` uses it, `shop-main` keeps running, and the output names it with the hint `siding down` or `siding gc` + +#### Scenario: Dry run +- **WHEN** `siding teardown --dry-run --json` runs +- **THEN** it prints a `Teardown` document with `dryRun: true` listing the proxy and the network, and removes nothing + +#### Scenario: No proxy +- **WHEN** `siding teardown` runs on a machine without `siding-proxy` +- **THEN** it reports that there is nothing to remove and exits 0 + diff --git a/openspec/changes/phase-2-overview-and-budget/tasks.md b/openspec/changes/phase-2-overview-and-budget/tasks.md index bd0c9ff..a4c9d5e 100644 --- a/openspec/changes/phase-2-overview-and-budget/tasks.md +++ b/openspec/changes/phase-2-overview-and-budget/tasks.md @@ -2,61 +2,66 @@ ## 1. Foundations -- [ ] 1.1 Add `Stdin` and `IsTerminal` to `cli.Options` (wired in `cmd/siding/main.go`, fakes in the test env); verify `just test` passes and no package-level variable is a seam -- [ ] 1.2 Add the error codes `NoCapacity` (exit 3 in `Execute`) and `NotInteractive` (D11); verify with the error-code table test and a test that `NoCapacity` exits 3 with an `Error` document -- [ ] 1.3 Extend `internal/compose` with `container_name`, bind mounts, healthchecks, memory limits (`mem_limit`, `deploy.resources.limits.memory`), replicas (`deploy.replicas`, `scale`), and names and labels of top-level networks and volumes; verify with a second captured config in `internal/compose/testdata/` and table assertions +- [x] 1.1 Add `Stdin` and `IsTerminal` to `cli.Options` (wired in `cmd/siding/main.go`, fakes in the test env); verify `just test` passes and no package-level variable is a seam +- [x] 1.2 Add the error codes `NoCapacity` (exit 3 in `Execute`) and `NotInteractive` (D11); verify with the error-code table test and a test that `NoCapacity` exits 3 with an `Error` document +- [x] 1.3 Extend `internal/compose` with `container_name`, bind mounts, healthchecks, memory limits (`mem_limit`, `deploy.resources.limits.memory`), replicas (`deploy.replicas`, `scale`), and names and labels of top-level networks and volumes; verify with a second captured config in `internal/compose/testdata/` and table assertions ## 2. Merged and the git directory -- [ ] 2.1 Extend `internal/git` with the shared merged check (ancestor of the default branch, or upstream `[gone]`), runnable with `--git-dir` for a missing worktree, and the gc conditions (not the default branch, default branch moved past the tip) (D7); verify with table tests against temporary repositories including a clone whose upstream branch was deleted and pruned, a fresh branch without own commits, and a deleted worktree -- [ ] 2.2 Let `down --volumes` accept a branch whose upstream is gone; verify with a fake-based test and the end-to-end guard case -- [ ] 2.3 Put `dev.siding.gitdir` on containers only (D8); verify with the override golden (`just golden`) and a test that networks and volumes do not carry it -- [ ] 2.4 Add `dev.siding.gitdir` and `dev.siding.budget` to the label conventions in CLAUDE.md and `openspec/config.yaml`; verify `openspec validate --all --strict` passes +- [x] 2.1 Extend `internal/git` with the shared merged check (ancestor of the default branch, or upstream `[gone]`), runnable with `--git-dir` for a missing worktree, and the gc conditions (not the default branch, default branch moved past the tip) (D7); verify with table tests against temporary repositories including a clone whose upstream branch was deleted and pruned, a fresh branch without own commits, and a deleted worktree +- [x] 2.2 Let `down --volumes` accept a branch whose upstream is gone; verify with a fake-based test and the end-to-end guard case +- [x] 2.3 Put `dev.siding.gitdir` on containers only (D8); verify with the override golden (`just golden`) and a test that networks and volumes do not carry it +- [x] 2.4 Add `dev.siding.gitdir` and `dev.siding.budget` to the label conventions in CLAUDE.md and `openspec/config.yaml`; verify `openspec validate --all --strict` passes ## 3. Memory budget -- [ ] 3.1 Implement size parsing (`512m`, `12g`, bytes, `default`) and the estimate of a project (D5); verify with table tests including replicas and both limit forms -- [ ] 3.2 Rework `setup` to always run Compose and compare container IDs (D6), and add `--budget` with the label carried forward; verify with fake-based tests for created, updated, unchanged, a budget change, `--budget default`, a v0.1 proxy, and the updated `setup.json` golden -- [ ] 3.3 Make `up` refuse with `NoCapacity` (exit 3) when use of other stacks plus estimate exceeds the budget, and start anyway with `--force` and a warning; verify with fake-based tests for over, within, forced, and the stack not counting twice -- [ ] 3.4 Show the budget in `ps` (total line, `budgetBytes` in `StackList`); verify with the updated text test and `stacklist.json` golden -- [ ] 3.5 Document the budget in the README (`setup --budget`, the estimate, `mem_limit`, exit 3, `--force`); verify the commands run as written in the devcontainer +- [x] 3.1 Implement size parsing (`512m`, `12g`, bytes, `default`) and the estimate of a project (D5); verify with table tests including replicas and both limit forms +- [x] 3.2 Rework `setup` to always run Compose and compare container IDs (D6), and add `--budget` with the label carried forward; verify with fake-based tests for created, updated, unchanged, a budget change, `--budget default`, a v0.1 proxy, and the updated `setup.json` golden +- [x] 3.3 Make `up` refuse with `NoCapacity` (exit 3) when use of other stacks plus estimate exceeds the budget, and start anyway with `--force` and a warning; verify with fake-based tests for over, within, forced, and the stack not counting twice +- [x] 3.4 Show the budget in `ps` (total line, `budgetBytes` in `StackList`); verify with the updated text test and `stacklist.json` golden +- [x] 3.5 Document the budget in the README (`setup --budget`, the estimate, `mem_limit`, exit 3, `--force`); verify the commands run as written in the devcontainer ## 4. gc -- [ ] 4.1 Implement the pure gc plan (D8): `Merged`, `WorktreeGone` with and without a merged branch, stacks without `dev.siding.gitdir`, kept stacks with `Dirty` or `Unmerged`, other machines ignored, the default branch never; verify with a table test per scenario of the spec `stack-gc` -- [ ] 4.2 Implement `siding gc [--dry-run] [--yes] [--json]` with the question on a terminal and `NotInteractive` otherwise, removing by project name; verify with fake-based tests (answer y, answer n, no terminal, `--json` without `--yes`) and `gc.json` and `gc-dryrun.json` goldens -- [ ] 4.3 Document `siding gc` in the README; verify the shown commands run as written +- [x] 4.1 Implement the pure gc plan (D8): `Merged`, `WorktreeGone` with and without a merged branch, stacks without `dev.siding.gitdir`, kept stacks with `Dirty` or `Unmerged`, other machines ignored, the default branch never; verify with a table test per scenario of the spec `stack-gc` +- [x] 4.2 Implement `siding gc [--dry-run] [--yes] [--json]` with the question on a terminal and `NotInteractive` otherwise, removing by project name; verify with fake-based tests (answer y, answer n, no terminal, `--json` without `--yes`) and `gc.json` and `gc-dryrun.json` goldens +- [x] 4.3 Document `siding gc` in the README; verify the shown commands run as written ## 5. lint -- [ ] 5.1 Implement the lint rules and severities of the spec `compose-lint` (D9); verify with a captured config that triggers every rule and one that triggers none -- [ ] 5.2 Implement `siding lint [--file] [--json]` with exit 1 on errors and the `LintReport` document; verify with fake-based tests, a text test and a `lintreport.json` golden -- [ ] 5.3 Document `siding lint` in the README with the rules in one table; verify the table lists every rule the code has +- [x] 5.1 Implement the lint rules and severities of the spec `compose-lint` (D9); verify with a captured config that triggers every rule and one that triggers none +- [x] 5.2 Implement `siding lint [--file] [--json]` with exit 1 on errors and the `LintReport` document; verify with fake-based tests, a text test and a `lintreport.json` golden +- [x] 5.3 Document `siding lint` in the README with the rules in one table; verify the table lists every rule the code has ## 6. Overview -- [ ] 6.1 Extend the proxy's compose file with the file provider on `~/.siding/proxy//routes` and the host alias decided by `setup` (D2: the network's gateway when it is a local address, else `host-gateway`); verify with fake-based tests for both cases and the proxy compose file content -- [ ] 6.2 Implement the route file of `siding ui` (atomic write, removal on exit, replacing a stale one) and the listen addresses of D2; verify with unit tests in a temporary home -- [ ] 6.3 Implement the server: the page (D3) with stacks, budget and gc candidates, `GET /api/stacks` and `GET /api/gc` returning the same documents as `ps --json` and `gc --dry-run --json`, 405 for other methods; verify with `httptest` tests and a golden of the rendered page with fixed data -- [ ] 6.4 Implement `siding ui [--port]` (ProxyNotRunning without the route support, prints both addresses, Ctrl-C ends with 130 after removing the route); verify with a fake-based test that the route file exists while it runs and is gone after cancel -- [ ] 6.5 Document `siding ui` in the README; verify the shown command runs as written in the devcontainer +- [x] 6.1 Extend the proxy's compose file with the file provider on `~/.siding/proxy//routes` and the host alias decided by `setup` (D2: the network's gateway when it is a local address, else `host-gateway`); verify with fake-based tests for both cases and the proxy compose file content +- [x] 6.2 Implement the route file of `siding ui` (atomic write, removal on exit, replacing a stale one) and the listen addresses of D2; verify with unit tests in a temporary home +- [x] 6.3 Implement the server: the page (D3) with stacks, budget and gc candidates, `GET /api/stacks` and `GET /api/gc` returning the same documents as `ps --json` and `gc --dry-run --json`, 405 for other methods; verify with `httptest` tests and a golden of the rendered page with fixed data +- [x] 6.4 Implement `siding ui [--port]` (ProxyNotRunning without the route support, prints both addresses, Ctrl-C ends with 130 after removing the route); verify with a fake-based test that the route file exists while it runs and is gone after cancel +- [x] 6.5 Document `siding ui` in the README; verify the shown command runs as written in the devcontainer ## 7. Agent skill -- [ ] 7.1 Teach exit 3 in the skill (stop and tell the user) and keep `--force`, `--volumes`, `gc` and `share` out of it; verify with the updated skill test and `just golden` +- [x] 7.1 Teach exit 3 in the skill (stop and tell the user) and keep `--force`, `--volumes`, `gc` and `share` out of it; verify with the updated skill test and `just golden` ## 8. Devcontainer and docs -- [ ] 8.1 Add `just build-darwin` (arm64 and amd64, `CGO_ENABLED=0`); verify both binaries are Mach-O for their architecture (`file bin/siding-darwin-*`) -- [ ] 8.2 Make Docker-in-Docker the only devcontainer setup in CLAUDE.md, the README ("siding in a devcontainer" with its limits) and `openspec/config.yaml` (architecture and roadmap without Docker-outside-of-Docker, also in the draft `phase-5-remote-host` if it mentions it); verify `openspec validate --all --strict` passes -- [ ] 8.3 Document in CLAUDE.md's safety rules that siding used on the host is product use and that `hack/nuke.sh` leaves it alone; verify `hack/nuke.sh` contains no name of a product object (`siding-proxy`, `siding`) +- [x] 8.1 Add `just build-darwin` (arm64 and amd64, `CGO_ENABLED=0`); verify both binaries are Mach-O for their architecture (`file bin/siding-darwin-*`) +- [x] 8.2 Make Docker-in-Docker the only devcontainer setup in CLAUDE.md, the README ("siding in a devcontainer" with its limits) and `openspec/config.yaml` (architecture and roadmap without Docker-outside-of-Docker, also in the draft `phase-5-remote-host` if it mentions it); verify `openspec validate --all --strict` passes +- [x] 8.3 Document in CLAUDE.md's safety rules that siding used on the host is product use and that `hack/nuke.sh` leaves it alone; verify `hack/nuke.sh` contains no name of a product object (`siding-proxy`, `siding`) ## 9. End-to-end and integration -- [ ] 9.1 Extend `internal/e2e`: a budget too small refuses with exit 3 and `--force` starts; `gc --dry-run --json` lists a merged stack and keeps a dirty one, `gc --yes` removes the merged one with its volumes; `siding ui` answers through the test proxy at `siding.localhost` and the route is gone after cancel; verify with `just smoke-stacks` and that no `siding-test-*` object is left -- [ ] 9.2 Run the flow by hand in the devcontainer: `setup --port 8080 --budget 2g`, two stacks, `ps`, `siding ui` in the host browser at `http://siding.localhost:8080`, `lint`, `gc`; record the result in the phase summary -- [ ] 9.3 Run the flow on the Mac with `bin/siding-darwin-arm64` against Docker Desktop, including `siding ui` (the VM case of D2) and port 80; record the result and fix what differs -- [ ] 9.4 Update the status in CLAUDE.md, the README and `openspec/config.yaml` to v0.2; verify `just test` passes +- [x] 9.1 Extend `internal/e2e`: a budget too small refuses with exit 3 and `--force` starts; `gc --dry-run --json` lists a merged stack and keeps a dirty one, `gc --yes` removes the merged one with its volumes; `siding ui` answers through the test proxy at `siding.localhost` and the route is gone after cancel; verify with `just smoke-stacks` and that no `siding-test-*` object is left +- [x] 9.2 Run the flow by hand in the devcontainer: `setup --port 8080 --budget 2g`, two stacks, `ps`, `siding ui` in the host browser at `http://siding.localhost:8080`, `lint`, `gc`; record the result in the phase summary +- [x] 9.3 Run the flow on the Mac with `bin/siding-darwin-arm64` against Docker Desktop, including `siding ui` (the VM case of D2) and port 80; record the result and fix what differs +- [x] 9.4 Update the status in CLAUDE.md, the README and `openspec/config.yaml` to v0.2; verify `just test` passes + +## 10. Teardown + +- [x] 10.1 Implement `siding teardown [--dry-run] [--json]` (D12) with the `Teardown` document; add network removal by name when unused to `internal/docker`; verify with fake-based tests per scenario of the requirement "Teardown removes the proxy" and `teardown.json` and `teardown-dryrun.json` goldens +- [x] 10.2 Document `siding teardown` in the README (also in "Once per machine: the proxy") and add it to the end-to-end test (teardown with a running stack keeps the stack and the network, without one removes both); verify with `just smoke-stacks` and the README commands in the devcontainer ## Workflow follow-up diff --git a/openspec/config.yaml b/openspec/config.yaml index 7497aa7..daac87c 100644 --- a/openspec/config.yaml +++ b/openspec/config.yaml @@ -21,8 +21,9 @@ context: | - Humans see what runs, what it costs and what can go after a merge. - A memory budget keeps the machine usable; later, stacks can run on a remote Docker host. - Status: pre-alpha. Phase 0 (repository, devcontainer, OpenSpec, CI) is done; Phase 1 (v0.1: - stacks per worktree, routing through siding-proxy, the verdict of status) is implemented. OpenSpec is the only planning source: `openspec/specs/` is what siding + Status: pre-alpha. Phase 0 (repository, devcontainer, OpenSpec, CI) and Phase 1 (v0.1: + stacks per worktree, routing through siding-proxy, the verdict of status) are done; Phase 2 + (v0.2: the overview siding ui, the memory budget, gc, lint) is implemented. OpenSpec is the only planning source: `openspec/specs/` is what siding does today, `openspec/changes/` what it is about to do (the roadmap is the `phase--` changes), `openspec/changes/archive/` how it got here. CLAUDE.md holds the working agreements, guardrails, safety rules and conventions in full; read it before @@ -82,20 +83,25 @@ context: | every stack by Host and path; no stack publishes a port. - Labels on every container, network and volume siding creates or adopts: `dev.siding.repo`, `dev.siding.branch`, `dev.siding.worktree` (absolute path), `dev.siding.machine` (stable - machine id); on containers also `dev.siding.agent` (optional; network and volume labels - must not change between two `up`s, since Compose recreates a network whose labels changed). - Service labels: `dev.siding.route` (`/`, `/`, `false`), `dev.siding.port` (the HTTP - port of a service with several), `dev.siding.seed` (`volume|script|none`). Volume label - `dev.siding.shared`. The machine and + machine id); on containers also `dev.siding.gitdir` (the common git directory) and + `dev.siding.agent` (optional), never on networks or volumes, whose labels must not change + between two `up`s, since Compose recreates a network whose labels changed. Service labels: + `dev.siding.route` (`/`, `/`, `false`), `dev.siding.port` (the HTTP port of a + service with several), `dev.siding.seed` (`volume|script|none`). Volume label + `dev.siding.shared`. On `siding-proxy`: `dev.siding.budget` (bytes) and + `dev.siding.host-alias` (what `host.docker.internal` resolves to). The machine and worktree labels exist from v0.1 so that remote hosts need no migration later. - Development and every test run in a devcontainer with its own Docker daemon - (Docker-in-Docker); see the spec `dev-environment`. + (Docker-in-Docker); see the spec `dev-environment`. There is no Docker-outside-of-Docker + variant, and siding inside any devcontainer is supported with Docker-in-Docker only: + mounting the host's Docker socket gives code in the container root on the host, the + opposite of why agents run in devcontainers. ## Roadmap (draft changes phase-1 to phase-5) - v0.1 (phase-1): `setup`, `up`, `ps`, `status --wait --json` with log tail, `logs`, `open`, `down`, hostname and path routing, Claude Code skill. - - v0.2 (phase-2): web overview at `siding.localhost`, `gc` (git-aware), `lint`, memory budget - (refuse, exit 3), Docker-outside-of-Docker devcontainer. + - v0.2 (phase-2): web overview at `siding.localhost` (`siding ui`), `gc` (git-aware), `lint`, + memory budget (refuse, exit 3), `just build-darwin` to try siding on the Mac. - v0.3 (phase-3): snapshots and `--seed`, queue for capacity, parallel build limit, learned peaks. - v0.4 (phase-4): idle stop and wake on request, stopping idle stacks under pressure. From 240a4dbf37af0802d613509fb8f5347b82015b2b Mon Sep 17 00:00:00 2001 From: tweinmann Date: Wed, 28 Oct 2026 22:07:40 +0000 Subject: [PATCH 3/3] Archive phase 2 and merge its specs The deltas of phase-2-overview-and-budget become the main specs overview, memory-budget, stack-gc and compose-lint, and update agent-skill, cli-contract, dev-environment, proxy, stack-identity, stack-lifecycle and stack-list. Co-Authored-By: Claude Opus 5.5 --- CLAUDE.md | 15 ++-- .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../specs/agent-skill/spec.md | 0 .../specs/cli-contract/spec.md | 0 .../specs/compose-lint/spec.md | 0 .../specs/dev-environment/spec.md | 0 .../specs/memory-budget/spec.md | 0 .../specs/overview/spec.md | 0 .../specs/proxy/spec.md | 0 .../specs/stack-gc/spec.md | 0 .../specs/stack-identity/spec.md | 0 .../specs/stack-lifecycle/spec.md | 0 .../specs/stack-list/spec.md | 0 .../tasks.md | 0 openspec/config.yaml | 2 +- openspec/specs/agent-skill/spec.md | 7 +- openspec/specs/cli-contract/spec.md | 20 ++++-- openspec/specs/compose-lint/spec.md | 58 +++++++++++++++ openspec/specs/dev-environment/spec.md | 23 ++++++ openspec/specs/memory-budget/spec.md | 56 +++++++++++++++ openspec/specs/overview/spec.md | 51 +++++++++++++ openspec/specs/proxy/spec.md | 50 +++++++++++-- openspec/specs/stack-gc/spec.md | 72 +++++++++++++++++++ openspec/specs/stack-identity/spec.md | 9 ++- openspec/specs/stack-lifecycle/spec.md | 9 ++- openspec/specs/stack-list/spec.md | 10 ++- 28 files changed, 352 insertions(+), 30 deletions(-) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/.openspec.yaml (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/design.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/proposal.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/agent-skill/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/cli-contract/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/compose-lint/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/dev-environment/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/memory-budget/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/overview/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/proxy/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/stack-gc/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/stack-identity/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/stack-lifecycle/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/specs/stack-list/spec.md (100%) rename openspec/changes/{phase-2-overview-and-budget => archive/2026-10-28-phase-2-overview-and-budget}/tasks.md (100%) create mode 100644 openspec/specs/compose-lint/spec.md create mode 100644 openspec/specs/memory-budget/spec.md create mode 100644 openspec/specs/overview/spec.md create mode 100644 openspec/specs/stack-gc/spec.md diff --git a/CLAUDE.md b/CLAUDE.md index 6f8e3e6..85c4ac6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,14 +14,13 @@ changing architecture. ## Status -Phase 0 approved (2026-10-28). Phase 1 (v0.1) approved and archived (2026-10-28, -`openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing`): `setup`, `up [--wait]`, -`status [--wait]`, `ps`, `logs`, `open`, `down [--volumes]`, `skill`, routing by host and path -through `siding-proxy`, `just smoke-stacks`. Phase 2 (`phase-2-overview-and-budget`, v0.2) -implemented, waiting for approval (2026-10-28), not yet archived: `siding ui` at -`siding.localhost`, the memory budget (`setup --budget`, `up` exits 3), `gc`, `lint`, -`teardown`, `just build-darwin`; tested by hand in the devcontainer and on the Mac with Docker Desktop. -After approval: `/opsx:archive`, then Phase 3 (`phase-3-snapshots-and-queue`, v0.3). +Phase 0 approved (2026-10-28). Phase 1 (v0.1) and Phase 2 (v0.2) approved and archived +(2026-10-28, `openspec/changes/archive/2026-10-28-phase-1-stacks-and-routing` and +`…-phase-2-overview-and-budget`). siding today: `setup [--port] [--budget]`, `teardown`, +`up [--wait] [--force]`, `status [--wait]`, `ps`, `logs`, `open`, `down [--volumes]`, `gc`, +`lint`, `ui`, `skill`; routing by host and path through `siding-proxy`; a memory budget; +`just smoke-stacks`, `just build-darwin`. Next: Phase 3 (`phase-3-snapshots-and-queue`, v0.3), +starting with `/opsx:propose`. ## Working agreements diff --git a/openspec/changes/phase-2-overview-and-budget/.openspec.yaml b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/.openspec.yaml similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/.openspec.yaml rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/.openspec.yaml diff --git a/openspec/changes/phase-2-overview-and-budget/design.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/design.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/design.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/design.md diff --git a/openspec/changes/phase-2-overview-and-budget/proposal.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/proposal.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/proposal.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/proposal.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/agent-skill/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/agent-skill/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/agent-skill/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/agent-skill/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/cli-contract/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/cli-contract/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/cli-contract/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/cli-contract/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/compose-lint/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/compose-lint/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/compose-lint/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/compose-lint/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/dev-environment/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/dev-environment/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/dev-environment/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/dev-environment/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/memory-budget/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/memory-budget/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/memory-budget/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/memory-budget/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/overview/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/overview/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/overview/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/overview/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/proxy/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/proxy/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/proxy/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/stack-gc/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/stack-gc/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/stack-gc/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/stack-gc/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/stack-identity/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/stack-identity/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/stack-identity/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/stack-identity/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/stack-lifecycle/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/stack-lifecycle/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/stack-lifecycle/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/stack-lifecycle/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/specs/stack-list/spec.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/stack-list/spec.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/specs/stack-list/spec.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/specs/stack-list/spec.md diff --git a/openspec/changes/phase-2-overview-and-budget/tasks.md b/openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/tasks.md similarity index 100% rename from openspec/changes/phase-2-overview-and-budget/tasks.md rename to openspec/changes/archive/2026-10-28-phase-2-overview-and-budget/tasks.md diff --git a/openspec/config.yaml b/openspec/config.yaml index daac87c..b823bca 100644 --- a/openspec/config.yaml +++ b/openspec/config.yaml @@ -23,7 +23,7 @@ context: | Status: pre-alpha. Phase 0 (repository, devcontainer, OpenSpec, CI) and Phase 1 (v0.1: stacks per worktree, routing through siding-proxy, the verdict of status) are done; Phase 2 - (v0.2: the overview siding ui, the memory budget, gc, lint) is implemented. OpenSpec is the only planning source: `openspec/specs/` is what siding + (v0.2: the overview siding ui, the memory budget, gc, lint, teardown) is done. OpenSpec is the only planning source: `openspec/specs/` is what siding does today, `openspec/changes/` what it is about to do (the roadmap is the `phase--` changes), `openspec/changes/archive/` how it got here. CLAUDE.md holds the working agreements, guardrails, safety rules and conventions in full; read it before diff --git a/openspec/specs/agent-skill/spec.md b/openspec/specs/agent-skill/spec.md index 2ecde2d..361b7e4 100644 --- a/openspec/specs/agent-skill/spec.md +++ b/openspec/specs/agent-skill/spec.md @@ -17,9 +17,10 @@ stdout and write no file. With `--json` it SHALL print a `Skill` document holdin ### Requirement: What the skill teaches The skill SHALL tell the agent to run `siding up --wait --json` after a change that affects the running app, to read the verdict and exit code, to fix the named service from the failure's -reason, hint and log tail, and to report the address when Ready. It MUST NOT tell the agent to -use `--force`, `--volumes` or `siding share`. +reason, hint and log tail, to report the address when Ready, and on exit 3 (`NoCapacity`) to +stop and tell the user. It MUST NOT tell the agent to use `--force`, `--volumes`, `siding gc` +or `siding share`. #### Scenario: The commands in the skill - **WHEN** the printed skill is read -- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1 and 2, and none of `--force`, `--volumes` or `share` +- **THEN** it names `siding up --wait --json`, `siding status --json`, `siding logs --json` and the exit codes 0, 1, 2 and 3, and none of `--force`, `--volumes`, `gc` or `share` diff --git a/openspec/specs/cli-contract/spec.md b/openspec/specs/cli-contract/spec.md index 0bdd6ed..93c9172 100644 --- a/openspec/specs/cli-contract/spec.md +++ b/openspec/specs/cli-contract/spec.md @@ -32,9 +32,9 @@ or given another meaning; new fields, kinds and codes MAY be added. - **THEN** documents still carry `apiVersion` `siding.dev/v1alpha1` and every earlier field with its meaning ### Requirement: Error document -A command that fails SHALL exit 1. With `--json` it SHALL print a document of kind `Error` -with a stable `code` and a human-readable `message`; without `--json` it SHALL print -`error: ` on stderr, followed by a hint line when there is one. +A command that fails SHALL exit 1, or 3 for `NoCapacity`. With `--json` it SHALL print a +document of kind `Error` with a stable `code` and a human-readable `message`; without `--json` +it SHALL print `error: ` on stderr, followed by a hint line when there is one. #### Scenario: An unknown flag with --json - **WHEN** `siding version --json --nope` runs @@ -46,12 +46,16 @@ with a stable `code` and a human-readable `message`; without `--json` it SHALL p #### Scenario: The error codes - **WHEN** a command fails -- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch`, `DirtyWorktree`, or `Internal` for an unexpected error +- **THEN** its code is one of `InvalidArguments`, `NotAGitRepository`, `NoBranch`, `ComposeFileNotFound`, `ComposeFailed`, `DockerUnavailable`, `ProxyNotRunning`, `PortInUse`, `InvalidRoute`, `AmbiguousPort`, `SlugCollision`, `StackNotFound`, `UnmergedBranch`, `DirtyWorktree`, `NoCapacity`, `NotInteractive`, or `Internal` for an unexpected error + +#### Scenario: No capacity +- **WHEN** `siding up --json` refuses because the stack does not fit into the memory budget +- **THEN** it prints an `Error` document with `code` `NoCapacity` and exits 3 ### Requirement: Exit codes siding SHALL exit 0 on success and when a stack is Ready, 1 when a command fails or a stack is -Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), 3 for -NoCapacity, which v0.1 never returns, and 130 without a message when interrupted. +Failed, 2 when a stack is not Ready in time (TimedOut, or Starting without `--wait`), 3 when +`up` refuses for lack of memory (NoCapacity), and 130 without a message when interrupted. #### Scenario: A verdict decides the exit code - **WHEN** `siding status --wait` ends with the verdict Failed @@ -61,6 +65,10 @@ NoCapacity, which v0.1 never returns, and 130 without a message when interrupted - **WHEN** the user presses Ctrl-C during `siding logs -f` - **THEN** siding stops docker compose and exits 130 without an error message +#### Scenario: Over budget +- **WHEN** `siding up --wait` refuses because the memory budget is exhausted +- **THEN** it exits 3 without starting a container + ### Requirement: Version `siding version` SHALL print the version, the commit (when the build knows it), the Go version and the platform; with `--json` as a document of kind `Version` with the fields `version`, diff --git a/openspec/specs/compose-lint/spec.md b/openspec/specs/compose-lint/spec.md new file mode 100644 index 0000000..bde8b07 --- /dev/null +++ b/openspec/specs/compose-lint/spec.md @@ -0,0 +1,58 @@ +# compose-lint Specification + +## Purpose +Defines `siding lint`, which tells a user before the first `up` what in their compose file +breaks parallel stacks or weakens the verdict, without changing the file. + +## Requirements + +### Requirement: Lint reads, never writes +`siding lint [--file]` SHALL check the compose file siding would use, through the configuration +Compose prints, and MUST NOT change any file. It works in any directory with a compose file, +inside a git worktree or not, and needs no running proxy. + +#### Scenario: Outside a repository +- **WHEN** `siding lint` runs in a directory with `compose.yaml` that is not a git worktree +- **THEN** it checks that file + +### Requirement: Findings +Every finding SHALL have a rule, a severity (`error`, `warning` or `info`), the service or +resource it is about, a message and a hint. `lint` SHALL exit 1 when there is an error and 0 +otherwise, and with `--json` print one `LintReport` document with the file and its findings. + +#### Scenario: A clean file +- **WHEN** the compose file has no finding +- **THEN** `siding lint` prints that it found nothing and exits 0 + +#### Scenario: Errors as JSON +- **WHEN** `siding lint --json` finds an error +- **THEN** it prints a `LintReport` with the finding and exits 1 + +### Requirement: Errors +`lint` SHALL report as errors what makes two stacks of the same file collide or `up` refuse: +`ContainerName` (a fixed `container_name`), `FixedNetworkName` (an own network with a fixed +`name`), `FixedVolumeName` (an own volume with a fixed `name`, unless it carries +`dev.siding.shared`), and `InvalidRoute` and `AmbiguousPort` as `up` reports them. + +#### Scenario: A fixed container name +- **WHEN** `db` sets `container_name: shop-db` +- **THEN** lint reports `ContainerName` for `db` with the hint to remove it, since a second stack could not start + +#### Scenario: A volume shared on purpose +- **WHEN** the volume `cache` has `name: pip-cache` and the label `dev.siding.shared: "true"` +- **THEN** lint reports `FixedVolumeName` as info, not as an error + +### Requirement: Warnings and info +`lint` SHALL warn about `HostPath` (a bind mount of an absolute path outside the worktree, which +every stack shares), `NetworkMode` (a service siding cannot route) and `NoHealthcheck` (a +service without a healthcheck and without a route path, of which the verdict only knows that it +runs), and report as info `PublishedPort` (ports siding resets) and `NoRoot` (HTTP services but +no `dev.siding.route: "/"`). + +#### Scenario: A host path +- **WHEN** `db` mounts `/var/lib/shop:/data` +- **THEN** lint warns `HostPath` for `db`, because every branch's stack would write into the same directory + +#### Scenario: A published database port +- **WHEN** `db` publishes `5432:5432` +- **THEN** lint reports `PublishedPort` as info, with the hint that siding resets it and how to reach the database diff --git a/openspec/specs/dev-environment/spec.md b/openspec/specs/dev-environment/spec.md index 5eda724..67fcaf5 100644 --- a/openspec/specs/dev-environment/spec.md +++ b/openspec/specs/dev-environment/spec.md @@ -25,11 +25,16 @@ The Docker engine version is pinned in `.devcontainer/devcontainer.json`. ### Requirement: A Docker daemon of its own The devcontainer SHALL run its own Docker daemon (Docker-in-Docker), which also hosts every stack siding's tests create, and MUST NOT mount or reach the host's Docker socket. +Docker-in-Docker is the only devcontainer setup; there is no variant with the host's socket. #### Scenario: docker inside the container - **WHEN** `docker info` runs inside the devcontainer - **THEN** it reports the container's own host name, not the Docker Desktop VM or the host +#### Scenario: No Docker-outside-of-Docker +- **WHEN** someone looks for a devcontainer configuration of this repository +- **THEN** there is only `.devcontainer/devcontainer.json`, and no configuration mounts `/var/run/docker.sock` from the host + ### Requirement: Devcontainer settings The devcontainer SHALL be named `siding-devcontainer`, set `SIDING_DEVCONTAINER=1`, forward port 8080 inside to 8080 on the host for the proxy and never to another host port, and set @@ -170,3 +175,21 @@ obvious fakes such as `not-a-real-token`. #### Scenario: A golden file needs a token - **WHEN** a test's output contains a token - **THEN** the test uses an obvious fake, and the golden file holds only that fake + +### Requirement: macOS binaries +`just build-darwin` SHALL build static siding binaries for macOS, `bin/siding-darwin-arm64` and +`bin/siding-darwin-amd64`, in the devcontainer, so siding can be tried on the Mac against +Docker Desktop without installing Go there. + +#### Scenario: Try siding on the Mac +- **WHEN** `just build-darwin` has run in the devcontainer +- **THEN** `bin/siding-darwin-arm64 version` runs on an Apple silicon Mac and reports `darwin/arm64` + +### Requirement: siding used on the host +What a human creates by running siding on the host (`siding-proxy`, the network `siding`, their +stacks, `~/.siding`) SHALL count as use of the product, not as development footprint: +`hack/nuke.sh` MUST NOT remove it, and it is removed with siding's own commands. + +#### Scenario: nuke.sh and a real proxy +- **WHEN** the maintainer ran `siding setup` on the Mac and then runs `hack/nuke.sh` +- **THEN** `siding-proxy` and the network `siding` on the host stay, and `docker compose -p siding down` removes them diff --git a/openspec/specs/memory-budget/spec.md b/openspec/specs/memory-budget/spec.md new file mode 100644 index 0000000..cebedc8 --- /dev/null +++ b/openspec/specs/memory-budget/spec.md @@ -0,0 +1,56 @@ +# memory-budget Specification + +## Purpose +Defines the memory budget that keeps the machine usable with many stacks: where it is kept, how +the memory of a new stack is estimated before it starts, and how `up` refuses when it would not +fit. + +## Requirements + +### Requirement: Budget on the proxy +The budget SHALL be kept as the label `dev.siding.budget` (bytes) on `siding-proxy`, set with +`siding setup --budget ` (`512m`, `12g`, or bytes) and removed with `--budget default`. +Without the label the budget SHALL be 75 % of the memory the Docker daemon reports. + +#### Scenario: Set a budget +- **WHEN** `siding setup --budget 12g` runs +- **THEN** `siding-proxy` carries `dev.siding.budget=12884901888`, and running stacks keep running + +#### Scenario: Default +- **WHEN** the proxy has no budget label and Docker reports 16 GiB of memory +- **THEN** the budget is 12 GiB + +#### Scenario: setup keeps the budget +- **WHEN** `siding setup` runs again without `--budget` +- **THEN** the budget set before stays + +### Requirement: Estimate of a stack +The estimate of a stack SHALL be the sum over its services of the memory limit +(`deploy.resources.limits.memory`, else `mem_limit`), or 256 MiB for a service without a +limit; a service scaled to several replicas counts once per replica. + +#### Scenario: Limits and defaults +- **WHEN** a stack has `db` with `mem_limit: 1g`, `web` with no limit and `api` with `deploy.resources.limits.memory: 512m` +- **THEN** its estimate is 1 GiB + 256 MiB + 512 MiB + +### Requirement: up refuses over budget +`siding up` SHALL refuse with `NoCapacity` and exit 3, before starting anything, when the +memory in use by every other siding stack on the machine plus the estimate of this stack +exceeds the budget. The message MUST name the budget, the use and the estimate. `--force` +starts it anyway. + +#### Scenario: Over budget +- **WHEN** the budget is 4 GiB, other stacks use 3.5 GiB and the new stack's estimate is 768 MiB +- **THEN** `siding up` exits 3 with `NoCapacity` and the hint names `siding gc`, `siding down` in another worktree, `siding setup --budget` and `--force` + +#### Scenario: Forced +- **WHEN** `siding up --force` runs in the same situation +- **THEN** the stack starts, and a warning on stderr says the budget is exceeded + +#### Scenario: The stack itself does not count twice +- **WHEN** `siding up` runs again for a stack that already runs +- **THEN** only the other stacks' memory and this stack's estimate count, not its current use as well + +#### Scenario: Within budget +- **WHEN** the other stacks and the estimate fit into the budget +- **THEN** `siding up` behaves as without a budget diff --git a/openspec/specs/overview/spec.md b/openspec/specs/overview/spec.md new file mode 100644 index 0000000..7ad2bdf --- /dev/null +++ b/openspec/specs/overview/spec.md @@ -0,0 +1,51 @@ +# overview Specification + +## Purpose +Defines `siding ui`, the one page where a human sees every stack on the machine, what it costs +against the budget, and what could go after a merge, served only while the command runs. + +## Requirements + +### Requirement: Overview in the foreground +`siding ui` SHALL serve a read-only overview until it is stopped, at `http://siding.localhost` +through the proxy and at the local address it prints. When it stops, the route MUST go away and +nothing of it MUST keep running. It SHALL NOT change any stack. + +#### Scenario: Start and stop +- **WHEN** `siding ui` runs and the user presses Ctrl-C +- **THEN** while it ran `http://siding.localhost` showed the overview, and afterwards the proxy answers 404 for `siding.localhost` and no process of siding is left + +#### Scenario: No proxy +- **WHEN** `siding ui` runs while `siding-proxy` is not running or lacks the overview route +- **THEN** it fails with `ProxyNotRunning` and the hint `siding setup` + +### Requirement: Only this machine +The overview SHALL listen only on addresses of this machine that the proxy can reach (the +loopback interface, or the proxy network's gateway when the Docker daemon runs on this +machine), on `--port` or a free port, and MUST NOT listen on all interfaces. + +#### Scenario: Not on the network +- **WHEN** `siding ui` runs on a Linux machine with a LAN address +- **THEN** a request to that LAN address and the overview's port is refused + +### Requirement: What the page shows +The page SHALL show every stack as `siding ps` lists it (repository, branch, state, address as +a link, memory, worktree), the memory in use against the budget, and the stacks `siding gc` +would remove, with the reason. It SHALL refresh itself every 5 seconds. + +#### Scenario: Two stacks and a merged branch +- **WHEN** stacks run for `main` and for a merged, clean `feature/login` +- **THEN** the page lists both with links to their addresses, the total against the budget, and `feature/login` under "can go" with the reason `Merged` + +### Requirement: JSON endpoints +The overview SHALL answer `GET /api/stacks` with the `StackList` document of `siding ps --json` +and `GET /api/gc` with the `GC` document of `siding gc --dry-run --json`, so the page and +scripts read the same data. Every other method MUST be refused with 405. + +#### Scenario: Same document as ps +- **WHEN** `GET http://siding.localhost/api/stacks` is requested +- **THEN** the body is a `StackList` document equal to what `siding ps --json` prints at that moment + +#### Scenario: No actions +- **WHEN** `POST http://siding.localhost/api/gc` is requested +- **THEN** the overview answers 405 and removes nothing diff --git a/openspec/specs/proxy/spec.md b/openspec/specs/proxy/spec.md index 8a142cc..d877277 100644 --- a/openspec/specs/proxy/spec.md +++ b/openspec/specs/proxy/spec.md @@ -9,8 +9,8 @@ on the network `siding`, started by `siding setup` and bound to the loopback int ### Requirement: Setup starts the proxy `siding setup` SHALL create the network `siding` if it is missing and run the Compose project `siding` with one container `siding-proxy`: Traefik with the Docker provider on the network -`siding`, restarting unless stopped. It SHALL print the proxy's address, with `--json` as a -`Setup` document. +`siding`, restarting unless stopped. It SHALL print the proxy's address and budget, with +`--json` as a `Setup` document. #### Scenario: First setup - **WHEN** `siding setup` runs on a machine without the proxy @@ -18,7 +18,7 @@ on the network `siding`, started by `siding setup` and bound to the loopback int #### Scenario: JSON - **WHEN** `siding setup --json` runs -- **THEN** it prints a `Setup` document with the proxy's container name, network, image and port +- **THEN** it prints a `Setup` document with the proxy's container name, network, image, port and budget in bytes ### Requirement: Loopback port The proxy SHALL publish exactly one port, on `127.0.0.1` only, `80` unless `--port ` names @@ -38,8 +38,9 @@ another. siding MUST read the proxy's port from the running container, not from ### Requirement: Setup is idempotent `siding setup` SHALL change nothing when the proxy already runs as requested, and SHALL -recreate only the proxy container when its port or image differs. Running stacks MUST keep -running and be routed again once the proxy is back. +recreate only the proxy container when its configuration differs (port, image, budget, or the +proxy configuration of a newer siding). Running stacks MUST keep running and be routed again +once the proxy is back. #### Scenario: Second run - **WHEN** `siding setup` runs twice @@ -49,6 +50,10 @@ running and be routed again once the proxy is back. - **WHEN** a stack runs and `siding setup --port 8080` follows `siding setup` - **THEN** the stack's containers are not restarted and it answers on port 8080 +#### Scenario: A proxy from siding v0.1 +- **WHEN** `siding-proxy` was created by siding v0.1 and `siding setup` of v0.2 runs +- **THEN** the proxy is recreated with the route to the overview, on the port it had + ### Requirement: Only siding stacks are routed The proxy MUST route only containers that carry `dev.siding.repo` and enable Traefik, so that other containers with Traefik labels on the same daemon are never exposed through it. @@ -64,3 +69,38 @@ other containers with Traefik labels on the same daemon are never exposed throug #### Scenario: up before setup - **WHEN** `siding up` runs on a machine without the proxy - **THEN** it fails with `ProxyNotRunning` before starting any container + +### Requirement: Route to the overview +The proxy SHALL route `siding.localhost` to `siding ui` while it runs, through a route file +that `siding ui` writes into `~/.siding/proxy//routes/` and removes when it stops. The +proxy SHALL reach the machine siding runs on as `host.docker.internal`. + +#### Scenario: A route file +- **WHEN** `siding ui` has written its route file +- **THEN** the proxy sends `Host: siding.localhost` to `siding ui` within a few seconds, without a restart + +#### Scenario: A route left behind +- **WHEN** `siding ui` was killed and its route file is still there +- **THEN** the proxy answers 502 for `siding.localhost`, and the next `siding ui` replaces the file + +### Requirement: Teardown removes the proxy +`siding teardown` SHALL remove `siding-proxy`, its files under `~/.siding/proxy/`, and the +network `siding` unless a container still uses it. It MUST NOT touch any stack: running stacks +keep running as plain Compose projects, and teardown names them. `--dry-run` removes nothing; +with `--json` it prints a `Teardown` document. + +#### Scenario: Nothing runs but the proxy +- **WHEN** `siding teardown` runs and no stack exists +- **THEN** `siding-proxy`, the network `siding` and `~/.siding/proxy/` are gone, and `~/.siding/machine-id` stays + +#### Scenario: A stack still runs +- **WHEN** `siding teardown` runs while the stack `shop-main` runs +- **THEN** the proxy is removed, the network `siding` stays because `shop-main` uses it, `shop-main` keeps running, and the output names it with the hint `siding down` or `siding gc` + +#### Scenario: Dry run +- **WHEN** `siding teardown --dry-run --json` runs +- **THEN** it prints a `Teardown` document with `dryRun: true` listing the proxy and the network, and removes nothing + +#### Scenario: No proxy +- **WHEN** `siding teardown` runs on a machine without `siding-proxy` +- **THEN** it reports that there is nothing to remove and exits 0 diff --git a/openspec/specs/stack-gc/spec.md b/openspec/specs/stack-gc/spec.md new file mode 100644 index 0000000..bb48c4e --- /dev/null +++ b/openspec/specs/stack-gc/spec.md @@ -0,0 +1,72 @@ +# stack-gc Specification + +## Purpose +Defines `siding gc`, which removes the stacks that can go after a merge or a deleted worktree, +and never anything that could hold unmerged or uncommitted work. + +## Requirements + +### Requirement: Merged +A branch SHALL count as merged for `gc` when it is not the default branch, its worktree has no +changes, and either its tip is contained in the default branch while the default branch has +moved past it, or its upstream branch is gone (deleted after a merge, a squash merge included). + +#### Scenario: Merged and the default branch moved on +- **WHEN** `feature/login` was merged into `main`, `main` has a commit after it, and the worktree is clean +- **THEN** `feature/login` counts as merged + +#### Scenario: A fresh branch +- **WHEN** an agent just created `feature/new` from `main`, without a commit of its own +- **THEN** it does not count as merged, and gc leaves its stack alone + +#### Scenario: Squash merge +- **WHEN** the PR of `feature/x` was squash-merged and the remote branch deleted, and `git fetch --prune` ran +- **THEN** `feature/x` counts as merged, although its commits are not in `main` + +#### Scenario: The default branch +- **WHEN** a stack runs for `main` +- **THEN** gc never removes it + +### Requirement: What gc removes +`siding gc` SHALL consider the stacks of this machine and remove: for a merged branch, the +stack's containers, networks and volumes (reason `Merged`); for a stack whose worktree no +longer exists, its containers and networks, and its volumes only if the branch is merged +(reason `WorktreeGone`). It MUST NOT touch git worktrees or branches. + +#### Scenario: A deleted worktree with an unmerged branch +- **WHEN** the worktree of `feature/wip` was deleted, but the branch has commits `main` lacks +- **THEN** gc removes its containers and networks, keeps its volumes, and reports them as kept + +#### Scenario: A stack from an older siding +- **WHEN** a stack without the label `dev.siding.gitdir` has lost its worktree +- **THEN** gc removes its containers and networks and keeps its volumes, since it cannot tell whether the branch is merged + +#### Scenario: Work in progress +- **WHEN** a worktree has uncommitted changes or its branch is not merged +- **THEN** gc removes nothing of its stack and lists it as kept with the reason `Dirty` or `Unmerged` + +#### Scenario: Worktrees stay +- **WHEN** gc removes the stack of a merged branch +- **THEN** the worktree directory and the branch still exist + +### Requirement: Ask before removing +`siding gc` SHALL list what it would remove and ask before removing it. `--yes` removes without +asking, and `--dry-run` only lists. Without a terminal, or with `--json`, it MUST fail with +`NotInteractive` unless `--yes` or `--dry-run` is given. + +#### Scenario: Interactive +- **WHEN** `siding gc` runs in a terminal and the user answers anything but `y` +- **THEN** nothing is removed + +#### Scenario: An agent runs gc +- **WHEN** `siding gc --json` runs without `--yes` or `--dry-run` +- **THEN** it fails with `NotInteractive` and removes nothing + +### Requirement: GC document +With `--json`, gc SHALL print one `GC` document: `dryRun`, the stacks it removes (or would) +with project, repository, branch, worktree, reason, and the containers, networks and volumes +removed and kept, and the stacks it keeps with their reason. + +#### Scenario: Dry run as JSON +- **WHEN** `siding gc --dry-run --json` runs with one merged stack and one dirty one +- **THEN** it prints a `GC` document with `dryRun: true`, the merged stack under `stacks` with reason `Merged`, and the dirty one under `kept` with reason `Dirty` diff --git a/openspec/specs/stack-identity/spec.md b/openspec/specs/stack-identity/spec.md index d831c60..719619f 100644 --- a/openspec/specs/stack-identity/spec.md +++ b/openspec/specs/stack-identity/spec.md @@ -75,8 +75,9 @@ with another `dev.siding.worktree` or `dev.siding.branch` label, `up` MUST fail ### Requirement: Labels siding SHALL label every container, network and volume of a stack with `dev.siding.repo`, `dev.siding.branch` (the branch name, not the slug), `dev.siding.worktree` (absolute path) and -`dev.siding.machine`, and its containers with `dev.siding.agent` when `--agent` or -`SIDING_AGENT` names one. External networks and volumes MUST NOT be relabelled. +`dev.siding.machine`, and its containers with `dev.siding.gitdir` (the repository's common git +directory) and with `dev.siding.agent` when `--agent` or `SIDING_AGENT` names one. External +networks and volumes MUST NOT be relabelled. #### Scenario: Labels on a running stack - **WHEN** `siding up` has run in `/src/shop-login` on branch `feature/login` @@ -86,6 +87,10 @@ siding SHALL label every container, network and volume of a stack with `dev.sidi - **WHEN** `SIDING_AGENT=claude siding up` runs - **THEN** the containers also carry `dev.siding.agent=claude` +#### Scenario: The git directory +- **WHEN** `siding up` has run in the linked worktree `/src/shop-login` of the repository `/src/shop` +- **THEN** its containers carry `dev.siding.gitdir=/src/shop/.git`, and its networks and volumes do not, so a stack of siding v0.1 keeps its networks on its next `up` + ### Requirement: Machine id `dev.siding.machine` SHALL be a stable id of the machine siding runs on: derived from the operating system's machine id by a one-way hash, never the raw id. Without an operating system diff --git a/openspec/specs/stack-lifecycle/spec.md b/openspec/specs/stack-lifecycle/spec.md index 3de30f3..fad5fce 100644 --- a/openspec/specs/stack-lifecycle/spec.md +++ b/openspec/specs/stack-lifecycle/spec.md @@ -87,8 +87,9 @@ compose file is gone, and keep its volumes. It SHALL print what it removed, with ### Requirement: Removing data keeps work `siding down --volumes` SHALL also remove the stack's volumes, but MUST refuse with -`UnmergedBranch` when the branch is not merged into the default branch, and with -`DirtyWorktree` when `git status` lists changes, unless `--force` is given. +`UnmergedBranch` when the branch is neither merged into the default branch nor has a deleted +upstream branch, and with `DirtyWorktree` when `git status` lists changes, unless `--force` is +given. #### Scenario: Unmerged branch - **WHEN** `siding down --volumes` runs on a branch with commits that the default branch lacks @@ -106,6 +107,10 @@ compose file is gone, and keep its volumes. It SHALL print what it removed, with - **WHEN** siding finds neither `origin/HEAD` nor a local `main` or `master` - **THEN** the branch counts as unmerged +#### Scenario: Squash-merged +- **WHEN** the branch's PR was squash-merged, its remote branch deleted and `git fetch --prune` ran +- **THEN** `siding down --volumes` removes the volumes without `--force` + ### Requirement: Logs `siding logs [service…]` SHALL show the logs of the stack's services, following them with `-f` and limited with `--tail `. With `--json` it SHALL print a `Logs` document of the last diff --git a/openspec/specs/stack-list/spec.md b/openspec/specs/stack-list/spec.md index 3a7b27b..0b4c8a1 100644 --- a/openspec/specs/stack-list/spec.md +++ b/openspec/specs/stack-list/spec.md @@ -34,17 +34,21 @@ some do, and `stopped` when none does. `ps` MUST NOT probe the proxy; the full v ### Requirement: Memory in use `ps` SHALL show each stack's memory in use, the sum over its running containers as the Docker -API reports it, and the total over all stacks. +API reports it, and the total over all stacks against the memory budget. #### Scenario: Two stacks - **WHEN** one stack uses 300 MiB and another 500 MiB - **THEN** `siding ps` shows 300 MiB and 500 MiB for them and 800 MiB in total +#### Scenario: The budget +- **WHEN** the budget is 12 GiB and the stacks use 800 MiB +- **THEN** the total line of `siding ps` reads `800 MiB of 12.0 GiB` + ### Requirement: StackList document With `--json`, `ps` SHALL print one `StackList` document with one entry per stack: project, repository, branch, worktree, machine, agent, state, address, memory in bytes, and -`worktreeMissing` when the worktree is gone. +`worktreeMissing` when the worktree is gone; and the total memory and the budget in bytes. #### Scenario: No stacks - **WHEN** `siding ps --json` runs and no stack exists -- **THEN** it prints a `StackList` document without stacks and exits 0 +- **THEN** it prints a `StackList` document without stacks, with the budget, and exits 0